Transcript: Wagtail CMS - Tom Dyson
Hello and welcome to Django Chat, a weekly podcast on the Django Web Framework.
This week we're joined by Tom Dyson to discuss Wagtail, a leading content management system built upon Django.
I'm joined as always by Carlton Gibson.
Hi Carlton.
Hello.
And Tom.
Hi, hi everyone.
So before we dive in, Tom, do you want to do a quick background on Wagtail and how you got into Django?
Yeah, sure.
So uh I'm I'm one of the two founders of Torchbox.
We're a digital agency based in the UK.
Uh 70 people now split across two cities, uh Oxford and Bristol.
But we've been going for a one long time.
We started in in 2000, and um we've been using Django since really the early days, so 2008.
I think I made the first ever screencast for Django, which is uh I'm I'm I'm still quite proud of.
You you can look that up.
And what year what year was that?
Is that 2005?
2006, 2007, maybe.
I'll I'll send you a link.
Um okay, yeah, we'll include that in the show.
The funny thing is that it I I picked the same bit of uh Django Reinhardt guitar from for that screencast that you use for Django chat.
So no one has mentioned it because it's yeah, it's sort of the Django Reinhardt uh jingle.
And yeah, I always even yeah, I uh I wonder if listeners picked up on that, but you did, so maybe you're the first.
I'm sure many others did too.
So so yeah, we've been we've been big fans of Django for a long time.
In fact, um uh Andrew Godwin, um who uh I hired was sort of as a as a fresh face university student in in Oxford, where where we're based, and um he joined Torchbox and uh it's actually for one of our client projects that he he wrote South, which of course became the Migrations Framework, which is now part of Django.
And Simon Willison, one of the two authors of Django, also worked here for a while on a project called the Carbon Account, kind of ambitious but ultimately doomed project to uh help companies track their carbon footprints in a very precise way.
So so I feel like we've got a bit of uh Django heritage that we're very proud of.
Um yeah, I I'm impressed.
I mean, because I know Django was uh cre created in Kansas, but I mean Simon, who's British, was was there.
I'm impressed by always the the large English footprint.
I guess that's you know, as an American, I sort of assume the world revolves around the US, but there's a large number of very important Django folks.
I mean, both of you, Tom Christie, Andrew, um Simon, you know.
I wonder if there's something specific about England or that's just you know it's a global programming community.
I I was struck up by that as well, actually, just in in Copenhagen and uh at DjangoCon Europe, where where Carlton and I were last week.
And uh yeah, there did seem to be kind of a high proportion of of British people there.
And and yeah, I'm not sure the reason.
Um Carlton, do you have any any guesses?
Yeah, so uh yeah, I do.
I mean, like so I don't A, British people probably notice British people, that so there's a there might be a slight bias in our observation there, but um uh it America and Europe and Australia were have been try I think, you know, if you look at the the greatest hitters, there's it's not just Britain, it's Europe as well, continental Europe, um, America, Australia.
There's a you know, we're trying to make the that that contribution base more diverse, but historically that's been that's been where it's at.
And then uh Tom, I believe so uh Wagtail, you you were at the agency, you found a way to convince a client to help fund its initial uh generation, is that right?
That's right in the day.
Yeah, so we'd been uh I guess what we're best known for in the last 15 years is building big public-facing websites, content managed sites, generally for for people making the world a better place.
And um we in 2007 we we adopted Drupal, uh very well-known PHP open source content management system that's um uh you know it's got a lot of momentum, particularly in that nonprofit space and uh a fantastic community and very very powerful piece of software.
And we built a lot of sites on Drupal that we were really proud of for big organizations, and um, but we just increasingly found it uh a technology that was hard to love.
And um at the same time, we're building apps uh in in in Django and just you know really really enjoying the the kind of the acceleration that we got from from using from using Django and Python.
And um, and I guess we we didn't we didn't set about to write another content management system because it's you know it's it's a kind of uh it's something that you you hope somebody else will do for you, and um, and there are lots of great tools out there, but we were commissioned by the Royal College of Art, very prestigious arts educ um university in in London, who themselves had done a big uh evaluation of content management systems and and hadn't found quite what they wanted.
And and so it was an opportunity for us to to build something that we've that we felt was gonna learn from the lessons of of using other systems um uh on our favourite framework on Django.
And that was in late late 2013, and um soon afterwards we we open sourced it in um February 2014.
And was that always the plan to open source it?
Was that like part of the agreement, part of the the contract to build that?
It was part of the contract, and um, and we were very happy that that that that was you know a requirement, and we've you know been big supporters and users of open source over the years, but I guess we hadn't anticipated, you know, because there's there's different ways of open sourcing things.
You you can just r build your project for your client and then take out the secrets and put it on GitHub.
But um uh but we wanted to do a bit more than that.
But I guess we we were slightly taken aback by the response that that Wagtail had, and we hadn't we hadn't anticipated that it would quickly uh become you know a re a popular tool.
And as as of last week, I think it hit um 7,000 stars on GitHub, which I know is uh you know it's just one metric among many, and it's uh you know a bit of a kind of a vanity metric, but it is it's I think that's I think that means that after Django Rest Framework, it's the biggest Django project um on GitHub.
Yeah, I think that's possibly probably true.
Well, I think I I saw that I mean in terms of people using it, I mean there's um Royal College College of Art, I think Oxford, the British NHS, NASA.
I mean, it's I was I was impressed by how widespread its usage was.
Um I think I think people always you know, we we we put those big names up up the front because they they give reassurance, and that was particular particularly important in the early days of Wagtail when for our clients, some of whom were quite risk averse, then uh they could they could see the benefits from the the UI point of view and they they they believed us that it was going to be quicker to build their sites, but they were nervous about something that they hadn't heard of before.
And and but now being able to reference Google and NASA and the NHS makes a big difference.
And particularly in the UK, the NHS has been, you know, that that's a really big deal because um the N National Health Service is, I think, still the world's fifth largest organization.
And this is not just a subsite, this is the main NHS site, this is NHS.uk, and um they they did the migration from uh kind of a massive Microsoft um infrastructure in a very public way, so they're you know really happy to speak about it.
And it's um, you know, this this this is a site that's very well known to anyone in the UK, and it's that looks like the first place you go to if you're worried about the kind of strange new marks on your arm or something, and uh you you know, um, and uh uh you know it's it's an an important public sector site.
And um I I feel like this is it's a kind of a it's a big story not just for for Wagtail but also for for Django and and Python.
Yeah, totally, totally, totally.
That sites because people always say oh Django doesn't scale or that you can't build it.
You you can like these are these are some of the world's biggest sites built on Django and
Yeah, I mean NHS, Instagram, it's pretty much as big as it gets.
Um so I I so Carlton, because you're in addition to being a Django fellow, so involved with uh Django Rest framework, I hope we can talk a little bit about what it's like to run a hugely popular open source project that relies on Django because you have these dependencies, and I imagine there's a number of similarities in terms of keeping your own projects up to date and then linking it up with Django, which has this aggressive release release cycle in part thanks to your work, Carlton.
Well, my new team's work as we discussed in the other episode.
But yeah.
Yeah.
But yeah, but I mean, because it all starts with Django, right?
I mean, and so with Wagtail, I mean, I think that's maybe something that is confusing for people who haven't used it, is you can start a new project with Wagtail, or you can also, uh, just like Django Rest framework, take an existing Django site, right, and and add Wagtail on top of it.
Um Do you have a sense, Tom, of which of those approaches is more common?
Is it starting out initially with Wagtail or adding it onto an existing Django project?
I think the former is more common, so so starting out with Wagtail, but we're really keen to stress that it that Wagtail is just a Django app and um and it it sits alongside all your other apps and it's it should be straightforward to integrate.
And you know, a lot of the time uh there are the the the kind of features in in Wagtail that we feel we can't really honestly take credit for because we're just standing on the shoulders of the things that you know that that that Django has already provided.
And and also you know, the fantastic ecosystem that you get with with Django and Python means that if there are things that our clients want to do that that aren't available out of the box in Wagtail, it's always uh you know a pip and stall away.
Right.
Yeah.
I think I think you uh the release cycle is even more aggressive than Django, right?
I mean, I think I saw somewhere you had said every two months, but I think the latest was in December of twenty eighteen for 2.4.
Is that correct?
That's right.
So we're we're we're more like three months at the moment.
The the the target was two months, and uh I think I think we might revise that because two months maybe feels a bit uh a bit too tight.
But this again was in this is I guess in response to what uh some of the lessons that we learned with Drupal.
And um there was a long period between Drupal 7 and Drupal 8, about three years, and uh it was during that pro that that upgrade cycle, it wasn't really clear how long it was going to be.
Um, you know, there were features that they wanted to to to build into Drupal 8, and um uh you know they weren't gonna launch it until they were happy with it, which is you know that's a fine, that's a like an understandable approach.
But it means that if you're someone who's about to invest in Drupal or or has an existing site, then you're left in a bit of limbo waiting to know, you know, should I wait or should I build on you know two-year-old technology?
And then when it comes to to the upgrade, there's you know loads of improvements in Drupal 8, but the difference between seven and eight was so big that for many of our clients the job of upgrading is comparable to starting again.
And we we really don't want to be in that situation.
So we have we've strived to have a predictable frequent upgrade cycle.
And you know, we aim for two months and we we've achieved that sometimes, but I think I think it's gonna be more like three months.
But the idea is that when you do the upgrade, it shouldn't be more than a 20 or 30 minute piece of work, and you're all although the you know you're not getting a a huge list of amazing new features every time because it's the the cycle is shorter, you feel like you're just taking advantage in a more immediate way of all the work that the community is doing to make the software better.
Yeah, because unshipped software is like inventory, right?
It's like stuff you've got in the warehouse that you're not it's not on the site.
No, get it out, ship it.
Like just in time delivery.
I like that.
That's that's that's a good I'm gonna use that metaphor, Colin.
But um the uh I think for Django that works exactly the same way.
Like we've got the night, it's nine months more likely for Django, and but it's reliable and it ties not just into the upgrade process that you've talked about, making upgrades easy, but it's also about confidence in the supported versions policy.
So they know, you know, users know that their their version will be supported for that 18 months, and then they've got that time to upgrade, date, and you know, just everything.
This the whole ecosystem is more healthy if there's a regular cycle that's you know relatively frequent.
And is there still do you you do LTSs for Wagtail?
Is that still the case?
Yeah, we do, yeah.
And um, again, the duration of those LTSs is kind of under discussion at the moment.
We've had um some funding from one big but uh currently secret source who has extended the the duration of the LTS uh for their own benefits.
So that's that's a that's a nice thing to do for the community, I think, because uh, you know, it i having a longer LTS is an easy thing to ask for, is an easy thing to ask for, but um you know it comes at a at a cost for the maintainers who need to make sure that uh it's not just the current release that they need to handle of their only security updates, but you know, a release from six more nine months ago.
Um and it's not you know it's not a huge consideration, but it's uh there are it's it's it's one of the the kind of the small ongoing costs that that open source maintainers have to be aware of.
Also affects the users who are on the LTS thing, you know, there's a kind of m LTS mindset that and you you end up dr getting closer and closer to the end of life date and you still haven't updated.
Whereas if you if you don't rely on the LTS, if you can not rely on the obviously it's not always possible, if you can be in that position, then you've kind of got a different attitude to updating, which I think is much more healthy for your product, not just the the the the dependencies.
But I think I've I've I've something I've learnt in the last couple of years is that um while it's you know we we can be idealistic about and uh and make it as easy as possible for people to to to be part of a rapid upgrade cycle, it's just not always possible for the for the ways that some especially enterprise y organizations operate.
And for them, it's uh you know the the idea of um of having to assign even a small amount of budget to an upgrade every two or three months feels a bit scary and and they want to know that there's a there's a longer LTS.
So I think that's something that we have to be conscious of.
Yeah, I no, I do accept that.
I just I kind of think that um the same very same companies buy vans and cars and other motor vehicles and they don't object to allocating budget to servicing them.
Yeah, Carlton and I try to be aspirational and push companies to uh yeah, be more active with the company.
No, it's a
mindset, it's a mindset thing, right?
They're still updating.
Uh but I mean, even um uh I mean Carl Meyer has a Django under the hood talk, uh he's Django Core, and then at Instagram and mentions even Instagram, which is maybe the Django site, just went right from one three to one eight and just saw what broke.
So um Right.
Yeah, it happens.
Well, so I'm curious with Wagtail, because Carlton and I have talked about how at the end of the day, how small the number of people who really make Django happen is what's the sort of core size of um contributors?
Like how how does that compare to Django?
So we have uh we have what we call the core team, and uh that's currently 19 people, um five of whom are here here at Torchbox.
But um, you know, I'm I'm happy that that five is is no longer the the majority of the core team.
Um that was a kind of a goal of mine early on in the project.
I think in terms of the I I don't have metrics, although I guess I could look at lines of code in in GitHub, but um uh in particular Matt Westcott here at Torchbox, who's the lead developer on Wagtail, is is is pretty much full-time on the on the open source product.
Um so he's full-time on Wagtail, and uh so I I I think um in terms of lines of code, we're we're probably still in the majority.
But I'm I'm really pleased that the the core team is growing.
And the core team importantly isn't just uh kind of back-end serious backend developers, it's people now with like really great UI expertise and people who can write documentation and people who are good at growing communities.
Yeah, well that's I mean, that's one of the things about Wagtail is just the first look, the UI is is beautiful and the documentation now fantastic.
I mean, I know you you've mentioned in the past that documentation was something you've spent a lot of time focusing, I guess, on that that first user or that first use perspective, right, rather than sort of the deep docs that's more relevant to people who already know it.
Right.
And I I think there's still still much more we can do that.
I'm I'm really interested in that first that first 30 minutes experience because I know that's the way that I I work, that other I uh explore new technologies.
It's like in a lunch break and I maybe have half an hour and I want to be able to pip install or you know, get it running on my laptop and see something working, and then if if if that experience goes smoothly and feels good, then I'm excited to you know recommend it to one of my colleagues or look into it more seriously.
And um, you know, that probably maybe I'm too impatient, but I think I think I think that's not uncommon.
And I think
that's standard.
Yeah, so I just think providing doing the best job you can in that in that first 30 minutes is is really important.
Um and that's just not you know, even in Python that's not as simple as it could be because uh we had a good comment recently um because uh we have this uh get started in seven lines in in Wagtail, and it starts pip install Wagtail.
Um and then uh someone raised a GitHub issue saying uh I couldn't pip install Wagtail because um they said I didn't have permissions, and and it's because we assume that uh you're gonna do it in a virtual environment, but you know, for for someone some there might be someone who's interested in who's heard about Wagtail and is interested in it and hasn't got as far in in their kind of Python development as knowing about virtual environments.
So so then we then we'll be able to do that.
How do we uh do we need to explain virtual environments or can we link somewhere?
And and you know, actually there there isn't even a root a kind of completely standard way of running a virtual environment.
No, not across
works across operating systems.
So that that bit's a bit tricky.
It'd be really good, I think, in in Python generally to to to do better at that, to to support those those new 30-minute users.
I think there's a there's a pep to I think there's a new pep to sort of remote get rid of virtual environments in Python, right?
That's the the idea.
I think there's some work being being done on that.
But yeah, I mean I think I I heard as well that when you first launched um the project, your team would had been was using Vagrant internally, right?
And also made the assumption that everyone um I guess that today that would be sort of Docker, right?
Would be the Vagrant equivalent.
Right, right, sure.
And yeah, this is another thing that uh that we learned is that we yeah, we we made I think too many assumptions about the way that other people run run Django projects uh based on our own experience.
So I think we we assumed PostgreSQL and um I had a a few other things and you know a way of running virtual environments probably.
And um and so we quite quickly in the first six months started pulling those out as we realized that that just wasn't a match for the way that other people were working.
But it's probably not something you can predict a priori, right?
You can try and explain it as simply as you can, and then you realize, oh, actually there's hidden assumptions here and it's hidden assumptions there, but you're not the whole point is the hidden assumptions, you're not gonna be able to spot them in advance.
So it's kind of okay.
Well, you as long as you improve on it, Carlton.
No, but like, yeah,
I agree.
I'm agreeing and anding.
It's it's it's an ideal.
Yeah.
Well, so what's the what do you think is the profile of a Wagtail user?
Because I could see someone who doesn't maybe know Django saying, oh, well, this is an easier way to get into Django.
Is that the case, or is it more an existing Django dev who wants to, you know, have a CMS for clients?
What would you say is the background?
I think that's shifting.
And to start off with, it was definitely this uh that that that persona of someone who's a Django developer who wants content management for their for their app, for their client.
And uh, and uh you know, we want to be a good choice for that person.
And we want to explain the differences between us and and some of the alternatives.
Um, but I think as uh as Wagtail's reputation grows a bit and um then I th I hope that it Wagtail will start being an interesting CMS for people who aren't Django developers, in the same way that a lot of people who use WordPress, you know, I'm sure don't know PHP or might might not even know what PHP is, but would are still interested in using WordPress.
Yeah.
So the interesting question then is the hosted solution, because the the vast majority of WordPress users they want hosted WordPress, not
hosted, and also they want themes, and that's uh that's a really interesting question for us.
One of one of our our principles with Wagtail's design is that um we should be completely unopinionated about the kind of site that you want to build, about the uh we don't we we definitely don't want to inject any markup, for example, um, or make it make any assumptions about how you how you want to define your models.
But the um the flip side of that is that uh when you run Wagtail start, you know, you do these seven lines and you in the kind of ta-da, you uh you left with a like a single home page with no styling.
And um and it's a it's an underwhelming experience for the first time, is it?
So clients finding a way to kind of bridge that gap between being unopinionated but also you know uh helping people start with something where they they kind of understand what they could get is a is a challenge for us.
Yeah.
And still and you still have to kind of define your page models and you know create your own templates, and you know, there's a lot of work to go from the st the I've got it started and I've got the nice admin UI and all of that stuff to a fully featured site.
So so it's hard to imagine how what the steps would be to from going from that model to a hosted model where where um Wagtail becomes you know something that's like a better option for someone who who doesn't yet know about um creating models.
And one one approach that I've been thinking of is something like um uh you know, so if if you're building a React app now, um you probably use this uh create React app, create a React app command, and that that you know that will stub out an app for you and you can continue using it, and then at some point if you want you can eject from that from that model.
So maybe we could have something similar where you uh you can you can have a kind of quick start Wagtail app that has a a blog and uh you know a custom user model or something, some some of the basic things that people want to do, and then you can eject out of it and and write write your own models and migrations.
Yeah, I saw one extension to Wagtail, I think it's called Code Red CMS, where it's very much designed for creating landing pages and marketing sites quickly, and it uses Bootstrap 4 and it's got some predefined models.
And yeah, it's kind of good.
If you want to build that site, it's it's that kind of site, it's it's very quick and good, but it's built on top of Wagtail and it's sort of made some of those decisions.
But underneath underneath the hood, it's you know, you've still got um a Wagtail project.
That's right, yeah.
It's a really interesting project, Code Red.
I I was uh slightly unnerved when I first saw it because it seemed to be, you know, it's a UCMS using Wagtail, but actually they they're quite clear in their intentions and I think they they expressed the differences in a in a nice clear way in the in the README for the GitHub.
And yes, just like you say, that you can you can build um a simple marketing site using Code Red really quickly, and but you can theme some bootstrap templates and um and then if you feel like it's uh you you need to customize it or extend it more, you can kind of eject out of that and just treat it as a normal Wagtail site.
Yeah.
Maybe let's let's talk about some of the features in Wagtail, because I think you alluded to it it it has a lot in it, and when you just fire it up for the first time and see the the simple home page, it's not really clear why you would use it.
But there's I mean, we can go down the list, right?
I mean, stream fields, search, images, I mean images in particular, right?
Because that came out of the first client, the idea that you can have focal points and automatic thumbnails.
I mean, these are really amazing features.
I kind of wonder what the process is where someone finds out about them.
Because I mean the documentation is great, but does someone um I I wonder if if there's like a killer feature, in your opinion, that brings people over versus maybe once they know Wagtail, they say, Oh, this is kind of keeps me here.
I don't I don't know there's a killer feature.
I I think I think maybe the killer feature in the early days was that the uh the UI was was kind of looked good and was thoughtful and was really focused around the needs of editors.
And um, and I think you know that there's lots of like wonderful open source software out there, but we we had the benefit when we were building Wagdale of uh some some fantastic Python developers, but also uh a UX team and designers who you know and and we took the time to to think really about the the editor experience and making it kind of responsive and seamless and and thinking about uh editorial workflow and making it look good.
And so I think that was probably the sort of the killer feature to start off with.
But the reason I think that one once we once you once you started using Wagtail, I think this uh Streamfield feature is probably what what what stands out.
And Streamfield is um so if I'd kind of describe it in terms of uh of other content management systems, you have um you you have two kind of uh two two possible models, um, broadly very kind of in a simplistic way of describing it.
You have the the model, which is probably more natural to to people to to Django developers, where you define all the fields that you think you're you're you're gonna need to to hold your data.
And uh so you have this really nice structured data, and it means that you can filter it and uh uh you can you're keep you're keeping a really nice clear distinction between presentation and and and and data.
Uh on the other hand you have tools like WordPress, which are really popular because they're easy to use, and uh you have one big field and you can you type the body in and you can use a kind of WYSIWYG type editor and copy and paste word stuff into it and have tables and inline images and so on, and it's easy to use, but you're you're kind of munging up the the presentation and the and the data, and that means it's difficult to export that in an API, for example, to a native mobile app or um and uh and uh so there are pros and cons to both approaches, but and and Streamfield was our is our uh
um our uh our our design for Streamfield was to uh to uh to try to bridge this gap to um to maintain structured data, but to give editors flexibility within pages.
Um so you can have blocks that that they're repeats and and are reordered and are optional, and so you can create like more interesting narrative type long form content, and um, but but it's still stored in a in a structured way using JSON in the back end.
Yeah.
So you kind of add a header block and a text block and an image field block and uh you know you build your page up from these components.
Yeah, uh yeah, exactly.
And uh and I should say this is this I think this was a pretty uh fresh feature when when we launched it, but it's not it's not unique anymore.
And there there are there are implementations of this in other systems.
In fact, I think the the new WordPress has uh has a quite good implementation of this.
But um, I think it's uh it's it it was really it was well designed, and I I think my colleague Matt Westcott has to take a lot of credit for that.
Yeah, I think that makes a lot of sense if you've had experience the frustrations of the other two approaches, whereas you know it may not seem quite as impressive uh for someone new to Django, but certainly for me, I that makes all the sense in the world not to get locked into that um yeah, one big block pattern because that causes frustration down the line.
But it's not exactly uh it's it's not it's not a kind of killer sales feature because it's you know it took me five minutes to explain that to you, and it's uh it's it's feels like quite a sort of subtle point.
Uh that ends up being important, I think, but uh uh is not you know it's not the kind of thing you put at the the top of the list of your your cool features.
Um perhaps you just need to work on your pitch then.
I think I do, because it's quite cool.
No, I think I think the way to do it is what you need is to just say here are the two dominant approaches and they both have um drawbacks.
Um but it yeah, it is something you need to kind of feel the pain first.
You can't just trust the pain.
Um so well so images too.
I was really impressed by all the
open images, right?
This is or maybe do you want to give the pitch on all the built-in image features?
Well, just before you do, for me it was amazing because in Django Land when Wagtail came along, there wasn't, you know, there were some um third-party apps out there that handled image uploading, but they none of them were particularly nice.
And then Wagtail did it really well.
Um, you know, you could upload images and as you say, select focal points and all these kind of things.
That's actually not very glamorous, but it's quite hard to implement properly.
And Wagtail kind of did, and it was like, ah yeah, this is really nice.
Oh, that's nice to hear.
But uh and I think I I guess it's um I was partly driven because our first client, Royal College of Art, uh, as you imagine, the very uh, you know, the the visuals are really important to them, and um, and so we needed to we we make sure that they were displaying their their image content in a really clear way.
And also something that I've just noticed a l as uh a lot in some pretty big high-profile news sites, you know, you often get this situation where you have uh on the on the details page, so like a news item, you have a nice big hero image, and um maybe the photographer has used like the rule of thirds and the the the the portrait is you know over on the on the left or the right hand side, but then on the index page showing the latest items, they'll do like a square crop, and um, and sometimes the crops are you know cutting off half the face of the the person the subject's about
And um, so that was uh uh and in Wagtail you specify the the the focal in the focal point.
So you you draw an area around the thing which you want to crop in.
Um, and I think that that's more flexible than allowing editors to to do actual crops because you don't know quite how that focal crop might be used that focal point might be used in in future designs so it means that um then you know when when you're resizing the image for different devices for example it's just going to make sure that it's always maintained around that around that point.
And I think it's it feels like a small detail but it's that it's it can be the difference between a site that looks uncared for and one that looks like it's really making an effort for its for its audience.
Yeah I think that's really important especially since Django is a predominantly back end framework that that love and care for the front end uh yeah it matters it matters a lot even to the most you know design challenged back end engineer they can tell when it's thought out.
But kind of like the whole point of a CMS is that you take the the the the sort of the hand cropping element of the the work out of it for people right that you don't want them hand coding HTML you don't want them hand cropping images.
So to have it done automatically and nicely is is the raison d'etra.
Yeah yeah no I agree.
And I actually there's been some really interesting work um in uh in the kind of third party ecosystem around image handling and one of my favorites is um by uh uh this uh someone from a Swedish agency Martin Sandstrom who um he built uh a plugin called Wagtail Autify and um this uses these um hosted machine learning tools so computer vision um services and uh the the one that works best is actually the Microso
ft one and as you upload your images it will send those images off to the to the image recognition service and um within you know a a few hundred milliseconds you get a a suggested title and tags for each image as it's uploaded.
And and it's
and and this was in your Django Corn Europe talk that you
that's right yeah yeah my talk was about uh was not about Wagtail it was about um how Django developers can uh can take advantage of these cheap amazing cloud machine learning services and sort of inject magic into their apps and and this is this is one really really nice example of that I think um and it's not you know the the you you always get a laugh when you show this because some of the some of the descriptions are are obviously inaccurate but actually you know they get better and
better and I think um the expectation is not that you just allow the the the machine to tell you the answer but you use it as a as an aid so it's um it's something that should augment the the editor experience for particularly for for large like really big busy news sites I think um I think you know it's it's a kind of thing that can really chip away at the the sort of the the the like the basic legwork that editors have to do.
So what's the process by which something like that makes its way into Wagtail?
I mean like South is a great example of you know migrations coming into Django is there an example of that or do you just really focus on Wagtail itself and just let the third party be third party since that's a lot to manage.
Yeah so that that's an example of something that isn't in Wagtail itself but but uh because you know it's it's it's really cool but we don't imagine that everyone wants to use that um but we we highlight it so there's uh there's an awesome Wagtail you know the you know these awesome X repositories that uh
I have the awesome Django one.
Oh wow wow congratulations um
well there was an older one but it's not being maintained so I have the new one.
Yours is more awesome and some some someone made a an awesome Wagtail and and we keep that up to date and you know that that's that's currently where we we highlight the kind of the community plugins.
But actually there's been some discussion about whether we should do a better job of that and you know there are there are a few apps that have their own kind of nicely presented marketplaces.
Not that I'd not I'm not really imagining a kind of commercial marketplace but but something that makes it easier for to to surface these kind of tools.
Yeah because the ecosystem's now got the size to the size where it's discoverability is difficult.
You can't just follow one account on Twitter and have it all appear in
all I think that's true.
Yeah.
Yeah well I know that's a chang a challenge for Django itself is uh you know Django itself doesn't have an awesome Django repo because then they're sort of being opinionated and I you know frankly as a book author I wish they had a books page um where they just listed all the Django books because there are very few but um you know when someone googles best Django books they find my blog post instead so I uh you know I understand the tension.
Isn't isn't one of the rules I don't know if it's an an unwritten rule one of the rules about the awesome X repositories is that they can't be started by the creators of X
Oh I'm not sure.
Well so to be frankly honest so I started the awesome Django repo again with the idea of oh this one's out of date and then I started down the process with the awesome maintainer of going through all the steps uh needed to do and then I just got swamped with stuff so it's technically not in full compliance but I you know my way of maintaining it is I just say this is basically Will's repo and you're welcome.
Do you accept submissions to it?
Um I should people have done them on on occasion I'll accept them but I don't feel bad about just making it pretty arbitrary.
Because the problem is it just bloats over time.
Because you just add more and more you don't want to take away and I mean I know a lot of the people too so I don't really want to be um so so I don't know it it's a it's a tough thing for any awesome or any repo as being you know that referee position but the curation is nice.
I mean you know for example like there's third party packages there's the Django packages site which lists all of them and you can filter a bit um awesome Django has I don't know twenty and it's you know it's not like strictly by stars it's basically the ones that I think are cool and I'm constantly adding new ones but um you know I was getting kind of burned out on the awesome Django thing and I was just like you know I'm just gonna do it for myself.
But yeah that's there's no perfect way to do it.
So no but
it's really difficult to field the requests like it so you asked earlier on about maintaining a large open source
car thing
it's exactly that no but it's exactly that problem is there's ever coming in streams can you add this can you add this can you add this and you have to choose otherwise it becomes unmanageable.
And I think that the issue the Django docs has a policy look we just don't link to third party things because they disappear and we have to keep them updated and they're not reliable so we need sites like awesome Django or awesome Wagtail to curate the list but it has to be curated otherwise it's just noise again.
Yeah so Tom if you could snap your fingers what would you like to change or what features would you like to see you know a perfect developer drop down and implement in Wagtail?
Uh I I don't think it's a I don't think it's a a kind of a brand new feature but I think the um a really important direction for us is supporting the the headless model better.
And um I think this is a clear direction in in content management generally and um we're seeing more and more of our own clients wanting it and talking about it and uh and and you know just more generally in in the industry and there's some really interesting products and services things like contentful you know proprietary hosted headless CMS.
And um so
this is where the CMS acts just just exposes an API which then clients use to build sites or whatever
I should have explained that so with
like Reactor View or or uh Elm Carlton right
so or even a even a publishing pipeline might consume content from the CMS I don't know if you're creating physical newsletters or something but the idea being that it's the CMS kind of stops at the API level.
Yeah that it's not it's not responsible for the presentation part.
I was gonna say that Wagtail actually supports this pretty well um and uh a really a really uh an early user in fact probably the first user of ITL in this way was the Hillary Clinton campaign who um who kind of secretly uh commissioned the uh the first API for Wagtail uh in order that they could use it in a headless way and um uh you know the people who were behind the the the Barack Obama campaign um you know kind of a lot of them made their careers out of the success of uh
that so we were waiting excitedly for the day that we were gonna say that we brought in
actually I had a a a colleague who worked on the Hillary campaign and I remember him mentioning uh Django and I was like what now he was sort of hush hush about it.
I don't know that that that ties that circle together for me
anyway that story didn't quite play out but um yeah but um there's
always 2020 no no no
yeah so they they commissioned the first uh the first API and so that that was a headless site and um and then actually uh Tom Christie uh the you know the the author of Django Rest framework rewrote the Wagtail API to use DRF which is you know that was that was a a good day that uh that PR landed um uh without warning you know it just just popped up in G
itHub that was pretty exciting and um so so Wagtail works well in that way but uh we don't do a great job of documenting it and um I think we we should have some good demos and blog posts and there are some features that we can add so in particular preview is is a bit of a problem with uh with headless CMSs generally because you know in in standard Wagtail when you hit preview we're able to to take the the data that that you've just been writing and push it into the into the template and you immediately see how it's going to look
If you're uh if you're but
this is if you're in in editor mode so not on a live site yeah
exactly yeah in in editor mode you're seeing and it's not even a draft it's it's it's just the changes you've just made even before you may have saved it as draft um and you really want that cycle to be to be quick and you know just for editors to be able to kind of play with their content and and immediately get feedback on how it's going to work.
And but if if Wagtail isn't responsible for the front end it's harder for it to to to represent that so we we want to find a a good generic way of being able to preview from from different front ends.
And that requires making some changes to the API that mean that we can store that uh that kind of um ephemeral preview data in the API in a way that's accessible in the API which has its own challenges because then the front end may have uh it needs to authenticate in a different way.
Yeah authentication is the one that jumps out.
Well it when you look at um changes in Django I mean specifically I guess async are there any things down the line on the roadmap that will cause uh will force major changes within Wagtail itself you know 3.0 and all the rest?
I think I I think async is the only one and async is just something that's really exciting for us and for for all Django products I guess because uh because of the the the the potential for you know rapidly improved performance.
Um are you a big believer in that?
Because I know yeah there's you know so it it definitely can work in some cases and at massive scale there's a there is a question of whether it's you know more than you need for a a moderate site I guess.
Right.
I'm curious what your personal view on
it I'm I'm I'm a believer in the in the potential performance gains.
I'm I'm I guess I'm uh a bit skeptical about what it's going to take for for the for Django and the Django community to adopt it in a way that realizes those gains.
There is there is secret news in that Andrew Godwin opened up a pull request on Django um master last week at the sprints right DjangoCon Europe Sprints uh introducing like that that first step into um an async handler at the very HTTP layer um and that's not the middleware and that doesn't let you yet write async views but that's that's definitely mergeable not maybe exactly is but with a review or two that's definitely mergeable and then that's we're on the road um and it's coming.
I think where the the async really could help a project with like Wagtail is where you've got to do multiple fetches and a REST API will pick um one model and then another model and another model it would be very easy if we had async views to write a kind of proxy view that did a couple of fetches and combine them without necessarily having to jump all the way into say GraphQL or something you know which is a totally different technology.
Exactly yeah yeah
I'm excited about that
I think another another relevant use case in in Wagtail is uh cache invalidation you know one of the one of the hard things and uh uh we um we Wagtail plays nicely with a lot of the the big caching proxies like Cloudfront and Cloudflare and um varnish and so on but uh those those invalidation calls can be quite slow and uh we you know you you want to check at what point the invalidation has successfully returned.
So I think the simple way is to run Celery and have a worker and keep checking but if we can just do that in an async way then that just simplifies a lot of code.
Yeah.
And so another feature that um with Wagtail is I guess it's an experimental is the A B testing framework.
Can you briefly talk about that?
Because that looks really interesting.
Yeah I'm uh this is um it's it's again it's not part of Wagtail itself it's uh although it's under the the the Wagtail organization in GitHub it's um it's a tool that was commissioned by a bank in in Norway and um and that's relevant because uh uh there are some great products and services for doing A B testing and one that a lot of people know about is Google Optimize and um and that works by um it's you know JavaScript that you embed and then you you have this UI that's off site that uh that lets you define
the different variants so the different options that your users might see but it means that when you load the page the JavaScript is having to rewrite bits of the DOM and um and as uh uh for for banks and and this is definitely drone nowhere and I probably I imagine in in lots of uh lots of territories it's not allowed to have third party systems rewrite the content of their of their
interesting yeah it's probably in the yeah in the US very draconian with how they yeah
right so um and
rightly so
so that that
rules out you know the use of these these kind of sophisticated front end tools and um so they commissioned us to build a uh something that was that worked natively in Wagtail and um and it's you know it's definitely simpler than those other tools to which you know uh you know have got teams of people working on them full time but it it allows you to create different variants of your of your content usually as as as different pages and then uh to to start running experiments and then um it uses a kind of middleware to to to swap between them and
uh typically you know setting a cookie to make sure that you that users are directed to the the relevant one each time and then has some simple reporting and um and it's not as I said it's not um it's not the most featureful A B testing but I think the important thing about E B testing is just to start doing it.
We have to have it everyone knows that it's it's best practice that uh you know especially if you're getting a reasonable amount of traffic you should just test which option results in more donations for your for your charity or uh sign-ups for your letter or whatever it is or you know or sales for your product and uh and then you know the the tools are there for you for you to to run those experiments and and pick the best one and then move on to do the do the next test.
So I think our focus has been on having something that works simply and is uh easily integrated.
Also from a web performance point of view like having JavaScript re-render the DON that's going to be quite slow right noticeably slow on mobile devices and all these things.
And there's whereas having it rendered server size is going to be you know the same speed essentially as not doing A B testing from a performance point
of view.
That's right and uh and there is still you know we uh kind of old timers remember the uh the so-called flash of unstyled content that we used to get and uh as I said before the CSS kick kicked in and uh and you still get the the flash of un un A B tested content uh with these tools which you can minimize in some ways but it's uh it's a bit tricky to manage.
There are still performance implications though because if you're doing A B testing uh server-side that makes it harder to do straightforward uh caching using Cloudflare or CloudFront or one of those tools so
okay yeah yeah yeah fair enough
well uh you know as we sort of wrap this up Tom one question I had for you is since so much of your work is with nonprofits and you seem very mission focused, you know, as I saw in a talk you mentioned the moral dilemma of you know open source software where Wagtail is used by lots and lots of people doing lots and lots of
things.
Yeah I wonder if you could just speak to that.
Because I don't you know I Django is used by lots of s lots of sites, porn sites, all the rest, um what your thoughts are on Wagtail doing that because most licenses you can't there isn't really an open source license where you can restrict the usage I don't believe.
No I I don't know of one.
It's it is an interesting uh dilemma I don't know if it's a dilemma because there's not much we can do about it but it's uh there there are definitely people using Wagtail who who we wouldn't work for.
Yeah.
And uh and and also the question comes up about you know and and some of those may some of those people may be asking questions on the on the Wagtail Slack or or you know Stack Overflow and so on and we we're committed to helping everybody in those communities so so you know we definitely will be by one way or another supporting people whose whose goals you know we we may not agree with.
But it's pretty hard I think to to try to you know forensically hunt down on the the who's on the right side of the line.
And I guess we generally feel like open source is a is a force is a positive force in the world and uh and people using open source are are you know in in a small way helping with that and so you know there's there's a there's a kind of justification for if I guess for supporting open source generally.
Yeah no I just I what was interesting to me is just that you asked the question because it's not something often asked um and certainly it should be in technology.
So I just thought and I guess that comes from you know many of your clients and running an agency and choosing who you work with that makes a lot of sense.
Right.
Yeah I'm sorry I don't have a more coherent answer.
So is was there anything Tom or Carlton we wanted to include as we as we wrap up?
Well what's the what's the future of Wagtail right?
Like you you sort of alluded to it but what's good you know if you look ahead um you mentioned headless um potentially themes and deployments are there where would you like to see it in a couple years if if all plays out?
I think the two main directions are the are the ones I've mentioned so one is making Wagtail feel like a an attractive option for for for anyone not just a Django developer and I think a lot of that is about about improving that that experience for the first 30 minutes which could be having a hosted service or just really really working on that on those those beginner steps.
And the second is the the headless thing and I think you know a a kind of quite clear goal for us could be to be the the top choice for a open source headless CMS.
So that would be whatever whatever the technology, if you, if you uh if you if you want to go headless, then you know you've got great commercial tools like like uh Contentful, but if you want to go open source then Wagtail's your top choice.
That that would be a good something for us to strive for, I think.
Yeah.
Well I I know Carlton and I have talked a a bunch about deployment in Django because that's you know as the as the author of books, I sort of I I get so many questions about the deployment sections, which I use Heroku which is you know as easy as it gets, but is still quite a leap.
And um so I often also think about how do I you know can you just push a button and solve this for people because it is it's it's really tricky even with platforms as a service.
Yeah yeah.
It would be amaz I mean even with um companies like Divio who get you you know they kind of help with that that deployment story you still you know you still got your Django project which isn't quite as easy to get off the ground as a WordPress you know for instance.
And so it it would be really nice if there were you know maybe on top of Wagtail or on top some some other way but if there were this kind of you can get a blog up and running pretty
I think that eject model that React has, I mean, is really nice because you know ejecting is it's pretty pretty forceful once you eject you're out but you can go a long way and um right like you can go quite quite a long long runway and I think they do a pretty good job of saying yeah once you eject you're on your own.
Yeah yeah I've been trying to imagine um you know from from a user point of view what what the best possible way of uh what the best user experience would be for deploying a site.
And you're right you know this is the question that that most people ask and you see this on Reddit and Stack Overflow.
It's you know I've I've been through the tutorial I've built my site now I'm stuck because I can't get nginx to talk to Unicorn ah
yeah well yeah it's
and it's tricky that stuff and it's you know and there's lots of different edge cases and different different ways that it can fail.
So my what
people make an entire living doing just that
it'd be nice to have a new manage.py command manage.py deploy and it says do you want to deploy just yes to uh Roku or Divio or uh nginx or you know um
yeah yeah we yeah we should have that Carlton I know you agree oh look
a and look a flying pony
yeah right well
yeah no I mean I
think yeah I I do agree I think that's um still the part of the story which is most difficult but
I think you just have to be really opinionated with it is is the thing is that you you almost need to not ask a Django developer but you know get a group of these beginners and you know optimize for that and you know we can talk about the trade-offs all day long but uh they can object when they need to but you know we're we we sort of have the wrong perspective on it I would say
I think you're right yeah talk to the beginners yeah
I mean you know the problem then is that this is the problem with Heroku and all these
platforms as a service is that it's a it's a business model challenge because so then you get all the beginners and intermediate people and then um once they become sites that would actually pay the bills they're tempted to uh to leave but I think even that is changing I mean um it just makes so much sense for most sites to have a managed hosting solution I but um great well thank you so much for joining us Tom we're gonna have links to all the things we've mentioned um is there anything Carlton you want to add?
No super I mean you know normally I'd say more but Tom's really hit all the points right on the head so I can just sit and listen.
That's great.
And is there a way for people to contact uh or contribute to Wagtail or get involved with it Tom?
Absolutely yeah so it's uh GitHub slash Wagtail slash Wagtail and uh there's a there's a Slack channel at um Wagtail.io forward slash Slack.
Those are the probably the the easiest entry points um or email me tomattauchbox.com I'm really I'm really happy to help anyone get started.
Wow that's that's a gutsy move giving your email out like that.
That's great.
Well and and and for this podcast uh you can always see new episodes at DjangoChat com.
Please leave a review and we'll see you all next week.
Bye.
Bye bye all right
thanks for joining us Tom