Transcript: Wagtail CMS - Tom Dyson

hello and welcome to django chat a weekly podcast on the django web framework this week we're joined

by tom dyson to discuss wagtail a leading content management system built upon django i'm joined as

always by carlton gibson hi carlton hello and tom hi hi everyone so before we dive in tom do you

want to do a quick background on wagtail and how you got into django yeah sure so uh i'm i'm one of

two founders of torchbox we're a digital agency based in the uk uh 70 people now split across

two cities uh oxford and bristol but we've been going for a long time we started in in 2000

and um we've been using django since really the early days so 2008 i think i made the first ever

screencast for django which is i'm still quite proud of you you can look that up and what year

what year was that 2005 2006 2007 maybe i'll send you a link um okay yeah we'll include that

you know what the funny thing is that it i i picked the same bit of uh django reinhardt guitar

from for that screencast that you use for django chat so no one has mentioned it because it's yeah

it's sort of the django reinhardt uh jingle and yeah i always even yeah i i wonder if listeners

picked up on that but you did so maybe you're the first i'm sure many others did too so so

So, yeah, we've been big fans of Django for a long time.

In fact, Andrew Godwin, who I hired as a fresh-faced university student in Oxford, where we're based,

and he joined Torchbox, and it was actually for one of our client projects that he wrote South,

which, of course, became the Migrations Framework, which is now part of Django.

And Simon Willison, one of the two authors of Django, also worked here for a while on a project called The Carbon Account,

ambitious but ultimately doomed project to help companies track their carbon footprints in a very

precise way. So I feel like we've got a bit of Django heritage that we're very proud of.

Yeah, I'm impressed. I mean, because I know Django was created in Kansas, but I mean,

Simon, who's British, was there. I'm impressed by always the large English footprint. I guess

as an American, I sort of assume the world revolves around the US, but there's a large

number of very important django folks i mean both of you tom christie andrew um simon you know i

wonder if there's something specific about england or that's just you know it's a global programming

community yeah i don't know i was struck up by that as well actually just in in copenhagen and

django con europe where carlton and i were last week and uh yeah there did seem to be

kind of a high proportion of british people there and i'm not sure the reason

carlton do you have any any guesses yeah so yeah i do i mean like so i don't a british people

probably notice british people that so there's a it might be a slight bias in our observation there

but um america and europe and australia would have been i think you know if you look at the

greatest hitters there's it's not just britain it's europe as well continental europe um america

australia is a you know we're trying to make the that that contribution base more diverse but

historically that's been that's been where it's at and then tom i believe so wagtail you you were

at the agency you found a way to convince a client to help fund its initial uh generation is that

right that's right yeah so we'd been uh i guess what we're best known for in the last 15 years

is building big public facing websites content managed sites generally for for people making

the world a better place and um we in 2007 we we adopted drupal a very well-known php open source

content management system that's um uh you know it's got a lot of momentum particularly in that

non-profit space and a fantastic community and very very powerful piece of software and we built

a lot of sites on drupal that we were really proud of for big organizations and um but we just

increasingly found it uh a technology that was hard to love and um at the same time we're building

apps uh in in django and just you know really really enjoying the the kind of the acceleration

that we got from from using from using django and python and um and i guess we we didn't we

didn't set about to write another content management system because it's you know it's

a kind of uh it's something that you you hope somebody else will do for you and um and there

are lots of great tools out there but we were commissioned by the royal college of art very

prestigious arts university in london who themselves had done a big evaluation of content

management systems and hadn't found quite what they wanted and and so it was an opportunity for

us to to build something that we've that we felt was kind of learning from the lessons of of using

other systems um uh on our favorite framework on django and that was in late late 2013 and um

soon afterwards we we open sourced it in um february 2014 and was that always the plan to

open source it was that like part of the agreement part of the the contract to build that it was part

of the contract and um and we were very happy that that that was you know a requirement and we've

been big supporters and users of open source over the years but i guess we hadn't anticipated

you know because there's different ways of open sourcing things you you can just build your

project for your client and then take out the secrets and put it on github but um uh but we

wanted to do a bit more than that but i guess we were slightly taken aback by the response that

that wagtail had and we hadn't we hadn't anticipated that it would quickly uh become

you know a popular tool and as as of last week i think it hit um 7 000 stars on github which i

know is a you know it's just one metric among many and it's a you know a bit of a kind of

vanity metric but it is it's i think that's i think that means that after django rest framework

it's the biggest Django project on GitHub.

Yeah, I think that's possibly, probably true.

Well, I think I saw that, I mean, in terms of people using it,

I mean, there's Royal College of Art, I think Oxford, the British NHS, NASA.

I mean, I was impressed by how widespread its usage was.

I think people always, you know, we put those big names up the front

because they give reassurance.

And that was particularly important in the early days of Wagtail

And for our clients, some of whom were quite risk averse, then they could see the benefits from the UI point of view.

And they believed us that it was going to be quicker to build their sites.

But they were nervous about something that they hadn't heard of before.

But now being able to reference Google and NASA and the NHS makes a big difference.

And particularly in the UK, the NHS has been, you know, that's a really big deal because the National Health Service is, I think, still the world's fifth largest organization.

And this is not just a sub-site.

This is the main NHS site.

this is nhs.uk and um they they did the migration from a kind of massive microsoft

infrastructure in a very public way so they you know really happy to speak about it

and it's um you know this this this is a site that's very well known to anyone

in the uk and it's like the first place you go to if you're worried about

the kind of strange new marks on your arm or something and uh you know um and uh uh you know

It's an important public sector site, and I feel like this is kind of a big story, not just for Wagtail, but also for Django and Python.

Yeah, totally, totally, totally.

Because people always say, oh, Django doesn't scale, or that you can't build it.

You can.

These are some of the world's biggest sites built on Django.

Yeah, I mean, NHS, Instagram, it's pretty much as big as it gets.

So, Carlton, because you're, in addition to being a Django fellow, so involved with Django REST framework, I hope we can talk a little bit about what it's like to run a hugely popular open source project that relies on Django because you have these dependencies.

And I imagine there's a number of similarities in terms of keeping your own projects up to date and then linking it up with Django, which has this aggressive release cycle in part thanks to your work, Carlton.

Well, mainly Tim's work, as we discussed in the other episode.

Yeah, because it all starts with Django, right?

I mean, and so with Wagtail, I mean, I think that's maybe something that is confusing for people who haven't used it is you can start a new project with Wagtail or you can also, just like Django REST Framework, take an existing Django site, right, and add Wagtail on top of it.

Do you have a sense, Tom, of which of those approaches is more common?

Is it starting out initially with Wagtail or adding it on to an existing Django project?

I think the former is more common, so starting out with Wagtail,

but we're really keen to stress that Wagtail is just a Django app

and it sits alongside all your other apps

and it should be straightforward to integrate.

And a lot of the time there are the kind of features in Wagtail

that we feel we can't really honestly take credit for

because we're just standing on the shoulders of the things

that Django has already provided.

And also, you know, the fantastic ecosystem that you get with Django and Python means that if there are things that our clients want to do that aren't available out of the box in Wagtail, it's always, you know, a pippin's all the way.

Right.

Yeah.

And I think the release cycle is even more aggressive than Django, right?

I mean, I think I saw somewhere you had said every two months, but I think the latest was in December of 2018 for 2.4.

Is that correct?

That's right.

So we're more like three months at the moment.

the target was two months and uh i think i think we might revise that because two months maybe

feels a bit uh a bit too tight but this again was in this is i guess in response to what some of the

lessons that we learned with drupal and um there was a long period between drupal 7 and drupal 8

about three years and uh it was during that pro that that upgrade cycle it wasn't really clear

how long it was going to be um you know there were features that they wanted to to to build into

Drupal 8 and um uh you know they weren't going to launch it until they were happy with it which is

you know that's fine that's like an understandable approach but it means that if you're someone who's

about to invest in Drupal or has an existing site then you're left in a bit of limbo waiting to know

you know should I wait or should I build on you know two-year-old technology and then when it

comes to to the upgrade there's you know loads of improvements in Drupal 8 but the difference

between 7 and 8 was so big that for many of our clients the job of upgrading is comparable to

starting again and we we really don't want to be in that situation so we have we've strived to have

a predictable frequent upgrade cycle and you know we aim for two months and we've achieved that

sometimes but i think i think it's going to be more like three months but the idea is that when

you do the upgrade it shouldn't be more than a 20 or 30 minute piece of work and you're although

the you know you're not getting a huge list of amazing new features every time because it's the

cycle is shorter you feel like you're just taking advantage in a more immediate way of all the work

that the community is doing to make the software better yeah because unshipped software is like

inventory right it's like stuff you've got in the warehouse that you're not it's not on the site no

get it out ship it like yeah just in time delivery i like that that's that's a good i'm going to use

that metaphor it's joel spolsky's i've robbed it but um all right the uh i think for django that

works exactly the same way like we've got the night it's nine months more likely for Django

and but it's reliable and it ties not just into the upgrade process that you've talked about making

upgrades easy but it's also about confidence in the supported versions policy so they know you

know users know that their their version will be supported for that 18 months and then they've got

that time to upgrade date and you know just everything this the whole ecosystem is more

healthy if there's a regular cycle that's you know relatively frequent and is there still you do LTS

for wagtail is that still the case yeah we do yeah and um again the duration of those lts is

kind of under discussion at the moment we've had um some funding from one big but uh currently

secret source who has extended the duration of the lts uh for their own benefits so that's that's

a nice thing to do for the community i think because uh you know having a longer lts is an

easy thing to ask for is easy thing to ask for but um you know it comes at a cost for the

maintainers who need to make sure that uh it's not just the current release that they need to

handle of their own security updates but a release from six or nine months ago um and it's not you

know it's not a huge consideration but it's uh there are it's it's one of the the kind of the

small ongoing costs that open source maintainers don't have to be aware of also affects the users

who are on the LTS think, you know,

there's a kind of LTS mindset

and you end up getting closer and closer

to the end of life date

and you still haven't updated.

Whereas if you don't rely on it,

on the lts if you can not rely on the obviously it's not always possible if you can be in that

position then you've kind of got a different attitude to updating which i think is much

more healthy for your product not just right yeah yeah the the dependencies but i think i've i've

something i've learned in the last couple of years is that um well it's you know we we can

be idealistic about and and make it as easy as possible for people to to to be part of a rapid

upgrade cycle it's just not always possible but for the ways that some especially sort of

enterprisey organizations operate and for them it's uh you know the the idea of um of having to

assign even a small amount of budget to an upgrade every two or three months feels a bit scary and

and they want to know that there's a there's a longer lts so i think that's something that we

have to be conscious of yeah i know i do accept that i just i kind of think that um the same very

same companies buy vans and cars and other motor vehicles and they don't object to allocating

budget to servicing them yeah carlton and i try to be aspirational and push companies to uh yeah

be more active but this is no it's a mindset it's a mindset thing right now still updating but i

mean even um uh i mean carl meyer has a django under the hood talk uh he's django core and then

at instagram and mentions even instagram which is maybe the django site just went right from

1.3 to 1.8 and just saw what broke so right yeah it happens well so i'm curious with wagtail because

Carlton and I have talked about how, at the end of the day, how small the number of people who

really make Django happen is. What's the sort of core size of contributors? How does that compare

to Django? So we have what we call a core team, and that's currently 19 people, five of whom are

here at Torchbox. But I'm happy that that five is no longer the majority of the core team.

That was a kind of a goal of mine early on in the project.

I think in terms of the, I don't have metrics,

although I guess I could look at lines of code in GitHub,

but in particular, Matt Westcott here at Torchbox,

who's the lead developer on Wagtail,

is pretty much full-time on the open source product.

So he's full-time on Wagtail.

And so I think in terms of lines of code,

we're probably still in the majority,

but I'm really pleased that the core team is growing.

And the core team, importantly, isn't just kind of back-end, serious back-end developers.

It's people now with really great UI expertise and people who can write documentation and people who are good at growing communities.

Yeah, well, that's one of the things about Wagtail is just the first look.

The UI is beautiful and the documentation now, fantastic.

I mean, I know you've mentioned in the past that documentation was something you've spent a lot of time focusing, I guess, on that first user or that first use perspective, right?

rather than sort of the deep docs that's more relevant to people who already know it right and

i think there's still still much more we can do that i'm really interested in that first that

first 30 minutes experience because i know that's the way that i i work that other i uh explore new

technologies it's like in a lunch break and i maybe have half an hour and i want to be able to

pip install or you know get it running on my laptop and see something working and then if if

if that experience goes smoothly and feels good then i'm excited to you know recommend it to one

of my colleagues or look into it more seriously and um you know that probably maybe i'm too

impatient but i think i think i think that's not uncommon and i think no i think that's standard

yeah so i just think providing doing the best job you can in that in that first 30 minutes is

really important um and that's actually it's not you know even in python that's not as simple as

It could be because we had a good comment recently

because we have this get started in seven lines in Wagtail

and it starts pip install Wagtail.

And then someone raised a GitHub issue saying

I couldn't pip install Wagtail because I didn't have permissions.

And it's because we assume that you're going to do it

in a virtual environment.

But there might be someone who's interested

and who's heard about Wagtail and is interested in it

and hasn't got as far in their Python development

as knowing about virtual environments.

So then we need to think about, do we need to explain virtual environments or can we link somewhere?

And, you know, actually there isn't even a kind of completely standard way of running a virtual environment.

No, not at all. It's gotten worse.

Across operating systems.

So that bit's a bit tricky.

It'd be really good, I think, in Python generally to do better at that, to support those new 30-minute users.

I think there's a, there's a pep to, I think there's a new pep to sort of remove, get rid

of virtual environments in Python, right?

That's the idea.

I think there's some work being, being done on that.

But yeah, I mean, I think I heard as well that when you first launched the project,

your team would have been, was using Vagrant internally, right?

And also made the assumption that everyone, I guess that today that would be sort of Docker,

right?

Would be the Vagrant equivalent.

Right, right.

Sure.

And yeah, this is another thing that we learned is that we, yeah, we made, I think, too many assumptions about the way that other people run Django projects based on our own experience.

So I think we assumed PostgreSQL and a few other things and a way of running virtual environments probably.

And so we quite quickly in the first six months started pulling those out as we realized that that just wasn't a match for the way that other people were working.

But it's probably not something you can predict a per hour, right?

you can try and explain it as simply as you can. And then you realize, oh, actually there's hidden

assumptions here and it's hidden assumptions there, but you're not, the whole point is the

hidden assumptions. You're not going to be able to spot them in advance. So it's kind of okay.

Wow. As long as you improve on it, Carlton.

No, but like, yeah, I agree. I'm agreeing and, anding.

It's, it's, it's an ideal.

Yeah. Well, so what's the, what do you think is the profile of a

Wagtail user? Cause I could see someone who doesn't maybe know Django saying,

oh, well, this is an easier way to get into Django.

Is that the case, or is it more an existing Django dev

who wants to have a CMS for clients?

What would you say is the background?

I think that's shifting, and to start off with,

it was definitely that persona of someone who's a Django developer

who wants content management for their app, for their client.

And we want to be a good choice for that person,

and we want to explain the differences between us and some of the alternatives.

But I think as Wagtail's reputation grows a bit and then I hope that Wagtail will start being an interesting CMS for people who aren't Django developers.

In the same way that a lot of people who use WordPress, you know, I'm sure don't know PHP or might not even know what PHP is, but are still interested in using WordPress.

Yeah.

So the interesting question then is the hosted solution, because the vast majority of WordPress users, they want a hosted WordPress, not a run your own.

and also they want themes and that's a that's a really interesting question for us one one of our

principles with wagtail's design is that um we should be completely unopinionated about the kind

of site that you want to build about the uh we don't we definitely don't want to inject any mark

up for example um or make it make any assumptions about how you how you want to define your models

but the um the flip side of that is that uh when you run wagtail start you know you do these seven

lines and you in the kind of ta-da you uh you left with a like a single home page with no styling

and um and it's a it's an underwhelming experience for the first time you said so

so clients finding a way to kind of bridge that gap between being unopinionated but also

you know helping people start with something where they they can understand what they could get

is a challenge for us yeah and still and you still have to kind of define your page models and

you know create your own templates and you know there's a lot of work to go from this that i've

got it started and i've got the nice admin ui and all of that stuff to a fully featured site so so

it's hard to imagine how what the steps would be to from going from from that model to a hosted

model where where um wagtail becomes you know something that's like a better option for someone

who who doesn't yet know about creating models and one one approach i've been thinking of is

something like um uh you know so if you're building a react app now um you probably use

this uh create racked up create react app command and that that you know that will stub out an app

for you and you can continue using it and then at some point if you want you can eject from that

from that model so maybe we could have something similar where you uh you can you can have a kind

of quick start wagtail app that has a a blog and uh you know custom user model or something some

some of the basic things that people want to do and then you can eject out of it and and write

write your own models and migrations yeah i saw one extension to wagtail i think it's called code

red cms where it's very much designed for creating landing pages and marketing sites

quickly and it uses bootstrap 4 and it's got some predefined models and yeah it's kind of good if

you want to build that site it's that kind of site it's it's very quick and good but it's built on

top of wagtail it's sort of made some of those decisions but underneath underneath the hood

it's you know you've still got um a wagtail project that's right yeah it's a really interesting

project code red i was uh slightly unnerved when i first saw it because it seemed to be you know

it's a new cms using wagtail but actually they're quite clear in their intentions and i think they

expressed the differences in a nice clear way in the in the readme for the github and it's just

like you say that you can you can build um a simple marketing site using code red really quickly

and but you can theme some bootstrap templates and um and then if you feel like it's uh you need

to customize or extend it more you can kind of eject out of that and just treat it as a normal

wagtail site yeah maybe let's let's talk about some of the features in wagtail because i think

you alluded to it it it has a lot in it and when you just fire it up for the first time and see the

the simple home page it's not really clear why you would use it but there's i mean we can go down

the list, right? I mean, stream fields, search images, I mean, images in particular, right?

Because that came out of the first client, the idea that you can have focal points and automatic

thumbnails. I mean, these are really amazing features. I kind of wonder what the process is

where someone finds out about them. Because I mean, the documentation is great, but does someone,

I wonder if there's like a killer feature in your opinion that brings people over versus maybe once

they know Wagtail, they say, oh, this is kind of keeps me here. I don't know if there's a killer

feature i think maybe the killer feature in the early days was that the uh the ui was was kind of

looked good and was thoughtful and was really focused around the needs of editors and um and

i think you know there's lots of like wonderful open source software out there but we we had the

benefit when we were building wagtail of uh some some fantastic python developers but also uh a ux

team and designers who and we took the time to to think really about the editor experience and

making it kind of responsive and seamless and thinking about editorial workflow and making

it look good and so i think that was probably the sort of the killer feature to start off with

but the reason i think that once people once you once you started using wagtail i think this uh

stream field feature is probably what what what stands out and stream field is um so if i just

kind of describe it in terms of other content management systems you have um you you have two

kind of uh two possible models um probably very kind of in a simplistic way of describing it you

have the the model which probably more natural to to people to django developers where you define

all the fields that you think you're you're you're going to need to to hold your data and uh so you

have this really nice structured data and it means that you can filter it and uh uh you can you're

you're keeping a really nice clear distinction between presentation and and and and data on the

other hand you have tools like wordpress which are really popular because they're easy to use

and you have one big field and you can you type the body in and you can use kind of WYSIWYG type

editor and copy and paste word stuff into it and have tables and inlining images and so on and it's

easy to use but you're you're kind of munging up the the presentation and the and the data and that

means it's difficult to export that in an api for example to a native mobile app or um and uh and

so there are pros and cons to both approaches and stream field was is our uh um our our design for

stream field was to uh to try to bridge this gap to um to maintain structured data but to give

editors flexibility within pages um so you can have blocks that that repeats and are reordered

and they're optional and so you can create like more interesting narrative

type long form content and um but but it's still stored in in a structured way using json in the

back end yeah so you kind of add a header block and a text block and an image field block and

you know build your page up from these components yeah yeah exactly and uh and i should say this is

this i think this was a pretty uh fresh feature when when we launched it but it's not it's not

unique anymore and there are there are implementations of this in other systems in fact i

think the new wordpress has a has a quite a good implementation of this but um i think it's uh it

it was really it was well designed and i think my colleague matt westcott has to take a lot of

credit for that yeah i think that makes a lot of sense if you've had experience of frustrations of

the other two approaches whereas you know it may not seem quite as impressive uh for someone new

to django but certainly for me that makes all the sense in the world not to get locked into that

um yeah one big block pattern because that causes frustration down the line but it's not exactly

it's it's not a kind of killer sales feature because it's you know it took me five minutes

to explain that to you and it's a it's it feels like quite a sort of subtle point that ends up

being important i think but uh uh is not you know it's not the kind of thing you put at the top of

the list of your your cool features perhaps you just need to work on your pitch there i think i

do because it's quite cool no i think i think the way to do it is what you did is to just say here

the two dominant approaches and they both have drawbacks um but it yeah it is something you need

to kind of feel the pain first you can't just trust the pain um so so images too i was really

impressed by all the yeah yeah no images right just gonna say go on this is or maybe do you want

to give the pitch on all the built-in image features well just before you do for me it was

amazing because in jango land when wagtail came along there wasn't you know there were some um

third-party apps out there that handled image uploading but they none of them were particularly

be nice and then wagtail did it really well um you know you could upload images and as you say

select focal points and all these kind of things that's actually not very glamorous but it's quite

hard to implement properly and wagtail kind of did and it was like ah yeah this is really nice

oh that's nice to hear but i think i guess it's um i was partly driven because our first client

royal college of art as you imagine the very uh you know the visuals are really important to them

and um and so we needed to we would make sure that they were displaying the the image content

in a really clear way and also something that i've just noticed as a lot in some pretty big

high profile news sites you know you often get this situation where you have a on the on the

details page so like a news item you have a nice big hero image and um maybe the photographer has

used like the rule of thirds and the the the portrait is you know over on the on the left or

the right hand side but then on the index page showing the latest items they'll do like a square

crop and um and sometimes the crops are you know cutting off half the face of the person the

subject's about and um so that was uh and in wagtail you specify the the focal in the focal

point so you draw an area around the thing which you want to crop in um and i think that's more

flexible than allowing editors to to do actual crops because you don't know quite how that focal

crop might be used that focal point might be used in in future designs so it means that um then you

know when you're resizing the image for different devices for example it's just going to make sure

that it's always maintained around that around that point and i think it's it feels like a small

detail but it's it's it can be the difference between a site that looks uncared for and one

that looks like it's really making an effort for its for its audience yeah i think that's really

important especially since django is a predominantly back-end framework that that love and care for the

front end yeah it matters it matters a lot even to the most you know design challenge back in

engineer they can tell when it's thought out but kind of like the whole point of a cms is that you

take the the the sort of the hand cropping element of the the work out of it for people right that

you don't want them hand coding html you don't want them hand cropping images so to have it done

automatically and nicely is is the raison d'etre yeah yeah no i agree and actually there's been

some really interesting work um in uh in the kind of third-party ecosystem around image handling

and one of my favorites is um by uh uh this uh someone from a swedish agency martin sandstrom who

he built a plugin called wagtail altify and um this uses these um hosted machine learning tools

so computer vision services.

And the one that works best is actually the Microsoft one.

And as you upload your images,

it will send those images off to the image recognition service.

And within a few hundred milliseconds,

you get a suggested title and tags for each image as it's uploaded.

And this was in your DjangoCon Europe talk?

That's right, yeah.

My talk was not about Wagtail.

It was about how Django developers can take advantage

of these cheap amazing cloud machine learning services and sort of inject magic into their apps

and and this is this is one really really nice example of that i think um and it's not you know

that they you always get a laugh when you show this because some of the some of the descriptions

are obviously inaccurate but actually you know they get better and better and i think um the

expectation is not that you just allow the the machine to tell you the answer but you use it as

a as an aid so it's um it's something that should augment the the editor experience for particularly

for large like really big busy news sites i think um i think you know it's the kind of thing that

can really chip away at the the sort of the like the basic legwork that editors have to do so what's

the process by which something like that makes its way into wagtail i mean like south is a great

example of you know migrations coming into django um is there an example of that or do you just

really focus on wagtail itself and just let the third party be third party since that's a lot to

manage yeah so that that's an example of something that isn't in wagtail itself but uh because you

know it's it's really cool but we don't imagine that everyone wants to use that um but we we

highlight it so there's a there's an awesome wagtail you know the you know these awesome

x repositories that i have the awesome django one oh wow wow congratulations um so there was an

older one but it's not being maintained so i have the new one yours is more awesome and someone

someone made a an awesome wagtail and we keep that up to date and you know that that's that's

currently where we we highlight the kind of the community plugins but actually there's been some

discussion about whether we should do a better job of that and you know there are a few apps that

have their own kind of nicely presented marketplaces not that i'm not i'm not really

imagining a kind of commercial marketplace but but something that makes it easier to to surface

these kind of tools yeah because the ecosystem's now got the size to the size where it's

discoverability is difficult right you can't just follow one account on twitter and have it all

appear yeah i think that's true yeah yeah well i know that's a challenge for django itself is

uh you know django itself doesn't have an awesome django repo because then they're

sort of being opinionated and you know frankly as a book author i wish they had a books page

um where they just listed all the django books because there are very few but um you know when

someone googles best django books they find my blog post instead so i yeah i understand the

tension isn't isn't one of the rules i don't know if it's an unwritten rule one of the rules about

the awesome x repositories is that they can't be started by the creators of x oh i'm not sure well

so to be frankly honest so i started the awesome django repo again with the idea of oh this one's

out of date and then i started down the process with the awesome maintainer of going through all

the steps uh needed to do and then i just got swamp with stuff so it's technically not in full

compliance but i you know my way of maintaining it is i just say this is basically will's repo and

yeah you're welcome do you accept submissions to it um i should people have done them on occasion

i'll accept them but i don't feel bad about just making it pretty arbitrary because the problem is

it just bloats over time because you just add more and more you don't take away and i mean i know a

lot of the people too so i don't really want to be um so so i don't know it's a it's a tough thing

for any awesome or any repo is being you know that referee position but the curation is nice i mean

for example like there's third-party packages there's the django packages site which lists all

of them and you can filter a bit um awesome django has i don't know 20 and it's you know it's not

like strictly by stars it's basically the ones that i think are cool and i'm constantly adding

new ones but um you know i was getting kind of burned out on the awesome django thing and i was

just like you know i'm just gonna do it for myself but yeah that's there's no perfect way to do it so

no but but it's really difficult to field the requests like so you asked earlier on about

maintaining a large open yes tell us carlton it's exactly that no but it's exactly that problem is

there's ever coming in streams can you add this can you add this can you add this and you have

to choose otherwise it becomes unmanageable i think that the issue the django docs has a policy

look we just don't link to third-party things because they disappear and we have to keep them

updated and they're not reliable so we need sites like an awesome django or awesome wagtail to

curate the list but it has to be curated otherwise it's just noise again yeah so tom if you could

snap your fingers what would you like to change or what features would you like to see you know

a perfect developer drop down and implement in wagtail uh i don't think it's a i don't think it's

a kind of a brand new feature but i think the um a really important direction for us is supporting

the headless model better and um i think this is a clear direction in content management generally

and um we're seeing more and more of our own clients wanting it and talking about it and uh

And just more generally in the industry, there's some really interesting products and services, things like Contentful, proprietary hosted headless CMS.

So this is where the CMS just exposes an API, which then clients use to build sites or whatever.

Exactly, yeah. I should have explained that.

With like ReactorView or Elm, Carlton, right?

Or even a publishing pipeline might consume content from the CMS.

I don't know if you're creating physical newsletters or something.

The idea being that the CMS kind of stops at the API level.

Yeah, that it's not responsible for the presentation part.

I was going to say that Wagtail actually supports this pretty well.

And a really early user, in fact, probably the first user of Wagtail in this way

was the Hillary Clinton campaign who kind of secretly commissioned the first API for Wagtail

in order that they could use it in a headless way.

And, you know, the people who were behind the Barack Obama campaign, you know, kind of a lot of them made their careers out of the success of that.

So we were waiting excitedly for the day that we were going to say that we brought him.

Yeah, I had a colleague who worked on the Hillary campaign.

And I remember him mentioning Django and I was like, what?

Now he was sort of hush hush about it.

That ties that circle together.

Yeah. Anyway, that story didn't quite play out.

yeah but um there's always 2020 no no yeah so they commissioned the first uh first api and said

that that was a headless site and um and then actually tom christie uh the you know the the

author of chango rest framework rewrote the wagtail api to use drf which is you know that was that was

a good day that uh that pr landed um uh without warning you know just just popped up in github

that was pretty exciting and um so so wagtail works well in that way but uh we don't do a great

job of documenting it and um i think we we should have some good demos and blog posts and there are

some features that we can add so in particular preview is a bit of a problem with uh with

headless cms's generally because you know in standard wagtail when you hit preview we're able

to to take the the data that that you've just been writing and push it into the into the template and

you immediately see how it's going to look if you're uh if this is if you're in editor mode

so not on a live site yeah exactly yeah in editor mode you're seeing and it's not even a draft it's

it's it's just the changes you've just made even before you may have saved it as draft um and you

really want that cycle to be to be quick and you know for editors to be able to kind of play with

their content and and immediately get feedback on how it's going to work um and but if if wagtail

isn't responsible for the front end,

it's harder for it to represent that.

So we want to find a good generic way

of being able to preview from

from different frontends, and that requires making some changes to the API that mean that

we can install that kind of ephemeral preview data in the API in a way that's accessible in

the API, which has its own challenges, because then the frontend may have needs to authenticate

in a different way. Yeah, authentication is the one that jumps out. Well, when you look at

changes in Django, I mean, specifically, I guess, async, are there any things down the line on the

roadmap that will cause uh will force major changes within wagtail itself you know 3.0 and

all the rest i think i i think async is the only one and async is just something that's really

exciting for us and for for all django products i guess because uh because of the the potential for

you know rapidly improved performance um are you a big believer in that because i know yeah there's

you know so it definitely can work in some cases and at massive scale there is a question of

whether it's you know more than you need for a moderate site i guess right i'm curious what

your personal i'm a believer in in the potential performance gains um i'm i guess i'm a bit

skeptical about what it's going to take for for the for django and the django community

to adopt it in a way that realizes those gains there is there is secret news in the andrew

godwin opened yes a pull request on django um master last week at the sprints right on europe

sprints uh introducing like that that first step into an async handler at the very http layer

and that's not the middleware and that doesn't let you yet write async views but that's that's

definitely mergeable not maybe exactly is but with a review or two that's definitely mergeable

and then that's we're on the road um and it's coming i think where the the async really could

help a project with like wagtail is where you've got to do multiple fetches and a rest api will

pick um one model and then another model and another model it would be very easy if we had

async views to write a kind of proxy view that did a couple of fetches and combine them without

necessarily having to jump all the way into say graph ql or something you know which is a totally

different technology exactly yeah yeah so i'm example i'm excited about i think another another

relevant use case in in wagtail is uh cache invalidation you know one of the hard things

And Wagtail plays nicely with a lot of the big caching proxies like CloudFront and CloudFlare and Varnish and so on.

But those invalidation calls can be quite slow.

And you want to check at what point the invalidation has successfully returned.

So the simple way is to run Celery and have a worker and keep checking.

But if we can just do that in an async way, then that just simplifies a lot of code.

Yeah. And so another feature that with Wagtail is, I guess it's an experimental, is the A-B testing framework.

Can you briefly talk about that? Because that looks really interesting.

Yeah, this is, again, it's not part of Wagtail itself.

Although it's under the Wagtail organization in GitHub, it's a tool that was commissioned by a bank in Norway.

And that's relevant because there are some great products and services for doing A-B testing.

and one that a lot of people know about is google optimize and um and that works by um it's you know

javascript that you embed and then you have this ui that's off-site that uh that lets you define

the different variants so the different options that your users might see but it means that when

you load the page the javascript is having to rewrite bits of the dom and um and as uh uh for

for banks and this is definitely during nowhere and probably i imagine in lots of uh lots of

territories it's not allowed to have third-party systems rewrite the content of their oh interesting

yeah it's probably yeah in the u.s very draconian with how they yeah right so um and right so so

that that rules out you know the use of these these kind of sophisticated front-end tools and

so they commissioned us to build something that was that worked natively in wagtail and um and

it's you know it's definitely simpler than those other tools to which you know uh you know i've

got teams of people working on them full time but it allows you to create different variants of your

of your content usually is as different pages and then uh to to start running experiments and then

it uses a kind of middleware to to swap between them and uh typically you know setting a cookie

to make sure that you that users are directed to the the relevant one each time and then has

some simple reporting and um and it's not as i said it's not um it's not the most featureful

a b testing but i think the important thing about a b testing is just to start doing it you know

everyone knows that it's it's best practice that uh you know especially if you're getting a

reasonable amount of traffic you should just test which option results in more donations for your

for your charity or uh signups for your letter or whatever it is or you know or sales for your

product and uh and then you know the tools are there for you to run those experiments and and

pick the best one and then move on to do the next test so our focus has been on having something

that works simply and is uh easily integrated also from a web web performance point of view

like having javascript re-render the don that's going to be quite slow right noticeably slow on

mobile devices and that's right and there's whereas having it rendered server size is going

to be you know the same speed essentially as not doing it right from a performance that's right and

And there is still, you know, kind of old timers remember the so-called flash of unstyled content that we used to get.

And as I said before, the CSS kicked in.

And you still get the flash of un-AB tested content with these tools, which you can minimize in some ways, but it's a bit tricky to manage.

There are still performance implications, though, because if you're doing AB testing server-side, that makes it harder to do straightforward caching using Cloudflare or CloudFront and one of those tools.

okay yeah yeah yeah fair enough well uh you know as we sort of wrap this up tom one question i had

for you is since so much of your work is with non-profits and you seem very mission focused

you know i saw in a talk you mentioned the moral dilemma of you know open source software where

wagtail is used by lots and lots of people doing lots and lots of things yeah i wonder if you could

just speak to that because i know you know jango is used by lots of lots of sites porn sites all

the rest um what your thoughts are on wagtail doing that because most licenses you can't

there isn't really an open source license where you can restrict the usage i don't believe

no i don't know of one it's it is an interesting dilemma i don't know if it's a dilemma because

there's not much we can do about it but it's uh there are definitely people using wagtail who

who we wouldn't work for yeah and uh uh and and also the question comes up about you know and

some of those may some of those people may be asking questions on the on the wagtail slack or

you know stack overflow and so on and we're committed to helping everybody in those communities

so so you know we definitely will be by one way or another supporting people whose whose goals

you know we may not agree with but it's pretty hard i think to to try to you know forensically

on the the who's on the right side of the line and i guess we generally feel like open source is a

as a force as positive force in the world and uh and people using open source are are you know in

in a small way helping with that and so you know there's there's a there's a kind of justification

for i guess for supporting open source generally yeah no i just i what was interesting to me is

just that you asked the question because it's not something often asked um and certainly it should

be in technology so yeah i just thought and i guess that comes from you know many of your clients and

running an agency and choosing who you work with that makes a lot of sense right yeah i'm sorry i

don't have a more coherent answer so was there anything tom or carlton we wanted to include as

we as we wrap up well what's the what's the future of wagtail right like you you sort of alluded to

it but what's good you know if you look ahead um you mentioned headless um potentially themes and

deployments are there where would you like to see it in a couple years if if all plays out i think

the two main directions are the ones i've mentioned so one is making wagtail feel like an attractive

option for for for anyone not just a django developer and i think a lot of that is about

about improving that that experience for the first 30 minutes which could be having a hosted service

or just really really working on that on those those beginner steps and the second is the headless

thing and i think you know a kind of quite clear goal for us could be to be the the top choice for

open source headless cms so yeah you know that would be whatever whatever the technology if you

if you uh if you if you want to go headless then you know you've got great commercial tools like

like contentful but if you want to go open source then wagtail is your top choice that that would

be a good something for us to strive for i think yeah well i know carlton and i have talked a bunch

about deployment in django because that's you know as the as the author of books i sort of

i get so many questions about the deployment sections which i use heroku which is you know

as easy as it gets but it's still quite a leap and um so i often also think about how do i you

know can you just push a button and solve this for people because it is it's it's really tricky

even with platforms of service yeah yeah it would be i mean even with um companies like divio who

get you know they kind of help with that that deployment story you still you know you still

which isn't quite as easy to get off the ground as a wordpress you know for instance and so it

it would be really nice if there were you know maybe on top of wagtail or on top some some other

way but if there were this kind of you can get a blog up and running pretty yeah i think that

eject model that react has i mean it's really nice because you know ejecting is it's pretty

pretty forceful once you eject you're out but you can go a long way and um right like you can go

quite quite a long long runway and i think they do a pretty good job of saying yeah once you eject

you're on your own yeah yeah i've been trying to imagine um you know from a user point of view

what the best possible way of what the best user experience would be for deploying a site and

you're right this is the question that that most people ask and you see this on reddit and stack

overflow it's you know i've been through the tutorial i've built my site now i'm stuck because

i can't get nginx to talk to unicorn ah yeah well yeah it's tricky that stuff and it's you know and

there's lots of different edge cases and different different ways that it can fail so my people make

the entire living doing just that be nice to have a new manage.py command manage.py deploy and it

do you want to deploy yes yes to uh roku or divio or uh nginx or you know um yeah yeah we

yeah we should have that carlton i know you agree oh look a look a flying pony

yeah right well yeah no i think yeah i i do agree i think that's um still the part of the story

which is most difficult but i think you just have to be really opinionated with it is is the thing

is that you almost need to not ask a django developer but you know get a group of these

beginners and you know optimize for that and you know we can talk about the trade-offs all day long

but yeah they can object when they need to but you know we sort of have the wrong perspective on it

i would say i think you're right yeah talk to the beginners yeah i mean you know the problem then is

that this is the problem with heroku and all these platforms as a service is that it's a it's a

business model challenge because so then you get all the beginners and intermediate people and then

once they become sites that would actually pay the bills they're tempted to uh to leave but i think

even that is changing i mean um it just makes so much sense for most sites to have a managed

hosting solution but um great well thank you so much for joining us tom we're gonna have links to

all the things we've mentioned um is there anything carlton you want to add no super i mean you know

normally i'd say more but tom's really hit all the points right on the head so i can just sit

and listen that's great and is there a way for people to contact uh or contribute to wagtail or

get involved with it tom absolutely yeah so it's uh github slash wagtail slash wagtail and uh there's

a there's a slack channel at um wagtail.io forward slash slack those are the probably the the easiest

entry points um or email me tom at torchbox.com i'm really i'm really happy to help anyone get

started. Wow, that's a gutsy move, giving your email out like that. That's great. Well, and for

this podcast, you can always see new episodes at DjangoChat.com. Please leave a review and we'll

see you all next week. Bye. Bye-bye. All right. Bye-bye. Thanks for joining us, Tom.