Transcript: Async Django - Andrew Godwin

This week we're joined by Andrew Godwin to talk about all things Django.

I'm Will Vincent, joined as always by Carlton.

Hey Carlton.

Hello, Will.

Hello, good to meet you.

This is really exciting because you've been responsible for many cool things in Django in the past and are currently working on really the future of Django.

So lots to cover around async and all the rest that you've done.

So maybe let's start it off with can you quickly give us your background on how you got into coding and uh Django in particular?

So uh like my first Fora into coding was a little bit unusual.

I started programming on a um Palm PDA, a Palm 5 to be precise.

Um there was basically like this little sort of basic app where you could like write basic programs and do little games, and like that's where I cut my teeth coding.

Um and I was one of those like annoying like early teenage coders because of this.

I was like 14, 15 on holiday with my parents in Cornwall, just coding in the caravan rather than actually like doing any outdoor holiday things.

Um but then

Yeah, exactly.

Um but then I I took from that went to do some PHP.

Um, so did a lot of PHP when I was sort of uh starting university, that kind of stuff.

And then I think it was about 2006 I went to work for a company called Torchbox, um, who are an agency in Oxfordshire, and that's where I met Simon Willison.

And Simon Willison, if you ever met him, is an amazingly enthusiastic person.

Uh and he within I think two weeks had taken me from writing PHP to writing Django, um, which of course was very young at the time.

It was uh pre-version 1.0.

And so that's kind of my route into Django.

Um, and then I've sort of worked with Django ever since, I'd say, at that point.

Um, but on and off, like so Torchbox also where um I wrote the first version of South.

And so it's also my you know, I I think two summers later, because I I was I was doing summers there during university, so I'd do university during term time, come the summer break, I would then go and work at Torchbox for a few months.

Um, and I basically sort of released South, and this is back when like Russell was still doing uh Django Evolution, there were still like lots of options for migration frameworks in Django, and I got this email from Jacob, who I never met and like vaguely heard of at this point, Jacob Catelyn Moss, that is.

He's like, Andrew, like are we we've seen your release?

We're having a panel at DjangoCon, the first DjangoCon in Silicon Valley.

Would you like to go?

And he's just like, Oh no, it's fine.

Like, no, the DSF will cover it.

Like, like we'll we'll pay for you and give you a scholarship to come out there.

So, like, and this is like maybe three or four weeks until the conference.

So at four weeks' notice, I basically go, okay, and then book a flick ticket to America, which I've never been to, fly out to Silicon Valley in like maybe the most alienating thing, and but then find this this wonderful community of people who are so supportive, and like even that first Django con, I think, s like settled me into Django properly.

I know it may not be in the DSF in its current form, but like some version of something like like Jacob was like, Yes, we can we can cover the costs for you, basically.

And that that's what got me out there.

So that's super.

And then that we heard tell that there was um some kind of celebrity deathmatch panel um about migration frameworks.

Russell had the rather excellent um Django evolution, and I had South.

And we sort of this look this like we did a little presentation, we did a little chat, but like through the years that followed, um, South slowly became more and more dominant until eventually it sort of morphed into the rewrite that became Django migrations that are now built in.

Um what led you to there?

Was it has it all been Django, or like many developers have you done non-Django things at various companies along the way?

So like after after university, I went to work at Torchbox for a a year or so.

I went then did some freelance work and worked with a few different companies, like Global Radio, who used to be running Django, that kind of thing, um, and a few other small companies.

Uh, and then eventually I settled into doing some contracting for Lanyard, which was a sort of event, a site for discovering professional events was the idea.

So, like, you know, here's all the conferences, and you can follow your friends.

And that was run again by Simon Willison.

So, like my career in Simon's touch points along a lot, a lot of points along the history.

Um, and

Um, and then like you know, I enjoyed work contracting at Lanyard.

Eventually, Simon offered me a okay, short, short divert here.

Just before this, I had applied for a job at the British Antarctic Survey to be the on-site sysadmin at one of the bases in the South Pole.

Uh not South Pole, but in Antarctica.

Um Halley 6.

Uh, and unfortunately, um, I that was the year that uh the BBC documentary series Frozen Planet had aired.

Um, and so the number of applications that position went from two to about 40.

And I got this very apologetic call from the wonderful people at the Antarctic survey going, it's such an unusually good year, and like any other year, you we would have hired you, but there was somebody who was just more experienced than you, and we went for him instead.

And so Simon very nicely waited a week and then offered me a full-time role at Lanyard after after I missed out on my Antarctic uh base position.

Like I get a chance I'm going down to Antarctica.

Um, but it's it's not easy to do as as you can imagine.

Um, but then shortly afterwards, like I started working for Lanyard, and then in 2012, 2013, something like that, um, Lanyard got acquired by Eventbrite, and I've been Eventbrite ever since.

I don't know how much you can say about that, and but can you sort of talk about the evolution of the async thoughts and the thoughts around channels and yeah, for sure.

Things like that.

Um, like these days, especially, like it's just grown and grown over the last five or six years.

But interestingly, very little of that is async.

A lot of my work that is based on like seeing the challenges Eventbrite faces and like how would we tackle them?

Like, what is the best approach?

And so some of those ideas we have tried internally at Eventbrite, some of those ideas are more like they doing things in a large company is different to doing them on a volunteer project, so they're better done on Django and then maybe brought in later in the future.

Um, but a lot of those do stem from like, you know, Eventbrite has a slightly unique traffic pattern in that it deliberately like we deliberately DDOS ourselves like multiple times a day because our model is we have these things that go on sale and everyone arrives at once for that event.

Like in e-commerce, like normally e-commerce is like, oh, Christmas is terrible, or like Thanksgiving or Thanksgiving or Black Friday, but everything else is quiet.

Whereas we just have like this very like gentle demand curve and then these massive spikes like a few times a day as events sort of go on sale.

And so that's an interesting scaling challenge that some of the things in async are particularly good at coping with.

Right, exactly.

So is that sort of a continuous throughout the day thing then where like Europe to America and Asia, or is it um does it matter sort of the geographic range of where those spikes are, or it's I

Um but there is you know, eventbrite's still very heavily in Europe and America, as that's where they tend to consecrate.

But yeah, that like it has started being like a well, they happen at 3 a.m.

uh Pacific time too, right?

And like nobody in the SF office is awake at that point, but we have a wonderful Madrid office now.

Madrid might be around, and like it's trying to it's trying to like make sure that you have global coverage, but also you have to just start to code to cope with those things at some point, right?

Like you you can't treat everyone specially, but like, well, we just have to design for this, we have to expect it.

Um, and we can't design for it by over-provisioning the entire site so that it runs at full capacity all the time and just wastes money.

So there's an interesting balance there that async really helps enable.

So you must be using Kubernetes and have problems with ingress tables and all the rest of the other things.

Like I do work on that.

I'm an SRE.

Um but yeah, like a lot of the like spinning up servers on demand is a difficult proposition.

Um, especially so like our Django app, it's it it's in it's in or near the million lines of code mark.

Um it takes 30 seconds to import all the Python modules into memory.

Like they're just so there's just so many Python modules that just the import itself is slow.

And so like we even have that like cold boot problem of like, well, it's a big app, spinning it up takes a lot of time.

And we're trying to break it up and make it into smaller pieces, but it takes time to turn a monolith into into services.

And um I saw, I don't know if this is something you've exp um uh come across or whatever, but um I saw on the Sentry um blog that they were talking about you rewriting a little elements in Rust or something like that to make you to get speed gains there.

Is that something you might do as well?

Like VenPy's still a very heavily Python company.

Um and pretty much like the one case where we needed a speed up, like uh PyPy, as in PYPY, gave us that speed up, right?

Like it costs a bit more RAM, but like it made our our webhooks delivery like seven times faster, I think.

And so there are places like if you need that trade-off, you can still do it within the Python ecosystem.

I think there is one piece written in Go, but that's because it was a time before async HTTP and Python was ready.

Like so, like we have um like a live seat map viewer where you can see like here's the seat map at the venue, and the seats like vanished in real time as people pick them.

So that the whole thing is like live streamed, and that that was written sort of four or five years ago before WebSockets and Python and async HTTP was a good thing.

And so that's all that's written in that particular bit is written in Go.

Because I think a lot of this is as a Django educator, I get a ton of questions about this because it seems sort of like sexy in the future of Django, but I think most people don't really understand what it is.

Um actually.

I'm curious, could you could you give the backstory of ch of channels and kind of how that's morphed over time to where we are today?

It starts with uh Aimerick Augustin, one of the uh Django uh contributors, and he is responsible for things like um the big transaction rewrite, for example.

Um and Aimerick did a lot of work um quite a few years ago now on uh an attempt at doing WebSockets with Django.

He has sort of a game of life example where every cell ran its own socket and that kind of thing.

And I looked at that and I talked to him a few times and I was interested.

And also this at this point, like migrations was done.

I'd had a year or two clear of it, so I sort of understood, like I'd recovered a bit from the burnout that comes with pushing yourself on migrations, as you can imagine.

And then I was like, okay, well, I want a new project.

What seems to be the best thing to do for Django?

Like, what is what is Django's future?

And at that point, WebSockets was the new kid on the block.

It seemed to be the thing everyone wanted.

And so I was like, well, okay, like let's let's take a serious look at this.

And of course, as I did with South, I was like, well, South succeeded by being an external project, integrating into Django, and then over time becoming very popular and eventually being merged in.

So, like, well, let's follow the same principle, right?

And so I was like, okay, so channels is gonna try and be this separate um sort of third-party app, uh, except we're gonna sort of fund it under the the DSF's auspices.

So um the wonderful folks at Mozilla gave us a grant for I think $115,000 of or thereabouts um to fund it uh for a year or so, or year or two, in fact, ended up being.

And so like that helped build it out and helped have the research.

And a lot of what channels is is basically like how do you take a synchronous Django and not touch it, but somehow wrap asynchronous around it.

So particularly for WebSockets, especially.

It doesn't really try and address asynchronous HTTP.

It is, it is, it's in there, like you can do it, but it's not easy, it's not really designed for that.

It's much more about you want either normal, boring, synchronous HTTP with Django, or you want flashy WebSockets with channels.

But there's very little middle ground between those two.

It was a Django Under the Hood.

So we can link to that so you don't have to repeat that.

Um so I want to talk more about that, but actually I it brings up can you talk about so move merging South into Django?

What was that like?

Because from the outside, I started with Django when South was there, and then just kind of magically it happened.

But what was that process like for you personally and just the logistics of the Django community?

Because that was is one of the largest, you know, third-party packages becoming into Django, and and I many third party ones, you know, Django Rest framework don't and probably won't.

So curious, what was the personally, what was that experience like?

Because I'm sure it was a ton of work and work in ways that aren't obvious from the outside.

Yeah.

So like Django migrations is a from scratch rewrite of South.

Basically, I took the I think six or seven years at that point of lessons from maintaining a migration framework and going, okay, like if I could do it differently, here is what I would do.

And this is the one chance we get to have breaking backwards compatibility, right?

So, like we're gonna be right on migration path, but that migration path is gonna be like not particularly fluid.

Like you have to like basically move to migrations and then run a command to synchronize all up and you're done.

And so, like that basically I sat down, I planned it all out, I went to Kickstarter.

Um, we ran it as a Kickstarter, and I think I raised like 17,000 uh pounds sterling, I think, something like that.

And that helped again fund it.

So, like a lot of um one of the things I do is I I work four days a week, and the fifth day each week is for open source work, which is I'm very thankful I've had from Lanyard, I've had from Eventbrite.

Like they've been very supportive in both of these.

And so like that money pays the pays for my fifth day a week to work on the project.

And so with migrations, uh when I was back at Lanyard, I believe this was, um, I was doing doing that, and I was like, okay, let's you know raise the money so I can fund it and take the salary cut and it makes sense.

Um and then I sat down, planned it out, went to the community and said, Here's my pitch.

Like I this is before the I think this is before the depth process, so like we didn't have a depth, so I was like, I just pitched on Django developers and I pitched um like the core team at that point.

Was like, okay, like this seems like what we want.

Does anyone disagree?

And I got very little pushback.

I think the one pushback was like, I have problems with South.

I was like, yes, I do too.

Here is the list of products, here is how we're gonna fix them.

Um and I think and everyone agreed that like it's a problem that pretty much every install of Django faces, right?

Like almost everyone has that problem with the URM.

And so that all lined up, and then I sat down, spent six months to a year doing the full rewrite, fully testing it, and then released it in Django 1.7.

Uh, and then a few extra fixes, and then thank I I'm very thankful to uh people like Marcus and other people who took over maintenance of it, like Ride sort of done that and burnt out a bit.

Um but yeah, it was it was an interesting time.

Like, you know, looking back, you think, oh, it wasn't that bad, you know.

And at the time you're pulling your hair out, and but it was it was well done.

I remember, you know, it was one of the big milestones in Django's move from the early days to the later days, I guess, when the the 1.7 with the migrations coming in.

Exactly.

I mean, so was it six months you just went off and did on your own?

Like, how often were you showing the work to others?

Like how how did that like so how would it be different now with async stuff versus what you did back then?

Like I I I got approval, I I went into the shadows for six months and came back with a basically a beta version.

It's not a good strategy, but it's also like as a developer, it's my default, and it's hard to get away from that.

And so like one of the things.

I mean, you can't sec have people second guessing you as much as you're second guessing yourself with some decisions.

But at the same time, like having people to bounce off of is also crucial to like getting good development done, right?

And that one of the things I've learned about EventPry is like how to run large projects.

And so, like, if you look at the async proposal versus like the migrations proposal, if you could go back and look at it, um, they're structured very differently.

Like async is structured in a way where it's like, well, this is a very big project, it needs to be across multiple people, it needs to provide value in incremental steps and not be one big dump, and it needs to be like exchangeable between like different people, and and like it we can pull out any time.

Like that that's kind of like it's run, like how I'd run things as a lead engineer at event right, it's run in that way.

If like we have to have this be small incremental steps that are all achievable by themselves, rather than Andrew goes into a bunker for a year and writes it.

It's it's it's a lengthy document, but I'm I was struck when I read it that it was how iterative it was.

I mean, I was quite impressed because it's I know it's very difficult to not just have the idea, but to have the roadmap.

And um, yeah, I guess that that makes sense that doing that professionally guides the process of then doing it in an open source way.

Like I've I've taken things from open source, like documentation culture, into a vetbrite, and that's really helpful.

And Simon has too, right?

Like there are things that go both ways, and like running large projects is not a thing you see a lot in open source, and I I'm determined to not have this be a burnout center again, right?

Like that that I I've learned from the last two times, I would hope.

Um, and like even if we just do the first part, but that's a win, right?

Even if we just yeah.

Even if the if it even if it's just the view layer that gets, you know, the that gets async, that's kind of okay.

It'd be nice if we get to the ORM and we get all it, but if we have the view layer, then we can write proper async apps and we can embed other ASGI apps and we can do all sorts of really exciting things with just that.

And right.

Once you've opened that door and you can do async def views, then like the community can fill in things like if you want to do an RM, you can, but maybe you can pull in like Tom Christie's databases library and do async stuff with that, right?

You have all these different options.

And so that really to me is the way of doing this of like, let's get that first thing unlocked.

And like that first piece of work is arguably the most like solo piece of work.

Like, like I don't intend VP done by me alone on trying to some collaboration, but it needs to be tightly controlled, needs to be like have high, like I'm not saying the word high touch, but like lots of contact between different people who are working on it and make sure it's a singular, a singular vision so that it comes out right.

Everything else can be like, you know, like async forms, async RM, async cache can be farmed out to different teams.

That's gonna work fine.

And hopefully it's a new contributors as well.

Like I'd love to be like, hey, like we need the cache layer to be async.

We can write you a clear definition, we can give you some guidance and mentorship.

Let's bring on some new um contributors to Django as part of that process.

So it kind of almost has to be right or nearly right first time.

It it it you know, if we mess up the form layer, well, we can redo the form layer, we could have, you know, it wouldn't be the the end of the world, but the the view layer would be would be annoying to have to introduce breaking changes later on.

You immediately jump to like 31.0.

Like most of the rest, like you know, I remember when the current forward system was called new forms, right?

Like most of the rest of Django has changed over that time, and the view layer has mostly stayed the same.

Um, like one of the things people don't realize is that Django predates Whiskey.

Um, like Whiskey basically came about kind of from Django and Turbo Gears and other contemporary frameworks, and so like in some ways, Django doesn't have a like Django is not Whiskey all the way through because it predates it.

Like you go look at like Flask and VirgSoig, like they are basically written around Whiskey.

I've been I've talked to the Flask maintainers um and like David Lord about like what it would take for him to support async, and he's like, Well, like it's so deeply ingrained in our stack that it's a lot harder to untwist, and it's possible.

I mean he's got some ideas.

But whereas Django, like we have that distance, which means that both it's very old code, but at the same time, it's kind of not too totally tied to Whiskey.

And that's great that that's a yeah, major strength today, that abstraction.

I wonder, yeah, I sort of wonder how Flask will handle that.

Um, that we get the place for the same.

Yeah.

It's but again, a testament to the original developers, it's run fine for a decade, right?

Um, but yeah, again, like I want to touch it once.

I want to, and crucially, I want to keep the same kind of abstraction because like I think one of the things that appeals to people at Django the most is like that very simple like mapping of like, hey, you have a function or a class, you put it in a URLs file, bam, it's done.

There's no like weird routing, there's no magical objects, there's no like tracing attributes or like implicitness, like it's very explicit, and I want to try and keep that design.

And so the the ASCII ref, the the the ASGI standard is is just gone to its 3.0 version or its version 3.

And so can you talk just quickly about the ASCII standard and is that kind of stable now?

Is is 3 gonna be around with us for a long time?

Um so uh in many ways, ASGI was one of the goals of the channels project, um, which was to like channels was both trying to make WebSockets work, but also research into what it means to make an asynchronous Django.

Like channels was always meant to be the predecessor to Django itself becoming async, but when I started that project, I had no idea what that would look like.

Um and we also didn't realize WebSockets wouldn't be nearly as popular as we thought they would be, right?

Like these days it's like nah, like HTTP 2 has made like Ajax a lot more reasonable, and like LOM polling is still around, and sockets are a pay, they're still used, but they're not as popular as maybe they're not like every site's not using them.

And so ASGI sort of ASCII one was um this sort of weird internal channels format.

ASCII 2 was born out of the channels 2 rewrite, which we'll maybe go into in a little bit, um, but it's basically sort of like a better version, and then Tom Christie has been instrumental in in ASCII 3, of like running a full ASCII framework and using it in production and having channels used in production and talking to other frameworks.

So, like, you know, like we've had weigh-in from a decent number of Python web frameworks on ASCII 3 to try and make it something that both seems implementable by servers.

Like I've talked to a few different server authors too.

They're like, this seems okay.

Like no one says it's good because no one wants to write new server support, but like people sit generally seem like it seems fine.

Um, and web frameworks agree with this too.

And crucially, ASCII 3 is the most looking like whiskey yet, um, which I think was always the goal of like it's meant to be a spiritual successor, but not a direct successor.

Why, you know, what's this?

Well, you know, what came from Tom's talk was very clearly that, well, we don't want to have to change language just because we need that extra throughput that uh async will give us.

And so Python's got to have async, and ASCII's a great candidate for a Python standard for uh web frameworks there.

And then, well, why does Django have to have it?

Well, because Django's the batteries included web framework in Python, and so you it may not be that we need everything that ASCII could offer, but we need something, you know.

You we don't want to have to change a web framework just because you want to build async views, and so I think it's really important.

Um, and that well, that was kind of interesting from the conversations that Django got around that.

Like migrations is a very hard problem, and I think async ranks in the same category.

Like it's a thing that we should solve so everyone else doesn't have to.

Like anything security related or like networking related, I think, falls in this category.

Like Django, like, you know, for example, doesn't provide things like you know, nice looking UI libraries because it's it's better solved by other people, but like solving async is a thing that like Django kind of should treat as its core competency, right?

Like it has to be baked in.

And in the security chapter, basically it's uh you know, you can take off the defaults that Django provides, but think pretty carefully about doing that because it just gives you the guardrails and it does give you the option to take them off if you want to write raw SQL or you know, but if you don't want to have CRS, CSRF tags, but it just sort of handles it for you, right?

Because security has also that, oh my god, it's such a deep well of of knowledge.

And once you get beyond the sort of basic attacks, it's it's almost kind of infinite.

And that's the point of these the uh the monthly updates to Django and all the work.

So yeah, so one of these areas that's definitely if someone else can handle it for the community, or the community, a small subset of the community can handle it for Django developers, it's a huge benefit and doesn't need to be redone each time.

You can remove the guardrails.

Like some web frameworks don't have that, right?

Some web frameworks are all or nothing.

Um I think famously like like I don't wish to talk bad about Rails, but my experience working on Rails projects was like Rails was an all or nothing.

Yeah, exactly.

Um I spent a f I've a good six months working on Rails projects and and it was uh very informative.

Um but where it's

Should I use Django?

Yeah.

Because Django then makes it completely customizable and you can swap out what you want, but it comes at the cost of more upfront time.

And so that's kind of what I tell people on the ease of convenience.

Rails is pretty much near the top, but when you want, you know, the fact that Django is, I don't know, three quarters, 80% of the way there, but lets you customize it with experience, you appreciate that trade-off, I think.

Um, but it but it's a little bit but it's a little bit of a bump when you're just starting out.

It's not all magic.

Rails is definitely easier from from like step one.

But like Django strength come like you know, like Eventbrite still runs Django.

We've replaced like half of it in various pieces, right?

Like we run a different like user authentication framework and cookie system, but like we kill still have the old CSRF system, right?

We run a different cache layer, but we still run things like the day the ORM pretty much untouched.

Um, so that's kind of part of the way, like the part of the value I see in Django itself.

Yeah.

Because I was wondering, I know they have Facebook has its own solution, if that's um if you if you can speak broadly, is that like where are the pain points at to the extent these a couple sites that are at massive scale that use Django, where does Django kind of break down at super scale, or is it really just specific on the needs of the project?

Um but like one of the problems we face at Eventbrite is that running at scale means having like it's it's more a problem of engineering teams, right?

Like you can't have all the teams running on one big project.

Um it's much easier to have them working on like separate, for whatever word I say microservices, but they're not micro, they're sort of like medium-sized services.

And what that means is like you now have this whole like service call and routing problem you didn't have before.

And crucially for async, you have this thing where like you're gonna your like main view or template logic is gonna call like four or five other things to service different parts of the page and wait for them all and come back.

If you wait for those all serially, you're gonna be waiting a lot longer than if you call them all in parallel and get the results in parallel and put them all on the page.

Because those things act independently.

I think that's one of the things of Django, like at least I've seen talking to other big sites too, is that like making sure it can split up more and more and become more independent like that.

was like.

Because lots of people took channels one, they built good sites, they were using it very well, and yet you weren't happy, and other people using it at scale perhaps weren't happy.

So, what was the what was the issue with channels one?

Can you if you could explain that?

I think that's interesting.

So that's like if I make a big jump like that, it's because there's a good reason.

And in this case, it's architectural.

Um, channels one was written in architecture where you had a server that received WebSockets and HTTP requests, you had a separate server that ran Django and the functions, and both of those talk to a central bus, which was called the channel layer, which is basically Redis for what for most installs.

This is an interesting architecture in theory, and in again, in theory, it's it it scales really well, but in practice it doesn't.

Um, in particular, it's very much it's much more fragile, and you're adding in another point of failure to the web flow.

Like you've gone from running a single server process to running two different processes and also a communication there between them.

And in the end, like I just couldn't overcome the difficulty of doing all of that at one time.

Like deploying it was massively confusing.

And in many ways, the reason we did that was because channels one, its goal was to work with the LCS at the time, which supported Python 2, which meant we couldn't use async.io.

Channels two is a re-examining of the same problem that channels one was a solution to, but with the different constraint of you can support 3.5 and up.

And like given those two different constraints, we end up with two different designs.

We're like to support Python 2, you can't have asynchronous code.

You have to run a different process.

Like Redis was your asynchronous scheduler, basically.

That's how it worked.

Whereas with channels two, we're like, well, we have async.io now, we can do this internally in in Python.

And sure, there are a few issues with async IO, and it's not totally perfect, but having it in process is way easier.

And having the deploy bee run it like Whiskey is also a lot easier to convince people of like, hey, like you just run a server and pass an application object like you used to, rather than running a read Redis cluster that you can't shard properly and all that jazz.

Right, and because you you as well, like deploying Django is hard enough for your average um project without making it extra hard by having this whole DevOps layer.

I'm very impressed it worked as well as it did in production.

Um I'm very happy that it did, but it just wasn't conducive to long-term maintenance, and crucially, it's not a pattern we could have taken Django itself into.

Like we could never make Django look

like that.

Okay.

I'm curious how how how do you tell that you're burned out and then what sort of techniques have worked for you?

Because I think this is something that hits almost everyone.

Maybe not Simon.

But uh for everyone else and for yourself, what kind of how have you spotted it and then what's you know what works or is it beyond just taking a break from all the extra work that you're doing?

But like with that caveat in place, like my my personal signs are generally a lack of enthusiasm, um, mostly for doing anything, right?

It's almost like apathy.

So like I like I go from like um like the two of you who are here can see behind me on the video that I have like there's a sewing machine, there's a 3D printer behind me, you can't even see the other like I do a lot of making in my spare time, right?

Like when those hobbies start suffering is when I know I'm suffering from burnout.

Um it's when I'm like like like my making and 3D printing and and fabrication and like my my other like crazy art projects suffer.

That's when I know like out overall I'm being burnt out.

And then the way to I found to do that is to just step back.

Um I've been more forceful with myself over time.

I used to be like, oh, if I step back, like I feel guilty, like the community depends on me.

Like everyone's felt like in a different way, be it like your company or your spouse or anyone depends on you.

Um, and in the end, like I found that like just you know, like with channels two at the at the start of the year, I was like, like I have realized I'm burnt out.

I give myself there's a I get I get a month, right?

There's a month to try and find a handover period.

And Carlton very thankfully stepped in to help at that point.

But I was like, like, you know, it's better to take this project, say like it is officially done, than to try keep pushing through the burnout.

Because like this, but a big problem I have with documentation sometimes too, is like it's much important to say, like, this, like, be honest, right?

This is not maintained, is much better than saying, like, oh, this has the look of being maintained, but it's not actually under the hood, and that's very different.

I like absolutely

Um, the only feedback you ever see as a maintainer are the problems people have.

Like, like I for like seven, eight years of South, for all of migrations and for most of most of channels, like I only ever saw people going, it doesn't work for me, it's ruined my life.

Like, I never like the only time I ever got positive feedback basically at conferences.

People come like, oh yeah, we're using your thing in production to like three million dollars.

I was like, What?

Like, so like I I encourage you, like if you're listening and you have and like there's an open source project you use that you think might not might be in this problem, like an email to the creator or maintainers saying thank you is always welcome.

Like I get a couple of those every month, and they're always welcome.

Like just plenty of people reach out and go, like, your work is amazing, thank you very much.

Um, like it honestly, like those are what keep me going.

Like that it's it's you know, otherwise you just see like everyone trying to do weird things in databases and not succeeding, and it just it feels like a a huge load, right?

Like having having that empathy for everyone is difficult.

So it's it's certainly helpful.

And yeah, even for me, like I in a way, books are a form of open source, like numbers don't mean anything, it's the personal emails, which aren't that many that uh make it meaningful and push through the sort of the negative, like, yeah, this this is broken kind of stuff, which does get tiring.

Like like when we work in a volunteer community, it's important to respect each other and make sure like each we we like, you know, like I try and make sure that I thank everyone in person, wherever I can, who does work that helps me helps me out.

And I think just you know, if we just have more of that going forward, and like the Django community is already one of the best communities I've ever been in for open source, right?

Like it may I joke that sometimes I stick around for the people, um, but it's it's kind of half true, right?

Like the the group of people around Django are some of the nicest, kindest, and best people I've ever like had the pleasure of interacting with and working with and supporting and just being around, and that's that's a huge part of it.

I've been using Django for 10 years, I'd never been to an event, and I came and I was like, Oh wow, this is what I've been missing, and it was just already there, and in contrast to other environments where it perhaps not, it's like it's really noticeable.

It's not just because I like going to different different random states in the US, it's also because like the there are people there who are very like I see them twice a year, but they I see them some of my best friends, right?

That's kind of the way Django works.

I experienced that, and but most people can't go to Django Con yet use Django.

So that was really this podcast came about in part to open up people to the broader community and the people like yourself who have these amazingly outsized-sized roles, yet uh you know, I would say the majority of people using Django don't know who you are.

Um and more of them should.

Like one one of the thing Well, yeah, yeah, like maybe one of the things that we've got to do.

It's nice to um yeah.

Uh what one of the one of the weirdest experiences for me was Django Con 2009, 2010.

Like what second or third one.

Um, someone came up to me and went, Oh, are you Andrew Godwin?

I was like, Yes, so I know that.

Oh no, I I've just I just know you from online.

Like that was the the turn of like people knowing who I was without me knowing them.

And it's honestly a very strange experience.

Um, I I've got much more used to it now, and like I try and meet new people every Django Con and try and push beyond my admittedly like very difficult like introvert boundaries to meet new people.

Um, but at the same time, like, yeah, there's a lot of people that like DjangoCon is by its very nature has to be an exclusive event, right?

Not everyone can travel, not everyone is in the right country or can get visas or whatever.

And so I think it's also important that we make sure Django is supportive in other ways too, and like we have the a good community online and you know, in different mediums like that.

At the one of the sprints at Django Con Europe.

They're like, Oh wait, who are you?

Oh, you're that podcast person, right?

Yeah, yeah, yeah.

Well, and I I mean, one thing with the Django community, uh from where I sit, so because I think I sort of sit hopefully maybe near the beginning of the tunnel, and you guys are both at the you know, very Mount Everest of it, is there's so many people learning Django for the first time who it's their first web framework.

Yeah.

Or um, I mean, just the variety of people I interact with is unbelievable.

It's you know, it's so it's undergrads doing computer science.

I mean, just last week it was multiple tenured Ivy League professors in, you know, economics or some other profession who for or field for whatever reason want to learn web stuff.

And they're, you know, you know, one of them is like uh on track for I I looked looked him up on track for like Nobel Prize uh in economics, and he was stuck on chapter three of my book with Heroku.

And I was like, oh, well, you know, it's I got stuck on it too, like a lot of people do, but it's this both this great equalizer and also people are just using, you know, Django and web as a piece of whatever they're um already doing because they see that to be knowledgeable in the modern world it's such a a benefit.

And so it's um I you know, I just see all this crazy upside to Django as being one of, if not the default, entryways to web development for people, you know.

So I I think I would bet the Django will be way bigger than it is now in a couple years because those all those points are going the right direction, I guess.

I mean you I I met you for the first time at PyCon.

I mean, PyCon is really not about Django, and yet it's massive.

Yeah, exactly.

Like it's very easy to fall into the thing like, oh, like we've all been working on this for like five, ten years, we're now all experts, we should solve the expert problems.

And like one of the things I try not to lose per I I do sometimes, I try not to lose perspective on is like the beginner view.

Like, what does it mean to sit down for the first time, use this thing, and try and understand the con like the web is still revolutionary, right?

Like we need to keep empower people in that way.

Like, what is it to sit down and stand this and like not know how anything works?

Like, I don't want people to know what async is until like at least you know, chapter nine of any potential book, right?

Like they should not have to care about it.

I mean, so that professor, he was it was the structure, he was using start project and he didn't include the period.

Um, you know, even though I specifically said three times you need to include the period, even though I had the source code, you know, because uh, you know, so it's so I'm very aware of like of you know of that thing.

And even actually last week I got on Friday, I got stuck on a no reverse match error, and I was having an existential crisis over it because I'm just how much time have I spent on this?

I can't believe I'm having this issue.

But you know, I I asked Carlton, uh Jose Padilla, a whole bunch of people, and you know, it it wasn't like everyone immediately saw it, even among you know, this like murderous row of Django people.

So um programming is very humanizing, like that.

And it's just uh async async doubly so because like I I often struggle to think about like I was taught like asynchronous like proof theory university, which is like it's one of the very few things that come back to go.

Actually, that was useful, I and I have used it again, but like it's so hard.

Like I regularly sit down, just like just sit there, like lie on my bed with like a drink of like I see or something going like, what is this problem?

How do we solve it?

Like it's not easy.

But I also think we should protect you from it unless you really want that.

It's not right, it's not gonna be like that.

Um, like it's like you've got more to think about, there's there's more to have in your brain at once, and I honestly will find any excuse to write synchronous code I can because like synchronicity and the gill and everything, sure, it's slower, but it makes it so much easier to think about and understand.

And like I value that sometimes.

Like if the code's not performance critical, that's great.

And like that's and most yeah, most code isn't exactly yeah.

Yeah.

We're gonna link to your um proposal.

What are some what would you say if someone says, well, I know Django, but I'm async is still new to me in terms of, well, here's how you uh learn more, and then here are maybe some areas that could use help on.

Um but there there isn't

Go is not quite it's similar but not exactly the same.

Like just like interesting, like like how do you even think about the problem?

How do you reason about it?

Like that's what's missing.

Um at the same time, it's Python, so I suggest you just like find some, just like find a project, just hack on it.

Like just make a little thing that fetches URLs in parallel.

Like the classic example is you want to fetch like 10 URLs in parallel, and you can do that in like five lines of asynchronous Python.

But like do that and like just break it and try and work out why it breaks.

Like that's that's my approach to stuff.

Um and let's say and for Django ASIC in general, um, it seems like uh DEP 9, which is the the big long document you referred to earlier, um, is uh seems to have sort of settled down and the feedback slowed down, so we're gonna start pushing it through to the proposal process for the technical board.

Um and once that's done, I'm gonna start working on both a how to get involved um sort of blog post or document and also a funding one because the funding is a big part for us.

Um so like look look forward to those.

I don't have them yet, unfortunately.

Um I took a

But uh at time of recording, I just come back from uh two weeks of vacation and a week of PyCon before that.

So uh that those are not times when I get get stuff done, should we say?

Anything to add on that front?

Oh, well on that front in particular, no.

Um, one thing you mentioned a couple of times is documentation, and I know you're really keen on writing good documentation, and the channels docs are just phenomenal, to be honest.

So can you talk about your philosophy there and stuff?

Because I think it it's something that a lot of people could learn from, and I've learned from myself.

Um so, so like, like I write documentation for myself in the future, is is the line I use, right?

It's like, well, okay, like what it and to me, documentation is not about how how to use, it's about how to think, right?

It's like if you are approaching this project, here is the frame of mind, here is the way that as the author or as a maintainer I approach this project, and like that will by itself convey to you like how to approach this.

Like, like the channel stocks, for example, a good portion of his concepts.

It's like, here are the concepts, and here are the gl here's a glossary of the words we use elsewhere, because that will get you so much further than me guiding you through it with with like hand on hand, because like I could give you lines to run, but you wouldn't understand at the end.

Like, my philosophy is always like documentation is to teach and to inform you a thing exists and to show you how to approach it, but and like you do want like detailed instructions when you get there, but you have to have the other parts too, and that's kind of how I approach it.

And I think when you say that, the fact the channels docs in particular, why they're so great is because you also do sort of a tutorial on it.

Yeah.

Um you sort of meld those together, but um that's a lot of work to do.

Like there is presumably a part five or part six of the Django tutorial that is and now async with some jazz hands in it.

Um but like as I said, like you know, there is no good tutorial out there to learn how Python async works.

I think it probably then falls to to uh Django as a project to supply that.

Um my hope also, like, you know, documentation is a crucial part of engineering.

Like if you if you haven't even talked about engineering, like engineering is part programming, part documentation, part communication and collaboration.

And like those parts are all pretty much equal in my experience.

And people often put way too much emphasis on programming and not enough on the other two.

Like written and verbal communication is incredibly important.

And so a big part of this project is like I want to bring on people that say, like, hey, like, like we just need people to who are good at explaining and understanding.

Honestly, like beginners are the best at this.

Like, if you are a beginner or relatively new to Django or even to a concept, you are in the best position to explain what was difficult about it.

Like your example earlier of the period, right?

Like, obviously, you you you anticipated that one, but there are things I can't anticipate that like beginners go, oh, I found problem X really hard.

I was like, Oh, that never even occurred to me.

And so, like, there's an amazing opportunity here for like beginners and people even like one or two years in, they still remember.

Like, I've forgotten what was hard about Python because I started in like Python 2.4, where people are going, oh, decorators are new and exciting.

Like at this point, like I'm not, I I'm on some ways a terrible teacher of Python.

People come to me and go, like, oh, how'd you learn Python?

And they're like, I can tell you, but it won't help.

Because like all the things that were there in 2006 are totally irrelevant now.

It's a different way of having it.

They've gone.

Well, I find if you just ask, if you think something simple, ask someone next to you to do it.

Like, like install Python, here's you know, on a Windows machine, go for it.

Like, and you're like, oh my goodness.

But yeah, it's it's it's so many things like that where I want to like make sure that again, like we talked about beginners earlier, right?

Like, beginners are a crucial part of Django's ecosystem.

I've always wanted to make sure, and like so are big companies.

I don't want to have us like captured by big companies, right?

Enough people who work with Django and Python in general work at big companies.

Like, I want to I want to occasionally forget about the big scale problems and come back to I'm writing my blog, like how do I get that stuff working properly, right?

And like I do still maintain my own Django power blog, half for this reason.

Um, it is still on Django 1.10, I think, unfortunately.

Um, but it does use migrations at least.

Uh and like that's kind of my my test bed for like I'm just like a normal like contractor again.

Like, how do I handle this stuff?

And it's a useful spectrum to have, I think.

Yeah, I think why is oh go on.

So in some ways that solves some problems in that uh it doesn't seem like you feel the burden of building something custom for Eventbrite or someone else for a specific company.

It's it can be done in this you know community-based way, whereas many other features on other projects, you know, do have that, you know, a large company wants something custom and then you have trade-offs.

Like Kubernetes is pretty much written how I would think about an enterprise-level orchestrator, which means it's very hard to get into and you can't really run it on a single box.

It's getting better, but like that and that that's what I wanted to avoid.

And part of channels one and channels two, and like the years of research and talking to people has been working out like what is best for everyone, like talking to different people up and down the spectrum in different countries and different industries of like, hey, like what do you need?

Like, what's useful for you?

And I think like the culmination of that is like the async proposal is pretty much like it's the node of like here is the place where all those needs cross, and like everyone everyone can agree on this stuff.

And I I I'm hoping like the general lack of people going, oh, this is a terrible idea, is is proof of that, right?

Like everyone seems to go, oh yeah, no, we do we do need this.

Um, but also it's it's a very difficult thing, and like, yeah, like if it was just for Eventbrite, it'd be a very different solution to the problem.

But that's not my goal here.

I think minor agreeing with Andrew.

Uh okay.

So I'm sorry.

So you uh didn't your mum tell you about that as well?

No, that's all super.

I mean you have a lot of other projects as well, Andrew.

What would you like to say as we wrap this up?

Um like it if you like if you if you're interested in like electronics and fabrication, like you know, I have a YouTube channel with some stuff or or on on Twitter.

But honestly, like the the thing I like if you listen to this podcast, you're probably involved with Django.

Um what I encourage you to do is like look out, like we'll get a post on the Django blog probably, but I'll tweet about it as well.

Like, like look out for like us talking about like how to get involved, and like I really want to run async in a way that not only brings in new contributors, but also like is spread out properly, and like we spread the workload and we have everyone understanding and agreeing on on how things work.

So I I'd love people to like even if it's just like copy editing documentation or write writing some new tests, like there's so much work that we can do that can be paralyzed out that you know don't don't feel shy about stepping up.

Um, don't feel like you're gonna have the burden of the world coming down crushing on you.

Um like it's not like that, at least not at first.

Um and yeah,

Yeah, exactly.

Like, not even with this, like maybe a local like Django Girls meetup or like PyLadies or other meetups, like open source takes volunteer work and it takes many volunteers to make it sustainable.

So that that's my call to action.

All and all I'd just add on to that is that as well, if you feel like you need help with that, reach out because there are people in the community that will support you in contributing if you want to contribute.

I have the I it's I've seen it time and time again, like especially in the Django and Python communities.

Um, I generally expect the best people I'm usually rewarded, which is which it says a lot.

Yeah.

So regardless of the question, it's it's sort of validating and and it and it provides and it's interesting because it provides that lens on, oh, I've I forgot that this was hard, or I forgot, you know, it that makes sense that with your lens, this is how you think about it.

So it's it's helpful as well to even someone who's an expert.

Totally.

So reach out.

Well, thank you so much for taking the time to come on.

Um as ever, people can find this episode on DjangoChat.com.

We're on Twitter at Chat Django, and we'll have links to everything in the show notes that we talked about.

So thanks again, Andrew, for making the time to do that.

Thank you for having me on.

It's been great.

All right, bye.