Transcript: Boost Your Django DX - Adam Johnson (Ep105 Replay)

I'm Carlton Gibson, joined us every by Will Vinson.

Hello, Will.

Happy New Year as well.

And uh today we've got Adam Johnson back with us, who's uh um, you know, we you know who Adam Johnson is, but he's uh Django Technical Board member, Django author.

So um you've all got speed up your Django tests, and coming up now soon is um improve your developer, Django developer experience, which we're gonna talk about today, and various other things.

Adam, thank you for coming on the show again.

Adam just won the Malcolm Krednick Memorial Prize.

Yes, he did, which is well deserved.

So it was a very, very easy choice um for the board.

Congratulations, very well deserved.

Yeah, well done, Adam.

Like um, you do you're massive.

You're doing posts all the time, you're you're putting out packages after packages after packages.

You're always there on the mailing list, you know, giving people the the welcome pointers.

So super.

Yeah, I'm honored to receive the prize, and uh it came in uh the worst year of my life, so it's uh nice and helped me a lot.

We have lots to talk about.

I would maybe I think it's gonna be a lot of the two of you talking, but I would love to tee up the Django technical board.

Um maybe we could tell folks what that is and what they do since Adam, you're on it, and that's quite a big deal.

It's also a relatively new thing, um, and how that works with the Django code base for new releases since 4.0 came out just a couple weeks ago.

So the technical board is five members uh voted in for each release cycle.

Um with uh where the buck stops with decisions on whether things go into Django or not.

Like quite often, like features are clearly like a good idea, so it's okay for a few people to suggest it and the fellows to go and review.

But uh for larger scale changes or contentious changes, um a technical board vote would be held and we'd need uh consensus whether or not something would be added.

Um in the same way for a community, that's what the Django Software Foundation handles non-technical things, but the DSF has no sway over the technical board decisions by design.

I actually I should know.

It's every number release.

Um there's like uh maybe eight or nine people who've stood um over that time.

So most people who are standing are are are getting elected at least once.

Um it's just kind of cycling between a few people, I think.

Um and part of that I think is because we still have a quite narrow, experienced contributor base.

Like the the qualification for the technical boards, you have to be involved in the day-to-day running.

I mean, you don't have to have been there forever, but you you do have to be active on the discussion list um and things like this.

And we still don't have a very broad contributor base there.

So to you know, so that there's not necessarily a wider pool to draw from, which is something we you know that we kind of think about.

It's very hard because you have to have been contributing for years before you're really eligible to be on the technical board.

You have to have some deep knowledge of various parts.

Like it's chicken and egg.

Yeah.

So it's you know, you can don't have you don't even have to be an individual member uh to be voted on, though it is voted on by the individual members, so you do need to be known in the community.

And um all those things are true.

Um, I think one thing I found this past year, I think it's basically for the first time since I've been on the board, this is my third year, we had basically all the same people re-elected.

And um diversity is nice.

I mean, I'm not planning to stay indefinitely, but it is easier logistically to have the same people because certainly for two years terms, because you kind of know how to do things, whereas initially that first couple of months, people are ramping up.

So when we my first year there was or my second year, there was like half the board was new, and so a lot of time was spent kind of getting people up to speed.

So anyways, internal baseball, but maybe longer term.

There is something I think to multi-year terms just to kind of avoid the ramp up and ramp down, but at the same time, a recycling of voices is healthy as well.

Absolutely.

Isn't it's every is it every major release cycle now?

So it's every eight months or every two and a half years for the technical board.

Right.

Yeah.

Yeah.

It's a ceremony that goes very past very quickly, doesn't it?

You know.

I think there's something to, you know, perhaps a you know, three out of three, two or two two, three-year term, just to avoid the nuisance, because it does take up quite a bit of time.

Um anyway, so something for all of us to discuss with our bodies.

So it not quite often we discuss on the mailing list, and we just come to a consensus, right?

Yeah, and it's it's but there are a few issues.

So one that has been really important for um for was the discussion about typing because there was no consensus at all.

And um, you know, there was a depth and there was massive discussion on the mailing list, and there was a you know, maybe even being draft PR, I don't know.

Um, but you know, Maris and I just aren't in the position where we can go, oh you know, we decide to merge this or we decide to close this.

So we asked the technical board for that, and you were involved in that discussion then.

Um so it would be nice to re um reapproach that decision at some point.

Yeah.

Um yeah, the I mean the second point is who's gonna do the work because it's it would be a lot of work to add the type and it's integral.

Um so would it be fair to say that you know you'd be you'd be keen now to think again about adding those?

Um I mean my one thing that I've I've kind of been flirting around is with around the widget API because you know there's lots of there's lots of areas there where it's like what do I get?

What exactly do I get out of value for data dict here?

And you know, I could really I could really appreciate some typing there.

But my kind of thought is, oh, I wonder if we could do this just in say the widgets file, you know, just in one kind of module where it's it's at the the kind of leaf, you know, that the widgets don't really impinge on anything else, they're not impinged by anything else.

Could we type those?

And that might be a sort of progressive way of doing it.

I don't know.

Um, yeah, all these leaf modules, like all the things in Django UTols, um, there'd be perfect candidates to begin with.

Uh but we'd need to discuss is there value in adding just a small amount of type pints initially, or is that gonna make it harder for people?

Because then they're gonna have oh, we're trying to use the type pints, but everything else is showing up as any and causing us headaches.

Some research is definitely required.

Interesting.

Interesting.

Um so as I went to bed last night, I saw there was a there's a new security release, and this morning I see in the notes, Adam, you've done up a whole blog post on it, which is insane.

So I wonder if we could briefly talk about that.

And that's a chance to mention that you're also on the security team of Django, which is a separate group.

Yeah, for a few months now.

Well, I just assumed you were part of it the last couple of years.

Let me just give a bit of background there.

Um, so there's a I don't know, about 10 members, some of the more uh the older contributors that have been around for and you know, they're super knowledgeable, um, but not necessarily involved in Django day-to-day anymore.

And, you know, uh there's a f a few people who are still very active, like Marcus Holtzman and uh Florian Apollon.

Um but we were finding that you know it was me and Marius, and maybe Marcus, and maybe Florian commented, and we we just needed some some more people who sort of directly involved, kind of very regularly.

And so um amongst the security team, we talked about it with Adam and Nick Pope came up as and so we invited both of them and they came in and they've been super helpful and active because we get a lot, not a lot of how many app reports would you say we get Adam in you?

Would you say that's about it?

Yeah, yeah.

I mean I I'm I'm I'm not a security expert, and a thing comes in and it's like I don't know, is this and without the security team, I there's no way I could handle those, you know, those issues.

Like I mean, yeah, okay, some of them it's easy, no, that's not a problem.

Some of them, oh yeah, that clearly is, but then there's this vast area of that's a really difficult question.

Um yeah, because we'd have a lot of back and forth before even our pull request is open, um, to be like, oh, we could do this, we could do that.

Um yeah.

So there were there were three fixes into this release, right?

Yeah, shall we run through them quickly?

Okay, so the first one was a denial of service possibility in user attribute similarity validator.

Now that's a mouthful, isn't it?

Um that's a password validator that um checks that the password given isn't too similar to a user's other attributes, like their email address.

So you they're not just recycling their email address or adding a one and exclamation mark at the end for their password.

Um, but uh who was it that reported that one?

Uh Chris Bailey reported that if you submit a very large password, then it has a kind of n squared or exponential runtime, I'm not sure which.

Um you have to submit like a very large password, like a hundred thousand characters, but then that can lead to like many seconds of processing on a on an average server.

And you know, submitting a hundred thousand letter A's over Gzip is not much of a request body for the client to send, but then it can result in in many seconds of processing, so it becomes like a denial of service vector.

Someone submitting loads of these requests could cause your server just to hang up.

So the fix that um I came up with along with Florian, Colton, and Marius uh is to avoid checking the similarity if password if the password is like so much longer than the attribute value that it's not going to be similar regarded as similar anyway.

I mean, are you like who do is you didn't did you come up with that or is that something that other security researchers have used as a best practice?

I don't think there's much groundwork for using this kind of algorithm out there.

Um yeah.

And then there was various back and forth and different approaches.

It was really nice.

This is Adam's point about like everyone throwing in their ideas is that there were two or three different takes, and it's like, yeah, actually this one's quite sweet, and you know, it's a nice ratio that this is a realistic cutoff at different scales of input and whatnot.

And then there's two more.

Okay, we'll go through them a little quicker because I'm a bit less familiar.

Yeah.

Um, the second one was with the dict sort template filter.

This is one that I've not seen used, but it takes a list of dictionaries and then sorts them by a key or perhaps an attribute.

Um and uh security researcher at Sonar Source, Dennis Brinkrof, submitted us a report that um you can use it to access um like arbitrary attributes and inside of a string you can access the characters of the string one by one.

So if a user managed to submit which order to sort your list of dictionaries by, they could sort them by character zero, then character one, then character two, and see the sort order changing, and then they could infer the value of that attribute.

And if that value was like security sensitive, like a password or API key, they can kind of infer, okay, this this user appeared at the top of the list, so their password starts with A, and then when I sort by character two, they're in the middle of the list, so that's probably something around L.

And then it can kind of combine that with other info to get a statistical guess at what their what their secret value is.

I think that's

So instead of going user slash one or right, because as you said, it could apply to bank records or something important.

You don't want someone to infer the order or the number.

And then number three, I can read it.

So there's a potential directory traversal via storage um dot save class, which so that's a file storage API thing.

So when you're uploading a file.

And unless it's something I directly have, don't get a chance to fully dive into it or even know the process behind it.

So yeah,

And there was a number, and this is kind of like the last one.

And you'd have to use the storage directly, which kind of nobody does.

So it's already been bulletproofed around the outside.

But the idea would be that you know, you you go from the media file, the media directory where you're meant to be saving it, to somewhere else on the file system, which you know you're not supposed to have access to.

No, you know, if you do everything right, that can't, you know, that's already ruled out by various other bits in Django, but you know, this is the sort of last bolt down on that area.

Um, there might be a bit of a rewrite coming in the future.

Who knows?

It's the last possible one in that that that cluster of you know, that that cluster of upload handlers and files, and we because we've gone over them.

No doubt there'll be one next week.

Um, it's pretty, pretty good.

This was again by um Dennis Brinkrof and Sonosource.

So I guess they were running some kind of scanner or just deciding to audit open source projects.

So yeah, they found it.

That was cool.

So 4.0 came out.

I guess Carlton, this was in your your week notes thing where I saw where um I don't think it was on the release notes, where you listed all the active contributors.

And Adam, you were one of the largest ones.

We'll link to that.

It's not in the release notes, right?

I don't think the formal no,

No, so I've so so one so what I've got for 4.0 is a breakdown of new contributors, of people who like I've called them on a roll, people contributed in the last in 3.2 as well, and then people who didn't contribute in 3.2 but had previously and could put those in the category of welcome back.

And what I want to do is write that up and then put that somewhere either on on my site or on Djangoproject.com if we can find a suitable place for it, and then keep that rolling for 4.1.

And as we go forward, that would have happened already if it weren't for you know 2020 or the usual excuses.

Um but that that's coming, and the idea there is just to have a bit of fun calling out contributors and to you know, particularly for new contributors, you make a contribution.

If can we find a way of just flagging that?

And you know, I it was it was on my list to get done for the end of the year, and that just didn't happen.

So, okay, over the next month or so I'll pull it up and you know finish finish that off and have that go going forward.

But that that the idea there is to do something to call out contributors a little more than we have, you know.

Um, I don't know.

It should it's just a bit of fun.

I think it's important to highlight those things, you know.

Like it's it's all right if you're sort of economically privileged and blah blah blah, you can contribute um loads of time to open source and you can spend the time to get a contributor record, and you know, but if a lot of people they need some recognition for it to, you know, so I've made one contribution.

Did that did that pay back to me?

Well, no, it didn't because it just disappeared into the void.

Whereas if we can put it on a website somewhere and then they can point to it, it it adds to a little, you know, for someone getting into tech, it adds a validation point that might help them get a job, and then they think, oh, you know, I can contribute, you know, I can continue.

We need to to to do things to smooth the pathway there, or we're never going to change the situation we're in.

It's always going to be a slow point.

We're not gonna you know instantly have a super diverse contributor base.

But if we don't do these things, we're never going to have that contributor base.

Like I think you tweeted there was one template, like there's like a one-line thing with you and Marius, you know, and then you have you know signal receiver function stuff.

You know, so there's I think it's great that you highlighted that because that's something where it seems small, but like that's exactly an onboarding way versus going into like a core piece of Django and thinking that's what it takes to do a PR.

Right.

I just was reading the docs, I found that there's a class that exists, but the setting that you'd put that class in didn't document it.

It said, here's the list of built-in and listed two of the three.

So it's just adding the third one on that list.

Um and and I think there are like a number of these small things out there you might spot when reading the docs.

And um it's great to just go make that change um so it helps everyone else in the future.

I mean, I know you're a consultant and so you poke around in a lot of code bases, like how do you find so many things?

I mean, because I guess I'm just not using Django to the full board the way you are, but I mean, the the range of PRs you have, you know, SQLite, signals, templates, I mean, does it is it client work?

Is it uh a docs mismatch, or is is there any pattern there?

Because you find so many.

Um I also enjoy reading the code, and uh I'll sometimes take that as first reference, even skipping the docs where I'm like, oh, what's that method on on the storage, for example?

I I have Django open in a sublime text window all the time.

Um yeah, I guess the third thing is I'm I'm always just uh finding it fun and expanding my knowledge and my curiosity by diving into things I'm not really not not really looked at before.

So like SQLite, I was just interested in SQLite a bit more because they came out with that new release that SQLite has strict tables now.

I was like, oh, I wonder how how good Django's SQLite support really is, and could you could you get that strict thing in there?

And then when I was reading the database backend, I came up with that optimization PR.

I was like, yeah, this seems fun.

I could do it.

Like, hey Adam, can you look at this?

You know, obviously you did all the work on Django MySQL and you know, you've been uh contributing there for a long time.

One of the questions we always have from prospective new contributors is how do I get going contributing to the ORM?

So can I put that question to you?

What might you what would be your kind of initial thoughts if somebody's coming, you know, somebody wants to get involved and they want to get involved in the ORM specifically?

I think the problem with the RRM is that like it does like 99.9% of all the queries people need, but most developers probably only know how to write like 99% of those, so there's this kind of ton of extra features that are hard to discover, and some people might be tempted to just try and help, but it's already there in some way.

Um yeah, I I I don't know if there's an easy path into the RM.

I know there's like a James Bennett three-hour talk at a Django Con that is like a deep dive into the ORM.

And um, it's a few years ago, but the fundamentals won't have changed.

If you can sit through that, you're definitely gonna be a good RM contributor.

Um, I still really want to.

It's it's no slag on him, it's just it gets deep fast.

Yeah.

You said uh lot there are lots of features that are kind of hard to discover.

So, like, you know, do you think um do you think we there's a there's a bit of a docs shortage in terms of um learning more advanced SQL features or SQL features and how to use them with the RM, like annotations and aggregations, and then the expressions API and you know all those you know output field types and blah blah blah, all that all that stuff which is it's documented, but there's no real easy learning path for that, I would think.

Yeah, definitely.

I've done a number of posts on that.

If you read the check constraint docs in Django, they are like purely a reference.

They're not gonna explain to you what a check constraint really is and how it's defined on the database.

Um and we don't really have like a topic guide there.

So if someone was looking for a topic guide, that'd be to contribute, that'd be a good place to start.

Um perhaps that's a good first RM contribution.

Um it's like in migrations, we have the reference for all the different operations and um the topic guide of like how to do X in migrations is actually where you can learn how to really use that stuff.

Um yeah, like that that has a few topics now, but could definitely be expanded.

So like how do you add an index concurrently?

Like we have that now for Postgres um in Django Control Postgres, but there's definitely not a topic guide like here's what you should be doing to use it.

Okay.

So there's all these features, but yeah.

I don't want to overrun you.

Do you no no no I wanted to ask so so your new book um was it Boost Your Django Developer Experience?

Is that that's the title?

Yep.

Because you've put out, I mean, we have a long list in the notes, so many projects, posts, um, but as you've been writing the book, so I'm really interested in that and if you could expand on that topic.

So the the book is um just trying to uh describe and like categorize all the tools that aren't Django itself, but like live around it that can really help you like accelerate your development.

Um and I guess book-driven development is where I start writing a section, I'm like, okay, I think I know the best way that I've seen to do this.

I get halfway, I'm like, actually, you know, that I'm just describing some snippets that you could copy paste.

Instead, it would be better if there was a package out there that would would combine all this for you.

Um, so like a good example there is a Django Rich, a package I I just created to integrate Will Magugan's uh rich that does nice terminal output with Django.

Um currently it it only provides like a new management command.

Yes, Carlton.

Because uh a while ago there was a a PR to get colored output into the Django test runner, and it didn't make it because of maintainability concerns, but perhaps a third-party package wrapping rich as a test runner, but that'd be a nice thing to see.

I I did briefly look into it and was like maybe the package could do this, but uh unit test is really the blocker there because you've got to override this class, then that class, and then Django's text test results and its debug results inherit from the text results.

There's like a chain of like four classes that refer to each other as subclass, and it's it would be a pain.

Okay.

If it was there.

The rich package only provides a management command that gives you the way to wrap Django's output with rich and then turn it off if you're piping to a terminal, uh another terminal command.

I mean, that was one that I know came out in I guess asking the two of you, you know, is that a future core Django thing, or what are there in just clear limitations on that?

Because that's pretty helpful.

Um briefly, Django browser reload is a tool that you install and then in development, whenever you hit save on a template or save on a Python file that causes the server to restart, or a static asset changes, then it will hit reload on the browser for you automatically.

Um it does this with a different approach to what I've seen before, in that it sets up a very small like JavaScript-based listener in the browser that's listening for a stream of events from a Django view that captures them, and then that that JavaScript worker it hits reload on on the most recently opened tab that you have for your development server.

So if you've opened five of them, it's only gonna reload one.

I was inspired to do this because I was writing about DX.

I knew that this is a big problem.

Like people have this in JavaScript, they have like even the hot hot replacement where the page doesn't reload, it just swap in one section.

But there's not really been something I've seen at Django that works amazingly.

I came across one package, I think, for fast API, and this was using Selenium to control it.

I was like, that there's gotta be a simpler way.

I don't want to have force everyone to install Selenium and use a Selenium development browser to do this.

There's got to be a way in JavaScript, and yeah, it didn't take long to fumble through the JavaScript APIs and come up with something very basic.

No, that's really interesting.

That that is a common problem, and that's an elegant approach, which I guess time will tell, but I don't know why it wouldn't work.

But I guess when it's a third-party package, you know, people can use it on their own, and if there are issues, they'll come to the fore.

Well, can you put it into a third-party package and can we see how that goes for you know an quite a long time?

I mean, you know, Jenga REST framework is 15 years, 10 years out there, you know, it's it can live in a third-party package forever.

Um, and so what's the case for bringing it in?

You know, that's you know, Jamai T-Bug toolbar, that would be a you know, a candidate, but that's never gonna get merged into core.

Um, you know, these kind of hyper useful pack third-party packages, they will they can just stay as third-party packages.

I don't know.

It's but we can't just go write new feature, bam.

I mean, so you've I think you've put well, I've I've had a chance to look at it, as says Carlton.

Um, but you know, topics covered, because this is your second book.

You also have your speed up your Django tests, and this will come the book will be released a couple days after this comes out.

So we'll have a link.

Everyone should pre-order.

It'll be released at 12 noon GMT.

That's the same as UTC right now, um, on the 10th of January.

We're recording on the fourth, it'll come out in the fifth.

So um we'll have a big prominent link to that.

But it but anyways, yeah, to the book.

So what was you know, because I I find the question is always what not to include, right?

Because you see the first draft is like, let me just dump everything in, and then it's like what to strip out, and then it's like, okay, and is this actually what I want to say as you were, you know, with these areas, you start to give it a beginner's look with some experience, and then sometimes you come up with different approaches.

So it starts off with like looking at docs and then how you make virtual environments and manage dependencies, and it goes on to other things like later, it's um things that are actually in your code, like your settings um and models and migrations.

Um so yeah, it's about everything that's like alongside the actual writing of Django code.

There's not much like here's how to build an X in Django, but it's here's how to make it faster for you to build X in Django.

Um so faster and but more bug-free.

Um you're very right about cutting things.

There's so many things that are actually involved in the process.

One thing that's helped me a lot is I've it added a dropped file in the repo with all the topics that I've dropped in bullet point.

That could have been like a whole chapter on Git, a whole chapter on GitHub, yeah, whole chapter on text editors.

Well, right.

Similarly, like I I I in this book I say, hey, you could go install my pie, and then it's like, I can't cover type pints here.

That's no real way of even starting.

So here's just some links to resources.

I did wonder if that was kind of you know drop um blogging the first draft of there was going to be a typing book come out of that.

I I wrote those um for my own education as well as to like help push typing a bit, because it it should be possible, I think, to like search how do I do type hints for this Python feature and find a reasonable explanation.

So I just try to cover many of these spaces.

And I think a book would just be like a bundle of these posts that you could go read.

Like a cookbook,

I feel like that's maybe an approach I should have done.

Because I I struggle more with the profession my more advanced books than the beginner one.

Because the beginner one, I feel a little more confident saying, I, you know, I don't need to dive into everything, like just sort of trust me on this, or you know, here's how you can go deeper.

But the more advanced ones, it's still kind of the same thing, but it's harder to do.

Um, I feel a little worse about waving my hands, or maybe I'm just not as c clear on it, and I'd have to say, you know, okay, here's three put ways to do it.

I like this one, but I don't feel the same level of confidence on that as I would on the beginner thing, where it's like, yeah, well, we're just gonna backfill a little bit here.

Um sorry, so I'm incredibly impressed that you're I mean, of course, you have the background to write these books, but I feel in some ways they're harder to write, but but taking a more from the side, you know, not just doing like the most badass Django project ever approach, like that's that's a recipe for failure.

That's like building a Django puzzle, which is what I have in my professional's book.

So don't do that.

Um it's very easy to say to someone, hey, go set up pre-commit, and then like the pre-commit docs don't say, here's how to do pre-commit on a Python project with Flake A, ISO, and Black.

Um whereas that that's that's what most people want, that's what Django does.

Um, and yeah, there could be some opinion.

Um I do respect your book swell, and I think the fact that you're a bit uh uncertain about what's going into the professional's book is just an indication of it being senior, right?

At senior, you have to make trade-offs.

That'll be my my fourth book.

Right, Carlton?

Like because you go to the pre-commit docs, you're like, okay, uh, it takes it can take all day to get it set up properly.

Whereas you can you give the recipe, look here, bam, this is how you get pre pre-commit going in your Django problem, your Django project quickly.

So one thing I wanted to ask you about, because you met you seem to manage so many open source projects, and um part of that, surely it must be, tell me, is that you've got this tooling kind of lined up to automate the the repetitive parts of managing a package.

Is that right?

Absolutely.

Um, but it it lets you like run many run commands in multiple directories.

Um it's git aware, so it has like a status command if you need that.

Um so I I kind of treat all my open source packages as one large repo with like some variability in each of them if necessary, but as little as possible.

So if there's a new Python version, what I'll I'll I'll do is I actually write a script that will make the appropriate changes in the right files using SD, that's a Rust-based replacement for sed.

Um and so I just run my repos and run that script and do the ch ch in all those files.

So as long as everything stays relatively uniform, I can do I can upgrade, you know, 33 packages, I think it is now, in the same time that many people could do one or two.

I was wondering the same thing.

I'm I'm trying to like nuke uh open source repositories I have because I just can't deal with maintaining them.

And and so I I love seeing that you're going the other way.

Um I mean your output honestly just makes me tired, but I think I'm just tired of those kids.

But I'm just like, oh my God.

Like every you know, because your weekly posts, which people should sign up for, it's like, well, a couple packages, a couple blog posts, um, a couple core commits, you know, plus everything else I do.

It's awesome.

I it's it's honestly it gives me energy though to see that you can be so prolific

Can you tell us a little bit about that?

Because that's that's

That's very exciting.

Yeah.

Um so I wrote this between August and September.

Um it's a code rewriter that um will upgrade your Django syntax, and it knows about some changes as far back as 110.

Um because they they were there.

Um Bruno Aller wrote a version of this before called Django CodeMod, and this used um Instagram's libcst, which is their concrete syntax tree library.

Um there are a couple problems with that underlying library though.

First, it's like incredibly slow because it's a Python parser written in Python, and it also like turns features on and off depending upon Python version, so it's like got all these branches.

Um and then the second would be that they don't really update it for new Python versions.

Instagram is still running their own fork of 3.8, as I understand.

So there's 3.9 and 3.10 syntax that's just not supported.

Um, so the end result is that Django Code mod is like okay for most people, but it's not something you can add in the way that I like it, which is you know, it up it runs through on your pre-commit hook.

So you never can commit old syntax.

Um, it will just get rewritten when you try.

Um so Django Upgrade is like a rewrite of Django Code mod, but it's using the same approach that a tool called PyUpgrade uses.

Uh, PyUpgrade just uses the built-in Python syntax tree and tokenizer to figure out what changes to make.

Django Upgrade does the same thing.

So if you're running it on Python 310, it automatically supports all the features in 310 because the Python standard library will support that.

It's it's quite sort of in-depth.

It takes it's quite slow going, I would I'd say

Um and and especially the especially the going back from the AST to the tokens, because when you're at the token level, you're basically manipulating like this thing.

All we know about it is it looks like a name.

We don't know what it relates to in this in the actual source tree.

Is it a variable name or a class name and assignment?

So yeah, you're fiddling around with those tokens is quite fun.

Then we don't break stuff unless you know there's a really a good reason to do so.

And I was asking on Twitter today after I was like, you know, because I was releasing 2.2 um as well as 3.2 and 4.0.

And I was like, who's still on 2.2?

Like, what you know, because I don't use the LTS myself.

Um and I tweeted out, and it's it's a you know, agency, a couple of big agencies, so dab apps, um Jamie replied, and then um Tob Tobias from Cactus replied, um, saying, you know, well, we use it because uh our clients, it's very hard to justify the eight-monthly upgrade, whereas a sort of two-yearly upgrade, we can we can do that.

And um other other replies were like, look, it's stability.

We use the LTS for stability.

Um, and one of the big challenges is oh, we want to deprecate this, um, but people won't be able to upgrade.

And you know, we you know, when we took changed, I don't know, to using um you had to import the view rather than just use the string view name.

I think you know, actually we lost Instagram over that change, you know, they weren't prepared to go through and so they fork Django, and you know, they're forever on the fork Django now rather than on mainstream Django, and maybe they'd have gone anyway.

But ah could we avoid that?

And well, one one hope of avoiding that kind of breaking change is automatic script.

So when we do introduce a deprecation, there's also a matching script that will go through your code base and correctly update it.

Uh is that is that is that a pipe dream too far, or is that is that realistic hope that we might have?

Um, it's a it's one thing to like deprecate it and add a warning, and then that warning just shows up inside in your test suite.

It's like, here's a thousand things to go change.

It's like that's no fun.

Um so yeah, there there's definitely a place for this.

Um I'd be excited to see it going forwards.

I couldn't spot anything in 4-0 that can actually be automatically rewritten um without huge amounts of effort.

So definitely a balancing act.

There's perfect changes in 4.0 that we can't automate without it.

Like um we brought in um the uh zone info, time zone changes for Django 4.0, and there's this Django.utils.timezone.utc constant, which is just uh you know, it we should get rid of that ideally.

But that's everywhere, and so is it worth the check?

We could just leave it there forever, but if we are gonna get rid of it, we need to get rid of it for five, you know, before 5.0 because it needs to go at the same time as PyTZ.

So, you know, could we could we could we use PyUpgrade to to make re A deprecate that remove it, but B make removing it, you know, just run a script.

That would be a cool thing.

And I guess we we just replace it with the the one in Python's date time module, right?

It's it's just replace it with datetime, datetime.utc.

But like it would be it would be nice because my only reticence about removing it is that it's everywhere, you know, everywhere in the ecosystem people have been using that.

So if there was a script that came along with with Django 5.0 that said, hey, you've got to remove this, but just run this and it's done, that would be an amazing um you know proof of concept of of that this is the future of upgrading Django, because stability, right?

That's that's the USP.

And and there's there's a good argument to remove it, right?

Because it looks like it's a different thing.

And once it's just an alias for Python's one, all you've got is like the potential for confusion.

We're keeping it to avoid the churn, but ah

Oh, I should be using Django's one.

Oh no, it's exactly the same, it's just an alias.

Okay.

Um and they weren't ever documented, but when they were removed, I found the old commits, and people were debating, should we really be doing this?

Yeah, okay.

Yeah.

So we should push that for 5.0 then.

Yeah.

At the same time, there's there's like a oh thank God, like something I can worry about something else.

So long term, what else?

Do you do you know what you know to work on projects, books like pie in the sky, um, things you have planned around Django?

So I'll be getting back into that and no doubt diving through people's code bases.

I'll spot some new things that I I'm interested in in changing.

Yeah.

Um for the book.

Uh my next writing project will be to go update the speed up your Django test one for 4.0.

You got me on 3.2, but I got you in 4.0.

For one of my books.

I'm still uh still waiting to get when I push that out, it's like I Well I double checked, I was like, I beat Adam in something.

Congrats.

It's not a big change.

It's a pretty small change.

Or at least I've I think we I think for yours as well, right?

It's not I don't think trying to remember.

Are you did you pin your repositories?

Like, I you know, I am 99.9% sure you're making a mistake.

But I just like sometimes I have to just like wait a couple days and be like, You sure you want to check, you know.

Um so it's good and bad.

It's like I'm pretty sure I'm right on this um because I've put it in this little box.

Um, but anyways, yeah.

Um so when I go through my updates, yeah, yeah.

Like I literally I've thought of running a script to like check everything, but I also for now manually go through everything.

Um as much for like the the text part, because I you know I used kind of whinge when I read old things I've written, and then like, oh, I could rephrase that or improve it.

So it's like a forced update mechanism.

I I don't know if that's what you do if you go through and read the text or you have you probably have a test suite you can run and just see what's broken.

I I definitely go through and read things um in in speed up your Django tests, at least because some features were still under Durham.

I signposted, hey, in Django 4.0, you'll be able to do this.

So it would just be a matter of updating it and checking that what I wrote there is still applicable.

Yeah.

Yeah, I try to say, I think this is coming.

Um but I mean also I think you know it's good to have some distance from it, right?

I mean, eight months is hard, which this is the first last year's the first time I missed an update cycle, but it's almost too close because it you sort of need some distance and yeah, and things change to like revisit the same stuff.

Anyways, yeah, that makes sense.

I I wish you well on the update challenge.

It's like a band playing the greatest hits, is how I look at it.

You know, it's like some for me, like new stuff, and then some for everyone else, which is updating things I've already written.

So anyways, not to make this about writing books, Carlton.

I know you love those discussions.

Okay, let's

Adam, thank you so much for coming on.

Um good job with the book launch.

Everybody, remember you can pre-order until the 10th.

Um boost your Django DX.

Um, thanks for coming on and thanks for everything you do to the Django for the Django community and well done again on the Mount Contrendelic ward.

Couldn't think of someone who deserves it more.

Bye bye.

Bye bye.