← Back to Show Notes

Transcript: Django the Good Parts - James Bennett

Carlton Gibson

Hi, welcome to another episode of Django Chat, a fortnightly podcast on the Django Web Framework.

I'm Carlton Gibson.

Joined us everybody by Will Vincent.

Hello, Will.

Will Vincent

Hi, Carlton.

Carlton Gibson

Hello, Will.

And this week we've got with us James Bennett, who's a longtime Django contributor, been on the Django board, and well, you know, all-round hero of mine.

James, thank you for coming on.

It's really

James Bennett

thanks for having me on.

Carlton Gibson

Massive honour to have you.

Um we always talk about um to start off with, like people's backstory and how they how they got into programming, how they got into Django and all the rest.

But am I right in thinking that you did philosophy?

Because I did philosophy, I was yes.

Will Vincent

But you bailed, you didn't get a PhD like Carlton.

You quit early.

James Bennett

I I was trying to.

Um, I was a terrible student.

Uh so I did not get into a PhD program.

Will Vincent

Oh, I didn't know that.

James Bennett

So I was growing up, my family didn't have a lot.

I didn't actually have a computer at all.

Um, like I had a little bit of experience with an Apple II at at school.

Okay.

James Bennett

But it wasn't until I went off to college that I had reliable access to a computer and to the internet.

I didn't have my own computer until my second year.

And I was taking courses, it was a small college, and so it was a department not just of philosophy, but a combined department of religion, philosophy, theology.

Okay.

James Bennett

I was taking a course on ancient Hellenistic religions.

Carlton Gibson

Nice.

James Bennett

And at the end of the course gave us an option.

You could either write a 15-page research paper on an assigned topic or produce a website with five pages.

That's easy.

And I was a lazy student.

Yeah, good programmer.

James Bennett

So I learned a little bit of HTML, made a website, and kind of was interested, decided to start learning a little more, learned some HTML, some CSS, some JavaScript.

Uh I was working as sort of a department assistant.

I was the guy who would go, like when they needed something from the library, I would go pick it up for them, or they needed copies made of things.

And my advisor uh found out that I had learned how to do this and was like, hey, our department's website could use a little refresh.

Would you like to do that?

And we'll pay you for it.

And so I did.

James Bennett

And then I did his personal faculty page for him.

Um and that was kind of the start.

I spent a summer when I was supposed to be doing much more research than I actually did, uh, learning PhP and a couple other things.

Uh, build my own like personal blog engine.

And then I graduated, applied to some PhD programs, didn't get into any, got kind of a boring job at an insurance company, and saved up a little money, and then went into business with a friend of mine who was a designer, because you could do that.

This was like 2004, um, 2005.

You know, you could do that back then.

You could just say, I'm going to go into web stuff, and it would work, and started doing contract work, uh, mostly PHP in Perl.

And I was learning Ruby and Rails shortly after that came out.

I'd bought the Rails book.

Um, I had learned Python from reading Dive into Python uh by Mark Pilgrim.

But at the time, Python web development was basically Zope.

Right.

James Bennett

And where I was living, I was living in Virginia, there were people around me, like there were user groups that I could go to for tech stuff that uh some of them did Python, but they all did government contracting, so like US federal government websites with ZOA and like hugely complex requirements and everything.

And then one day this thing called Django came out.

Um, and I read about it.

I decided to play with it, and I really liked it.

And I remember I wrote something on my blog, not the one that I have now, but an older one that thankfully is is gone, um, about how I really like really like this Django thing, and I really wish one of my clients would let me use it.

And I kind of joked about maybe I should go on strike until somebody says I could use Django.

And a little while after that, I got a message from Jacob Kaplan Moss saying, Hey, the journal world is hiring.

Would you like to apply?

Carlton Gibson

Fantastic.

James Bennett

And so I did, flew out to Kansas, uh, interviewed with them, they decided to hire me, and so I moved out there and that started my like proper Django.

Carlton Gibson

So that's right in the crucible.

That's like, you know, you're there at the Yeah.

James Bennett

So one fun thing, that that little website that I did for that course, um, thankfully that also is not online anymore, but for a long time it was, and because it was on a.edu domain, I kept getting email from people assuming I was an academic expert on the topic and being like, yo, I've been reading this.

Can you offer any you know follow-ups, any anything else that I should study?

Because you were like the webmaster address at the bottom.

James Bennett

Yeah, I I was I was not a professor when I did that.

I never am going to be.

I was a student who was lazy and trying to avoid writing a 15-page paper.

Will Vincent

So is it fair to say that programming seemed to f fit your mind though?

I mean, it I don't hear quite the same struggle that mul other people seem to have like learning multiple languages.

It seems like it, you know, you obviously had to do the work, but it made a sense to your mind how programming works.

James Bennett

Python did.

Python has always fit my brain.

Um PHP and Perl, I had to put in some work, JavaScript.

I had to put in some work, HTML kind of clicked easily, but that was the age of view source learning.

Carlton Gibson

I don't know how to do that anymore with it.

James Bennett

Um I mean, it's also that I just had, thankfully, because I was doing it over a summer when I had not many other responsibilities, I had the time to just kind of like bang my head against it.

Like my first little PHP blogging application was not good, but I was very proud of it.

Will Vincent

Yeah, well, that's always how it is, right?

I mean, any even I was telling someone the other day, like you look at code from a year ago that you know someone else wrote, let alone you wrote, and generally you're like, ugh.

Yeah.

Will Vincent

I mean, it's just how it goes.

Yeah.

James Bennett

And and yeah, like I said, Python kind of fit my brain.

Like PHP, I give a lot of credit to PHP for making web programming so accessible to so many people, like at that point in history, because up to then it was basically you had to learn CGI, which is kind of a complex and tricky programming model, and probably you had to do Perl, which has its own quirks and is kind of complex to learn at first.

And PHP made that so much easier.

Because it was just write this, and you know, you can you can intersperse a little logic with some HTML and just upload it and it works.

Yeah.

Carlton Gibson

Yeah.

I mean that there's that quote um about make solve the web problem quickly.

That was its kind of goal.

Yeah.

Um I st I still hold that in my mind, you know, as you know, whenever I'm thinking about Django and what's good about Django and the strong points, it can try and hold that, you know.

We t Python still hasn't got the deployment story of just stick it up there and it'll work.

But yeah, that solved the web problem quickly.

I you know, I I kind of think that that's something that Django has going forward as well.

James Bennett

We we never had the kind of like mod PHP of just upload some files in a directory and your web server will run them for you.

Carlton Gibson

No, we didn't.

But the that general goal, I'm still fought, I still hold it.

And yeah, you know, when you say you give credit to PHP, it's exactly right.

It's like, you know, how did I get in?

I picked up the PHP and MySQL programming book.

James Bennett

Yeah, and and if PHP had not made it that easy, I don't know if I'd be here.

Carlton Gibson

Yeah, thanks.

Will Vincent

One of the things I've always been impressed by, James, is that you've stayed so involved in Django itself, both on the technical side and also the community side.

I mean, this is true.

I mean, when I joined the software the Django Software Foundation, you were on there.

You were also actively involved with the technical things, and you had you actually drove the development, I believe, of the the technical board, in part because you know you were, I think, the person you were involved, you had your hands in everything.

James Bennett

So the the technical board was uh uh with was Jacob and Adrian, and I I want to pause and give them a ton of credit because they set a really great example and they did some great things for Django.

I mean, from the very beginning with opening up to other people being given commit access to then stepping back, like one of the biggest milestones and potential problems for a project or a community like Django is whether it can survive the loss or the moving on of its founders.

And they set it up to succeed at that, and then they stepped back and they said, We don't need to run this for the rest of our lives, and then we got the technical board.

You know, we still had at that point the technical board and the committers, the core team, as the decision makers, but they deserve a lot of credit for that because they set Django up to succeed and be hopefully sustainable in the long term.

Because there are other projects that never really get over like the the their founder or their kind of like old guard team and just kind of cling to that for as long as they can.

Carlton Gibson

But part of that is being, I guess, being open to the yes, they I mean I remember when they stepped back and it was you know BDFL, but BDFL until we step down and then ah, what's gonna happen now?

But they Django's was had quite a big contributor base and is was historically open to new contributors.

And do you is that cultural or do you is that

James Bennett

I think they I think they did a lot to encourage that.

Um they they were very open to feedback.

Uh I mean, even before Django was publicly released, they were kind of showing it around to people and asking opinions.

Um, you can hear stories of like Ian Bicking being asked about the thing that became Django and offering his opinions on it, and kind of you know the the ORM used to be a code generator, and there was feedback to yeah, maybe maybe don't do that.

Like yeah, I think they really set the tone for that by by being open to feedback to other people jumping in and helping out and being willing to not be gatekeepers of Django forever, but saying, oh well, this person's doing good work.

Let's give them a

commit bit.

Carlton Gibson

Yeah.

And one one thing I've always liked is the um the approach for consensus, looking for consensus, and that you know, it's like, well, you know, actually there's there's not a consensus for agreement, so we'll just hold off here or we'll we'll carry on the discussion to until we see if we can find that.

I think that you know it's a real strength.

Um it's not just one person going, no, I think that's

James Bennett

I know people get frustrated because it seems like it can take forever to get something done in Django, but I do feel like Django's kind of slow and steady approach to we want that, but we'd rather have the right version of it than have a version of it right now.

Carlton Gibson

Yeah, no, I mean it's uh the other day I was um commenting on the channels repo about you know the development's slow and it's been slow the last couple of years, but it's inexorable, you know.

Year after year it slowly moves forward, and yeah, you know, one way to speed it up is you know, I'd say one way to speed that up is to join in, but you know, the the reality is that Django's a slow-moving beast, you know?

Yeah.

James Bennett

And I think it's also been a benefit in a way because Django has maintained a lot of integrity of what Django feels like, despite all the changes that have happened.

Um when I when I did my ORM tutorials at Django Con, I I mentioned this.

I would take like the uh the actual blog model that's used on the Django project weblog, and go through different stages in its history and be like, doesn't this look awfully familiar every single time?

Yeah, okay, nice.

James Bennett

And I think that's contributed a lot.

The the the kind of pace and the kind of well, let's do it right, even if it takes a little bit longer, has helped Django maintain that integrity over time of this feels like Django, even as new features get added, even as things get rewritten.

Will Vincent

Well, that's even I was thinking about that today because um black, the Python formatter, just got out of beta and step eight has been around and approved so that you know once that milestone is reached.

So you could say, well, another framework could have thrown black in earlier, but you know, there was a process, there were all these extra steps, people committed to that process, and it will be in there.

And when it's done, it's done the right way.

People can see the discussion, you know.

So I think, as you said, taking a little bit longer, having a process wins out in the end, even and people can always just use it themselves, too.

I mean, that's the thing with the ecosystem and third-party packages channels.

Yeah, there's there's like an off-ramp for I really want to do this.

Um frustration has an avenue.

James Bennett

Yeah, and Django is always kind of encouraged doing things third-party to begin with, and building consensus in the community around, yeah, this is a good implementation, this is a good way to do it.

And then bring it into Django.

Yeah, so that happened with migrations, that happened with channels, you know.

Yeah, SC stuff.

Yeah.

Yeah.

Will Vincent

I mean, and even the sorry, go ahead, go on, we will.

You will.

Will Vincent

Well, I I want to hear you two talk about the technical part.

I was gonna say, even just the the community part, the the DSF, you know, that we had Anna on recently, last week, um, who's the current president, and she's been on the board for five years, and I've been on now three years, and there's a natural cycling of people on the board, but there is the sort of passion and knowledge hopefully gets passed along.

I mean, you were secretary, James.

Um, it's you can't assume that someone is going to volunteer for 10 years on the board or as a core committer, though you've you've managed to do it a long time, James.

So there's a healthiness in terms of having that model and you know, do what you can.

When you have to step away, you can, and that keeps the goodwill in the community.

Um, right.

I mean, I know for you, right?

Like you had a whole spike spike of work stuff on top of all the Django things you were doing a couple years back, which made you need to step away, but you know, you're still here you are, right?

You haven't rage quit, hopefully.

So well, I've

James Bennett

reduced my involvement a lot.

Um yeah, I used to be on the DSF board, I served some terms on the technical board, I used to be a lot more active on the developers list.

Um these days, the only thing I really do in the Django project is I'm still on the security team.

And not super active there, but every once in a while I'll jump in

Carlton Gibson

timely and do it a

James Bennett

special review something.

Carlton Gibson

Timely import call it, you know, thing.

I hope so.

James Bennett

So yeah, I I think that again, that's kind of the project and the community are healthy enough at the moment.

And I I think again that goes back to like Jacob and Adrian setting things up that way, but they're healthy enough at the moment that people can step back, and other people are stepping up to do things.

Carlton Gibson

I mean, but you gotta hope that you can step back because you know, I mean, open source and burnout go hand to hand, like you know.

Yeah, coffee.

James Bennett

I wasn't quite intending to have like the the mic drop of the new governance and then alright, I'm out, but

Carlton Gibson

so you mentioned the governance, so that that was the um the abolition of the old um Django core and then um moving the the man the sort of uh the governance of the framework to the DSF membership um and yeah but the the the the um the rebranding of roles so there's mergers and committers, uh mergers and releases and

roles like this, and then the technical board taking that role.

That was depth depth, right?

Yeah.

One one thing that I was worried about, and that I've you know I've got you here, was that the the the kind of knowledge base, the sort of the old knowledge, the old, you know, the the established knowledge to sort of just disappearing as you know you know I looked at I looked through the old list of the core committers and they're all like all the people that built all the features, and if they're not there anymore, I was just worried that we'd lose that that kind of tribal knowledge slightly.

I wonder what you think now.

James Bennett

I feel like I feel like people are still willing to to like answer questions.

Um I I've joked with uh Russell Keith McGee a couple times that that my role in Django now is to be a grumpy old man who remembers what happened.

Yeah, yeah.

James Bennett

Um and yeah, every once in a while just jump in.

But at the same time, there are plenty of other people filling that role.

Like I saw um you know Tim just recently uh in

Will Vincent

Tim Graham, yeah.

James Bennett

Yeah, in thread about email configuration, jumped in with, well, here's this old ticket, and it never really got anywhere, and here's here's some thoughts from when that happened.

So, yeah, there are other people stepping up, and and like Carlton, you know, you you've been around long enough that you are you have a lot of that context now.

You've seen

Carlton Gibson

the the last few years doing the fellow role, like so basically to live in track.

That's that's where I've you know m deepened my knowledge massively.

So okay, I was working on REST Framework, and then, but to come in and and have to go through track, and when an issue report comes in and have to find the history and exactly why it was, and yeah, uh you know, you do that for a while, and it's like okay, yeah, I can I can my knowledge Tim's knowledge of the issues is amazing.

James Bennett

It's just impossible now.

But I used to like 10 10 years ago or more, I used to every once in a while just go through every open ticket in in Django's track and read all of them, see if any of them could be closed, see if any of them needed to be bumped.

Like, is there anything like any easy like documentation tickets or things that can be promoted as let's get this one done for the next release?

Yeah.

James Bennett

We can't really do that anymore, but you you still yeah, you you you develop that feel for what's out there, what's open?

It's like, oh yeah, I remember there was a ticket about that.

Carlton Gibson

Yeah, and you could try and find it, and that that works, and then you you know, and but uh just the other day, Claude um there was a discussion on the uh about um what was it?

I can't remember the exact details, but Claude was like, Are you talking about this ticket?

And then you know, a

10-year-old ticket.

I'm like, Yes, Claude, thank you.

Carlton Gibson

That was the exact ticket I was I I need to find.

So yeah, I mean, so I guess the history is still there.

And what's in you know, what's interesting, you talk about um Jacob and Adrian, you see a 10-year-old discussion or a 12-year-old discussion or 15-year-old discussion where they're you know handling it in a very gracious and open way, and it's exactly right.

Super.

Will Vincent

James, another thing I want to ask you about is you have given a lot of conference talks over the years that I mean that's how I first knew about you is seeing these talks.

And um I wonder if I could ask you about so this was from PyCon 2014, Django the Good Parts.

Okay.

I don't know if you recall your thoughts there, but I thought that was a really I probably still have interesting one.

Yeah, you but that that idea, I mean, there was, you know, Java, I guess that was when what's his name?

There was JavaScript the good parts, right?

Yeah, that that was the thing.

And but it is what one of my favorite talks is Jacob's from I think PyCon 2017, which is a three-hour build your own web framework where he basically goes through these are the decisions you make.

And you know, do you write your own templating language?

Do you use something else?

And what do you use for the web server?

And it sort of distills uh a top architect level view of of these things.

And anyways, if you recall that, you know, your the good parts talk.

I I watched that a couple years ago, and it might be interesting to see if it still holds up uh eight years later.

So let's see.

Will Vincent

I have we'll put the slides in the notes if they're I'm sure they're up somewhere.

James Bennett

Yeah, they're somewhere.

Um I'm sure they're online.

So let's see.

What did I suggest?

Um HTTP abstractions, the request response cycle, the slow and steady approach to features, and apps as the good parts of Django.

I'm sure there are more good parts.

Those are just kind of the ones that I that I focused on.

Um I think that still holds up.

Like I think Django's you know, request response cycle and HTTP abstractions are still quite good, especially and this was in comparison to doing like your own raw WISGI type programming.

And for people who've never done that, like WISGI is kind of CGI adapted to be a Python thing.

Like it is, it is literally the CGI programming model, just kind of lightly adapted for Python.

So there's all this, like the the way for people who've never done CGI, the way that it worked was you would have your script and the server would put information about the incoming request into environment variables and the body of the request into the standard input stream, because it was a Unix programming model, and then start your script.

And then you would do your processing and send your response, you know, headers and response body out on the standard output stream.

And so you had to do all of that processing in the script of knowing which magic environment variables have which things and how am I supposed to parse them and how am I supposed to handle them.

And WISGI is similar in that it passes you a dictionary of the environment, and the keys, in fact, in that dictionary match up to the traditional CGI variable names.

Um, and then you know you return.

And WISGI is a little more composable, but every single step, like if you have you know one WISGI app wrapping another, both of them have to parse that environment dictionary.

Um like the the response API is kind of complicated because you have like this thing that you call to start the response and then you return the response.

And so it's I really do think that Django's abstractions are are pretty good.

Carlton Gibson

I think there's evidence for that with the move to ASGI as well, because it was Andrew did an amazing job, and it like he was just able to not just able, I don't know just about it, he was able to um bring in an ASGI request that maps kind of parallel to the Whiskey request, and inside of that beyond that request object, kind of the insides of Django were like, yeah, I I don't even notice.

It's you know so we've been able to put in a totally different um request handling sort of application layer.

And Django's untouched virtually.

Virtually.

Will Vincent

Well, we had uh Carl Mayer on from Instagram on the podcast, and I think he said pretty much the only part that hasn't been rewritten by Meta, Facebook, is is that very part of Django, and he thought they are they thought they would never probably swap that out because they didn't need to, whereas they'd change the ORM and you know everything else.

But that bit was pretty rock solid.

James Bennett

I mean, there definitely are other like libraries and frameworks that provide the request response abstraction as objects in a in a similar way.

I just think that Django that's one of the things that Django got right.

And I'm sure there are other good implementations, but I think Django's is really good too.

Um we talked about like the the slow approach to features, so I think that still holds up.

Um apps, absolutely.

Um I I've been I've been a fan of Django apps.

Like I gave a talk at the very first Django Con, uh, which is 2008, and hosted at Google about like designing what I thought were good Django apps, and I've talked about that a lot before and since.

That the app abstraction of you can build sort of a self-contained unit of functionality and then compose and reuse uh so things like you can build a contact form, you can build a blog, you can build like all of these different types of features and functionality and compose them together to build a Django site.

And I think that's really, really crucial because just providing that abstraction at the level of not like a database model or a view, but at the level of a whole interconnected unit of functionality for a website has been really, really crucial to Django's success because it gives you that level to think about and it gives you the ability to package those things up

and distribute them, or even if they're internal, you know, reuse them across different projects and saves you a lot of time, makes it a lot easier to reason about these things.

And so I've always felt like apps were kind of like the secret weapon that made Django so good.

Will Vincent

I should mention you have a Django contact form app among several others that folks.

James Bennett

Yeah, I need to I need to do the Django 4.0 updates, and and I got a, I think it was Marius, uh pinged me to point out that um the new form rendering is is actually gonna require me to do some work with the contact form because the contact form already had a method that uses the same name as what was added for the new form rendering.

Will Vincent

Carlton, I know you had a question, but I just wanted to go on, Kevin.

Well, I just wanted to say something quick, which is that apps, uh when I people who are new to Django, so e especially people coming from another framework, apps is actually something that trips them up or they they're a little not angry about, but they kind of like, well, this is stupid.

Why do why does Django have apps?

I want it to be a certain way.

And I see this various discussions, it's like, well, you can the whole point is you can do it however you want.

You can have one app for your entire site, or you can rely on abstraction, and then you can also rely on these third parties.

But that concept, maybe it's coming from different frameworks, but that's something that someone coming from another web framework, I find, trips them up initially.

Yeah.

Because there's confusion over what it means.

James Bennett

Yeah, it is a thing that I've had to like I've had to teach and I've had to explain like at a few different companies with people coming from other stacks, and and their assumption is, oh, the app is the thing you deploy.

And it's

Will Vincent

projects versus apps, yeah.

James Bennett

Like the the project is the minimum deployable unit of Django.

But you know, the app is kind of the atomic unit of functionality.

Will Vincent

The question is, well, where do you draw the line?

It's like, well, there's guidelines, but that's so I think that's you know, that's the piece that is unsettling to to to those folks.

It's like, okay, but but how?

Yes.

James Bennett

And and it is it it can be tricky, and like I'm not gonna say that I always do it right.

It it's not something that there is a single rule you can follow that always gives you the right answer.

Will Vincent

Right.

Well, that's kind of what I tell these people is say, well, yeah, maybe it's an extra step or thing to think about, but Django on that balance of customizable, Django is actually quite customizable considering it has guardrails.

And that comes at the cost of the apps and some other configs where it could be rails, it could just go boom one way, but it makes it easier to be malleable within a construct.

James Bennett

It lets you it lets you turn things on and off.

It lets you have you know different configurations of features that you can kind of toggle.

Um I think it also helps that Django it that Django itself has that approach.

So like the apps in Django contrib, uh things like the wall admin and sessions, that you can see that you are composing these things together to get it.

Just one more, right?

Yeah.

Will Vincent

Carlton, what were you gonna say?

Carlton Gibson

I'll follow on from that just one more comment.

It's that people get kind of addicted.

And what I've seen as a working Django programmer for many years is you could go to a company and they're stuck on the the old LTS and it's just about to go end of life, and they they're struggling to update because they've got you know 30 third-party apps, and 29 of them aren't really um aren't really maintained properly, and uh then they're in a kind of bind.

And so over the years I kind of developed a kind of um very conservative approach to taking on a new app, and you've got to be you know, you must be cautious here.

You mustn't you must check it's properly maintained.

You must that's the counter, that's the counterpoint, I guess, is that you can

Will Vincent

third party app, you mean Carlton?

Carlton Gibson

Yeah, third but yeah, the third party app.

If you're gonna take on a dependency, like the and then you you find yourself kind of stuck.

So I I I tend not to use the LTS myself, but I asked on Twitter recently, why you know, if you use the LTS, why do why do you use it?

And the one of the biggest responses was, well, you know, by the time all our third-party apps are are updated to be compatible, you know, we go LTS to LTS, and that can serve us better than you know, trying to I think

James Bennett

one thing like the the the API compatibility promises that we have now make it a little more make it a little nicer for a lot of places to go LTS to LTS because you know Django itself kind of makes the prom

ise that if you're running on this LTS and you have no deprecation warnings, you can safely upgrade to the next LTS.

Yeah, yeah.

James Bennett

Uh without having to change your code.

Now, you know, you you would want to test any third-party stuff you're using.

But that gives a very nice upgrade path.

And and I've dealt with setting up dependency management workflows at at several companies and and how to how to keep things up to date, and that is a perennial issue.

Carlton Gibson

Yeah, and I recently updated a project on 111, and it was remarkably smooth to get it to 2.2 and then 3.2.

And it was just like, you know, and I didn't bother with the intermediate ones because I, you know, Ramad warnings clear those up, and it was it was that was lovely, to be honest.

Much yeah, better than the old days.

James Bennett

But it it does, it does get complex when you've got, I mean, even if you're not using that many like Django apps, you're probably using some number of third-party Python libraries and things off you know, PyPI, and uh you need to keep those up to date even if you're not upgrading Django.

Yeah.

Yeah, so that's that's a fun thing to do.

Will Vincent

Well, uh, so I wanted to ask you, you you wrote a post on how I'm testing now from 2020, which is related to this that I I thought was pretty interesting.

And testing is something we love to ask practitioners about since there's quite a variety of ways it's done in the wild.

Um so we will link to that article, but I wonder just again, you've you've worked at a number of companies using Django.

With how do you think about, you know, practically speaking, proper testing?

You know, what are the tools you like to use in terms of the hierarchy of priorities that you like to see on in a code base?

James Bennett

Um I like to see tests.

Like any tests are better than no tests.

I'm not particularly like a test-driven development person.

I'm kind of I'm skeptical on most like capital letter methodologies for building and managing software projects.

Will Vincent

Um like full stack developer.

James Bennett

So I I may write something about this in the future.

Like you know, I I I understand that a lot of these methodologies, there's some essential insight that somebody caught once and then tried to propagate that, but it's the everything else that grows up around it and kind of all the rituals and practices that because because so often like the the things that make a project successful or not are not really you can't replicate them to the

next team, the next project, the next thing.

Like it's it's nobody's figured out how to do that yet in software.

So um, but yeah, I do like tests.

Um I for my own personal stuff, I write tests.

I use Tox as my test runner because typically I'm um like I'm testing against a bunch of versions of Python and Django for like Django apps.

Sounds like Carlton.

James Bennett

Yeah, Tox makes it very easy to manage that matrix of things and keep them running.

Um I use I prefer the Python standard library's unit test module over PyTest.

Um left two places I think.

That's interesting have been PyTest shops, but I I prefer the unit test module.

Um can I tell what that is?

Carlton Gibson

Like what is because PyTest is very popular now, and you know, we still use unit test for Django's own test suite, but w what are your own reasons for favoring that?

James Bennett

Um so I mentioned this a little bit in the blog post uh that that I understand, or at least I think I understand what PyTest is trying to do with separation of concerns in testing, um, and how it wants that.

I'm not the biggest fan of dependency injection in Python.

Um and the particular way PyTest does it, I just feel like I at least I kind of struggle with it because I feel like fixtures can be getting injected and and found and loaded from so many different places that it's hard for me to know what's available.

Wait, where where did that one come from?

What does that one do?

It it just I have a harder time reasoning about everything that's going on versus like a unit test style uh where everything is right there on that class and I can see all the things it's doing, or if it's from a parent class, I can see what it inherited from and I can follow the import chain.

Will Vincent

It's a bit like function-based views versus class-based views.

Like Carlton, I saw there's you know, you were waiting to Twitter.

Someone had

oh, I foolishly got on that, and I foolishly got so

Will Vincent

I I saw I glanced from a distance and said, braver than I am.

It's just I like

James Bennett

class-based views.

Um I I think I think Django's built-in ones could have been done differently, but I understand why they were done the way they were.

But in general, I'm a fan.

Carlton Gibson

Yeah, I mean, my the the the claim I made was that they give you um uh a namespace to break your your logic up in.

And I think the issue is that we don't teach, we teach here's one line and you've got a list view, and that's kind of deeply mysterious.

Whereas if we taught, you know, a class with a get handler and then you declare your view, then it's just it's exactly the same as a function-based view at that point.

Um, but it's just got a bit more structure if you need to break your your you know, if you need to break it up.

And I think, well, yeah, that was my point.

Is that hang on, what are we what are we arguing about here?

Are we comparing function-based views to Django's generic class-based views?

Or are we comparing it?

James Bennett

And I also, I mean, I feel like I lived through the entire era of generic function-based views and of that being function-based views being the way you did things in Django.

And I remember how much more difficult it was to like add functionality.

It's like, okay, now I need another keyword argument, and I need another, you know, conditional in the body of the function to handle that keyword argument, and I need to account for it everywhere.

And if you if you ever go back and look, like maybe you know, link one of the old Django documentation uh versions with like the argument signatures of the old generic views back when they were functions.

Like it's it's a gigantic list of stuff.

Will Vincent

See, this is that grumpy old man wisdom that we need on the young folks, Carlton, right?

James Bennett

So like and yeah, like the class-based the like the generic views in Django right now that are class-based are the way they are because they tried to replicate exactly what the old function-based views could do.

Like we we wanted the functionality before and after to be the same, and so that's why they were written the way they were.

That's why, you know, if you wonder about like choices that seem weird in terms of what's broken out into a mix-in and what's inherited into the it's because there was a deliberate attempt to preserve exactly the set of options you had with the old ones, so that you could just go straight from one to the other.

Will Vincent

Well, Carlton, you've mentioned many times, I mean, Tom Christie has Django vanilla views, which is perhaps a clean approach at doing it without trying to mimic the old function function based.

Carlton Gibson

Yeah, I mean the vanilla views are they're sort of they don't handle all the the they do the the basic create views and um you know the uh the the form handling views and then the list and detail.

And for those, they they're kind of you know at 1.8 time they were sort of feature equivalent, but they're just a more direct style without the mix-ins.

And so I always recommend people because there's only like four files in that package, I always recommend people to go and read the source code of Django Vanilla views because it kind of it it's really eye-opening.

It's like, oh yeah, actually, what's going on here is just turning requests into responses, and you know, there's not too much to it, and then yeah, you go back to classy class-based views and the Django ones, you know, ah, the same thing's happening, and it's that's a nice learning.

James Bennett

At any point, could Django simplify the ones we have, or I mean it feels like there's too much of a compatibility issue.

Carlton Gibson

Yeah, no, not now.

No.

I mean our our our USP is we don't break stuff, right?

Yeah like and we can't we can't change the generic class-based views that we ship.

Uh I you know, maybe, but no.

If you ask you,

James Bennett

I mean uh I remember w when when the old form system or the the manipulator system as it used to be called, um, and and then there was Django.new forms, which became Django.forms years later.

So, you know, Django.new generic.

Yeah.

Oh Carlton.

Carlton Gibson

Okay, if you write the dep, I'll I'll plus one it.

How's that?

James Bennett

Okay, I think I think you're right, there's just too much backwards compatibility and it it's too important a feature to to rip out and replace like that.

Carlton Gibson

It's not it's not that it's bad code, it's great code.

It's great code, it's well documented, it's well tested, it's got everything going about it, it's just there's a learning curve, and it's not perhaps as clear as we'd lay out if we started fresh.

James Bennett

But and at this point it's not clear to people why that learning curve exists, because that complexity is entirely, as you mentioned, due to the desire to maintain exact compatibility with what the old function-based views did.

Yeah, yeah.

Carlton Gibson

Yeah, I mean, I think you mentioned that really, if we're honest.

But that's okay.

Well, you meant

James Bennett

you mentioned, you know, we we don't we don't break compatibility as a policy.

That you know, when the class-based generic views were introduced, they had to be uh exactly equivalent feature set.

Yeah.

Carlton Gibson

Okay, interesting.

So Gornwell, did you want to

Will Vincent

Yeah, I was just gonna ask about service layers.

Carlton Gibson

Well, okay, because there's a big elephant in the room, which is the ORM, right?

So Yeah,

James Bennett

and just to wrap up like the original question about testing, like it's not that it's not that I think PyTest is bad.

Like it it's it's really good, it's really well

written, and I do

James Bennett

appreciate what it tries to do with separation of concerns.

It's just for for me personally, it doesn't quite fit my brain, and I find it a little harder to reason about.

Like that's why I prefer the unit test style.

Okay.

Will Vincent

Service layers ORM, pick one.

Carlton Gibson

Well, let's talk about your you've taught the ORM, you've um you know contributed to the ORM.

How did here's a question I'd like to ask ORM contributors.

How do we make how if you're if you've got a young contributor who wants to come along and they want to get into contributing to the ORM, how do they approach that?

How what's the the route in to grokking the ORM?

Because it's it's big and scary and multifaceted.

Will Vincent

Um even Tim said so.

Yeah.

Tim Craig.

Yeah.

Will Vincent

He said that was the one area.

I was like, where are you where are you not rock solid?

And he's like, well, the

James Bennett

mostly we've mostly relied on like every once in a while, so like I'm sure you've heard Russell tell the story of um Olys and Detka when she first started contributing and picked this really deep, like gnarly ORM ticket to do that, like when she submitted the patch, it took Russell and and someone else like a couple hours with a whiteboard figur

ing out what was going on and that yeah, this is the right solution.

And she just kind of like tossed that off in the course of a sprint.

And we we we've always kind of relied on the the kind of genius level.

I don't I don't know how she understands the ORM that well.

Will Vincent

Um it's like walking in late to class and you know solving the proof that's never been solved, right?

The MIT.

James Bennett

Yeah, so so Django has been very lucky to have people like Ola who just show up every once in a while and can do that.

Um we we've we we need to do better with that.

Like the ORM, I would say first try to understand the different abstractions, like start with you know, models and managers and query sets, and then you know, maybe look a little bit at query, look at database backends, look at the like the query object, um, look at the SQL compiler, like learn to think of you know, query.

I actually think a good analogy is to learn how the template system works, and learn how the template system like parses into a tree of nodes and then walks through rendering all of them because what the ORM does effectively is build a tree of expressions and walk through with a SQL compiler, turning them into SQL, uh, and kind of understanding that pattern and what the major abstractions are, and then you can start reading like some of the scary source

code in you know query logic and I haven't looked at that in a while.

But it it it is

Will Vincent

all comes back to compilers, right?

That reminds me, we we had Carson Gross who who wrote HTMX, and uh he's a big fan of writing uh programming language from scratch and teaches them now.

And he was he had some line, he's like, Oh yeah, you know, I just wrote a programming language for this problem, it's easier than you think.

I was like, for you.

James Bennett

I've actually I've been thinking about maybe doing a series of blog posts on like let's build the Django template language.

Okay, and just go through like okay, here's where we're gonna parse it.

It because the the Django template language is incredibly simple.

And the implementation is not that hard.

Like if you have if you're if you're really motivated, you can probably read through it and work it out yourself.

And and if not, like a little bit of guidance, I think, will get you there.

Um like just introducing the concepts and what's going on.

Um, and then it when it all kind of fits together in your head and you understand, oh, that's how it does that.

And now you're ready to learn how to do that.

Will Vincent

Plus one to uh writing to writing that.

Yeah, yeah.

Carlton Gibson

No, I'd I'd read that thread.

James Bennett

Yeah, I've got I've got some things I want to write.

I need to decide.

I've thought about turning that ORM tutorial into a book, but I don't know if I'm ever gonna get around to it.

Will Vincent

Oh Carlton loves when we talk about when I talk about books with potential guests.

Well, you wrote a book.

You wrote a I have your you wrote yeah, I have that.

Uh that was you built a blog in it, I believe, right?

James Bennett

Yeah, there was a blog and I think something else that was built in that one.

Will Vincent

Yeah, that was I

James Bennett

don't know.

I had a copy of it somewhere.

Will Vincent

OG Carlton, as they'd say.

Yeah, OG.

Carlton Gibson

See, don't taunt me.

Don't taunt me.

Will Vincent

There's a whole well, there's just a Twitter thread, you know.

Carlton Gibson

My children are like, Dad, you have to play more Fortnite because I don't know any slang or any of these history abbreviations.

James Bennett

Am I am I missing out on uh on what's going on on Twitter or am I not missing out by

Carlton Gibson

missing out?

No, you're not mi you're not missing out.

It's Will's just teasing me.

Okay.

Carlton Gibson

So following okay, so learn the abstractions to get into the ORM.

And then recently there was a go on, it was recently you wrote there was a um few articles on um using Django's or the ORM versus service layers and things.

And I think that's really interesting because like why Django, why the ORM, like versus other approaches.

That's I think that's a massively interesting topic.

We can link to those blog posts, but it'd be nice to

Will Vincent

we talked about with some guests back when you wrote it, James.

Actually, we okay looked at those.

So here's an expert, here's an OG expert out in the wilderness saying, giving opinion on Django that's like I mean I feel like

James Bennett

with the ORM the ORM more than anything is kind of the the the scenic one on of John of Django at this point.

It's the thing that if you're not using it, why are you using Django at all?

Um like the the request response abstractions are nice, but there are other libraries that will give you request response abstractions in Python.

So basically it's if you're not using the ORM, why Django at all?

Is kind of the question.

And I'm not sure the answer at that point is to use Django.

Um, like maybe you want some of the abstractions, you want uh like the concept of apps, but you want to use another ORM with it, or you

you're not using a

James Bennett

database at all, you know, that can make sense.

And you can do the kind of minimal Django.

Like uh I I think some people have even put together examples of how to do a very minimal Django.

Will Vincent

Yeah, it might be a talk on it.

James Bennett

Um But as far as the ORM itself, like like putting service layers in front of the ORM, like the reasons people give for wanting to do this are always you're going to end up reinventing the ORM or you're going to end up introducing a lot of complexity to make something like sometimes people do it for testability so they can swap out a fake database at run at during test runs.

Um, some people want to like hide the data access logic and you know have separate representations of like business objects and business logic.

And there are design patterns for that.

Like I mentioned in the blog post several times, like if you really want that, and if you really believe you want, you know, data objects and business objects and explicit logic mapping between them, you want the data mapper pattern and you want to use SQL Alchemy.

Um, and you want to use a data mapper ORM.

Um, but Django is not a data mapper ORM, it's an active record ORM.

And you're you're trying to take this design pattern that works fundamentally very differently from how Django's ORM works and push it onto Django, and that's not going to go well, and you're going to end up having to rewrite large chunks of the ORM to simulate ORM functionality in your service layer and introduce a whole bunch of complexity.

Like, I think in a lot of cases, the problem is, and I touched on this, I think, in the first blog post on that, is people trying to untangle interdependent code.

Like these really complex code bases where it's like everything does everything.

Like I worked on code bases where um like there were some models that had a policy that you always had to do um save with explicit field list because you didn't know what else might be modifying that model at the same time.

And so, you know, make sure to only actually um save the fields you're changing to the database.

Um, like and I tried to give what I think are some good guidelines for how to have models as the API of your app and what kind of access from other apps and other code into a set of models to allow.

Um and basically like things like you shouldn't ever reach into another app's model and start tweaking its fields directly and saving it.

Like, like don't do that.

Um, instead, you know, expose logical methods.

Like the joke that I've used a few times that I've always heard is you know, if you want to walk the dog, you don't reach down and pick up its leg and put it down and pick up the next leg and put it down.

You ask the dog to walk because the dog knows how to walk.

And building models that know how to do things rather than having to reach in and grab this field and set it to this, and grab this field and set it to this.

Um minimizing the amount of internal structural knowledge you need about those models because their public API, their list of methods that they expose gives you the operations you need.

Carlton Gibson

And then at the the table level, you might do put that on the manager rather than um the model itself sort of thing.

James Bennett

Yeah, like like row level operations on the model, table level operations on the manager class, alternate constructors on the manager class.

But yeah, I think that's it's hard to do.

I I've advocated for it.

I've never seen it completely pulled off successfully.

I I don't know that I've ever personally managed to completely pull it off successfully, but at least putting in the effort and trying to do it gets you a little better results.

Um and and then the rest of it was kind of like, well, what do I do with these really complex things?

Like, you know, somebody's canceling their subscription, and there's like six different models that need to be updated when they cancel their subscription, and it's like, well, if you put that in a service layer, like saying, Well, it doesn't make sense for this thing to be on one model, well, it also doesn't make sense for it to be in one service method because it's doing all these different things.

Like that that's moving it around while keeping it that complex doesn't actually help.

Um, so that that's the other kind of issue there.

And there are patterns for that, but that's just a matter of like familiarity and experience, and sometimes it's going and looking things up and being like, how how do people do this?

There there are I think Brandon Rhodes has been trying to do like design patterns for Python, his Python patterns repository.

I don't know if he covers like anything other than like the the original design patterns book, but there's a lot of stuff out there on how to like how to handle these kinds of like complex cross-cutting concerns in code and how to break it down so that you don't end up with like the one method that does everything and touches everything.

Will Vincent

I had one sort of larger question.

Is there sp any more specific ones you had, Carlton?

Just James?

Carlton Gibson

Yeah, I mean just on that, I think you know that the whole point you m we made right at the beginning about you know doing the web program quickly, like the the ORM fits that for me, the ORM fits that niche really, really well.

And most times if you're trying to do the web problem quickly, you don't need these extra complex constructions around it.

You just need to make sure that as you grow you you maintain hygiene because

yeah, it

Carlton Gibson

will get gnarly, it will get intertwined, it will get difficult.

But that happens to every project, no matter what architecture you use.

Will Vincent

So my uh concluding question is open-ended, which is so as as we record, you're you're you're you've you're not currently working for a company.

Yeah um I've

James Bennett

been unemployed for just about a week and a half at this point.

Carlton Gibson

Oh wow, okay.

Is that home-baked bread behind you?

I see.

Yes.

Carlton Gibson

Yeah.

James Bennett

Uh I didn't think that.

Right, okay.

James Bennett

So we didn't think it is brioche that was that was homemade.

But I

was not the one

James Bennett

who made it, but I have been enjoying it.

Yeah, right.

Nice.

Will Vincent

So my question to you is what kind of opportunity, given your experience, is interesting?

Like you've worked at small companies, large companies, you've tackled all sorts of web Django problems.

What, you know, where you're at now, what kind of things would be interesting?

I don't I don't know.

Will Vincent

Have you managed teams?

I feel like you've been more on the code side, but I don't know.

Like what so anyways, what what are you looking for?

What would what's interesting now to you?

James Bennett

I've never been a manager.

I don't really have interest in being a manager.

Um, I've been like sort of climbing the individual contributor ladder.

My last job, I was my official job title was staff software engineer.

Like I I put up a blog post about it where I said that I like working someplace where I can use my Django experience and be a resource to a team that perhaps is using Django and Python, but not being defined by it.

Um being able to do more than just be the Django guy at at a particular company.

And it's like it doesn't have to be any single particular thing, but just getting to branch out, getting to learn stuff.

Like I spent the last year, I didn't set out saying, I really want to get good at Docker, um, and I'm not really good at Docker, but I I s ended up doing a project that involved learning, you know, quite a bit about Docker, how to do Docker files, Docker compose, orchestrate, you know, Docker-based projects.

And that was really interesting.

And because they were Django services, they were being Dockerized, like there was an opportunity to use the Django specific knowledge.

But it wasn't just, oh, we're we've got a Django person, let's have him go do Django things.

Will Vincent

I would love to see a blog post on that, by the way, because that's a perennial issue, isn't it?

James Bennett

Well, I don't know that I

Will Vincent

think Docker approach for Django.

James Bennett

I don't know that I think it's

Will Vincent

Kubernetes and it's another.

James Bennett

Like I I did I I did a thought was okay at the time, and other people are gonna pick up and iterate on now that I'm gone.

Yeah, it it's that that that's the kind of thing that I look for is I don't necessarily know what's going to be interesting to learn, but having the opportunity to branch out and learn other things and do other things that aren't just being defined by Django.

Um, because it is important to me that like Django is important to me, and the things I've done with Django are important to me, but it's also important that I'm to me that I'm not one-dimensional.

That's not the only thing I can do, that's not the only thing I know, it's not the only thing I can learn.

Will Vincent

And do you think that's a I see issue?

Because the tech industry isn't great around ice in you know individual contributors.

There's there's a sense that trees only grow so high.

Um other other fields don't have that, but in tech there is a little bit of that.

James Bennett

Yeah, like once once you specialize in something, it can be hard to be like, you know, people talk about you know wanting the like the kind of T-shape of like, you know, really deep knowledge in one thing, and you know, then kind of okay knowledge in a bunch of other things.

Um, but I don't know how many companies actually succeed at that and how many teams actually succeed at wanting that, because getting getting to a level of of specialization with something, you can like it, but that doesn't necessarily mean you want it to be the only thing you do for the rest of your career.

Will Vincent

So so you're switching to front-end JavaScript development only, right?

Is that what you're so

James Bennett

when I was hired to journal back in 2006?

I believe I was officially hired to be their JavaScript guy.

Um you you can see how well that worked out.

Uh like at the time I was like I knew Python, but most of what I did professionally was like a little bit of back-end work and then like front-end, like HTML, CSS, JavaScript translating, you know, ideas and designs into actual implementation.

And you know, 2005, 2006 era front end was a whole different thing.

Will Vincent

But well, like would it be more interesting to be the fifth employee on something with traction where you need to rewrite everything to make it perform it, or would it be more interesting to be the 50th and have to specialize a bit more?

I'm sort of curious, you know, where you're at now, which which of those two problems would be more interesting?

James Bennett

I don't know.

I actually did talk to a company that was hiring for um their fourth person and first to dedicated back end person.

Um, and that sounded really interesting, like the the chance to kind of like start from scratch and and have a hand in building everything.

Like we you know, we we're programmers, we always love the idea of just starting with that clean sheet of paper and saying we're gonna do it right this time.

Will Vincent

This time is different, yeah.

Yeah.

James Bennett

Um but I've also been talking to places that are that are much larger and are looking for like someone who can come in and be like a resource and a mentor on Django stuff, but also do other things.

And then that's that seems like that seems really good to me.

That seems like where I want to be.

Um like maybe one day I'll do the you know first backend person process and and start everything from scratch, but I don't think it's gonna be this time around.

Will Vincent

Well, you could also just consult for equity, you know, with the the smaller one, right?

Because I feel like a lot of it is somebody who doesn't have a uh the wisdom who might say, This is how we're thinking about doing it, and in a couple hours you can save them so much time, but you don't necessarily have to do it yourself.

James Bennett

I don't know.

I've I've done I've done contracting.

Contracting is expensive.

Uh it's expensive to be a contractor in the US at least.

Will Vincent

Uh expensive for you or for them?

James Bennett

Um expensive for the contractor is was my experience of doing it.

Like, because there's so much overhead that people don't think about from being full-time contracting.

Like healthcare.

James Bennett

Yeah, like healthcare, 100 year old retirement, you pay more taxes.

Uh you pay an extra 15% tax for being a contractor.

Will Vincent

Um I'm self-employed.

Yeah.

James Bennett

Yeah, so so you you know you know this.

So it can be

Carlton too.

James Bennett

Yeah.

It can be tricky.

Carlton Gibson

Oh, you said one you you said one thing, so I'm gonna you said mentioned mentoring.

So I have one more question I'd like to ask you, and then we'll let you go.

But what what do you think it's something that I'm always thinking about?

What does it take to mentor somebody well, do you think?

The challenge I find is people come and they say, Oh, we want to mentor, but they don't bring anything to the table.

And how do you how do you find that balance?

James Bennett

I don't know that I have a great answer.

Um, in in my professional career, like working for companies, I've had a little bit of success getting people from kind of like post-beginner intermediate into like advanced intermediate to senior.

Um, like working with people who already had some basics under their belt and getting them kind of to the next level there.

I've had some success with that.

Um, I have never felt particularly great about my ability to start someone like from scratch and get them, you know, zero to at least kind of like a reasonable like beginner-junior, you know, first job level of knowledge.

So I don't know exactly um how to answer that.

Like the things that I've been successful with have mostly been like pointing people at resources that are kind of interesting and kind of showing, like, here's a pattern that you might not have run into, but we can do that for this thing.

Um like not necessarily telling them, oh, do it this way, just kind of showing them what options are out there.

Yeah.

James Bennett

And letting them build.

Because I I think that is the kind of thing that's one of those things that as far as technical knowledge, as as you grow, um, is kind of your mental library of patterns and possibilities and options that you at least know are out there, even if you don't consider all of them for every single thing you're gonna do.

And so I've had some success with that, but I I don't know I don't know how to replicate that necessarily, and I I don't feel like I've had great success with um with people just starting out.

Okay.

Uh the one the one time I did

James Bennett

Google Google Summer of Code, um, I I was probably a terrible mentor, and I was very lucky because my assigned student was Yanis.

Yeah.

James Bennett

So I I got the easy mode.

Yeah, yeah.

So you Yannis being like Django's second biggest contributor of all time.

So yeah.

James Bennett

Yeah, and and like so many other things too.

Yeah.

James Bennett

Like we we used to joke that everything you use has has been written by him at some point.

More or less, yes.

Super.

Carlton Gibson

Okay, let's let's call it that because we run up to time.

James, thank you so much for coming on and just sure sure sharing your opinions.

Oh, thank you.

Will Vincent

Yeah, so we have links to everything uh in the notes.

Check out James's talks if folks haven't seen them already.

And we are at Django Chat, chat Django on Twitter, and we'll see everyone next time.

Join us next time.

Bye bye.