Transcript: Test and Code - Brian Okken

This is Will Vincent joined by Carlton Gibson.

Hi, Carlton.

Hello, Will.

And we have Brian Akin with us, the host of Test and Code.

Uh hi, Brian.

Um, thanks for uh setting this up.

We we want we both want to talk to each other and we both have podcasts, so we decided to uh do a joint podcast and talk about testing PyCon Django and and do it together and see how it goes.

Um I'm I actually really like your yeah, so uh we're both I'm uh I said we're both the fans.

I was gonna say we're both fans of each other's podcast, but I don't know that about you guys.

I'm a fan of your podcast at the very least.

Um I I think

So okay.

So that's sort of like the one-sentence description, right?

Like we want to talk about specifically testing in Python today with you on our on our side of the fence.

I didn't know what we were going to talk about.

So um uh I'm I'm glad to know now.

So it's awesome.

Um, but uh so Will and I have met.

Uh Will uh so I'm Brian Hawkins, uh Will Vincent, uh Carlton Gibson.

Um I feel like I know both of you, but I have never met Carlton, I don't think.

Um Carlton, you've never seen

I'm sure I came on one uh podcast with you.

I'm sure I've been you and Michael together at some point.

No, did I not do that?

I'd maybe I

I don't I forget if you were on that, Brian, uh Brian.

I mean there's so many, you know, it's I we had our guest on recently and I forgot he'd been on before, which is a bit rude of me.

So we're at that stage.

But so I'm I run a site called learnjango.com and have some books teaching Django.

Carlton just finished five years as a Django fellow, so one of two people paid to actually work on Django and do the releases.

So that's kind of the the quick and dirty on us.

But we met at uh Django Con 2018 in San Diego, which was my first Django event, my first tech conference, real tech conference to be honest, either two.

And such a great time.

And I was introduced to the concept of the hallway track, because of course I was like, I'm gonna see all the talks in person.

That's what you do.

And then I realized all the poo bahs were not in the talks at all.

And I was like, that's a little rude, but then I realized there's this hall this hallway track thing and all these great conversations, you know, uh outside of the the main event.

And so this podcast came around as an idea of trying to mimic that and share it with others, because I came away saying, Wow, like I had no idea, you know, how great the community was and all these things I learned.

And is there some way to share that with others?

And that was on my side, the genesis.

And then I asked Carlton um if he would be my first guest, and he said, How about co-host?

And here we are.

Because uh I'll I will let uh Carlton talk eventually.

Um I guess uh with Will and I both have the personality of like talking for like forever around stuff, so um it'll be a miracle, but it will get you in.

But the the I did a whole bunch of uh talking with people at um at PyCon just just this last weekend, and the um it was interesting that you bring that up because sometimes it's uh awkward and to get started to podcast started.

You get somebody on the podcast and you and at first you want to like talk about what you're gonna talk about, and then you you know to try to get started, but in the hallway track or something, or just at the at the the floor, um the sponsor floor or whatever, uh you can run into somebody and say, Oh, hey, I you know, cool, hey, good to meet you.

Um tell me what projects you're working on, and you can just start running and and you get a 15-minute conversation in with no stress at all.

So um yeah, I'd I'd love to re-train.

So there's still

I'm like, okay, I have to like talk about a project that sounds cool.

Um so yeah.

Um well, so I cut you off, but Carlton, what's um is that your story too?

Are you you're sticking with it?

Or yeah, what was your side of the of our engagement?

You know,

So he gave a really good talk, and so you know, I'd went to his talk um and uh we we started speaking after that, and yeah, and then sometime later Will said, Oh I'm thinking about doing a podcast.

And I was like, Oh, that sounds really cool.

But my favorite podcasts are ones where people kind of chat, it's very difficult.

A solo person just by themselves, and so you know, I was like, why don't we have you know, we'll we'll add this name, I think.

You may even add the domain Django chat or something.

I was like, that's perfect, let's just chat about Django.

So nice it's so episode.

Like I know you and Michael, um Kennedy have um, or how does that all work?

Your your podcast network, because you have a couple great.

Um he and I were kicking around the idea individually before we even met of doing podcasting the way back when.

And so he started um we started Python, I started testing code, but it started out as the Python test podcast, and I just got tired of saying that.

Um testing code is easier.

Uh although I've been I've had also people tell me that they they thought the podcast was testing code uh instead of test and code.

So I've been a little bit uh conscious lately to try to enunciate.

So it's test and code.

Um, but anyway, the and the testing is first because testing should be first.

Um and then uh

Uh and he was and at the time there were those it was just those two uh podcasts about testing.

And then there was another one uh like to podcast in it, started around the same time also.

Um but Michael and I were both in Portland, and um I made a conscious, I mean I uh both of us I think individually made a conscious choice to support each other um and uh promote each other's podcast and everything.

Um, you know, two two social networks uh could have a larger audience than one.

And um, and then we were doing the our individual thing for maybe a year.

Um I can't remember the exact time frame, but he contacted me and said, hey, it it'd be kind of fun if we did a podcast together.

Um and uh of a news format sort of thing of like a few news items uh every week.

And and uh so we just we tried it and it and I love the the I love the format.

It's a it's a really fun thing to be able to we talk about like four four to six topics, but you with six if we have a a guest host on, but everybody picks a couple topics to talk about, and it could be any anything, um uh anything Python related usually.

Um or sometimes mostly we thought it'd be news most of the time, but sometimes it's just a cool project that we haven't covered yet, and it could be a an existing older project, and um, and a lot of people have given us feedback that they just uh they just constantly have like a list of things they want to try out because of our podcast, which is nice.

But the the having another co-host gets you to do it every week.

Um so uh mine's solo and uh so if it um but I I I kind of glad that I had that flexibility.

So but I I don't do it every week anymore.

Um there were a couple years where I've tried to do weekly, and um uh and some and some of the episodes are um interviewing people, and some of them are just a topic that I want to talk about a little bit.

And uh kind of like and I was inspired a little bit by the um what there was like uh program tea tea time or something.

There's there's a tea related uh or coffee break or something related programming podcast that I was listening to.

There's just like one dude talking about some topic for 10 minutes.

Uh and I'm like yeah, say that again.

Um so uh but um yeah, the the motivation of having so a guest or a guest or a sub host or whatever.

What is it?

Co-host, that's it.

Co-host, co-host, yeah.

I just looked it up uh uh three years ago.

So we that's that that was the connection.

Oh, yeah.

Because I've been I'd I'd done one with uh Michael on TalkPython, and then you and I actually recorded one years ago that there was like technical difficulties or something.

Yeah.

Um so uh but Carlton, come on, get a word in.

Like well,

no, no, no.

Okay, well, so I would yeah, I would say.

I mean, it's so much easier with a co-host to just sometimes you're like, I'm out of it this week, like you need to do the lifting, or hey, like like Carlton's been great about we started off weekly and then we've been going every other week and we take breaks sometimes because kind of the key thing is not to burn ourselves out.

And we we have thousands of listeners, but it's not like our it's not a moneymaker for us, so we you know, we want to stay sane.

And at this point, for me, it's frankly a chance to talk to interesting people and like have an organized chat with Carlton.

Like, you know, everything else is secondary, like you know, I guess you know, marketing our stuff a bit and all the rest, but it's really just nice to talk shop since you know we're all sitting in our own little caves somewhere, you know.

We're not like I'm not overwhelmed with social stimulation with work.

The the being willing to talk with somebody is like or having being being okay with asking somebody to talk with you is something that a podcast gives you that's really pretty cool.

Because I I've reached out to people that I would have never just reached out in person and said, Hey, you don't know me, but uh can we talk for like an hour?

Um massively valuable time.

Um, so but with a podcast, it's like and it and also it also one of the things, lovely things that podcasting taught me is accepting rejection.

Uh, because it's it's okay.

It's it's okay if somebody says, no, I don't, I I don't want to do that, or I don't have time, or something like that.

And and I've realized that I don't feel bad at it at all.

I um it's fine.

But um, what is uh so so Carlton, you just got done being a Django fellow, and I I guess I'm not I'm not involved with

I do a little Django, but I don't know what a fellow is.

Um, and it's it's got a lot of history, and um just give you an idea on the the Django issue tracker, there's um three to five new trick tickets opened every single day.

So there's like a thousand plus tickets every year, and you know, and that's gone on years and years on years.

Um the security reports that come in pretty much every week, and they need triaging, they need looking at, they need resolving.

We do so in 2022 with the security team, there were 10 CVEs handled through Django Security Progress uh Processes reports, for instance.

There's you know a constant stream of pull requests that need to be dealt with.

There's releases monthly, there's major releases every eight months, there's you know, there's community handling that needs to do.

And it turns out that on a project the size of Django's, it literally cannot be done just on volunteer effort alone.

And so, how long ago now?

I don't know, maybe eight years ago, nine years ago, there was the um they created the Django Fellowship Program where people were contracted.

Um, initially Tim Graham, and then I came, I joined five years ago, Marish Felicia four years ago, and Natalia, whose surname I don't know but really should now, um has just started as the new fellow now.

And that that um having a contractor, having someone who's people who are paid means that the tickets get triaged, means that the pull requests get reviewed, means that the releases go out, you know, on time every month, means that the security patches are done.

Um, and I honestly believe that the fellowship program is sort of the reason why Django is being able to survive and not just like and didn't die essentially, that it got to that kind of um fruition of an open source project where the original contributors started to fade away.

And why did Django not just collapse in on itself and and and fade away?

Well, because of the fellowship program, and it's I honestly believe it's one of the um you know the reasons why Django is sustainable and reliable and will continue now till probably the heat death of the universe.

Um and then

Yeah, good point.

And there's so Django has a Django, the Django Software Foundation, and if you're an individual member of the Django Software and Foundations, that's the equivalent community award in the in the Django community.

Okay.

There was originally one until I came um started.

I think there might have been the original pilot, there might have been an overlap with it.

Um Burker did it for a few months.

Yeah, yeah, yeah.

A little while.

I don't know the very dark history that I'd need to um ask others.

Um, but Tim Graham did it for four years by himself, which was amazing.

He did it full-time, four years by himself.

I've always said, so when Tim stepped down, um, you know, there was some conversation.

Well, Carlton, do you want to carry on?

Do you want to go full time?

I said I didn't really want to do it full-time, and I didn't want to be the only fellow because it's really hard.

Like you're you're on the you're on the sort of the front edge of the fire hose.

You know, it's a it's a really big um jet of traffic that comes at you, and a lot of it you have to say, sorry, this isn't a valid um request, or you know, we're not gonna add this.

You have to close a lot of tickets as won't fix, and people get very upset when their idea is rejected, and some sometimes you get it on the nose.

You know, people aren't necessarily meaning to take it out on you, but they do.

So one of the and so to have some support is important.

Does Django have a similar sort of idea?

There's going to be elections for the new the new version of that um coming up for the 5.x series now.

But um if if in Django we try and work by consensus.

So normally there's a mailing list or the the forum where there's a discussion, and if there's not a clear consensus, then it can be you can we can ask the steering council to have a vote and and say what the way it should be.

That doesn't happen very often.

We're quite good at you know reach trying to accommodate and reaching some consensus, but sometimes it does have to go to the steering council and they have to just make a decision.

Carlton, you slipped a couple in.

But what was it?

There was one in particular.

There was something

the last idea.

I don't remember.

I've I've blanked it all.

It's okay.

So I think I'm in week three of stopping, and I I and and still so the the Django Con Europe is coming up at the end of May, and I've said I'm not gonna look at Django Django, and I'm not gonna look at track unless you know if someone CCs me or at mentions me, then that's fine.

But I'm not just gonna watch the notifications because five years of every comment, every commit, every pull request, every ticket, every uh you just need to step away and cleanse the habit of checking that.

So I don't really know what's going on at the moment, which is lovely.

Uh well, I was just gonna add Brian.

Um, but like Python, Django was set up as a nonprofit.

So the fact that it's a nonprofit is why there are some volunteers who run a board.

We could talk about that.

I I spent the last three years as treasurer on the board running the money.

So um, but that's what funds the fellowship program, and that's kind of how it's very community-based, which generally works well.

It means we go a little slower, but there's a there's a churn of people.

So when someone gets burned out, like you know, not burned out, but five years Carlton needed a break.

Like as treasurer, three years I needed a break.

Um, there's other people there, and you know, we're still involved in the system.

Like Carlton just released a Django Skunk Works project, you know, last week.

So um there's a recycling that keeps it fresh.

And I'm really happy that you guys are here because I get a lot of questions about since I run uh testing code, wrote a book on testing.

One of the most popular questions I get is how do I test my web application?

Um and uh I'm not a professional web developer, I'm a professional embedded software engineer.

So um uh not my wheelhouse really, but I uh so my my deflectiveness sometimes is just you run through the API instead of the uh the front end then.

But that's not really that's not really fair, um, maybe, but um, but Django's and I uh I was the first time I interviewed you, I didn't have enough experience with Django to to ask decent questions also.

So there was technical.

I'm still learning Django.

I'm in the I'm I'm a Django newbie, um, but I am starting a project and I'm thinking about this of like, how do I make sure it's working right?

So I'm really glad uh I don't know how to start this conversation other than how do I how do I start testing with Django and what do I need to think about?

Have you like how are you starting?

Are you like because I have I'm yes, just yes, I'll leave you with that.

What what are your what's the process for someone like you who wants to start Django and is wants to know how to do it properly?

So I start I did I did a like uh starting Django and I started the the tutorial and the tutorials I I read slow and I'm very impatient.

So I um I'm like there's gonna be a long time.

I to to to it reminded me of my first uh first couple months at Hewlett Packard when I when I first started out.

So I was a compute pure computer science person.

I got hired uh to do um I was work working on uh satellite testing at first, um, but then uh quickly moved into oscilloscopes.

So doing scopes, no spectrum analyzers.

And one of the the one of my managers said um you really under should understand electrical engineering, um, and gave me this big Horowitz book or this this huge gray book of of and I'm starting like learning DC circuits and stuff, but I'm working with microwaves, and I'm like, there's gonna be it's gonna be so long before we get to microwaves.

I'll just rather I'll I'll just rather the I'll cheat.

And so what I did was I learned the I I was just nice to people at work and learned the experts in the RF stuff, and I learned enough electrical engineering to talk to the electrical engineers, and so I kind of feel like that with Django.

I'm like, who do I need to find some experts in Django and I'll just learn how to talk to them and that will be good.

Um, so I have a start I found a starter project.

Uh there was a and I don't even remember the name, but somebody that knows what they're doing, and uh I'm starting with their

Can you tell me who it was?

Well, I'll I'll I'll I'll speak out loud.

So there's the big one is there's one called Cookie Cutter Django, um, which that's probably the default that people find.

And then there's a host of slightly smaller ones.

I run one of them, that's why I was sort of curious if you came across it.

But there's uh I think there's a gap in terms of people who are experienced programmers who want scaffolding to kind of jump in.

Um there is there's only one really paid starter project for Django, um, Sas Pegasus that Corey Zoo runs, which is fantastic.

So the the sort of market question is is there room for more?

Um, or uh, you know, he's he's I think he's doing that pretty much full time.

Um sorry, I'm just I'm just I'm getting free market research from you because you know, Django developers, all of us after some period of time have our own personal starter project, just like all the consultant, you know, the agencies have their own.

And so it's it's one of these standards you know, you're having a pint and you're like, you know, what if I charge for my starter project?

Like how much work could it be?

You know?

Uh and you know, it's a it's the code is the least of it.

Well, the answer is that's kind of why I was

Yeah, yeah, but it might be worth it.

I think there's room.

I think that there could be dozens of script.

I maybe dozens would be confusing to the marketplace, but um, but a handful of uh I could easily that there's different target audiences too.

So, like you were saying, the uh Sas Pegasus, um, that's a particular kind of thing that's as far as I my brief looking at it, it's um targeting uh uh paid uh subscription-based kind of projects, which not all projects are like that.

So your project might not be.

So you said testing.

Um, it does have you know a lot of great things.

The test the payments thing seems to be the one that lets people open up their wallet as opposed to uh sorry, Carlton.

I I'm just like this

As I sit in a SiteWord co you know podcasting booth.

But anyways, yeah, so anyways, Carlton, testing.

How where where should Brian start?

So Django in Django.test, it has a uh the module Django.test, it has a client.

And what you can do with that client is you can make a web request to a URL.

So you can say get client.get, you know, forward slash homepage, you know, whatever the URL to the path to your home page is.

Um it might just be just slash, it might be nothing, it might be just you know the index page.

Um and then it will simulate the actual request and it will it will run the request through the the web stack, the request response stack, it will give you back the response, and it will also add a few extra features on the response so you can test things like um the template context.

What so you can get the HTML and you can say, Did the HTML include my header?

Or you might not that might be a bit hard if you're looking for a particular fragment of HTML.

So you might want to say, was this template rendered with this context?

So you can get the which template was rendered, which context was it, what was the status code?

What was um, you know, did it redirect to a login view, for instance?

So say you've got an authenticated view and you want to check the authentication works, well, you'd make a request to that with an anonymous user and an assert that it was redirected to your login page, for instance.

Um should use the text client.

Does it does it go?

Do you have to have a server running or no?

That doesn't.

So that's that's the basics that will do most of your cases where really you're not you don't need the browser behavior.

You it just it talks, it runs the test client will sort of use your Django app internally, so to speak.

It will make a request to the Django app and get the response, and you can test the HTML.

But that's not the browser behavior.

So there's a level up where if you need, like, you know, say you've got some JavaScript on the page and you need to test that the JavaScript actually does function, then you would need to use a thing called Selenium or Playwrights, the new the new kid on the block there.

I'd you know, I don't have much to say about which is better.

There, a lot of people are liking playwright, but I don't know if the integrations are as mature, is all I can say.

In Django's test, we we use Selenium.

There are lots of tests, and we use that for testing the admin.

So, for instance, in the admin, there's um JavaScript to add a new inline related object in the in the you know, when you you've got the admin, you've got the inline objects, you want to add a new one.

There's a bit of JavaScript to do that.

Does that work?

Well, we have to use a Selenium test for that because it's all JavaScript logic.

Um, and that does need that.

That will, when you run that Selenium test, Django will fire up um a live server.

It's called live server test case, and it fires up a live server, it opens a Firefox window or whatever your Selenium driver is using, it clicks on the actual buttons as you as you describe in the test, and it it runs it.

So you can do that kind of more um that's like I guess they call it fully end-to-end testing or something like that.

You can do that Selenium testing, but those are difficult to write and they're more brittle because every time you change your CSS or you know the position of a button moves, sometimes it doesn't, the test breaks, and you have to to fix those.

So I would say save those when you need them, but the first port of call is Django's test client.

So let's say um I've got a uh uh application where um somebody has to log in, and it might be like there's different levels of log like permissions and stuff.

And I want to think, okay, I've got a different kind of per person logging in, and what what do they have access to?

Does this workflow work?

Uh go going through some stuff.

Can I do all of that with the test client, or do I need to do playwright or something like that?

Like in in most cases, you would do so.

I, you know, I'm thinking uh you might have a view where you start off in the foot, and you might you might just test you might test a sort of a view, a post, a form post view in in a kind of cup couple of tests together.

You might break them up into separate functions, or you might just go, right, first of all, make a get request to the page to load the form.

Um, then make a post request with the data, then check that you know the the object was created.

But you can you can authenticate with the test client so you can give it you know log in with this particular user, and you can either do that by sending it to the login URL or it's got a shortcut method where you can kind of say, look, log this user in, and then we'll do the run the rest of the test because you know there's a lot there's a teeny bit of baller plate going via the login view and then back to the other view, which you might not want, might not want or need.

If I I don't want to do a workflow thing, I'm I'm saying, well, well, I I assume that this kind of a user is already logged in, and there and there's already data there and stuff, and I want to just test this little thing.

Is there like there?

I'm assuming there's setup stuff you can do to like yeah,

So Django, Django ships some some test cases which are based on unit tests.

So you you know, normally in your test set, you'd define a setup function, and that's where you do your your fixtures where you you populate the database with whatever you want, and you um you know, you set so what is it?

You you you I can't remember the three A's, but you set it up and then you do some actions and then you assert the um what you're looking for.

It's a range.

Arrange.

There you are.

Yeah, yeah, yeah.

You arrange what you want in the test case, and then you do some actions, and then you assert that it was and it and um the Django test case uh test cases provide some nice assertions, like assert that a query set is equal to, or assert that I don't know, a warning message was raised, or assert read that it that it uh a redirect to a certain URL or um these kind of things.

It's got a lot, you know, it's quite mature.

Uh extends unit tests.

Okay Django, Django's just client extends,

Whereas uh I don't think that PyTest Python extends it, it's just um it's just raw unit tests.

But the um but the the 2021 developer survey from uh Jeff Brains and Django um says that 39, only 36% of the people use unit test and 39 use py test.

That's because everyone's read your book, Brian.

And I really do highly recommend that book as well.

Um so yeah, you can do that.

So there's a there's a plugin for PyTest called PyTest Django, which gives you lots of it gives you, for instance, a fixture for the client and it gives you a fixture for um I know the database access.

So um PyTest Django will not allow you to access the database unless you tell it, no, no, no.

Here's a pot, you I think there's a marker or a fixture, you can use either, but if you use one of them, it lets you access the database.

Um whatever.

I think there's even a live um server fixture.

So if you want to do these kind of Selenium tests, I believe I'm not 100% sure because by preference I do use um Django's test cases.

Um myself, but I have used I've used PyTest plenty and it's it works just as well.

It works just as well.

It probably has similar sort of the similar test cases built in or something, hopefully.

Okay.

Um, because the Django's developer survey is like the people that would might even know that there's a JetBrains-based survey out there.

Um, so this is where this all sounded really incestuous.

So Django had a survey like 2015 and then hadn't had one for years.

Meanwhile, Python has one which is very helpful to the community.

So one of the things when I got when I got on the Django board in 2020 is I wanted to do the survey.

So that was one of the things I worked on was it being in charge of that.

So hopefully it happens this year in my absence.

But um, and we've partnered with with JetBrains, is you know, so we fly, we we Django fly blind because we don't track downloads, we don't track anything.

Really, how does it being worked?

It's what you know the fellows see in the issue tracker, it's what's in the forums, and it's what people talk about in the hallways.

We know that we're massively blind to a lot of things.

So anyway, so the survey was brought about to try to shed some light.

Now, we know that a lot more people use Django than take the survey.

I think it was something like 8,000 people used it last took it last year.

And this is a communication problem that uh being on the board, Django has.

How do people find out about anything about Django?

It's like, well, if you go to the Django website, used to be you go to the Twitter account, like there's no email newsletter for Django.

Um, and this is a whole separate discussion Carlton and I have had many times.

I think even on this podcast, but there's there's definitely a lack of light being shown in terms of how are people actually using Django.

I you know, anecdotally, the numbers I see in the survey match with what I my impression.

Yeah.

But you know, I'm a American, you know, going to Django cons, right?

I'm not like

Like they just it's they they go they download Django, they're using Django.

Django says these are the testing tools, they use those testing tools if they test at all.

I was gonna say I'm just grateful if they test at all.

Um so yeah,

well, yeah, that's the thing.

Is that still the term used, or is that still a thing that people do?

Um so you can you can populate a database and then you can do this.

You've got this command to dump data where it will dump it as JSON um files, and then you can load load them again with the load data command.

It's quite you know, it's got you know all sorts of useful uses.

Um, but you can also specify those those dump data files as fixtures for a test case, and it will it will load them in kind of prior to to you running your case as fixtures.

Yeah, I don't think people use that as their first port of call, and I don't really recommend people using that as their first portal.

You're much better off in um setup in setup or setup class data, which is a class method that only runs once rather than once per test.

Um creating the the the um the objects you need you know using the ORM or using factory functions that you've you know written to speed up, much like a pie test fixture that you're familiar with.

You know, you create the the fixture function that does that creates the objects.

You would do the same, but you just instead of having it as a fixture function which you can't ever find, you'd have it in the um the test case, sort of just above the test that's tested.

Um I want I kind of want to like know if I've covered all the the things I've added.

Is coverage do people use coverage?

Does that make sense?

Um there's even a plugin which will help you cover your phone.

And oh, I didn't test the else branch, you know.

Okay.

Django comes with all these tests itself.

So there's how do you know what is a core Django behavior versus what is something you did, right?

So I think sometimes people end up trying to test Django behavior.

So generally, Carlton, you're making a face.

No, no, no, no, no.

Carlton and I both uh come down on the side of kind of write your code and then test it.

Or better yet, when something changes or when something breaks, then you add, then you add the test.

So that that's a you know sort of an in-between system because anytime so I you know in my books, I lead, I have tests and I show people how to do this, and I'm like even the beginner's book, like boom, we test, we're testing everything.

When you've changed something, then you add a test.

So just because you see hello world, like you know, you test, does it show hello world?

You don't need to test that the view got the templ.

I mean, you can, but like it's only when it's something new.

Like I don't test the admin, for example.

Like I don't test that the admin works because it should work 'cause unless I've changed it.

If I have changed it, which which Django lets you do, then I would run a test to see, oh, does does this customization uh work as planned?

So it's when you customize something you want to test it.

Um, so don't don't you doesn't just doing anything change the admin, or do you have to like would you mean change how admin works or well

So you can there's built-in ways to how you display your your data, whether you want it kind of how you want it listed.

You people can and do abuse the admin, so you can kind of use it as a poor man's CMS.

Um or maybe there's valid use cases for it, but when you're so there's there's there's changing kind of the the graphical layout of the admin, which Django gives you tools to do.

You don't need to test that.

But if you're I mean, Carlton, you've seen it used and abused.

This is one of the things as a fellow, people people, I mean, there's all sorts of thoughts on the admin and trying to make it be more than it is.

So use the admin.

The admin is a power tool and it's wonderful, and it's one of Django's secret sources, and you know, push it to its limits.

But people, when people are like, oh, can I open it up to the world?

No, no, no, no, no.

No, just write a custom view.

It's gonna be much less work to write a custom view than it is to try and force the admin to do this funny thing that it was never designed to do.

So there is a sort of there needs to be a limit to what the admin can do, not least for maintainability's like, like you know.

And I was like, I I I don't want to build something like that because that's well well one, I'm considering I'm talking about like 20-year-old Django probably or 15-year-old Django.

Um, so it's even that's probably better now um than it was 15 years ago, of course.

But still, um if I just want to have somebody enter some stuff, I don't want them to have to go through and look at the data tables and stuff.

Um that's that's too much.

Uh you know,

There's built there's permissions that the problem is Django lets you kind of do a lot and you can like you can have super, you know, you have a super user and you can have different access.

So you can give different people access to the admin and permissions it up a little bit.

And then, you know, just as you said, it's like, oh, do I really need to build my own custom thing for this when it's kind of working well?

Like, you know, at some point, yeah, you do.

That that's kind of how it plays out, right?

Like it's just me, it's me and Carlton.

Oh, now we're a team of 10.

There's some non-technical people, we really want them to be able to like answer customer support.

So we have set on permissions, so they can't touch this, but they can enter this.

But they're still like that's when you get into trouble.

Or you start like saying, hey, the admin's slow, because I'm trying to view like a million records within the admin.

I mean, there's uh multiple DjangoCon talks about you know using and abusing the the admin.

So it's a very natural progression.

I mean, I I I probably abuse it too a little bit, but it's more like in yeah, it's easy, yeah.

It happens, but you should avoid it.

I'm like using all the stuff, and I'm not really changing how Django works.

I'm just um building a website.

Um, so the I'm assuming my my approach probably would be test the thing like the end user experience.

Um, the the somebody goes to this page.

And is that is that when you said that that's is that testing the views at that point, or yeah, okay, or the models.

The view is like the controller in the old MVC thing.

It get it go the request comes into the view, and the view's job is to turn that request into a response.

Okay, so in order to do that, it put it probably goes to the ORM and it says, Can I have some some some records from the database?

And then it grabs a template from the file system and it puts the two together and it creates a um um it creates a response.

Yeah.

You want to test well, does that load?

So you you know, for any, if you've created a nice view, create a a test which is that hits it and checks that it returns 200 and maybe that the template was correctly rendered or something, you know, that so that it it works.

Because one day you'll I don't know, you'll do something and then you'll just break that view.

And that you've got just that test that checked the 200, that it that it returned an okay response, will tell you that you broke something.

Now, yeah, you know, if if your view has complex logic and it could it might return a 404, so you know if you go to somebody else's repo on GitHub, it goes 404.

It just said pretend you know, if you haven't got the permission for it, if it's a private repo.

It just says 404, it doesn't exist.

So lots of URLs are like that, where if if you've got permission to access the the object, it will show you the response 200 and render the template.

But if you haven't got permission, it will just say, oh, it didn't exist, or it will redirect you, or it will say 403, like forbid forbidden.

So those kind of different responses that you might get, they're worth testing.

And then they're quite high-level, they're quite they're almost integration test level rather than unit test level.

But you can spring those around and they give you quite a lot of coverage and quite a lot of sort of um juice for your money, if that makes sense.

Um and they're quite easy to write.

Um anyway, go on, Will.

Because I think tests in a Django context, people have no idea where to start, in part because they don't understand how the web works, how Django works, let alone how do I test it.

So it's just a lot of confusion.

Yeah.

But you kind of write the same tests over and over and over again.

Like tests are super confusing until they're incredibly boring, to the point where like we've like ChatGPT and all this stuff, like throwing that at like what should I test would be a great use case, actually, because a lot of times you're writing the same tests over and over and over again.

Um and so I was for you know, for you or for someone who's like, what test do I write?

Well, you can look at open source projects.

Like if you go to the awesome Django repo, there's a list of some, see how they do it.

You can, if you type in like Django testing tutorial, you'll probably see like Django has docs, Mozilla has one, and then I have one that kind of gives you 80% of all you'll ever need.

Um, like in my books, I show you how to do it.

But I mean if you're doing CRUD and you're doing forms, you're really you're like test, as Carlton said, testing the model, testing the view, testing the template, testing the URL, does it return what I want?

Does it not return what's not expected?

And you just kind of do that at infinitum um at a certain point.

So just like find an example of something like it if it's CRUD form stuff, which you know, 90 whatever percent of of the web is, you're kind of writing the same test over and over and over again.

Well, one of the things Right Carlton?

Two setups.

And then when you have logic, it dep, you know, at a base level, as your logic becomes more complex, yes, then then things change and you know you want to test that out.

But if you're just kind of the the basics of like, does this page display the information I think it does in the way I think it does, like it's kind of the same thing over and over again.

And so what people cut quite often do will um embed domain logic into manager methods or methods on the ORM themselves.

So for instance, um, you know, I will quite often say I've got a project, uh, a project model, and I don't it's a list of projects, and I users are only limited to which projects they'll they can access, you know, via an organization or via a team, or you know, maybe they're the owner.

And so on the project model, I'll need them what's called the manager, which is the objects bit in Jack, you know, project.objects.filter is the RM API, but that manager, I'll add a method called for user, and then I can pass in the current user.

And then you're writing more of a unit test type of approach where you literally you've created a few instances, assign some of them to the user, create some for a different user, filter the the query set, and make sure only the users um uh uh projects are returned.

Okay, that kind of thing is is embedding your domain logic, right?

You you're checking that that works, and then that that manager you'll then use in your view, and you don't necessarily need to test again that it was correctly filtered because you know that the base query set was filtered to that you to the request user, for instance.

Um, just as an example.

Like that's a more I would say intermediate advanced level book on testing.

And he certainly knows his stuff.

So um, you know, uh you could do an hour on like how do you not write the same test twice with him?

So you would you are you gonna be using PyTest?

Because you're you're the expert on PyTest.

So will you be using Py test for your for for testing your project?

Yeah, uh

It's about managing fixtures because in the in the unit test world, the the the fixtures go the the arrange goes in the the setup method of the test case cast and it's right next to it.

In PyTest, they do the dependency injection thing where the sort of the fixtures live somewhere else and then they they get injected and one thing i found as projects scale is that it becomes increasingly hard to track down where the fixtures come from so do you have any kind of advice for you know managing your fixtures to keep them maintainable to keep them trackable because I I sort of end up at the point where I'm like right clicking in the IDE and then the I to go to definition and I'm like please take me to the right place because I'm just not sure you know is it scales up

But only if you've got I'm not I'm not a PyTest you know I use it but I'm not like a PyTest efficient identity by choice I'll use the unit test method.

So I'm asking

Well the the most obvious way now is the dash dash fixtures command line flag so if you if you run in like either as a whole you can run uh if you run all your test suites but if you just run run as if you were gonna run all the test suites like or your particular test and add the dash dash fixtures uh command line it tells you all the fixtures that are available and where they're defined.

It still is um but but you could get more information if you did dash dash fixtures with like uh verbose like dash v also but um and the verbose used to be the only way you could get the actual file name and line number where it's defined and now by default it gives you the file name and line number so um it's really not hard to find them but I also you don't have to put them somewhere else uh you can put them right in the test file if you don't want to go hunt for them don

't put them somewhere else you can put them right there so they're either they're either in the test file or they're in a conf test file somewhere.

Um and then I get I sometimes feel a bit embarrassed as if I'm doing it wrong or something as people

Uh because I because I want them I want the setup to be obvious that it's not part of the test.

The other thing that you get is if a failure you can assert within your fixture um and if an assert fails or any any uh with Py test any exception causes a failure but if it's in a fixture any exception uncaught exception causes an error so that distinction between error and failure is did my did the the thing that will go wrong happen in the setup or did it go happen in the test itself?

So would you would you kind of recommend not not um not overcommitting to being super dry and that you reuse a fixture every would you rewrite a fixture?

Would you like so say you've got the same you know you you um different tests different places in your project you you need a project fixture would you would you be tempted to rewrite the project fixture closer to where it's being used to avoid a bit of confusion

the and the conf test can it can go all the way up and the cool thing is you can start out at a file level and then maybe move it to a directory level if you want to share it between tests and then maybe you can move it up a directory but the tests don't have to be rewritten um the the you just move where you've defined the fixture it's it's not um it doesn't have to be the the test I love that you don't have to rewrite the tests with unit tests you have to you have to change the code if

you if you if you decide you want this fixture that fixture logic to be at the at the file level or a class level that's a completely different thing.

Actually you only have class and function level right um I think there's

something like that.

level or something um and the test doesn't the test still doesn't know about it the test just says uh it gets the fixture stuff the other thing I really like about fixtures is that the setup and teardown are in the same file or in the same function um they're just they're just divided by a yield keyword whereas in unit test you've got the setup and teardown completely separated into two different functions yeah

the issue there is just I think the answer to my question is is just maintain hygiene you know because I've been on projects where I've been a bit like ah where's the where these where's this fixture and where's that fixture and I think it was just got a bit messy over time it's all

and then also um being able to uh do type hints in it so if they're they're well type hinted um uh like VS Code and PyCharm will also like do comp code completion for you uh with your the returned object um so um but I'm I'm looking I probably will get you guys back on because I'm so I've got this project that I haven't written any tests for because I haven't actually have yeah don't have any running code for it yet but um but I it'll be an

interesting uh interesting progression to build this thing up and find out like um how I feel about the Django Django testing environment and I'm I'm I'm somebody that likes shiny new tools so I'm sure I'm gonna try out like the Django test plus I mean it's got a plus in it it's gotta be good um and um and I've heard people talk about factory boy I'll probably like try that out and all these sorts of WYSIWYG not WYSIWYG but whiz bang features and things

that but I also don't like too many things because if I pull in a a dependency

in?

Because you know so many different asserts it's can be hard to remember.

So some of the helper functions are helpful especially around Django to say like well assert that the user's logged in or something I don't you know I it might be easier as an assert a helper function than it is as code.

Yes.

Well what you're what you're verbalizing is very common in that like I want to try out all the things but like where do I even start because you know Django has third party ecosystem which is a huge strength and you can go to Django packages.org and see however many thousands and thousands there are.

But like which ones do you want to use first right?

So if you went to the testing section there's a lot um so there is a a curated list and again I'm plugging my own stuff again here but it's called awesome Django that uh I run with actually the person who runs Django packages now uh that has a list of you know how how many is it?

You know a dozen right with this idea of trying to say okay well those of us in the community, there are some packages that have been around for a little while, have a certain number of stars that are maybe a a good first place to look rather than just what's new and shiny.

Um, because that is, you know, and there's also things like there's a Django newsletter and and stuff to try to both expose people to new things, but also remind people of existing things that are plugging along, like like Django Debug toolbar, for example.

Like that's a package that is, you know, I just did a post on like what do I think are the top like I think I said 10 packages, third-party packages you'd use.

Like there's a whole discussion of this.

I think everyone's got that probably top five.

But who maintains it and who uses it, like someone started it, and then there's this group called Jazz Band, like so the maintainability and the fact that um positive things are still just like Django itself, like there's a tremendous amount of stuff in the most recent release of Django 4.2, and yet from the outside people say, Oh, Django's kind of just Django, like, right?

Like, I don't know if you had that with your books.

I mean, you're on you're on the second edition, I'm on like the seventh edition, and I basically rewrite them all from scratch, and I'm like, it's so much better, but like everyone's like, oh, it's just still there, right?

So like it's hard to get people to appreciate, like, you know, just being still around is a huge sign of success generally for for a code repo, for a book, for a project.

Yeah, um, but it doesn't seem like that, maybe unless you've maintained one yourself.

I mean it it had changed, so I I did add new things, but it was more around the teaching.

So the uh I I didn't I didn't feel that the first edition uh was the appropriate ramp of difficulty.

And so I the the main impetus for rewriting it was I'd like to start people out with the real basics and then introduce fixtures, try to do it gradually.

I think I but I was a little bit more abrupt than maybe I should be, but um, but that's fixtures coming in chapter three, and then just gradually add on complexity so that by the end they're pretty much.

I mean, if you really understand the whole PyTest book, I think you're a PyTest expert at that point.

Uh I'm not teaching 100% of PyTest, but it's it's the 80% that or the 50%, 60% that 80% of the people need uh every day.

Um, and then some extra fun stuff.

Like I really want people to write plugins more, um, because it's a great way to and packages, it's just a packaging packaging extensions, uh, because that's how they should be sharing code.

Um even in within even if I don't do a PyPI package internal to a company, people should be sharing their fixtures with the plug-in packaging system because it's there, why not?

Um, but I do I see people trying to share like trying to share fixtures between projects in some wacky way, and I'm like, there's a built-in mechanism, just do that.

Um, so uh that might seem like a fancy thing, but you get into companies and people have those needs, even if they're not writing their own open source project.

The the final thing is I want to I'm guessing I don't want to spend 80% of my time testing or thinking about testing.

I want to probably spend 80 to 90 percent thinking about my problem, and I only want to spend like 10 or 20 percent in the test thought process.

Is that drive with how I can develop Django, or do I need to skew those percentages?

Like, yeah, like like so you you mentioned about testing first, right?

So when you're developing your sort of domain bits inside the view, the bits that are gonna go into the the view layer, and they're gonna be like the view layer is gonna call out to those to do what do its job, then you know you can do test-driven development and you can write nice little tight tight unit tests for those, and you can build them as you go, and that's that's where you're having the fun.

But the actual testing around the outside, Django is well tested, it works.

You can lay down these simple endpoint tests that we've talked about, and they will give you confidence, but they're not they're quick to write and they're not difficult, and they give you a lot of coverage for not much.

Um, okay, good.

You know, I mean it's like like you know, and again, not to uh I'll give you a copy.

Like you just peek at my book, just flip to the testing sections.

But like I show how to test like a basic thing, how to test a crud, how to test something custom.

That kind of gives you 80% of what you're gonna use in a normal setting.

Okay.

And I was like, Oh, you always write the test.

He's like, Well, if I don't write the test, I have to click it in the browser.

Yeah, like like so just write the little endpoint test.

And he was like, just say it's quicker than writing in the clicking it in the browser every time I make an update.

Just yeah, I've got a test, I know it works, and as long I can keep coding as long as I don't break that test.

And then, yeah, once or twice I'll check it in the browser.

Um, I want to

Um I let's just a little bit, I'm joking.

Um, is I think of I think of testing first um when I'm thinking about coding, but I don't always write the test first.

I I just want to make sure that test the idea of how am I gonna make sure it works is in my mind while I'm coding.

Um that's that's my advice I give people is um just think about how I'm how how am I gonna verify that this works.

Um you can do it afterwards, but just make sure you verify it.

That's one of the things I like about coverage because um if I forget, I can run coverage and go, oh, I didn't even cover, I didn't run, I don't test that at all.

So but that's good.

Um I enjoy so your your pot how do I find the podcast you guys at Jang?

It's Django Chat.

Okay, and yours is what's what's testing code?

The and is spelled out, so test and code.com.

Yeah.

And I'm glad I'm so glad uh just as a side note, Carlton has been mentioning that question around fixtures in Py test for like years.

So finally, like I I saw him nodding, so some something new to think about on this issue because people have raised PyTests, and that's always been the bugaboo that you you were burned in some corporate projects on it, and so and I can't help you out on it.

Would that would give

I'm gonna give that a go.

I didn't I didn't know about this, so I'm gonna go and pick pull out a PyTest project and give that a go right now.

Well, um uh so fun talking with both of you, and I wish you all luck.

And um, I'm gonna go get some coffee and wake up.

They they had me wake up at five in the morning to do this.

Can you believe that?

Well, and I'll just end with we are also taking we're taking a little bit of a break until the fall.

We might slip in a special episode, but um we'll be back.

We'll be doing repeat episodes, but Carlton's not saying no.

So this is our last one for a little bit.

All right.

Thanks for coming on,

Okay, bye.