Transcript: Test and Code - Brian Okken

Hi, welcome to a cross episode of Django chat and testing code. This is Will Vincent joined

by Carlton Gibson. Hi, Carlton. And we have Brian Ocken with us the host of testing code.

Hi, Brian. Hello. Thanks for setting this up. So yeah, so we're trying something new.

We we want we both want to talk to each other and we both have podcasts. So we decided to

do a joint podcast and talk about testing PyCon, Django, and do it together and see how it goes.

Yeah, very nice. I actually really like your... Yeah, so I said we're both fans. I was going to

say we're both fans of each other's podcast, but I don't know that about you guys. I'm a fan of

your podcast, at the very least. Well, I listen to your podcast as well, so...

Okay. Well, yeah. Fair enough. Good enough.

So maybe to get out of the way, for our listeners who may not know, you are the author of a book on PyTest, and this is a Test & Code podcast, so that's sort of like the one-sentence description, right?

We want to talk about specifically testing in Python today with you on our side of the fence.

Well, that's great. I didn't know what we were going to talk about, so I'm glad to know now.

so it's awesome um but so will and i met uh will uh so i'm bryanock and uh will vincent uh carlton

gibson um i feel like i know both of you but i have never met carlton i don't think um carlton

yeah i've been on one i'm sure i came on one podcast with you i'm sure i've been you and

michael together at some point no yeah i don't do that i'd maybe we we did we did a talk python

i don't i forget if you were on that brian brian i mean there's so many you know it's

we had a guest on recently and i forgot he'd been on before which is a bit rude of me

so we're at that stage like you know so um but we met the yeah sorry so i just got back from pycon

and you were um you were about to tell me before i stopped you when we started recording

is to is how that that you and uh carlton met somehow or this your your podcast started somehow

with a hallway chat chalk or something.

A hallway tractor.

Yeah, so I'll get some speaking in now

and then Carlton's the real expert

so he can fill in later as we get into the details.

But so I run a site called learnjango.com

and have some books teaching Django.

Carlton just finished five years as a Django fellow.

So one of two people paid to actually work on Django

and do the releases.

So that's kind of the quick and dirty on us.

But we met at DjangoCon 2018 in San Diego,

which was my first Django event, my first tech conference, real tech conference, to be honest,

too. And such a great time. And I was introduced to the concept of the hallway track, because of

course, I was like, I'm going to see all the talks in person. That's what you do. And then I realized

all the poobahs were not in the talks at all. And I was like, that's a little rude. But then I

realized there's this hallway track thing and all these great conversations outside of the main

event and so this podcast came around as an idea of trying to mimic that and share it with others

because i came away saying wow like i had no idea you know how great the community was and all these

things i learned and is there some way to share that with others and i was nice on my side the

genesis and then i asked carlton um if he would be my first guest and he said how about co-host and

here we are so yeah isn't that incredible because um i will let carlton talk eventually

i guess with will and i both have the personality of like talking for like forever around stuff so

um it'll be a miracle but it will get you in but the i did a whole bunch of uh talking with people

at um at pycon just just this last weekend and the um it was interesting you bring that up because

sometimes it's uh awkward and to get started to a podcast starter you get somebody on the podcast

and you and at first you want to like talk about what you're going to talk about and then you you

know to try to get started but in the hallway track or something or just at the at the the floor

um the sponsor floor or whatever uh you can run into somebody and say oh hey you know cool hey

good to meet you um tell me what projects you're working on and you can just start running and you

get a 15-minute conversation in with no stress at all so um yeah i'd love to try so there's still

there's still i'm gonna but there is still a little tension you know because we're all as

programmers often we're introverted and you know there's still that sort of nervousness of

approaching people sometimes oh yeah and also i'm trying to the i remember the first time i

was trying to impress everybody i'm like okay i have to like talk about a project that sounds cool

um so yeah um well so i cut you off but carlton what's uh is that your story too

you're sticking with it or did i make all that up or yeah what was your side of our engagement you

yeah will gave an awesome talk on um authentication in the jungle rest framework because it's quite

complex and you know it's a token auth and you know jwts and session auth and all these different

options he gave a really good talk and so you know i went to his talk um and uh we we started

speaking after that and yeah and then sometime later will said oh you're thinking about doing

a podcast and i said well that sounds really cool but my favorite podcasts are ones where people

kind of chat it's very difficult a solo person just by themselves and so you know i was like

why don't we have you know we'll we'll have this name i think you may even add the domain

django chat or something i was like that's perfect let's just chat about django so nice that was it

so so episodes later yes because you normally do this one it's just you right like i know you and

michael um kennedy have um or how does that all work your your podcast network because you have

a couple great podcast network i guess we do have a network uh right you each have your you each you

have your own and then you do one together isn't that yeah so um he and i were kicking around the

idea individually before we even met of doing podcasting way back when and so he started um

we started i started testing code but it started out as the python test podcast and i just got

tired of saying that um test and code is easier uh although i've been i've had also people tell

me that they they thought the podcast was testing code uh instead of test and code so i've been a

little bit uh conscious lately to try to enunciate so it's test and code um but anyway the and the

testing is first because testing should be first um and then uh oh we can talk about that

uh michael started uh talk python to me around the same time but but i was a little uh i was

a little upset because he was doing it better than i was uh and he was and at the time there

were there was it was just those two uh podcasts for about testing and then there was another one

uh like to podcast and it started around the same time also um but michael and i were both in

portland and um i made a conscious i mean i uh both of us i think individually made a conscious

choice to support each other um and uh promote each other's podcast and everything um you know

two two social networks uh could have a larger audience than one and um and then we were doing

our individual thing for maybe a year um i can't remember the exact time frame but he contacted me

and said hey it'd be kind of fun if we did a podcast together um and uh and of a news format

sort of thing of like a few news items uh every week and uh so we just we tried it and it and i

love the the i love the format it's a it's a really fun thing to be able to we talk about like

four four to six topics with six if we have a guest host on but everybody picks a couple topics

to talk about it it could be any anything um anything python related usually um or sometimes

mostly we thought it'd be news most of the time but sometimes it's just a cool project that we

haven't covered yet and it could be an existing older project and um and a lot of people have

given us feedback that they just uh they just constantly have like a list of things they want

to try out because of our podcast which is nice but the the having another co-host gets you to do

it every week um yeah so uh mine's solo and uh so if it um but i i kind of glad that i have that

flexibility so but i don't do it every week anymore um there were a couple years where i've

tried to do weekly and um uh and some and some of the episodes are um interviewing people and some

of them are just a topic that i want to talk about a little bit and uh kind of and i was inspired a

little bit by the um what there was like a program tea tea time or something there's there's a tea

related uh or coffee break or something related programming podcast that i was listening to

there's just like one dude talking about some topic for 10 minutes uh and i'm like the developer

t yeah say that again maybe developer t yeah that one yeah yeah um so uh but um yeah the motivation

of having a guest or a guest or a sub host or whatever was it co-host that's it co-host

yeah well we were we were guests on um pie bites number i just looked it up uh three years ago so

So that was the connection.

Because I've been, I'd done one with Michael on TalkPython,

and then you and I actually recorded one years ago

that there was like technical difficulties or something.

So, but Carlton, come on, get a word in.

Well, no, no, no, I'm just waiting until we swing around to testing,

because I want to talk about testing.

Okay, well, so I would say, let me sing the praises of Carlton.

Yeah, I want to talk about testing, but just on running a podcast, though,

Like, there's no way I'd be doing this if it was just me still.

I mean, it's so much easier with a co-host to just sometimes you're like, I'm out of it this week.

Like, you need to do the lifting or, hey, like, like Carlton's been great about.

We started off weekly and then we've been going every other week and we take breaks sometimes because kind of the key thing is not to burn ourselves out.

And we we have thousands of listeners, but it's not like our it's not a moneymaker for us.

So we, you know, we want to stay sane at this point.

for me it's frankly a chance to talk to interesting people and like have an organized chat with

carlton like you know everything else is secondary like you know i guess you know marketing our stuff

a bit and all the rest but it's really just nice to talk shop since you know we're all sitting in

our own little caves somewhere you know we're not like i'm not overwhelmed with social stimulation

with work oh that's a that's a good point the the being willing to talk with somebody is like

or having being being okay with asking somebody to talk with you is something that a podcast gives

you that's really pretty cool because i i've reached out to people that i would have never

just reached out in person and said hey you don't know me but uh can we talk for like an hour um

massively valuable time can i just have that please yeah um so but with a podcast it's like

And also, it also, one of the things, lovely things that podcasting taught me is accepting rejection because it's, it's okay. It's, it's okay if somebody says, no, I don't want to do that or I don't have time or something like that. And, and I've realized that I don't feel bad at it at all. It's fine. But what is, so Carlton, you just got done being a Django fellow.

And I guess I'm not, I'm not involved with Django enough yet.

I do a little Django, but I don't know what a fellow is.

Okay. So Django is...

Django just appears and updates itself and that's...

Well, it's, it's quite a big project and it's, it's got a lot of history and it just gives you an idea on the Django issue tracker.

there's um three to five new trick tickets opened every single day so there's like a thousand plus

tickets every year and you know and that's gone on years and years on years um the security reports

that come in pretty much every week and they need triaging they need looking at they need resolving

we do so in 2022 with the security team there were 10 cves handled through django security progress

processors reports for instance there's you know a constant stream of pull requests that need to

be dealt with there's releases monthly there's major releases every eight months there's you

know there's community handling that needs doing and it turns out that on a project the size of

jango's it literally cannot be done just on volunteer effort alone and so how long ago now

i don't know maybe eight years ago nine years ago there was the um they created the jango

fellowship program where people were contracted um initially tim graham and then i came i joined

five years ago marish feliciak four years ago and natalia whose surname i don't know but really

should now um has just started as the new fellow now and that that um having a contractor having

someone who's people who are paid means that the tickets get triaged means that the pull requests

get reviewed means that the releases go out you know on time every month means that the security

patches are done um and i honestly believe that the fellowship program is sort of the reason why

django has been able to survive and not just like and didn't die essentially that they got to that

kind of um fruition of an open source project where the original contributors started to fade

away and why did django not just collapse in on itself and and fade away well because of the

fellowship program and it's i honestly believe it's one of the um you know the reasons why django

is sustainable and reliable and will continue now till probably the heat death of the universe

um and then python's adopting it now too right yeah good point i was just gonna say that was

the the python developing residence program that they've now got is directly inspired by

the success of the django fellowship program okay but apparently we just didn't like the name fellow

um well there's already python fellow so you can be a psf um psf fellow um which naming is hard

right yeah so there's a community award um for contribution contributions to python which is

the psf fellow ships and they're they're awarded quarterly i believe and there's a django has a

Django Software Foundation, and if you're

an individual member of the Django Software

Foundation, that's the equivalent community ward

in the Django community.

So that's

everything I know. And how many fellows

are there at a time?

Well, currently there's two,

and there were two. There was originally one

until I came. I think

in the original pilot there might have been an overlap

with it. Berger did it for

a few months. Yeah, yeah, yeah, for a little while.

I don't know the very dark history that I'd need

to ask others um but tim graham did it for four years by himself which was amazing he did full

time four years by himself i've always said so when tim stepped down um you know there was some

conversation more to carlton do you want to carry on do you want to go full time so i did it part

time for five years i said i didn't really want to do it full time and i didn't want to be the

only fellow because it's really hard like you're on the you're on the sort of front edge of the

fire hose. You know, it's a, it's a really big jet of traffic that comes at you. And

a lot of it, you have to say, sorry, this isn't a valid request, or, you know, we're

not going to add this. You have to close a lot of tickets as won't fix. And people get

very upset when their idea is rejected. And some, sometimes you get it on the nose, you

know, people aren't necessarily meaning to take it out on you, but they do. And so to

have some support is important.

board one of the things that python has is this is well we used to have the bdfl and now we have

the steering council does django have a similar sort of idea yeah so there is a technical board

which was called technical it's now called the steering council it's going to be elections for

the new the new version of that um coming up for the 5.x series now but um if if in django we try

and work by consensus so normally there's a mailing list or the forum where there's a discussion and

if there's not a clear consensus then it can be you can we can ask the steering council to have a

vote and and say what the way it should be that doesn't happen very often we're quite good at

you know reach trying to accommodate and reaching some consensus but sometimes it does have to go

to the story at steering council and they have to just make a decision well if you're a fellow

though like can you do you have enough power to just say um wow i want this cool new feature i

just want to put it in no um when you're leaving you can right carlton you slipped a couple in

i i may have burnt some social capital getting some things through quickly that needed to be

done but what was it there was one in particular there was something that you know was one of those

that this isn't controversial that's why i'm mentioning it there was i have no idea i've not

i have no idea i don't know i've blanked it all it's okay no comment i'm on official detox so i

stopped like two weeks ago so i think i'm in week three of stopping and i and so so the the jango

con europe is coming up at the end of may and i've said i'm not going to look at jango jango and i'm

not going to look at track unless you know if someone ccs me or mentions me then that's fine

but i'm not just going to watch the notification because five years of every comment every commit

every pull request every ticket every you just need to step away and cleanse the the habit of

checking that so i don't really know what's going on in the moment which is lovely uh well i was

just going to add brian so django's history but there were there were two bdfls who stepped down

actually before guido did um but like python django was set up as a non-profit so the fact

that's a non-profit is why there are some volunteers who run a board we could talk about

that i just got off i just also left i spent the last three years as treasurer on the board

running the money. So, um, but that's what funds the fellowship program. And that's kind of how

it's very community-based, which generally works well. It means we go a little slower, but there's

a, there's a churn of people. So when someone gets burned out, like, you know, not burned out,

but five years, Carlton needed a break, like as treasurer three years, I needed a break.

Um, there's other people there and, you know, we're still involved in the system. Like Carlton

just released a, a Django skunk works project, you know, last week. So, um, there's a recycling

that keeps it fresh uh well one of the things we wanted to wanted to talk about was testing a little

bit and i'm really happy that you guys are here because i get a lot of questions about since i

run a testing code wrote a book on testing one of the most popular questions i get is how do i test

my web application um okay and uh i'm not a professional web developer i'm a professional

embedded software engineer so um uh not my wheelhouse really but i uh so my my deflectiveness

sometimes is just you run through the api instead of the uh the front end then but that's not really

that's not really fair um maybe but um but jango's and i uh i was the first time i interviewed you

i didn't have enough experience with jango to to ask decent questions also so there was

technical I think same for me I don't even want to look at my responses yeah so but uh but I have

a um I'm not saying I'm a Django expert yet I'm still learning Django I'm in the I'm a Django

newbie um but I am starting a project and I'm thinking about this of like how do I make sure

it's working right so I'm really glad uh I don't know how to start this conversation other than

how do I how do I start testing with Django and what do I need to think about

i want carlton to take that but i want to just ask you are you so as someone who's you know you

know how to program you're newish to django are you using a starter project have you like how are

you starting are you like because i have i'm yes just yes i'll leave you with that what are your

what's the process for someone like you who wants to start django and wants to know how to do it

properly well oh i'm not sure so i start i did i did a like a starting django and i started the

the tutorial and the tutorials i i read slow and i'm very impatient so i i'm like there's gonna be

a long time to to it reminded me of my first uh first couple months at hewlett packard when i

when i first started out so i was a compute pure computer science person i get hired uh to do um

I was working on satellite testing at first, but then quickly moved into oscilloscopes.

So doing scopes, no, spectrum analyzers.

And one of my managers said, you really should understand electrical engineering and gave me this big Horowitz book.

This huge gray book of, and I'm starting like learning DC circuits and stuff, but I'm working with microwaves.

And I'm like, it's going to be so long before we get to microwaves.

I'll just rather cheat.

And so what I did was I learned the – I was just nice to people at work

and learned the experts in the RF stuff.

And I learned enough electrical engineering to talk to the electrical engineers.

And so I kind of feel like that with Django.

I'm like, I need to find some experts in Django,

and I'll just learn how to talk to them, and that will be good.

Um, so I have a start, I found a starter project.

Uh, there was a, and I don't even remember the name, but somebody that knows what they're

doing and, uh, I'm starting with there.

You're killing me.

Can you tell me who it was?

Well, I'll, I'll, I'll, I'll speak out loud.

So there's the big one is there's one called cookie cutter Django, um, which that's probably

the default that people find.

And then there's a host of slightly smaller ones.

I run one of them.

That's why I was sort of curious if you came across it, but there's, uh, I think there's

a gap in terms of people who are experienced programmers who want scaffolding to kind of jump

in. There's only one really paid starter project for Django, SAS Pegasus that Corey Zhu runs,

which is fantastic. So the sort of market question is, is there room for more? I think he's doing

that pretty much full time. Sorry, I'm getting free market research from you because Django

developers all of us after some period of time have our own personal starter project just like

all the consultant you know the agencies have their own and so it's it's one of these standard

you know you're having a pint and you're like you know what if i charge for my starter project like

how much work could it be you know uh and you know it's a it's the code is the least of it well the

answer that's kind of why i was the answer is probably it's more work than you think and it's

probably good that it's oh for sure yeah yeah but it might be worth it i think there's room i think

that there could be dozens of maybe dozens would be confusing to the marketplace but um but a

handful of uh i could easily see that there's different target audiences too so like you were

saying the sas pegasus um that's a particular kind of thing that's as far as i my brief looking at it

it's um targeting uh paid uh subscription based kind of projects which not all projects are like

that so your project might not be so yeah well that's what i was going to ask you so you said

testing i mean he uh that project has a lot of things in it um it does have you know a lot of

great things the test the payments thing seems to be the one that lets people open up their wallet

as opposed to um sorry carlton i i'm just like i like i like your mba wisdom as you do the market

analysis on this oh god as i sit in a psych ward co you know podcasting booth but anyways yeah so

anyways carlton testing how where should brian start um okay so you should start with django's

test client so Django in Django dot test it has a the module Django dot test it has a client and

what you can do with that client is you can make a web request to a url so you can say get client

dot get you know forward slash home page you know whatever the url the path to your home page is

it might just be just slash it might be nothing it might be just you know the index page and then

it will simulate the actual request and it will it will run the the request through the the web

stack the request response stack it will give you back the response and it will also add a few extra

features on the response so you can test things like the template context what so you can get the

html and you can say did the hbml include my header or you might know that might be a bit hard

if you're looking for a particular fragment of html so you might say was this template rendered

with this context so you can get the which template was rendered which context was it

what was the status code what was um you know did it redirect to a login view for instance so say

you've got an authenticated view and you want to check the authentication works well you'd make a

request to that with an anonymous user and assert that it was redirected to your login page for

instance um so you should use the test card and does that go through does it does it go you have

to have a server running or no that doesn't so you can there's all there's a level up from that

so that's that's the basics that will do most of your cases where really you're not you don't need

the browser behavior it just it talks it runs the test client will sort of use your django app

internally so to speak it will make a request to the django app and get the response and you can

test the html but that's not the browser behavior so there's a level up where if you need like you

know say you've got some javascript on the page and you need to test that the javascript actually

does function then you would need to use a thing called selenium or playwrights the new the new

kid on the block there i'd you know i don't have much to say about which is better there a lot of

People are liking Playwright, but I don't know if the integrations are as mature, is all I can say.

In Django's test, we use Selenium.

There are lots of tests.

And we use that for testing the admin.

So, for instance, in the admin, there's JavaScript to add a new inline-related object.

You've got the admin.

You've got the inline objects.

You want to add a new one.

There's a bit of JavaScript to do that.

Does that work?

Well, we have to use a Selenium test for that because it's all JavaScript logic.

that does need that that will when you run that selenium test django will fire up um a live server

it's called live server test case and it fires up a live server it opens a firefox window or

whatever your selenium driver is using it clicks on the actual buttons as you as you describe in

the test and it runs it so you can do that kind of more um that's like i guess they call it fully

end-to-end testing or something like that you can do that selenium testing but those are difficult

to write and they're more brittle because every time you change your css or you know the position

of a button move sometimes it doesn't the test breaks and you have to to fix those so i would

say save those when you need them but the first port of call is django's test client but like a

workflow so let's say i'm i've got a uh application where um somebody has to log in and it might be

like there's different levels of law like permissions and stuff and i want to think

okay i've got a different kind of person logging in and what what do they have access to does this

workflow work uh going through some stuff can i do all of that with the test client or do i need

to do playwright or something like that pretty much you can do most of it with the test client

like in in most cases you would do so i you know i'm thinking uh you might have a a view where you

start off in the foot and you might you might just you might test a sort of a view a post a form post

view in in a kind of couple of tests together you might break them up into separate functions or you

might just go right first of all make a get request to the page to load the form um then make a post

request with the data then check that you know the the object was created but you can you can

authenticate with the test client so you can give it you know log in with this particular user and

you can either do that by sending it to the login url or it's got a shortcut method where you can

kind of say look log this user in and then we'll do run the rest of the test because

you know there's a lot there's a teeny bit of baller plate going via the login view and then

back to the other view which you might not want might not want or need and then um i'm assuming

i'm hoping that there's like some uh there's enough structure there that we can if i don't

want to do a workflow thing i'm saying well i assume that this kind of a user is already logged

in there and there's already data there and stuff and i want to just test this little thing

is there like i'm assuming there's setup stuff you can do to like yeah yeah so django django

ships and some test cases which are based on unit tests so you you know normally in your test set

you define a setup function and that's where you do your fixtures where you populate the database

with whatever you want and you um you know you said so what is it you you i can't remember the

but you set it up and then you do some actions and then you assert the um what you're looking for

arrange it's a range yeah yeah arrange there you are yeah you arrange what you want in the test

case and then you do some actions and then you assert that it was and it and um the django test

case as test cases provide some nice assertions like assert that a query set is equal to or assert

that, I don't know, a warning message was raised

or a redirect to a certain URL or these kind of things.

It's got a lot, you know, it's quite mature.

So like Python, Django, the default test framework

is a unit test-based thing.

Well, it extends it, yeah.

It extends unit test.

Django's test client extends, yeah.

Whereas I don't think that Python extends it.

It's just raw unit testing.

Just easy.

But the 2021 developer survey from JetBrains and Django

says that only 36% of the people use unit test

and 39% use PyTest.

Yep, so you can do that as well.

That's because everyone's read your book, Ryan.

And I do highly recommend that book as well.

So yeah, you can do that.

so there's a there's a plug-in for pytest called pytest django which gives you lots it gives you

for instance a fixture for the client and it gives you a fixture for um i know the database access

so um pytest django will not allow you to access the database unless you tell it no no no here's a

i think there's a marker or a fixture you can use either but if you use one of them it lets you

access the database um whatever i think there's even a live um server fixture so if you want to

do these kind of selenium tests i believe i'm not 100 sure because by preference i do use um

django's test cases um but myself but i have used i've used pi test plenty and it's it works

just as well that was one of my questions so i'm glad it works it works just as well it probably

has similar sort of the similar test cases built in or something hopefully um it's the pi test the

pi test django package has got the things that you're looking for when you're migrating away from

in the Django test cases.

OK.

But I was just curious if you thought these numbers also

were real, because the Django's developer survey is

like the people that might even know that there's

a JetBrains-based survey out there.

Well, there's back story on the surveys.

this is where this will all sound really incestuous.

So Django had a survey like 2015

and then hadn't had one for years.

Meanwhile, Python has one,

which is very helpful to the community.

So one of the things when I got on the Django board in 2020

is I wanted to do the survey.

So that was one of the things I worked on

was being in charge of that.

So hopefully it happens this year in my absence.

And we partnered with JetBrains.

So we, Django, fly blind because we don't track downloads.

We don't track anything.

Really, how is it being worked?

It's what the fellows see in the issue tracker.

It's what's in the forums, and it's what people talk about in the hallways.

We know that we're massively blind to a lot of things.

So anyway, so the survey was brought about to try to shed some light.

Now, we know that a lot more people use Django than take the survey.

I think it was something like 8,000 people took it last year.

And this is a communication problem that being on the board, Django has, how do people find out about anything about Django? It's like, well, if you go to the Django website, used to be go to the Twitter account. Like there's no email newsletter for Django. Um, and this is a whole separate discussion Carlton and I have had many times, I think even on this podcast, but there's, there's definitely a lack of light being shown in terms of how are people actually using Django.

django i you know anecdotally the numbers i see in the survey match with what i my impression

but you know i'm a american you know going to django cons right i'm not like there's a there's

a just a whole mass of people out there using django in their you know real world situations

who are you i think don't even know that pi test is a thing like they're just it's they go they

download django they're using django django says these are the testing tools they use those testing

tools if they test at all okay i was gonna say i'm just grateful if they test at all um so yeah

well yeah that's the thing if they test at all they're probably i would imagine there's a big

chunk who aren't filling in the the survey that are just using the django tools but within the

developer community i think yeah it probably is representative now one of the other another

question i had was um early on in reading about django and i i'm not sure if this this changed

there was a notion of uh fixtures that were like things that set up the database

is that still the term used or is that still a thing that people do or it is a thing um

so you can you can populate a database and then you can do this you've got this command to dump

data where it will dump it as json um files and then you can load load them again with the load

data command it's quite you know it's got you know all sorts of useful uses um but you can also

specify those those dump data files as fixtures for a test case and it will it will load them in

kind of prior to to you running your test as fixtures yeah i don't think people use that as

their first port of call and i don't really recommend people using that as their first

port of call you're much better off in um set up in setup or set up class data which is a

class method that only runs once rather than once per test um creating the the um the objects you

need you know using the orm or using factory functions that you've you know written to speed

up much like a pi test fixture that you're familiar with you know you create the the fixture function

that does that creates the objects you would do the same but you just instead of having it as a

fixture function which you can't ever find you'd have it in the um the test case sort of just above

the test that's tested and is it is it difficult to like let's say i'm i'm uh getting started with

testing my Django application, but I have a lot,

I mean, Django pulls in a lot of stuff.

I kinda wanna know if I've covered all of the things

I've added, is coverage, do people use coverage?

Does it make sense?

Yeah, no, it works really well.

Yes, there's even a plugin which will help you cover your.

I think Ned maintains, Ned Batchelder who runs coverage,

I think he also runs the coverage Django plugin.

But that will test coverage of your templates, which is quite good, because you might have a little bit of logic in your templates, like a for loop, and then an if, and then an else branch, and, oh, I didn't test the else branch, you know?

Okay.

I think, Brian, for people new to Django but who know what they're doing, one of the issues is you don't need to test core Django thing.

Django comes with all these tests itself.

So how do you know what is a core Django behavior versus what is something you did, right?

So I think sometimes people end up trying to test Django behavior.

So generally, Carlton, you're making a face.

No, no, no, no, no, no.

I was nodding.

I was nodding.

I was lining up, you know, some.

So, I mean, so you said earlier about, you know, test, test first and, you know, test

driven development.

That's a whole discussion.

Carlton and I both come down on the side of kind of write your code and then test it or

better yet when something changes or when something breaks, then you add, then you add

the test.

So that, that's a, you know, sort of an in-between system because anytime, so I, you know, in

my books, I lead, I have tests and I show people how to do this.

And I'm like, even the beginner's book, like, boom, we test, we're testing everything.

When you've changed something, then you add a test.

So just because you see hello world, like, you know, you test, does it show hello world?

You don't need to test that the view got the temp.

I mean, you can, but like, it's only when it's something new.

Like I don't test the admin, for example, like I don't test that the admin works because

it should work because unless i've changed it if i have changed it which which jangle lets you do

then i would run a test to see oh does does this customization uh work as planned so it's when you

customize something you want to test it what do you mean by changing the admin um so don't don't

you doesn't just doing anything change the admin or do you have to like would you mean change how

admin works or well so you can yeah that that's that's a valid point so you can there's built-in

ways to how you display your your data whether you want it kind of how you want it listed you

people can and do abuse the admin so you can kind of use it as a poor man's cms um or maybe there's

valid use cases for it but when you're so there's there's there's changing kind of the the graphical

layout of the admin which django gives you tools to do you don't need to test that but if you're

i mean carlton you've seen it used and abused this is one of the things as a fellow people

people i mean there's all sorts of thoughts on the admin and trying to make it be more than

it is the official line is that the admin is not your application right so use the admin the admin

is a power tool and it's wonderful and it's one of django's secret sources and you know push it to

its limits but people when people are like oh and can i open it up to the world no no no no and can

i customize this thing no just write a custom view it's going to be much less work to write a custom

view than it is to try and force the admin to do this funny thing that it was never designed to do

So there is a sort of, there needs to be a limit to what the admin can do, not least for maintainability sake.

That's actually something that I, one of my first exposures to Django was an internal project, not at the company I'm at now, a previous company, that was the user interface essentially was the admin, at the admin level page.

And I was like, I don't want to build something like that because that's, well, one, I'm talking about like 20-year-old Django probably or 15-year-old Django.

So even that's probably better now than it was 15 years ago, of course.

But still, if I just want to have somebody enter some stuff, I don't want them to have to go through and look at the data tables and stuff.

That's too much.

Well, there's permissions. The problem is Django lets you kind of do a lot and you can have a super user and you can have different access. So you can give different people access to the admin and permissions it up a little bit. And then, just as you said, it's like, oh, do I really need to build my own custom thing for this when it's kind of working well? At some point, yeah, you do. That's kind of how it plays out, right?

like, it's just me, it's me and Carlton up now we're a team of 10. There's some non-technical

people. We really want them to be able to like answer customer support. So we have set on

permission so they can't touch this, but they can enter this, but they're still like, that's when

you get into trouble or you start like saying, Hey, the admin's slow. Cause I'm trying to view

like a million records within the admin. I mean, there's multiple DjangoCon talks about, you know,

using and abusing the admin. So it's a very natural progression. I mean, I, I probably

abuse it too a little bit but it's more like yeah it's easy yeah it happens but you should avoid it

so if i'm so django's big and my application if i do a relatively simple application i'm changing

just a little bit i'm like using all this stuff and i'm not really changing how django works i'm

just um building a website um so the i'm assuming my my approach probably would be test the thing

like the end user experience um the the somebody goes to this page and is that is that when you

said that that's is that testing the views at that point or yeah okay or the models well so the what

does the view do the view is like the controller in the old mvc thing it get it the request comes

into the view and the view's job is to turn that request into a response okay so in order to do

that it probably goes to the orm and it says can i have some some records from the database and

then it grabs a template from the file system and it puts the two together and it creates a um

um it creates a response yeah so test what you want to test you want to test what does that

load so you know for any if you've created a nice view create a a test which is that hits it and

checks that it returns 200 and maybe that the template was correctly rendered or something you

know that so that it it worked because one day you'll i don't know you'll do something and then

you'll just break that view. And that you've got just that test that checked the 200,

that it returned an okay response, will tell you that you broke something.

Now, if your view has complex logic and it might return a 404, so if you go to somebody else's

repo on GitHub, it goes 404. If you haven't got the permission for it, if it's a private repo,

it just says 404, it doesn't exist. So lots of URLs are like that, where if you've got permission

to access the the object it will show you the response 200 and render the template but if you

haven't got permission it will just say oh it didn't exist or it will redirect you or it will

say 403 like forbid forbidden so those kind of different responses that you might get they're

worth testing and then they're quite high level they're quite they're almost integration test

level rather than unit test level but you can spring those around and they give you quite a

lot of coverage and quite a lot of sort of juice for your money, if that makes sense.

And they're quite easy to write. Anyway, go on, Will.

Well, I think it's, yeah, I was going to say, because I've, I mean, like you, a book author,

I've thought about how to present this to people. Because I think tests in a Django context,

people have no idea where to start, in part because they don't understand how the web works,

how Django works, let alone how do I test it. So it's just a lot of confusion. But you kind of

write the same tests over and over and over again. Like tests are super confusing until they're

incredibly boring to the point where like we've like chat GPT and all this stuff, like throw in

that at like, what should I test would be a great use case actually. Cause a lot of times you're

writing the same tests over and over and over again. And so I was, you know, for you or for

someone who's like, what tests do I write? Well, you can look at open source projects. Like if you

go to the awesome Django repo, there's a list of some, see how they do it. You can, if you type in

Django testing tutorial, you'll probably see Django has docs, Mozilla has one, and then I

have one that kind of gives you 80% of all you'll ever need. Like in my books, I show you how to do

it. But I mean, if you're doing CRUD and you're doing forms, you're really, you're like, as

Carlton said, testing the model, testing the view, testing the template, testing URL. Does it return

what I want? Does it not return what's not expected? And you just kind of do that ad infinitum

um at a certain point so just like find an example of something like if it's crud form stuff which

you know 90 whatever percent of the web is you're kind of writing the same test over and over and

over again well one of the things right carlton two seconds i mean you know it depends and then

when you have logic you know at a base level as your logic becomes more complex yes then then

things change and you know you want to test that out but if you're just kind of the the basics of

like does this page display the information i think it does in the way i think it does

like it's kind of the same thing over and over again well i'm i'm gonna take that as a challenge

then because i don't like to write the same thing over and over again okay but then you get to a

another level in django where you for instance you're into um you're dealing with the orm and

so what people quite often do will embed domain logic into manager methods or methods on the orm

themselves so for instance um you know i will quite often say i've got a project a project model

and i don't it's a list of projects and i users are only limited to which projects they'll they

can access you know via an organization or via a team or you know maybe they're the owner and so

on the project model on the what's called the manager which is the objects bit in january you

know project or objects dot filter is the rm api but that manager i'll add a method called for user

and then i can pass in the current user and then you're writing more of a unit test type of

approach where you literally you create a few instances assign some of them to the user create

some for a different user filter the the query set and make sure only the users um uh the projects

are returned okay that kind of thing is is embedding your domain logic right you're checking

that that works and then that that manager you'll then use in your view and you don't necessarily

need to test again, that it was correctly filtered, because you know that the base

query set was filtered to the request user, for instance, just as an example.

Yeah, we should also say, as a guest, and to get some free help, Adam Johnson, who's on the

technical board of Django, has written two books, one of which is Speed Up Your Django Tests,

which I think addresses more of the concerns you have. That's a more, I would say,

intermediate advanced level book on testing and he certainly knows his stuff so um you know

you could do an hour on like how do you not write the same test twice with him and i wanted to ask

i wanted to ask you brian though so you would you are you going to be using pi test because you're

the expert on pi test so will you be using pi test for your for testing your project yeah i mean

okay planning on it yes okay so so i have a real question about pytest and you're the expert and i

really want to ask this it's about managing fixtures because in the in the unit test world

the the fixtures go the the arrange goes in the the setup method of the test case cast and it's

right next to it in pytest they do the dependency injection thing where they sort of the fixtures

live somewhere else and then they get injected and one thing i've found as projects scale is

that it becomes increasingly hard to track down where the fixtures come from so do you have any

kind of advice for you know managing your fixtures to keep them maintainable to keep them trackable

because i end up at the point where i'm like right clicking in the ide and then the to go to

definition and i'm like please take me to the right place because i'm just not sure you know

as it scales up funny if you've got i'm not i'm not a pi test you know i use it but i'm not like

of PyTest aficionado, by choice, I'll use the unit test method.

So I'm asking...

Well, one of the things that I used to complain about...

So there's a couple ways you can do it.

The most obvious way now is the dash-dash fixtures command line flag.

So if you run in, like, either as a whole, you can run, if you run all your test suites, but if you just run as if you were going to run all the test suites, like, or your particular test, and add the dash dash fixtures command line, it tells you all the fixtures that are available and where they're defined.

Okay, that's quite cool.

It used to be a very, a fairly quick list.

It still is.

But you could get more information if you did dash dash fixtures with like verbose, like dash V also.

And the verbose used to be the only way you could get the actual file name and line number where it's defined.

And now by default, it gives you the file name and line number.

So it's really not hard to find them.

But I also, you don't have to put them somewhere else.

You can put them right in the test file.

If you don't want to go hunt for them, don't put them somewhere else.

can put them right there so they're either they're either in the test file or they're in a conf test

file somewhere yeah but yeah okay i mean because one thing i've i did i have done is use test

classes in the like arrange my tests in test classes to keep them grouped together and it's

something that i'm used to and then use the pytest runner you know and it all works the same it works

perfectly um and then i get i sometimes feel a bit embarrassed as if i'm doing it wrong or something

as people i i mean i i there's a lot of fixtures that i write that are only for one test uh yeah

because i because i want them i want the setup to be obvious that it's not part of the test

the other thing that you get is if a failure you can assert within your fixture

and if an assert fails or any any uh with pi test any exception causes a failure but if it's in a

fixture any exception uncaught exception causes an error so that distinction between error and

failure is did my did the the thing that will go wrong happen in the setup or did it happen in the

test itself and that's quite a little that's a good one so would you would you kind of recommend

not not um not over committing to being super dry and that you reuse a fixture every would you

rewrite a fixture would you like to say you've got the same you know you you um different tests

different places in your project you need a project fixture would you would you be tempted

to rewrite the project picture closer to where it's being used to avoid a bit of confusion

no i i'll put it wherever wherever at some central point so you can have a conf test file at all at

any directory um or multiple ones you can only have one pi test any file but you can have multiple

conf test files and the and the conf test can it can go all the way up and the cool thing is you

can start out at a file level and then maybe move it to a directory level if you want to share it

between tests and then maybe you can move it up a directory but the tests don't have to be rewritten

um the the you just move where you've defined the fixture it's it's not um it doesn't have to be

I love that you don't have to rewrite the tests.

With unit tests, you have to change the code.

If you decide you want that fixture logic

to be at the file level or class level,

that's a completely different thing.

Actually, you only have class and function level, right?

I think there's...

Yeah, but you might move it to a shared module or to something like that.

But the other thing I like is you can change the scoping of it.

I mean, so you can say, like, reset the database to be a completely clean, absolutely no changes in it.

But that might be too time-consuming around every function.

So you could broaden it up and say, I want this to only happen at the beginning of the session.

And then if you've gotten bugs all over the place, you can go, well, that was a mistake.

And you can dial that down to like a module level or something.

And the test still doesn't know about it.

The test just says it gets the fixture stuff.

The other thing I really like about fixtures is that the setup and teardown are in the same file or in the same function.

They're just divided by a yield keyword.

Whereas in unit test, you've got the setup and teardown completely separated into two different functions.

yeah no interesting yeah you're good so it's you so the issue there is just i think the answer to

my question is is just maintain hygiene you know because i've been on projects where i've been a

bit like ah where's the where these where's this fixture where's that fixture and i think it was

just got a bit messy over time it's um uh both uh they but it's gotten really a lot better even

just recently like vs code just recently implemented in the last few months implemented

the ability to just right-click on a fixture

and go to the definition.

And then also being able to do type hints in it.

So if they're well type hinted,

like VS Code and PyCharm

will also do code completion for you

with the returned object.

But I probably will get you guys back on

because I've got this project

that I haven't written any tests for

because i haven't actually have a yeah don't have any running code for it yet but um but i

it'll be interesting uh interesting progression to build this thing up and find out like um how

i feel about the jango testing environment and i'm i'm i'm somebody that likes shiny new tools

so i'm sure i'm going to try out like the jango test plus i mean it's got a plus in it it's got

to be good um and um and i've heard people talk about factory boy i'll probably like try that out

and all these sorts of whizzy wig not whizzy wig but whiz bang features things that but i also don't

like too many things because if i pull in a dependency you kind of mentioned all i mean

there's not that many like they're right are there other other i mean there's like maybe one or two

other big ones but there's not a million commonly used third-party things with django but there's

probably a million uncommonly used ones so it's uh but are they but why are they why are they

uncommon though right why are they uncommon i don't know people don't have their own podcast

to plug them on uh one thing i will say is you can use django which i think with django um with

pi test django with the plugin you can use django's test cases just with the pi test runner and it

just works and so you you know you can experiment writing your test in the django way and then you

can mix in fixtures and you can mix in plain asserts because one thing i really love about

pytest is just being able to use plain asserts rather than self.assert and what was it again

because you know so many different asserts it can be hard to remember but there's stuff that like

you can't it's going to be hard to write as an assert so some of the helper functions are

helpful especially around django to say like well assert that the user's logged in or something i

And, you know, it might be easier as an assert, a helper function than it is as code.

Yeah. Well, what you're what you're verbalizing is very common in that, like, I want to try out all the things.

But like, where do I even start? Because Django has third party ecosystem, which is a huge strength.

And you can go to Django packages dot org and see however many thousands and thousands there are.

But like, which ones do you want to use first? Right. So if you went to the testing section, there's a lot.

So there is a curated list, and again, I'm plugging my own stuff again here,

but it's called Awesome Django that I run with actually the person who runs Django Packages now

that has a list of, you know, how many is it?

You know, a dozen, right?

With this idea of trying to say, okay, well, those of us in the community,

there are some packages that have been around for a little while,

have a certain number of stars that are maybe a good first place to look

rather than just what's new and shiny.

Because that is, you know, and there's also things like there's a Django newsletter and stuff to try to both expose people to new things, but also remind people of existing things that are plugging along.

Like Django Debug Toolbar, for example.

Like that's a package that is, you know, I just did a post on like, what do I think are the top, like, I think I said 10 packages, third-party packages you'd use.

Like there's a whole discussion of this.

I think everyone's got that probably top five.

But who maintains it and who uses it?

Like someone started it and then there's this group called jazz band.

Like, so the maintainability and the fact that, um, positive things are still just like

Django itself.

Like there's a tremendous amount of stuff in the most recent release of Django 4.2.

And yet from the outside, people say, Oh, Django is kind of just Django.

Like, right.

Like, I don't know if you had that with your books.

I mean, you're on, you're on the second edition.

I'm on like the seventh edition and I basically rewrite them all from scratch.

And I'm like, it's so much better, but like, everyone's like, Oh, it's just still there.

Right.

So, like, it's hard to get people to appreciate, like, you know, just being still around is a huge sign of success generally for a code repo, for a book, for a project.

But it doesn't seem like that maybe unless you've maintained one yourself.

Well, so the second edition of the PyTest book wasn't really because the PyTest changed too much.

I mean, it had changed.

So I did add new things.

But it was more around the teaching.

So I didn't feel that the first edition was the appropriate ramp of difficulty.

And so the main impetus for rewriting it was I'd like to start people out with the real basics and then introduce fixtures, try to do it gradually.

I think I was a little bit more abrupt than maybe I should be.

But, but that's fixtures come in in chapter three, and then just gradually add on complexity. So that by the end, they're pretty much I mean, if you really understand the whole by test book, I think you're a by test expert at that point. I'm not teaching 100% of by test, but it's, it's the 80% that or the 50% 60% that 80% of the people need every day.

And then some extra fun stuff, like I really want people to write plugins more because it's a great way to, and packages, it's just a packaging, packaging extensions, because that's how they should be sharing code.

Even with, even if I don't do a PyPI package internal to a company, people should be sharing their fixtures with the plugin packaging system because it's there.

why not um but i do i see people trying to share like trying to share fixtures between projects in

some wacky way and i'm like there's a built-in mechanism just do that um so uh that might seem

like a fancy thing but you get into companies and people have those needs even if they're not

writing their own open source project so yeah and if you're the person who knows that then as well

then that's valuable too but the um uh we're kind of getting long but i i do want to come back

because at some point because there's a lot here the the final thing is i want i'm guessing i don't

want to spend 80 of my time testing or thinking about testing i want to probably spend 80 to 90

thinking about my problem and i only want to spend like 10 or 20 in the test thought process

does that drive with how i can develop django or do i need to skew those percentages i'd say that's

fine like yeah like like so you you mentioned about testing first right so when you're developing

your sort of domain bits inside the view the bits that are going to go into the the view layer and

they're going to be like the view layer is going to call out to those to do do its job then you

know you can do test driven development and you can write nice little tight tight unit tests for

those and you can build them as you go and that's that's where you're having the fun but the actual

testing around the outside django is well tested it works you can lay down these simple endpoint

tests that we've talked about and they will give you confidence but they're not they're quick to

write and they're not difficult and okay give you a lot of coverage for not much so people shouldn't

be scared to try testing django projects um this isn't no okay good i mean in my in my well i put

But yeah, I put it like, I don't think I put it in the hello world, but I put it on like the second project that you do, you know, I mean, it's like, like, you know, and again, not to, I'll give you a copy, like you just peek at my book, just flip to the testing sections, but like I show how to test like a basic thing, how to test a CRUD, how to test something custom, that kind of gives you 80% of what you're going to use in a normal setting.

okay when a colleague of mine when i was a junior was very keen a senior colleague was very keen on

writing tests and i was like oh you always write the test he's like well if i don't write the test

i have to click it in the browser yeah like he was like so just write the little endpoint test

and he was like look just say he's quick than writing in clicking it in the browser every time

i make an update just yeah i've got a test i know it works and there's not i can keep coding as long

as i don't break that test and then yeah once or twice i'll check it in the browser we didn't even

talk about ci at all in this conversation you know all that yes we'll have to get back into that

hooking up ci and everything um i want to get up actions i want to clear my name though uh because

i was disparaged earlier um let's just a little bit i'm joking um is i think of i think of testing

first um when i'm thinking about coding but i don't always write the test first i i just want

to make sure that test the idea of how am i going to make sure it works is in my mind while i'm

coding um that's that's my advice i give people is um just think about how how how am i going to

verify that this works um you can do it afterwards but just make sure you verify it that's one of

the things i like about coverage because um if i forget i can run coverage and go oh i didn't even

cover i didn't run i don't test that at all so but that's good um i'm enjoying so you're you're

how do i find the podcast you guys it's django chat so we're django chat.com okay and yours is

what's what's testing code uh test i think it's testing code.com the and is spelled out so

okay there you go yeah well yeah we should we should do this again in the fall when you know

You have a little more experience getting your hands dirty.

And I'm glad, I'm so glad, just as a side note, Carlton has been mentioning that question

around fixtures and PyTest for like years.

So finally, like I saw him nodding.

So something new to think about on this issue, because people have raised PyTest and that's

always been the bugaboo that you were burned in some corporate projects on it.

And so, and I can't help you out on it.

So yeah, well, I mean, people, the pros in your team learned it when there was no Dash

dash fixtures would that would give you no i'm gonna give that a go i didn't i didn't know about

this so i'm gonna go and pull out a pie test project and give that a go right now okay well

um uh so fun talking with both of you and i wish you all luck and um i'm gonna go get some coffee

and wake up they had me wake up at five in the morning to do this can you believe that that's

what it that's what it takes well and i'll just end with we are also taking we're taking a little

of a break until the fall we might slip in a special episode but um we'll be back we'll be

doing repeat episodes but carlton's not saying no so this is our last one for a little bit

all right so i slide that in yeah good to talk to both of you thanks for coming on

okay thanks brian okay bye