Transcript: Python Tooling - Hynek Schlawack

Hi, this is Will. A quick note that my website, learnjango.com, now supports free tutorials

and premium courses, meaning my three books, Django for Beginners, APIs, and Professionals,

are now available. I'm running a 50% off Black Friday sale until December 1st.

More information in the show notes and later on in the show.

Hi, welcome to another episode of Django Chat, a podcast on the Django web framework. I'm Carlton

Gibson, joined us ever by Will Vinson. Hello, Will.

Hi, Carlton.

Hello, Will.

And this week we've got with us Hinnik Twalek, who's Python core contributor and all-round, well, you know, I'm a big fan.

So, hello, Hinnik. Thank you for joining us.

Thank you for having me.

And I love how everybody has problems to peg me anywhere because I'm so over the place that it's just this dude in Python and we love him.

Yeah, no, that's exactly it because I was going to say you're a Python core team, but you are on the Python core team.

But that's not where you hang out mostly these days, right?

Yeah, I'm not, sadly.

Like, my time is finite.

Are you more on the Go side of things?

Or you're still equally in the Python world?

Or how would you divide that time?

No, no, no, no.

I'm on the Python side.

Okay.

But a vast majority.

Like, we use Go only when we need something that is faster than Python,

which is very rare.

Like, Python is fine for us most of the time.

Or when we need a single binary, which happens more often.

We need a setUID or something like that.

otherwise it's all python okay and so i'm just going to get jumped straight into there because

um you talk about being past faster than python um now there's this whole python 3.13 and free

threading and things like that coming along is that going to make a lot of difference because

i'm quite excited about it i'm like this means i can stick with python i can do my threading my

template rendering in a thread and it won't slow down but what's your view because you're a bit

more closer to the metal i think um i think i'll give you the most senior software engineer answer

that there can be uh it depends so good yeah because let's be honest we've been doing pretty

fine for like 30 years without free threading right so um i am absolutely pro free threading

I love it.

I'm very excited.

I was one of the people advocating for this experiment to start.

But you need a parallelizable problem to take advantage of parallelization, right?

And people also are not great at writing parallel code,

which will probably change because in Python,

we do not have good apis for parallelism with threads because it doesn't make a lot of sense

they are just not great um at some point you just run into the gill so nobody ever took the time to

like build really good abstractions for free threading and this is something i'm really

excited about when we will build that but uh i don't i don't expect any world-changing uh

things happening right now or for most people but i believe for many people it will be life-changing

like when they have computational things right like the data science people i think they will

be much happier than us although their stuff is already usually enforced right on c and uh

it doesn't have to go so we will see but um it is okay and so to be clear it is necessary to

future proof python i think that is true but i don't expect that uh our code suddenly but i'm

gonna be super fast or something it's just we will see it will be an evolution i think it will

be more boring than most people think okay so no no instant new world i was going to ask them what

about the sub interpreters thing because it was all sub interpreters sub interpreters then

suddenly free threading came out of the the blue and then i've hardly heard about sub interpreters

for a while yeah i mean uh eric snow still working on them and talking about them and

And I think they are also exciting and I think they are orthogonal to free threading.

And I think if they ever reach a certain stage and don't get like the whole oxygen pull out of the room, they can also be very interesting in the future.

But they have some limitations, right?

Like you can only exchange certain types of data between sub-interpreters while you can share anything using threads, which is both good and bad.

which which is of course the problem with yeah so there's been this this presentation by yuri

solivanov at last pycon where he talked about this amazing um i don't even remember how the

proper name for that is but it's these are the structures that are uh they don't change but

you can change them in a way that they remember the changes so basically applying patches to

to dictionaries and to lists and there that makes them frozen and you can share them across

barriers like sub-interpreters i think the correct name is stms or something like that

but uh didn't go any anywhere yet it's still not public i think it's in their async pg or in their

hdb report somewhere but you would have to copy it out so i guess my one more question in this

sort of way then we should cut back to the beginning because we didn't even ask you or

ask anything we just jumped straight into performance is do you think then that these

these over the medium term these techniques will keep python competitive with you know the goes

and the rusts of the world i mean i mean there's two questions right uh what is medium term for you

and python right now is the most used programming language in the world right

and the most popular ones like asking if it's competitive is is kind of weird um

i mean but there is this thing at the moment where it's like if you don't write rewrite

everything in rust the world will end and i know that we have that every time there's a new

technology or a new option but yeah you know yeah again i think it's it's important it will

enable new things but it is and i said before when we were like advocating for these changes

we do not have the fantasy right now or the imagination what we could get out of this

because we don't have it right um we cannot imagine from the primitives that we have now

what could be we will have to do we have to try it we have to play with that and then probably

someday out of some company will come some amazing framework that will make everything easier and

everything will just fly and everybody will be happy but um i don't think you can expect like

there's gonna be something in a year or something there's gonna be some nice things um i think async

IO works a little bit better when you have free threading

and things like that.

But it will take

something big that

will add an abstraction that people love

to use to be an actual change.

But again,

it's great for the future.

Alright, I'll ask a question.

So we have a big Google Doc of things

we want to discuss, but I wanted to make sure we ask about

your day job, which is at

VarioMedia? I'm probably saying

that wrong. It's not a real word,

right? Okay. So we have German

companies we call it vario media okay nobody's get offended if you pronounce it english yeah

we are web hosting company we do email hosting we are domain registrar and uh we run a kind of

tight ship with not a very big technical team which informs a lot of the things that i do and

say like i cannot afford having um janky systems that page someone every night or something like

because that someone is always me so i cannot just uh push something to production and hope

someone else will have to deal with that yeah we're only serving the german-speaking market

so uh that's why probably not a lot of your listeners have heard of us and our hosting is

mostly lamp so for the web hosting part yeah so yeah exciting times right now with wordpress

Well, I can't...

If you go hot, I go mad.

You can't leave us there.

Yeah, how is...

Is that a model for, you know,

the Python and Django worlds, you think?

Have a dictator who goes crazy?

What do you mean?

Have a dictator who goes crazy?

Someone's going rogue?

Yeah.

I mean, Guido was apparently smart enough

to just abdicate before he can get crazy, right?

Right, right.

It seems like it just...

Maybe it's almost...

I mean, that's the thing,

is that Matt had a reputation,

deserved or not,

for being relatively sane

and a, you know, good in quotes person.

And so it's...

Yeah, I remember early Masodo

and he was like this,

he was kind of celebrated, right?

Like he took over Tumblr

and integrating things into the Fediverse.

Right.

I mean, given enough time

and enough money and power

and I guess this is the lab experiment playing out.

Yeah, hell hath no fury

like a billionaire scorned

or something like that.

Right, I know.

Well, I mean, it's interesting

in the Django space for, you know, Wagtail, right?

Which has been a big thing for a long time,

but especially as people are starting finally to think

maybe WordPress isn't all it's, you know,

maybe we should look beyond WordPress.

I'm curious to see if that brings more momentum

to things like Wagtail.

What do you, Carlton, you're making a-

The thing with WordPress is that it's PHP

and it's dramatically easier to deploy

than anything that will ever be with python so um you can just drop a bunch of files on a server and

it works and this is why it's successful like uh you cannot win the war against this one on

technical merits like with feature lists or something that's just not gonna work i think

i think there's it's more likely that there's gonna be an open source a new open source fork

of wordpress my sort of experience with it was of trying to sell alternatives was always it was

about plugins and themes it was you know people didn't care whether it was php or python or ruby

or whatever but what they cared about was plugins and themes and and only because i didn't know how

much of a pain it is to run right well i'm i'm curious your so your your day job i mean clearly

it's interesting and challenging to you like what are kind of the things that keep you engaged with

it right because you could be doing we're going to talk about all these open source things that

you're doing um but of all the challenges you could take like this seems to be one that you

seem to you know enjoy and stick with um i'm just curious if you could talk about running a web

hosting kind of company because that's something a little different than most of our guests are

working on um i mean my open source projects are results of my work at the web hosting company

because i don't do open source just for fun like these are projects that i need that i use myself

because I don't have time to maintain software that I'm not using.

So this is all our stuff that we use.

So you can see by that alone that my challenges are quite wide.

Okay, fair enough.

This is probably the reason why I've stayed there for over 20 years

is because thanks to the fact that we are a small team,

I have a lot of different jobs in a way.

I don't think I could work in a company where I just work on one Django application eight hours a day.

So this is not my life.

I have a wide variety.

I mean, there's some Go.

I write web services.

I write async IO code.

Our domain registry thing is done in async IO and works pretty well, actually.

So, yeah, I wear a lot of hats and I can kind of sort of decide myself when I change them.

So there's a lot of freedom.

I mean, there's a lot of pressure and responsibility because, again, if something breaks, it's my fault and I have to fix it myself.

But I have a lot of challenges to choose from.

And, of course, there's this thing that the thing that I do, I am the best at the company.

There's nobody over me.

I cannot get promoted or anything like that.

So I've hit the ceiling here, which is why I have this understanding with the company that they let me work on open source projects and engage in this community.

That was like literally what I told them to either you give me opportunity to learn from others there by serving a community, by having an exchange with the community, or I will have to go somewhere where I can grow professionally.

Well, here we are.

Yeah, that kind of autonomy is deeply rewarding, right?

It is. It is. It is. I'm a big fan for like these slow productivity books from Colin Newport and those things. And it's like this career capital that you can build up. And it is not that I'm working less. It is just that I work more on things that interest me right in the moment, right?

Like, I could take, like, two or three weeks off and transfer our projects to UV and just do this deep research of UV.

I don't, like, if I don't have a deadline, I have time for R&D, basically.

Yeah.

On the other hand, nobody else is doing the R&D.

So this is the other side, right?

Like, what I don't do doesn't happen.

Well, that's sort of the dream, right?

I mean, Carlton, for you, Carlton, right?

like coming off of being a fellow for five years speaking of autonomy you know look at what you and

marius are doing like you're not going and being engineer a thousand at fang right because it just

doesn't translate no so yeah i'm just working on um a single django app eight hours a day

but it's just you but it's just you though but yeah but it's just it but it's me and i get to

choose it and i get to set the the thing and i'm you know it's it's not a 10 year old project yet

it's so you know it's still very much fleshing out very much enjoying but it is you know me

driving the direction and that's the rewarding bit it's yeah fantastic um you mentioned cal

newport in it i've heard you talk about hustle culture and you know yeah um yeah i mean not a

fan right like neither him nor myself like um i mean there's treading that balance between doing

your thing and promoting it and but and just being out there sort of slimily selling it yeah i mean

that's another hustle thing right like hustle culture can mean different things like uh the

thing i usually talk about is that uh just do more do it now do it as fast as possible

and then there's of course this constant hustling and shilling and uh i just don't feel good about

that so i don't do that and uh yeah it might be that's just performance not great for my

coding or art it's just like performance coding and art right if you're spending all your time

talking about how hard you're working how hard are you actually working is like right it's like

can you go when you go up to meetups when i lived in san francisco for a while i tried to go to

meetups and they're great at first but then you realize it's like the same people saying having

the same conversations and everyone who's actually doing something is like hunkered down doing the

work rather than bro i don't even sleep anymore you know i'll say one of the things i found a

little disappointing about the Bay area was realizing that even though everyone was, you

know, a lot of people were, you know, nerds in quotes, they were still super all the negative

male stereotypes. So it wasn't just like a, you don't have to be a jock to fulfill some of those.

Anyways, I think we have a name for that.

Um, yeah. Okay. I want to ask one more question. Then I, I know Carlton has a bunch, but so I,

Well, I have to bring up, you have a 2011 blog post where you talk about, quote, Django's ORM is still a joke and reference that you preferred Flask and Pyramid.

So I have to ask, I know it's 13 years ago, do you still feel that way and maybe provide more context around that?

Okay, the quote is very much informed by Buck and Django that, by the way, is still open and can legally drink soon.

It's not the one that is linked in a post, but...

We'll put it in.

Yeah, I have a fun story.

I have a fun story for you.

So as part of R&D, I was trying out Django to use in our systems,

and it ended very, very poorly.

Mostly, like the key reason is that Django is a great framework

for Greenfield projects, but we do very few of those.

we have this existing architecture

the company

exists for like 25-26

years now

we run a weird database

and I had to

fight Django

a lot more than it was helping me

to make it fit our

infrastructure and what happened

there is that we have a

we have

a database table that

has a composite primary key

ah that's nearly coming in and i'm just make i make the long story short i changed the password

of everybody in our company because the table was the primary key was username and domain

and the orm never told me that it's only gonna have the last primary key you define

so when I

set the password hash

for a certain user

I change it for everybody

in the company

and all of our bots and whatever

so that was a lot of work to fix again

yes I can imagine

I can imagine

once upon a time I forgot a where clause

on a delete query

that went very wrong

for similar reasons

well I didn't forget anything I just didn't know

and Django didn't tell me

You know, so that little negative story about Django from time to time.

By the way, the Google Summer of Code project from this year is going to finally get the composite primary keys moving forward.

That's going in and after, I don't know, 20 years almost.

20, 19 years. The bucket's open for 19 years. I checked today.

Exactly, exactly.

But that ties into, you know, a lot of what we're going to go on and talk about is that Django does things a particular way.

and it's not necessarily the way you might do it from a kind of pure perspective so let's just put

that out there and then go and talk about your open source work because you've done so much and

i do want to get onto it and cover it with the with the listeners i think the thing you're

probably most famous for is the attas package is that fair yeah that's definitely the one with the

most downloads and with the biggest impact on python itself yes so i mean that's the attas for

But for people who don't know, it enables you to define a kind of very, very succinct class.

And it was the direct inspiration for data classes, which I think most people would have known of.

Yes, that's correct.

Like, I mean, adders predates type hints.

So that's a thing.

There's this funny syntax most people have forgotten about, but people back then really loved.

The package name is just ATTR, and the decorator is at adder.s, so adders.

And you define attributes by assigning them

to atter.ib, like atrip.

Hilariously, not everybody got it.

And people just imported those symbols.

And there was like at s class and then x equal ip.

Yeah, so anyway, this was like the first good class building

toolkit for Python.

And it was really popular, too.

At some point, people started asking

to get something like this into the standard library.

I still remember I got the email from Guido when I was at Amsterdam airport,

which is a good coincidence, traveling to PyCon US.

And we met up there with Eric Smith and did the first planning stage of data classes.

Okay. I mean, it's difficult to get something into Python, right?

I mean, like something big like that.

Because normally the talk is always about trimming it down and having less.

Yeah, it's usually not a good idea to put an existing package into the standard library.

like we have this moniker of saying that uh packages go to the standard library to die

and it's just that's a good one so we have this same debate in in django right so python was

is batteries included and i i remember when i first picked up python i came from php while i

picked up python it was just amazing because you know the standard library providers i don't know

most of the things i wanted um and django is very much a batteries included web framework but we

have this same tension between keeping things maintainable keeping them fresh and being

batteries included because if you trim everything away then where do you go you know the example i

always have is a um is a production web server i'd love it if there was a production grade web

server in the standard library so i could just sort of import it and create you know but i

understand why it's not there but in the same time i want it because i want i like that batteries

included philosophy i yeah i kind of miss it in a way so what many people don't quite understand and

that includes also when we talk about packaging like i made a few videos on packaging and i get

like these comments like i wish this would be in python that would be done by python but there is

no such thing as the python there's just a bunch of core developers who are developing c python

myself included, albeit not very active.

And they are not necessarily equipped

to write a production-ready web server.

Right, web server, yeah, right.

They have no interest in solving packaging problems.

I mean, just because you're a Python core developer

doesn't mean that you're capable

or interested in these kind of problems, right?

We cannot just tell them to do that

because nobody's paying them to do that.

So this is just like a discussion

where people just quite understand

what they're asking for

because it's not an abstract concept, right?

It's not like Mr. Python can reallocate developers

to do something different.

It's just a bunch of people with own interests.

Yeah, no, no, no, entirely.

I mean, I guess, do you have any thoughts then on,

because this is the other solution,

is let's just put everything on pip

and then ask people to install it.

But the problem then is

there's an awful lot of things on PyPI.

um and how do you how is it how's a newcomer to the community do you find the right packages the

right reason how am i ever going to go and find at us short of being in the python community and

reading the blogs and hearing it come up how am i going to discover it so many branches that we

could take right now uh obviously they should just listen to your podcast to learn about uh

important things like that yes um so we've had this in basically this the same discussion at a

Core, not Sprint, but Core, what's the name?

Core Summit, Core Summit at Python US.

Language Summit.

Language Summit, yes.

Sorry, I'll start again.

So we've had this topic at a Language Summit

a few years ago when I was there too.

And it was the question,

what kind of things should we put into the standard library and why?

And my answer back then was already

that we should put things there

that make people write better Python,

that leads them to a better design.

For example, data classes,

as much as I'm interested in using address instead,

I understand that data classes are a game changer

in making people write classes

into just screwing around with dictionaries

or, God forbid, named tuples.

And this is the things that should be in a standard library,

things that lead people to writing good Python.

And something, I don't want a web server in Python proper

because I want there to be competing ones.

I mean, there's this new one called Grainian,

which is written in Rust.

So there's no way that could be in CPython.

And now we're talking about it.

If we put a production-ready web server into CPython,

as the name says, it would be written in C,

which is illegal, as we all know now.

and um that's not great right i don't want a um freshly written web server written in c

in my standout library so yeah it was yeah i mean the the example came back to me this week i saw

julia evans who does the web comics the web scene and uh she listed um had a list of points about go

that she liked and one of those points was that it's got a production ready web server in it and

it came back to me i was like ah yes that was that was something i but this is a perfect example

because there is no mr python but there is absolutely uh a google and google needs a

production ready web server so they can just as well put it into the go standard library and write

it in whatever they want to write it right so that's the big difference right that's yeah that's

lovely okay brilliant so swing back to attis can you so i'm i use attis all the time i really like

kit i think it's it's you know as you say it lets you define these little classes and they

so then when you're using them you're not indexing into dictionaries you're not missing keys you get

type you know autocomplete and you get all of the other things there's one thing that you don't

include and you deliberately don't include it i wanted to talk to you about that and that's

validation like data validation and you've got this line in the docs about how data validation

should happen outside your could you just explain that tape because you know it's not there are

are other opinions and i'd love to hear you talk on that yeah so this this harks back to orms man

like we didn't we never finished the topic so this is great no we we're getting to it we're

getting to it no no this this is perfect so um how do i start this so yes uh address does not do

uh validation by itself i mean it does have validators and these kind of things but it is

not primarily meant to do validation because i want my my um my data that i've worked that i've

my business code with i don't want that to do validation i don't want to have any information

about how the code is coming from or whatever like my core domain model i want to have pure

such that i can uh such such that it is in a perfect shape for me to write that uh business

code and there's been an amazing talk at the last um pike escates called the rising sea where the

speaker shows how a problem is radically easier to solve if you have proper data model around that

if you take the first if you first take the step to transform the data that is coming in

and he's using i think some something from code advent and transform it into something you can

work with and then solve the problem and that makes it the code that solves the problem radically

radically simpler and easier to understand and shorter and nicer and everything and this is where

this comes in like um i don't mind the existence of pidentic or whatever like it's popular right

now but i think it is a problem when we use the same classes we use for validation also for our

business logic and this is the same thing with ORMs because these are two kinds of design pressure

from two sides like the ORM is basically your data model it's the tables so your models in your code

are going to reflect that fact I mean you can probably rename fields and do some joints and

whatever but in the end this is called table driven design because because the models that

come out of your tables are defined by the table and in my experience i mean maybe i'm weird but

in my experience you maybe have a say on how these tables look at the beginning but over time you lose

control over the tables one way or another maybe another team is working on that or someone else

is relying on it. This all should

not happen. This is all bad, but it

happens all the time.

Talk to anybody who's running production code

if they can just change

their data model. They probably

can't. So that's

what's coming from the database.

Validation is the same thing

just from the other side. You probably

cannot just change your API,

the JSON structures that are coming in

once you've defined them, right?

So now you have this

really, you have this

two types of design pressure coming from two sides and in the middle is your business code

and business code is supposed to be the pure one the important one it makes decisions if someone

goes to jail or if someone gets uh married or whatever right and uh this code in my opinion

should not be affected by how the json is looking from the outside because someone decided five

years ago it should be so this doesn't mean that for example pidentic is bad absolutely not i mean

people can use it and then transform into the data that they need for their business code but

nobody does that like i've been literally asked some from someone like usually people ask me why

should i use others if there's data classes so this time someone asked me why should i use data

classes if there's pidentic and yeah right this to me is an attractive nuisance because

you you do these two things like you have this data coming uh from the outside and from the

database and it's extra work to shape them so you probably are not gonna transform it and you end up

with this weird code that is affected by how someone decided five years ago how to make the

tables on how to make the API. So this is the thought behind that. And this is why I prefer

other ways to do validation. And I also I feel like you need normal classes too, at some point.

So it's just more layered. Hi, this is Will again. My website learnjango.com is now fully

live and I'm having a Black Friday sale 50% off all three of my books. With the new site,

you get lifetime access and free updates, meaning you'll always have access to the latest version

of everything. I've spent almost a year working on it and managing all the user permissions,

payments, and the rest. Carlton and I will have a future episode taking a deep dive look into our

respective projects this past year. But if you want to support me and the show and get a killer

discount, now is a good time to take advantage of the Black Friday sale, which is good through

December 1st. And the analogy with the Django projects is exactly right, is that you start off,

you've got these nice models, they're really clean, they're tidy, they're expressive, they're

quick and then over time you've got i've got an extra method here i've got an extra field here i

knew it does get more complex and then what people end up doing is creating a kind of

a mapping layer to move from the the table gateway which is the the the django model into

some more business logic type cleaner models or service layer or a you know a layer that's

decoupled from that narrow rm but i mean that's the right way right i i just don't i'm not sure

if it's the most common way well i think a lot of people there's that tension right between yeah

carrying on and struggling with or you know and it's it's what's interesting is when do you make

this this change or when at what point is it worth because that extra step that's what i said

right it's friction like you have this these classes and they almost work so you either you

can make this nice transformation that you just described or you just make it somehow do right

if statement here if statement there and you end up with terrible business code

and but that's the that's the same with any growing software project right it's not dependent

on rms or data mapping classes or you know well not necessarily because you can control

your uh the fringes of your code so so i think like my job is to deal with whatever is coming

from the outside and i consider the database also the the outside and i can transform it

into something i want to use and the same same goes the other way i this is in my power i

can write some code and if i don't do that that's on me and i'm just being lazy and uh i deserve

what i get but there is no tension if i do it from the beginning if from the beginning i say okay

i'm using uh a validation library that just transforms into normal classes on the one side

and on the other side i use repositories or something like sqlc that uh spits out normal

classes and then just control those because then there's no friction to add it right in the other

case you already have something that's almost working and then adding a little bit of clutches

and a little bit of dirty works here and there it's the easier way to add a whole layer of

transformations and it's also slower it's like it's another indirection yeah no yeah it's that

that that notion of an attractive nuisance which i think is quite is the nice metaphor it's like

It pulls you into doing the wrong thing in a way.

I want to ask you about UV and your YouTube channel.

But before we switch, you have a number of open source packages.

Are there any in particular you want to talk about or call out?

I mean, there's too many for us to go through all of them.

But are there any that you're particularly interested in or are working on at the moment beyond adders?

If I have, like, my most underappreciated project, I think, is services.

For many reasons.

For many reasons.

And I understand those reasons.

Okay, well, tell us.

Give us the pitch.

Give us the pitch.

That's the beginning.

It's so hard to pitch, like, especially to Django people, because you're just used that everything is a global variable somewhere.

Like animals.

That was not meant at Django.

I just have a friend that I have this constant discussion

because he does the same thing in other frameworks too

because he's convinced that having singletons in a module

is the better way.

And actually he says if it's good enough for Django,

it's good enough for me.

So fair enough.

So the thing about services,

it is kind of hard to explain in a general way

if the person who's listening

is not already cognizant of the problem it's trying to solve.

I mean, everybody heard of dependency injection

at some point in their life,

and they probably think it's something with Java and XML.

And one of my, actually my second YouTube video

was about how dependency injection

is actually just passing arguments into callables

and by that way, making things testable.

And how you get those things

that you pass into those callables,

there's different ways to do that.

There's dependency injection frameworks that I personally find complicated.

And then there's something that's called service location, which is what services is doing.

And the big picture is that you have a registry where you define how to create certain objects, as in types.

And then you have, let's just use Django as an example.

Then per Django request, you would have a container

which you can use to create those resources,

which can be database connections,

which can be API clients, it can be anything.

And of course, the boon is that at the end,

first the container cleans it up behind you

so you don't have to make sure

that your API clients are being closed

or whatever resource you're using.

And you can easily replace those objects for testing.

you don't have to monkey patch to replace your database which with with django it's a bit more

complicated but there is no an official django services plugin where also james bennett is part

of so i'm sure it's pretty good okay oh yeah i mean so the two things you mentioned there was

your video where you introduced this um we talked mainly about dependency injection but then you

mentioned services at the end and you had a lovely line in there which just when you said it i was

like, oh, yes, is that all side effects, anything that can happen, like you write to a file,

make a network request, make a database thing, all side effects happen through an object

that's passed into the function.

And you say, this is the key bit, right?

Is that instead of having the database connection card coded in there and you do it, you take

that as the reference.

And then if you want to pass in a mock or a test fake or whatever, you can do that very

easily and test that logic separate to, did it actually make the request or did it actually

you know without having to do at mock you know patch yeah yeah um i just wanted to

to quote brandon roads again where monkey patching is software bankruptcy and he's right

i want to say just one more thing it was underappreciated yeah uh and it is like when

i talk to people about services and describe what it does there's two reactions one reaction

is complete blank stare no idea what i'm talking about and the other is like this look of pain and

recognition and and uh like my ex my at this point my estimation is that the average company has like

3.5 implementations of something like services in their company just half-assed because that's

literally what they say yeah we've did the same thing but it's half-assed so you don't say

and uh the problem with spreading is that this is very much a project that you or a package that

you use when you start a project it's a decision that you do when you start something new it is

non-trivial to apply to an existing project like even i don't i don't use services in all my work

projects like i i just keep adding it when i get to work on my project or two or so every now and

then but it is hard to sell because people only need it maybe once per year or something and

yeah the growth is very slow and again you have to have this intellectual curiosity to just

deal with these kind of things which most people don't have because they don't care or i don't know

okay james james yes james bennett who you mentioned there he wrote an essay a long time

ago again about you know the django patterns and he talks about he was arguing against using a

service layer it's about no lean into the active record record notion um and one of his arguments

that he gave there was that you know people talk about needing a service layer in case you need to

change the db that's just never going to happen in real life and it's you know i'd be interested

to get him on again and talk about you know how he feels and his development of um the django

services like um component i mean i'm not gonna tell uh django people what to do and i think

james is on the money when he says that if you are working within a framework like django you

should you should do as the django notes right like it um it doesn't make a lot of sense to use

a fully-featured framework like Django

and then start using it like Flask.

That's not going to go great.

I mean, I had to do that, basically.

That was what I had to do due to my constraints,

and it was absolutely no fun.

I have replaced at this point zero databases

in my applications over the past 20 years,

so I don't think that is a good argument at all.

like a service layer can you use an orm2 i think like it really depends how you exactly define it

for me a service layer is more defined by the fact that it doesn't do any view things in that sense

and what i'm personally leaning into nowadays is that i'm trying to do something kind of called

like a functional DDD.

It is a lot of inspiration from functional programming

where you do a lot of things with immutable data

and it's basically data in, data out

and then you apply the changes,

which is even more testable than everything else

because it's just data, right?

I would not know how to use something like that

with a heavy framework like Django.

Yeah, it's difficult to bring in these ideas.

You bring them in sort of slowly and think about them piece by piece

rather than bolt.

There is no one step.

Oh, yeah, let's just switch to doing it in a totally different manner.

That doesn't exist.

But to me, a service layer is mostly about controlling transactions

and not doing anything with views,

which already makes it more testable

because you don't have to view stuff to test things.

right and it's like this separation of layers it's um yeah and i understand that repositories

for example are also not very popular in uh django for good reasons but i could just drop

words like cqrs or something like that but that's and we would be here all day yeah okay so look one

more thing i want to talk to you about in this kind of software engineering world i guess it's

just you've got this fantastic talk and essay combination on subclassing and subclassing in

python um and perhaps you could give us your talk and then perhaps we could talk um give us your

overview of that and then perhaps we could talk about you know a couple of examples from the

world because django uses subclassing very heavily right you subclass models to create a model you

subclass the admins to make an atom you subclass the forms to make form so you know you probably

have views on that be interesting to just django is a child of its time right so what what are you

gonna do uh yeah i've been very involved with the twisted project which is from a similar time i

think it's slightly older but it had the same problems like everything is template subclassing

everything is subclass and um but at some point they realized that or we i just have this memory

of us sitting bristol at a sprint and eating thai food and watching this uh seminal talk by

Augie Feichler and Nathaniel Manista

and the name was like the end of object

inheritance and the beginning of a new modularity

and it was kind of the talk that in a

Python community kicked off the

change of

thought that maybe we don't have to subclass

everything and

I'm not sure

where I'm going with that I'm just what I wanted to

say is that Django

just started at a certain time so it

has certain patterns

ingrained

and nobody's gonna rewrite the whole thing

right and it probably also be weird to just change the paradigm suddenly i mean probably

would be the right thing but uh this is hard i think this is really really hard okay so let me

let me phrase it let me give you something guy because i quite like the subclassing i think you

know your subclass and model it's fine and i read your essay and i agree with all you know i kind

of go through and i agree all the points i think but you know also yeah if i just take the model

and subclass to if i just take jango's model base class and then subclass it for my model and i'm

not doing crazy inheritance structures i don't it seems kind of okay you know it's it's like i get

all this behavior i get all the the things defined it's not like i'm losing myself in a million

different call chains and what's so bad i mean you know that's a rhetorical question yes uh i mean

obviously again it's all trade-offs right like in a case of models the subclassing is mostly a

mechanical thing to just decide to do it it could just as well be a class decorator and the difference

between a class decorator and a subclass approach like this which i demonstrated in the talk is

you just get everything inherited into your model so your models are very heavy and it depends from

when that's a problem and when it's not a problem

but you're on a slippery slope either way

but I don't find that kind of subclassing as bad

and again, we are in Python

so some things only work when you do subclassing

but the point that I was making in the talk

is that we have now two languages, Rust and Go

that have proven that you don't need subclassing

as a concept to write great software

you might need subclassing in python to achieve certain effects like you have to subclass

exception or something like that like for ontological reasons and there's different

types of subclassing which i also talk about for example specialization it's fine like go

has specialization they are just cheating they call it embedding it's specialization it's nothing

else it's um it just works in a very narrow way and i use subclassing in the same narrow way

in my code but it is something you have to know like it is an active decision it's active design

decision how you deal with those sharp edges while when you use composition you have to take

effort to to break it you know what i mean like um subclass is basically broken by design you get

everything in all the borders are destroyed um you you get a lot of things happening by accident

and a lot of surprising behaviors and if you go crazy like like you say like if you have like this

whole hierarchy diamond shaped what what do i know right then it gets extra hard so there's

many points that you have to take into account and but of course like just subclassing to effect

to achieve a certain effect is not bad nobody's going to jail for that but you have to be

cognizant of the rules and you have to be cognizant on um how to deal with the complexity

and you don't have to do that with with um with uh jesus christ

with the composition yes oh god sorry that's okay no so so it's probably i guess that the

take on is it's like a cold hygiene thing again like it's another one of these method methods of

making sure you don't fall into you know unmaintained or making your code harder to

maintain in the future as you as you move forward and this is the thing when someone asks me for

advice i'm going to tell them if you don't know don't do subclassing because it means that you

cannot make this decision in an informed way so yeah and you give a good framework yeah what once

you have the knowledge once you know what uh madame liskov is uh telling you to do and these

kind of things sure wield a tool but but it's a much sharper tool than composition okay there's

one it's just one bit on the essay i agree with the essays i read through it there's just one bit

of the essay that i do disagree with it's right at the end you you talk about uh or you give this

example template method as being at the part the sort of pattern you really hate but it's something

that um because writing django web views all the time they're all nine they're all the same you

know all crud views 90 of them are crowd views and 90 of the operations of the crowd views that

are identical and django's class-based views are very much template method right you override

the one little bit you need out of the template in order to make it make your the particular view

that you're working on in that time yeah sure why me hate template method so much yeah i hate them

So much, because they create this cyclical dependency

and this indirection through two classes,

and you're jumping up and down the hierarchies,

which is like the worst part about making things unreadable.

I've just recently written an internal project

where I also use template subclassing

because the development experience of using the resulting object is much better.

So again, it's all a trade-off, right?

But I wouldn't build a whole web framework

based on template subclassing.

But it's the views, right?

You've got an algorithm which has got some known steps,

and those known steps, sometimes you need to plug out a little,

well, that's template method, you know.

Yeah, but it probably could have been solved

with a strategy pattern or something like that, you know.

It just depends where you're putting in the code,

but it's hard to talk about this from thin air.

No, okay.

So we've got here in the Google, you

get 300 million monthly downloads

across all of your software projects.

MARTIN SPLITT- Yeah, but it's more

of Python getting incredibly popular suddenly.

Like, I mean, to get into the top 20 of PyPI downloads,

you need like more than 300 million downloads.

Like this is, it's insane.

Yeah, that is insane.

Okay, go on Will.

Okay, so two things.

We want to talk about UV

and I want to ask about your YouTube channel.

So maybe I'll start with that.

So your videos are fantastic.

I mean, I look at them

and I see you're doing a lot of production

and you've got quite a few of them, how did you actually get started, I guess would be the

question? Because lots of people think, oh yeah, YouTube channel, but it's quite a lot of work

and quite a lot of work to do it well, which I would say you're doing. So what was that journey

on starting your channel and then doing real production? Because again, you're not just

one straight take, you've got graphics and cuts and you're doing a really nice job with it.

Yeah, well, thanks for saying that.

I mean, I disagree.

I think my videos are still terrible, but it's glad that my viewers...

The ones I would do would be way worse, right?

I mean, you know, but I think until you've tried to make a video, just seeing even what you've done, I see hours of time in simple things that someone wouldn't if they hadn't done it themselves.

Yeah, you see absolutely correctly.

it is so much i mean starting the channel was the usual programmer's hubris just thinking that's

how hard can it be and um i'm gonna have to tell you the reasons too i started like reason number

one and that did not work at all is that i hope to just get better at speaking english because i

just get to talk more into the camera but uh that doesn't help if you're recording like one video

per month because because i'm so slow at the production because as you said production is

really hard sorry i have to stop you i mean your english is almost flawless i i don't really know

how much more you would improve to uh thank you for saying that i mean it's true but the main

reason was was really uh the implosion of conference invitations like uh giving conference

talks was my main creative outlet in my life because i cannot draw i cannot clap on a beat

or something like that, but I can give conference talks

and I've been told pretty good ones.

And I derived meaning from that too.

Like I told you before, small company.

So if I want to have an impact to more than 20 people,

I have to do something public.

And the interaction with the community

and the invitations, conferences around the world.

I mean, I've been to Japan, I've been to three US,

I've been to Russia when it still wasn't that bad.

It just stopped in 2019, and whenever I talk to conference organizers,

everybody's just complaining how they are just out of money.

So basically this whole bubble just burst for me.

And my other creative outlet, which is writing, has also kind of tanked

because whatever Twitter and Google are doing there right now,

like everybody who has a blog will tell you,

unless you are not on Hacker News' front page, nobody reads this stuff anymore.

So I decided to go where the people are

and YouTube is apparently the second biggest search engine

in the world nowadays,

probably after TikTok, I'm guessing.

I thought you were going to say Google, but yeah.

There's some logical synergies with conference speaking, right?

Because I'm doing like this talking head video stuff,

which is basically giving a talk.

I had a photography side business many years ago

until I realized that having a photography side business

means looking into Photoshop after work,

which I did not enjoy at all after a while.

But I kind of understand how lighting works.

And it still took me way too long

to figure out how to record in my tiny room here.

But yeah, I'm getting somewhere now.

Right, well, it's all about the key lamp

and then the overhead and then the side one.

You've got the shadows working.

I mean, I can tell because I've, like,

yeah, it's not so easy to do.

Yeah, I basically have one big light

on the right top right uh a small light to the left which is basically just uh like a lamp i mean

it's that is for webcams but i just rotated into me talking to it and i have a silver fill disc

under my chin because i have deep set eyes and uh yeah i record using my iphone to an ssd

oh do you use a microphone cameo or what's it what's it called is it cameo or they're

the software for using your iPhone, it's with a C.

No, no, no, I record directly on the phone.

Like the new iPhones, they have like this raw and log recording,

so very high quality, but it's very big files.

So I'm recording directly to SSDs on that.

So I connect the SSD to it.

There's like this crazy contraption that I connect to my phone,

which is also like leading out an HDMI to my iPad,

so I see what I'm doing because it's right next to a wall

because this room is tiny.

But I record the audio directly to my computer.

And this shouldn't have taken more than one afternoon,

but it took me weeks of experimentation,

and Lukasz Langa was helping me a lot,

especially with the audio and everything.

And, yeah, it's a process, yeah.

But learning about it...

I'm sorry to interrupt.

I was going to ask what video editing software are you using?

So I started using DaVinci Resolve,

which is free, but I did not quite warm up to it.

So I switched to Final Cut Pro,

which is like this Apple software.

And I'm really good with Keynote,

which by the way, I'm using for the slidey stuff

in my videos.

So I felt more at home

because DaVinci Resolve really feels

like a Linux application, which it is.

And like in the best and worst possible way,

it's like super powerful,

but I just couldn't get really happy with that.

Like, so I switched to Final Cut Pro after,

I think with my third video or something like that.

So yeah, this is also very expensive, by the way.

So like Final Cut Pro, 300 bucks,

then you spend even more money

on the plugins and everything, right?

So yeah, I couldn't be doing this

without my GitHub sponsors.

And since I'm talking to one right now,

so thanks for your support.

Well, I think you're right about that.

Thank you for turning out all the great stuff.

Yeah, the conference talks, I mean, our friend Jeff Triplett has just launched Django TV to try to put more light on conference talks.

But conference talks, even though they're available on YouTube, you look at the viewing numbers, it's, you know, a couple hundred, maybe a couple thousand max.

Whereas, I mean, your video…

My subclassing talk has 5,000.

I think this is my best talk I've ever given.

Like in every way, the subclassing one, it has 5,000 views.

And so it's like nothing, right?

Yeah, I know.

Like the UV one had 20,000.

Right, yeah.

Okay, well, I know we're coming up on time.

We have to ask about UV.

Separately, I would ask you a million questions about YouTube.

But so UV, right?

So you have, I think, two main videos, right?

Like one asking, is it the answer?

And then a second one saying it is the answer.

But what I was really impressed by is you really go in depth.

I think they're both about 20 minutes.

And, you know, so give us the pitch, right?

But I was really impressed.

You could easily have done just a quick clickbait, you know, short thing.

But you really dove into why it's so great, but also how it could be improved going forward.

So you stand by, I guess, as a question, you stand by the most recent video.

You're still on the UV train, and you

think this is the packaging answer we've

been looking for in Python?

Yeah, definitely.

It is like nothing before.

So yeah, for the foreseeable future,

I think it is the future of Python packaging.

So you want to give me pitch for my videos or for UV?

I wasn't quite sure what you were asking for.

Oh, yeah, I guess I do this sometimes.

I just, Carlton knows, I just talk.

I guess I was just saying people should watch the videos, but I was giving you a compliment

that they're really in-depth, you know, so people should go watch them.

You can't sum them up neatly, but I was, yeah, far more in-depth than the usual videos, right?

If I see something on a topic I'm interested in, usually it's someone just saying, this

is interesting, but you, yeah, I guess we don't have time to dive into all the nuance

that you cover, but you do a good job with it.

Yeah, it's mostly a result of lived experience.

I'm not talking about things that I've read up somewhere,

but I've been part of all these shenanigans

around Python packaging.

And I've been around for a long time.

So I remember when setup tools finally got freed

and was taken over by the community.

And when it was a big deal,

I remember when wheels came to life

and it stopped being a big deal to install a binary extension.

So, of course, that's a gift of age, I guess.

So I have the perspective and the context to talk about these things in this way

and just knowing so many of those problems that they are actually solving.

And I have the humility to say that they are not really solving my problems.

Like I was reasonably happy before, like PDM worked great for me.

UV is much faster, but I can see how important UV is

for the Python community because, again,

at a community-wide scope, I don't care about the speed at all,

which also, like, there's been this NPM blog post

about why NPM should not be rewritten in faster languages.

And I think this is a red herring.

Like, if UV would be just as slow as PDM

or pip it still would be a huge deal because people complained about python packaging being

unclear and janky people things were broken and it was unclear what tool to use when there was a

bunch of 90 solutions like and you have to know which 90 where your 100 are in that spectrum so

to use the right tool these are the things that people are complaining about not that pip is too

slow that was never an issue at least not on a serious thing right and um i don't think this

is possible to achieve in python like we've tried and hatch came close but only by using rust helpers

again like uh i think wrote this pie app thing so some rust or zig or c is inevitable but don't

use cc is illegal now at least in the us we all know that but now like making like modularizing

it would make things more complicated again right and you cannot ship like a packaging tool that has

a binary extension that would just make it that would that doesn't make things worse not better

so um so you think it's the one tool thing that's the real it is the one tool that doesn't break

that is my main point and i think that also resonated most with the audience of my video

when i make making this uh this point and i of course i don't want uv to be the only tool right

I want PDM to exist and do well.

I'm happy to keep pip the default installer

so people can use it if they want to.

And I don't want to convince anyone to use UV.

It's fine.

Just use whatever you want, right?

But I'm really glad that we've just solved

the biggest Python problem we've had,

rivaled only by the GIL.

But I think this is a bigger deal than the GIL

because people run only months later

to using python into threading problems but the initial impression of installing python of using

python of installing an application of starting an application that is something that you have on day

one and that's a huge that's a big huge difference and people are like looking for hair in the soup

which as a czech living in germany two very compliant heavy countries i fully understand

and we also totally need to do risk management like we need to plan for when astral's vc overlords

will pull the rug under us but i wish the conversation were more solution oriented and not

and less like per clutchy like yeah there are there are bad aspects to this like this is

absolutely true but also this is something wonderful that we've been waiting for we've

been begging people to use money to solve this problem and now someone did so now it's our job

basically to make the best out of that i just have one question on on it which i i saw a post this

week um from nolan lawson but he was talking about the he was saying why he's skeptical of

rewriting javascript tools in master languages yeah that's the one i meant same post you're

referring to yes but the point i took from that i thought was quite good was that um the thing with

when a tool is written in python or in his case javascript users can sort of debug it themselves

when they hit a problem they can get in they can understand the the what's going on but if it's

written in a another language a compiled language suddenly that ability to dig into it as is gone

and i wonder if well i wonder if that's a loss so i'm i'm gonna say the word again uh it's a trade

off and okay it absolutely is a loss and there was this um really interesting mastodon thread

started by Jacobian, by your very Jacobian, where he kind of shitposted, kind of lamented

the fact that you used to have to learn C to contribute to Python, then you stopped

having to learn to C to contribute to Python, and now you need to learn Rust.

And that absolutely is a loss.

But the question is, what are we getting for that?

And I have to say, I have not debugged PDM or pip many times.

so for me this has not been a loss in a practical sense and if it helps all those people that have

been so frustrated by python in the past and by the way many of those are in the comments of my

video it's not even funny like how many people are just airing their grievances with python packaging

so there's so much pain that still needs to get out so those people are there and this is helping

them like they are the ones where we say this is the one tool use it and at some point you can still

look at other tools, but for now

with this one, you don't even

need to go to python.org

you just download this one thing

Well, I think we're out of time

we could still talk about so many things

but I feel like this is a

This wasn't even half of my notes, man

I know, we'll have to have you

on again if you'll come

Well, yeah, thanks so much

for indulging all our

silly little questions about this

but it's good to be able to pick your brain

and get your responses yeah thanks for listening for my to my rambling i'm used to script everything

out because of course english is my third language so it's very hard for me to just have coherent

thoughts uh life yeah so which is well we have links so nice to lie to me all the time no no it's

i'm not sure i could have so i speak spanish and catalanx i live in spain i'm not sure i could have

an in-depth technical conversation about

ad hoc, about software engineering

in Spanish or in

Catalan, I'd struggle. So I

think you do phenomenally. Thank you.

And the funny thing is I've had a much harder time to speak

about technical stuff in German or Czech because

it's just not... Because it's not

the language of... Yeah.

Yeah.

Okay. Well, we have links to all these things.

Definitely check out

YouTube channel, personal site.

And again, thank you so much for coming on.

We are at DjangoChat.com

and we'll see everyone next time.

Bye-bye.

Bye.

Bye-bye.

This show was brought to you by LearnDjango.com.

Free tutorials and premium courses

to help you level up and stay informed

of all the latest best practices

building websites with Python and Django.