Transcript: Django's Async Future - Tom Christie

Hi, and welcome to another episode of Django Chat, a weekly podcast on the Django Web Framework.

I'm Carlton Gibson, joined as ever by Will Vincent. How are you, Will?

I'm great. How are you, Carlton?

I am marvelous. And today we're joined by a very special guest, Tom Christie,

creator of Django REST Framework, and well, you know who he is. Hey, Tom, how are you doing?

Oh, pleasure.

Thanks for coming on.

there's so many things to talk about let's just as a broad structure i gotta ask you about jenga

rest framework and then really want to talk about encode and async and the future of all that um

but maybe before we get to that so how did you learn how to code because you're doing some really

cool almost like bare metal programming type stuff with async and i know you you have a formal

background in computer science is that right? Yes yeah so I started out the way I guess

quite a lot of people who are my age who are in the programming industry did which was writing

games for a ZX Spectrum and doing a bit of you know basic programming there and also on

um on whatever thing you know the the cube basic as well wasn't it that was the the windows dos one

i think it came with snakes and something else gorillas was another one okay both of you

no no gorillas is where you threw the bananas right and you had to throw the throw it up and

hit the other building with the source code so you could go and have a look and then try and

figure out how do i turn that into a flying triangle instead or yeah i thought you're

going to talk about we had nicholas tollerby on talking about the what's carlton what's that

british thing that you all did the bbc the bbc micro did you did you play around with that as

well tom because in the states we didn't have anything like that no no no that was not a thing

But, you know, it was kind of by accident trying to figure out, well, what are you going to do once you get to leaving school age and just ended up doing a computer science degree.

and um i've i've worked in a few different areas kind of since then so i ended up moving into

when i finished my degree i then went and did a master's in computer speech and language processing

it just seemed like an interesting area i was really interested in artificial intelligence

at the time so i went and did that and um then worked in that area of the industry for a while

and then like most people who were on that small master's course with me we eventually all ended

up moving out of speech recognition because it was so it was such it's such a too soon domain

well yeah it's still like in terms of where the money was at yeah it was too soon because it was

right before you know that was all before siri came along and you know a big boom in all of it

but even so i think it's always quite a difficult space to work in a way because it's only really

with the mega corporations and then you have this little research team and it's such a specialized

area that you're working in um personally i found it quite frustrating eventually so

well there is and i'm sorry i was just because i know that um with current options there's

for doing um audio to text one of the few non-megacorps is speechmatics which is i don't

think a megacorp um but it's based in the uk but maybe they're bigger than i think they are

in terms of i don't i don't i don't follow that scene anymore so i played around with transcribing

our podcast and it's like yeah it's like google amazon you know microsoft and then there's like

a uk one and they had a nice drag and drop option not you know configure an api to test it out and

yeah sorry i interrupted please no that's okay um and then ended up

slightly by chance really but ending up working in networking so worked with a content distribution

network um who were doing stuff with peer-to-peer and building caches to live inside isps in order

to save them traffic on all the peer-to-peer stuff that was going through their networks

um did that for a while and then eventually at the same time as i moved to brighton decided okay i

just want to have a complete change of stuff and i would like to be able to again with both of those

things they were really really kind of um but they were very deep technical kind of areas

but i didn't know how to build my own website right or my you know or just to do those sorts

of things and that freedom to be able to okay right i know the basics of how to put stuff together and

build stuff that i can do with and eventually you know build up the skills that you can use to

potentially build a company or do something like that rather than you know the backend stuff and

when i started um yeah when i started kind of working in the web industry uh that was you know

i guess about a year later was when i started working on rest framework right and so did you

find django at that point uh yes i guess that the company that i was doing that i was working with

at the time must have been using it no actually i think they weren't i think they were using they

were using java and that was massively frustrating but yes i had found right so i went looking for

django right to corral them into look we should do more of this stuff because that stuff's really

painful for me well and was the the natural language things was that in python um because

most of it is now um so was python around when you're doing your degree it it certainly wasn't

very big on the scene right and i've struggled to remember at what point i switched over from

bash scripting and pearl to oh wait a minute this is really really helpful like this is much better

i think it probably was yeah i think it i think it probably was after i'd left the speech

recognition stuff because i can remember like some of the more impressive things that i did

when i was there were very complicated sets of bash scripts and pearl strung together because

for all of the um with all the stuff where you're dealing with large corpuses of uh audio and

you know you need to do so much wrangle basic wrangling with them you kind of got that stuff

and then you've got your like very detailed here's my bit of c that right you know actually

does the signal processing and pulls the waveform out into a set of different signals that we can

yeah and that's still the case though right because a lot of the scientific python libraries

are extensions around the underlying c implementations and stuff so it's it's not

necessarily changed at the implementation level even now today possibly so well well that background

actually that makes a lot of sense i was just saying before we recorded your your talk at

jingo con europe this year on async and future of django was fantastic and you did a really good

job of breaking down going a level deeper level deeper on the layers of abstraction that i think

is often assumed because people work at their own um level but given that you've jumped about a whole

bunch of those levels professionally it makes sense how you would be able to wrap it up together

you sort of kept going up the abstraction layers in a way to um you know working with web stuff

but anyways we'll link to that talk i think for for people interested in async your talk and

andrew's godwin's talk which we'll link to from django con this year i think are the

the two places i point people to um so i want to dive into that stuff but did you

what else with your story so you were doing web stuff and at some point you did um django s

framework and then encode right should we finish the professional yeah so what's the origin what's

the drf origin story what's okay yeah sure so what happened was basically there's this one

big motivating factor i have no idea kind of where it came out of but there was the idea was

why on earth is it that out of all these apis that we're building none of them are web browsable

why on earth is it that when we're having to interact with our web apis we're having to do

that from the command line rather from than just being able to point a web browser at it

um http is designed with content negotiation so that you can serve different representations to

different kinds of clients so that a programmatic client could get json but a web browser could

request html instead and i had a couple of ideas about how you would do it because there were

there were things that made it awkward to do so for instance browsers not being able to

post json inside one of their regular forms i mean this was a way back so even where javascript

was was a bit different then as well but even sending non-post requests you know to send a

put or a delete you know the http verbs are available forms themselves don't natively support

it right and i had a couple of ideas about well look here here are different ways that you could

get around that and it seemed to me such an obvious and such a beneficial idea that the web

api should be web browsable that you know i looked into oh can i use one of the existing frameworks

at the time to do this and eventually went no i don't think so let's just you know let's build

let's build a proof of concept first and show that you can do this thing and then it just kind of

snowballed from there you know it was the big like that was kind of always the big motivation

because i thought it was such a you know i thought it was an important idea i ended up wanting to

pile more and more time into improving the framework so i could show people how i thought

we ought to be interacting with apis and the browsable api is still like you know i don't

know how many years later now so seven eight nine years later perhaps the browsable api is still

really cool you know you fire up a new api and you you go to it and it's like oh yeah great and okay

at some point you know you get nested serializers and all the rest it doesn't handle those but

forget up and running it's like oh wow this is super and i don't have to crack open the command

online and i can play around i can create some models and still it's you know still an awesome

thing yeah i mean obviously you know i think there are nicer things that we could be doing with it

but um time and time and effort yes well there's i mean it's a great for i mean i think of a lot

as the admin for django itself where it just works it's fantastic and you get it out of the box and

then if you need more you can use postman or something the same way as you know you shouldn't

force your django admin to do too much but it's just that first time experience of like you can

see it and it works you know that's really really powerful i mean so for example like in my my book

my book um on django apis i mean it's all using the built-in um web views and i say you know you

can go to postman you can do all this other stuff but really most of what you need especially when

you're starting out it's just the first time first time i saw visually those things actually i think

that's the case for many people they they you know because the command line just doesn't do it

yeah well having having the immediacy and the accessibility of it but that's interesting you

almost worked backwards so you you started with that idea rather than saying you know tasty pie

or twisted or the other ones you know the serializers were the issue you almost sort of

started with the end goal there is that correct um yeah yeah yeah definitely yeah yeah it was um

it was tasty pie and piston at the time i used tasty pie i never used piston i should well i

should have done what jasper noah the the author of piston did which is found bit bucket and then

sell that to atlassian for who knows how much that's why he's not on the scene anymore right

Yeah, he's still sitting on a desert island feet up.

So, you know.

Wow.

Well, so Django has frameworks.

So 3.11 is coming out.

Well, it will be out by the time this is released.

It's so feature-complete as is.

What do you see as the future for that project?

And then we'll get into all these other projects.

I can't necessarily think of what more it needs beyond polishing,

but, you know, you two wrote and maintain it.

yeah so at the at the moment i would say it's just in its mature phase now if there if there is

another big jump i think that to me the obvious place would be so we have the we have an admin

like view in there at the moment right but it's not great it's not that great um and i think like

the obvious place that you could put some improvements in would be making it so that

rather than the browsable api defaulting to look like here's a bunch of json and here's some form

fields it's like well here's your tab for the json and here's your tab for i want to see this

data displayed in a table and be able to paginate through it with pagination controls and so on

so yeah if there's another like big jump to happen there i think that's probably where i'd be most

interested in putting time in that's kind of interesting i mean there's also you know there's

also i guess yeah yeah that's that's where i'd be most interested right because there's someone who

users DRF for building projects in the commercial environment have done you know it's how I found

it I was building iOS apps and I needed a back end with an API and a new Django and I was playing

with tasty pie and I was like this isn't working for me and I found DRF and I was like this is

working for me for me still in that still thinking about that environment it's the it's the integration

with the open API tooling the schemas the client generations and all that stuff and that can't be

part of REST framework really,

but integrating with that ecosystem

and making that more fully fleshed out,

I think that that's where the growth is

for over the next short term period at least.

Because I think that's, you know, as a jobbing programmer,

that's what you need, you know, you need your API quickly,

but you also need all these client libraries.

And if you can produce decent docs and, you know,

code examples and, you know, and all that stuff,

The other area that I'm particularly interested in as well is, you know, at the moment we have what you have if you're coming to a new project and you want to build an API, you've got two different slices of option.

and one of your options is you use a completely hosted service something like firebase

or something like that and you build within their service

and you have to be happy to kind of live within the set of constraints that they provided and

know that you're going to stay on their platform indefinitely or you go the other route and you

work with a framework like django or rails now i think you have to host your own really nice space

that's somewhere in the middle which is you know what's the easiest way to get put you know express

it like this like what's the easiest way to get started with django oh well start building your

admin and your api out in this hosted service here and you can put some you know you can start

putting data in and so on and you can start interacting with it as an api and when you're

ready here's this big green let's go button and you hit that and you've got your code and you're

ready to start running that so that's another area that i'm interested in working in so the

other big project that you was around right at the beginning was make docs right which is

which is incredible that it even exists because sphinx was the big beast in the room and you

created this other thing was it is my burning question was it just that you hated restructured

text so much that you wanted to use markdown or was there another no it's just i enjoyed working

with markdown so much um you know in the tour and the tooling for markdown was a lot better you know

as well like there's so many markdown editors and yeah um and support for markdown came to github

before support for restructured text and you know all all these sorts of things um and yeah it's

you know it's a trade-off obviously like you know it's still all these years later it doesn't have

interlinking um just pushed out a package that does um you know you point at your python package

and you can you can inject your uh your doc strings into your you know your markdown

your pros is this a wrap around around docutils uh is it it's um it's a mcdocs equivalent of

uh is it autodoc that right okay this is an equivalent of autodoc but for mcdocs

so it'd be like creating api documentation yes it is yes placing api documentation within

your prose documentation so that came out last month and we're using that so far with httpx

uh i can't remember if i'm using it in any of the other projects yet okay and what what's that

called for our listeners we'll put it in the show notes uh automac docs maybe i can't remember okay

we'll find it we'll find it it's in the show notes i'm using it now so i've forgotten the name

we're just okay abstracted away that's really cool so you've just said the big exciting thing

as well that you've mentioned httpx so tell us what that is uh yeah so actually i'll i'll start

from what was the motivation and then i'll kind of come to where it is now and what it is now so

the gap that was missing was well two things in particular number one how do you you know what's

the async http client for python so you know with requests you're using the thread concurrency model

and you're generally only able well you're only able to open as many outgoing HTTP requests as

you have threads running whereas if you're using the asynchronous model so and that might be you

know say maybe 10 or 20 concurrent requests whereas in the asynchronous model the process

of making a request doesn't block all of the other little async tasks that are off running

concurrently alongside your other one thread of control that you're looking at right now and

you're able to serve thousands of concurrent HTTP requests now the big question was well we've got

requests at the moment but how do we bring async functionality to requests so i've put a bunch of

time into trying to look at what would be a good way of doing that and eventually that's uh evolved

into you know what let's build a complete http client from scratch all the way through because

we need to build one that is async capable up front and center and also you know like we're

in a different landscape now than when requests started so let's start by building an HTTP client

that uses type annotations all the way through so it's very very precise about exactly what the API

and um and another big thing that it started to provide is http2 support right so there's

there's one other kind of halfway incomplete python http client that supports http2

um but there wasn't really anything and fully mature that had that in place so the two big

things were let's look at async support and let's look at http to support um what it's evolved into

is a an http client that is largely requests compatible so wherever possible the api

meets up with requests so i can come along as a long-time request user i can get me import httpx

i can do my get and it will be more or less the same yeah that that supports async uh right now

that means two things so we support async io but we've also just added properly support for the

trio async library as well can we have a little sidestep what's the difference there

async io versus trio what's what's good no but you know in 10 words what's the difference between

those two uh so so trio is an alternate implementation instead of async io and

of like the event loop the big thing is is that uh yes the trio people were bashing me over the

head because it doesn't use event loops or something but um and it had essentially it's

more thoroughly designed it's got this really nice core tenant called structured concurrency

which doesn't just apply to trio but is a general idiom for that um that they've kind of discovered

for look here's a sensible way to deal with concurrency without having callbacks all over

the place without potentially leaking tasks that you forget about having uh you know well-defined

semantics for what happens when tasks are cancelled and so on it's a really thoroughly designed

alternative to async io it's a bit awkward because the two things are completely incompatible

but from the point of view of httpx it's largely irrelevant we've got these two tiny little back

one for asyncio and one for trio and we're able to support both of those uh yeah both of those

different platforms um the plan is we what we did have we did have sync and async support for httpx

momentarily i've dropped out the sync support because we needed to re-engineer the way that

that works previously the the sync code was kind of bridging onto the async code so it would seem

to the user like you're running synchronous code but actually you've got an event loop running

under the hood an async code running underneath and we're actually switching that round so that

instead the sync version of the client will be a pure sync version of the client and the async

will be a pure async one and the folks working on who worked on urllib3 which is the thing that

powers requests have some really smart ideas about how to do that so we've been working on that basis

right super and i was looking i've followed that conversation i was a bit um i was a bit like i

don't drop the sync support because i just want to be able to pull in httpx as my replacement but

having looked at it um async ko has this nice function called run which you can just call from

a synchronous context and you can pass it your coroutine which will be your httpx gets yeah and

it will it will on without you having to pull up your event loop and run the event loop yourself

it'll which is only a few lines but it'll do all that for you so yeah well it's not importantly

the sync support is coming back yeah and i mean i don't know when's the podcast coming out

well when do you want to come out yeah well anyway hopefully hopefully it's already back in

right yeah by the time you listen to this folks sync and async hcd client that additionally

supports blah blah blah right so hang on before we before we lose the thread here you just mentioned

url lib3 now so one thing that um i was sort of under the impression that um the requests um one

of the things about requests is it was built on kind of some older tech which wasn't quite as

robust and then the underlying networking layer has kind of been redone and url lib 3 is part of

that but all of that's a bit vague to me because i haven't dug into it and haven't researched it

could you what's going on there like what's you know url lib 3 is that is that what we need to

be using and is is i don't know what's going on how would you describe that stack has it been

rewritten or is it so the uralib3 team are also working on a new version of that which brings

synchronous and asynchronous support okay and that's the underlying personally what i would

like to do is and i don't think is that difficult is provide the ability for httpx to be used with

that once that's available um i mean we've already implemented a slice of it as well ourselves

but that's the mature battle tested battle scars um you know deals with all these different edge

cases that they've hit in the wild when servers over the course of 10 years ways so whereas we've

got this kind of that this is clean hcb works like this this is what we expect version it would be

really really beneficial to kind of have both those options on the table and be able to figure

out where we're going from there so that's still a work in progress right but that's kind of

underlying level that that layer underlying layer is being yeah yeah they're working in the in the

exact same space as us and a whole bunch of the work that they've done so for example

their approach onto how do you build code that is both sync and async is what we're now

tacking for in httpx because it's smarter than the approach we had before it's going to work

out better for the project well i think one of the things i took away from from your talk as well

tom for people listening who aren't or are new to this space is that async requires rebuilding all

the layers in the stack which is sort of why you know why is the rest framework person working on

http but it's because it you sort of have to rebuild everything for this new async world

uh yes but i you know um that's and that's certainly the motivation but what i hope as well

is um you know as mature and great as requests is i think it'd be really great to have a new

http client that does all the same slice of stuff as it because i'm really pleased with the work

we've done on hcpx like the design is is really nice and i think folks kind of coming to it now

and digging into the code base will be able to get a handle on oh right okay i see how i mean

that's kind of true for requests as well actually but um you know we've kind of been stuck in this

place where there's just this one entity and it's backed onto url of three it would be nice for us

to freshen up and all that again and it's stable right httpx is because it's like 0.8 now is it

it's stable if you pin to no but what i mean is it's stable isn't the right word it's um the word

i mean like it's fully functional fully you i could put it into a project it does everything i

need there's like because the north point what does the north point what's left for a 1.0 then

so yeah you should pin to a median release version so you should pin to 0.8 or 0.9 you know

0.9.x or 0.9.x and the big thing that will still change is once we add sync support back in

then you've got to go well what are we calling the async client and what are we calling the

sync client and what are we calling async requests and what are we calling sync requests

so maybe an import changes or something like that yes you've got a namespace here and you've got to

name it but there's not actually a massive amount more technical work required to get there

um and once it hits 1.0 that's when we'll have the right here's our uh formal deprecation policy

here's what you can expect to happen once we upgrade to version 2 in a year's time or

so on all that stuff so i really want to ask you about starlet um oh cool so let's just assume

And so ASCII, their async web service gateway interface,

we could talk about that, but let's assume that exists.

Starlet's really interesting to me,

and you've also done some work building out,

you've been putting the code,

like a new demo project with it, I believe, right?

So this is, let me tell you what I think it is,

correct me so this is you're building up so lightweight framework using ascii that the

ideas of which could eventually roll into a larger thing like django itself um is that how would you

describe the motivations because i look at it as like it's really educational for me but um

you know you have so many projects that sometimes i'm like wait what does tom work like you've got

like a dozen active projects that are all cool and all i've been trying to keep that more and

more neatly garaged these days but do you i mean what you have i mean django rest framework httpx

asgi um uh uvicorn api star like yeah i mean there's kind of um there's kind of three projects

okay so there's django rest framework there's httpx yeah and there's the async web stack

the async web stack includes well we need an async server so here's uvicorn that's your async

web server we need an async web framework so here's starlet we need a way of interacting

with async database drivers that's at a nice level so here's the databases package

we need an orm eventually i mean so the orm package you know that's kind of more on my horizon

but kind of got the basics in there so there's that bit um and yeah and one or two other bits

and pieces around the periphery there and then carlton what's the schedule for django three

releases for how these things are added in right three one is going to have or three zero has

ascii three one is yeah okay so we so we've just released 3.0 and that that can um run under an

ascii web server so uvicorn or daphne or the other one i can never remember hypercorn um

so you can embed and what's useful about that at this stage is you can embed django next to

your data set application or you know next to your starlet application it's not actually async views

so hopefully 3.1 will have async view capacity and for there you know from from the django

perspective tom's work is like breaking new ground and it's showing the way to go and it's like

it's just amazing and tom's talk from django con europe when it was called sketching out a django

redesign and he spent the entire time talking about starlet not the entire time but you know

basically was talking about starlet but how else do you show what django could become looking at

other than showing what an async web framework looks like.

And that's, you know, from the Django perspective,

it's like, it's amazing that this stuff's here,

the work that Tom's doing and seeing it.

And we can take some ideas

and we won't be able to take it all

because Django's got a lot of history

and it's always going to be,

it's Django and it's got to be backwards compatible.

And, you know, so it's a really exciting area

and there's plenty of room for Starlet

and Django to rob all the good ideas

or as many ideas we can.

and we won't be able to rob them all because Django's Django

and it's got a history and it needs to maintain that.

But yeah, 3.1 for Async Views, which is the bit I'm really excited about.

Something that's been really nice has been being able to see how

once we've got a standardised approach, ASCII,

that Andrew Godwin's designed, we're then able to go off

and I can go off and do a whole bunch of work

in this kind of completely different approach and different space

and he can go off and do right how do we start to bring this into django and they end up being

complementary right so okay and mix and match right yeah yeah so for example like we've got

the baseline functionality in django 3.0 of you can um you can be running in async right at the

very bottom level well what that means i mean i haven't dug into it in detail but you'd be able

to do some neat you know not stuff that you'll need to do all the time but neat ideas like okay

well we have this one particular this one particular case where we want this path to always

just proxy requests off to some other service and you could add in an ascii middleware to let you do

that yeah using http x to make the requests yeah and that's before there's it it just it starts to

give you opportunities but coming back to what will asked earlier actually because you kind of

crystallized the answer to that in my mind you said well what what is starlet i'll describe it

what do you think well actually what starlet is is essentially um how would i start to go about

designing the fundamentals of something like django if i was coming to it fresh now from where

we are now in the landscape exactly that's not to say it's any you know of course it's not remotely

at that level but that's because right let's put these foundations here let's put those foundations

there let's do this bit let's do that bit and um you know because when i especially when i started

working in the async space you know did things like okay well i've got uvicorn and that's built

here but my god you know trying to build some production service on top of that like what are

all the other bits of work that i'd have to do to do that it seems immense and undoable and

i don't want to have to work with these async database drivers it's not my

core expertise i don't understand how to do that but gradually as we fill in more and more of the

blanks um you know you're going to start to see you know a more and more mature framework ecosystem

yeah yeah and it's about providing the the abstraction so you don't want to handle

um asgi send and receive handlers you want a handler takes a request and you send back a

response and you want the framework to do the send and receive to asgi and you don't want to look at

that yeah yeah that kind of thing yes yeah yeah and yeah the way that it's designed is it's kind

of designed to be to be very lightweight but without sacrificing the kinds of abstractions

that you want to be working with at the framework level yeah and one thing you've talked about a lot

is the importance of all this stuff because we talk about async and it's all very you know

but why does it matter how important is this to python and django and other frameworks the space

where it's important particularly is python's position within the landscape right so two of

its biggest competitors in particular would be javascript and go and both of those when compared

to the synchronous python frameworks are able to handle vastly higher levels of throughput

so you know for each server that you're running you're able to handle a certain number of active

Now, that doesn't, you know, for vast swathes of businesses, that doesn't always matter. But for the heavy hitters, it does. And also, you know, a lot of businesses that they're starting out and they're hoping, well, one day we want this to be, you know, and we don't want to have started out in a technology that we then find we're fighting against the current.

with and the intention here is to build up the kind of framework where you've got all of the

wonderful productivity benefits that working with python brings without having to make the case of

it doesn't matter that it's less you know able to handle vast amounts of loads than javascript

without having to make the argument being able to say it's basically equivalent and of course it's

not for go but the goal posts are shifted that much closer that it's not gonna matter to your

team yeah if you can get if you can get 95 performance you know that's probably good enough

so anecdotally here in boston i hear there's a fair number of enterprise java company stacks

that you know they're switching over to new technologies in part because they can't hire

java developers because no one coming out of school knows java and so they're looking at

yeah those three python javascript go as modern current workforce can use um and maybe because

i flew i flew back from pycon on the plane with a whole bunch of these people saying i was like oh

why using python just for recruiting so it's almost right you know like but you know of those

those are the three kind of modern, I need something high performing. Well, and really,

yeah, people would say go and then JavaScript is mangled. So anyways, it's an interesting

perspective. And I think it's right that if you can just check that box, like again, in your talk,

you had great, you showed a bunch of benchmarks and kind of why they're not the standard benchmarks

people trot out don't really matter or are sort of incomplete, right? I mean, Carlton and I often

talk in this podcast about how Python is not what, you know, the programming language is not going to

be the thing that slows down your web stack anyway but even if you're looking language to language

it's easy to look at one of those and think python slow but it's really so yeah so let's talk about

some other aspects of it as well though right because um web socket support right for instance

that's one of the big now django has that with django channels but it's had to be kind of designed

you know into an existence whereas with you have a corn and starlet we're able to start

from a starting point of right we want to be able to handle long-lived connections such as web sockets

or streaming hcp requests now i know there's not necessarily loads and loads of good kind of

tutorial level examples yet of look at the interesting functionality that you can start

to easily build out with using these stacks but that's going to start to come and i think that

when it does and when you start to be able to show look here's a production ready framework

that you can start working with now and um look at those sorts of real-time things that you can

start to build with it i think that will be interesting um and the and the other thing is

forgetting about the async at all but just starlet in its own right i think is really interesting

because the you know having had 10 years of working with django and right how would i approach this

from scratch just the the way that it's stripped everything right down you know it's easy to say

i think that people will kind of get a feel for it when they've been working with it for a bit

but it's a very very low depth of complexity stat right and that's something that i can say

nebulously and is difficult to difficult to quantify but it's a you know i think it's a

really elegant design and i think that as the uh space on top of that starts to mature

that's going to kind of become apparent in the way that it allows you to kind of

uh you know the the framework ends up having a low maintenance cost so you can work on new

things quickly you can adapt new things quickly if you get things wrong it's easy to move stuff

around versus okay cool we're supporting you know versus jango's completely different thing which is

supporting tens of thousands of businesses and here's all the things we're carrying along with

us and we do a fantastic job of continuing to involve and support them um but it's also worth

putting time into great here's this new sprinter on the scene let's work with that for a bit yeah

oh yeah no totally totally totally and like jango's got 10 years of edge cases to

you know to accommodate and to maintain and rightfully so i and i always say one of the

reason why it's still exciting is because we don't break stuff so you know that it's still

going to be there and reliable so no but you can use the new version you can update and you can

use the new features because you don't have you know i always argue that unless you're unless

you're going to really abandon your app don't be on the lts be on the latest major version and get

the new feature and have fun with it i mean yeah you've got to update every nine months but that's

just like taking you taking your car to the garage well so hosted api is the name of the the demo app

that um you built out and i guess i i want to mention to people listening if they're more

beginners jacob captain moss one of the django creators his he had a pycon talk 2017 which we'll

link to on if he were going to build django today from scratch what would he uh redo um because i

think when at least when i try to explain to like i work in a co-working space there's a whole bunch

of people using django who aren't developers so they're managing developers and when i try to

describe async i see their eyes um glaze over and i think part of it is they don't they don't

understand how the piece is what is the stack what is a web framework um so you know what is

whiskey how did all these things stack together so um you know i think of this as an educator i

want to write some courses on async but i think it really has to start with well here's all the

things you were using you didn't have to think about but like here's what the stack is here are

the choices because i i think until you can understand why async beyond a real time sounds

nice you have to see where what you're dealing with and you a lot of people haven't see i think

the thing is the reason that we're having to talk about the technical aspects is because it's still

immature yeah right and what i've been working on really hard is and you know hosted api is just

like kind of the very right let me figure out what we're still missing is until you can get to the

point where you can put your technology in front of people that really don't know an awful lot

about the space and say, hey, look at this cool thing. Isn't this what your business wants to be

able to do or to be able to build or, you know, to have your team work with? You know, and I'm

hoping to be able to get into that much more user facing space. Well, what do you think that example

is right because i think of well maybe it's like what is you know something you build the sync way

and the async way and you can just show it's massively more performant is there like i mean

it's like oh chat seems a little bit faster like what would be a good you know killer example right

like what's the like equivalent of like so a real-time admin right so a real-time admin

plus uh oh and look here's a bit of javascript that we've written on the other side and here's

a website where we've got these various kind of graphs that are updating and we can see

when we put you know either when we're putting data into the admin we can see that stuff updating

over here so kind of what hosted api is no you know maybe one day i'm skirting around you know

bigger plans or here's a simpler example right so um being able to when you when you start up

django right and you get your little welcome screen and then you've added in the admin and

you get the admin like what would be what would be really nice in async framework land would be

once you've started your new project and you go to a screen you can see uh like a real-time

graph of how many requests per second are we running at the moment and it's updating all the

time what's our throughput how many servers are running and currently connected to the same

channel at the moment to be able to see really responsive set of information like that and have

those be in your basic programmers toolbox so you know rather than django debug toolbar is the thing

is the kind of level that we're looking at you know responsive set of information that's coming

back straight away about you know those sorts of things and in the in the phoenix web framework

which is in a lick um which is built on the elixir language which is kind of like a functional thing

built on the early vm they um they've got a thing which is really the new hotness at the moment

called light which they call live view which is exactly this kind of real-time communication back

to the server and they can do you know inline validation send back the response updates in line

and it's it's like ajax but ajax plus plus plus plus plus because it's so quick and you know and

really easy to implement and they've kind of got that nice and be nice if starlet had that be nice

if django get can get there and it's time and you know for me you start you've got you take a

beginner and you run them through and you create a django model and they start and they spin up the

i'm in and they imagine doing that but just with with an async framework where you know they don't

even know about it's async they're just building something and they're and all of a sudden it's

like they're programming and they're learning to program but using this this really cool modern

async stack fantastic yeah

one of the things that is difficult though of course is trying to balance that you know trying

to balance django rest framework with okay i think that the revitalizing the http space in python is

really important so here's httpx with oh i think that the async web space in python is really

important so here's the async web stack right trying to balance those things and trying to make

the case for you know especially with in places where stuff is newer trying to make the case for

yes this is less mature no your business shouldn't necessarily be working on this yet unless you know

that you've got a good reason to but yes we think it's worth investing the time up front on and so

we come to the final and perhaps most important topic is encode oss and your work to build

the tagline what's that the tank sustainable software development right that's that's how you

uh yes yeah it could be well perhaps it was once upon a time but it was collaboratively

funded software development is what it is okay there you go yes that that that that chimes

because like we've we've mentioned what a dozen packages drf starlet h uvicorn you know

httpx how on earth can you do it and you know well yeah and there's a there's a short answer to that

is uh my time is fully funded i work on this stuff full time actually i think it's interesting

going back into the story of how that happened to come about because it's quite lucky really

um you know django rest framework 2 version 2 was when we very first ran a kickstarter for it

and at the time i was working with the web consultancy dab apps and they were really on

board with the idea of running a kickstarter for this so we kind of came to an arrangement where

he said okay we'll do a kickstarter and you know we'll figure out for however much money the

kickstarter makes here's how much of my time can be taken out of client projects and instead just

be put to work on rest framework and we we ran the kickstarter and it was wildly successful

you know for rest framework too i can't remember how many how long ago that was now but it was a

way back because i i it would i got involved with rest framework before version two but it was like

it was coming along and there was the kickstarter and it was it was years ago yeah and i think you

know the timing happens to be just right people could see you know what we wanted to do it was

kind of just about started emerging as you know like the more mature like for a long time tasty

pie had been overall the kind of the the slightly better choice in terms of if you just want to look

at what features have they each got but the timing just happened to be just right and you know we got

like 30 000 pounds from the kickstarter and i worked on rest framework 2 for i can't remember

but you know between six months and eight and nine months somewhere in that area which was fantastic

when we came to okay here's a new slice of work that i think needs to happen and rest framework

3 um again we got pretty lucky with the timing because mozilla had just launched their grants

scheme and i knew that although i wanted to run a big kind of initial fundraiser for it what i

actually really wanted was ongoing funding rather than a single kickstarter style drop

work work work now we're done and it wasn't really obvious to me that leaving my job would be a good

idea or not but because i'd applied to the moss grant the way it went was once that grant you

know once they came back and said yep here you go here's your grant which i think was about 50 i

think was fifty thousand dollars i think it was thirty thousand pounds again so that gave me

this huge chunk of runway you know like best part of a year and i said okay like tab apps is a great

place to work but i can see an opportunity here so i i quit and launched the um the ongoing

funding model and i thought well we'll wait and see how many companies pitch in with this and

what kind of level it gets to and if it's 50 of what i need then at least i'll know you know i'll

have a year's worth of figuring out and maybe i can go back to dab apps and work there you know

half time or whatever but as it happens we've just about got we've just about got enough to

keep me going full time on things and sometimes we've had a bit more and sometimes Carlton's had

contractor time on it sometimes in the past I've supplemented it with a little bit of contracting

and so on but on the whole I'm trying not to do any of that at all and just use the funded time

and and the pitch that I that I always go with if I'm ever on stage talking about it is

um you know i want to work for your company full-time for a hundred dollars a month

because that's that's what the case is that's and it's so clear then why collaboratively funded

open source development makes economic sense right it's like an insurance makes business sense

yeah it's like one of these plans to service your boiler like you know you make a subscription to

keep the the fundamental infrastructure of your business there isn't a there isn't a person

listening to this podcast who doesn't use res framework right you know if if all those people

fund it a little bit of all those companies fund it a little bit then it's totally sustainable on

the ongoing basis yeah i think too it's easier for people to understand that drf is is you and

a small team whereas i mean i'm constantly struck by django is just this amorphous thing even though

it is a half dozen dozen people who really do the yeoman's work um it's a little bit easier in a way

when you can people can see oh it's it's going to tom and his crew right it's not i know it's a

little bit of an easier ask because i think of this in the context of i'm constantly thinking

for the django software foundation i think there's a lot of ways they could get more money which would

help a lot of things um but it's a little bit harder than when you're literally on stage saying

you know I want to work for you for $100 a month and it's just like man if I'm using DRF for

anything as a business that's obviously a total no-brainer so I think yeah I think it's I think

it's really important that there's a very direct link between sponsorships and um the person doing

the work because I think it is a personal relationship I mean you know I I think there's

probably more that we could be doing in that space as well that um you know so for instance

things that i've kind of toyed with recently as well it would be great to get some more to get

somebody else in with a bit of funded time on the project well maybe the right sort of way to

approach that is you you have a a mentee a mentorship program and you specifically ask for

you know we want to run this program and we we need three companies to sponsor it yeah and here's

how much we expect you to buy and then it's a very direct relationship you know they can say

we're sponsoring this program right that person i think that is what they want at a corporate level

I mean, it's like giving to, you know, university. You want your name in the building versus just general fund, right? Everyone wants to point to their specific thing. I mean, because I think, again, in the Django context, I would posit that 95, 99% of people have no idea that a Django fellow exists, let alone that there are two, let alone that Carlton is one of them.

I mean, it's just totally—the DSF, nobody knows what that is because they don't really have to.

So it's—I like your idea of direct relationship from mentee.

I mean, just this year, the DSF with Jacob Kaplan Moss is mentoring someone on adding something to the funding page.

They got 1,000 people who wanted to be mentored.

And I think if you had a company that would sponsor that or something, yeah, that's brilliant.

that makes a lot of sense it's like a badge that a company can talk about and yeah have a as you

said a direct relationship between their funding and what they're getting out of it well it's a

lot easier for me and rest framework to be able to tackle that sort of thing because i can just

be very unilateral in how i want to approach funding right whereas that's more difficult with

a commute you know something where there's a bunch of different people who are all kind of

equally responsible for the health of the project to go to to make very like direct decisions about

you know what let's put the funding up front and center and let's be very direct about this

you know those conversations can be more complicated yeah jango's a big ship it takes

a while to turn you know yeah but and yet it you know if you go to a django con in the u.s or europe

or now africa next year i mean it's still a small number of people who really make it happen which

i mean this is like i i you know being by far the newest to this community of the of the three of us

i keep getting struck by first it's like wow this is amorphous thing and then i'm like oh my god

it's so fragile like i mean carlton's given some talks to this jango jango isn't you know this big

organization there's there's marius there's myself there's half a dozen people who are very active

and then there's you know 50 people who are kind of active and then that's it that's that's it and

you know people will email the django developers saying the django team and it's like well really

that's you if you step up well and it's true in so many places as well right it's you know there

there's a lot of places where there's these core bits of you go in python or any other bit of the

space where there's something absolutely critical and it's one person sitting in their garage

you know every now and again who's working working on it out of love yeah yeah well as we wrap up is

there anything else you want to mention highlight i mean there will be a couple thousand people

listening to this and you know it'll be there for posterity uh no just hi

come back in six months time and i might have something different to say to that question

so we'll see okay great we're gonna have links to all the stuff encode everything uh in the show

notes so please do check out all the packages and stacks that are being worked on i'm also i'm

hoping to go to PyCon in 2020 which is not something I do very often yeah I'm not often

I think yeah over in the States so it would be really wonderful to chat with loads and loads

and loads of people there because I'm like I say I'm not over that way much so and and but you'll

be at um DjangoCon Europe they've publicly announced it's in um Porto Portugal are you

gonna be there i haven't figured that out yet okay sorry to put you on the spot i think i'm

gonna try to cross over the pond uh for the first time i'm really excited to yes no i've been in a

bit of a squish at the moment so okay we can cut that bit out pretty much down okay so tom thank

you so much for coming on the show and listen from us and from the whole django community and

beyond thank you so much for the contributions you give and all the work you've done you're just

you know superhero thank you that's a pleasure yes thank you tom and this was jango chat join

us next time folks