Transcript: Django's Async Future - Tom Christie (Ep46 Replay)

Hi, and welcome to another episode of Django Chat, a weekly podcast on the Django Web Framework.

I'm Carlton Gibson, joined as ever by Will Vincent. How are you, Will?

I'm great. How are you, Carlton?

I am marvelous. And today we're joined by a very special guest, Tom Christie,

creator of Django REST Framework, and well, you know who he is. Hey, Tom, how are you doing?

Oh, pleasure.

Thanks for coming on.

there's so many things to talk about let's just as a broad structure i gotta ask you about jenga

rest framework and then really want to talk about encode and async and the future of all that um

but maybe before we get to that so how did you learn how to code because you're doing some really

cool almost like bare metal programming type stuff with async and i know you you have a formal

background in computer science is that right? Yes yeah so I started out the way I guess

quite a lot of people who are my age who are in the programming industry did which was writing

games for a ZX Spectrum and doing a bit of you know basic programming there and also on

um on whatever thing you know the the cube basic as well wasn't it that was the the windows dos one

i think it came with snakes and something else gorillas was another one okay both of you

no no gorillas is where you threw the bananas right and you had to throw the throw it up and

hit the other building with the source code so you could go and have a look and then try and

figure out how do i turn that into a flying triangle instead or yeah i thought you're

going to talk about we had nicholas tollerby on talking about the what's carlton what's that

british thing that you all did the bbc the bbc micro did you did you play around with that as

well tom because in the states we didn't have anything like that no no no that was not a thing

But, you know, it was kind of by accident trying to figure out, well, what are you going to do once you get to leaving school age and just ended up doing a computer science degree.

and um i've i've worked in a few different areas kind of since then so i ended up moving into

when i finished my degree i then went and did a master's in computer speech and language processing

it just seemed like an interesting area i was really interested in artificial intelligence

at the time so i went and did that and um then worked in that area of the industry for a while

and then like most people who were on that small master's course with me we eventually all ended

up moving out of speech recognition because it was so it was such it's such a too soon domain

well yeah it's still like in terms of where the money was at yeah it was too soon because it was

right before you know that was all before siri came along and you know a big boom in all of it

but even so i think it's always quite a difficult space to work in a way because it's only really

with the mega corporations and then you have this little research team and it's such a specialized

area that you're working in um personally i found it quite frustrating eventually so

well there is and sorry i just because i know that um with current options there's for doing um

audio to text one of the few non-megacorps is speechmatics which is i don't think a megacorp

um but it's based in the uk but maybe they're bigger than i think they are in terms of i don't

i don't i don't follow that scene anymore so i played around with transcribing our podcast and

it's like yeah it's like google amazon you know microsoft and then there's like a uk one and they

had a nice drag and drop option not you know configure an api to test it out and yeah sorry

i interrupted please no that's okay um and then ended up slightly by chance really but ending up

working in networking so worked with a content distribution network um who were doing stuff with

peer-to-peer and building caches to live inside isps in order to save them traffic

on all the peer-to-peer stuff that was going through their networks

um did that for a while and then eventually at the same time as i moved to brighton

decided okay i just want to have a complete change of stuff and i would like to be able to

again with both of those things they were really really kind of um but they were very deep

technical kind of areas but i didn't know how to build my own website right or my you know or just

do those sorts of things and that freedom to be able to okay right i know the basics of how to

put stuff together and build stuff that i can do with and eventually you know build up the skills

that you can use to potentially build a company

or do something like that rather than, you know.

The backend stuff.

And when I started, yeah,

when I started kind of working in the web industry,

that was, you know, I guess about a year later

was when I started working on REST Framework.

Right, and so did you find Django at that point?

Yes, I guess that the company that I was doing,

that i was working with at the time must have been using it no actually i think they weren't i think

they were using they were using java and that was massively frustrating but yes i had found

right so i went looking for jango right to corral them into look we should do more of this stuff

because that stuff's really painful for me well and was the the natural language things was that

in python um because most of it is now um so was python around when you're doing your degree

it it certainly wasn't very big on the scene right and i've struggled to remember at what point

i switched over from bash scripting and pearl to oh wait a minute this is really really helpful

like this is much better i think it probably was yeah i think it i think it probably was after i'd

left the speech recognition stuff because i can remember like some of the more impressive things

that i did when i was there were very complicated sets of bash scripts and pearl strung together

because for all of the um with all the stuff where you're dealing with large corpuses of uh audio

tagged words you know you need to do so much wrangle basic wrangling with them you kind of

got that stuff and then you've got your like very detailed here's my bit of c that right you know

actually does the signal processing and pulls the waveform out into a set of different signals that

we can yeah and that's still the case though right because a lot of the scientific python

libraries or extensions around the underlying C implementations and stuff. So it's not necessarily

changed at the implementation level even now today. Possibly so. Well, that background actually,

that makes a lot of sense. I was just saying before we recorded your talk at DjangoCon Europe

this year on async and future of Django was fantastic. And you did a really good job of

breaking down, going a level deeper, level deeper on the layers of abstraction that I think is often

assumed because people work at their own level but given that you've jumped about a whole bunch

of those levels professionally it makes sense how you would be able to wrap it up together you sort

of kept going up the abstraction layers in a way to you know working with web stuff but anyways

we'll link to that talk I think for for people interested in async your talk and Andrew's

Godwin's talk which we'll link to from DjangoCon this year I think are the the two places I point

people to um so i want to dive into that stuff but did you what else with your story so you were

doing web stuff and at some point you did um jingo s framework and then encode right should we finish

the professional yeah so what's the origin what's the drf origin story what's okay yeah sure so

what happened was basically there's this one big motivating factor i have no idea kind of where it

came out of but there was the idea was why on earth is it that out of all these apis that we're

building none of them are web browsable why on earth is it that when we're having to interact

with our web apis we're having to do that from the command line rather from than just being able to

point a web browser at it um http is designed with content negotiation so that you can serve

different representations to different kinds of clients so that a programmatic client could get

json but a web browser could request html instead and i had a couple of ideas about how you would do

it because there were there were things that made it awkward to do so for instance browsers not being

able to post json inside one of their regular forms i mean this was a way back so even where

javascript was was a bit different then as well but even sending non-post requests you know to

send a put or a delete you know the http verbs are available forms themselves don't natively support

it right and i had a couple of ideas about well look here here are different ways that you could

get around that and it seemed to me such an obvious and such a beneficial idea that the web

api should be web browsable that you know i looked into oh can i use one of the existing

frameworks at the time to do this and eventually went no i don't think so let's just you know let's

build let's build a proof of concept first and show that you can do this thing and then it just

kind of snowballed from there you know it was the big like that was kind of always the big motivation

because i thought it was such a you know i thought it was an important idea i ended up wanting to pile

more and more time into improving the framework so i could show people how i thought we ought to

be interacting with apis and the browsable api is still like you know i don't know how many years

later now so seven eight nine years later perhaps the browsable api is still really cool you know

you fire up a new api and you you go to it and it's like oh yeah great and okay at some point

you know you get nested serializers and all the rest it doesn't handle those but for get up and

running it's like oh wow this is super and i don't have to crack open the command line and i can play

around i can create some models and still it's you know it's still an awesome thing yeah i mean

obviously you know i think there are nicer things that we could be doing with it but um time and

time and effort yes well there's i mean it's a great for i mean i think of a lot as the admin

for django itself where it just works it's fantastic and you get it out of the box and

then if you need more you can use postman or something same way as you know you shouldn't

force your django admin to do too much but it's just that first time experience of like you can

see it and it works you know that's really really powerful i mean so for example like in my my book

my book um on django apis i mean it's all using the built-in um web views and i say you know you

can go to postman you can do all this other stuff but really most of what you need especially when

you're starting out it's just the first time first time i saw visually those things actually

i think that's the case for many people they they you know because the command line just doesn't do

it yeah well having having the immediacy and the accessibility of it but that's interesting you

almost worked backwards so you you started with that idea rather than saying you know tasty pie

or twisted or the other ones you know the serializers were the issue you almost sort of

started with the end goal there is that correct um yeah yeah yeah definitely yeah yeah it was um

it was tasty pie and piston at the time i used tasty pie i never used piston i should well i

should have done what jasper noah the the author of piston did which is found bit bucket and then

sell that to atlassian for who knows how much that's why he's not on the scene anymore right

Oh, yeah, he's still sitting on a desert island feet up.

So, you know.

Wow.

Well, so Django has frameworks.

So 3.11 is coming out.

Well, it will be out by the time this is released.

It's so feature-complete as is.

What do you see as the future for that project?

And then we'll get into all these other projects.

Like, I can't necessarily think of what more it needs beyond polishing.

But, you know, you two wrote and maintain it.

yeah so at the at the moment i would say it's just in its mature phase now if there if there is

another big jump i think that to me the obvious place would be so we have the we have an admin

like view in there at the moment right but it's not great it's not that great um and i think like

the obvious place that you could put some improvements in would be making it so that

rather than the browsable api defaulting to look like here's a bunch of json and here's some form

fields it's like well here's your tab for the json and here's your tab for i want to see this

data displayed in a table and be able to paginate through it with pagination controls and so on

so yeah if there's another like big jump to happen there i think that's probably where i'd be most

interested in putting time in that's kind of interesting i mean there's also you know there's

also i guess yeah yeah that's that's where i'd be most interested right because there's someone who

users DRF for building projects in the commercial environment have done you know it's how I found

it I was building iOS apps and I needed a back end with an API and a new Django and I was playing

with tasty pie and I was like this isn't working for me and I found DRF and I was like this is

working for me for me still in that still thinking about that environment it's the it's the integration

with the open API tooling the schemas the client generations and all that stuff and that can't be

part of REST framework really,

but integrating with that ecosystem

and making that more fully fleshed out,

I think that that's where the growth is

for over the next short term period at least.

Cause I think that's, you know, as a jobbing programmer,

that's what you need, you know, you need your API quickly,

but you also need all these client libraries.

And if you can produce decent docs and, you know,

code examples and, you know, and all that stuff,

the other area that i'm particularly interested in as well is you know at the moment we have that

what you have if you if you're coming to a new project and you want to build an api

you've got two different slices of option and one of your options is you use a completely

hosted service something like firebase or something like that and you build within their service

um and you have to be happy to kind of live within the set of constraints that they provided and

know that you're going to stay on their platform indefinitely or you go the other route and you

work with a framework like django or rails now i think you have to host your own really nice space

that's somewhere in the middle which is you know what's the easiest way to get put you know express

it like this like what's the easiest way to get started with django oh well start building your

admin and your api out in this hosted service here and you can put some you know you can start

putting data in and so on and you can start interacting with it as an api and when you're

ready here's this big green let's go button and you hit that and you've got your code and you're

ready to start running that so that's another area that i'm interested in working in so the

other big project that you was around right at the beginning was make docs right which is

which is incredible that it even exists because sphinx was the big beast in the room and you

created this other thing was it is my burning question was it just that you hated restructured

text so much that you wanted to use markdown or was there another no it's just i enjoyed working

with markdown so much um you know in the tour and the tooling for markdown was a lot better you know

as well like there's so many markdown editors and yeah um and support for markdown came to github

before support for restructured text and you know all all these sorts of things um and yeah it's

you know it's a trade-off obviously like you know it's still all these years later it doesn't have

interlinking um just pushed out a package that does um you know you point at your python package

and you can you can inject your uh your doc strings into your you know your markdown

your pros is this a wrap around around doc utils uh is it it's um it's a mcdocs equivalent of

uh is it autodoc that right okay this is an equivalent of autodoc but for mcdocs

so it'd be like creating api documentation yes it is yes placing api documentation within

your prose documentation so that came out last month and we're using that so far with httpx

uh i can't remember if i'm using it in any of the other projects yet okay and what what's that

called for our listeners we'll put it in the show notes uh automac docs maybe i can't remember okay

we'll find it we'll find it it's in the show notes i'm using it now so i've forgotten the name

we're just okay abstracted away that's really cool so you've just said the big exciting thing

as well that you've mentioned httpx so tell us what that is uh yeah so actually i'll i'll start

from what was the motivation and then i'll kind of come to where it is now and what it is now so

the gap that was missing was well two things in particular number one how do you you know

what's the async http client for python so you know with requests you're using the thread

concurrency model and you're generally only able well you're only able to open as many

the outgoing HTTP requests as you have threads running. Whereas if you're using the asynchronous

model, so, and that might be, you know, say maybe 10 or 20 concurrent requests. Whereas in the

asynchronous model, the process of making a request doesn't block all of the other little

async tasks that are off running concurrently alongside your other one thread of control that

you're looking at right now and you're able to serve thousands of concurrent HTTP requests

now the big question was well we've got requests at the moment but how do we bring async functionality

to requests so I've put a bunch of time into trying to look at what would be a good way of

doing that and eventually that's uh evolved into you know what let's build a complete http client

from scratch all the way through because we need to build one that is async capable up front and

center and also you know like we're in a different landscape now than when requests started so let's

start by building an HTTP client that uses type annotations all the way through. So it's very,

very precise about exactly what the API is. And another big thing that it's started to provide

is HTTP2 support. So there's one other kind of halfway incomplete Python HTTP client that

supports http2 um but there wasn't really anything and fully mature that had that in place so the two

big things were let's look at async support and let's look at http2 support um what it's evolved

into is a an http client that is largely requests compatible so wherever possible the api meets up

with requests so i can come along as a long-time request user i can get me import httpx i can do

my get and it'll be more or less the same yeah that that supports async uh right now that means

two things so we support async io but we've also just added properly support for the trio

async library as well can we have a little sidestep what's the difference there

async io versus trio what's what's good no but you know in 10 words what's the difference

so so trio is an alternate implementation instead of async io and of like the event loop the big

thing is is that uh yes the trio people were bashing me over the head because it doesn't

use event loops or something but um and it had essentially it's a more thoroughly designed it's

got this really nice core tenets called structured concurrency which doesn't just apply to trio but

is a general idiom for that um that they've kind of discovered for look here's a sensible way

to deal with concurrency without having callbacks all over the place without potentially leaking

tasks that you forget about having uh you know well-defined semantics for what happens when

tasks are cancelled and so on. It's a really thoroughly designed alternative to asyncio.

It's a bit awkward because the two things are completely incompatible, but from the point of

view of httpx it's largely irrelevant. We've got these two tiny little backends, one for asyncio

and one for trio, and we're able to support both of those different platforms.

the plan is we what we did have we did have sync and async support for httpx

momentarily i've dropped out the sync support because we needed to re-engineer the way that

that works previously the the sync code was kind of bridging onto the async code so it would seem

to the user like you're running synchronous code but actually you've got an event loop running

under the hood an async code running underneath and we're actually switching that round so that

instead the sync version of the client will be a pure sync version of the client and the async

will be a pure async one and the folks working on who worked on urllab3 which is the thing that

powers requests and some really smart ideas about how to do that so we've been working on that basis

right super and i was looking i've followed that conversation i was a bit um i was a bit like i

don't drop the sync support because i just want to be able to pull in httpx as my replacement but

having looked at it um async io has this nice function called run which you can just call from

a synchronous context and you can pass it your coroutine which will be your httpx gets yeah and

it will it will on without you having to pull up your event loop and run the event loop yourself

it'll which is only a few lines but it'll do all that for you so yeah well it's not importantly

the sync support is coming back yeah and i mean i don't know when's the podcast coming out

well when do you want to come out yeah well anyway hopefully hopefully it's already back in

right yeah by the time you listen to this folks sync and async that additionally supports blah

right so hang on before we before we lose the thread here you just mentioned url lib 3 now so

one thing that um i was sort of under the impression that um the requests um one of the

things about requests is it was built on kind of some older tech which wasn't quite as robust and

then the underlying networking layer has kind of been redone and url lib 3 is part of that

but all of that's a bit vague to me because i haven't dug into it and haven't researched it

could you what's going on there like what's you know url lib 3 is that is that what we need to

be using and is is i don't know what's going on how would you describe that stack has it been

rewritten or is it so the uralib3 team are also working on a new version of that which brings

synchronous and asynchronous support okay and that's the underlying personally what i would

like to do is and i don't think is that difficult is provide the ability for httpx to be used with

that once that's available um i mean we've already implemented a slice of it as well ourselves

but that's the mature battle tested battle scars um you know deals with all these different edge

cases that they've hit in the wild when service over the course of 10 years ways so whereas we've

got this kind of that this is clean hcb works like this this is what we expect version it would

be really really beneficial to kind of have both those options on the table and be able to figure

out where we're going from there so that's still a work in progress right but that's kind of

underlying level that that layer underlying layer is being yeah yeah they're working in the in the

exact same space as us and a whole bunch of the work that they've done so for example

their approach onto how do you build code that is both sync and async is what we're now

tacking for in httpx because it's smarter than the approach we had before it's going to work

out better for the project well i think one of the things i took away from from your talk as

well tom for people listening who aren't or are new to this space is that async requires rebuilding

all the layers in the stack which is sort of why you know why is the rest framework person

working on http but it's because it you sort of have to rebuild everything for this new async world

yes but i you know um that's and that's certainly the motivation but what i hope as well is

um you know as mature and great as requests is i think it'd be really great to have a new

http clients that does all the same slice of stuff as it because i'm really pleased with the work

that we've done on http x like the design is is really nice and i think folks kind of coming to

it now and digging into the code base will be able to get a handle on oh right okay i see how

i mean that's kind of true for requests as well actually but um you know we've kind of been stuck

in this place where there's just this one entity and it's backed onto url of three it would be nice

for us to freshen up and all that again and it's stable right httpx is because it's like 0.8 now

it's stable if you pin to no but what i mean is it's stable isn't the right word it's um the word

i mean like it's fully functional fully you i could put it into a project it does everything i

need is like because the north point what does the north point what's left for a 1.0 then let's put

it that way the big so yeah you should pin to a median release version so you should pin to

0.8 or 0.9 you know 0.9.x or 0.9.x and the big thing that will still change

is once we add sync support back in then you've got to go well what are we calling the async

client and what are we calling the sync client and what are we calling async requests and what

are we calling sync requests so maybe an import changes or something like yes you've got a name

space here and you've got to name it but there's not actually a massive amount more technical work

required to get there um and once it hits 1.0 that's when we'll have the right here's our

uh formal deprecation policy here's what you can expect to happen once we upgrade to version 2 in

a year's time or so on all that stuff so i really want to ask you about starlet um oh cool so let's

Let's just assume, so ASCII, their async web service gateway

interface, we could talk about that,

but let's assume that exists.

Starlet's really interesting to me,

and you've also done some work building out,

you've been putting the code,

like a new demo project with it, I believe, right?

So this is, let me tell you what I think it is,

correct me so this is you're building up so lightweight framework using ascii that the

ideas of which could eventually roll into a larger thing like django itself um is that how would you

describe the motivations because i look at it as like it's really educational for me but um

you know you have so many projects that sometimes i'm like wait what does tom work like you've got

like a dozen active projects that are all cool and all i've been trying to keep that more and

neatly corralled these days but do you i mean what you have i mean django rest framework httpx

asgi um uh uvicorn api star like yeah i mean there's kind of um there's kind of three projects

okay so there's django rest framework there's httpx yeah and there's the async web stack

the async web stack includes well we need an async server so here's uvicorn that's your async

web server we need an async web framework so here's starlet we need a way of interacting

with async database drivers that's at a nice level so here's the databases package

we need an orm eventually i mean so the orm package you know that's kind of more on my horizon

but kind of got the basics in there so there's that bit um and yeah and one or two other bits

and pieces around the periphery there and then carlton what's the schedule for django three

releases for how these things are added in right three one is going to have or three zero has

ascii three one is yeah okay so we so we've just released 3.0 and that that can um run under an

ascii web server so uvicorn or daphne or the other one i can never remember hypercorn um

so you can embed and what's useful about that at this stage is you can embed django next to

your data set application or you know next to your starlet application it's not actually async views

so hopefully 3.1 will have async view capacity and for there you know from from the django

perspective tom's work is like breaking new ground and it's showing the way to go and it's like

it's just amazing and tom's talk from django con europe when it was called sketching out a django

redesign and he spent the entire time talking about starlet not the entire time but you know

basically was talking about starlet but how else do you show what django could become looking at

other than showing what an async web framework looks like and that's you know from the Django

perspective it's like it's it's amazing that this stuff's here the work that Tom's doing and seeing

it and we can take some ideas and we won't be able to take it all and because Django's got a lot of

history and it's always going to be it's Django and it's got to be backwards compatible and

you know so it's a really exciting area and there's plenty of room for Starlet and

jango to rob all the good ideas or as many ideas we can and we won't be able to rob them all because

jango's jango and it's got a history and it needs it needs to maintain that um yeah 3.1

for async views which is a bit i'm really excited really nice has been being able to see how once

we've got a standardized approach ascii yeah you know andrew godwin's designed we're then able to

go off and i can go off and do a whole bunch of work in this kind of completely different approach

and different space and he can go off and do right how do we start to bring this into django

and they end up being complementary right so okay and mix and match right you can yeah yeah so for

example like we've got the baseline functionality in django 3.0 of you can um you can be running in

async right at the very bottom level well what that means i mean i haven't dug into it in detail

but you'd be able to do some neat you know not stuff that you'll need to do all the time but

neat ideas like okay well we have this one particular this one particular case where

we want this path to always just proxy requests off to some other service

and you could add in an ascii middleware to let you do that yeah using http x to make the requests

yeah and that's before there's it it just it starts to give you opportunities but coming back

to what will asked earlier actually because you kind of crystallized the answer to that in my mind

you said well what what is starlet i'll describe it what do you think well actually what starlet is

is essentially um how would i start to go about designing the fundamentals of something like

django if i was coming to it fresh now from where we are now in the landscape exactly that's not to

say it's any you know of course it's not remotely at that level but that's because right let's put

these foundations here let's put those foundations there let's do this bit let's do that bit and

you know because when i especially when i started working in the async space you know did things

like okay well i've got uvicorn and that's built here but my god you know trying to build some

production service on top of that like what are all the other bits of work that i'd have to do

to do that it seems immense and undoable and i don't want to have to work with these async

database drivers it's not my core expertise i don't understand how to do that but gradually

as we fill in more and more of the blanks um you know you're going to start to see you know a more

and more mature framework ecosystem yeah yeah and it's about providing the the abstraction so you

don't want to handle um asgi send and receive handlers you want a handler takes a request and

you send back a response and you want the framework to do the send and receive to asgi and you don't

want to look at that yeah yeah that kind of thing yes yeah yeah and yeah the way that it's designed

is it's kind of designed to be to be very lightweight but without sacrificing the kinds

of abstractions that you want to be working with at the framework level yeah and one thing you've

talked about a lot is the importance of all this stuff because we talk about async and it's all

very you know but why does it matter how important is this to python and django and other frameworks

the space where it's important particularly is python's position within the landscape

right so two of its biggest competitors in particular would be javascript and go

and both of those when compared to the synchronous python frameworks are able to handle vastly higher

levels of throughput so you know for each server that you're running you're able to handle a

number of active users now that doesn't you know for vast swathes of businesses that doesn't always

matter but for the heavy hitters it does and and also you know a lot of businesses that they're

starting out and they're hoping well one day we want this to be you know and we don't want to

have started out in a technology that we then find we're fighting against the current with

and the intention here is to build up the kind of framework where you've got all of the wonderful

productivity benefits that working with python brings without having to make the case of it

doesn't matter that it's less you know able to handle vast amounts of loads than javascript

without having to make the argument being able to say it's basically equivalent and of course it's

not for go but the goal posts are shifted that much closer that it's not gonna matter to your

team yeah if you can get if you can get 95 performance you know that's probably good enough

so anecdotally here in boston i hear there's a fair number of enterprise java company stacks

that you know they're switching over to new technologies in part because they can't hire

java developers because no one coming out of school knows java and so they're looking at

yeah those three python javascript go as modern current workforce can use um and maybe because

i flew i flew back from pycon on the plane with a whole bunch of these people saying i was like oh

why are you using python just for recruiting so it's almost right you know like but you know of

those are the three kind of modern, I need something high performing. Well, and really,

yeah, people would say go and then JavaScript is mangled. So anyways, it's an interesting

perspective. And I think it's right that if you can just check that box, like again, in your talk,

you had great, you showed a bunch of benchmarks and kind of why they're not the standard benchmarks

people trot out don't really matter or are sort of incomplete, right? I mean, Carlton and I often

talk in this podcast about how Python is not what, you know, the programming language is not going to

be the thing that slows down your web stack anyway but even if you're looking language to language

it's easy to look at one of those and think python slow but it's really so yeah so let's talk about

some other aspects of it as well though right because um web socket support right for instance

that's one of the big now django has that with django channels but it's had to be kind of designed

you know into an existence whereas with you have a corn and starlet we're able to start

from a starting point of right we want to be able to handle long-lived connections such as web sockets

or streaming hcp requests now i know there's not necessarily loads and loads of good kind of

tutorial level examples yet of look at the interesting functionality that you can start

to easily build out with using these stacks but that's going to start to come and I think that

when it does and when you start to be able to show look here's a production ready framework

that you can start working with now and look at those sorts of real-time things that you can start

to build with it i think that will be interesting um and the and the other thing is forgetting about

the async at all but just starlet in its own right i think is really interesting because

the you know having had 10 years of working with django and right how would i approach this from

scratch just the the way that it's stripped everything right down you know it's easy to say

i think that people will kind of get a feel for it when they've been working with it for a bit

but it's a very very low depth of complexity stat right and that's something that i can say

nebulously and is difficult to difficult to quantify but it's a you know i think it's a

really elegant design and i think that as the uh space on top of that starts to mature

that's going to kind of become apparent in the way that it allows you to kind of

uh you know the the framework ends up having a low maintenance cost so you can work on new

things quickly you can adapt new things quickly if you get things wrong it's easy to move stuff

around versus okay cool we're supporting you know versus jango's completely different thing which is

supporting tens of thousands of businesses and here's all the things we're carrying along with

us and we do a fantastic job of continuing to involve and support them um but it's also worth

putting time into great here's this new sprinter on the scene let's work with that for a bit yeah

oh yeah no totally totally totally and like django's got 10 years of edge cases to

you know to accommodate and to maintain and rightfully so i and i always say one of the

reason why it's still exciting is because we don't break stuff so you know that it's still

going to be there and reliable so no but you can use the new version you can update and you can

use the new features because you don't have you know i always argue that unless you're unless

you're going to really abandon your app don't be on the lts be on the latest major version and get

the new feature and have fun with it i mean yeah you've got to update every nine months but that's

just like taking you taking your car to the garage well so hosted api is the name of the the demo app

that um you built out and i guess i i want to mention to people listening if they're more

beginners jacob captain moss one of the django creators his he had a pycon talk 2017 which we'll

link to on if he were going to build django today from scratch what would he uh redo um because i

think when at least when i try to explain to like i work in a co-working space there's a whole bunch

of people using django who aren't developers so they're managing developers and when i try to

describe async i see their eyes um glaze over and i think part of it is they don't they don't

understand how the piece is what is the stack what is a web framework um so you know what is

whiskey how did all these things stack together so um you know i think of this as an educator i

want to write some courses on async but i think it really has to start with well here's all the

things you were using you didn't have to think about but like here's what the stack is here are

the choices because i i think until you can understand why async beyond a real time sounds

nice you have to see where what you're dealing with and you a lot of people haven't see i think

the thing is the reason that we're having to talk about the technical aspects is because it's still

immature yeah right and what i've been working on really hard is and you know hosted api is just

like kind of the very right let me figure out what we're still missing is until you can get to the

point where you can put your technology in front of people that really don't know an awful lot

about the space and say, hey, look at this cool thing. Isn't this what your business wants to be

able to do or to be able to build or, you know, to have your team work with? You know, and I'm

hoping to be able to get into that much more user facing space. Well, what do you think that example

is right because i think of well maybe it's like what is you know something you build the sync way

and the async way and you can just show it's massively more performant is there like i mean

it's like oh chat seems a little bit faster like what would be a good you know killer example right

like what's the like equivalent of like so a real-time admin right so a real-time admin

plus uh oh and look here's a bit of javascript that we've written on the other side and here's

a website where we've got these various kind of graphs that are updating and we can see

when we put you know either when we're putting data into the admin we can see that stuff updating

over here so kind of what hosted api is no you know maybe one day i'm skirting around you know

bigger plans or here's a simpler example right so um being able to when when you start up django

right and you get your little welcome screen and then you've added in the admin and you get the

admin right what would be what would be really nice in async framework land would be once you've

started your new project and you go to a screen you can see uh like a real-time graph of how many

requests per second are we running at the moment and it's updating all the time what's our throughput

how many servers are running and currently connected to the same channel at the moment

to be able to see really responsive set of information like that and have those be in your

basic programmers toolbox so you know rather than django debug toolbar is the thing is the kind of

level that we're looking at you know responsive set of information that's coming back straight

away about you know those sorts of things and in the in the phoenix web framework which is in

which is built on the elixir language which is kind of like a functional thing built on the

earling vm they um they've got a thing which is really the new hotness at the moment called light

which they call live view which is exactly this kind of real-time communication back to the server

and they can do you know inline validation send back the response updates in line and it's it's

like ajax but ajax plus plus plus plus plus because it's so quick and you know and really easy to

implement and they've kind of got that nice and be nice if starlet had that be nice if django get

can get there and it's time and you know for me you you start you've got you take a beginner and

you run them through and you create a django model and they start and they spin up the admin and they

imagine doing that but just with with an async framework where you know they don't even know

about it's async they're just building something and they're and all of a sudden it's like they're

programming and they're learning to program but using this this really cool modern async stack

fantastic yeah one of the things that is difficult though of course is trying to balance

that you know trying to balance django rest framework with okay i think that the

revitalizing the http space in python is really important so here's httpx

with oh i think that the async web space in python is really important so here's the async web stack

right trying to balance those things and trying to make the case for you know especially with

in places where stuff is newer trying to make the case for yes this is less mature

no your business shouldn't necessarily be working on this yet unless you know that you've got a good

reason to um but yes we think it's worth investing the time up front on and so we come to the final

and perhaps most important topic is encode oss and your work to build the tagline what's there

sustainable software development right that's that's how you uh yes yeah it could be well

perhaps it was once upon a time but it was collaboratively funded software development

is what it is there you go yes that that that that chimes because like we've we've mentioned

what a dozen packages drf starlet h uvicorn you know httpx how on earth can you do it and

you know well yeah and there's a there's a short answer to that is uh my time is fully funded i

work on this stuff full time actually i think it's interesting going back into the story of

how that happened to come about because it's quite lucky really um you know django rest framework

two version two was when we very first ran a kickstarter for it and at the time i was working

with the web consultancy dab apps and they were really on board with the idea of running a

kickstarter for this so we kind of came to an arrangement where he said okay we'll do a

kickstarter and you know we'll figure out for however much money the kickstarter makes here's

how much of my time can be taken out of client projects and instead just be put to work on

rest framework and we we ran the kickstarter and it was wildly successful you know for rest

framework 2 i can't remember how many how long ago that was now but it was a way back because i i it

would i got involved with rest framework before version 2 but it was like it was coming along and

there was the kickstarter and it was it was years ago yeah and i think you know the timing happens

to be just right people could see you know what we wanted to do it was kind of just about started

emerging as you know like the more mature like for a long time tasty pie had been overall the kind of

the the slightly better choice in terms of if you just want to look at what features have they each

got but the timing just happened to be just right and you know we got like 30 000 pounds

from the kickstarter and i worked on rest framework 2 for i can't remember but you know

between six months and eight and nine months somewhere in that area which was fantastic

when we came to okay here's a new slice of work that i think needs to happen and rest framework

3 um again we got pretty lucky with the timing because mozilla had just launched their grants

scheme and i knew that although i wanted to run a big kind of initial fundraiser for it what i

actually really wanted was ongoing funding rather than a single kickstarter style drop

work work work now we're done and it wasn't really obvious to me that leaving my job would be a good

idea or not but because i'd applied to the moss grant the way it went was once that grant you

know once they came back and said yep here you go here's your grant which i think was about 50 i

think was fifty thousand dollars i think it was thirty thousand pounds again so that gave me

this huge chunk of runway you know like best part of a year and i said okay like tab apps is a great

place to work but i can see an opportunity here so i i quit and launched the um the ongoing

funding model and i thought well we'll wait and see how many companies pitch in with this and

what kind of level it gets to and if it's 50 of what i need then at least i'll know you know i'll

have a year's worth of figuring out and maybe i can go back to dab apps and work there you know

half time or whatever but as it happens we've just about got we've just about got enough to

keep me going full time on things and sometimes we've had a bit more and sometimes Carlton's had

contract time on it sometimes in the past I've supplemented it with a little bit of contracting

and so on but on the whole I'm trying not to do any of that at all and just use the funded time

and and the pitch that I that I always go with if I'm ever on stage talking about it is

um you know i want to work for your company full-time for a hundred dollars a month

because that's that's what the case is that's and it's so clear then why collaboratively funded

open source development makes economic sense right it's like an insurance makes business sense

yeah it's like one of these plans to service your boiler like you know you make a subscription to

keep the the fundamental infrastructure of your business there isn't a there isn't a person

listening to this podcast who doesn't use res framework right you know if if all those people

fund it a little bit of all those companies fund it a little bit then it's totally sustainable on

the ongoing basis yeah i think too it's easier for people to understand that drf is is you and

a small team whereas i mean i'm constantly struck by django is just this amorphous thing even though

it is a half dozen dozen people who really do the yeoman's work um it's a little bit easier in a way

when you can people can see oh it's it's going to tom and his crew right it's not i know it's a

little bit of an easier ask because i think of this in the context of i'm constantly thinking

for the django software foundation i think there's a lot of ways they could get more money which would

help a lot of things um but it's a little bit harder than when you're literally on stage saying

i want to work for you for a hundred dollars a month and it's just like man if i'm using drf for

anything as a business that's obviously a total no-brainer so i think yeah i think it's i think

it's really important that there's a very direct link between sponsorships and um the person doing

the work because i think it is a personal relationship i mean you know i i think there's

probably more that we could be doing in that space as well that um you know so for instance

things that i've kind of toyed with recently as well it would be great to get some more to get

somebody else in with a bit of funded time on the project well maybe the right sort of way to

approach that is you you have a a mentee a mentorship program and you specifically ask for

you know we want to run this program and we we need three companies to sponsor it yeah and here's

how much we expect you to buy and then it's a very direct relationship you know they can say

we're sponsoring this program right that person i think that is what they want at a corporate level

I mean, it's like giving to, you know, university. You want your name in the building versus just general fund, right? Everyone wants to point to their specific thing. I mean, because I think, again, in the Django context, I would posit that 95, 99% of people have no idea that a Django fellow exists, let alone that there are two, let alone that Carlton is one of them.

I mean, it's just totally – the DSF, nobody knows what that is because they don't really have to.

So it's – I like your idea of direct relationship from mentee.

I mean, just this year, the DSF with Jacob Kaplan Moss is mentoring someone on adding something to the funding page.

They got 1,000 people who wanted to be mentored.

And I think if you had a company that would sponsor that or something, yeah, that's brilliant.

that makes a lot of sense it's like a badge that a company can talk about and yeah have it as you

said a direct relationship between their funding and what they're getting out of it well it's a

lot easier for me and rest framework to be able to tackle that sort of thing because i can just

be very unilateral in how i want to approach funding right whereas that's more difficult with

a commute you know something where there's a bunch of different people who are all kind of

equally responsible for the health of the project to go to to make very like direct decisions about

you know what let's put the funding up front and center and let's be very direct about this

you know those conversations can be more complicated yeah jango's a big ship it takes

a while to turn you know yeah but and yet it you know if you go to a django con in the u.s or europe

or now Africa next year, I mean, it's still a small number of people

who really make it happen, which, I mean, this is like, you know,

being by far the newest to this community of the three of us,

I keep getting struck by, first it's like, wow, this is amorphous thing,

and then I'm like, oh, my God, it's so fragile.

Like, I mean, Carlton's given some talks to this point.

Django isn't, you know, this big organization.

There's Marius, there's myself, there's half a dozen people

who are very active and then there's you know 50 people who are kind of active and then that's it

that's that's it and you know people will email the django developers saying the django team and

it's like well really that's you if you step up well and it's true in so many places as well right

it's you know there's a lot of places where there's these core bits of

you go in python or any other bit of the space where there's something absolutely critical and

it's one person sitting in their garage you know every now and again who's working working on it

out of love yeah yeah well as we wrap up is there anything else you want to mention highlight i mean

there will be a couple thousand people listening to this and you know it'll be there for posterity

uh no just hi

come back in six months time and i might have something different to say to that question

so we'll see okay great we're gonna have links to all the stuff encode everything uh in the show

notes so please do check out all the packages and stacks that are being worked on i'm also i'm

hoping to go to pycon in 2020 which is not something i do very often yeah i'm not often

i think yeah over in the states so it would be really wonderful to chat with loads and loads

and loads of people there because i'm like i say i'm not over that way much so and and but you'll

be at um django con europe they've publicly announced it's in um porto portugal are you

gonna be there i haven't figured that out yet okay sorry to put you on the spot i think i'm

gonna try to cross over the pond uh for the first time i'm really excited to yes no i've been in a

bit of a squish at the moment so okay we can cut that bit out pretty much down okay so tom thank

you so much for coming on the show and listen from us and from the whole django community and

beyond thank you so much for the contributions you give and all the work you've done you're just

you know superhero thank you that's a pleasure yes thank you tom and this was django chat join

us next time folks