Transcript: Channels

Hello and welcome to another episode of Django Chat, a weekly podcast on the Django Web Framework.

This week we're going to talk about the very popular Django Channels package.

I'm Will Vincent, joined as always by Carlton Gibson. Hi, Carlton.

Hello there. Hi, Will.

So, Django Channels. This is a very popular package with a long history and ties into bleeding-edge Django.

So let's describe. So what is Channels? I'll tee that up to you.

So it's been around for a couple of years and you are currently the maintainer of it, though it was originally created by Andrew Godwin.

And really, it was a first attempt at tackling async.

Yeah, exactly. Exactly. And so it's now on like version two.

Version one was the first go at, well, how might we make Django async?

I think Andrew talked about this a little bit when we interviewed him.

Yes, and he has some talks as well that we'll link to talking specifically about channels as well as async.

But yes, our interview with him, he talked quite a bit about async itself, modern async.

And so the first thing it does is it enables you to, it will wrap your existing Django application in an async handler, an ASCII handler, much like there's going to be available in Django 3.0, almost, you know, very similar.

And so it will enable you to use an ASCII server to serve your Django application.

And then on top of that, so that's the first thing.

And that's kind of interesting.

That's what it will be in Django 3.0.

On top of that, it has WebSocket support.

Yes, yeah.

And it also has proper async view support.

So you can write async def views, which will handle, say, HTTP,

but in a properly async way all the way down.

And it also, there's another interesting bit it has, which is called a channel layer, which enables you to kind of talk between applications.

So you need a backing, a back for that.

So the one that's in play is Redis.

There's a channels Redis package, which sits on top of that.

And that enables you, for instance, to have, say, group chat, for instance.

So you open a connection to my website, and it's got a WebSocket that enables you to do async WebSocket communication back and forward to the web server.

And I open one as well.

And then the channels layer bit enables us to send messages, not just back and forward to the server to ourselves, but to a group, which then fans them out to everybody in the group.

So we could have a chat room or a multi-user chat or these kind of things.

has a course on building a uber clone so real time with with taxis um we'll link to that um

yes these are okay so yeah that example you know and you could you could communicate using um a

mobile app for instance to the channel server and that might be able to talk between apps or

you know i think an uber driver has an app and you have an app and the two communicate via a

server using probably web sockets or similar yeah yeah those web sockets so but the so i guess the

question for people that was so what should i use now because channels uh right has been under very

active development in a way the new async stuff is a bit of a okay so the first so the first thing

to say about that is the status of chance so i've so andrew has stepped back from channels and he's

now working on async in django and i've stepped in and i'm just maintaining it and there's a massive

difference between when django andrew was actively developing channels and i'm i'm doing maintenance

and I have I do this isn't part of my Django fellow work so it's just volunteer basis so I

have an hour a couple of hours each week where I triage tickets and I fix bugs over the course of

the medium term so we have a release now maybe two three releases a year just fixing one or two bugs

rather than Andrew actively actively developing it and pushing it forwards but it is maintained

it's great and it there are there are issues and if you want to contribute I'm happy to help you

contribute and we'll get those issues fixed and we can put new releases out but it's it's good and

it's there what should you be using django 3.0 just about to come out if all you want to do is

wrap your application in um and serve it via an ascii server use django 3.0 because that's all

um that's that part of channels will be in django cords no point using channels for that

um eventually we'll remove that from channels but not until 2.2's end of life i suppose

because that doesn't have the ASCII support.

If you want to use WebSockets, then yes, channels is a great solution for that.

And if you, at this stage, want to write async views,

then still channels because 3.1 should have that in Django core but doesn't yet.

And if you want to play around with the channels layer stuff,

which is kind of exciting, it's kind of like it enables you to build, say, some…

it's kind of like a proto message bus. It's not, it's not a full scale one. You know,

there are some full scale applications that you have to run, but if you just want to play around

with passing messages between different clients, channels will serve that purpose. And so it's a

good, um, starting point for that. And if you run into difficulties or scaling issues or whatever,

then perhaps you can move up to one of these more fully fledged message bus

applications at the time. So that would be my answer. And what should you be using? Depends

what you want to do as ever well and i think the history of channels is interesting just in terms

of shedding a light on django and how it's structured because so channels is not part of

django but it was added as an official django project in 2016 so this is i guess a little bit

like django rest framework is actually i don't know if rest framework's an official project i

don't think that it's not but rest framework was originally under tom christie who was the creator

rest framework his um sort of um github account and his personal account and then he created encode

oss software which is like um they seek to do sustainable open source software development so

you can fund the projects so the main the biggest project there is rest framework but also the new

httpx which is a fully async requests compatible http client it's going to be or library it's going

to be yeah we must have tom on to talk about that because that's the thing you and i talk

and get really excited about but i don't think everyone else knows about it yeah i mean it's it

uses ulm lib3 it's fully async compatible it's got multiple backends um async io and trio it's it's

you know it it's going to be ubiquitous in the future yes yeah yeah it's it's really hot and

also he's got the starlet asgi micro framework there and some other things are all under encode

So REST Framework's under that, and you can sponsor Encode to help fund REST Framework and make that sustainable.

But why is Channels?

But Channels is an official Django project.

Why is that?

I think because it was clear right from day one that async was where we wanted to go.

It was clear to Andrew, certainly, and to, you know, but it wasn't something that we could just bring straight into Django.

Django is mature and stable, right?

This idea that it's really exciting because it's mature and stable and you can be on the latest release because you know it isn't going to break and it's all these wonderful things that enable you to have all the new features.

But that means we can't just bring something, a massive architectural redesign in and put it straight into Django, straight from the word go, because that's not mature, stable behavior.

and to be honest that turned out to be very prescient because channels one looked like it

was the perfect solution and it looked great but it turned out it had scaling issues and so that

required the whole channels to rewrite and if we'd have had to do that inside django core that would

have been that would have been end of story yeah well and i want to mention too well i'll link

there is a thing called django enhancement proposals based on python has the same things

of which um having official projects is one of them the new async stuff i'll link to that that's

that's kind of where these decisions are made i would say right or you know these big changes to

django um so so there's a whole depth on async and that had to be discussed at length and then

that was approved and now it's being implemented and we're we're at stage one now with django 3.0

where Django speaks ASCII.

And then stage two will be

that you can write async def views.

And stage three will be the ORM

and template layer and these other.

Well, I think it's worth repeating

how these things are structured

because I'm always struck by the fact

that Django seems mysterious

and otherworldly to people

when in fact it's all out there in the public.

And if you're curious

or want to be involved in Django stuff,

there is the Django core.

Well, not Django core.

what's it called? Not Django users. Django developers, Google group, which we'll link to.

And there's also a separate Django users, Google group, though. Now there's a new discourse forum

board, which we'll link to, which is an effort to make it a little more beginner friendly. But

anyways, all this is out in the open. So the question is who decides? Everyone and no one

is basically the answer. If you're active. So if you want to get involved, I'd say subscribe to

Django development. It's not very high traffic, you know, a few emails a week. And if you haven't

got the capacity to read them, don't read them, you know, just sort of have a rule that puts them

in a folder automatically. And then you can look at the folder once every so often, or just go to

the Google group once a week and sort of browse through it and have a quick read. It becomes more

relevant as you get in, you know, the more time you spend hanging around, the more it's like,

oh, I read that, you know, okay, it makes sense. It's. Yeah. And I know that Andrew mentioned at

DjangoCon that there is a section on async

in the new forum in part because he didn't want the Django developers group to just be

async issues. So I think, I think this is great to have just a better UI around discussing these

things. But again, it's, it's all out there in the open. There isn't someone behind the curtain

deciding, um, if you have opinions, get involved. Okay. And, and the channels projects, like,

So if you go to GitHub, the Django organization, there's Daphne, which is the web server.

There's Channels, which is the main library.

And there's Channels Redis, which is the behind one.

As well as that, there's ASCII Ref, which is the reference implementation of the ASCII standard, which is the async standard for web servers to talk to applications like Whiskey, but ASCII.

Those four repos, that's where most of the discussion happens.

And if you want to watch those repos, GitHub will send you notifications.

And then, you know, you can join in the discussions.

And if, you know, if you want to help get involved in channels, I'm really happy to help get you set up, help, you know, help talk you through what my understanding.

And we, you know, write some tests and we can fix the bugs.

Yeah, well, because there are, I wish we had better visibility on this.

There are, I know, anecdotally, places using channels for real on, you know, professional projects.

I just don't know how public that is, but you can definitely use it right now.

And if you need something right now, you should use it rather than waiting on Django itself.

Yeah, I mean, you know, even though it's, you know, a few people that help a little bit,

even though it's just me and I don't have much time,

and I certainly haven't had much time this year for various personal reasons,

but the number of open tickets on those three repos combined is quite low.

there aren't many, you know, it's not like there are loads of outstanding bugs or whatever. There

are a few and we need to chip them away over time, but it's nothing that, you know, a little bit of

input from the community and we wouldn't have those all wrapped up. So it is, they are good

quality packages and they do work for you and, you know, you can use them. You can do async already.

Yeah. Well, and that's the interesting thing that Andrew's talk at DjangoCon he alluded to,

which is that, you know, three years ago, kind of everyone thought WebSockets and all this stuff

would be everywhere now.

And it turns out that the need

hasn't quite matched.

That isn't the case yet.

It's not quite as ubiquitous.

Everything doesn't have to be real time,

even though when WebSockets came out

and HTTP2 and all these things,

it's all kind of there now.

But waiting, I would say,

for use cases beyond a chat app

or taxi riding,

I think these will come.

But it's interesting that the demand

isn't is more the issue than the technology generally speaking it yeah i think it is

interesting i mean somebody there was an email to django developers quite recently i can't you know

it doesn't matter exactly when but it was saying well why aren't web sockets in django cool and

it's like well exactly this point is it it turns out that they're not the silver bullet people you

know there was a lot of excitement about it oh we have to have this we have to have this

they're okay they're great they work but there are you know other solutions they've got

server side events where you have a slow http connection and you fire updates just down the

wire so most of most of what people use web sockets for is like i just want an update from

the server yeah well it turns out you can hold an http connection open yeah and you can just

have the server push more data when it wants you don't need a whole the whole web sockets because

the trouble with web sockets is you need client libraries and then you've got javascript and

you've got you know loading issues and you've got well what's the quality of the javascript client

and you know so a lot of issues we get on channels turns out to be well i'm using a broken web sockets

library right well since i think we have a couple minutes maybe can you so make the case for why

actpx is so exciting what does it oh what will it solve that's not channels at all but no i know but

it's but it's related it's a flavor it's related ish well for instance i don't know let's say you

want to um fetch web pages you want to you know you're writing a scraper no one writes a scraper

but let's say you wanted to write a scope you could it's fun to do you want to no but okay but

if you're using requests um which nothing requests a great lie but it's been the cornerstone of

python development for you know many years but you've got this this same sync versus async problem

in that you're using up a whole thread a whole worker for each request yeah whereas if you can

do this using async io you can that same worker can probably get throughput of 100 times yeah i

I mean, it's just a massive change of, what's the word?

Change of scale.

It's an order of magnitude.

Yeah, it's a step change for sure.

Improvement in your perform, right.

But if you, so if you wanted to do,

if you wanted to make web requests at scale,

you're massive savings.

Or if you even wanted to do it at small scale,

so you want to get

little amazon micro thingy bob you could probably run it using httpx on just a single instance and

you'd be able to make all the web requests you ever want to make versus oh hang on i need to

scale up i need two of these to get the throughput that i'm looking yeah no um so that's i mean

that's the primary difference it's it's but also it's written on the new um underlying networking

layer um i don't know much about that can you so well no i don't know all the underneath but um

anecdotally, you know, Pete, that we'd have to get, yeah, we'll get them on, you know,

we've had to talk more sensitive, but request was built over a whole URL. It wasn't that great. And

blah, blah, blah, blah, blah. And so it was a nice API on top of some kind of not very nice

foundations. Whereas HTTPX is built on top of the new URL lib3 and it's all, you know, 100% type

annotated and it's, you know, yeah. So this is, these are the exciting things. It's, it's, it's

a async and b it's built on better foundations i would say those are the two take-home points for

me we'll have to as i say we'll have to get tom on and ask him well and i and i think one last

thing i would say it's again repeating uh andrew's talk on the state of async and django from django

con where he even said that you know async is hard to reason about and hard you shouldn't just

take it lightly i mean he was envisioning that even in an application or website that needs it

it should probably only be 10 to 20 percent of the website it's not that something everything

needs to switch over to async it it's that for certain things it'll have big benefits and the

way it's being written is you can toggle back and forth within an existing django project which is

sort of nice yeah i mean so yeah exactly this you want to write 90 of your views just like you

always have and then where where does async going to help it's like if you're fetching out to a

third-party service or if you're um you know if you've got to fetch something from the database

and get some other data from somewhere else and then combine them you know those kind of um cases

where you're io bound and you're not tying up a whole worker for this but you're you know your

bog your bog standard quickly serve a response that could just might as well just be sync right

well and i'm again with three one i that's what i'm you know as a content creator teacher that's

when I'm going to start banging out a whole bunch of tutorials, because with the combination of ASCII

and the views, you'll be able to demonstrate this stuff and then, you know, ORM and these other

cases. But I think that's when it really will be real for a lot of folks, the built-in async stuff,

I would say. Yeah, for me, like the exciting case with async views is these kind of, I've talked

about it before, but these kind of proxy views. So say you've got a REST API and you've got these

endpoints one for books one for authors and you want to combine them so you can get a nicely nested

response that contains lots of data so a mobile client only has to make a single request

well then a proxy endpoint instead of creating you know a whole different api you can have a proxy

endpoint which makes these requests for you and combines them how the client needs it and then

returns those that will always be i o bound so if you can use async there for me that's a really

exciting news yeah that's what we get excited about carlton things like that that's what gets

us out of bed in the morning well you know that's my new program um all right everyone this is a

this is a tech podcast right well everyone thanks for listening if you have questions you can reach

us at chat jango on twitter uh all the new episodes are on jango chat.com website and we'll

see you all next week bye-bye all right take care bye-bye okay so we thought we were done but carlton

really wanted to get some extra points in go yeah no right so channels and it turns out i thought

channels and httpx were two different topics it turns out they're not because the these proxy

endpoints that we were talking about that's the perfect use case for an async um http client right

so you've got an http request comes in you want to proxy out to say three separate endpoints and

you want to make but you want to make those requests to those three separate endpoints in

an asynchronous way so if you were to just use requests you'd block the entire worker whilst

you were waiting for them to come back but if you were using httpx you could fire off all three

they'd all cut they'd all happen at the same time they'd come back together and you can get your

response compiled and sent back to the client nice and easily so you could do that already using

channels and httpx and when django has async def views you'll be able to do it in django directly

boom mind blown that was it that was just like yeah but we're already there you know these these

these we already have the tools in play now for um being able to write these nice async views

okay everyone that's our last point we wanted to get in um yeah we really are okay bye