Transcript: Third Party Packages

Hello, welcome to another episode of Django chat. I'm Will Vincent, joined by Carlton

Gibson. Hello. And today we're going to talk about third party packages, which are often

referred to as a secret sauce of Django. So to start things off, why do we have third

party packages, Carlton? Why would you say, you know, Django is a batteries included framework,

it comes with a lot out of the box why not have everything in the box because it's been possible

to maintain essentially um yeah and for 80 of projects they wouldn't need 80 of the functionality

um so batteries included is it's the 80 cases it's almost every project should need everything

that comes with the core framework and then in order to keep it maintainable but also to keep

it relevant. Extra functionality lives outside the core. So we wanted to go through our favorite

third-party packages and we'll list to them because there are, let's say thousands. There's

a website, Django Packages, that lists all of them that we'll link to in the show notes. There

are tons and tons and tons, partly because Django with its projects and app structure lends itself

quite well to separating out functionality for these things. So if you build it for yourself,

you can make it available to other developers and then, you know, can be community projects.

But it's always been encouraged, right? So now we've got 10 year history of people being

encouraged to build third party apps, and then building third party apps and making them

available. And so there are just so many, almost everything you think of, there's a good app out

there that does it, or there's one that's at least good, good enough for you to think users are kind

of um inspiration to copy it in your own project yeah yeah i i think it's often it's a great way

to learn if you're to if it doesn't do exactly what you want to look and see how someone

implemented functionality because again with the web almost nothing is unique um it's someone's

had the problem and tackled it before and um just one caveat is that you before you install a third

party app just have a look and make sure it's well maintained um because what can happen is

you install a third-party app and it works great and that's fine but it doesn't get updated

and over time then when you want to upgrade Django you can't upgrade Django because you're tied to

this third-party app so just just vet them before you install yeah don't don't go crazy I mean in

the way to you know how do you know something's active I would say you should have a fair number

of stars so probably a couple hundred and you should see some action within the last three to

six months if not daily well okay but no django app well yes but like be sensible but django

app conf which is a um a package i maintained it's used by django compressor and a few other

applications but i this week i released um a new version which just changed the

packaging metadata to say it supports python 3.7 and the latest djangos and things like this

but it hadn't been updated for two years but it had zero issues and zero pull requests yeah okay

that's that's very great yeah so you know it's not that it hasn't been updated like things which

are very stable they don't need updating every week yeah fair enough i know i brought it up

because it's it's sometimes hard to hard to tell but anyways all the packages we're going to talk

about today are vetted and so if they're what you need um you should feel as comfortable as one can

using them and we should note we're not going to be so there's also what's a third-party app there

are additional things like wagtail django cms which um i guess they're technically apps but

they're really more like frameworks on frameworks yeah frameworks on frameworks so we're not

wagtail and cms they're full cms frameworks and they're amazing and they're great but they're

they've got their own ecosystems they've got their own dependencies oh wow you know yeah so we're not

going to talk about those today, maybe in a future episode. So let's start right at the top. I mean,

I think everyone would agree the number one third-party package, and most people probably

don't know that it's a third-party package, is Django REST Framework, which is how one builds

RESTful APIs with Django. It is what the majority of Django, professional Django developers,

I would say, use when they use Django. And yet it is not part of Django core, which, you know,

you're a co-maintainer on it so perhaps you can speak to why it's not part of django itself

just historical i guess um to create json responses was you know there's a json response

class and then there was a um a package called tasty pie which um enabled you to do restful apis

which was a third-party package and then long came rest framework and it was never it was just

it just was built as a third-party package and it survived as a third-party package and it continues

to this day but it's yeah i mean for me django django is framework right now what else do i need

yeah no exactly so it's um that's probably the number one third-party app that everyone will use

um and we will we'll do an entire episode or episodes on on it it has its own third-party

packages um that we're not going to talk about today yeah but that's um you know another big

one so i'll actually in in the in the news is django channels um because you've um you've taken

on a new role with this right carlton yeah so andrew godwin who created channels has stepped

back from that um so hopefully that gives him a bit of capacity to work on bringing async support

into django core itself um but he stepped back from the maintenance maintenance role in django

channels and I've stepped up to do that with a couple of other people so we're just keeping

that ticking over but that gives async support to right this is really a future yeah the main

the main use case I guess right now is for web sockets right if you want a web socket thing

then Django channels is the way to go but also if you want to build async applications using Django

you can do that right now with channels and then we'll see what happens with the work that Andrew

comes in perhaps over the next six or twelve months we'll start seeing how Django itself will

provide support for Async

Gold. But Django Channels, if you want

WebSockets, Django Channels.

Yeah, exactly.

Moving on. So I would

say the one that i use on every project is django debug toolbar which is a way to view your your

requests so to see how efficient you know your your blog list is this is a small one um it's

i can't think of why you wouldn't want on every every project yeah it's just super it lets you

view template context variables which and which template you know you always this template extends

that template that includes this template which templates actually did get rendered and what were

the contexts that they actually saw this is just super lets you view your sql requests lets you i

know you can and you can create plugins for if you want to view extra bits and bobs you can um yeah

so panels so both to you know hop into your own project or a new project and see yeah that what

is the template structure and especially for performance you can really see um you know how

many queries are involved. So that's really a must-have. I can't think of a project I've worked

on that didn't have that. Another one that I especially like is Django Extensions, which is a

way, it's several tools, but the really nice one is you can add, what is it, plus, right? So you

can do Python run server plus. So if you want to go into the Django shell, it will automatically

preload all the things that you'll need it has a whole bunch of it's really a swiss army knife

of sort of additional tools that as you become more experienced with django are just really nice

to have yeah my my favorite one there is it will graph your models for you and output those in a

in the dot format the graph is format which then i drag into omni grapple which is a capable editor

for editing those but you can drag that into anywhere and then it gives you a nice um kind

a class diagram of your models which is lovely to have and you can edit that and you can it's

one i use a lot so django extensions super yeah yeah i i don't use all those all those parts of

it you just mentioned i should i should look into that um well which ones do you have we've got a

list of things well so jack so another one i maintain is django filter which is awesome if

you want to create um filter if you want to allow people to filter by url parameters so you know if

I put in size equals a bit large and I want to filter my query set.

Well, you don't just want to expose that raw user input

and then feed it into the ORM because that's dangerous.

So Django Filter will create a form which will then validate that the value is sensible.

It will then take a Quilter set and it will filter it.

And you can very easily and very declaratively create filter sets for your models

which enable you to filter by the parameters that you want

and you can filter across relationships and things like this.

And Django Filter integrates with Django itself

but also with Django REST Framework too.

So it enables you to create very easily

filterable endpoints on your APIs, which is kind of cool.

So, you know, I've got a list of blog posts

I want to filter by those that are tagged Django.

Easy peasy.

Yeah.

Yeah, I mean, that's a separate security thing around

just what you sort of alluded to it with URLs,

you know why you often probably don't want to include the primary key um you know for blog

posts because it'll give away the number of blogs that you have right like you would use a uuid or

you could slugify it um yeah there's a nice little thing called hash ids which is worth looking up i

did so oh i don't i don't know what they do is they take the number and they pass it through a

little algorithm and it comes up with a um with like a say seven or eight digit random string

and it can and it's reversible and that that's great but it's it's they're slightly longer than

one two three ten yeah and it's not necessarily giving away that you know you have 100 you know

blog posts on your site and the order the also important the order is not um deducible right so

you can't predict what the next one will be right yeah i mean you know slugs the problem with slugs

if people haven't experienced this is you know what do you do if you know your url structure is

you know my website.com slash blog slash the name and i write one called hello world and then

carlton writes one called hello world they can't exist at the same url url endpoint so you have to

deal with um the conflicts collisions of that sort so that's why um so one thing is you were

just on slugs while we're here we might as well say i quite like using unique for month or unique

for year so you put a um in the url you might have 2019 or two for february and then you might

have a slug which is unique for that month and then it's not unique forever yeah only one that

month and i kind of quite like using that sometimes and that's an option on model fields

you can if you've got a date time field on the model you can give unique for month and say it

has to be unique on reference to this date time field yeah i mean i didn't know that i mean there's

also an argument around not putting the uh you know not time stamping things in the url because

lots of people don't like that now lots of people don't like but if it is that's an interesting

approach if it is a traditional blog like a web log yes there's a difference between kind of what

you want to have put out evergreen content where you don't want the date in the url yeah and a web

blog where it literally is saying hey today i did this there's no harm then that being grouped by

month or by year or yep anyway yeah yeah um so what else do i like i like jango watson which is

a super package which enables you to make use of your database's text searching features um

it's slightly perhaps viewed as slightly out of date now because jango has got more full text

search features built in but jenga watson is is very good and worth looking at and it supports

all the different backends oh great i haven't i haven't used it well it's great if you it enables

you to um set up an index on a model and then you can pass in a query set and you can just get back

the you can get back a query set which only matches which the search and then you could

filter them further so you might say filter by a tag then search for a string and then filter again

by, you know, publish this, whatever you want it to.

but it's it's super um janga watson that's called the other one by by the same author is called um

janga reversion and this enables you to keep um model histories of your models so say you've got

blog posts and you wanted to be able to go back in history um and say oh what i've edited it and

then it's been edited again it's been edited again but actually the version two ago twice

ago i need to go and look at again django reversion lets you do that oh yeah that'd be very useful

he's a very you know he's two two good apps there that i like yeah well another one next on our list

that i really like um is called django all off which is what you would use for authentication

again because the challenge is that django does not come with a built-in user registration sign

up form um so there's a couple packages out there i like django all off the best it does i guess two

things. One is it enables social authentication. So if you want to add Google, Facebook, Twitter,

it's very straightforward to do that. The other is that it provides a whole ton of customizable

features. You can swap in email instead of username. There's a whole bunch of additional

settings. So, you know, it is a third-party package. So, you know, like, for example,

with email, this is a common, common one. The Django default is to have username, email,

password, which is a little bit outdated right now. More common on websites is to have login

or sign up be with email and password. So you can do that yourself. You need to dive in and

use the model managers, which is a little bit of an advanced feature for people.

Or you can, if you know that you need social or you want some of these additional features,

you can just use Django all off and it's a bit faster to get up and going. So in my first book,

Django for Beginners, I show how to do, we use Django AllAuth in some of the later chapters.

And I think in my new book, I'm going to show, for the example with email, I'm going to show

how to do it with model managers. But we're going to end up using Django AllAuth because if you do

want to have social authentication, which a lot of people do, it's a great way to do it and not

have to roll your own solution. Yeah, and it will capture the OAuth tokens you need to hit the API.

If you've got Twitter login and you want to add Twitter features to your website, you've got the right credential to then make user requests to the Twitter API on the user's behalf.

Yeah, yeah.

You can see in the admin.

I mean, all that auth stuff is a bit of a minefield.

You really don't want to roll your own solution.

One thing I was going to say about AllAuth was that it needs a better tutorial because it is quite opaque.

But then you've just said it's in your book.

I have one.

I have one.

Brilliant.

There you are.

Yeah, yeah.

Of course, there is one from, I think, four or five years ago called the Missing AllAuth Tutorial.

yeah do a google search for django all auth tutorial see what comes up right okay because

because because i you know this is this is how i learn i get frustrated and

i write it up um no i remember when i started using all auth it was like wow this is this is

powerful it does exactly what i want but i have no way of understanding this um apart from reading

the code and working it out and it is a package that i use and i was thinking well it needs

tutorials but of course you've written one already well yeah well that's i mean and you know that is

the the thing about all these third-party packages is you get a tremendous amount of power but then

you need to learn how to use the packages and um you know as i sort of mentioned with django all

off you know i've gone full circle on it where i you know i didn't i didn't initially know how to

you know years ago how to do email instead of username i found django all off i sort of you

know yeah clues together how to do it and then i wanted and sort of learned how to do it the raw

way and now most cases i use jangle all off um because i understand what it's doing and it's

managing the complexity but it's a bit of a circle in the same way um you know i would say generic

class-based views are versus function-based views no and the other thing i like about all we've

turned this into an all-off podcast now but the thing i like about all of as well as it enables

users to manage multiple email addresses which is nice if you want to um if they they want to

control notifications if you need to send a user notifications they want to say look i want

notifications to this email address not that one and it enables them to add multiple email addresses

and choose which one's their primary and things like that so it's what something that i like about

it anyway we should move on yeah and i and i include in in uh django x which is my django

starter project because i think it's um it's a great i think it's it's it's something i use in

almost every project all right moving on um so we have ones for deployment which are or static files

which is static files yeah yeah so white noise do you want to talk about that uh well yeah okay so

there's two there's two strategies here static files can be a pain so the white noise but let's

explain so static files you can when you're just prototyping you can store them um within django

but you really don't want to do that you want them on a cdn for performance reasons yes yeah yes um

the thing is you can have that within django so you know when i teach this i i just sort of show

how to set up static files and stuff and then and we use it and then later i show how to um

you know deploy using django storages or something on s3 or something i don't i sort of go i go

progressively with it right but the thing the thing is that the static files app will serve

your views in in development is perfectly well and so they've got this little if debug that

we all use and that's perfect but you can't use that in in um in development so then white noise

comes along and it gives you a view which will serve your static files and it adds the right

caching headers and it does all these nice things for you that's so that's one strategy so it's

called white noise and then the other option is django storages which if you want to put your

static files on S3 or as your storage is or Google Blob Store, it gives you the tools to do those.

Yeah, because I think you can use, White Noise in particular integrates well with Heroku.

Right, okay.

In general though, you do want to put your static files on a CDM.

Yeah, so even if you use Django storages and put your static files in S3, for example,

where you then put CloudFront in front of it

to speed that up or whatever.

Yeah, well, we can get into all that later.

Another big one.

Anyway, those two good ones.

Yeah, yeah, no, those are great.

Those are ones I use all the time.

Environment variables.

This is something you should use in your projects.

And Django environment is sort of the default.

Well, yeah, there's also .env,

but Django environment is sort of managing

your environment variables.

Do you want to—I'm blanking here on how to explain the package properly other than it helps you manage your environment variables.

Yeah, no, it helps you control your environment.

I mean, this is a fuss, right?

We want to keep—we want to use—we want to do the right thing and not put sensitive variables into our settings files, so we keep them in the environment.

But then we've got to remember that we need to export them into the calling process before we can launch Django.

It's all a pain.

Django environment will help you manage all that.

Right, because you'll use different variables, you know, locally versus deployment.

Yeah, yeah.

And you don't want to put that in source control.

For example, like your secret key or your, you know, your Stripe ID if you're using payments or something.

Yeah, or your database, you know, your AWS.

Yeah, your database.

What's it called? IAM access key thing.

Yeah.

You need to keep that secret, so environment variables for that.

Okay, fine. We'll skip on that one.

Django Guardian is another nice one.

So Django has a nice permissions system, which by default, the Django control.auth permissions will say, yes, you can access blog posts.

Yes, you can access, I don't know, the sales catalog.

But if you want to restrict access to individual objects, then Django Guardian will help you do that.

So you can create object permissions for this specific blog post

or this specific item in the sales catalog.

And that's a useful addition.

Django doesn't provide that.

Yeah, permissions and authorizations is very much an intermediate level thing,

but every single project you want to lock those down and set up the hierarchy.

And that's, again, every project you're doing that.

Yeah, I mean, you know, once you get beyond a certain size of team, it will always be the case that there's a category which people can only access a subset of.

And as soon as you need that, you need object permissions.

And Django Guardian is the go-to solution for that.

Yep.

So another one, I'll tee this up.

Django Compressor is one I really like that you maintain, which is for compressing your static files.

Yeah.

So you should do that.

You should use Django Compressor or something like that.

on all your projects yeah i mean so what's really good about it as well as if you use pre-processors

like um sass or less or post css or um you know i use it for elm but other people can you use it

for react and what you you do is you just create a template block which has got you with the

compressed tag and then you put your script tags in them as normal so you might put your or your

link tags for your css um or your script tags for your javascript but you give them a special media

type which will be like for sass so content style sheets slash sass and then when it sees that it

will run it through the pre-processor and convert your sass into css so that you can load it straight

into the browser and in development it keeps them separate so that you can trace it back to the

individual style sheets but then for production you run a compress command and it compiles them

all into one file and you can minimize them and you can gzip them and you can run it whatever

of pipeline you want to and then it puts them into the static static files folders for white

noise or django storages to deploy and job done and it's yeah no it all ties in very nicely and

and you know there's a number of local versus production things with django that um you know

we've talked about a couple that these packages help you with because you'll you know you want

to get up and running quickly and um locally but then for production it's a different set of

requirements often yeah and so anyway i really love that and i'm not a front-end developer and

so if i work in a you know a team where there's a you know professional front-end developer they

they will replace django compressor with their own build system and their own whatever and we'll

work out the deployment but for me as a django developer i can do something that's semi-respectable

quite more than semi-respectable just using django compressor and then when people need to go that

bridge further, they can do that. Yeah, no, it's a really nice optimization. You know, maybe so one

more, this is sort of a fun one, Django admin honeypot. So a best practice is that one will

notice that every Django site has the admin at slash admin. So if you were a malicious actor,

you could write a script and find out if a site was a Django site. And then you could try to

hack it. There's a couple ways you can safeguard that number one is change the URL. So don't

don't have admin at admin um django admin honeypot lets you see people trying to break into your

admin um so it's a you know it's sort of a fun one it's a good reminder that you should

harden your admin um there's actually a number of steps to do on that but that's sort of a fun

one if you want to see someone in particular trying to break in your site versus just random

bots on the internet yeah right brilliant all right okay so look well that's a whole long list

right um but check the show notes for more on those this is a just a quick run through of our

favorites there are as we said millions or thousands of others um there's a group on github

called jazz band which maintain a whole group and so the danger with a third-party app is it's

difficult to maintain over time and so jazz band take on packages which need help maintenance and

then they maintain them as a group and so they keep these packages going and they're lively so

anything in jazz band you can rely on as being well maintained check out django packages as will

said if you think there's any packages that we haven't covered or should go in depth on please

let us know you can do that at django chat.com there's a way to contact us we're also on chat

django and we'll see you in the next episode okay bye