Transcript: Security

I'm Carlton Gibson.

Hello, Will.

Hi, Carlton.

Good.

I'm good.

I'm uh this week we're gonna talk about security, which I'm kind of excited about because it is a deep and important topic.

And I think it's interesting when I'm teaching beginners and I sort of say and then show them the web is a really, really dangerous place.

And it's why framework like Django that takes a lot of effort and a lot of precautions, it really helps you out.

Um so I think we're gonna talk about what are these classic dangers and then what does Django do to help you with them, and then how do you configure it?

Yep.

So let's start off.

So why is the web dangerous?

Um the web is dangerous because there's lots of bad actors, but really bad bots out there trying to hack your site for whatever reason.

Um we've talked about this in other episodes.

If you have a slash admin page, you know, as a default, you should change that because you or I could create a little bot that goes around to every website out there and looks for a slash admin and tries to

Because it's not that we create that bot.

It's that there are scripts down, there are kits downloadable off the internet that have have scripts to check every known web vulnerability in every known framework and how to set that to every addressable IP IP address that your computer can reach.

So it's like you just have to set up your thing and fire off this kit, and this will go and find every server out there and try every vulnerability against that server.

This is this was called there's probably still is script kitties back in the day where you can download and run something you don't know how it works, but you can still cause a lot of damage.

People think, oh, you know, it's not gonna affect me.

I've got a small site, I don't, you know, this security.

No, this security, your isn't if you have a weakness in your application, it's not if your application will get hijacked, it's when.

It's right.

Um, they've publicly announced they're not gonna make it over to Python 3 by January 1st.

So that's a big invitation to hackers just on the Python side, not to mention um Django's updates coming up.

You wanna you wanna stay up to date.

But let's talk about so the biggest risk too.

So expanding

I would say before we get into the technical things, the so the biggest risk is what's called social engineering, which is people.

So that's some individual who has access to things is compromised.

Either they're disgruntled, but more likely they click on a phishing link.

So someone sends them an email that looks like a real email, they click on a link.

I mean, this is what happened in the US in our elections.

Um, some people in Hillary Clinton's staff clicked on phishing email links, click on one link, and it goes to a site that looks like a site but's not.

Um, we'll get into how that kind of works.

But and then that person, because that person is their level of permissions are now owned by Russia or whomever.

So the biggest risk is always people.

So you want to make sure that you set permissions.

You don't give permissions are set appropriately.

You want to have a lot of permissions levels in your uh application.

You also probably two-factor authentication is a good idea.

I know there's some talk about uh eventually rolling that into Django itself, but permissions I would say is the biggest one.

Like, for example, do non-engineers need right access?

No.

Yeah.

Oh, wouldn't it be nice if I could just do this thing?

Well, do you you need to do that?

Is it not just for the sake of security better that you can't do that and you have to go past this through this other person who can?

And if you don't do these things, they will happen.

So it's it's you know, it's important to do.

I mean, the number two risk, I would say, I'm curious what you think, Carlton, after people, so social engineering would be uh updates.

Just keep up to date, keep up to date with Django.

Most of the minor releases are security related, uh, especially third-party packages.

For example, Django Crispy Forms um just went from up to one point one point eight point zero, but um the one from 1.71 to 1.72, there was a uh security issue there, right?

That I know Adam Adam Johnson and you were mentioning I should update my books to uh fix that.

I guess what else would they be?

Maybe like some egregious bug fix, but we'll

So Django itself, like the main, the main, the current major version, so currently 2.2, will receive bug fixes for the um the current for that current version, but bug fixes for new features, right?

So if it was a new feature introduced in 2.2, then you'll get a bug fix in 2.2.4, say.

But beyond that, the extended release, all you know, the 1.11.25, that's the LTS.

Yeah, the LTS, all those L all those long year and a half, two years, three years of whatever it is of um security releases, they are, you know, they are security releases.

They we they would also get data loss bugs, but those are so few and far between as to you know almost never happen.

Um so the majority of patches there are are security fixes, and most of them are public in that you know, there's a there's a um uh a database, a CVE database of um security exploits, and you publish those so that people know about them and know what the mitigation is, and you know, people who are interested in security subscribe to that those mailing lists and they know about them so they know what to update.

It's really important that you keep up to date.

And as I said, it was as we were talking about at the beginning, it's because for every vulnerability, someone will write the and it's not someone evil, it'd be a penetration tester, a pen tester, right?

Someone whose job it is is to find out whether an application is secure so that they can secure it.

But they write

Yeah.

It can be used to uh for penetration testing, which is a legitimate use, or it can be used for penetration, which is obviously not so good.

And we've we've talked in another episode about how to update uh Django.

There's all these great deprecation warnings, and what you want to do is you want to go through and run.

Oh, I'm forgetting off the top of my head.

What is it?

It's with the W flag, what's the code?

And if you put warnings all or wall, I mean it is because it really is like wall, but warnings all, then all these deprecation warnings um or pending, so there's kind of levels.

A pending deprecation warning means it's not quite deprecated, but it will be soon.

And then a deprecation warning means hey, you really better fix this now because it's going to disappear very shortly.

Um, but by default, pending deprecation warnings are silent.

So with it without this W all flag, they don't show up.

But you want to run that at least time to time, or at least in perhaps in your um in your CI as a different build so that you can um you can see these deprecation warnings, and then that you've got ages and ages to fix them.

So yeah.

So you've got like three whole versions to fix it.

Like 3.3.0, 3.1, 3.2, and then 4.0.

There's only gonna be 3.2.

So 3.2 will be followed by 4.0.

So what else?

Uh well if if you're curious on, I'll just slightly plug myself, if you're curious on how to actually step through all this, I have a whole chapter, actually a couple chapters um in my book, Django for Professionals, on doing this, on making something production ready.

Um, because the information is out there, but it may not be in the most concise form.

So what's another step?

Environment variables.

This is another must-have for security.

Basically, anything you don't want to put secrets in source control.

So you don't want something, for example, your secret key in your settings file, API keys.

You don't want that floating around so anyone who has access to your code base, any employee of any level can see them.

So you create an environment variable which basically lives separately.

And how would you uh how would you describe it, Carlton?

What's how would you

describe it's a environment variable?

As it's launched with keys to strings, which um the the launching process can set and it will be inherited from the the launching process.

Hence, you know, when you're in in bash, you might you might have you might have put export um Django settings model.

The export says and pass this down to the um child processes.

You can set environment and you can use those um to inject these environment variables into your settings file.

So so when you first start Django Start project, you get a settings file and it's got a generated secret key for you.

And that's fine for when you start.

But before you deploy, you want to change that to use an environment variable, and then you can keep that in a .m5 file people use a lot.

And then you know there's various ways of launching with with that environment with those environment variables in your in your process environment.

And you probably want to generate a new secret key because chances are you've done a couple commits.

And actually there's a there's a I'll put the command, there's a neat command you can use Django's uh generator, but basically it's a 50 uh character 50 character long string, random string um that's generated.

Yeah, I

And if that seed's known, then you know, then all of that can be um broken because the algorithms are uh are known and they're it's good that they're known because then they can be audited, but the seed has to be kept secret, otherwise they don't work.

And again, any commit ever that just one commit out of a thousand that happens to have the secret key or some other secret there is available to anyone who hacks into your GitHub, GitLab, whatever, anyone who has access to the code base.

So change it.

Yeah.

In that, you know, like UUIDs or or whatever secret keys, you can just generate a new one.

It's it there's not much cost to doing it.

Um and I would mention, so what else?

Uh I mean the main ones, we've talked about this in a deployment episode um where you go on at length, but you know, turn debug off.

Um you want to set your allowed host so you can't just let any old person come in.

And if you run the um that there's a deployment checklist built into Django and there's docs on it, python manage.py check uh dash dash deploy, it will say, Hey, here's all the things you want to make sure are configured properly for production.

And then for me, the thing that you you need to do more than you know, the sort of general rule is handling user input.

So the goal here is you must always filter input, and that always means always pass it through a form with whatever validators are sensible or you know, a rest framework serialized or something like that.

So you filter all input and then you escape all output.

So if you know you take if you're gonna take user-generated markdown and render it to HTML and put it on a page, you must you must run that through a sanitizer like bleach, or you must escape it, right?

Now obviously you have to, if you're generating HTML, you have to run it through a sanitizer like Bleach because if you escape it, you'll just see, you know, HTML in your HTML page.

So to escape something, um so um Django gives you the um the escape utility, which will, for instance, take all uh opening angle brackets and convert them into um and LT semicolon HTML entity.

So instead of it look so if you put if you had angle bracket p close angle bracket, instead of it appearing as an HTML tag, a paragraph tag, a p tag in HTML, it would appear as um the HTML entity less than the HTML entity greater than P in between.

So it would it would be viewed and you'd see it as as the HTML on the rendered page.

Yeah, it just makes it less.

So I actually have if I have a I'll have a link, an HTML escape tool that I built just with JavaScript that um on my website, wspincent.com slash HTML escape tool.

So you can play around and you can escape or what is it?

Um uh unescape.

Yeah.

Anyways, these are it's

Anyway, let's go.

Um you can't trust someone out there and you want to do all these, you know.

So if you use use Django for

Great, great folder.

Do go and rummage through there.

HTML.html, and then there's escape, there's unescape, there's uh conditional escape, format HTML, there's all sorts of good stuff in there.

Yeah.

Strip tags, all sorts of extra.

So we could uh so let's see.

So what are the common attacks?

I don't think we covered this in a deployment um episode, but just just very briefly, common web attacks you should be aware of.

So the top one is, or one of the top ones is SQL injection.

So someone has your form and tries to put some SQL in there, like delete database.

No, I don't know.

it would be it would be fun.

Because the problem is it's you don't want to play around with this on real sites, and a lot of real sites are just uh shockingly susceptible to these things for one reason or another.

Like you have to you you have to like you know, so say you're using Jango, you have to get the connection, then you have to pull out the cursor, then you have to concatenate your string, and then you have to like you know that stuff's known, but it's not in any of the the books.

You have to you know dig down and and work out how to use that.

So you wouldn't be able to get user-concatenated SQL into your database if you're using the the ORM unless you tried.

You'd have to really Right.

So what else?

Um XSS cross-site scripting, someone injects a little bit of JavaScript into your page.

This is why you want to automatically escape um user input.

CSRF is a big one, cross-site request forgery.

That just basically exploits the trust in a user's browser.

So this is why whenever you're doing uh a form that has a post in Django, you want to put in those CSRF tag to load in the CSRF middleware, which will puts in a random secret key, basically as a cookie that doesn't let you so so what how why does this work?

It's like if you're logged into two different um, what's the classic example?

If you log into a banking site, a server sends back a uh session token.

Again, this is because HTTP is stateless, so each request is kind of independent.

But what happens if I get in a separate tab, I have an email open, someone sends me a link, and it sends me to something that looks like my banking site, but it's not actually the banking site.

But my browser already has loaded my uh token.

So basically that person on this fake banking site, because I'm logged in in another tab, in this new tab, can act like me and can do all sorts of nefarious things like take out money because I'm still logged in.

So this is, I mean, sort of the fundamental trade-off of the statelessness of the web is this security problem of how do you know that another tab open is actually the correct one.

And it if the post data doesn't include the correct CSLRF token, then the post request will be um rejected even if all the other fields were valid.

So if you're rendering a form, you have to add that, yeah.

I mean that's I wonder if almost never would you not want to have that there automatically.

Because the modern web frameworks, not just Django, but all the modern web frameworks handle this in more or less the same way.

And it's kind of as long as you follow the practice and you use CSLRF, you really are protected.

You know, it's it's a it's kind of nice.

Are there some others that well I can go through?

So click jacking, that's another one.

There's a middleware to protect against this.

That's if someone puts a hidden iframe.

So an iframe could be when you put another site in a site.

So for example, if you have a Google Maps embedded on a site or a YouTube video, but what happens is someone can just because it looks like something doesn't mean it's what it is.

So someone could show an image of a kitten and you think, oh, that's a cute kitten, but actually it goes to Amazon and tries to make an order or something on your site.

So Django's uh clickjacking middleware checks basically whether a resource can be loaded within a frame iframe on the page.

Uh is that is that right?

Um yeah, almost

I can't think that I've wanted to actually embed my own site in a wiframe more than a few times ever.

Like if you'll if you'll if you're discuss, right, where they've got that comment widget thing.

Yes, you embedding then okay, then you need people to be able to embed your site in an iframe.

But if you're not discussing it,

Yeah, it's D I S U U D.

Whatever it's how you pronounce it.

Yeah.

Yeah.

Um But I, you know, if you're not them, when have you ever wanted to put your site in an iframe?

But on a Django site, um, I mean you can still do it, but that that's the only example I can.

But it's also the the danger of a widget is you're trusting that widget and all their friends to be good actors.

Um and they may not be, or maybe they are, but then you're still using the widget and someone takes control of it, or you know, so this is

But if you're in this kind of situation, then what you actually want to do is have two Django apps.

You want to have your main Django app, which is sensible and doesn't let, you know, it's got all your on-site functionality and it um doesn't let anybody embed an iframe.

And then with the same project, you just want another whiskey.py file somewhere else that serves a an optimized and minimized um Django app, which just does the iframe necessary bit and it doesn't have access to all the other stuff because you don't want the you don't need to have all that in all of it in one application.

The same with async stuff.

You might you you might need only a couple of endpoints which need the new async stuff, or just serve those from a with a sl with a different worker, with a different um you know server, and have most of it going through the you know tried and tested whiskey pipeline, and then just have you know your web sockets coming into a different worker, a different um server endpoint or a different you know process.

So so what's the takeaway?

So security is important.

Django basically has you covered, but you have to use Django appropriately.

And that would means I would say definitely run the deployment checklist, yeah, make sure you're using environment variables.

If you do those two things, Django will hold your hand.

Uh there's a host of smaller things, but that gets you most of the way there.

If you if you pass.

And if you do that, the major attacks are covered.

Yeah.

If if you don't you know that's the big danger, is um.

They do.

They do.

And forms also do um check validation and uh forums will also protect you.

Okay, so then you've got a you've there are filters like um uh UR IRI to URI, where which you should run any user-submitted um URLs through to make sure they're incorrectly URL encoded.

And like escaping's a bit more complicated than Django does it all for you.

Yes, if you just if you if you just have the um if you just take the user submitted content and you you put it into a template, Django will auto-escape it.

But sometimes you don't want it escaped because uh you it an escaped URL won't work.

So you need then to run it through the appropriate filters to make sure it's safe to present that URL.

And you mentioned too just the Django Bleach package, which we'll link to.

I wonder.

So this is so say you want to you you you actually want to present user-submitted HTML.

So you can't escape it because it ceases to function as HTML once it's escaped.

So you have to sanitize it, and bleach is a sanitizer, and it will um enable you to um provide a very limited set of tags and a limited set of attributes, and anything which doesn't match those gets stripped out.

So for instance, GitHub have a markdown editor field, right?

They you can't put any old HTML in there.

There's a very limited set of HTML.

You try putting a strip script tag in there, it just won't appear because they they they filter that out.

I I didn't even know, sorry, I didn't even know there was a Django Bleach package.

I don't know what this is.

I mean, that's something I find more and more as I'm progressing in my Django journey, is that I see, well, specifically with Markdown, I'm building something that I'm using Markdown, and there's a whole bunch of Django Markdown packages, but I mean I can also just pull in, what is it, Markdown 2, whatever the Python library is and create a custom template, uh, custom filter.

So uh it's sort of a nice place to get when I look at an app and can think, do I really want all that functionality and all that um trust that comes along with it?

I mean, a lot of times it's helpful, but if I can just do it manually, more and more I would rather uh roll it myself if I can do it concisely.

The number two um risk we said was um not updating.

And why don't people update?

They don't update because there's a third-party dependency which isn't compatible with the new version.

Yes.

So don't take on third-party dependencies is is a big issue.

We just had one with Django today.

Python 3.8 has been released, and there's this traceback lib that we use um with the test runner to format um format tracebacks.

And that's great, but it's got a bug that's fixed in master um against Django against Python 3.8, but it's not been released yet.

So we now have to we've had to insert a conditional, skip this test if this version is lower than because in order to get around that.

And Django doesn't take on dependencies for exactly this reason because then we're we're tied on them.

And okay, that will be released soon, and then you know we'll be able to drop that skip if, but projects really get stuck.

Oh, I'm using you know, this third-party package, which hasn't been updated in three years, and it's not compatible with any supported version of Django.

So I'm on an end-of-life Django because of some third-party package that I took on, because it saved me half an hour on the first day.

That's so true.

I think I think half our listeners are gonna go, mm-hmm, and half are gonna be like, that's an interesting idea.

I mean, not for not unless you know it's easy to do.

Is it when what's its update cadence?

So, like, you know, okay, you to we mentioned crispy forms that had been updated.

It hadn't been updated for a year and a half, but it hadn't need to be, because it was 2.2 compatible a long time ago.

And we've just updated a release which is 3.0 compatible.

And since my talk at DjangoCon Europe, I've had loads of new contributors to help me get the bootstrap stuff going much better.

So I'm really pleased with that.

Um right, but the whole point is Crispy Forms is stable as whatever.

A year, a year and a half between releases is not a problem for crispy forms.

But a package where it's new, you know, it's only had 30 commits, it's only got one contributor, and it be you've got to be really cautious about taking that on.

Can you can you get what you need from it without having a third-party dependency?

If you can, I would recommend it.

I I almost wonder, it'd be nice if there was a like a check mark that you could see, oh, something hasn't been updated in a year, but that's because it's fine, or it's because

And if you're not knowledgeable, you may well, there may be some, and you may look and see, well, this hasn't been updated for a year and a half, and I see a whole bunch of PRs.

I don't know if I can trust it.

And it turns out those are minor things versus a oh, like there isn't Python 3 support.

That yeah, um, I don't know the answer to that.

I mean that took me a long time.

I used to be like, oh, you know, I can just bring in this third-part dependency and bang some stuff out, and then I you just find yourself getting into a um a spider's nips.

Is that the right way?

A spider's nest?

But tangled up.

You get tangled up in these dependencies which aren't really giving you much value.

Hopefully that helps everyone.

We'll have links to everything.

As ever, we are at DjangoChat.com, chat Django on Twitter, and we'll see you all next week.

Bye bye.