Transcript: MySQL & Security - Adam Johnson

Hi and welcome to another episode of Django Chat. I'm Carlton Gibson, joined as ever by Will Vincent.

Hi Carlton.

Hi Will. And today we have special guest Adam Johnson with us. Hi Adam.

Hi Carlton and Will.

Thank you for coming on. So Adam, you're going to have to tell us about yourself. So who are you and how did you get into Django? Let's begin there.

Yeah, I am a member of the core team since 2016, but I started working with Django back in 2012 at my first full-time job, Memrise.

Since then, I've worked with Django at other companies, Yplan, Timeout, Genius AI, and now I am a contractor offering services under my company, AWS, Adams Web Services.

So now in university, had you done anything with web frameworks before Django?

I actually started building my first website when I was 11 years old.

As a little freelancer, I was paid a very small sum for building the podiatry website for a friend of the family's business.

Then when I was 16, me and my brother set up a shareware service.

offering a game creation software as Red Spunk Studios.

And for that, we had a PHP website with no framework.

So I'd kind of experienced the world of frameworkless web development,

just throwing some files on a server.

Yeah, the classic PHP.

Yeah, crack open the template, open the tag, start writing some stuff.

Yeah, that's good.

Well, I wanted to ask you about, you mentioned Memrise.

So I was at Quizlet, I think at the same time, or maybe some overlap.

And Quizlet was a PHP no framework site.

Oh, really?

Because it was originally built in 2005, the founder, when he was in high school.

So there was lots of learnings around that fun.

But it's still going. It's scaling.

So had it developed a kind of internal framework or was it?

I think it was just pieced together is the honest answer.

and sort of Frankensteined, and that's not uncommon,

especially when the creator was young

and it was pre these really great PHP frameworks

like Laravel and stuff.

So it was hosted on joint servers.

So it was a long time ago, 14 years ago,

setting up a website.

I mean, way more complicated across the board.

I mean, payments, ads, all that stuff

was just light years more complicated.

I think deployments were easier

because you'd drag and drop the folders onto the server.

Well, the joke is the R-Sync and a can of beer, right?

It's just watch it upload, and if it didn't work,

you just re-upload it and edit it live and FTP,

and oh, yeah, it's done, yeah, it's fixed.

Well, so I think at Memrise,

were you the only web developer on the team?

Is that correct, when you were there?

No, no, there was a period when I was the only web developer.

Okay, because it was already quite a large site,

because I know the founders knew each other back,

this was 2010 to 13 when I was at Quizlet,

And we knew of Memrise and, you know, even back then.

Yeah, I was at Memrise 2012 to 2014.

And in around mid-2013, there was like quite a big staffing change.

So we ended up with one person looking after the mobile app, me looking after the website,

and a part-time contractor handling DevOps where the service needed rebuilding effectively.

So that's when I learned quite a lot of the depths of MySQL

and the value of doing DevOps properly.

And was it always Django?

It was Django before you came onto it?

It was Django before I came onto it, yes.

And I remember doing the Django tutorial and thinking,

oh, wow, all this structure to building a website.

This is just fantastic.

Right, especially coming from PHP without a framework.

Yeah.

One of my first two weeks, I did the tutorial,

And I was like, yeah, I've been using it since.

And you've been a MySQL fan ever since too, right?

I mean, because Memrise was dealing with a lot of data even back then.

And so I'm sure you got to roll up your sleeves

and feel some real database scaling fun.

Yes.

I remember I had to sit down and read the MySQL book for scaling websites

called High Performance MySQL.

And that's by some of the key people in the MySQL community,

Peter Zaitsev, the CEO of Percona, who are a MySQL consultancy. Just digging through that.

We used Percona at Quizlet. We had them at the highest tier of retainer and

they were very needed.

Yeah. We hadn't quite hit that scale, I don't think, when I was working there,

but we had a lot of data. One table was 100 gigabytes.

Oh, wow. Well, we also had, again, growing pains. It's been a long time in Quizlet's

big and so i don't think andrew reminds me telling this but at one point he we didn't really have

much permissions i mean we had two developers at the time or when i first joined two developers

and andrew went in to delete a record and dropped the entire database by accident um so that was

that was a fun five six hours and we called it percona like hey we need some help here you know

those backups you said we had yeah yeah no but i was i remember they set the gold standard for

a consultancy because we had, you know, generally scaling or whatever challenges and we weren't,

you know, experts and they, it would always be someone different, usually in Eastern Europe,

and they would just almost immediately be able to diagnose and fix the problem. It was pretty

miraculous, actually. Yeah, one of those things where they just know everything about the system

inside out. I definitely got the impression reading the book and their blog posts that

they know everything there is to know about mysql yeah i mean we almost couldn't pay them enough it

was just you know even though it was a lot it was i was just blown away that they always knew it and

i i guess it makes sense though i mean they're consultants they've seen you know that was our

first time building a site at scale but they've been through it so the issues were not unusual

so adam like you you mentioned you're on the you're part of the core team and the the sort

of the thing you're known for there is working on the orm and working on the mysql side and you

maintain a Django MySQL package as well so could you tell us a little bit about that what that's

what that does and what that brings to the table yeah the package is how I got invited into the

core team and so the package is unoriginally called Django MySQL and we were using MySQL at

wide plan as well. And I saw Django Contra Postgres come in from Mark Tamlin's great

work and the Kickstarter there. And I got a bit frustrated that MySQL could do some

of these things as well. And we had some use for this feature or that feature. So I first

developed my stuff in-house and then I was very gratefully given the chance to open source

it under my own name. And then it kind of snowballed from there. I tried to cover as

many features as possible i found it quite enjoyable just writing django fields and things

like that making a package well tested and then at the third django under the hood i had quite a

long chat with josh smeaton who's also a core member and a few months later unexpectedly i

found myself invited into django core it was a very nice experience just seeing everyone who'd

voted for me and i'd had some impact with my work okay yeah that's super and so so my story i began

with php and mysql back in the day and i was using it very happily for a long long time and then you

know the whole um contrary postscript uh postgres thing came along and i started using postgres

because i wanted the extra functionality that was available and all of a sudden over the course of

two three years mysql just dropped away and i think now for me personally and i think postgres

now is in a position where it has a lot of the mind share but it's not necessarily the most well

it may be the most capable database or not but what is mysql got to offer for us that perhaps we

you know we're in this django world where django comes packaged with this extra functionality for

postgres why should we be considering mysql because i think that's a great topic for users

yeah i think mysql has had a bit of a troubled past after its acquisition by oracle because

Oracle have their own database as well, so they've been seen as not focusing on MySQL's

development and it definitely lagged behind feature-wise from Postgres in many respects.

But now the ecosystem there is a bit stronger.

So Percona, who we were talking about, maintain their own fork where they have a number of

fixes that they've merged in from their consultancy experience.

And then there's MariaDB, which is a completely hard fork at this point that implements features faster, has a faster release cycle, and is effectively the Postgres of the MySQL world at this point.

It's fully open source.

And if I'm a user, should I really be looking at MariaDB?

At this point, yes, in my recommendation at least.

If you're maybe more of a corporate user,

you might still want to look at MySQL

because Oracle's bigger

and maybe it'll be easier for you to get a contract with them.

Right, okay.

You can get a support contract, that kind of thing.

Yeah.

But I think MariaDB's in a better position going forwards.

The place that I think MySQL shines over Postgres still

is in replication.

So Postgres has had a load of different replication methods in the past, and then they've

eventually merged one into the core after changing their minds that replication should be something

the server should do. Whereas MySQL has had it from the start and is a lot more flexible.

One use case I've seen a few times is having a replica of your master database,

but still being able to write to it on perhaps a different table and you can't do this in postgres

one of my clients who i'm with right now they just like dump and load the database every hour or so

into a replica so that they can have a a writable replica

the take-home there would be that for scaling up for big data you think mysql's got

is more competitive in that area yeah i think there's probably more big distributions of my

school in the world than postgres by quite a lot of yeah i mean i don't know orders of magnitudes

like facebook is mysql youtube is mysql i don't think last time i looked none of the alexa top

100 were postgres right okay and that's got to tell you something you think yeah yeah okay

interesting but maybe that's also just like like sampling bias right like maybe these companies

started to exist and scaled up and fixed mysql on the way and if you wanted to create a big company

you could also use postgres and fix the problems on the way okay and so out of the features that

contrary postgres gives gives us what does um django mysql give us i think it about matches it

um the key thing is the json field which um you and i are both mentoring for uh an implementation

in core that works on all database back yeah so hopefully that will be in 3.0 it's it's close we

should get there hopefully fingers crossed yeah but uh when i started like postgres had it my

well maria db had just implemented it and my sql um but it wasn't clear that there's like complete

overlap with all the databases yet and so if you're on uh mysql or maria db today and you

want to use a json field then you can use jagger mysql yeah okay and i mean the thing is as well

like um anything that's in um the core orm is going to have to be support all the four

database going to have to support mysql and maria db postgres oracle and um sql light or sqlite

um which maybe it can't do but in reality your project isn't using four databases it's using one

so if you're on mysql or you're looking into mysql you can use your package right exactly and there

are a few little extras in there that will never work on other databases because there's some

feature that mysql committed ages ago yeah okay one thing i found useful is the user locks

so you can use mysql or maria db as a locking server with a string so you can create a lock per

say user if you need that and and in postgres they have such a feature but you only get to

lock by an integer between one and four billion so even having two applications on the same

database server becomes a property yeah okay yeah uh this is my own ignorance but does

MariaDB MySQL have the same sort of full-text search support that Postgres has built in?

It does not have exactly the same. There's Sphinx, which I think you can use under MariaDB,

but I've never really tried. It has a basic full-text search, but it's definitely more

basic than Postgres is. Yeah, I mean, back in the day, I used to use that. That was like kind of

the the okay first well poor person's you know search that you could use it and it worked and

it was not bad and if you didn't want to scale up to you know a proper search engine well i i asked

just because i'm giving a a talk at jango con um this fall where i'm gonna i need to learn a little

bit more about the postgres options because i'm aware of them in the abstract but i haven't

implemented myself so that's sort of an excuse to to get to do that always a good idea with talks

to be like, I wish I knew more about this.

Let me propose a talk on it.

Well, I think I have an open ticket on Django MySQL

to add some of the extra search support

like Django Contra Postgres has.

So if you want to look at that.

Right.

Actually, I have one more question.

Since you're both core,

you guys mentioned Oracle still gets support.

Who's working on Oracle in the open source world?

How does that keep up with the other ones

where you have folks like yourself adam who are doing such such great work that's sort of an open

question like is is is it just sort of chugging along because i don't get the sense that anyone

on their free time is you know keeping oracle current with django but well no well like marius

is our sort of um who's the other django fellow at the moment he did he um he's our sort of oracle

superstar he knows it really well he knows all the different versions and the new versions come

up and he makes sure it's um maintained and he you know if an issue comes in that you know there's

some incompatibility with oracle 12 point whatever and he's in there and he knows the details and

you know without marius we'd be in trouble there um he really he really takes that on

okay we haven't had him on the show yet i don't know him personally but i assume there's one or

many people uh doing that work to keep it up so i was just curious who they were i think at this

point it's one i know josh smeden used to but changed job and then doesn't want to touch

oracle again effectively i think that's what oracle does does to people no but like we we

have to get maris on now because we're you know we're not going to defend it but he will defend

it and he he's like look it's not so bad it's nice he's got these things and you know he'll

tell us all about what the positives are um well and shouldn't oracle fund django a bit as the only

Yeah, don't get me started.

Yes, they should.

Yeah.

Okay.

Yes.

I think you're more likely to see Microsoft funding

before Oracle funding.

Well, on that note, hosting,

there's a bunch of questions we wanted to talk to you about.

So I think in particular, you know AWS, right?

Is that the one you...

Yes, AWS as in Amazon.

Yes, Amazon, not Adam.

So we've done an episode or so talking about...

talking about this but what's your take on amazon web services versus um platforms as a service like

koroku or divio how would you if someone came to you for advice how would you advise thinking about

that and then maybe we can get into some horror stories or production stories of how using them

my general advice these days is that a pass is is the solution people are looking for and

And Heroku, DBO, they work great out of the box.

And Heroku and DBO have both also contributed to Django

and definitely keep their integrations working

and their tutorials up to date.

There's also Elastic Beanstalk, which is the pass from AWS.

Yeah, they changed the name of it.

It's something, I think it's something else.

There's a newer Elastic Beanstalk.

Oh, did it?

Yeah, but it's the, I'll look it up.

But it's, yeah, Amazon has a pass option.

Yeah, I've heard less good things about that, like more mixed reviews.

I know Christian Glass, who's a relatively active Django blogger and user,

he raves about it, but everyone else I've spoken to is like,

I wouldn't use it again.

I've personally found the docs very opaque,

and it's not the platform-as-a-service story that I found elsewhere,

you know, with Google or with Heroku or with Azure,

you kind of it's quite simple and i found that personally my personal experience with elastic

beanstalk was i'm running through treacle here like trying to get make progress and at that point

i'd just rather spin up an ec2 and yeah do it the old way so they i found it so actually well

this is for anyone listening um why you should blog so i i blog you know django hosting things

and a post that i wrote came up which i totally forgot about so it's called aws codestar that's

there. Yes. That's like the integrated, it brings in the code pipeline CI service and the code

commit to get hosting. Yeah. So I think to Carlton's point, they're trying to be more like

Heroku-like option. But yeah, so AWS, I mean, just using S3 or CDN or CloudFront, I don't even know

where anything is. I mean, the underlying tech is so great, but the user experience is just garbage

for me how is it as a power user do you know where to go because and they change the interface every

six months too so i just every time i use it personally i get incredibly frustrated but i

don't work on huge projects myself so it is a bit of a maze and um like with what 300 services or

something i think the only same thing is to maintain a kind of white list and like just

just be like, we're gonna only touch these ones.

Yeah, yeah.

Oh, exactly that.

So at timeout, I was in charge of the DevOps team.

And over the course of a year,

we migrated 26 applications to AWS, so one every two weeks.

Oh, wow.

And the main way we did that was with EC2 instances.

We had a bit more of a poly-lingual environment.

So, some Java, some Scala, JavaScript, PHP, and Python.

So, we kind of set out the ways to do things

and stuck every application through a very similar,

similar setup.

The main way we achieved this was infrastructure as code,

using the AWS CloudFormation tool.

So we could just copy paste the setup

one app to the other as code and a lot of it ends up being a bit of like cargo coating like

we know that we need this resource in that resource but we only know it because it's in

that other template um but like you don't like so um you know interesting you say you use cloud

formation rather than um i know chef or ansible or whatever because is it answerable okay because

it's sort of tied into the amazon system then right um but it's exactly like that if you've

got an ansible role you're just going to reuse that role you're not going to necessarily dig

through the details every single time because that defeats the purpose that is meant it's meant to be

kind of pluggable i know i need i need a cache i need you know a database i need like now here's

my ansible script deploy it brilliant it works and maybe it's not 100 tuned for that particular

service but it only took 20 minutes versus you know days it used to take to provision machines

by hand and install the right dependencies and you know i wouldn't go back to this yeah i think

it's um a bit of the same like there's so much on offer in terms of services that you do have to kind

of lego brick it up like package it um but that is why it's so overwhelming to begin with and i

I think many of the decisions in terms of design are aimed at capturing some corporate client's particular use case and they don't really care about the average use case.

what I think of is like when you create a lambda function you can hook it up to a service but you

in confirmation at least you also need to declare the middle thing which is the permission for the

one service that you provision to talk to the other service that you provisioned and I think

that should just be the default like why do I need to give permission to things that I already just

told to do things so like I am I am and permissions and access policies and it's that's where it gets

to be like oh this is really hard now like you know you get through it and it's but it's

it can be a slog but you know it's powerful right super powerful so what was the when you're

switching over maybe for for listeners who haven't ever done that kind of switch so you

yeah you have it on aws i guess maybe just like the dns part how do you know

so you have a duplicate copy how do you feel comfortable you can switch over and then what

is that switching over process like um i assume is it just the dns what what else you what else

are you doing and yeah one of the key things that we would do for every switchover is build a

checklist and discuss it with the team try and get as many points in there and it depends on your

uptime goals so if you're happy with your website going down for an hour then switching the dns

can be the easiest switch over and the most complicated involves like moving each individual

service inside inside that so maybe you move the database then you move the cache service

okay and you maintain like a network connection like a vpn between the old data center and the

new one um yeah and as for like checking that a copy is as good as the one that's running

and the best way is to edit your etc hosts file to point to the new dns record okay effectively

and browse that see that it looks right and maybe if you have a list of the top 20 urls to check

or someone who uses the system to step through any automated testing also helps

and then always do it on like a friday afternoon right before the weekend so just after you've

Right, exactly.

You just press the button as you walk out the door.

What about, do you have any experience with containers for deployments?

I only ask because for my new book, Django for Professionals, we use Docker and then

we use containers on Heroku.

And this is a growing trend that makes a lot of sense to me, but I don't know.

Is that in the AWS world?

Can you do containers within AWS?

Yeah, there are a number of solutions on AWS, and the naming of them has gotten

some ribbing as well.

I think one was Elastic Kubernetes Service on Elastic Container Service.

Classic.

Classic.

Yeah.

Yeah.

I personally have avoided containers up to this point.

Timeout would have been one point where we reconsidered it, but ultimately went down

the EC2 route. And we looked at using EC2 with the same benefits that containers tend to bring,

which is like a frozen image. So a new instance is faster, create things configured from like

variables outside of that server. So you can use the EC2 user data, much like you can use

environment variables in Kubernetes without so much of the complexity and the tool chain.

VMs and EC2 in general, any VM provider, they're pretty stable in what they offer.

Whereas I find even like every six months, there's some blog post that tells me something

brand new about containers that I know I'd have to rework everything just to keep up.

Yeah, it is definitely a moving variable.

But I mean, and then at the same time, Kubernetes is clearly built by and for big companies

with their needs.

So, I find that the leap from a couple of containers to Kubernetes is a really hard

one to make.

Yes.

And I understand you need tools on top of it, like Helm, to really make it usable again.

Yeah, I think there is a bit of a trend towards AWS and containers being the kind of IBM decisions

of our era.

So, nobody got fired for choosing IBM.

nobody got fired for trying containers because that's what everyone's doing right

that's sort of a horrible thought but probably true um well it's also i i mean just you mentioning

your experiences and so much on the devop side speaks to something carlton i have mentioned on

this podcast which is that on the spectrum django is actually kind of a front-end framework um it

sort of doesn't really you know at massive scale django or another framework isn't the issue in

terms of scaling up it's the database it's all these other things that are yeah very separate

from your programming languages and your frameworks yeah definitely um i think one of the

questions i've come across on core recently was simon willison's answer to does django scale

and the answer was it's just like any other framework that reads the database

it will go horizontally it's going to be the database that's it yeah well he said as much

on the podcast and we had Andrew Godwin

on as well. It just came out

today as we record.

I know Simon sort of flippantly said, oh, it's sort of

boring how you shard and go

horizontally, but I think his

definition of boredom is different than most people's.

So, switching gears.

Hang on, just before we switch gears, there was an old

magazine when I was

cutting my teeth. The hardest

thing you go is from one server to two.

When it's all on one box,

you know that's great you can learn loads and then you reach the limit of that one box and

you have to scale to two boxes and that's that's like a whole life-changing experience but then

the third box that's just the same as what you did to get to the second box and yeah you know

i think i think that still holds true something for me to look forward to i haven't gone off

gone off the one box i think there's also the quote there's only three numbers your computer

program should take as limits, zero, one or infinity.

Yeah, exactly. So testing, I want to talk about testing, because I know you've written and spoken

about it. And it's a question we get a lot about. And maybe specifically, we can talk about PyTest.

So for folks who don't know, this is, I guess, in addition to built-in tools that I think you've

worked a lot with that a lot of people in a professional setting use pytest but maybe people

who are learning django don't understand what it is or why you would use it over unit tests in the

built-in django test runner yeah i'm a big fan of pytest and it's definitely the more pythonic

way of testing and i remember learning django's unit test and having done a lot of

java at university i thought it was quite natural but then i came to realize it doesn't really fit

fit in very well with the rest of more like idiomatic Python

code that I was writing in my views and models.

So I think PyTest is the only way

I would write tests these days.

And for Django, you install the PyTest Django plugin,

and it hooks into your Django settings

and configures the databases just like Django's test runner

does.

Yeah.

I mean, I would agree anecdotally.

almost everyone i know running a production site uses pytest carlton does that ring true for you

yeah well like so i'm sort of in a kind of halfway house and i use pytest um as the the testing

framework of the pytest command i use that every time and i love i absolutely love using plain

python asserts because i can never remember whether it's self-assert you know has more than

or less than i can never remember those 52 methods i can just about remember assert equal assert true

you know assert num queries is kind of handy in django land um but i'd much rather just write

assert some python expression and then you know a message that's kind of handy um but the thing i

still do quite like and still use a lot is um unit test test cases because i like them to group

use them to group so i'm using pytest to run them and i'm using plain asserts inside the test cases

but I'm still using that unit test class for the grouping that it gives me.

And that's just sort of where I'm at.

It's not a religious thing.

It's just kind of where I've reached.

I haven't delved into the further realms of PyTest with the fixtures

and the parametrization and those kind of things,

which all look really cool,

but just one more thing for me to learn at this stage.

Well, Adam, you have a nice blog post on speeding up Django test cases, right?

Yes, I wrote this post, what, last week?

Oh, is that right?

Okay, yeah, I saw it.

Yeah, I guess it's the 15th of July.

Yeah, so we'll link to it.

99 days ago.

We'll link that in the notes.

Yeah, this was with a client recently, EV.Energy.

They're an electric vehicle charging optimization company.

I hope I've done them justice in that explanation.

So they have a not huge Django app at the moment,

But it had tests running for, what was it?

It was about six minutes, and I got them down to two minutes or so.

And the main way of doing this was by changing the test case class from Django that was in use.

So for historical reasons, they'd used the transaction test case,

which I think is slightly badly named.

And this has more complicated database rollback behavior than the test case, which you're recommended to use generally.

And they'd use that because it fixed a bug in their testing.

It made sense at the time.

And the speed difference would have been noticeable when they were just maybe a few hundred tests.

But now their test suite had scaled.

It was now taking, um, a not insignificant amount of time.

Six minutes is still pretty fast.

Yeah, well, that's true.

Right.

It's, it's pretty fast, but it's also long enough to be a nuisance.

So yeah, it's, it's go make a cup of tea or browse Reddit time.

It's not just wait for it.

Um, so, uh, you can go read the blog post and see why it made

a difference to switch.

to test case for most of their test classes. But I really wrote the post because I thought

this is a great story of where to look in Django testing to have fast tests.

Yeah. And related to testing, I know you've also written on

Coverage, I think, which is a fantastic package for people. And actually,

Ned, who maintains it, is here in Boston. I need to get him on. But could you talk about

coverage. How would you describe coverage to people who haven't used it before?

Sure. So coverage is when you run a program and you keep a record of which lines were run.

This isn't so easy in compiled languages, but Python makes it relatively easy. And

Ned Batchelder, who creates the coverage Python package, has maintained it for many years at this

point, maybe 10 years or more. So when you run the program with coverage, you get to see which lines

have run. And so the main use case for that is to create a test suite and check which lines of your

actual program, in our case, a Django app, have been touched by the tests. And the goal you can

set is to try to get to 100% coverage. That is, every line in your program has run.

Well, do you think that's a good goal?

I think it's a good goal. And in fact, I go one step further with the branch

option, which is every branch has been taken both ways. So if there's an if in your code,

it must go through both true and false. So you can't have a line of code that does like

if something that's only true in your test suite oh so you might have like if user is admin yeah

and then to run all your tests with admin but then you haven't tested the normal user

experience of your website this is the mind of a django core contributor people

that's fantastic i didn't know about that i love that yeah um so my build post guides you through

how to set up Django for coverage testing two ways,

one with the Django built-in test runner

or one with PyTest,

which you install the PyTest Cove plugin,

which sets up coverage.

I think everywhere I've worked at

has had not 100% coverage,

apart from once when I worked on an app on my own.

Well, because there is an argument

that it's sort of a goal that isn't worth the last 10%.

and so it should just be used as a metric of that's kind of why i asked i've heard people

espouse that theory that you know the last five ten percent isn't worth the time i think

yeah i think it depends on the last five to ten percent of what we discussed

if you're building a nuclear power plant i sincerely hope you have better testing than

just 100 coverage you know you're testing 100 of the scenarios hopefully but this is the

the software quality versus cost issue right so everyone's always like oh software engineering

doesn't deserve the name because it's impossible to build reliable software it's no it's not

impossible there are standards you know nasa have these these standards which you can use but they

cost five times as much to build the software so you take that to a company and say hey we're going

to build it to nasa standards they're just going to throw it out the door they want it shipped

quickly and it's got some bugs it doesn't matter um you know it's the there's always that well same

thing with uptime going from an hour to you know a second or something is exponential in cost and

yeah like one nine no problem two nines yeah no problem six nines that's hard yeah yeah um well

and related to security i also wanted to get a chance to talk about your talk you gave at jangle

con europe this year on security headers because that's i liked how you which folks was fantastic

which is fantastic a couple people before you had said security is boring and you emphatically made

the case that it is not so perhaps you could give the highlights of of that talk sure um so the talk

which has a corresponding blog post um goes through how you set up seven different headers

on a django site to make it more secure and this is to pass the grade on a website called

securityheaders.com and which is by a security researcher called scott helm and will help you

get like some assurance that you are keeping your users secure security can be seen as boring

because it's often like checking some boxes or re-auditing things you've already written

but i like to think of it from the perspective of a hacker and i kind of got into web security

as a teenager on a website called hack this site.org which i checked recently it's still up

it teaches you like the basics of all the different attacks hackers might run and then

these security headers that you can activate in Django four of them come out of the box

they help you protect against all these relatively basic attacks and if you're running a Django

website without these headers on you're definitely at more risk and they're very well known simple

attacks that you can protect against yeah the defaults there are these are things you have to

turn on or these come turned on automatically and so some stuff is on by default in django and

i think of the headers is only the x frame options that is set by default yeah if you run the

deployment checklist you'll these these will spring up but most of them and this is only top

of mind because i just wrote these chapters for my book only most of them are not are turned off

by default yeah and i opened a ticket after my talk to consider turning them on by default

from Django 3.0.

How's that going?

Because that sounds like a good idea.

Did we accept that?

Yeah, I haven't got around to that,

but I think Marius took a stab at it.

Well, I think the challenge for maybe

beginner-intermediate folks is setting up

the local and the production environment for testing

because you want different things in both environments,

and that leap, do you do it with containers?

Do you do it with different settings files?

It's not the responsibility of Django core

to explain that, but I think that's probably the hardest part of doing that because everyone

does it a little bit differently.

I think some of these headers are perfectly safe to activate both locally and in production.

So definitely like X-Frame options is one where there's a default and Django can just

switch to the more secure default.

XSS protection is another one that just helps the browser protect you against injection

attacks.

I'm pretty sure there's nine things

that in the deployment checklist you need to switch.

And one of those, I think two of those,

it's like debug and allowed hosts,

and the other seven are largely header related.

Yeah, there's all system checks for these

that if you run manage.py check with the deploy flag

with your production settings, you get the messages.

But I've found a number of my clients slash friends

have not gone through the deployment checklist when deploying.

So are there any other projects or packages you want to give a shout-out to?

I have a projects page on my site that describes some of the things I make.

Aside from Django MySQL, there are a number of smaller packages that work with Django.

One of them is for a security header.

That's Django Feature Policy.

That's for a draft security header called Feature Policy

that enables disabling features in the web browser that you don't want to use.

Like webcams, say, or something like that.

Yeah, exactly.

So if someone injects adverts onto your site,

you're not going to let them ask for the webcam of your visitors.

Sounds like a good idea.

Yeah.

Those are the main things I'm working on, you can see, on my projects.

And then how are you finding, I'm always curious, life as a freelancer?

because you've worked at companies obviously the mix works for you how would you describe the

difference between those two lifestyles why makes it switch yeah um i made the switch in january

mostly because i wanted to explore the ecosystem a bit more i might say so um i find it quite

stimulating to jump between different projects and help people out in various capacities and

And it's quite interesting in that regard.

Also, I think it's helping me make some better decisions for Django Core.

I'm no longer stuck on one big website that I'm thinking about.

Like, oh, most users are ending up not doing this,

like not activating these security headers.

So hopefully I can feed that back in.

The lifestyle-wise, like at the start, I didn't have much work coming in.

I found it a bit stressful thinking about how am I going to pay rent this month

for pay for that but on the flip side like it's a lot more flexibility and freedom and when work

does come my way sometimes i'm turning it down these days yeah and the the interesting thing

with freelancing is the pipeline is that you've got to always see what's coming up next and you

you know if you get sucked into a current project and don't work the pipeline you're suddenly the

project finishes and you've you've got some money and you've got a nice break but you haven't got

work coming in it's like ah i needed to be doing the pipe the client development the business

development work whilst the other project was still going on and learning to manage that is

you know it's it's it's the freelance but it's the business side of freelancing yeah well there

is the middle there is a middle ground i um where you you work for a consultancy so they sort of tee

up the projects for you and you sacrifice a little bit i guess in terms of pay but you don't have

that you don't have to spend all that time on business development and marketing and all the

rest that does you know it is it can be challenging right to spend all day teeing up projects and not

actually coding and not actually getting paid yeah i've got a number of friends who do contracting

that way in london it seems to be quite a popular way of moving developers between projects as well

and a lot of companies are engaging with that yeah great well adam thank you so much for taking

the time to come on and share all your work on django with us yeah super thank you thanks for

thanks for coming on adam that's great oh and we should mention carlton for people want to listen

to the podcast you can find it on your favorite podcast player it's on django chat.com and if you

have feedback you can leave it on the site or on twitter we're at chat django so again adam thank

you so much for taking the time thank you adam thank you very much for having me