Transcript: Caching

I'm Will Vincent, joined as always by Carlton Gibson.

Hello, Carlton.

Hello, Will.

And this week we're going to talk about caching, which is a power tool of all developers, but may not be familiar to uh folks newer in their career.

And Django has some fantastic built-in support.

So we're going to get into all things caching.

So, Carlton, what is caching?

Why is it important?

Caching's good if you uh, you know, say you've got some database query that takes quite a long time and you're doing it all the time.

Maybe you want to cache that.

And that just means keep it hanging around so you don't have to make it again.

And ideally, your cache gatefetching, so if you store a result that you got from the database, you can put it in your cache, and it ideally it's quicker to get it back from the cache than it is to go to the database.

That's the idea.

Yeah, and that general idea that rather than spinning up the physical disks, if you load it into memory like the RAM, it's going to be faster.

That's the general idea.

Yeah.

Um if there's a I know you're rendering a complicated template and you've you've taken, you know, you've gone to the database, you've got some data, you've rendered it into a template.

There's no reason why you've you've got then some HTML out of that, which was computationally expensive.

There's no reason why you couldn't cache that on the disk, because then all you've got to do is fetch it from the disk and serve it straight away rather than do all that heavy computation before.

But normally, normally we cache in RAM.

But that's the basic idea, is you you pre-process things so they can be loaded faster.

Um and I I mean I think something that's uh maybe a little confusing is it's such a broad idea, caching.

I mean, so if you index a database, if if folks have heard of that, that's basically a cache.

But then you can also in Django, you can, well, for example, Django has a built-in caching system uh where we'll get to where you store it, but there's four options to give folks a sense of how to think about this, where you can do per site.

So you can just cache everything.

So if you have a Django site, but it's basically a static site, if it's a blog that's not changing, you can just do a, I think it's basically one line, we'll link to this in the docs, and just per site cache everything.

And so after the first time, the first time it loads, someone will come in, hit the site, it actually needs to process, but then after that, everything is just served from memory.

Um and actually on that topic, let's talk about uh refreshing the cache hot and cold, because I think that's an important thing.

Because there's this idea that you know cache has to be run, it has to be hot.

So for example, if I've changed, I have a blog, simple blog site, I add a new blog post, the first person who comes and hits that site is going, is not going to be cached even if I have caching run.

So either I just say, Well, the first folk, first person who comes in is gonna take that performance hit, or you can um heat, you can what's the term?

You can run the cache in advance.

You can pre-heat the cache, right?

So what you can do is something when you publish your own blog post, you can go and check that it appeared on the site.

So normally you cache per page, right?

Like Django's got this nice page caching option where you like each individual page has a so-called.

So yeah, so the four options per site purview, uh template, which I guess is what you mean by per purchase.

I think so.

I can never

remember that.

So this is something you want to play with in reality, so in production, like you can make predictions all day long on local usage, but you really just want to see how your site actually performs.

But I would say with God, we're just butchering this, heating up the cache, warming up the cache.

I played around with this a ton.

And on a big site, it's worth it, maybe, but it's also fine to just say, you know, if I have thousands of visitors, the first one on this page when I do a change, it can be a little slower for them and they'll live.

So the pay, however long it takes to go and get your blog post out of your database, to render it into the template, put it on the page.

Right, okay, the first time that's a bit slow.

And then the index page needs to change as well, because that, you know, the one where it lists the first five blog posts, the most recent five blog posts, you need to update that.

So questions about invalidation that we can come back to in a minute.

But so that first the first person who logs that is a bit slow.

If that's you, brilliant.

Well, I I did that.

I I manually, again, this is early days a startup, I would manually manually go through and uh reheat God, uh, these pages.

Um, but you know, in practice, when you're dealing with hundreds of thousands of users, it all isn't.

I don't think you want to do that.

I mean, you could if you're updating it all the time.

I recall setting it for a very long period of time, though off the top of my head, I can't remember what a long period of time is.

I think maybe it was a a month.

Oh, right.

I mean, I've I've uh the thing is and

Even an hour, right?

It's really not gonna kill your Django application, right?

But if you've got 20,000 people all at once, that will kill it.

So if you can cache that blog post even for an hour, it means that the the Django app is only really doing the hard work once every hour or once every day or once every week.

Um yeah, I think so so there's a timeout, so there's arguments you can pass into the cache uh caching framework built into Django, and I mean the docs give an example of um 300 seconds, so five minutes as you know, substantial period of time.

Yeah, I I think all that what I said was way too long.

Whatever.

This is why you want logging and other information too on your site, so you can see actually how fast the page is loading.

Um, you know, it's a balancing act, basically.

But in general, cache everything.

Because um Nginx will serve files off the file system, you know, far more efficiently than any application you can ever write.

And so if you've got a blog post which perhaps you update, I don't know, never, why not tell Nginx to cache it on the file system?

And then it's just like Nginx, for Nginx perspective, it's just like it's serving a static site and it's like not even talking to your back end.

And that's really quite easy to configure.

You you give it a path and you say, look, file cache, and you you give it the module, the the amount of time you want to cache it for, and it'll just do it.

Um so that's worth considering.

I remembered um using Varnish, which is a a proxy cache layer um when I was doing this all on DigitalOcean.

What's what's your take on Nginx versus Varnish?

I mean you could you could use both, right?

Because they do different things.

I mean, I remember Varnish was like a huge speed bump, maybe the biggest of all the things I did.

So your what's your what's your base setup?

Your base setup is you know, just for example, I mean you might be using Apache or whatever, but let's just go with one example.

You're using Nginx, we're gonna call them with Django, okay?

You've already got Nginx in play, and it's it's got first grade um caching module that's really easy to configure.

You can use that, and that will that will really will cope with you know, probably 90% of sites out there.

That's perfectly good enough.

And then if you are really pushing it to the limit, then you're going to investigate whether or not you need another dedicated caching layer on top.

It sort of scratches that developer itch, but it is, I'm certainly guilty of spending way too long getting that last five, 10% when it was totally unwarranted.

So it um I would say be aware that this is fun and feels binary.

Um, and so a lot of times you'll you know neglect things like talking to users that is a little more gray.

Um, okay.

So where do you put the cache?

So let's talk about so it historically, so memcache was the well the first big popular caching layer, though these days I think Redis, almost everyone would say Redis.

If you're starting from scratch, you would use I

It's got some fancy.

I there I've seen we'll link there's a detailed analysis.

I believe in most cases it's actually faster.

I mean, but look, back in the day, and so we're talking um, you know, early 2000s when memcache was the option.

You'd run memcache, you'd be there even into

Because it came out of I'm sorry to interrupt, but yeah, I remember like it came out what like Live Journal or something in 2005.

Like it was, I don't I think it was like the first major uh caching of that type.

Um and still per brilliant, right?

It still works and no reason what not to use memcache unless you're already thinking about using Redis.

And again, it's like how many components do you want in your stack?

So if you've got Redis running, why not use Redis as a as a cache back end?

So I guess yeah, the general thing, um, memcache is a little bit simpler, but if you if you need if you're gonna need Redis things anyways, you might as well just use Redis for all of it.

Um well, so let's talk about those things.

Why would you use Redis?

So I mean basically for any Q like task, so email is one example.

Um what are some other examples that come to mind of when you would so so I guess we're I'm confusing two things here.

So there's caching, and then you'd also use something like Redis for Q based.

Yeah, because you want to you want to use a queue.

So let's take a good Q package.

So you you know everyone always talks about Celery, but celery's overkill for you know the majority of use cases.

So what's a good package?

That's nice and simple, and that's got a Redis backend.

So you'd pip install Red, or you know, apt install Redis, and then you pip install Django Q into your project, you know, a little bit of settings magic, and you're up and running.

What do you put in there?

Anything that you want to put out bad.

So you know you're rendering a PDF.

You're something that's gonna be process intensive that would

You're doing any of these tasks that we talked about, we talk about all the time, and then you've got you've already, at that point, you've got Redis in play, so you might as well use it as your Django cache backend, for which you'll need a couple of packages or a package.

There's a couple of options, right?

There's Django Redis and Django Redis cache, and I can never remember what the difference is between these two ones.

Every single time I start a new project, I have to go and search for history.

What did I use last time?

And is it still as good?

There is a difference, but it's I can't remember either.

Like I so I I was looking this up before we started the talk.

Last time I did it, I used Django Redis cache.

I've been very happy with that.

It turns out I've used that loads of times in the past, but I've also used Django Redis loads of times in the past, and I have no idea why.

I don't know which one's good.

I think the general the state of play on that is yeah, we we keen on that, but it needs a Django enhancement proposal adept, it needs someone to step up and and write the thing.

Um but in principle, in a you know, two, three, four versions time when someone's actually got around and and written it, there might be Redis cache backend in Django itself.

I mean, like, you know, uh just the the one thing we haven't talked about, it's not just the pages, but the template fragments.

Sometimes templates are computationally expensive to render.

And if you've got, I don't know, let's say you're converting um user-submitted markdown to HTML.

Okay, first of all, you've got to render that as HTML using markdown, and then you've got to run it through a sanitizer like Bleach, which is HTML5 lib, which is not necessarily the fastest library in the whole world.

You if you can cache the output of that rendering, then the next time you have to do it, I mean you could cache it in the database.

Say say you've got that markdown stored in a model field, you could have an extra model field for it for the rendered HTML.

You could do it at save time, but equally you might do it by caching the template fragment.

If you just flip around the switches for per site per view and just see how much faster it is, I mean it is orders of magnitude faster, obviously, to serve from a cache.

So I would say that would be the way to play around with it is just just simply Django debug toolbar, um, and then you can there's more complex tools to see how fast in production your pages are loading.

So you know, if you can

Is is there's actual time, the time your Django application took to serve the response versus the perceived time that the user had on the endpoint.

So, you know, let's say your and your Django application took 300 milliseconds to go to the database, render the template, serve the response.

You know, that seems is that fast, is that slow?

Who knows?

But let's say you're loading, you know, two megabytes of JavaScript, which took two and a half seconds to be responsive and to load on the on the client, the client isn't going to notice if you halve your response time from your Django application.

They're just not gonna notice because it pales into insignificance.

So quite often you'll see the front-end people talk about this a lot.

So, i.e., how much JavaScript are you loading?

How much how many images are you loaded?

I mean, the images aren't even the thing, it's JavaScript.

How much JavaScript are you loading?

How long does that take to pull into the page?

Especially if you're doing one of these um uh single page applications, these um client-side rendered things where it's got to load all the JavaScript, then it's got to pull the um data from from an API, and that's the bit where your Django app does its thing and it takes 300 milliseconds.

And then it's got to render all that into the page before the user says, Oh yeah, the page loaded.

And that whole perceived time, I mean, it reminds me of it.

So Instagram back when it came out, um, because I was actually working at Quizlet, like just next door to them.

Um one of the things besides filters, one of the things that I remember being a wow moment was so this was still when the uh cell reception in San Francisco was terrible.

A lot of places, what they did is they, as soon as you you picked a image you wanted to load and start typing in all the information, in the background they started loading it.

So it felt really fast.

You didn't, you know, press the button and then wait five, ten seconds.

It felt instantaneous.

And I'm sure some others had done it, but that was one of the first apps I saw that you know basically said, We're gonna we're gonna blow up your bandwidth or your cell phone plan.

But it you know, in the background, and now that's a standard process.

Anytime, I don't know, Tumblr or something, your or Instagram still, I'm sure you know, when you're loading an image, first thing you do is you load the image and then you type in a whole bunch of stuff in the background, it's already processing, so you can just click the button and go load it.

And I think the reality of cache, you know, you can you can Django gives you these caching tools and you can use it to speed up your Django response, but in a lot of cases, the real work is on the front end.

And you know, do the basics in Django, get it, get it performant, use that Nginx caching layer that we talked about, but don't sit there then worrying about micro optimizations when you've got a front-end app that's likely to offer better return on investment for that optimization time.

And I guess in the last I guess major point I would say is it really it it depends.

It depends on the type of app that you have.

How often is the data updated?

Is it personalized for every user?

So for if you think of Facebook, you know, every you and I log, not that I have Facebook, um, but if I did, you and I log in, there's different content being loaded there.

I'm sure I know that in the background Facebook is periodically loading those things into cache.

So when you log in, it's there.

But how often does that change?

If you have a timeline feed or Twitter, right?

I mean, that's updating quite a lot.

So that would be a little more challenging than a blog or something that doesn't update as much where you can be a lot more uh aggressive with the time limits that you uh caching.

It's it's like how aggressively can you cache it?

Is like for a static blog post, and do you have a mechanism for invalidating it, right?

So let's say you've cached a blog post in your you know, Redis, whatever, using the Django backend.

Are you able to identify that by um key such that when you update it, you can, you know, use you can when you in your save handler, wherever you put that save handler, you can say, Oh, and invalidate the cache.

So there's there's two problems in computer science, right?

Naming things, cache invalidation, and off by one errors.

Yeah, yeah.

So often you are finding yourself, you're like, well, I'll just cache everything all the time, but um, it's more expensive than uh database space.

Because everyone's like, right, let's go straight from RAM.

Well, RAM can get expensive, but file system can be cheap.

And you know, it's this, it's so there's this um, you know, when there's this idea about the the different latencies of different things, you know, L1 cache, blah, blah, blah, all the way down to um reading something off the disk and or getting something over the network.

It's like how far up that scale can you move your relevant thing?

It's just a question of thinking about you know your requirements and your performance things and all the rest, you know.

It's like algorithm design all over again.

Um, I would at last point, I would mention this.

So there's a there's a book that's a couple years old at this point, but still very relevant called High Performance Django by the folks at Lincoln League that uh talks about caching, but talks about a lot of these performance, because this all comes around performance, so that's definitely worth a look.

Uh we'll put the link for that in the show notes.

Um yeah, they so yeah.

So end of the

I don't I don't know what the latest published books.

So it's not that the advice especially around this stuff is wrong but I would love to know about more up-to-date things.

I mean as far as I know the um so Tango with Django just released an updated book.

That's a classic that's been around um but there's still I still think I'm almost the only one with 2.2 versions of my books.

Okay.

Um so it's yeah if you think of if you find those we'll we'll put them in the show notes.

But um that you know I guess it's nice in a way that that's the stuff that doesn't change as much.

I mean that's the the the challenge and opportunity for me as a content creator is I have to update things all the time which be can become tiring but also makes me do it better.

But it means that a lot of um I I sort of look longingly at these things like um algorithms and stuff that are more promoted.

Like it's it was the

Oh what you know I've got it it says 2.2 and I've got 1.8 what's going on here like you know yeah I remember that I remember being in that exact position and I couldn't I you know hating barriers and my wall head on the wall for ages.

Anyway that's a cheerful note to finish on

The episodes actually are also on YouTube the audio only if you prefer that I keep putting them up there and there are some subscribers but if you prefer YouTube go check it out.

We'll see you all next time