Transcript: Postgres - Craig Kerstiens
Hi, welcome to another episode of Django Chats, podcast on the uh Django web framework.
I'm Carlton Gibson.
Joined us every by Will Vincent.
Hello, Will.
Hi, Carlton.
Hello, Will.
And today we've got with us Craig Kirsteins.
Um th Craig, thank you for joining us so much.
Yeah, thanks for having me.
No, again, I'm really excited to have you on because you're sort of massive hero of mine.
For people who don't know you are, I'm gonna say you're the you're the database guy, right?
Yeah, I think I've definitely been pegged as that.
Like uh basically it's been a lot of time in in DevTools and around Postgres.
Um I I keep joking that like the next time I do something, it's gonna be not Postgres and back to DevTools, but
No, it's not.
It'd be Postgres again.
Oh no, it's uh the third time and I'm I'm done with it.
But uh no, I've been around you know Postgres and in that community a while, but um really you know, a mix of DevTools and kind of work with a bunch of DevTools companies, know those circles well, um, spent a long time at Heroku.
Um like came on to Heroku not to do anything Postgres at all.
I came on there to actually build and run their Python team.
So um and ended up then doing Postgres there again.
So So
how how okay, so that's a nice little segue.
Well, how how does one start?
I'm in a Python lead, and now all of a sudden I'm you know get further and further into the Python.
Well, just one index at a time.
Yeah, yeah.
Um, so I guess I can, you know, back way up, right?
So I've I've been at like Fortune 100 companies, um, had joined a small database startup way back um in the the last recession, not the current one, but um, and uh they were they extended Postgres to become a a streaming database and basically do map re map reduce on data as it came in.
Um so yeah, back in the at the time, um like uh complex event processing CEP was a common thing.
So streaming databases were a thing.
There was like Astrodata, Natiza, Greenplum, um, and I joined TrueViso, and it was interesting.
We we had, I think MySpace was a customer.
Like if you remember back when MySpace was like, hey, I do, I
do, we do.
Yeah, that's that's where I learned the web on MySpace.
Um and uh like we were handling all their ad traffic, right?
Like, how do you like process all this in in real time and manage it and report on it?
And uh so I joined there, they were 15 people, they grew to be about 30 people, dwindled back down to 15, and was kind of ready to move on, and did some consulting, bootstrapped my own thing, um, actually in the wedding space.
It was one of those like scratch your own itches, right?
Was engaged, getting married at the time, and uh was like built this product, didn't know anything about marketing, and like got written up in like the Wall Street Journal of like top wedding apps of the sea.
Like, it was like, oh wow, cool.
Um, and then I got married and I'm like, I hate this.
Like, I was literally going to meetups talking about the season's dress cover, and uh uh had known some of the Heroku folks, had been there, did some consulting with them, and uh, but it's like it's all Ruby, right?
And I'm like, I don't want to go work at a Ruby company.
Like, Heroku was the cool thing at the time, but myself being more in like the Django world, I'm like, yeah, it was like this is a cool thing I'd heard about, right?
But I never experienced it.
And uh, so I show up there and uh I'm doing a little bit of consulting, um, some business analytics stuff and other things for them.
And uh they're like, so can we show you this thing?
We got this like build pack thing, and you know, when I was bootstrapping my own thing, we spent two or three weeks like getting our deploy process good and like cool, you tar it up, you ship it over here.
Like everyone did this, right?
And like to have stable builds and releases, and I forget we were running on a Linode VPS or something like you know, like everyone did back then.
And uh I was just like, wow, there was three weeks of my life.
I wish I could get back because I could have just get push and it was up and running.
And uh Python was slated to be like the I think the next the second or third language, and it got pushed back because of Java and others, so it like it ended up being one of the last major ones re released at the time.
Um, but I was like, okay, who's releasing this?
Who's driving this?
And they're like, oh, we don't know.
Like, we just have this thing, like we built it and here it is.
And um, so I ended up joining to like I was the first kind of build pack maintainer there.
Then I hired Kenneth because I'm like, I'm not actually a good Python developer, like I know that ecosystem and world, but like there's so much better out there.
So I I joined Heroku to to kind of focus on that, not Postgres at all.
Okay, okay, fantastic.
Can I can just go back and ask?
So, how did you get into programming initially?
Do you it sound did you have a like undergrad degree?
Like how why why why did Python hit for you?
Because this is before Python was kind of the hotness.
Um, so I I started coding at like age 10.
Like, um, to me, like code's always been an interesting means to an end.
Like, I I like solving problems, but like to me that intersection of like code and technology and business.
So, like my my degree in college was actually over in the business school, but a minor in CS.
Um so like I I I generally say in like you know, the best of the best, like developers, I'm I'm like, I can't keep up, but like put me in your average eight to five engineering job, right?
Like, I think the people listening to like the the podcast, right, deeply care.
Well, there's a whole world of developers that like that show up, write some code and go home and don't think about it, right?
And don't care if they get massively better.
And I I think that's I'm kind of envious of that at the time.
I mean, I was just gonna say that doesn't sound I mean, maybe not the not caring about getting better, but the rest of it sounds good.
Yeah, I mean, like, I'm kind of envious, like, hey, you can turn it off, right?
For me, uh, you know, I'm out here in the Bay Area and it's like everything is code or tech constantly.
So I I appreciate the like, you know, the family and friends that are doctors and nurses and not in tech, right?
Um so I don't I don't subscribe to the idea that you have to to truly to be a good developer and contribute, you don't have to like live and breathe it and say this is you know the end all be all.
But uh yeah, to me, always that intersection of of tech and business was interesting.
Um we I think it was a a colleague of mine, Daniel Farina, that like he was like, Yeah, this Django thing, it seems like it has good docs.
It's like well opinionated, it's well structured.
I mean, this was back in like was it like 092 maybe that we got started on it, like early days.
Um it was kind of data rails, and it's like the docs on Django seemed better.
So um, and I think it had the time the the batteries included tagline, which I loved.
Like, it's like great, I need auth, I need an admin panel, I need like it was all, I mean, all the things that are still true today were incredibly true way, way, way back 15 years ago.
Yeah, right.
I mean, um that talk about how you might as you might as well ask for a pony because it's got you know it's got all these things that like now it, you know, when people say, Well, where's the pony come from?
And you tell that little story, they sort of glaze over because they're like, well, all every web framework's got that.
But you've got to understand that no, really, every web framework didn't have that.
Yeah, I mean, it was you know, you look to the Ruby community and the Rails community, and there's like still five different authors, and finally an auth gem one out, but for the longest time, it was everyone was changing every you know few months.
Yeah, and Ruby's still part of the I mean, you know, Rails is still very much a batteries-included full stack framework, right?
So you expect yeah.
I mean, to me, I think Django goes even further.
Like Rails is close to it, but like misses some of those pieces.
Um, and I have to I have to be careful because you know I have many, many friends in the the Rails and Ruby community who do I who do I appreciate.
We have a DHH on here, so we you know we're we're all compatriots trying to solve the same problems.
I mean, I get okay, so while we've got why you got you talking about Rails, I mean one thing that people often say about Rails is it's more opinionated.
Um I I think it so it's more opinionated, I think there's less batteries included.
I think there's right, um, like there's not an admin dashboard, right?
There's not user management.
There's not uh like there's there's a few things that now it's standard.
Oh yeah, you pick this up off the shelf, but I mean, even 10 years ago, that wasn't a decided, you know, like how do you do user management in Rails?
I was gonna say one of the things um that Rails had as an advantage was that Heroku you could pretty much just start it and push it to Heroku and it worked.
And the I think the Rails book, you know, the first chapter I remember when I was learning Django back 10 years ago was just like, you know, do do do boop, you're live.
And there's nothing like that on Django.
So like I have a couple books on Django, and my goal was actually like I want to get you as quickly there as we could, but it's still four, five, six steps, just because so that I think comes to I like that distinction you made of batteries versus a pin of you know guardrails, but it definitely feels like a beginner still thinks rails is like guess you what you need, and it's like kinda, but then once you actually know what you're doing, you're a little bit more on your own.
But but that's um we'll we'll talk later about this.
But there's the whole new post Heroku um hosting platforms, and they're I like I just wrote the docs, the Django docs for fly.io, which is one of them, um because they didn't they didn't have any.
Um but it's interesting how there's not quite a like a Rails bias, and and there's also you don't use Git at all.
Like in fact, if you don't have a Docker Ignore file, your Git will be pushed up there.
So like it's a whole kind of different it's similar but quite different in subtle ways if you're used to the Heroku way.
Yeah, I I think we're gonna have a few interesting years.
You know, I don't know if we want to get into it now or later on, but like a few interesting years where I don't I don't know what the the six I mean and Heroku's still there, right?
The three tier's gone.
But um I think it's you know it's an interesting time, and to me it's kind of the a little bit of the writing on the wall, right?
Like I I saw someone that was, you know, ex Heroku, used to work at Heroku and was, you know, thought it was the right thing to do to get rid of the free tier, and then came back a few weeks later and was like, hey, I just ran like a Rails uh girls boot camp, and like I didn't think about the implication of this.
Like developers learned on Heroku.
Like, if you were at a Django boot camp, Py Ladies, like, I mean they every PyLadies boot camp ran on Heroku that I I'm aware aware of.
Like I I helped with a bunch, like we sponsored stuff, and it was like, here, it's just easy, right?
Like getting a dev stack is still a huge pain.
And you didn't have to have a full dev stack to be able to push it up to Heroku and see something at the end of the day.
Yeah, every con like, you know, my books, everyone's everyone's stuff is broken now.
Um or not broken, but you have to pay.
And it's less than having to pay.
I mean, I think like we've Carlton and I Carlton and I have had some discussions, like there's a valid thing to say you shouldn't have unlimited free because there's a real cost and there's fraud.
But um you know, not like they all this one of them told us like they all want to be the second place, right?
They want the marketing benefit, but not the hassle of being the free place.
Um but it is a yeah, it's harder, it's harder to learn.
Um, I mean, I I'm just try trying to tell people like I'm switching to fly just to my books because I need to pick one.
Like, you know, you hopefully you paid for this book and didn't steal it.
So like paying a couple cents to host a site, like you know, that should feel normal, right?
Like it there are real costs involved, but it is just a drag on the whole cycle.
And to your point, there it's gonna be several years before there's a clear winner, right?
Like a lot of these are series B and C, they've raised money.
Probably one or two get acquired, one will win, and the rest will go away.
So in the interim, it's frustrating for people teaching beginners.
Yes, Carlton.
Well, I I so I mean, unless you want to say something to that point, Craig, but I wanted to ask you more about the DX side of it, the developer experience side of it.
So Heroku clearly got it right.
And from your perspective, which is an amazing perspective, having been there and worked in it and you know, created it, what do you think um are sort of like the key things that people should look out for or that are missing from the the new ones, or you know, what do what do you think the sort of the holy grail is, like the the you know, yeah.
I mean so there's there's two pieces in there, right?
Like at Heroku, we we agonized over DX.
Like we we spent so much time discussing and debating, you know.
Like, I mean at the time, requirements.txt was not a standard.
Right, okay.
Like, you you go, I mean, I remember speaking at a a Pi Code Comp in Miami, asking how many people actually had a requirements.txt and an audience of Python, and there were like three hands.
Um it was like it was not a thing, and and it was like, but then you go and talk to people, and how did you manage like a reproducible build?
And it was all over the map.
Um and now it's like, you know, whether it's you know pipinv or like what I mean, I know there's plenty of debates on, but it's like, okay, requirements.txt generally works as well enough, like well enough a standard now, right?
Without it being a that uh I'm not aware of it as a standard, but like it may be, right?
Like codified somewhere.
But we agonized over that and kind of we looked across language communities and said, what do we need to push as an agenda to like not for us, but to like better improve?
How can Python go and learn from Ruby?
And how can Ruby learn from Java?
And how can um database URL was a canonical thing?
And like I I had yeah, I forget if how many debates I had, whether it was with Jacob or Andrew or who, they were like, no, you just do this, and I'm like, no, you shouldn't be putting config in code.
Um and they're like, yeah, but it's it's like and man, it was some hard-fought battles.
Um and like persistent connections in Django.
Um, I remember at a a Django Con Europe standing up, giving a talk, and Android argued tooth the nail with me that like it's not a problem.
And I showed him the chart of like latency to do TLS.
I'm like, okay, do you connect to your database with TLS?
And he's like, yeah, you should, it's secure.
And then I showed him the TLS negotiation, and it's like, oh, here's 100 milliseconds.
And like, that's just to grab a connection.
He's like, like, and and I asked how many people uh, and so this is Andrew, he knows how to run production Postgres, right?
And I'm like, all right, how many people are using PG Bouncer?
Because that's what you should do.
He'd tell you to do that, and there were like three hands in the audience out of like 400 people, and it was like, I I loved publicly trolling him, right?
Like he's sitting there on the front row, and I'm just like, I'm giving them this like subtle stare as I'm asking the question in the audience, because it's like, no, it's not a problem.
This is what you do, and everyone says to do this, and we agreed that it's the right thing to do, and no one does it.
So, um, being highly opinionated and just pushing for the thing that was right, right?
Like taking the long database URL wasn't a standard for three years, right?
And we just gradually pushed and nudged and had the conversation and and built the community and the connection, right?
Like, I I have a lot of very good friends having never contributed a line to core of Django at all, that like when they're in when Jacob's in town, like he's coming over for dinner to catch up.
Um, and uh, I mean that invite goes broadly out to like anyone I know in the dev world.
Like, I'm very happy to like host like it's a small world and community that we're a part of, but um really pushing for that like this is the right way to do things and and just seeing it through, it not giving up on it.
Um, and that's a hard thing to measure.
Like, yeah, I I you see it when you know it, and if the answer is go to docs, or oh, here it like here's this error message, or here's how to treat like it should just work, like it should just fade away and be magic.
Okay, good.
That's really nice because I saw what one thing I sort of see with the um a lot of the new batches they're trying to cover every possible use case and every every possible way of doing it, and because of that, it's quite complex.
There's too many moving parts sometimes.
It's it's it's hard, and I um I'm excited by Fly and what they're doing.
I think a lot of what they're doing with the networking stack is really interesting.
Um, like there's a whole bunch out there, and some are more exciting to me than others, and they're they're exciting because they have an opinion and a vision.
Yeah, I don't think any of them have nailed the same simple developer experience of Heroku yet.
Well, they don't, you know, maybe this leads into crunchy data.
I don't think any of them truly have managed databases yet.
Um, which, you know, is like they have they'll give it, they'll give you Postgres, but it's just uh running in a VM.
It's not not like Heroku has it and not like dedicated services like crunchy data.
And it my sense might just from the outside sense is it's taking them a lot longer to do that than they thought it might.
It's it's it's hard and I wouldn't wish you know it.
It sucks.
Um like I wish I had someone to run my databases for me.
Um I mean at the time at Heroku, we we joked that we had all these Rails developers asking for database, and we thought, how hard could this be?
Turns out it was a whole lot more work than we ever expected.
Um like Wally, which did all of the like underlying disaster recovery, continuous protection stuff.
Um like we were good and lucky.
Like we wrote that in a weekend thinking, hey, we needed something, like, and shipped it.
Luckily, about a month before the great EBS apocalypse, I think that was April 2011.
I don't know if people remember when like Netflix, Reddit, like the entire internet was down for a week.
It was terrible.
And like a lot of times, like when Heroku would go down in the early days, like we would we would go down and we'd check and see like is Reddit up and Netflix up.
And if they're down too, like we're we're screwed.
Like we're we just wait for like can Yeah,
like it's it's the whole availability zone that's gone.
It's not just uh
Yeah, it's like Northern Virginia is offline and half the web is gone.
Yeah, it was um that that happened.
Like we I mean AWS was offline for a week in April of 2011, pretty much.
Um and we had shipped WALL E just like luckily a month before.
And so we were able to recover databases with no data loss, whereas RDS just told customers, like, sorry, we lost the volume, like back then.
Yeah.
Um, and so I mean, I'd like to say that was amazing foresight, and like we knew we knew what we were doing.
It's like a little bit of lucky timing, and uh I mean, now we very much know what we're doing, like a very experienced team that you know at the tail end of our time, at my time there, it was after five years, we were running managing like 1.5 million Postgres databases.
We we joked that a one in a million problem was once once a week, once a day, and that was across eight engineers.
So very small, proficient engineering team.
Uh but uh I know Kurt over at Fly really well.
And his viewpoint is like it's at this point, it doesn't make sense for a pass to also run a database as a service.
Like they're specialized enough, like it's a distraction.
I think I think we may be past the point where like if you want the best service for Postgres, you come to someone that does Postgres.
You want the best service for Redis, you go to someone that does Redis.
You want the best pass, you go to the pass for the past.
And it's um I I know Kurt's gonna you know ping me after this and be like, all right, Craig, so that means you're gonna run the databases for Fly, right?
And we we go back and forth every few months.
And we have customers that use Fly and use Crunchy.
And like I've I've I've run a pass, I've worked at you know Heroku and and ran a lot of Heroku Postgres, but a lot of other areas of post of Heroku as well.
Um it's a lot of work to do either one of them.
And right now we're really focused on Postgres really, really, really well.
Okay, so given that we've uh talked about Crunchy, let's uh div in there.
So this is a managed database service, a bit like RDS, but you know, more um as I say, boutique, more bespoke more.
So I'm a big fan of RDS.
I always recommend people get a managed um Postgres.
Why should I use Crunchy data?
What's the you know, what's the yeah?
So I mean I'll I'll uh I'll give a few reasons, right?
Like um one, we contribute about a third of the upstream code to open source Postgres itself, right?
So, you know, if you care about supporting open source, right, like we contribute to a lot of it, right?
If you're like, how do I contribute to Postgres?
Well, guess what?
We we employ people that contribute to upstream Postgres about a third of the code.
So if there's a you know, there's sometimes not a lot of love lost for Amazon, right?
Out there making a ton of money, how do I how do I contribute in a meaningful way to open source?
Supporting us is one.
Um we we are just Postgres.
Like at whether you go to you know Microsoft or Google or RDS, like they they have people that run databases and they have you know app dev teams and all sorts, like we are just Postgres, and we also understand it from an app dev perspective.
Like I've been trying to make a positive impact on Postgres and for app developers.
Like at Heroku, we contributed a lot to get JSON B committed to um to Postgres itself.
Um we at CITUS, when I was there, we donated a portion of our equity to the Postgres community and foundations.
So like when CITUS was acquired, it meant money for the Postgres community.
Um, so like we're we're deep in there, and because of that, you're gonna get a great Postgres experience.
Within RDS, like you open a tick, and you you may or may not get to the product team.
Like no, you won't.
No, you won't.
Yeah, yeah.
I'll say I'll even if you won't say that, I'll say that.
You won't.
And for us, right?
Like, if there's a bug in Postgres, like I can reach over to a colleague and have them upstream a patch to it, right?
So like there's no issue too deep there.
Um, I I was told by our marketing team, I can't say like our support isn't crap, but like our support isn't crap.
Like, we give really good support.
Like um, I tell people, like, open up a ticket, ask a question.
Like, um and I think we have a unique, like it's not on the product side necessarily.
I think it sort of is and sort of isn't, in that like I get app developers, like, and app developers are terrible with your database.
Like, how many don't run with PG Bouncer?
And you should, and like, even if you have persistent connections in Django, you're like, no, no, I set my connection pool.
I'm like, that actually wastes memory in Postgres.
Like, you've reduced latency of establishing a connection, but every idle connection to Postgres is about 10 megs of wasted memory.
It's like, oh, so I should run with PG Bouncer.
Yes, but like one connection pool is not the same as another connection pool.
Um, they're like, which one do I want?
I'm like, both.
You want both of them.
Uh but you're talking about performance things.
I mean, I remember I was at a startup called Quizlet, which has decent traffic, and it was um MySQL, and you know, at one point we did we didn't have like a lot of uh permissions for who could go in and do something, and the founder went in to change a record and dropped the entire database, which wasn't oops.
Um we got it back, but then after that we had like Pirkona on the highest level of support to uh, you know, to your point of like people saying and not doing.
I mean, if you're running a website or running a startup, like you're definitely not focused on the database on the database to the degree you should be, just because you have a million other things to do.
Yeah, no, I mean I I I laugh a little at like situation, but like I've seen it before.
I've had customers Friday at 4 p.m.
be like, uh-oh, we ran our test suite against production.
What's the first thing in the test suite?
Like truncate database.
Like, and I'm like, all right, take a deep breath, tell me exactly when you ran the command, right?
Do you have and we can like restore your database to exactly no data base data loss.
Like it's gonna take a little while, but like we can watch the right-ahead log replay and see when it like the exact time that it happened and give you the exact moment before you dropped all your data.
And so, like, we have your back there.
I mean, we've got some stuff that detects like N plus one queries in your database coming soon.
Like, none of you in the Django ORM have ever written an N plus one query at all, I'm sure, right?
Yeah.
User.groups.name.
And it's it's not that like developers are bad, but you've got other things to worry about, right?
You can't be an expert on everything.
And like I I found the Django community loves Postgres, but also you you're missing half the features and half the perks of it because you don't have time to dig into it.
Well, this as well is one so one of the things one of the things that Crunchy Data's like on my radar for is that you've got um support for uh PL Python, so the you know embedded Python in there where it's RDS, like you go down their long list of which supported PLs and Python's not there, and it's like but I you know I I'm not gonna go and learn this one in order to write that stuff.
So there are more features there as well, like you can get a richer Postgres experience.
Yeah, I think that sums it up for like richer Postgres.
Like, because we are everything Postgres, because we believe so strongly in it, like I want to give you everything that Postgres can do, right?
Like, I you know, when we ship PL Python, I'm like, I wonder what if?
And we so we ship it also with SciPy, NumPy, and Pandas in there.
And I was just like, you know what?
I wonder if I could write a recommendation engine inside my database.
And I started live tweeting this, like when we we shipped it, and uh I was like, I should have tested that I could do it first.
I'm like, could this how wrong could this go?
Um and there's like 20 lines of PL Python and two hours later, and like, here's a recommendation engine inside my database.
And there's all sorts of reasons you shouldn't put that in production.
But maybe you should.
Yeah, no, well, this is one question is how much how much of my application logic should I move to the database?
Because I remember uh years ago, and it must have been years ago because I was using MySQL and I I learned about triggers and I was like, this is going to be quite good because you know, round trip to the database, that's costly.
Let's see if I can and it was fine to sort of start with, but then it started getting out of hand and I had no real management of it.
So the things that I missed and that I still don't know if they really exist are source code control or version management and like testing, decent testing.
And so, like I, you know, one question I have for you is okay, let's say I use PL Python.
Uh, how much of my logic should I embed in there?
And how can I have good software engineering practices so that I can roll back, that I can test, that I can show.
And are there frameworks to help me do this?
Because if I was doing this in in Python, I'd just use Django.
If I was doing it,
there's a little bit more than there used to be, but it's still not perfect, right?
There's still not the easy release process version management as much as you'd expect.
Um, testing, there's a little bit more with things like PGTAP, or like you think about the um the other kind of supporting pieces, right?
That I can go and take a fork in my database so I can grab a snapshot of production down the staging, and I can run this as a like we have more tooling around the database of like, I mean, we have customers all the time that take a copy of production, move it to staging like via automation on a Friday night, slim it down so they don't, you know, developer, all their developers aren't working with a one terabyte database, but it's like here's the latest 10 gigs of production data.
So we have realistic test data that we get every Monday morning, right?
Every every developer's gets a real copy.
And I, you know, I think they built it in less than a day with all our APIs and automation and like the hey, we give you these constructs to build and work how you want, right?
Um, and we see organizations work very differently, so we haven't said here's the codified one way to do this.
Now we have thoughts of how it evolves, but it's still, I think, early for that type of CI CD for your database, right?
Um so it's it's not perfect, and that's why I don't think you should always go like gung ho, like let yeah, swing all the way back and do all of the logic in the database.
I think if you're doing data-intensive operations, that's a good line where you can do it easier in the database.
And my, you know, Postgres, I think, can do a lot.
And so it's start there instead of adding another tool or system.
Full text search, start there before you add Elastic.
Because how are you getting the data into Elastic?
Now you've got two systems to maintain.
Um, geospatial stuff, post GIS is great.
Like you start to talk about a recommendation engine, and it's like, it's it's 20 lines of PL Python, and it's gonna be slow and not great, but you refresh it every hour or something like that, right?
Now, if you were to do this otherwise, you're gonna do be like, okay, I'm gonna go and install Debezium and do change data capture from a database, feed this into Kafka, I'm gonna run some model in Spark, and now I'm gonna cache the results in Redis.
And now, and I like that's not an exaggeration from 90% of companies, and now I've got five systems to maintain versus like 20 lines of PL Python inside my data.
Like, and your point of like, well, how do I version it?
How do I test it?
Well manually, like it's 20 lines of PL Python as opposed to five systems, right?
Yeah, yeah, yeah.
Okay, that's super as well, because that's that's that ties it.
I saw you tweet something about how um like Postgres and neither Rails or Django, you know, but then being like the money stack, like if you just want to get something done,
it just works.
Like it's it's it's it's um it's is profanity okay on the podcast or not?
Well it depends.
You get the idea, get whatever done, right?
Like that's that's what it is.
Um it's like I just wanna and one fun thing, uh total kind of non-segretary, but like one fun thing for me of of like running a past or database as a service is seeing everything like people build.
Like I I talked to one person who's uh uh he he surveys snakes for Canada.
Like that is his job.
He's like half like engineer and half like out in the woods surveying snake and populations of snakes, and he uses like post-GIS because it's like the best geospatial database.
And he's like, Yeah, I'm not a real developer engineer.
I'm like, you totally are like the stuff you're doing here is he's like, Yeah, but I'm not a real eight to five software, and I'm like, because he's his background's in snakes, but then he's using like post-GIS, and so getting to see all of these fascinating things like come in full circle, right?
To me, like tech is a means to an end, it's a fascinating one.
And so by being at kind of the dev tools databases a service intersection, I can see like um, you know, we have one person that our team jokes is like the cow social network that like tracks cows, like it's an app that like tracks their eating habits and other things, and it's like who how is this a business and a startup?
And like, what?
And but we know this, like walk down the the street and like walk into a retail store, and I'm sure there's a problem you can see tech solving.
So it's it's fascinating to be at that intersection and like Django, Postgres, um Django or Rails, to me, they're very similar.
Just just work, like they're not new and shiny, they just work, and now I can go focus on solving a problem.
Yeah, and and you haven't got 15-bit moving parts, which then you need three engineers just to keep the moving parts in the air.
Never mind about write the application logic.
Yeah, yeah, absolutely.
And so that's why I I mean I totally appreciate the question, right?
Of like, okay, well, how do you test this?
How do you version this?
But sometimes you may not, if it doesn't change once a year and you, you know, how did we how do we test things before?
Well, we with like extra code review and eyes on it and like you know, manual testing.
It's like, oh my goodness, crazy that we could we could do those things again.
I've got such a little evil devil on my shoulder going, oh yes, yes.
It's like it's it's like when I got the remote um development um extension in VS Code, and all of a sudden I can't so I can just open a folder on my server now.
Oh, I can.
I wasn't good off.
All these things in balance, right?
If you've got a 20-person team, how are you doing this?
How much is it changing?
But is this you know, we used to like rsync files to deploy, right?
Like, like, hey, guess what?
It still worked back then.
Like we we we brand websites on the internet same as we do today.
Um just fine.
Okay, well, I I wanted to um ask you one more thing about um crunchy data, then perhaps we can swing around to um you know, perhaps a few more Django questions that we've got lined up.
But um the question I had was I I saw you recently, so when you first launched, I was a bit like, oh, but it's quite a lot more expensive at small scale than an RDS, because a small RDS costs you I know eight, ten bucks a month, and you know, then you can scale up as you need to, whereas crunchy data is more like in the $30, $40, which is absolutely reasonable, but you know, it's for a smaller project that's perhaps prohibitive.
But then last month, month, two months ago, you did you went for 10 bucks a month or something for the minimum, which I just think is a power move.
So go tell us about that.
Yeah, I mean, so initially we were really focused on like being a production ready, like database.
Like backups, disaster recovery, monitoring, metric, like everything you truly need, right?
Like PG balancers built in.
Like you don't need to go and layer it in.
Um, I don't know of other providers that give it just right out of the box.
I think Heroku kind of has an add-on for it now.
Um, but like on RDS, there's this separate RDS proxy that you go pay for.
Like, no, this is what it takes to run a production database.
And um, we're running a lot of very large production databases.
Now I care about hobby developers, but we needed to nail the the advanced stuff.
Absolutely.
Like, we now we're going kind of down market that it can be there for everyone.
And it's, you know, as you say, 10 bucks a month for a full Postgres database with PL Python and with you know PostGIS, and you can create more databases.
Heroku, you know, when they had their kind of like lighter tier plans, it's like, here's a million rows, and I'm like, a million rows?
What is that?
Like, um, and it's one you get a shared multi-tenant database thing.
This is your own dedicated instance.
Um and then the
key point, the key point there for the hobby developers is I can create as many databases in that instance as I want.
And so, you know, if if I've got if I'm running 50 micro projects, hobby microprojects, I can still run it on the same one 10 buck instance, and then that that's as close to zero as you're ever gonna get.
Like, well, so we also don't technically zero, but it's effectively zero.
And so, uh, you know, we were talking kind of like about Pylades and some of their workshops.
Uh, if you spend less than five dollars in a month, we don't build it.
So it's not a five dollar credit, but if you're below that five dollars in a month, like great, it's on us.
Um and so you can go and like suspend your database and idle it.
So if you're like building and developing, right?
And you're like, hey, I'm um, we've got a couple universities now that are are pointing their students at us, and like, great, idle it like 5 p.m.
and on the weekends, and like it's there.
And then when you're you know developing and learning and building out your app or your project, turn it back on, right?
And then if you want to, hey, if you want to have your portfolio live to demo and show off what you can, like great, it's 10 bucks, right?
It's man like less than you paid for the textbook, right?
Um, but then archive it, it's sitting there, and you're just paying for the storage, and if you're below that five dollars, it's it's there on us.
See, I think that's I I just think that's a super power move.
I think that's you know, that's gonna grab a lot of people's attention.
I think, and rightly so.
I think when I saw you put that out, I was like, yes, that's good move.
So can I can I ask about because I'm interested in how people learn how to code, are these do you have more insight on like these these universities?
Like, is this their databases course?
Is this like like an intro like CS50 kind of thing?
Do you have like how are people from where you sit, how are people kind of learning how to code these days?
Because I'm always curious and can't it, it's the I never get a consistent answer from people on like how they're actually like like building a website, right?
Still many universities don't have a course on it.
If anything, it's an adjunct or like it's not a core thing web in like an undergrad CS degree.
I I'm I've talked with a few that seems to be a little more common now, and it's you know, I think it's very much the basics of building a web app to showcase something, right?
Um, one of these is actually in a I don't know if it's an MBA or a master's program.
I think it's a master's program, but it's typically like people out in industry, so it's not like I'm getting a CS master's, like coming back and wanting to kind of level up on their technical skills to kind of advance their career, right?
Um, so and this one is like on the geospatial side.
And so geospatial, it's like you do anything geospatial, you're using PostGIS, likely.
And so, like, how do you build an app to showcase this?
And it's you know, a lot of the apps are are you know simple little apps, but it's hey, I can showcase that hey, I'm doing this analysis here.
I want to show off a website, not deep, deep on the SQL side, but they touch just enough of it, right?
Where hey, the database is here.
Um I'm definitely seeing more universities teach here's how to build a web app.
And but the the problem is, man, like the way they they plan out a course, right?
They'll work on a course and develop it and um plan it and establish that curriculum for like two years, and they do it, and you know, guess what?
Everything about that we talked about on web development and how to build a web app is has changed.
Like um what is it like remix and next.js now or the front-end frameworks, and I'm like, I I don't even man, I miss jQuery, right?
Like I like I don't know
uh HTMX, yeah.
It's HTMX, yeah, but yeah.
So um I I think it's hard for like universities to like how do they with the pace at which things change, how do they keep up and do all that, right?
I I don't know that I have an answer to that.
The answer is they should use Django.
Well, but even Django.
Well, like I had like a Jang at DjangoCon, there, you know, a couple prof professors who are using who I teach Django are looking at my books, and you know, yeah, to your point, they have to have it they're planning now for next fall, and it's like, well, the you know, the book's gonna change, like Django might change a little bit, and it's not not so much if you've been in the space, but if you're a beginner, a newcomer, it's enough to kind of break everything and they can't really plan around it.
Yeah, and I mean the uh for me, especially on the front end side, that the pace is.
Oh yeah, well forget about forget about that, yeah.
That that that's the one that's like, oh man, it's just like anxiety immediately of of I don't know anything about like the front end world anymore.
Like it like it changes so so quickly.
Can I ask you, so you're sort of unique in that you you're not a pure Django person, though you have a strong background in Django, like are there other frameworks that the Django community should look at in terms of hey, there's some good stuff going on there, you know, databases or otherwise?
Like I'm constantly looking at um Laravel impresses me um in terms of being batteries included, but from where you sit, are there other ones beyond you know rails that Django people should look at?
I mean I think you you nailed it on Laravel.
That's the big one that comes to mind of like a a similar and um like kind of modern, you know, PHP is a new thing.
Like things I didn't I and I so I I launched the Python support at Heroku.
I also launched the PHP and the Go support.
So um, and it was you know, the PHP one was like, wait, what really?
Like Heroku official PH, it's a different world now, right?
Um It's huge.
I've got a buddy who's a consultant and he's constantly I mean it's it's you know, I don't want to throw out stereotypes, but it it there's a lot of money in in PHP and a lot of people tech stacks that just work.
I mean, hosting's easier, right?
They've got you know, Laravel itself has Forge or whatever it's called, you know.
So whereas in Django, it's like you know, so
um, I mean, on the you know, the front-end side, I think Next.js, uh Next.js and remix, um, like we we've gotten on the remix run like bandwagon ourselves, and I I'm not knee deep in it, but it as a front-end framework, it seems like next and remix both solve some of the problems that I think existed with vanilla react, and and you know, we we complicated things on the front end, and it feels like they're getting
a little better.
I part of me is still confused, it's like why do we need a state machine on the front end?
Like, why why doesn't like request response just work?
I there's there's people in that world that'll educate me at some point.
Like, I'm I'm I'm sure there's a good reason.
I hope maybe, but um just
the pendulum just swings, right?
Yeah, it it entirely does, right?
And I'm I feel like I'm an old curmudgeonly, like uh, you know, yeah, just use Django and Postgres, like build your app, make your money, solve your problems, and like you don't need anything else.
Um uh which but I also think that's that's fairly true.
Like you can live in that world and it's great.
You don't have to relearn something every 12 months.
Um, not to say you don't learn new things in that that stack, like Postgres advances.
Django advances, but um that stack works pretty well.
I was gonna ask, are there obvious things Django could add to our official Postgres support?
I mean, I think the last the last big one was I think full full text search.
Um I mean we've we Django has felt historically like we couldn't pay for code or really push things absent a Kickstarter or some things, and we found out through the P Python Software Foundation that's not necessarily the case.
So this is one of the things is we're kind of looking for doable, you know, three, six month projects that Django can support, and especially around Postgres.
So what yeah, what cool features does Django not have support for that maybe we should?
It's a great question.
I feel like Django's done really well.
Like the the JSON support's good, the full text search support is good.
Um I I don't I don't know right all I mean it's I don't know that it fits in Django, but maybe it does.
Some of the stuff around like um constraints.
Like I don't like um like enums aren't uh you know there's an internal Slack discussion about enums of like, hey, has anyone seen them out in the wild these days with Postgres?
And the answer is most uniformly no.
Um it's like use a check constraint in instead because then you can easily change it and relax it and you don't have to do a table rewrite, it's not as invasive, like, oh okay, like I I don't know if an enum in Django maps down to like a check constraint, which is maybe a more modern way of doing that, right?
Or um exclusion constraints with range types.
How do you say like you can't be registered for a university class that you know overlaps times?
Like range types and check and exclusion constraints are amazing.
Like if you actually had to enforce that in code, like you're going to wind up with like race conditions.
And if you remember back to like university scheduling, it's like, no, no, it's just like a class with 30 people.
How like there's no way two people will be registering at the same time.
And I remember, like, oh yeah, my registration opens at this time, and I I want the 1 p.m.
I do not want the 8 a.m.
class, so I'm like, there when it opens, ready to go.
And yeah, did you just fill up a class?
Because like they both had their session working in, you know, you know, multiple web servers, and like so like relying on the database, I think a little bit more for like constraints is a big one.
Constraints are so valuable, and yeah, go ahead.
So that so it's good you mentioned that because that's one of the good feat that has come on quite a long way in um Django over the last few versions.
Um so with first of all, unique constraints and now you know uh I have to check exactly which you know which constraints we've got covered there.
Um but um Django 4.1 now that has it so when you do validation on a form, the database constraints that you've got can are checked for that form validation as well.
So it goes back and we'll go.
And so it's like it's not we first of all, it was like, yeah, you you get an integrity error when you save because you violated the database constraint, but now you get the nice form error and you can go and handle it.
And it's like, yes, that's so that's lovely that you say that because it's one of the features that's been pushed hard over the last few versions and has really flourished.
So that's lovely.
I don't I feel like Django does in general does a great, great job.
Um, so um
one one question I had uh on this topic was one thing that's always really interesting me about Paste Postgres is being able to take a create a custom type so you could you've got a um a relation and then you can use it as a field in another relation.
It's like I I've never sort of sat down and thought if that would even if that's even feasible to make work at the Django ORM level, but that would be kind of amazing.
Yeah, and it's what that's one of those things where it's like when you understand that you're like, oh man, now what can I do with this?
And it's like, wait, uh, not as many things as I thought.
It's like oh and like I mean, I've seen it used out in the wild, but not as common, right?
Like it's one of those, like, all right, this is a deep database hack thing for a very specialized case.
It's not one of those.
I would be more scared if Django supported it like easily in the ORM that it's like, oh man, we've got people now building monstrosities.
Um I I so I mean I I could actually see a reason for Django not supporting it because you know, cool, I've built this table and now I want to embed the table and embed the table inside a you know another column and another column, and now I've built a NoSQL database inside Postgres that's also relational, like it I could just be see it being turtles all the way down.
Okay, good, good.
That's a
good answer.
Don't go in there.
I wanted to ask you um some fun questions about personal coding setup, if you don't mind.
Like so let's maybe we'll go quickly.
So text editor of choice these days.
Uh usually sublime text.
Oh, okay.
Nice.
Um Mac or Windows.
Mac.
Or Linux.
Yeah.
Okay.
Good man.
Um Git Vis Visual Editor, what do you use to see your Git?
I no command line all the way.
Okay.
Like that.
Um what else?
What what's another like cool personal like tool setup you have that maybe others aren't using?
Um, so I mean, I, you know, uh my I don't spend as much time in code, right?
So my stack is like Notion Linear uh for like project management, um superhuman for email.
Um I think my like my default these days is like when I pick up a new tool, uh, if it doesn't have like a command palette, like a command K, like then cool.
I'm not I'm gonna like look for another tool.
So um it's actually a thing that we're building into Crunchy Bridge.
Uh, I'm hoping it ships early to Q1 next year of like command palette inside Crunchy Bridge, where it's like, great, you want to find slow queries, command K database name, slow queries.
Like you can imagine all these things that like I forgot where to go and to what to do, like queries over five minutes, right, on your database.
Just Command case, start typing, and it'll take you to that screen where you could kill a query.
So I'm I'm stoked to I think maybe be the first, depending on you know it who listens and tries to to outrun us and ship it.
Um yeah, I'm a big fan of tools with like a command palette to just like I can be a power user, but also like this take like this as an average user turns me into a power user.
Yeah, it's discoverability, right?
Because you just start typing and all the map possible matches come up.
It's like, oh, it's that one.
Right, brilliant.
Yeah.
How how do you think so?
The fact that you mentioned that Crunchy is like about around a third of open source contributions to Postgres, like Django doesn't have anything like that.
We have massive companies that use it.
That it's sort of like the bigger you get, the less you're in you contribute to Django.
Like any any thoughts on how we can do better about getting some of these massive organizations to actually that are using Django?
Because what happens, I think, is that they kind of get off the versions and then they have a custom fork like Instagram has this, and they're kind of not really using Django itself.
Or I don't know.
Like it it we'd love to have them, you know, that's such a great thing for Postgres, right?
We're always like like Postgres had a table at DjangoCon.
Like Django has no tables anywhere, right?
Like Postgres seems to have figured something out there that Django needs to.
Well, but I think it's uh it's not the big companies, right?
It's you know the smaller ones that are doing kind of the heavy like, you know, we're a hundred person company, right?
Like look look at AWS, right?
Like, are they running a proprietary fork of Postgres that they should be contributing things back?
Yeah, probably.
Like, like they're our Instagram, right?
Or Instacart are like um, so yeah, I mean, I I think there's I I don't know how to get the big ones to, right?
And I think that's the reason of like, hey, how do you support insustained open source?
It's you know, some smaller companies building models around this, right?
Um, we have people that are full-time working on Postgres.
And like we're we're we're a hundred person company.
And it's like, you know, could we take that person and that salary and and build it back into product or otherwise?
Sure, but we're committed to Postgres, right?
And I think there's an opportunity for Django, um, you know, the mix of of consultancies and smaller shops, like to do that, right?
And and product companies, right?
Like at Heroku, we were for a little while headed down that path, right?
And I remember way back at Heroku, there was there was Jangi and there was um uh what was Andrew's uh startup?
Um I forget.
Oh uh Lanyard or
no, no, before that.
He was building a Python for Heroku.
Oh, I don't oh, I don't know about that.
Yeah, so Heroku for Python.
Um I emailed him ahead of time.
There were three of them that existed.
Um it kills me that I can't remember now what the name of it was.
Um but uh yeah, how do you build product companies around you know Django that then also contribute back?
That's a a a good question.
I think I'd say Postgres is a little lucky, maybe a little bit of skill, and then some some super intentional kind of companies doing that.
Can I can I ask, is there any um sort of business case to be made that it pays for itself in terms of expertise and um you know the ability to react to problems more quickly?
And so you're paying for these engineers to work on Postgres, but because of that you have in-house expertise that's at a significantly higher level than you would otherwise have.
I I think so.
I mean it's the the pitch I give, right?
Like you can't
But right, but does does it does it does it cut the mustard with the business type?
So they go, oh yeah, actually, yes.
Yeah, I mean for for us, absolutely, right?
So I think it's a case by case, but it's like, you know, you you don't need a Postgres contributor on staff as a Django development shop, right?
Does that mean you can't discover a Postgres bug and need help with it?
Yes.
Now guess what?
Even if you wanted to pay the salary of a dedicated Postgres person as a Django shop, you you couldn't hire them because you don't have interesting enough problems.
Like you you wouldn't cover one bug a year, right?
It's like they're they're bored with the work.
Now, as a Django shop, right?
Man, having someone that's like me deep in Django Core, and you got you know five clients, simultaneous projects, like great.
Oh yeah, here's an interesting one.
We heard this from users, right?
Here's where we can go and upstream something like that model can work really well.
And what I tell our customers, like, why do you pick us over someone else?
Like, great, when you have that Postgres bug that you could never get to the bottom of, like, forget even if you you could if you wanted to pay the person, you can't hire the DBA because you don't have interesting enough problems.
But guess what?
Here, like, your problem is interesting to them, and like let's pull them in, right?
And so when you need that, that one week of the year, you've got it with us.
Yeah, I mean, I've done a lot of freelancing, and one when I started contributing to DRF in particular, it absolutely changed the dynamic because instead of going to shops, they'd be like, Oh, you sure you can do this, they'd be like, Oh, you maintain that that dependency that we're building, our product, yeah, and that just changed everything massively.
I mean, Zapier, they have um Simon Charrette, who's one of our big ORM contributors, like you know, he's but they give him a lot of time to work on Django itself, and they must obviously see the benefit of doing that.
Yeah, so I mean it's a model that I think works, and I wish I would see more of it.
Um, we we did some of it at Heroku, um, managed to do some of it at CITUS and when I was at Microsoft a little bit, and and Crunchy were doing a ton of it, you know.
Um at Heroku we did it across various languages and other areas too.
So I've seen the model work outside of the database world.
Um it it takes some buy-in, right, at the business level, um, and it takes interaction on the technology side too.
Like it I I have found that like hiring someone just to work on open source off on an island isn't as good.
It's like you gotta you gotta get the communication.
Like, what do you see in out in the field?
What are custom like what problems are your developers right now?
How do you get that communication?
Like, communication's hard.
You've gotta build that in, and then it really shines.
Okay.
So what's the AI disruption for databases, right?
Like, what's the copilot or is there one coming?
Uh there so much is already there in Postgres, right?
Like, you point me at a Postgres database, and I can tell you, like, you have this query that was probably generated by this ORM and it's like missing with like uh select related, right?
Like I can look at the query from inside the database, PT statements already has it.
And so I, you know, I'm optimistic.
We're we're doing a lot of RD and investing there to be like, hey, I can tell you, like, from not knowing you are a Django app or not knowing you are a Rails app, because like Django has these conventions on the tables are named this way, the join looks this way.
I can probably tell you to search for this in your app code and change it, and now this will improve your your performance.
So um
that that's like my favorite.
Like, there's like a meme of like, you know, AI, and then like you pull back the curtain, it's like if then, if then like you know, but it works.
It works so well.
So I mean I feel like we're really close to a lot of this with Postgres, and so um, I'm maybe oversimplifying it, but I think it's like a lot of this simple stuff goes a really, really, really long way.
And is that what you're excited about coming up in the Postgres world?
You know, if that if I would say, you know, what's what's on the horizon that we should look out for?
For the Postgres, I mean, so for that's definitely what I'm excited about for Crunchy and what we're focused on and like coming soon from us.
From the Postgres world, you know, the pluggable storage stuff, right?
Like Postgres got pluggable storage backends uh a couple years ago, but we haven't seen like another predominant one emerge.
So like there was Z Heap for a little while, then Z Store.
Um there's there's a few others we haven't seen one win yet.
So I but those sort of things move at a different timeline and pace, right?
On like years.
So it's not like oh, we'll see it next year and it's gonna win.
I think it's like five years before we have another standard pluggable storage backend.
Um, extensions like are awesome.
Like Postgres that I'm aware of is the only database that allows these like deep, low-level extension hooks that can change the query planner, change the storage engine.
Um it's super exciting.
And it's not like there's like one extension.
It's like as Postgres evolves, it's like more hooks get exposed, so you can do more things that you never thought of possible, right?
Like Postgres is great at full text search, at geospatial, JSON.
Um I think it's like just missing like being a graph database.
Like it's good at time series.
Um there's the Apache Age project.
I that's one thing that's I think.
I get asked about that every like nine months.
So it's been around a while.
I don't know of anyone using it out in the wild.
If someone's using it out in the wild, like drop me a like, I'd love to hear about it.
What's the experience like?
I'm pretty easy to find.
Search like Craig and Postgres and like shoot me a note, a DM, an email.
Um I I haven't heard of cases out in the wild, so I don't know enough.
Like I get asked like every six-nine months, like, what about this one?
And I'm like, I still don't know.
And it's been like five years now, because it's not a new project, it's new to the Apache project, but it's it's not a new extension.
Okay.
So we're coming up on time a little bit.
Are there any questions or recommendations you could give to us?
You know, what what could Django do to make Crunchy or Postgres' life easier?
Does anything jump off your jump to mind?
Uh I think the biggest thing is like, don't I find developers, and I I hate to use the term developers, like I am a developer, just a really bad one.
And it's like developers,
developers, developers, yeah.
Uh don't be afraid of the database.
Like I find that way too like like an app developer that's a really good developer shows up and apologizes or like I'm not a database expert, so like don't judge me.
And it's like I I like don't judge me for all of my ter like we all have terrible code, like all over our apps, right?
And like, don't judge me from a year ago, because now I'm a better developer.
Like, just you know, don't talk to my future self, you know, because then I'll have been a bad developer right now.
Um, like don't be afraid of the database, don't be afraid of SQL, right?
Like, you don't have to become an expert at it, like, but like that willingness to learn, I feel like sometimes we abandon it when it comes to databases as developers.
Like, we'll we'll we'll dig into advanced caching and you know, templating and um, you know, like how does DNS work?
Does do any of us know, right?
But we'll go and learn about that before we'll learn about the basics of SQL.
And I I find there's just like fear and aversion to databases that I wish people would just kind of like, don't be afraid of it.
Be a little willing to dig in, and then it you get so much more out of it.
Yeah, good.
So and there are these tools, like there's the what is it?
I think one of the like the Harvard places has these like SQL, like ways to go deeper with SQL, right?
Because I think part of it is you if you're using Django, right?
Like, okay, learn just like very basic SQL and then you can use the ORM, but like there isn't that need unless you're like at scale to kind of dive into SQL as part of it, like part of why it's a gap for developers.
Whereas maybe as you grow, like caching, okay, like I don't know, yeah, I don't know.
It's it's kind of like how do you become a a uh uh a database person, right?
Like you said, you like is there a straight yeah, it's just like a weird sort of you sort of end up there as opposed to like starting there in a career, whereas like web, you can kind of stay in web.
Yeah, so I mean it like so one thing like crunchy, we have like a Postgres playground, which we put like Postgres in your web browser.
Oh yeah, that's right.
I saw yeah, that's that was this year that came out, right?
You can like follow along, like it boot it's running Postgres, not on the cloud somewhere connected to a like it's in your web browser, and it'll load.
We have like exercises and guided tutorials that it's like oh yeah.
I'm gonna link to that.
I remember sorry, yeah.
I remember that that was really exciting when that came out.
Oh, cool.
I just I just looked and we shipped like four or five new tutorials there um that I I didn't know about um since yesterday, I think.
Um uh but it's like you you have the question like uh a query I write at every single job is like like user signups or like revenue, and then it's like growth week over week or month over month, right?
And I I bet your first thing is gonna like I'm gonna jump to the Joe Django ORM and write a query that's probably terrible, and then I'm gonna iterate over stuff and do comparisons over a list.
Um why not just write it, like try to write it in SQL and see how it comes out?
Because it's gonna be more performant than anything you do in the ORM for that kind of like reporting stuff.
Um, and like just a willingness of like, hey, if I'm asking questions of data and it's some kind of report, like can I can I try SQL?
Can I give it a shot?
Right?
That's I don't know that there's uh an answer for like how do you get better at it, except for like when you have the question, see if SQL can answer it some of the time.
Doesn't have to be all the time, but at least some of the time.
Yeah, there I mean there's for me with beginners, I often say if you're writing raw SQL, you're doing it wrong.
Um so I think that's part of it is like, you know, yes, you can do it to see things and at scale, but I think maybe that's also part of it is there's this sense of like, you know, there's the Simon Charretes of the world who've were smarter than you, so like don't even touch it, right?
I mean, that was Simon's whole talk at DjangoCon was like it's the ORM is not this scary thing that only you know three sorcerers understand.
I I think it's like if you're asking questions of data, right?
If I'm not doing a quick, like, you know, the like quick lookup, like if I'm asking a question about the data, right?
I'm doing some kind of analysis, SQL's the right tool for it, right?
If I'm and so like those cases, right, where you want to ask, how many users did this, how many people did that, like what happened to this over this period of time?
SQL's a great thing, right?
And that's that's when it's useful to to dig into.
Yeah, and the other thing is like when you dig into SQL, like don't worry about it being performant, worry about it being readable.
SQL is a powerful language.
Um, if you like, you know, do do either of you like writing SQL?
I like the results.
Okay.
Um, if I ask that to 100 people, I'll probably get like five to ten yeses, right?
Like it's not a lot of people.
Have you any of you, uh either of you ever liked reading someone else's sequel?
No.
That's like no way, not happening.
Definitely not.
Yeah, zero out of a hundred, yeah.
But like, I can write sequel that it's like, you know, with like use CTEs, use common table expressions, like build like building blocks that it's like, no, like a DBA is gonna be like cringe.
This is this is so unoptimized, and it's like, yeah, but developers can read it, and you can come back and see what's going on and tweak it and change it.
And like, right, CTEs are amazing for readable sequel, and like I'll indent my SQL like I will my code.
That is and it's like like writing readable SQL, and people are like, oh, actually, this doesn't have to be what I thought about, which is like three joins on one line, like what's happening, and you just cry, right?
Yeah, yeah, no, entirely.
I think that was that's why we ended up in this position where it's like, no, use the ORM, don't use SQL, because you'd have basically junior mid-level coders writing, hand writing the worst SQL in the world, and then you'd come and you'd need to maintain that, and it was unmaintainable.
And so it was like, no, no, no.
We it the the the correct thing to do is to use the ORM and the abstractions it provides, but there's another there's another use case, there's another category of things, which isn't like right in the middle of your application code, you're slapping some horrible, you know, insert query.
I think I have like some block, I forget, it's like I thought it was some like horribly executed joke about like um like my approach like my sequel, like like how I like my not my sequel, but like how my how I do like queries, and it's like some basics of indentation and CTEs, those two things will make all the difference.
Yeah, okay, super.
Anything else, Carlton?
I think we're well, I was just gonna say, do you run uh Postgres Weekly?
Uh I I helped start it.
I do a very, very, very little bit for it now.
It's uh Peter Cooper does all the heavy lifting.
It's kind of um like early on it was, you know, hey, there wasn't.
Now that you we've gotten it going, right, it's kind of like great, it's easy to discover the things, people tell us that it's there.
Um there's there's clear kind of outlets and mediums.
Way back then it was like, where is Postgres content?
So um I did a lot more curation in the early days.
Um now it's a lot less and a lot of of Peter Cooper, but like um, yeah, it was years ago.
We were both speaking at a conference, and I'm like, so Postgres Weekly, are you gonna do it?
And he's like, because he had the domain, and he's like, but I yeah, I don't know anything about Postgres, and I I can't curate it myself.
I'm like, well, he's like, I got all the mechanics in place.
I'm like, this sounds great.
So I I did start curating it, and like he's got an awesome back-end system that manages it, ships it out, like it's so it's great.
So it's I highly recommend it.
I, you know, I tweet a lot about Postgres.
Um, there's the crunchy data newsletter, Postgres Weekly, um, a number like if you're hey, I want to dig in a little bit.
And Postgres Weekly has always been targeted more at app devs, like people think, oh, a database newsletter.
That's we're not targeting DBAs.
If you want to go like deep on like Postgres stuff, join the Postgres mailing list.
It's a great resource.
Um, the hackers, that Postgres hackers mailing list.
If you want to learn about databases, it's a wonderful place to lurk.
But if you're just hey, I'm a Django dev and I just kind of want to know about the cool new things that I can take advantage of, Postgres Weekly is an awesome resource.
Yeah, no, I would I subscribe to that, I have done for a number of years, and it's like my little window into the Postgres world.
And I I I truly appreciate it.
So I want to say thank you to you and to Peter for doing that because it's important.
All right.
I think we're we're at time.
So um we'll say thank you again, Craig.
This was this this is great.
This is really interesting to to hear all these things.
And I, you know, I think my biggest takeaway is your your point about the next generation of host like no nothing's gonna replace Heroku.
It's gonna be like it's enough to do a database well, it's enough to do a pass well.
Like that makes all the sense in the world, even just talking to you about some of the challenges and things that that you see.
So I'm gonna have to think about that as I change my recommendations to people in my various books.
Um Yeah,
I think it'll make the world a little messier, but I think it's like it is it's probably the right thing for a little while.
Cool.
Thanks for having me.
It was fun.
No, well, thank you for coming.
A real honor to have you on and would like you know to have a chat with you.
It's super.
Thank you.
So we have links to everything, Crunchy Craig's personal site.
Um, and thank you, Craig.
Thank you, Carlton.
We'll see everyone next time.
Bye-bye.
Take care.
Bye-bye.