Transcript: Self-Hosted Open Source - Michael Kennedy
This episode is sponsored by MailTrap, an email delivery platform that developers love.
Try for free at mailtrap.io.
Hi, welcome to another episode of Django Chat, a podcast on the Django web framework.
I'm Will Vincent, joined by Carlton Gibson. Hello, Carlton.
Hello, Will.
And we're very pleased to welcome back Michael Kennedy from TalkPython. Welcome, Michael.
Hello, hello. It's amazing to be here on your show. Thank you, guys. Good to see you.
Thank you for coming on, Michael.
You are one of the original inspirations for this podcast. I was, I think, a guest on yours,
and we were just talking off air. That was over five years ago. So it's time flies.
I can't believe it's been five years. It's just mental.
I know. I know. But I said TalkPython. You have this whole constellation of things now
as well, right? You have Python Bytes. You have your training courses. I feel like I'm
missing something. Is that the full suite of things?
That's pretty much it. Yeah. A tiny bit of conference speaking and stuff,
a little bit of YouTube. But primarily,
TalkPython, Python Bytes, and the courses around it.
So you mentioned conference speaking. I saw you flashing by on the news
this week or last week about some keynotes somewhere exotic. What have you been up to?
Yes. Yes. I had a wonderful time. I got to go to give one of the keynotes at
PyCon Philippines this year, which was very excellent. I think I'm over jet lag,
and then we had daylight savings. And so, you know, that's how it goes.
Yeah.
Did the kids get up earlier or later? I can't remember.
They are forced to get up earlier than they want.
Right. Okay.
Yeah. We lose an hour. Yeah.
But it's glorious. It's glorious. I live in the Pacific Northwest in the US, and it is like,
we have two seasons. We have summer and rain, and also that comes with darkness. It'll be dark at
like 4.30 in the winter and rainy. And the sun was out till past seven last night. I feel like
summer's already here, even though it's not. So no regrets.
Yeah. It makes a big difference.
I'm on team daylight savings. Let's go.
So how did you go in the Philippines? Tell us about your keynote. What did you talk about down
there?
The keynote was super fun. It was called The State of Python in 2024.
Oh, that's good. Because I wanted to ask you about that. I was like, what are your hot takes,
Michael? Go on. Give us your hot takes.
Yeah, my hot takes. What are my hot takes? Well, I think it's interesting to consider that
in the Python space, it's so distributed.
It's so distributed, the directions, and it's like Brownian motion almost. Stuff is just moving
around and whatever channels it takes and where traction gets for various projects, that's where
the future goes, right? Even amongst the core devs, there's a bunch of them working on things.
So if I went to a keynote or you went to a keynote at Apple, Tim Cook, Tim Apple would come out and
he would tell you, this is what we're doing with our new stuff. And this is what the team has been
doing.
You go to Microsoft Build or Google I.O., there are people who come out and say, here's the
direction we've decided for you and us as a community. Hope you like it. There's none of
that in Python, right? But partly because who's going to speak for the 10 different web frameworks
and the 1,000 different data science things and all that kind of stuff, right? So I tried to put
that together. I tried to say, all right, where's Python been last year? What's important and where's
it going in the future? So fun stuff.
Okay. So my, I mean, one question, one, not question, one problem I have, you know, I'm quite
involved, but I'm very involved in Django and I don't know anything that's going on over there.
And all of a sudden I'm like, oh, wow, there's this brand new thing. How do you keep up? How
does, how does one keep us kind of overview of what's going on in the different areas?
I have a bit of a cheat card in that regard.
Yeah, well, you got Python bytes.
Yeah, I got Python bytes, right? I got TalkPython.
And I got Python bytes. With TalkPython, I can, I can dive into deep topics. Like one of the big
things is, you know, the whole astral UV stuff that's going out, right? And so I'm talking to
Charlie Marsh in three hours or something about UV, right? So I get to ask all those questions
and kind of figure that stuff out. But then Python bytes, like six, seven, eight topics that you've
found every week, and you can not just get to, you have to find another six for next week. And so it
just keeps you kind of on top of always searching.
But for the, for the, for the rest of us, we should just listen to your podcasts, right?
That's how we keep up.
Take it on coast.
Yeah, exactly.
So do you, we talked about this last time. I mean, when you're so involved in talking about
and producing things around Python, do you still get to code a little bit, right? Because you had
recently, we're going to talk about infrastructure and deployment, but like,
in terms of actual Python, Python, do you still get to keep your hands?
Yeah, it seems like I would just be busy talking, right? But there's a ton of infrastructure
and stuff going on. There's kind of a whole DevOps side, but you know, the,
the talk Python training stuff is it's basically its own SAS, right? Evolving over time in lots
of ways. I just noticed I was doing some stuff in GitHub. I'm, I'm trying something different. I'm
always busy. You know, I talked about all these different projects and then I have people working
on the courses and I've got, you wouldn't believe it, but I've got a lot of people working on the
courses and I've got a lot of people working on the courses and I've got a lot of people working on
how much email and it's always just pulling me away. So I have like big ideas for stuff that I
would like to add. And it's always like, well, maybe next week. So I was working on something
I'm calling platform week where turn off the email, turn off, I have no meetings. Who knows,
maybe even I'll go stay by the beach or something. I'm just kind of more alone. I haven't decided,
but just sit down, come up with a plan. Like if I had 40 hours to just write code
and sort of tackle the bigger problems that I've been thinking about, I would do that.
That I should, what would I do? So I was working on that and I realized just for the courses I'm on
issue 1,690 or something like that. It's mostly me, but also a little bit, a little bit of stuff
is in there around, say like our mobile apps where I worked with another developer and things like
that, but that's a lot of code. Yeah. And what's, what is it now that it keeps you excited? Like
when you're, you're coding away and you're actually just focusing on the code side. What are the
exciting bits that you, you like, Oh yeah, I like doing that. Or I like this. Yeah. There's just
always stuff to learn. There's always stuff to get better at. You think, okay, I've really got this,
this web stuff nailed. And then HTMX comes out and you're like, Oh my God,
this changes everything. Right. I know you both are big fans of that. And
you're like, okay, I got HTMX nailed. I got, I got the web stuff nailed. Oh,
maybe we could deploy it in a whole way better way. This whole thing I've been doing, you know,
I could, I could completely change that as well. And it's, it's never ending. I, I think the fun
thing is, is it's every time you kind of go through that process, it's better. Like Python is better.
The web frameworks are better. The infrastructure and tools like databases and web servers all the
way to the hosting, like the whole spectrum is just like, Oh, this is even cooler than last time
I dove into this aspect of it. And so it's, it's worth doing that a few times. Yeah. The excitement
is like, it,
last few years, it's been like, Oh, wow, we've got, you know, new layout things on the front end.
We've got, you know, CSS finally catching up with where we wanted it to be. And, you know,
and then on the backend, all the async thing stuff has come through and it's, it's always fresh.
Yeah, absolutely. I just, just yesterday I talked about Mountaineer, the web framework,
and a few weeks before that was fast UI, not fast API, but fast UI from Samuel Colvin and team. And
those are, those are both super interesting projects. There was like Python,
deeply integrated with a react in interesting ways. I don't do a ton of react, but, you know,
like when you write a function in your web app in Python and Mountaineer, it automatically
creates like an API connection for you. And it generates a TypeScript type safe, like front end
thing to call that and keeps them in sync. And there's just a lot of, there's a lot of neat
stuff out there, right? Well, it's good to have that attitude, right? Because I think you can
also, after a decade plus getting this, you can fall into the trap of thinking this is just
reinventing what I already know, which is a little true, but it's also, you know, generally better,
which is why people are taking the time to reinvent the same things. But that might be more
if you, you know, if you're three years getting really good at something and then, you know,
a manager says we're switching and get up to speed and you're like, ah, I have a feeling
that's more where that, that feeling comes from or you're, you know, the market changes and it's,
yeah, it can take a little time to feel like you're catching up, even though you've been
one of those areas. It can. And if you're, if it's not your choice to go down that new path,
that can be frustrating, right? You're like, ah, we, we were really good. We just server rendered
code. What are we doing? Like, why do we got to change all this? And honestly, I still think
that that's, that's true. I think it's, it's pretty amazing, which I guess is why HTMX is
also really interesting. It's not as bad in Python though, right? I mean, we could be in
JavaScript, not knocking the language necessarily, but it just changes.
So fast, the lifetime, you know, and, and backwards compatibility seems to be something
that people are like, ah, maybe if it's convenient, you know, I just saw that they're working on alpha
on a tailwind form. Like all that does is fill me with dread. I'm just like, okay,
what is not going to work anymore that I don't quite realize combination of 10 classes are going
to break. Yeah. I'm quite excited about the tailwind for, because it looks like it's bringing
CSS. It looks like, you know, using native layers rather than the tailwind layers and
things like that. So it might be, you know, that might be nice to, because it's not the
case that you have to stick with your tailwind forever. You don't have to do everything in,
in JavaScript, right. But you can work it back into your style sheet. And if they go
closer that way, I think it might take away a few of the, um, kind of religious criticisms
of it.
Excellent. Well, it looks exciting. So we'll see. Will, are you a tailwind
fan? Do you, do you believe in tailwind?
I have a redesign of my main site that is not publicly live yet. That's in tailwind,
but I have to say I'm not a tailwind expert. So I worked with designers. Imagine that who
did an amazing job on this site. So I'm sorry.
I think we talked about that, uh, on Mastodon.
Yeah, maybe. Yes, that's right. Yes. You were, you were looking for, um,
yeah, I used, uh, YupGup. Um, we're great who do a lot of work in the Django space. So,
uh, I highly recommend designers. They're a lot cheaper than programmers and it's, um,
so yeah, so I've, I mean, I, I definitely wouldn't say I'm super comfortable with tailwind,
but I'm sort of really happy about that. I have enough to deal with, with your tailwind
interested.
Yeah. To that point of just like, I like I'm, I'm pretty familiar with bootstrap. I've used
a lot of bootstrap and there was a discussion of like, should we use bootstrap? Should we
use tailwind? And it's like, well, I know that whatever it is, we'll probably have to change
it in five years. And tailwind seems pretty established and a little easier for me to
maintain once someone else does it. Um, but I don't have to worry about architecting it. I mean,
cause so that was the decision and you know, it's a, it's a little annoying to have to have a
separate server running to, uh, uh, you know, have NPM going, but it's not the end of the world. Like,
yeah.
Yeah, that's true. It's not the end of the, but you do have to figure out, okay, well,
there's these little scripts like watchers and stuff. I've got to write. Yeah. Yeah.
And then deploying it like is, you know, it was a day to like fully figure out like, okay.
Like even just on Heroku, which is still like what I use, like, it's like, okay, wait, where's
the thing for this? I mean, that's the, you know, we, we, we both work on courses. Like
I'm like, how does anyone figure this out? Like why, why does this take so long? Why
isn't there just like an answer? Like I will pay money for an answer. Like someone just
can just like save me two hours for 10 bucks or 20 bucks. Yeah. But I guess it's the change
is too much and maybe it's not worth anyone's time, but, um, so it was, so yeah, I'm trying
to fight the don't change things attitude on all this.
Oh, just, just on that. I saw a thread this, I saw a thread this week and it was from a
front end and they were like, all I hear is front ends too complex. Front ends gone off
the complexity cliff. Front ends crazy with its complexity. But what were their, like
what are you supposed to do? Just sit there and not look for the better solutions, not
try the better things, not progress. And at each step their argument was that, you know,
it made sense that we brought in web pack. It made sense that we switched to this. It
made sense. And you know, they were just searching for a better solution and it's all right.
It's all right for us. Python deaths go, Oh, you JavaScript is crazy. Was that kind of
crazy?
I think, well, I think it's two things, right? It's the people pushing the envelope are large
companies with big staff who.
Have maybe needs that the rest of us don't have. Um, so does your average website need
all these things? Probably not. Um, and at the same time, you know, selfishly, we all
want to feel like we understand everything going on in our web app, but obviously as
it matures, that becomes harder and harder to say with a straight face.
Yeah. Yeah. It's probably true. Also the big teams is probably more true. The big teams
that they don't, not the people working on them don't necessarily understand everything,
all the moving parts, right? Cause you've got teams.
It's team members that deal with like, Hey, how do we keep it in production?
Right, right. Um, well maybe that's a segue to infrastructure because that's, uh, one
of our big topics and you have a number of notes to dive in. I guess, have you, you've
changed your infrastructure or you're thinking about it? Where, where does all this research
come from?
Oh, no, this is all, this is done. This is, is kind of the, the, the infrastructure equivalent
of what I refer to as the platform week. Like I'm just going to take some time, focus on
this problem.
I'm going to put everything down, suffer the, the backed up inbox later and try to dig my
way out and get a bunch of changes. I think this was, how was it? What was really the
impetus for this?
Yeah. I was going to ask how, why do you, why dive into this? Right? Like what, where
is that point where you say, you know, because you're, you have traffic, but you're not Google
level, right? Like how do you decide it's worth, it's worth customizing things beyond
just, you know, paying a platform as a service bill.
Yeah, absolutely. So I just see, I don't know that I would say it's big and it's not totally
small. I'll just give people a sense. Uh, we do through all the talk Python things,
we do around 9 million Python backed requests a month and we do about 10 to 12 terabytes
of traffic a month. So it's a decent amount, but it's not, it's certainly not Google. Like
that's a minute on Google or especially YouTube or something. Right. But it's a, it's a, it's
a non-trivial amount of sort of traffic. Right. So for a while I had, so let's take a step
back. So for a while, at least in the early days, I kind of had this, what I think is
not really the way anymore that a lot of people think about, but I had this belief that the
cloud was small commodity things, right? The cloud is, the cloud is best at giving you
a bunch of small things.
Right. So I had this belief that the cloud is best at giving you a bunch of small little
things. They're super cheap. I can get a $10 thing here and a 12, you know, $12 thing
there if I want to optimize the CPU to be an Intel, whatever, and so on. And I can piece
my stuff together with that. Right. So I have a server for this server for that. I got some
backend servers, I got some failovers. So I got, and I ended up with like eight servers
sort of in that constellation of things. And I thought, okay, this is great. So I've got
the main stuff. I've got failover servers. If I need to do maintenance and say like the
kernel is patched.
And there's a security patch on the main one. I've got to reboot that server. There's not
too many interesting tricks I can do besides just making it reboot. And so what do you
do? So do you use some kind of load balancer to switch it over to the cheaper backup one
or, you know, something like that. Right. That was kind of how I was doing things. And
it real, I realized like I've got, yeah, go ahead, go ahead Carl.
I just wanted to bump him. Cause like you said, 9 million requests a month. I just quickly
did the math. There's about 6,000 requests a minute or something.
It's quite a lot, right? So you can't just reboot your server because you've lost 6,000
requests.
It's yeah, it's a lot.
It's all right. If you're just running your blog and it's like, oh, one person, maybe.
Yeah.
Anyway, carry on. Sorry. So you, so how do you handle the fall over and how do you handle
the load balancer you were saying?
Yeah. Well, I was just sort of cycling between these servers and using, you know, sort of
IP magic and stuff like that to, you know, sort of dev ops-y stuff. But what I realized
was, well, and I have all these servers to maintain and I've got to patch them all. And
it's just a bit of a hassle. And then, I mean, you could make a platform as a service
argument. That's, that's certainly a thing. And you could talk about Heroku, but there's
an advantage to not being a Heroku as well. And then what I realized is, well, if you
got to take the database server down as one of those things, they're all going down.
Anyway, you know, there's, there's, there's probably 10 seconds of downtime per month.
That's unavoidable unless I get into a more complicated database cluster with replication.
And now it's like, okay, I am not Google. I'm not YouTube. It doesn't justify that level
of uptime. I'm willing to take 10 seconds of downtime a month. Very rarely, like more
if I screw up something, but you know, which does happen, but not that often. But anyway,
there's kind of this single point of failure anyway, and I have all these servers and then
that was one half. And the other half was, and I think this is where a lot of interesting
stuff is going to happen. And the conversation here is there's actually a ton of open source,
similar things to what you might use, right?
So let me just give you one example, MailChimp. MailChimp used to be a cool little startup.
You could get your mail over there and you could sign up people and you could make a
and it was amazing and then it grew and it grew and then it was bought by a financial company
that just wanted to turn the screws on folks from what i can tell i'm sure there's people who don't
just want to turn the screws so no offense but it's it's got a slightly different business model
and it doesn't have that like i'm supporting the small the small person anymore right and my bill
is just going up like a hundred dollars every two months like hey we've guess what your bill is now
a hundred dollars more sometimes because there's more subscribers sometimes just because well
we've added a new feature you're not using so it's a hundred dollars more again you know it was up to
like i don't remember something like six six hundred dollars a month or something i was like
all right this is getting out of control like and so i realized there's like a couple of really
cool options for just self-hosting something an open source version of this
all right well they've got these eight servers where do i put this self-hosted thing well i'm sure
not going to put it like right next to anything else that has access to my database that because
i'm not in control of that code how much do i really trust it you know i trust it but
there's trust it and there's put it on inside the room with you you know like that that animal looks
friendly oh it turns out it's really aggressive at night i can't believe it's in my house like
what have i done right like that kind of thing so i'm like all right well i need to set up something
to kind of manage that right like all right well maybe i need to set up another server or
maybe i should just this is ridiculous and i should start using something like docker
or containers or something so like that's one half of the the story and the other one is
i was interviewing or reading different people talking about like this cloud commodity a bunch
of small things maybe that's not the right way like
for example i got a chance to uh have a mark rasanovich the cto of azure on talk python
we sat down and we were talking and it's like oh yeah we're moving towards more consult more
big servers rather than a bunch of small commodity servers and they have they have
some that are up to ridiculous levels like single servers you can go get that are you know 1700 core
with many terabytes of ram and they every every time they put out a new one they have a different
adjective for it so it started out with godzilla then super godzilla and now the one that they have
uh that's really big is like mega super godzilla beast or something like that right but they're
they're running out of ways to describe like how ridiculous it is um so so that and the david
heinemeyer hansen stuff like leaving the cloud i don't know if people have been watching that but
like super godzilla and now the one that they have that's really big is like mega super godzilla beast
similar thing we're going to get like a couple really big servers and we're just going to use
modern software to sort of devops our way through this better than and so i kind of went down that
path because i'm i'm like all right well i want to take advantage of not just python but all these
cool open source things for both running your code for for extending services for getting away from
unfair sass to having not just unfair sass but you know your your gdpr statement has
this other service we're sending your data to at a very high price we pay for the privilege of doing
so and then here is their gdpr statement and their cookie statement and they're this and they're that
and it just it makes operating an online property a little bit of a pain a little bit fraught with
that kind of stuff not to mention blocking right like if it's some kind of api endpoint that uses
their domain.javascript thing and maybe that's blocking it's not blocking it's blocking it's
tracking right so there's just all these challenges of integrating with other people's stuff
so there's a lot of advantage to running things yourself but then you don't want to let the wild
animal like into your house what do you do yeah okay good so i want it so one thing i want to pick
up on before we perhaps switch over to the container side of it it's just on hosting your
own email list is the big scary beast that they always wave at you is deliverability is is getting
your your email delivered and if you self-host
how do you handle those deliverability issues how do you yeah that's certainly uh a concern right
so there i people should disentangle two things there's running your own mail server and then
there is running the database that holds the list of people on your mailing list and so for example
one of the things that i chose uh there's a ton of them uh you could choose but i chose something
called list monk which is free self-hosted sort of docker first
type of of i don't even think this one has a hey you can pay us money
um list monk dot app yeah so when you go there it says well how do you use the thing
it right there it says okay what you do is you literally
uh use this uh say create a docker compose yaml file that has a list monk latest and it also
has um you set up postgres i think it i think it's postgres there's a big
basically two um two servers that you got to run there yeah uh the they use the post postgres
thing and then just the web app which i think that's written in go yeah i'm pretty sure it's
written in go but it doesn't matter because you don't have to think about it you just go there
and it says here here use this is how you plug it into your infrastructure right as part of
uh your system so back to your question carlton um that plugs into
what people are probably already using to deliver mail so it has it like a drop down for are using
amazon ses are using sendgrid are using mailgun which you've already got your dkim and pk whatever
stuff set up and you because because you're already sending a ton of mail out of that right
yeah okay right because that's the thing is that when you like running a platform as a service you
have your transactional emails already you know someone signs up reset password that kind of thing
through let's say sendgrid and then you can separately
have your database that holds your newsletter which you also send through the same what's what
is it ses what's what are they even called there there's some acronym for ses is the amazon one
right simple yeah simple email service but i there's something yeah i'm sure there's an
alternative and an opposite of transactional email service but yeah yeah so yes they send it
this they send the emails yeah the email sender yeah so carlton i'm not hosting
my own mail thing that seems fraught and also it's not okay and then as long as you're well
yeah exactly and my reputation is 100 at sendgrid so as long as i'm well behaved
maybe it's 99.9 or something now but i gotta put on my old man hat like 2010 when i was at
a company called quizlet we had millions of users and we had to do all the i guess primarily
transactional stuff that we sent a couple a couple marketing in quotes emails and we had
and it was just like just stored in like a php app and you know it's basically impossible
then it's worse now to just send a million emails or two million emails unless you have
this built-up reputation right because gmail yahoo like it's it's undoable and i think mailgun
actually had just come out of y combinator at the time but like yeah so like you can send them but
they're no one's gonna get them yeah they're not going anywhere no they're not going anywhere it's
it's it's a nice little game
there but but yeah that's for anyone who's thinking i'll just self-host at all like
it's not going to go through because there are these guardians around reputation and if you get
a certain number of spam it's actually this is what i tell like my non-technical friends i'm
like if someone's sending you email just flag it as spam and they will stop like they will hard
stop because unsubscribe is just kind of like a option it's like please unsubscribe me there's
dinged essentially and they will automatically pull it yeah yes i actually set up a a thing that
will listen it'll set up a webhook that'll listen to unsubscribe events across different things and
so sorry and and well that's obvious but spam requests and bounce requests and and those types
of things that are not typically we're sending back responses and through zen grid they'll tell
you hey somebody somebody
marks you as spam or something like that at least on the major platforms google and microsoft and
stuff and so you can then unsubscribe them so you don't keep sending it and keep adding to your bad
score and so that's right that's some of the interesting things that you can do uh it took
a little bit of right code to make it happen but it seems like that should be built into one of
these services right it's built into this list monk thing but it somehow doesn't come back from
zen grid correctly i don't know i'm like it's it matters enough that i'm going to just take an hour
and write a little api that just listens to this email doesn't like you for some reason unsubscribe
now and as this primarily this is for like marketing emails as opposed to transactional
exactly yeah so the transactional you already had through i forget which one you said that was all
yeah sangria this right right because the marketing ones that's where mailchimp and
convert kit and all them come in and yeah they were actually um there's a jango hosted one um
button down yeah but you have to pay for it but sure
as well yeah that looks cool we're gonna have justin duke on um a little bit later to talk
about it awesome but yeah that that seems to be the progression right it's like hey we're indie
and then all of a sudden intuit owns you yeah exactly they and it's like free for the first
couple thousand and then it's like wait how much is my bill yeah yeah it's always struck me that
email providers are quite expensive you know you look at them and it's always like okay it's only
but it scales up and it's like oh wow actually it gets it gets expensive but the transactional
ones aren't that expensive no no no the marketing ones yeah yeah the marketing ones like marketing
ones it's like 100x yeah like markup and it's like you're still just sending an email like and
my issue is like i want like a base version like i don't want to do a drip thing like i just want
to send email and i'll have like personally i'll happily pay someone not to just do that and yeah
so we're gonna have to talk to justin from button down and these other solutions and i'm gonna have
solutions you know because i'm not at a scale but there seems to be no in between you know broadly
speaking right it's really i guess designed for larger companies and the marketing departments
once they get hooked in like they're not writing an api endpoint they're not looking at this they're
just like just throw money at it and move on yeah absolutely and there's uh i don't know i guess
i guess that's just the space but there's that whole area is just i i kind of feel like they're
just overcharging now you look at places like hubspot and they're just like they're just overcharging
and they're like you know what it seems totally reasonable that it's a thousand dollars a month
to have a few landing pages you know what i'm not really sure that that's true oh by the way you can
pay that in 12 months you know yearly chunks as well so it's like wow okay right i didn't know
that well there's no i loan i need a loan to get like marketing software right i mean there's a
company here in boston that just ipo that's um django based or was originally um clavio with a k
that i believe also does
like like if you look at a lot of like e-commerce companies they handle
you know marketing emails essentially and um i know there's is it still around there's a company
called drip that was doing oh yeah yeah drip is that that kind of thing yeah but like clavio is
like that but like 10 billion dollars or whatever it is so like huge companies um but yeah i guess
there just isn't a market for like solo or smaller places right they don't really care right and
it's like well you can go open source if you can otherwise
we'll just we'll take your money yeah anyways button down shout out we'll talk to justin about
it i think he's trying to tackle that challenge but the space in general doesn't doesn't focus on
that this episode is sponsored by mail trap an email delivery platform that developers love
an email sending solution with industry best analytics smtp and email api sdks for major
programming languages and 24 7 human support try for free at mail trap.io so michael you decided
you were going to stick everything all on one
one big box what kind of box did you get and where did you get it and is it a physical thing you have
to drive to the colo and change the hard disk at two o'clock in the morning or is it i think you
know i mentioned the the dhh for leaving the cloud and those guys went and spent half a million
dollars on servers like that's not you're also not you're also not 37 signals probably for the
general people listening who would consider but it's totally reasonable to say i'm going to go
to somewhere like digital ocean or linode or somewhere you know one of the things that is
struck me as like interesting but you often get sort of too deep into the machine and then you're
stuck is just go to aws and look at the the console of services that are there you know it's
like you gotta zoom in just to see what they are anymore there's so many sort of thing right what
is going on here and you know it's it's totally reasonable that all right this this database thing
is like starts at twenty dollars a month okay that's fine and we got this function list and
then oh we're gonna need caching for this thing then over there and then all of a sudden your
code doesn't run in general places it only runs in this one particular cloud and the aggregate of
this because you now started doing too many ses sorry s3 transactions reads and writes so now
that's expensive but you got nowhere to go or whatever right bandwidth is high then then you're
I think it can be frustrating.
And also, it can kind of take you away from just the simple Python solutions.
Like, then how do you even test that kind of stuff locally end-to-end?
You know, it's like, well, I don't have all these.
I know there's containers you can set up, but, like, at that point,
you're running a different sort of complicated infrastructure just to write code.
So to answer your question, Carl, then just I got, like, a $60 server over at DigitalOcean
and just put everything on that.
Right.
There's some really interesting things.
You get a lot of stuff for a $60 server, right?
You do, yeah.
It's 8 gigs, 4 CPUs.
What's really interesting is the bandwidth.
So if you did 10, let me just do the quick math.
If you did 10 terabytes of traffic, the CPU, the EC2 is actually not as expensive as it used to be.
But if you did 10 terabytes of traffic on Amazon, it's $921.
It's $921 a month, right?
It's $0.09 a gigabyte, right?
Which doesn't sound that bad, but then you multiply it by big numbers and it goes up.
DigitalOcean, you get most of that for free.
It's, like, bandwidth is, like, $50 a month.
I was like, oh, that's interesting.
That's quite a big difference, no?
It's not a non-trivial difference.
And, you know, it scales linearly with how much traffic.
So as you get bigger, it's just different.
Why is that?
Is that DigitalOcean trying to compete with AWS in one of the few ways they can?
Every machine that you get, every, basically, tier of a machine that you get at DigitalOcean,
I think Linode is this way, too, but since my acquisition, I don't really know anymore.
You get one terabyte of included bandwidth with each little machine you get.
And if you get one big machine, I think I get six or eight terabytes of credit.
I get it for outbound traffic.
Whereas on Amazon, it's like, well, it's just nine cents a terabyte or a gigabyte.
It's not so bad, right?
But then it adds up.
I think there is a quite generous starting thing.
I mean, most small sites wouldn't pay anything for bandwidth.
Yeah, yeah.
And even if you were a small site, like, also, it just doesn't matter, right?
But if you start shipping bigger things like MP3s and videos.
Yeah, videos.
As soon as you're hosting videos, you're like, ah.
Yeah, yeah, yeah.
Well, and you, yeah, you, so you don't use, like, I don't know if it's still the case.
For a long time, Vimeo had, like, unlimited for, like, $99 a month or something.
Is that a sigh there?
Talk to me about hosting videos.
Hosting videos.
So, yeah, I had some stuff going on with Vimeo for a while.
Paid for their business tier where you could white label, re-host it.
And it was actually a really good deal for a while.
And it solved a problem because video delivery, CDNs, all that stuff is a super hassle, right?
You know, re-encoding stuff so it streams the right, you know, on Android.
It wants a different size and format than it does on iOS than it does on, yeah, it's a nightmare.
So, they handled it.
I was happy to pay them.
And then they changed their policy, like, oh, if you have too much bandwidth, then you're going to start paying.
There was a famous example.
I can't remember the exact details because it's been so long.
But there was, this is somebody who was not going through the business plan.
They just put, like, as if you were on YouTube.
They put up a video and it went viral.
It was a creator type, not somebody trying to create a platform or anything.
And they got a letter from Vimeo.
You know, saying you're going to need to pay us, like, $1,000 a month for the bandwidth that your viral video is costing us.
And I'm like, what?
I just put, like, you're a video.
We posted a popular video.
It wasn't popular at first.
It became popular.
And, yeah, they had some changes in policy that just really kind of put the quash on that.
I don't remember the exact details.
But around that time, it feels like it was five or six years ago that it.
It was, it just, things changed there and it didn't make any sense anymore.
Kind of like MailChimp.
Like it was cool for a while that you could do stuff there, but things change.
Well, what is, I think this is also Boston-based.
There was another like more expensive, better Vimeo.
What is that one called?
There was like one other one, right?
Is it Wix, something like that?
Yes.
Wix is the building.
It was like that, something like that.
Yeah, something like that.
I can't remember.
But its pricing was, even from the start, was like kind of outrageous.
I think, yeah, it's probably back five, six years ago.
I was looking at this.
It was still, it was like, I was like, who can afford this?
You know, it's like big companies, I guess, big companies with marketing departments
that just want to pay money and have it solved.
But yeah, I guess it had all this analytics and everything.
Yeah.
Yeah.
It was awesome for the right group, but not for the wrong group, I guess.
So what are you doing today?
Yeah, you're, no, you're right.
It is Wix.
You're correct.
Yeah.
Okay.
They're still around.
I guess I was thinking about Weebly, all these weird names.
There's something that's like Wix, though, for building web apps.
It's like a Squarespace competitor.
I don't.
Yes.
I don't remember exactly what that is.
So basically, I threw this whole process.
I realized, oh my gosh, there are an amazing set of open source projects that sometimes
you can pay small amounts of money to.
Or.
If it makes sense, you could also self-host it, right?
This list mug example I gave you, you have to self-host it.
But another one that's really, really interesting is the web analytics, you know, Google analytics
alternatives sort of thing.
Probably the best known one would be Plausible, Plausible.io.
You guys know that one?
Yeah.
Yeah.
Yeah.
So those, those folks are based, I think, in the EU and are much more concerned about
GDPR.
And I think what, you guys know, Carlton, maybe you know, you live, they're in your
neighborhood.
There was some ruling, either France or Germany, that basically said Google analytics violates
the GDPR to some effect.
Yes.
And then another one, which are those pairs that said Google fonts violates the GDPR,
right?
Right.
Which is even a little more subtle.
But you, like, so the trick, I mean, I know more about the Google fonts.
It's something I've looked into and considered.
And I always, if I want to host a web font, I will always download the web font and host
it locally because actually...
They don't let you do that anymore, Carlton.
Yeah, they do.
They do.
No, no, no.
I was there yesterday downloading a font.
They've got it downloaded.
Google, Google, Google does.
Yeah.
But, but if you like, because I went through this with the design process, even all these
fonts, I was like, we're talking to designers.
I was like, I'll pay a couple hundred dollars to download a font.
And it's like, nope, you can't.
Anything that's not Google fonts, basically, they're, it's a SaaS.
Fonts as a service.
Fonts as a service.
It's...
Yeah.
And it's so expensive.
Right.
And I'm like, you know, like there's, I forget the font that, you know, OpenAI's font that
they're using is, you know, I calculated the bill based on like, obviously they're getting
a discount, but it would be like tens and tens and tens of millions of dollars a month
just for the font.
And yeah, anyway, so just side rant, like, yeah, I just, let me just download the thing.
And yeah, I guess Google lets you download them, but all these other fonts, I was like,
oh, I want a bougie font.
I love fonts.
And I was like, nope, that's like...
It's basically not possible for the majority of...
Yeah.
Okay.
So anyway, so I do always download them because I don't want to, for my, you know, even forget
the EU regulation.
I don't want to put my used, people come to my website in good faith.
I don't want to then have tracking stuff embedded in there.
I'm 100% with you.
So that, you know, big, big megacore can form their shadow profile.
I'm just like, no, I'll download it and self-host it and that's fine.
So the thing with the analytics is like, which are the, which are the ethical ones?
Which are the ones which don't do exactly that?
I'm going to leave some out for sure.
But so plausible.io is really interesting.
They've got a, just analytics as a service, which I think probably makes sense for most
people, right?
But it's buried deep down in there.
I mean, it's not buried, but it's, it's not front and center.
Click here to self-host it, but you can self-host plausible as well.
But one that I'm using is something called Umami, which is more.
Yeah, I've heard of that.
Umami.is.
It has a pricing version where you can just go get it your own for $9 a month.
Or you just go, it's got a GitHub 20,000 stars and fired up your, your Docker compose file
and, and off it goes.
And what's really especially interesting about analytics is the,
I actually took Google analytics off like four or five to four years ago and just turn
it off because I'm like, it is not just a privacy violation to basically have it, but
it is also kind of useless because so many apps, web browsers, pie holes, everything
just block Google analytics outright.
And so you're getting a really partial view of what your web traffic is doing, but with
anything self-hosted, you can put it on your,
your domain and it just looks like part of your web app.
So for example, for Umami, I self-host it in a container on the same big server with
all the other things.
I've got like 15 things running over there in there.
It's on the same domain, the same IP address.
So when browsers look at it, go, yeah, that's fine.
That belongs to part of this web app, right?
Yeah, it's just a, and you get near a hundred percent, unless you turn off JavaScript, you're
getting almost,
you're getting almost a hundred percent accurate view into what's happening in your world.
Yeah, which is obviously much more useful.
That's a pretty, that's a pretty unique thing.
Most things are not like that, but web analytics are so subject to this privacy blockers, tracker
blockers, ad blockers, which I'm not ranting, I guess, at all.
I think that they're serving a very important role, but you know, as a website builder,
you end up on the receiving end of, you know, other people misbehaving badly.
So you're untrusted as well.
Kind of like the same email story, right?
Like, you know, people have abused the system.
We don't trust anyone anymore, unless you're Gmail.
Back in the, back in the day, you didn't have a web blocker, but then you had pop-ups.
So we have to have a pop-up blocker.
And I mean, it's, it's just warfare.
I'm not turning it off ever.
I'm surprised there isn't like a documentary on like, you know, the dominant web blockers,
right?
Cause the ownership structure is deliberately unclear and it seems, seems like they're owned
by like Google and Apple, like, but it just isn't like for real, you know, not to get
too Joe Rogan on that.
On this, but, um, yeah.
No, go on.
We need to break some numbers, but let's get over.
The dark analytics state is there.
That's right.
Yeah.
The deep, the deep, the deep analytics.
The deep, the deep, the deep analytics.
Don't, don't let deep analytics.
But I think it's, I'm pretty sure that I remember taking like 10 minutes once to look at it
and it was just like, well, we don't know who owns it, but there's all these carve
outs for these companies and like, who would want to, who would want to own, who would
want to own it and has a lot of money?
Like, oh, we're going to block everything except for our ads.
You know, something like that.
Right.
Right.
That's right.
Yep.
Yeah.
So I actually run, um, next DNS, uh, for my DNS and that, that stops everything.
That's beautiful.
Even my television has ad blocking built in my, all your iPhone apps or Android apps
built in ad blocking.
Cause that network endpoint is unreachable.
So if you, so like Amazon prime now for their videos that you just get a blank thing when
it tries to put ads on there.
I, you know, I haven't watched an Amazon video.
Since they put ads on there.
So yeah, I think generally the stuff that's in the stream is kind of come through the
same source, but in most of it, you know, even if you go to, if you go to some mobile
app and you hit a thing and it opens up an embedded browser, like the ads are gone every
now and then my daughter, daughter was like, dad, could you put the ads back on, on the
network for a little bit?
I'm like, why do you want that?
She's like, well, if I watch this ad, then I'll get a credit in my game and I can't watch
the ads in my game because you're stopping.
I'm like, we're winning.
But yeah, no, I was thinking this cause my kids, like for the longest time, you know,
they've never seen ads on a screen.
And now one of these things or something they want to watch and they figured out that my
kids like Legos.
And so now it's like, Hey, I want this Lego.
Okay.
What does it cost?
What?
Oh my God.
You know, like, so it's, but it was, you know, in a way it was this was, is this golden
time of kids not seeing ads?
Like, you know, cause growing up, like.
Like the radio that like, you just bombarded with it.
Right.
Whereas most kids, you know, well, I guess, you know, if you pay for YouTube premium or
get a family plan.
But anyways, yeah.
So when someone looked at that and was like, you would pay more to not see those ads.
Exactly.
Exactly.
So, yeah, I think anyway, so back to the analytics, I think, you know, there's all these different
ways in which that, that information either is somewhat ill gotten by sending it through
a big tech or just.
Just corrupted because it's so partial from, you know, the different levels of ad blocking.
So I think that that kind of self-hosting stuff is there.
I want to give, I'll give you guys a whole long, interesting list of things here in a
second, but I want to kind of just step back for a second and talk about like this eight
little servers versus one big server.
And if you think about, you could say, all right, well, the, the big server is really
roughly every app has about the same Ram as it had before.
Right.
In a, at least in a sense, right.
If they're all still using a decent amount, but if you have a bunch of small servers and
there's some spike in say one of your APIs, or there's a traffic spike in one, like talk
Python has like a big spike for some reason, Python bites had one.
Cause some guy was trying to DDoS it the other day, but it was fine.
I didn't mind.
Anyway, if for whatever reason, good or bad, it has a spike.
Uh, and it's on this sort of commodity, a bunch of small pieces world, it quickly runs
into the limit.
Like, well, that server has one CPU or it has two CPUs or whatever it gets when you
chose it and that's it.
But if you put them all together in some kind of like Docker cluster type of thing, those
spikes almost never overlap, right?
The reason that the courses would spike the first is the reason that talk Python versus
Python buys versus the APIs that run the mobile apps, whatever, there's no, almost no correlation
between the spikes.
Of one and the other.
And so it can, each one of those has almost all compute available to scale out and do
its thing.
So it's really interesting to think, you know, like I've got this little you mommy Docker
thing running and this list monk thing running and all the other apps and, um, web apps running
and they might have these different spikes, but they all have like a higher headroom to
do whatever they need to do for less.
Like I'm paying.
40% of what I was for the eight servers, the little baby eight servers to get one bigger
one and you get more compute out of it.
It's really interesting.
Yeah, no, I like that.
I'd like that.
It's yeah.
Love it.
Yeah.
It's, it's fun to think about, but so, all right.
So I mentioned a couple of these things and neither of them, uh, just, just before you
got gone, but just before you do that, let me jump in.
So you're running, so you said 15 services on this, on this box.
So I presume that isn't just some sort of long running.
Yeah.
It's a long screen session with, you know, shell, it's 15 tabs open and how are you,
how are you managing docker containers?
I believe maybe 17 with the different databases and stuff, but yeah.
Have you got some nice UI that, you know, helps you manage those or, oh, uh, not really,
but I, I, it didn't come with one because he's got docker and compose and different
things, but I used rich, you know, the Python library.
Yeah.
And did basically, uh, an in-memory join on the report that a bunch of docker and Linux
things return and it'll create a table.
And then with rich, something that's really awesome is one, you get color as I'm sure,
you know, but what people maybe don't know is rich has a context manager for live redrawing
of content.
So you say with live, here's the table and then you just start changing the table and
it'll change the UI without even.
Um, without even flickering or anything just over SSH.
So that's basically that.
And are you all familiar with B top?
No.
Oh, B top magic.
So B top, um, it's github.com slash a wrist.
Oh, Kratos slash a B top.
Yeah.
A monitor of resources.
Yeah.
But what's amazing, they don't have,
if you scroll down to the graphics section, like maybe half the way down that page there,
it's, you know, this thing I described with rich, this is not based on rich, but it's
like kind of that to the logical extreme.
It looks like a, uh, uh, Atari or Nintendo 64 game almost.
And it's just live updating of all these different panels.
Here's all your CPU across, you know, individual and total aggregate CPU.
Here's your network traffic, your processes.
Um, this is what your, your disc and memory usage looks like all updating live with graphs
and stuff.
So like, that's kind of the thing I've been using is just like, look at the, the server.
What's interesting is when you're running Docker containers, all the processes in Docker
all show up as separate processes in, in the, the, the process monitoring tools, right?
So you can actually see this part of my Docker container, like this worker process for this
server.
Service is using this much Ram and this much CPU.
And here's the graph over time and stuff like that.
It would be super cool.
If just thinking about like your custom stack here, it's almost like if some naive person
came in and you just had these dropdowns like, Oh, I'm using MailChimp, I'm using these different
things.
And then it was like, here's, here's a recommended other, you know, I don't know, you wouldn't
monetize it in any way.
So what's the point?
But like, cause as much as things are custom, there is some degree of generality around
all this, which, I mean, you know, Carlton, I almost hesitate to ask, you know, you have
spent quite a bit of time on a Django specific thing, um, button that was going to do some
of this for, um, for Django.
I guess it's just hard to draw that line and get enough people in where it's like, yeah,
I'll pay, you know, it's, it's not as much as all these services, but it's enough to
someone put a wrapper on top of it.
Yeah, no, I think there's a market there.
Um, you know, yeah, no, it's going really well.
So I'm, so I,
I stopped fellowing and I immediately in order to pack the bill, pay the bills, had to pick
up a new, um, thing.
So I've been working on a project called paths, which is an app in the environmental and social
governance space, but I'm deploying that with button on exactly the tooling that I want
to use.
And it's all going great.
And it's, it's super.
Um, and I think there is a market there.
It's just a question of, um, productizing.
So you build an app and then you, you want to productize it.
And it's, it's sort of three times the, the, the, the, the.
Three times the time to productize something as it is to just build the app that works for you.
Um, and it's that bit that is tricky because I'm already building one application for as a full-time job.
How am I going to build another?
Well, I slowly take away at it and I'll roll it out over time now.
But for me, it works.
It works well.
It's lovely.
I think there's a market there, but it needs somebody to sit down and, you know, have the time.
And where do you get that time without the funding to, to back it?
Just raise money.
Yeah, but as soon as you raise money, then there isn't a market there anymore.
That's, that's the rub.
That's so what have we talked about on this podcast?
We've talked about MailChimp used to be cool and then they sold out and they, then they became really expensive.
We talked about, I don't know, a Vimeo that did used to be good.
It was the best offer for hosting videos on the internet.
And then obviously they need to make more money to, to, to satisfy their, um, their investors.
And all of a sudden it's like priced out and it's corporate only.
And it's the same in this deployment world is like Heroku.
What happened to Heroku?
Well, it got bought out.
It got bought out by, you know, the, the big company there, and then they, they're like, we can't do this free plan because it's, it just doesn't satisfy.
And that's totally understandable.
Yeah, yeah, yeah, yeah.
I, I don't know how much their policy changed, but like they, they kind of got down the same path.
Yeah.
You can't have it both ways.
Either it's a small bootstrap thing, which you need to kind of grow organically, or you take outside money and then that you can't, you just can't serve the, the, the hobbyist developer or the small, the small independent developer.
You're looking at corporate.
And there's no way around that.
That'll be our first question to, uh, Justin of button down, but he's, but he's doing it that way.
Right.
He's, he's, it's a small thing.
He hasn't taken a lot of money, a lot of intent and a lot of, and at some point, right.
Like maybe you don't want to work on the same thing for forever.
Right.
And so, um, I guess, you know, and tech is so weird because it's infinitely scalable.
Whereas like a more real world example, like, was it like, um, the like oatmeal?
I like to get from this, like Bob's something or other, it's this old guy on it.
And like, yeah, red mill, right.
Like, you know, he could have sold to mega corp.
He sold it to the, uh, employees.
So now it's employee owned, like where I'm from in Vermont, there's a number of companies that have done that, but you know, that means that Bob or whomever, he could have made a lot more money.
Right.
But, um, you know, yeah.
And I guess it'd be nice if there was some.
Tech equivalent, I guess maybe there, there are some a little bit, but more around like agencies.
I know like torch boxes, employee owned, I guess it's really hard around products to do that in the tech space and stuff that has running costs, right.
Infrastructure bandwidth, uh, compute any outside money, then it's going to, yeah, no, but I think about outside as soon as you.
take the outside money you need the bigger revenues and you know the clock is ticking
on your 10x and you better find a way to get it but yeah i mean not to make this all about 37
signals but they famously took money from jeff bezos like way back in the day because i think
they were facing some of these pressures and it allowed them to you know stick it out right like
maybe there's oh you find a benevolent billionaire who'll buy 10 for uh some amount of money you
know i like i like your base camp app i want it to exist i'll just make sure it doesn't go away
something yeah yeah yeah their whole leaving the cloud series that david honeyman and henson wrote
is is super even if you disagree with them it's super worth your time of looking through it it's
quite interesting so you guys are i'm sure familiar with awesome list right so one of the things i
want to just give a shout out to is kind of like this whole thing we've been discussing kind of
that so if people check out products.com
containerize.com that is like an awesome list for you know pick your self-hosted thing
so you want to do blogging here's here's the choices you want to forum here's these are all
open source self-hostable right discourse or do you want vanilla you want you know node bb or
there's a marketplace they're all alternatives to you know shopify i guess would be the marketplace
equivalent right and they've like this is the one that's voted most by the market place
by by the users um you want chat like slack well rocket.chat is like the winning one apparently
here but there's you know what is that 12 other options um transactional email is it win winning
by github stars or they have some other what it's like uh it's it recommends it gives this little
like thumbs up and i think it's whoever runs this is like this is the one we've sort of given our
stamp of approval to more out of them
well yeah that yeah no it's very cool that's interesting right but then you end up like well
if i want to use any of these like well what infrastructure do i run this on you know what i
mean right you've got invoicing you're back yeah you're back to that right but i don't know i mean
every one of these things there's a massive learning curve to picking them up and taking
them on right it's not it's it's open source it's not postable but it's not free no it's not totally
free and i totally agree with that but what i would tell you is you will find a pretty common
here's a docker compose file do you have a place that can run docker compose if you can do that
then you can probably self-host this pretty well right you gotta have some kind of backup but
almost always it's postgres so you just run a postgres backup on the server you set up
and and figure out how to restore that later whatever that means for them
you know it's it's not as done once you start down this path you're like oh i've already got
10 of these things running here what's what's two more what's 11 yeah
you know because it's docker like you know you don't have to trust it as much you still got to
trust it with your data but it's not right in there with all of your other stuff on the inside
of the firewall and those kinds of things and so it's it's not as risky but it's pretty interesting
to go down this you know one um one thing i guess also two things here one is um have you heard of
elestio e-l-e-s-t-i-o elestio so these this is a it's not that cheap but it's it's kind of
empowering it's just this company that you say what is your provider it says hosted on digital
ocean aws vulture hertzner linode scaleway etc etc even on prem and then you say well which one
which open source project do i want so do you want click house you want uh open search you know
and they'll just set they'll manage the backups keep it running keep it patched
for you almost like as a service on your own infrastructure for like a bit of a cloud premium
but you just go in there and you type in you know one of these crazy things there's a really good
chance that they have it right there i think it's pretty fascinating i'll have a link to that and
it's a yeah all of these things that cloud premium it's like well what are you buying
what are you trading off because in my day
was when i was a youngster it was literally a sort of old guy who like had a beard
no but it was it was like yeah like it was a proper big beard and recumbent bike you know
he he had a full-time job and he was like you know whatever time of day he had to pay
like it was never like seemingly never sleeps but didn't do much didn't speak much but literally
his job was to keep the the actual physical server going and so when the cow came along and
they were like well you could get rid of that guy and that guy didn't get got rid of because
like he had you know he was he was just part of the furniture you wouldn't get rid of him
but that cost just disappeared and so okay the cloud server was more expensive than running
it your own but it was a lot cheaper than hiring the system absolutely absolutely and so you think
oh you know this service here that's going to be an extra few pence per hour but hang on what
are you trading off are you trading off it took you two days to work out how to deploy this
or do you need to hire somebody to manage all these things or can you just like pay an extra
hundred dollars a month yeah it's it's really interesting one uh django one that's really
worth um maybe you've already had these these folks on but have you looked at post hog post
talk no i've heard i've heard that no we haven't we haven't talked about i think i'm pretty sure
it's self-hosting let me just double check single platform to analyze test observe to
it's self self host yeah post hog what is the domain ending post hog.com provides open source
product analytics session recording feature flagging ab testing all of that as a service
or self-hostable and it's in django ah right well we're gonna get them on there right it's brilliant
does that not look interesting i'd seen i had now you bring it up i had seen it before i didn't know
it was built in django so that's yeah i mean that's what isn't that kind of cool like even
even the two even the two of them are built in django so that's really cool i mean that's really cool
even the three of us right like who do podcasts on this and it's like oh there's this thing with
like 17 000 stars that we didn't really know about that a lot of other people knew about
somehow somehow exactly how do we not yes almost 17 000 github stars and and written and it's a
really cool looking app single platform to analyze test observe and deploy new features for your
your web apps and and stuff so yeah that looks that looks quite slick
let maybe let me give you a one more category of interesting
yeah before we we close it out uh or we can keep going as much as you want but before we miss it
and i know um will that you're a fan of the heroku style of things right yes i am and then you end up
at my go ahead yeah i just i i just put a preface with at my scale it's i'm more than fine and you
know i know there was a lot of drama around heroku i don't want to i don't really want to give any
they're they shouldn't be required to run the internet for free yeah you know regardless if
they're just run by one person in a closet and no one else does either yeah exactly no one else
really does either exactly you know like digital ocean doesn't have free they have a four dollar
server which i don't know to me feels like free but you know uh it's not free right it still still
covers it for like the the super super indie little sites and just little small things and so
on but uh as part of this whole deal i came across
doku d-o-k-k-u are you familiar doku doku yeah i mean this is why i was wondering if you ran
something like if you used some kind of ui a bit like this but you don't you do it with the terminal
but yeah so doku is pretty interesting it's kind of it's got this you know create your own it's it
builds itself as open source pass alternative to heroku and it's kind of similar you create an app
and then you create a proc file and you get pushed to some branch and that makes it a little bit more
it go i talked about that on python bytes and people are oh that's interesting and somebody
wrote in i can't remember the name of them apologize said that's cool but have you seen
coolify c-o-o-l-i-f-y this is like uh one of the same things but a little bit a little bit more
full featured and a little bit better ui and a little bit more supported self-hosting alternative
for heroku netlify vercel i mean that covers a lot of
features there yeah free free ssl automatic database backups web hooks automation notifications
monitoring all the all that in just uh uh you know docker compose up
basically let me ask let me ask you as as an educator like what do you tell people who
just want to like learn web development right i mean because like this is kind of the issue is like
at the same time that most software engineering jobs are around infrastructure actually probably
you know if around the web but getting started like i don't know what do you say
someone comes to you is like i don't know what docker is i just have this i'm so excited i just
learned django and i want to host something and it could be uh yeah what do you say let's put the
data science folks off to one side because that's a whole different level okay yeah right because
they don't even actually have a web app but they need to somehow get it on the web right
i mean you've got to turn this spreadsheet into a web app with crud and off and everything yeah
which is more of it but but for the folks like i have a web app it does just what i want
on my machine how do i make it on everyone's machine right so i think there's two paths
as an educator you send people down one is just don't think about it this is what a connection
string looks like to a managed database server you get pushed to heroku or something some platform
is a service equivalent job done that's kind of the i want to make it happen i want to make it
happen and i don't really want to think about worry about or even hardly know about necessarily
like what do i need i just need it to just go and if i can get it here these people will take
care of it right that's the minimum path i think and that's probably the right path for a decent
number of people just like i wanted to get it to go the other side this isn't what i've ended up
doing more and i don't know people are like michael why are you doing this to me is is show
up on linux if if you because if you know how to set it up on linux you can run it on vms you can
run it on what do you got to do to do a docker container you put run or copy in front of all
the linux commands you know what i mean instead of saying cp the the docker file thing is copy
instead of uh you know apt install you say run apt install right capital r-u-n is the front of
that that thing to make it a docker command and so if you understand linux
you understand docker and that opens up all the you know containers as a service it opens up
kubernetes if somebody else is running the cluster like all it opens up that whole world
i think but there's a higher bar to say like okay well i'm a windows person and i generally don't
use the terminal and you're making me work in linux with no ui what what has become of me right
there's that challenge so i think there's this this bimodal you want to send somebody people
i think there's a trade-off as well with the past is that it's kind of like a dead end and that you
i've described it in the past the coldest that you go in and it's great to begin with and it's
lovely and pretty and there's flowers around and they spill the end but then you get to the end and
you're like oh actually i want to do something further and you have to kind of go back all the
way out out where you were to start again to get on to a more complex whereas if you go straight
to a vm it's slightly harder to get there to begin with but you can grow an awful long way before you
need to learn anything else at all and i 100
agree with that yeah the the sort of example that comes to mind is with channels which is the um you
know async add-on or web sockets add-on for django we the number one problem we have is people um
having difficulty connecting to redis because they've got a pass and they've they've got a
redis that they've spun up in the cloud somewhere and it's not simple to connect because you you're
doing it over the network and it's got to be https and it's got to be this and the other
whereas nobody in the history of
redis had a problem connecting to redis on localhost right it just never happened
yeah like if you've got a vm you can spin up redis sitting next to it and you're just you're
off and you start you never hit that set of problems and until you have a much bigger scan
you're like oh i need a much bigger redis well then okay you can put redis on a separate box
but by then you've you've kind of got the chops whereas like people really struggling early on
because they've gone the pass route and then they needed something which wasn't delivered by the
pass in its sort of cul-de-sac of first offer yeah yeah i think part of it too is it's enough to
create a website and market it and do all those things like in of itself and then to add in all
this other bit it's kind of like it's rare to have the success and the knowledge and the time
to do what you've done michael yeah whereas most people it's like on your own put up a portfolio
stuff and then go join a team and then you know i don't know i don't know i don't know i don't know
have the time to focus on an area it's pretty it's very rare to do all the things you're doing
at the scale that you're doing and i know you have some help but like still it's usually most
people are overwhelmed with no i totally agree with you and look when i started my stuff i ran
it on python anywhere and i'm just said here's my code run it i don't want to think about it
also bought by someone else yes i know conda anaconda i think yeah yeah yeah
then i slowly slowly over time kind of thinking like carlton said like ah it's just i need a
little bit something different right you know i think it was something like we're gonna have our
maintenance period will it be down for four hours like four hours i'm not really psyched on that
maybe i should learn a little bit enough to control my my my maintenance window to be shorter
you know four hours that's that's a long time and stuff like that right and i know i know we're
coming up on time but how how did you dip your toe
in right because it does feel a little bit like a boom like like were you able to do that gradually
or was it no i need to no super gradual it was how do i get like moving services yeah and then
well in the beginning i was just well in the beginning i just used sqlite because how much
how how how read heavy is a podcast right it's not that read heavy i mean sorry it's not right
heavy it's it's read heavy right so
that's actually pretty much fine for something that you updated a couple times a week
you know it's only one person updating it right it's one person exactly and me doing that yeah
exactly and so i started out with that and like all right this is easy i can i can productionize
this because i just let the file be there basically you know run it on on python anywhere
and like all right well maybe we need to move a drone server okay well maybe we need a better
way and stuff that's happening and stuff like you know and it it grows but it grows
you know it's you look you look back and i go look at all this stuff it's like it's crazy how
why do you choose this well it started like little little baby steps uh and you know after you work
with it while i was like well that's not that's not for me that's not that complicated but i know
that it is for others right you know the best advice i ever had was from you know senior when
i was a junior was like don't only add the bits that you need as you need them so you know you
don't worry about varnish caching and stuff you'll get to that if you need it like just
care about that yeah then yeah then you can dedicate the time yeah you should be so lucky
as to have a problem where you need that honestly yeah exactly exactly right or as i sometimes i'll
i like to tell people like when it's a problem it will be overwhelmingly apparent like you don't
need to you know
how will i know when a mind database needs to scale like well yeah when it becomes a premature
optimization like there's a quote on that like yeah and and that's true in code and that's true
and like how you try to run your code um well we could keep going on and on but we're a little bit
over time we'll have links to all these things um i should we'll put a link to you you have you
mentioned htmx you have a relatively new django htmx course absolutely folks should check out
um that seems i hope that's doing well for you
yeah yeah it's doing well worked with christopher trudeau on that and it's going well yes he's kind
of he's like your django person right he does yeah he's done a lot of stuff yeah not to limit him to
that but i when i see talk python stuff that does django i'm like oh that's chris yeah exactly
yeah um so thank you for coming on our podcast of course you have your podcast python bytes
talk python we'll have links to everything training around django stuff if they need it
and um
this was really fun this deployment as such is just an infinite topic and i think for you to
come in as pretty much a solo person at scale that's a really interesting perspective as opposed
to someone at a large company who came in late you know kate didn't didn't start from the beginning
and it's just like well here's my little piece of it but everyone wants to know how did you how
did you get there it's yeah it's really interesting to think of i mean my takeaway is it's so easy
and so
affordable i guess easy is not the right word it's so accessible right you know for a hundred
dollars a month or something you end up with infrastructure that looks like world-class
infrastructure right it's ridiculous so uh it's it's all within reach and hopefully people got
some cool resources as well thanks thanks boy so we are at django chat.com and we'll see everyone
next time bye-bye bye-bye this episode is sponsored by mail trap an email delivery platform that
developers love. Try for free at mailtrap.io.