Transcript: High Performance Django - Peter Baumgartner

Hi, welcome to another episode of Django Chat, a fortnightly podcast on the Django web framework.

I'm Carlton Gibson, joined as ever by Will Vincent.

Hello, Will.

Hi, Carlton.

Hello, Will.

And this week we've got Pete Bamgarner with us from Lincoln Loop.

Hello, Pete.

Thank you for coming on the show.

Hello.

Yeah, thanks for having me.

Thanks for coming on.

So, Pete, we've got so much to talk about.

We talk about Lincoln Loop.

We talk about things like high-performance Django, deployment, topics like that.

But we always ask, how did you get started and how did you find Django?

What's your backstory?

Yeah, so let's see.

I'll try to keep it short, but graduated college with a computer science degree right at the beginning of the dot-com bust, the original dot-com bust in the 2000s.

Good timing.

I moved to the mountains and was a ski bum for a couple of years.

And then I got into doing some computer stuff like computer repair and like kind of small business networking stuff.

And from that, folks, there was a really there was somebody doing kind of website and emails, email hosting in our town that would go on vacation.

And I think they ran a server out of the closet or something and the server would die and people wouldn't have email for two weeks while he was on vacation.

And so, you know, people start asking us, hey, can you help us out with your email and kind of figured out email.

And then with that came websites and so picked up PHP and WordPress and got into that and kind of painted myself into a corner a couple of times trying to build WordPress plugins for things that should have been much more than that.

and um had tried uh you know picked up ruby on rails and all that and and eventually came to

django and that was like that was running off of the subversion trunk days pre yeah pre 0.96

okay because because like i i've been using django for a long time you know like version

one 1.1 something around that that kind of time and you you've sort of always been on the landscape

you know link and loop was one of the first agencies that i knew about that were blogging

about django and using it and yeah yeah it worked out really well and you know uh i was just blogging

about my experience you know learning the framework and using it and all that and it turned

out uh the things that were stumbling blocks for me were stumbling blocks for lots of other people

So, you know, got some traction on that and did a kind of a lightning talk at, I think it was the first DjangoCon.

It was at Google and their Mountain View campus on some like customizations to the admin I had done, which really were not that like, it was kind of like some, you know, injecting some CSS in and stuff.

And people were like, you know, mind blown.

Like it was not, it was just early days, you know, people hadn't kind of thought of it yet.

So, yeah, just really feel like in the right place at the right time type situation, you know.

Do you recall why Django over, you know, Rails, especially at the time, would have been the obvious other choice?

The documentation was a big deal for me.

And I think I liked Python's, you know, people say Python, there should be, you know, kind of one right way to do it.

And that worked in my head better than where I think Ruby is like, there's a lot of different

ways to do it and, you know, use your creativity and all that.

I think that was harder for me to kind of wrap my head around.

So, yeah, but the documentation definitely was huge as well.

That makes sense.

Well, because it's easy to think about now versus back then, whereas now Python is much

more broadly used than Ruby.

Ruby is pretty much predominantly used with Rails. But back then, it was not so much the case. I mean,

it was probably a number of years later, I think it was 2012, when I was deciding which of the two

to play around with. And Rails had better beginner materials. And there were more, I was in San

Francisco at the time, more startups using it. But I just picked Django for some reason, I think,

because I just knew I had to pick one and go deep on it rather than bouncing around. Whereas now,

i think it's hopefully with my stuff and others there's more content for django beginners but it

definitely felt eight years ago like your average django person was coming in with you know if not

a cs degree knowledge of python whereas if you were you know all the boot camps were rails like

i think all the boot camps now are pure javascript pretty much but they still basically don't do

django at all they if they do have batteries included it's rails yeah yeah and i i think

i kind of part of the reason lincoln loop started was i i think i kind of saw like

there was a lot of rails specific agencies out there at the time and people focusing on it and

i saw you know django seemed like it was a couple years behind rails and figured you know hey maybe

you can get in now and and uh beat the crowd can you talk about um so lincoln loop so building out

that agency and you know the type of client work you've you've done over the years and how that's

Yeah, so it started, you know, like I said, I was doing business with kind of these small business networking and things like that and kind of dabbled in the website stuff.

And then I was working for another company and got more interested in the website stuff and sort of spun out a little business there.

And eventually we just kind of didn't see eye to eye anymore.

they saw it as kind of a value add on for their services. And I was more interested in building

kind of a standalone agency. So we parted ways. Um, but at the time all my clients were local,

I was just freelancing, um, most, you know, a lot of website hosting stuff and supporting,

you know, people setting up Microsoft outlook for their emails and all that. Uh, and got involved

with a couple of real estate companies here and was kind of frustrated with the state of the world

of real estate search so that's that was kind of my first big project i built in django and

built a bunch of tooling around that actually had a little product that i was kind of operating for

a while and and that was where i started blogging about it and and kind of shifted gears of of my

company from being, uh, you know, we build kind of brochure WordPress sites for local businesses

to, you know, we're focused on Django and all that. Um, and honestly, like the phone just started

ringing. It was kind of shocking to me. Um, I remember like one day I got a, you know, I mean,

I don't think that people were calling me directly, but, you know, getting an email, uh, and, you know,

got an email from National Geographic and they're like, you know, hey, we're looking at moving to

Django. And, you know, I'm like working out of a spare bedroom at the time, you know, all by myself.

And yeah, I was like, yeah, you know, and so pretty quickly had more business than I could

handle on my own. And at the time, it was really easy to find Django developers that wanted to

work professionally people were working with Django and there's you know personal projects

and on the side but there wasn't a whole lot of professional opportunity around it so uh I was

able to kind of just go through the open source community and start picking people out like hey

I've seen you you know I've used some of your software do you want to work on Django stuff and

they're like yeah great I'd love to I'm you know doing PHP stuff or working in Plone or whatever

so people kind of jumped at the opportunity but that's such a good story about like and you know

from the sort of business perspective of finding a niche because um you know perhaps a year or two

in front of where i was in terms of timeline but that i had that same phenomenon of living in a

world where it was a wordpress and you know setting up emails and managing domains and

it's kind of all it was really hard to make a living in that in that world because it was all

low low value low margin difficult stuff and then finding the way into for me it was um native app

development but you know as well as Django I you know I did build apps and then use Django for the

back ends but finding into the way into a niche where it's like actually there's loads of air

here there's loads of air around and then there's lots of clients who are desperate it's like all

of a sudden now this is a really sustainable absolutely yeah I mean there have definitely

been you know kind of peaks and valleys and um it was a long time before yeah it's always looking

for work made the leap to paying people full-time it was always contract work and per project and

then i kind of looked around and you know we all realized like i've basically been paying full-time

for three years you know um because there hasn't been a gap uh so yeah you know then making that

transition but that transition that must have come from ongoing business like maintenance of

ongoing projects and things right so you've got regular clients that you know we're going to pay

there wasn't not as much then like we i think we have more of it now uh it was more kind of

back-to-back projects and that was stressful um because you don't know when it's gonna the next

one's gonna come because if it stops should yeah i remember there was an agency in um barcelona that

won all sorts of awards they were doing like facebook's app and they'd won you know they

up on stages and all the big things and you know they were like wow everyone was watching them and

all of a sudden they just disappeared because i don't know the work dried up for three months and

they had 50 developers on that terrifies me that that ended business for you know a few years in

the beginning yeah and i remember you know as we're growing like i had a line of credit um with

the bank that was i can't remember it was attached to our house or a car or something and like

completely draining it every month to cover paying people while i waiting for you know checks to come

in from you know we were profitable but just the kind of cash flow wasn't there yet we didn't have

a uh you know buffer and um yeah that was uh i'm glad we're not there anymore it's stressful

yeah yeah no but i mean interesting to hear you know from a business point of view like your story

and it's you know i think there's three main things to talk about um your high performance

jango book deployments and then app pack and maybe i'll put it to you which how how do you how do you

string those together because we've sort of been circling around it but deployments and maintaining

a site anywhere but especially with Django is non-trivial yeah and you've done a lot of work

on this yeah um so I think one of the one of the challenges of being you know I don't want to say

like a front runner but like earlier in the game some of these stories weren't as well defined as

you know how do you set up a site that's um you know getting millions of page views a day and all

that uh so um we definitely did some learning by trial and error and there was some good

information out there i remember at the time um uh shoot the the guy that does sentry uh

he was working at like chess.com or something or i can't remember where it was he gave a talk uh

about like scaling Django and this was really early on David Kramer. Uh, and, um, uh, wait,

sorry, it's chess.com Django. Uh, no, I don't, I'm, I'm probably butchering this story. I can't

remember what it was, uh, where he was working. This was, yeah. Uh, a long time ago, but yeah,

you know, so there were some people doing it, but it just wasn't kind of common knowledge at the

time. And, uh, I think it was sort of piecing stuff together from what other people have learned

doing rails and and all that so um we did one large site that was like a gaming publication

and it was like a cms and like they had a little social network built in and stuff and

they would do these like drawings like first comment gets you know like a free so such and

such so they would like post something on their site and like immediately like it would just go

bananas and it was not just people reading the site it was like people trying to type in comments

and stuff so uh we learned a lot um through you know kind of trial by fire there and uh i think

the book came about we had a client that we worked with who was you know large uh another large cms

company and um they had a bunch of really smart devs on their staff and you know they came to us

and said hey you know we've got this scaling problem and our sites you know periodically go

down and we're really lost here we're just we're just stuck like we just could use another set of

eyeballs and and they kind of explain their system and how it's working and all that and then

like gosh like this isn't the way you should be doing it you know like and it was like where to

start yeah you kind of get those sometimes where it's like you know we they cobbled this system

all together and it's like the system is not working well can you help us fix this system

when really the solution is like, you should probably burn that thing to the ground and do

it a different way. Um, for sure. Of course you're going to have problems like this with the system.

So I, I, I kind of was doing a write-up for the client on, on sort of, you know,

our recommendations on, on how to proceed. And, um, I think like, and not to like,

I hate it when people are like, Oh, these people are idiots or whatever.

They were really smart people. And I think, you know, they made some early on architecture choices that probably made sense at one point that, you know, they carried on for a long time. So I think, you know, it's easy to come in in hindsight and say like, oh, that's all wrong. But at the time, I think, you know, things made sense.

But as I was kind of doing the write up for them, you know, the words were just like flowing, like it was really easy.

I was like, I should, you know, I could probably write more on this.

And so I was pretty sure I was coming back from a conference or something and sat down in the airplane and wrote like 5000 words on, you know, like how to set up scaling and how this should be done or whatever.

It's like, all right, I could probably do a book out of this.

but about that time you were doing a newsletter as well right you would do there was a link and

loop update or a friday like yeah we've gone through a couple of yeah we had we had like a

podcast and we had a newsletter where we were like collecting kind of like links that i think

jeff triplett's got something going with that now but yeah yeah we will yeah i'm just

however many years behind you

no it's it's pain like it's it's uh it's hard like i i was doing that the the newsletter thing

mostly on my own i just got sick of it um but i think a partner really helps because like

for the podcast for the newsletter even this awesome django repo that i have that has

i don't know 3 000 stars like actually jeff came on to that because i was basically ready to just

be like i can't like um so and i you know having that having a couple weeks where you can just be

like i'm just not especially when you're not really you know you're not really getting paid

for it so it's the easy to drop um yeah yeah that's nice um but i didn't know you had that

but it makes sense because i think there's there isn't a django thing which is why jeff and i

started that and um i mean it i think barely we have ads which covers the cost of the rails app

that we use to host it so it's not a you know it's for the community yeah yeah yeah totally

the django forum is a rails app as well right because that's the whole django ecosystem built

well because we got jeff and i got some emails from people saying hey how come you're using a

rails site for this it's like well if you saw that it was a rail site you would have seen that it's

also a company and you know yeah we could have built it i guess but i don't need i don't need

to build everything myself yeah yeah totally sorry so i i interrupted because i was having

reminiscences about like the the time because i bought i got the the the the high scaling the

high performance django the second it was out i was like and i was like hang on i heard about that

on your email and you would oh yeah yeah totally so you're on that you're on the plane you knocked

out 5 000 words you thought yeah i'm gonna do that as a kickstarter too right i think yep so

so i kind of was like all right i should make sure people are gonna want this before i

you know kill myself writing the rest of it so put it up on kickstarter and the kickstarter did

well and i think the only reason it got finished was because of the kickstarter because uh i i

write like the the first 5 000 words were really easy and you know it was relatively easy some of

the other stuff but there was a lot of stuff that was you know where it's like i'm pretty sure this

is the right way to do it, but now I really need to like, you know, if I'm going to write it in a

book, like I really need to figure it out. And so, you know, going through and testing stuff and

then parts where, you know, I was a little fuzzy and I needed to lean on some of the other people

we work with. Uh, and, um, it, it just got grueling towards the end. And I, I hear a lot of people

that have, you know, published a book say this. So I really think the only reason it got completed

was I had taken people's money and committed to a date that it was going to be done. And so it was

like all right i you know i have to do this um and and this part of the reason i haven't ever

finished a second version i've i've tried to pick it up a couple of times and uh i you know i i get

through some of it and i'm like i think i have ptsd or something from the from the first time i

can't force myself to sit through it um but you know like you were saying you know talking about

deployment i think one of the realizations as we were writing that was um that book is not

terribly django specific uh a lot of the things in it are kind of common deployment scenarios you

know you've got a a caching layer in front of some app servers and a database and you know you

cache some database queries and uh that's the general architecture and i think that's the thing

that when we did this consulting job that it was missing.

It was like they cobbled these other pieces together.

And so I think that's where I started getting more

into being interested in deployment

and kind of configuration and all that.

And these days I don't really do much Django development at all.

I'm kind of more focused on running servers

and uh you know doing deployments and docker and containers and all this fun stuff um which is a

little weird because it's kind of full circle from where i yeah where you started in some ways so

uh yeah that's um so one thing i wanted to mention with the book is uh so we originally

put it out in i can't remember if it's 2015 or 2017 but it's a few years old now uh i think i

still stand by a lot of information in it but there is some stuff in there that's outdated

either you know packages we were using that that aren't um around anymore uh and and like i said

i've been hoping to to do a new revision of it and it just hasn't come to fruition yet so we're

actually working with somebody now to get a free version of it published online

so you'll just be able to read um read it on the website so i'm hoping that'll happen in the next

month or two uh and then i i i have not wanted to sell it anymore because i don't feel great that

it's you know outdated but i also don't want to just throw it in the garbage because i feel like

there's still a lot of valuable information in there so this seemed like a it's run its course

you know uh and uh seemed like a good way to kind of uh partially sunset it i guess

well it's cool i mean go on i just could say i when i read it i think part of the reason why it's

much of it still holds merit is you know you wrote it at a level up from as i recall there wasn't a

ton of like code samples it was really more like as you said this is how you construct it and these

of the tools so even when i so that has longevity as opposed to like like for me like i have to every

nine months update my code snippets um which is like you know uh you know if they didn't sell i

definitely wouldn't do it so um i appreciate when i wrote when i read it that you wrote it at that

level because i think that is especially with deployment it is something where the big topics

are kind of the same and as i recall like i think the two big things you said is basically database

hits hurt you and you know cache all the things and then you broke down the steps and the second

half i think is when you got more into like nginx and and like the server itself um because this is

pre-platforms or i'm pretty sure you didn't mention heroku or any sort of hosted service

in the book no no and yeah we were kind of doing all you know stuff on a linux server like it could

be on aws but it wasn't you know serverless or you know on some other platform um but i you know

i i've always felt like that stuff's great for people that you know need to uh do all sorts of

weird custom stuff um but but heroku and the like are are really the place to be for your

kind of average person to even you know running a business or whatever

most people are not uh google scale companies and don't need to operate as if they were you know

carlton what were you going to say you had a something you're going to add there well i was

just what i was going to say about the um the the book and the the patterns in it there was a

another one though a riley one that you know i have on the shelf somewhere but i can't remember

the name of it right now but it was the same the examples were using mysql and php or something

like that but it was the same patterns you know it's it's that you know the write through cache

that you know the database optimize that scale it out have a read replica if you can you know

these kind of things which are exactly as you said kind of um implementation agnostic it doesn't

matter whether you did it in django or rails or you know elixir or whatever you're going to have

the same problems right right yeah and yeah i totally agree yeah you're basically you're

Your application is this, you know, your application, your database are the slowest parts of whatever you're doing.

And the more you can avoid touching those things, the better off you are.

Yeah. So I want to reference you gave a 2015 talk on Django deployments done right, which I watched before this.

And we'll put a link to that. That covered a lot of weight, you know, best practices.

Because I was thinking about that company because I'm wearing Quizlet shirt.

I worked at this company, Quizlet, that's now grown.

you know so why do companies get themselves in this holes of technical debt you know it's partly

because they didn't know any better but it's also partly because like they've got their own fires

to deal with so it's like even though it takes extra time it's like no one has time to spend a

couple weeks if even if they have the expertise to like build you know burn it down which is sort of

why consultants are great because you know you don't usually just sit there with nothing to do

in a company um unless you're going out of business and then you get all the time in the

the refactor until you lose all your customers right yeah and i think people you know i think

the the talk i i kind of hit on in a little bit but i i think people forget about the like the

human cost of all this yeah if you put you know yeah you're running a million miles an hour and

you don't have time to optimize your deployment process but if your site's going down half the

time you're deploying and your team is like you know totally stressed about it and uh you know

constantly like waiting for the next fire to happen like you're gonna get burnout people are

gonna leave like people can't do that for a sustained amount of time and kind of looking at

how do you give people a calmer safer environment and and there's benefits to both sides on that

Surely if your site goes down, that's an expense on the business side, but it's also taking a toll on your team and I think making that stuff so it's not stressful, it's not like, oh God, my hands are sweating moment when you push that button to deploy is huge.

and i i know i've been there like i've been in those situations where it's like this is terrifying

i i just waiting for something to topple over when i push this button you know it's when you put off

deploying you're like you've got a fix and you're like oh we won't deploy that yet and it's like no

no you can't you can't be in that position you've got it yeah if you've got a fix you've got to be

able to get it out like yeah yeah yeah well it's a bit it's a bit like test coverage too because

you know the business can kind of carry on for a while but it is you know burnout people will quit

like and then yeah totally yeah there's there's like day one there's very little value in brand

test right yeah like you know the the value comes two years down the road when somebody has to

refactor or upgrade and like they can do it with some peace of mind versus uh you know being having

to go through and click everything or whatever uh yeah i think that's a it's like a savings account

you know yeah it pays off later well i think if you're not technical i mean so i like i went to

business school and i talked sometimes so a lot of these people go in and are pms at companies and

so they're not technical and you know in a crunch they basically learn that if you know developers

will say it takes four weeks and they'll crack the whip and say we'll get it done in two weeks

And so developers will ship something that looks like it works in two weeks.

And the takeaway is the PM is like, oh, I just got to crack the whip on these lazy developers.

And developers are like, that was terrible.

There's no test coverage.

I'm only doing that a couple of times and I'm going to leave.

So they both feel like they won, but the company's lost.

So I always try to tell the non-technical people, like, you got to build in overflow for testing, deployments, you know, because I know you can't see it.

And it makes you sound good to your boss, but you're killing your team here.

so if you want them to stay around for more than a year you need to you know do something different

even just like like even like psychologically like with especially with testing or bugs in

particular like having like a bug squash day like at quizlet we would have it once a month we'd have

a day dedicated to bugs so that let the pm offload the um hierarchy to the developers it wasn't just

like i'm saying no all the time i was like okay i'm gonna clear the overhead space and then you

need to prioritize and then you know then you have fun with it instead of just like this endless queue

of you know uh these bugs these tests these deployments so there's also like a psychological

part to it all right and yeah that's that's it's hard to make the business case on that sometimes

um we deal with that with our clients you know they uh they may not see it um immediately but

but you have to explain you know yeah it's it's important that you upgrade this library and you

you know, even we're going to spend 10 hours on it and your site's going to work just like

it did before.

How's that sound to you?

I feel like every week on the podcast, I say the same thing, but it's like taking your

car to the garage to get it serviced, right? You, you change the oil, you change the brakes,

you change the tires, you, you know, you put new wash in the wipers, the car drives the

same, but you wouldn't.

do that right like you know and it's it kind of it's i feel like it's an unfortunate i don't know

if software development i guess maybe it's a it's a like a remnant of of software now being on the

public internet but it's it's unfortunate that you can't build something and just let it sit for

10 years and know that it's going to run 10 years later you know like the bit rod is a really weird

thing that like yeah you build something and then it just it you set it aside and five years later

you know a year later it doesn't work anymore yeah but there was an example this week of um

the python cryptography package they've upgraded and they you know they're bringing in rust to make

it memory safe i mean it's cryptography you want memory safety but you know there's people on i

know ubuntu 18 that doesn't have the right version of pip installed and so you know the install

breaks it's like you know there was a big fallout um yeah and all the all they did was make their

package better right right i feel like it's you know when people rage quit tech they often become

like you know woodworkers or something totally analog where things don't need changing but the

antidote is i try to tell myself well what about like a chef a chef makes something it's literally

gone in 20 minutes you know so you know on the spectrum of things like it does last a while

you know um but yeah you need to have like offline things that you do uh so deployments let's talk

about app pack um because this is something new that you've just i think just publicly announced

right can you say what this is yeah um yeah actually your your segue is kind of good i i'll

i'll jump ahead of myself a little bit but the one of the things so um all our clients are on

aws either heroku or aws for the most part um and one of the really cool things i think about aws is

how stable their apis are um i think you know you could probably write something in boto or boto 3

And assuming you can still get, you know, that version of Python to run, like it'll

probably work in a few years, uh, maybe five years.

Um, and I don't think you get that same kind of guarantee with other stuff.

Uh, you know, if I, we used a salt stack, um, as a configuration management tool in

the past and, you know, upgrades were always kind of scary.

It was like the service area is so big, what's going to break.

it's kind of hard thing to test, you know? So I've kind of over the years transitioned from

writing, you know, building and deploying sites as if they're on a Linux server and that Linux

server could be anywhere to taking more advantage of the, you know, I think the term these days is

like cloud native you know um like aws specific tools or if you're on gcp or azure or whatever

they've got some pretty nice stuff there like we've been using rds amazon's database service

for a long time and that's amazing like i would gladly pay extra money for them for me to never

have to think about a database again like uh yeah i don't have to worry about backups or you know

upgrades or whatever like it's just taken care of and that's that's huge like i'm happy to

run stateless services that you know like it's okay if the app crashes or goes away like we

can get it going again but yeah losing data is a different story um so so yeah when uh

we've kind of been slowly transitioning uh from from this like configuration management on linux

servers type scenario to more kind of cloud native stuff and using a fair amount of Amazon ECS,

which is their container service. It's sort of like Kubernetes. I'm not a huge fan of Kubernetes

myself, just because we don't operate at a scale where I think Kubernetes is really valuable.

you know, even our clients that get, you know, millions of page views a day, they're running

like one or two apps, you know, we've got multiple environments for them, but they're not running

a hundred, a thousand apps or, you know, different projects. Um, so, so for us,

I don't think Kubernetes makes a ton of sense. Uh, and I, I sort of like Docker was a tough one

for me um you know it's it's another kind of layer of abstraction but but there's value there

and being able to bundle your application and getting it out onto servers and amazon ecs kind

of lets you do that same thing as as kubernetes and just say hey i've got this image just run it

i don't care where it runs or how it runs or whatever just you know make sure it has this

much cpu and ram and and run it somewhere um which i i think matches the mindset of kind of how most

developers want to work on projects like i need i need this much ram for this thing and just run it

i don't care where it runs i don't want to have to secure it i don't want to have to deal with

any of that but like amazon's stuff is i think their their backwards compatibility is sometimes

like a blessing and a curse like some of it's just feels really archaic and like the amount

of hoops you have to jump through to kind of glue these services together or even figure out which

services to glue together uh is is non-trivial like you you need to it's it's not a whole nother

thing you need to know you know like before it was like okay i know php and i can figure out like

an apache configuration and i can you know ftp something to a server and run it and then it was

kind of like all right now i need to know linux and you know now i need to know yeah everything

but your app the tools like yeah it's gotten bigger and bigger and like aws was just like

another one you had to add on some extra layers of vpcs and security groups and you know it's like

ah totally so um we we built out a couple of solutions for clients uh on based around amazon

on ECS. And I still felt like, you know, this is just, it's too hard. Like it's too, you know,

I've got my head wrapped around it, but trying to pass this off to somebody else who's focused on

Python or JavaScript development, like good luck, you know, like not that it's like, you know,

you need to be really smart to do it, but it's just like a lot of stuff you need to know. And

it makes sense once you know it but it's it's too much so so anyhow that's where kind of app

pack came from and i think i started it when uh covid hit and i was like kind of stuck in the

house and it's like you know i i think there's i can i can glue some of these things together and

make this work a little easier and always kind of having in mind like heroku does this really well

And I, I would tell anybody, like, if you fit the Heroku model, like use Heroku, they're great.

Like that's, that's the general put, you push your code, they figure out like, yeah, it's Python and they put it on a server and you don't have to worry about.

They put it on an Amazon server.

Right.

Exactly.

Um, but for some of our clients, Heroku doesn't work out well.

Like it's, uh, it's either expensive.

um they don't have all the service you know you can get your app running and postgres and redis

and i think that's it um that that heroku itself offers so if you want elastic search

you got to go someplace else and get it and you have to run that traffic over the public internet

and you know you have to trust some third party with your data or you have to go to aws and set

it up yourself um and by the time you're doing that you're more yeah yeah exactly um uh and so

yeah like trying to figure out if i could sort of glue these things at amazon together which are

lower level um and make something that operates for the developers more like heroku um and if

you look at the services amazon offers and and you kind of you know amazon.com is built on amazon

right so the amazon uses and amazon watch services internally and you kind of start to see like okay

like if i was going to build like a microservices version of a platform as a service all the

microservices you need are there you know it's got a way to store secrets it's got a database it's got

a way to run containers it's got load balancers um so app pack was just kind of a way to

glue those things together and sort of spackle over all the extra knowledge you need you know

90 of the time everybody's going to set it up the same way um if you want to like follow best

practice or whatever uh so yeah um that's what i've been working on i started i guess last may

and so our site runs on it but that's not really that exciting it's pretty

trivial yeah but at least your dog fooding that's the important thing right yeah um and we have a

couple of clients uh using it and we're rolling it out to a larger client um this month and uh

kind of just trying to work out any kinks or you know figure out what what doesn't click for people

and we've been rolling it out to kind of some individuals as well so so the the way it works

is you bring your own amazon account but we kind of give you all these tools that set it up in the

right way for you and your data your app all run in your account and you can have your controls on

that um and we kind of just run uh so we built a bunch of cloud formation templates um which is

like terraform it lets you set up a bunch of resources there uh we have a cli and a web

dashboard um and uh an auth service that uh let's see all service and then uh we have something that

plugs into Amazon EventBridge, which lets us get notifications on what's happening in your

environment and then kind of figure out what needs to be done and send the right AWS API calls to

keep things moving along. So an example would be like, you know, you set up a project,

you push your code, a build triggers when the build finishes, then that sends a signal that

says, hey, this build was successful and our code can say, okay, now we need to figure out if there's

a release task like Django migrations that need to run. And if you define that, then we kick off

a task in ECS that runs your migrations. And then if those run successfully, then we kick off a

deployment that updates your site. And like that kind of stuff that seems like it should be really

easy and trivial is like actually kind of challenging in in aws and in kind of eventually

consistent container land and all that so um yeah we kind of take that off of people's hands

so it's it sounds like this isn't necessarily django specific though obviously you've optimized

it for django do you have is it currently or do you have plans for it to be you know rails or

whatever language framework it is so uh it's really cool there's a project out there called

build packs um cloud native build packs and that's i think originally it spun out of heroku

but a bunch of other systems are using it now i think google cloud run uses it

um i can't remember a bunch of other people use it and so um you can use all of heroku's

build system well not all of it but like a lot of it's been open source so um you just define

well it can go in and detect what your app is if it's a simple app so if it goes in and sees a

requirements.txt file it'll say oh this is python if it sees package.json it'll say oh this is node

and it'll build kind of following best practice and in a generally sane way um so so we're using

that so yeah it's it's totally i think there's eight different languages that heroku's build

packs support and there's other um uh other kind of like google has their own build packs

um i think the cloud foundry cloud foundry has some build packs out of their own but

heroku's some of them don't support python heroku's does so that's it's like it's like

a build pack is like an alternative way of building a container right than say a docker yeah so you

don't need to write a docker file but you get a docker yeah you get a compatible image out of the

other end um and i think that's like just another example of like this balloon of knowledge that you

need to have like the fact that people need to understand docker and how to write a docker file

uh to be able to deploy an app is crazy to me so yeah this this kind of manages that for you

without you needing to have Docker on your machine

and no Docker.

You can use it if you want, but you don't have to.

Yeah, that sounds so nice.

I mean, so my third book,

The Professionals One uses Docker

and writing it was a challenge

to make it accurate, but simple.

And I get so many Docker questions, you know,

and it's hard to, because I wanted it to be,

I do think Docker is very common

And also because Mac, Windows, whatever system, it's helpful that for me, I can just write the

same thing, though, of course, there's been some Docker Windows stuff. Anyways.

um i think it's docker max stuff now well no there was something i guess it was earlier last

year where like docker windows broke yeah anyways i was like of all the you know i'm using docker

for it to be the one true thing and you know it breaks for a while um but it is a total i don't

like as a content creator i don't like just being like yeah you need this here's like a couple

paragraphs just run this docker file and don't peek under the hood you know it's like a million

lines ago and it works until it doesn't and when it doesn't i don't know what to tell you

Yeah, exactly. Exactly. And we deal with that. A lot of our client projects, there's, I don't know, maybe 50-50 of folks at Link and Loop that develop locally with Docker. And if you're hopping from project to project, it's a kind of a godsend that you can just jump on another project and do Docker Compose up and things just work.

Or a team setting, for sure.

Yeah. But if you're, yeah, if you're working in the same project day in and day out and, you know, you know how to set up your own machine, it's, it can be a real hassle. Like it gets in the way. It's, it's slow. It's, you know, especially on Mac where you're like under the hood, there's a Linux virtual machine that's running everything. And how are you, you know, the way it shares files across that is kind of weird.

and the networking stuff's kind of weird

and just throwing it at somebody and say like,

oh yeah, just use Docker, you'll be fine.

That's a big leap right there.

That's what I do.

It works out okay, but yeah, I agree.

I mean, it would be nice if as programming develops,

we had more simplicity of tools.

I mean, we do, we have things packaged together

like AppPack, but it also seems like the bigger ecosystem

of what you need to know grows, right?

because people ask me like how do i learn how to program it's like okay well html css databases

sql python javascript you know i was like and that's just to get started and then you can try

my beginner's book um i wish i you know i wish there was a what is that balance right and then

and then docker and all these things come on and it just seems like it's sort of people just learn

it as they need it but they just kind of like band-aids right no one really has the time or

the resources to like fully understand docker fully understand you know whatever the tooling

because there's so much tooling but anyway so that's why you know especially with deployment

something like app pack heroku um button which carlton's working on these tools make all the

sense in the world i mean personally like i have no interest in ever spinning up a linux box like

i i'm i never want to do that ever yeah so i so i the issue with spinning up a linux box is what

do you do when you've spun it up it's like it's got nothing on it so you then you've got to install

everything you don't want to do that yeah and then you're i feel like you end up in the same place

there as like with the testing thing like yeah sure you can spin it up and you can like manually

install your packages like assuming you know how to do all this stuff and like get it running

what happens when you now need to you know you want to get a new version of python and it's not

easily installable on your system and you know how do you move all that stuff how do you manage

upgrading the system how do you make sure like somebody hasn't you know hacked into your system

or what like there's a whole lot of extra stuff beyond just like getting it running um which which

we know as developers you know like once you've gotten it running like that's you know maybe half

the battle i don't know like tests and refactoring it and making it understandable and commenting and

documenting it i think the test for me is that you it's not enough that you can deploy your

application you have to be able to redeploy it on fresh architecture and if you can't redeploy

it on fresh architecture with the same ease or you know it's going to take a bit longer because

you've got to you know spin up the new whatever but that's the test can you can you redeploy it

and if you can then you're in a good place for sure i want to hear about button a little well

it's it's so similar you know it targeting a um a smaller scale thing i'm like i'm looking at you

know helping individual developers who you know they want to get their app online it's more about

matt it so i'm building a native mac app and you know a native app where you can enter your

variables and enter your secrets so they're being coded in the app and then it will launch it and

get it into the right place for you with those environment variables populated so that you don't

You know, that difficulty that people have where they're committing secrets to the repo

when they don't know how to, you know, they don't know quite how to manage that.

And so it's more targeting that layer rather than spinning up a, you know, an ECS cluster,

which I think is a bit too much for the kind of user I've got in mind.

Yeah, absolutely.

But it's the same.

The underlying motivation is the same, you know, exactly the same.

To stick something on AWS, that's the right place to put it for me.

But you don't want to have to deal with all of that nonsense just to configure basically a correct AWS environment.

And the thing you said that's 100% correct is that 90% of people are going to do it exactly the same way every single time.

And it's that 90%.

Yeah, fine.

And for me, with Button, I'm, you know, out straight up saying, look, if you're already into containers and, you know, Kubernetes or, you know, you know, all this stuff, then Button's not for you.

It's Buttons for, you know, I've written my first Django app.

How on earth do I get it online?

Yeah, I think that's the test.

If you can, for whoever, you know, solves this problem to, you know, take the polls app and, you know, put that up the easiest, you know, because that's for someone new, that's what they want.

um but you know the docs isn't going to do that because django itself doesn't do that and um

you know in some ways it's like the early framework scrum right where django is one of many

uh python-based frameworks so hopefully there'll be multiple ones that sit between

learning aws and um heroku this certainly seems like there's plenty of room yeah i think so i

i mean i i think there really is like there's the you can't possibly serve you know i could

you know if i if i worked on button for the next decade i could cover like half of the the the

territory with features and it would be bloated and unusable it would be exactly the thing i'm

trying to provide a solution for um right and and so i can i can target one niche and p can target

another one and then and amazon's up there going look we're basically talking to infrastructure

engineers and if you're not one of them we don't want to talk to you and so well most people who

want to deploy an app they're not infrastructure engineers yeah yeah i i did have like the thought

of like yeah amazon could come by and like eat our lunch and you know they don't care it's not

worth it yeah but i i think you know i've seen this stuff like they've got light sale which is

supposed to be like an easier way to do some of this stuff and there's a project out there called

co-pilot which is supposed to make ecs a little easier to work with and like i still think it's

just like it's too far off from from where people want to be with that i posted an example on

twitter the other day so i was um you know part of what we do is like set up a load balancer for

people and add a certificate so it's got you know tls and all that and i was updating that so you

get uh logs from your load balancer because that's something people want you know and it was uh

when i finished building it all out it was 10 000 characters of json for cloud formation to

you know you have to set up it gets logged to an s3 bucket you have to set up your s3 bucket

you have to set all these like arcane iam policies to allow the load balancer to talk to the s3

bucket if you want to be able to like tear it down quickly then you need to write custom code

to empty all the files out of your s3 bucket before you delete it you need to write like

special stuff so by default it'll just keep filling up your s3 bucket like s3 is cheap but

you don't want to keep logs for you know five years so you need like a custom rule to delete

old logs and like there's just it's so much there it's mind-boggling to what you something you would

expect like this seems like it should be like a line or two you know like yeah i want logging you

know like that should be easy but uh it's it's not i just don't see how um like you describe in

creating that thing it's like yeah i have this thing now and um where i'm i was working on button

from um django con 2019 and i was doing really well until covid arrived and then it just ground

to a halt and at the end of the year i was looking at my diary and i was saying do you know what i

actually didn't make any progress from the whole time covid hit till the end of the year and i said

talking to my wife i said well you know i've got to find some time so i've got this button saturday

saturday morning where i can sit and work on it and my button saturday is just like creating these

you know these rules and it's putting them all together so other people don't have to

smiling at that smiling yeah yeah but amazon amazon have no interest in making that easy for

people well i yeah i think they're they're you know they're in the selling picks and shovels

business right like you can put it together however you want you've got tons of flexibility

like there's certainly value in that but uh it it gears it to a very specific user you know yeah

yeah exactly i think and especially to have a framework specific um solution so django you know

uh even heroku could be a lot better with how it describes django like its docs are pretty

out of date i know the the sort of django overlord they've had has changed a couple times over the

years so you know even heroku doesn't have a simple story for django um so yeah so just as a

you know just like stripe started off as payments for developers and now it's for everyone you know

it seems like that's a nice logical progression for a hosting platform too like if you can nail

the django piece because it is i mean heroku python anywhere i guess there's replet you know

there's just such a chasm to spending the time that either two of you are doing which most people

don't want don't know how to do or don't want to do so yeah yeah i mean we're not touching it right

now but yeah like do you use g unicorn or you whiskey or asgi or whiskey or uvicorn or you know

how do you store files like oh you need django storages to do that and you know how do i like

use dj database url to like there's all this this kind of knowledge that we have having done this

before that coming in as a new person is even that's like just that part's overwhelming just

getting a uh i i gave a django con talk on this that was like prepping your project for production

like it's uh it's a big deal it's a big difference between getting something running on on local host

versus uh getting it running elsewhere just even without all the how do you how do you do deployment

stuff just getting your your code ready yeah but to cut back to your where you started like

okay so you're going to deploy your app and then you think oh i need i need email i need to send

you know password right so oh god i've got to set up my domain for email i've got to verify the

domain and do those funny dk whatever the job of thing records and yeah how many people are doing

it with probably you know their gmail password for their personal account you know as their email

credentials you know google even stopped you doing that now it's like yeah they stopped i've got to

do our yeah yeah they used to be able like four or five years ago you could do that well then even

if you say i'm going to build like a really really basic django site and show you how to do it well

I need to show you email. And you know, SendGrid last year changed something major. So, you know,

so it's like, when I first built the first editions of my books, it's like on version five

for them, I had, you know, all these other services I would use. And now I'm just like,

really trying to strip everything away because they change so much and you know, beyond the

control. And yet, you know, as a jobbing developer, that's what you have to do is keep up to date with

these things so anything that can be hosted and out of sight out of mind it's like take the money

i don't want to deal with it for sure um so can people people there's a wait list right when

so people should sign is that the action item for people listening go to yeah pack.io and great

so um i i probably will start emailing uh some folks on that in the next week or two uh i'm

trying to keep it really small uh just to kind of get feedback and and polish the experience

i'm taking the uh i can't is it reid hoffman or you know if you're not embarrassed of your

first version you waited too long to launch yeah so it works as you know there's rough edges but

i'm guessing a lot of people probably wouldn't you know i know because i can peek behind the

curtains but a lot of people hopefully wouldn't even notice it uh um so yeah just we're kind of

trickling people in right now and um hoping that uh we get enough kind of feedback from that to do

larger public launch super that sounds great what any is there anything we've missed that

we haven't covered that you want to mention to folks uh i don't have much um no i think you hit

it all uh you know like i said um so yeah lincoln loop we're still doing you know django consulting

and and helping folks with either build django sites maintain them uh support them all that

um and and app pack's kind of been my my personal project over the last few months and

yeah look for the high performance django stuff online soon and yeah that's that's what we've

got going on super great well thank you so much for taking the time to join us i know we've long

wanted to have you on so thank you for accepting yeah awesome thanks so much for reaching out i

enjoyed it yeah thanks for coming on so as ever we are chat django on twitter django chat.com and

we'll see everyone in two weeks bye-bye bye-bye join us next time