Transcript: Django vs Flask - Michael Herman

In this episode, we're joined by Michael Herman, the founder of Testdriven.io.

I'm Will Vincent, joined as always by Carlton Gibson.

Hi, Carlton.

Hello.

Hi, Michael.

Thanks for having me.

So the title of this episode is going to be From Flask to Django because you've spent a number of years writing production level Flask code and recently switched over to Django.

So we're going to spend a lot of time talking about that switch and differences you've seen between the two frameworks.

But first, can you give a quick background on who you are and how you got into programming?

I'm an engineer/slash educator from Denver, Colorado.

And yes, I am the founder of Test-driven I.O.

which I will talk about in probably just a bit.

So my backgrounds, I started web development in the late 90s competing with a friend in Missouri.

We were building PlayStation websites and eventually moved on to the Lamp Stack and got into hardware during high school.

My undergrad years, I spent at uh University of Missouri, started in computer science, moved to philosophy, ended up getting a business degree.

After undergrad, moved to New York, got a job at Yahoo, ended up getting transferred out to San Francisco, quit Yahoo, got a master's degree, started working at startups, got into a lot of Python, web scraping, testing, and that's what led me to co-found RealPython with another friend of mine.

So RealPython is a uh

Or that's what it was initially when you were involved.

Uh this was back in 2011.

We ended up kickstarting two other courses, a Flask web development course slash web fundamentals course, and then a more advanced uh Django course.

And so, yeah, again, the philosophy behind it was learning by doing, experiential learning.

There was nothing really out there at the time in terms of beginner courses that had that uh experiential uh learning philosophy.

And so um, yeah, we we kickstarted RealPython.

Um, I got into more into tech education that way, taught full stack uh JavaScript for a couple years at a boot camp called Galvanize.

Yep.

Can I build anything other than just a basic crud app?

So went back into the industry, focused mostly on uh Flask development, and at the same time started testdriven.io, which uh focuses on teaching mid to senior level developers how to build, test, and deploy microservices to AWS.

So what were you using back in the 90s?

You said you started off was that Pearl or PHP or what?

Well good.

Yeah, I trumped your PHP.

Yeah, yeah, good.

That's it.

A lot of tables.

Yeah, good.

So without you know going too deep dive into Flask, I mean, for if listeners don't know, it's a micro framework with Python.

Um, what kind of struck you uh and you'd used Django before, but I think it'd been a couple years since you'd used it on a large project, is that right?

So I wanted to build a full stack uh web application with server-side templating.

And so with all the pieces there that you get from Django out of the box, I was able to move a lot faster uh with Django over Flask.

So you've been using presumably uh um some kind of microframework in j the JavaScript world like um express or right, Express?

So yeah, I did uh teach full stack JavaScript.

Uh I worked on the dev team there, and that was um a big Rails app and they had some microservices and Node and Express that I worked on.

Uh but after after leaving Galvanize, uh I worked for a company uh that was all Flask, and so I got back into Flask that way.

So really the decision for test driven kind of came down to Flask uh versus Django for me.

So when you started with Django, do you recall had things changed since you'd last used Django?

Because it'd been a number of years.

Did you have a sense that there'd been a bunch of changes like migrations and stuff, or had you used those before you switched over to JavaScript and Flask?

Yeah, South, um, the old migration system, you know, was now uh into the core in the core, uh, which is nice.

But I was already familiar with how the ORM worked and the data models and whatnot, so there wasn't that too many changes.

And I think that's really nice with Django.

It's it moves a little bit slower than I would say um some other uh Python frameworks, but because of that, it's it's it's it's much more stable.

So you know if you if you know how to build a model and you know how to filter and you know how to do those core operations, that's the same from from you know day one and to now.

And so and there's all this other stuff that you didn't know about, not that you didn't know about, but let's say, oh, there's annotations now, oh there's search now.

And when did that appear?

Uh I mean if I would say you know, regardless, uh in general, regardless of whether someone's wanting to learn Flask or Django, I generally start uh or I recommend that they start with Flask.

Uh it's a Flask is just a great tool for learning web development fundamentals in general, like HTTP, routing, uh, that sort of thing.

And those things are common uh you know across all uh web frameworks.

So I think also um one thing that I often overlook is that the source code for Flask is much smaller.

Uh it's much easier to read.

And so diving into the source code of Flask is is really nice.

You can see what's going on behind uh underneath the hood.

What and what about um for APIs?

Are you using an API with test driven um or or not at this stage?

Flask can do that in a couple lines of code, yet Django Rest framework is uh very elegant.

I just wondered if you had a chance to play around with DRF um recently versus doing it kind of raw with with Flask.

Uh so one of the uh courses on the uh test-driven site is a um a Django course that deals with uh Django channels, um sold asynchronous um fun and um uh Angular on the uh client side.

So uh I did kind of dive deep into DRF to really understand what was going on there, especially with the um authentication.

Yes, yeah.

And how do you find that?

So yeah, Docker, you know, it just simplifies the development process.

Uh yeah, there there's definitely you know hurdles in the uh configuration and 12 factor with settings and whatnot, .m files versus hard coding environment variables in the uh Docker Compose file.

But you know, once you sort of figure out how Docker is working and figure out how that you want to get those environment variables inside of your container, then it just makes the process a whole lot easier.

And are you um what what are you doing for deployment?

You've used a host of services that we've talked about offline before.

So it feels like 2014.

So I am in the process of moving everything over to the um Docker deployments, but right now I'm just um sticking with the uh slug compiler.

Because clearly you know how to set up a server and do it, you know, on a VPS if you wanted to.

What how did you think about that choice of using Heroku or, you know, a platform as a service as opposed to infrastructure as a service, given that you have experience with both?

It gives you logging and monitor monitoring.

Yeah, you can set up uh cron-like jobs, uh, all these different services that I would have to set up and manage and up up uh grade and maintain on my own.

Heroku just all does that all for me.

So I can just pass it uh Django and Green Unicorn or whatever, and then set up all those different extensions and you know I'm good to go.

That the you know it's all very well maintaining, I don't know, your Chrome file, your cron tab file, but like that's not the most fun in the world.

Can you talk about how you sort of prioritize features and you know, again, uh I think this is a trap sometimes when you go from working in a production environment, you it's easy to want to over optimize everything because you're just familiar with it.

But um there still needs to be some sort of process to getting to this optimized point across the board, and maybe we can talk about some of the optimizations on a Django site.

But do you recall kind of what was the kind of the MVP and then how you layered on complexity?

Um, because I think you you initially had what, a Jekyll site before, so you weren't starting from scratch with the test-driven domain, right?

You already you knew you were gonna have traffic.

So yeah, I did start with a uh a Jekyll site, just a you know, static site generator, and was selling uh the course as just a PDF.

And I wanted to move to uh more of like an interactive you know web application where the end users instead of downloading a PDF and then you know never coming back again, always have to come to the website and log in and and and go through the website.

It's it's more interactive for them, and it's also uh you know, customers will continue to come back and I can upsell them and whatnot.

So yeah, I basically wanted to port over the main functionality from uh Jekyll and then from there needed to add on you know authentication author authorization and then some sort of payment provider to um you know handle payments.

I think and Stripe are the two.

Was there another one you considered at all, or was just Stripe the the one you thought you'd go with?

There's a whole there's a number of different startups that have been built um on top of Stripe.

Right.

So I was thinking of you know sort of abstracting that out.

So I did initially start with one of those, Gumroad, and uh that was I was using that when I was just selling the PDF, but now that I'm not selling a digital download anymore, I moved away from uh Gumroad.

And then for um for email, because you must have uh both transaction and potentially marketing emails.

Did you set up a sort of any uh async process for that?

Or maybe you could talk through how you set how you set that up in a Django context.

So yeah, I'm using uh Django RQ, the extension uh for uh Redis Q.

And so to get the uh process out of the uh request uh response handler, just you know, puts the job into the queue and then have a separate process constantly looking at the queue.

Is there a job?

Is there a job?

Carlton, I've dominated the discussion.

Are there any questions that come to mind for you?

I mean, like um so the big beast in the room is celery, right?

Um and uh I'm just wondering w why you chose RQ over celery.

I mean, you know, I've got my own views there, but it'd be interesting to see what you've got to say.

I haven't touched celery in a little while.

Uh I didn't really need all of like the robust like um scheduling and retrying that I know that you get from Celery.

Uh so it was just sent basically sending transactional emails.

I have a couple of other things that I'm using the queue for, but yeah, I mean just from a functionality standpoint, I didn't need everything that Celery gave me.

Yeah, exactly.

Because I think this is a great chance for listeners to go through the list of you know, building out from scratch a Django site, you know, with the knowledge that you have.

Was there, you know, testing?

So how did you find, I guess, doing tests with Django, um, because it had been a little while, and then are you using any additional services as well to test your application?

So testing pyramid, you know, most of your tests are going to be unit tests, and you have some uh functional tests, and then at the very top of the pyramid, you do a bit of end-to-end or browser-based testing.

So a testing tool uh came out a few years ago, years ago called Cyprus, and uh that's for browser-based testing.

So it's really actually good for testing out um you know asynchronous uh type single-page applications.

So I would say Selenium is great for server-side templating type applications and page refreshes.

It doesn't handle all the asynchronous uh type stuff really well.

Cypress handles that really well.

So uh what you get out of the box with Cypress is is uh basically uh in-the-end tests or browser-based tests that are a whole lot less finicky and a lot easier to write.

And so because of that, the testing pyramid is for me, I write mostly browser-based tests and unit tests.

Um I kind of steer clear of like a lot of the functional tests at this point.

So what would you call a functional test in that context then?

Because you know, obviously like running a browser automation, that's obvious.

And then unit tests, you're probably you know, testing individual bits of logic.

Then how would you what would you call a functional test and how would you run that?

Okay.

So related to testing, uh I assume you're using uh some form of continuous integration as well?

I guess.

Um I think GitHub has their own one in beta, maybe GitLab has their own.

What was it about semaphore that um stood out to you?

I don't remember.

Other than you have to pick one and you know.

You're using it for um deployment as well, right?

Not just Can you talk about just the pipeline of incorporating it, I guess, in the project?

So continuous uh integration, continuous um delivery slash deployment.

Uh yeah, I guess in terms of the reason why I picked that is the way that I'm running my Cypress tests in pr in the CI environment.

So I'm not running the Cypress test inside the Docker container.

I'm running the Cypress test outside the Docker container, and they are testing against the Docker container and for or the web application that's running inside the Docker container.

So for whatever reason, I couldn't get that to work exactly right on Travis Circle, um, and I could get it to work on Semaphore.

Uh I have been meaning to maybe switch over to GitLab just because I I like uh having the um the source code and CI C D and whatnot built into GitLab, and GitLab has um faults for secrets as well.

So I like that you know, one-stop, you know, shop type platform.

So eventually I'll probably move over to some or move over to GitLab, but for right now, Semaphore is just working uh for me, and uh so there's not really a reason to switch.

Are you using a a staging server as well?

So you do a local change and then you you know then what happens before it gets to the live site?

So yeah, I do a form of like the uh feature uh branch workflow, um sort of like a Git flow uh type approach.

And so you know I create a new feature branch, uh I make some changes locally, I test it out locally, I you know, do a commit, push it up to a GitHub, that'll uh trigger Semaphore, Semaphore then you know run CI.

Um and if it's on uh for if if uh if it's running against just a feature branch, uh it won't actually do any sort of deployments to uh staging or prod.

But uh as soon as I do a merge request um another

So the reason why I do that is because I just want a place to be able to test it out on uh Heroku and just to make sure that um what's happening on my development environment locally is the same as um or is as close to what it will be when it's in production.

And then uh so then do you manually go and check before um committing uh to production from staging or how do you make that leap?

I mean everything's really covered by Cypress at that point so I don't really need to worry about it too much.

But I'll just do like one last visual check or you know if it's a blog post or something like that I'll have I'll send the link to um the blog post author and you know they can take a look at it before it goes live.

So just little things like that.

I like to be able to view it before it goes live.

And so are you um as you mentioned authors uh so test driven's what started with just you right but now what's what's kind of the master plan right because I I know you have another course um on there and are you are you you're adding blog posts by other contributors as well

Jason Parent he is uh writing another uh course right now um it's basically the the current um Django channels course with uh React instead of Angular um I'm breaking up the uh large monolithic uh course the microservices course my uh myself right now so that's gonna be like four or five different courses hopefully by the end of the summer um onboarding a couple other course producers um right now as well so just looking to scale that out I'm doing some design work right now with

um some other companies and and yeah hopefully the platform will have a couple more courses and have a nice uh uh makeover by the end of the summer sounds

all good

Setting up subscriptions that's uh going to be a lot of development work.

So yeah I'd say if I wanted to go that route it'd probably be you know three to six months worth of development to change that workflow.

This is just for resolving your your your um host name right yes right well okay

Like what's not solved when you're actually at at a production level?

It was uh yeah I don't know if you want to go down this route but it was it was definitely a nightmare.

The platform doesn't really go down at this point so uh which has been nice I think I've gotten like one email in the past um like six months about the site like going down but it was immediately back up so might have been a false negative but or

Heroku add-ons and then you know post Postgres and and Redis.

Does it interest the like the the straight Heroku dashboard will give you a number um of nice key metrics like that.

What sort of stuff are you logging right?

Because this is sort of an extra step on top of your your tests uh or can you talk through how you've how you thought about logging in terms of building out the you know Django side yeah

too much about that and that's you know definitely something there's a backlog for tickets to clean up the logger uh going back to the application performance monitoring I'm using Scout for that um that just hit me so that's another add-on uh that you get from Roku uh and Sentry is used for like the exception monitoring so that's that's anytime an error is thrown and it bubbles up then sentry will catch that and uh send me emails and whatnot.

They come out of the box uh with Heroku and so yeah Heroku just allows me to be able to just focus on my application and not have to worry about infrastructure at all.

I would say Heroku gets expensive quick but I'm a single person development team I'm not gonna hire a DevOps person anytime soon so I'm happy to pay whatever they tell me.

How much do you save?

And you know we talked to Tom Dyson and you know he made the comment that you know these from Wagtail Torchbox um Wagtail fame and he made the point that um you know the if you really do the maths and you include the developer time you can scale these things quite a long way before they they become uneconomical.

You have um for search, um this is relevant because um by the time episode comes out, it'll be announced that I'm giving a talk at uh DjangoCon on search.

Can you talk about how you went through that process of search and um because search is such a deep field and you've got a nice search implementation.

How did you go about that?

So I'm not doing anything um uh fancy in there, and actually I haven't looked at that code in like three or four months.

Oh, you've got to have Elasticsearch or whatever.

You know, you don't have to have Elasticsearch a lot of the time, you know.

You can do it with your database, and for a lot of sites, that's actually good enough.

And so I guess it is doing a very janky flavor of full text search, but I'm not taking any advantage of any of like the full test full text speed uh full text search features that you get from either Django or Postgres.

Got it.

And y and yet from the end user perspective, it's quite effective.

It I don't, you know, no complaints.

I have complaints about it, but I I'm a bit close to it, but uh yeah, I mean it's a just a very, very basic search.

There's not really any ordering to it.

Um so if you know you search for Docker and really it's just going to like sort that based on like the date.

So you know, there might be a blog post with 15 instances of Docker.

There might be a blog post that comes um you know after that that has uh one hit for Docker, and it's going to you know sort that based on the date.

So it's it's not even like concerned about how many different instances of Docker showed up showing up.

So it's a very, very naive uh search.

We're always tempted to um gold plate it.

We're tempted to do the thing which is the absolute um best engineering standards when it's a lot of times it's not needed.

And I do get a lot of emails, uh people asking, not directly about search, but indirectly, in the sense of like, do you talk about this in the course or do you talk about you know that?

And or where do you mention blah blah blah?

And in my head, I'm like, well, use the search, and then I'm like, Well, the search isn't all that great, so maybe I shouldn't tell them that.

Yeah.

That is nice, I guess, versus a well, I guess you can search a PDF as well.

I was gonna say you can point them to search and they can you know if they have the course, they can search through and and find it.

Though I guess if they don't if they haven't purchased the course, they only get what the first the first couple parts or chapters, they don't get the full thing.

So that would probably be why they would ask you that question.

So uh something else you're involved in, which we haven't mentioned yet, is your um conferences.

I mean, you speak a lot, but then also um Pi Colorado is coming up this fall, which you're involved with.

Uh, do you want to speak by that at all?

Uh the norm the main conference is gonna be the 7th and 8th, that's uh Saturday and Sunday, and then there's uh a workshop day uh that's similar to the PyCon workshops that's going to be on Friday the 6th.

So I'm the program chair, so I am organizing all the talks, all the speakers, and all of the workshops.

Well that's quite a bit of work.

And do you know uh are you gonna be able to attend DjangoCon, which I think is later later that month?

I don't know.

Yeah.

I'm probably not gonna make a decision on that for a while.

Maybe just to circle back to what we started with, um what are you've worked with so many different frameworks.

You've mentioned Flask, Rails, Express.

How do you think of you know which one to choose?

Like what are some cases where you know Flask is very well suited to something versus Rails or maybe Express?

Are there some examples that come to mind where you would pick a specific framework for a specific task?

Or would you start with a language?

But I'm not not sure how to answer that question.

But uh yeah, I mean some of the some of the use cases that I look for um in terms of Flask versus Django, I look at project size, I look at the the type of application, um, I also look at the uh the size of the team.

Um I'd say the application type is is is is a big one.

So you know, a website versus a web app uh versus a web service.

And I know not not everything is is uh cleanly bucketed you know into those three different things, but let's say a website is just static generated.

A web app is a full stack uh web application, it could be you know single page application, it could be uh something of server-side templating, and a web service is just could be just just a basic RESTful API.

So I like to think think in terms of those buckets.

And so if it's a website um that's just static generated, probably not gonna use Django for that.

So if it's a web app, um Django you know is great for that.

I think the Jam stack over the the past couple years, the um JavaScript uh APIs markup, uh sort of the Netlify stack, uh the serverless type type stack, uh is kind of eating away in into uh sort of uh Django's dominance as a you know as a web application framework.

And then uh sort of a web service, uh I like to use either Django or Flask for that.

Why?

Because every time I was like, oh well now I need a web form.

And now I know Netlify does this kind of thing now, but it's like, oh now I need a web form, or now I need authors now, and it was like always the case where I just had saved myself a little bit of time setting up my Django app and writing my first template view by just slapping up our HTML page, but then I had to go and retrofit it all every time, and it was like, ah, do you know what?

I give up on that.

And I found myself more and more just like I I'm not gonna experiment with the static site generated for this project because it's just gonna cost me time in the medium term.

And and and using like you know an alternative microframework, it's like it's always more time for me digging in the docs to to learn the other API than it is to just use Django.

But again, I guess that's familiarity.

Yeah, I mean that's a that's a good point.

I don't I don't think it's ever a good uh it's ever a good time when you're when you're working on a project to experiment with something new.

Just go with what you're familiar with.

It's it's easy, it's a lot easier to estimate how long it's gonna take to complete something.

And you know, maybe dabble here and there in some uh with some toy projects uh outside like a you know a project um that you're doing for for work or uh freelance or whatever.

But that's a good time when to like spin up like, hey, how can I actually rekeate recreate like this full Django web application using Netlify or Flask and serverless and whatnot.

So it's

Um how do you think about microservices?

Uh you know, when do you because I think I know you've worked with those professionally, but in terms of for your own, you know, for test driven, did you think about a microservices approach or where do you um how do you how do you think about microservices?

Just to give you a broad question.

When are they appropriate?

Like what what what are they what problems do they solve and what problems do they cause?

Because you know, they're typically used in larger companies.

Um you don't necessarily usually see a startup go with microservices.

And I don't know if that's you know just because that's you know, jet brain users in general, but I would say that's more of the web development industry and the trend toward the smaller frameworks, the microservices and the serverless uh platforms the past uh four or five years or so.

And you know, I guess going back directly to your question, microservices, um, I mean, they're they can be good.

They're not they're not like gonna fix, you know, like if you have a lot of problems in your company, um, if your development uh uh if your development teams are very siloed, it's not gonna fix that by any means.

Uh you you definitely need to address uh some of those softer skills first.

But microservices can um help.

Um it can help uh scale certain pieces of your application that get a lot of traffic um it can help with you know isolating you know bits and pieces of your application to de decouple it from you know other processes and whatnot so that can run independently it can be scaled independently so I I know microservices is is uh kind of a it's obviously a trend these days and you know there's a lot of you know questions um about it and you know how to go about it and micro micro front end frameworks or whatever

like has been a trend on uh Twitter the past like week and that seems bonkers to me but maybe uh next year this time I'll be talking more about you know micro web micro front end frameworks or you know whatever people are called micro front ends I think is the term um yeah but yeah but the but but the the sort of sorry I was gonna say so like the problems that you have with micro front ends is and the same with microservices um micro back ends or whatever is that the core state so how do you

how are you maintaining state across those applications if you if you need to and you know ideal world you wouldn't have to do that you could be able to scale and deploy all those things and you wouldn't have to worry about shared state but there's going to be a certain amount of shared state and that's um huge problem.

Is that something you want to expand on?

Because I thought Yeah you

wards the DevOps the sysadmin the uh SRE team

Think about what you can absolutely not compromise on and then base your decision on that.

And based on that sometimes you'll choose Django other times you'll choose Flask and that doesn't make one better than the other I mean they're really used to achieve the same things building web applications.

They just have uh very different approaches or the the means are are different.

So I appreciate that you've been you know sort of backed up the choices you've made and building out test driven because ultimately a a choice has to be made um you know many choices even for a basic website.

You gotta choose your web server you've got to choose your hosting you've got to choose like these you've got to choose your language you've got to go

more and more awesome.

So thank you so much for joining us.

If listeners want to reach out to you what's the best way Michael