Transcript: MongoDB - Aaron Bassett

Hi, welcome to another episode of Django Chat, a fortnightly podcast on the Django Web Framework.

I'm Carlton Gibson, joined as ever by Will Vincent.

Hello, Will.

Hi, Carlton.

Hello, Will.

And this week we've got with us Aaron Bassett, developer advocate for Mongo and all-round

good guy, member of DSF Board and all-round good guy.

Hello, Aaron.

Hey, Carlton.

Thank you for joining us.

Thank you for joining us.

Welcome to the show.

Let's kick off.

Let's kick off.

Tell us about yourself and how you got into Django and how we know you and all those things.

so that we've got your backstory.

Oh, gosh, yeah, so as you mentioned,

I'm a developer advocate for MongoDB,

which I know is not probably the first thing

people think of when they think of Django with MongoDB,

but I've been a member of the Django community for,

well, since before Django hit version one, to be honest.

I used to, before I got into developer advocacy,

I was a software engineer,

worked for a couple of different software agencies,

and one of them I was lucky enough

to be able to convert from a homegrown PHP CMS into using Django.

So that's kind of where my love affair started.

And I've been a heavy user of Django ever since.

Talked several times at DjangoCon Europe and DjangoCon US.

And as you mentioned, I've now been a member of the DSF board for almost a year now.

Elections are coming up soon.

I remember the first time I saw you was at DjangoCon Europe in Florence.

So you gave a great talk on the testing pyramid.

I'm glad you said that was a great talk because I was incredibly ill that day.

So I'm glad I didn't come across.

That was actually the conference as well where I decided to become a developer advocate.

I was still working in software engineering at the time for a company based out of Edinburgh.

And I'd come to Florence to give the talk.

had been asked to to help uh mark smith out at the nexmo booth um as they were a bit short-staffed

and then kind of realized like oh so you give talks and go to conferences for a living like

you don't pay to be here you get paid to be here and that's when i decided to make the move myself

that was what prompted it like the lifestyle or the the because it's lots of people in the agency

environment and it's quite high pressure and you've got these constant deadlines and you know

You know, I don't know about the agencies you worked in, but that was the agencies I worked in.

And so, yeah, what was the appeal of being a developer advocate?

To be honest, I'd kind of been doing it for many years before I officially started.

You know, whenever I used to live in Glasgow, many years ago, there wasn't a huge tech community there.

You know, there was a tech community in Edinburgh, and that was really seen as being the heart of kind of software engineering and development in Scotland.

but Glasgow really didn't have that same sense of community.

There was a lot of developers in Glasgow,

most of them working in finance,

but there wasn't really any kind of meetups

and there wasn't any kind of social events

or anything like that.

And a group of us got together and decided to fix that.

We were very community minded,

wanted to have an excuse to go out

and have a couple of beers with similar minded folks.

So we started a meetup called Webcentric.

the founders of that went then went on to find things like tech meetup in

Glasgow and Barcamp and CodeCraft and a bunch of other different ones.

But it was really that trying to look for a sense of community,

which got me into it in the first place.

And although I was still just working as a regular kind of engineer in like an

agency at the time, we started to like give talks at the meetups.

And as anybody who's organized kind of regular meetups with speakers will tell

it can be quite difficult to get speakers lined up every month like yeah yeah so it got to the

situation where i was speaking at a fairly regular cadence at these different meetups

you know anytime we couldn't have didn't have somebody to fill us the slot then it was like

okay well what's the topic of this month's meetup of html5 okay right aaron can talk about that

that's fine let's just put him in there but it wasn't until 2014 i think jango khan europe

the the time in poland when they held it in the big circus tent that i got up and gave like a

lightning talk yeah it was it was a great conference like i'm actually probably my

first ever jango khan and uh also my first time on stage i didn't give a full conference talk but i

i gave a lightning talk which if anybody's listening is interested in getting a conference

speaking that's a great way to like test the water you know not a lot not a lot of preparation

required you know five minutes slot everybody is so supportive you know it's just a nice fun time

you get up there it it's daunting like i was surprised anybody could hear what i was saying

because my legs were shaking that much i was sure that all you could hear was i'm tapping on the

stage like i was thought i was going to drown out everything i said but i came off the stage

afterwards it was like yeah that was fun it was terrifying but but fun i should have taken that

advice my first talk was like a 40 minute one at django con that's not true i did it at django

boston before that so to your point about having getting guests every month i'm not one of the

organizers but i've worked with them and there was a while i was speaking a bunch and even in

boston which has a lot of developers it is it's so much work to be a organizer of these things but

it brings so much to the community because otherwise we're all you know in our offices

on our computers yeah we we ended up switching it for um i was helping organize a pipe in glasgow

for a while and we actually end up switching it so instead of having like trying to do monthly

speakers it was then they would have a cadence of doing okay well one month we'll have uh talks

and the next month we'll do a dojo and then the next month we'll just have a social meeting let's

go to like a local bar and next one talks dojo bars and it's it's a lot easier to find speakers

kind of one every once every quarter essentially rather than than every month yeah and you can ask

them when you're meeting with them socially sort of sort of pull out topics that they may not have

thought of as opposed to you know an email blast to the community saying who wants to yeah pretty

much who wants to speak and it means any you've got a bit more time to organize it as well so if

you've got somebody who you know you maybe not be in the local community but you know um you really

want to hear from then you know it gives you more time to organize and maybe like get their travel

sorted and get them um through to the city and uh things like that as well i had a couple more

questions on um the sort of dev rel life before we moved on one was like three years later how

you're still liking it because you're traveling all the time and all the rest and then the second

part of that is how's 2020 been with covid and all remote and the massive change yeah so travels

obviously it's it's completely gone away and i um i went from i think at the peak i was doing

oh gosh i can't remember the distance was now but it was like equivalent to four and a half times

around the world like per year so i've spent 56 of my working time on the road okay so yeah a lot

of travel that sounds like heaven for some people in hell for a lot of others well to be honest i

i was really enjoying it i know like i've no commitments at home or anything like that so

it was really the time to kind of do that that part of my life and I at that stage I was kind

of well I was on the road so often that it didn't even make sense to have a lease for a mortgage

anywhere you know I was literally away from home more than I would have been there so I ended up

just the time I wasn't traveling for a conference is spending it in like airbnbs or visiting different

cities and stuff and it was great I don't think it's for everybody you know it's it's a very

difficult kind of lifestyle to maintain for a while living out of a 35 liter backpack

but for that time of my life like it you know I wouldn't trade it for anything you know I really

enjoyed it I got to meet some amazing people I got to give a lot of conference talks you know

and I got paid to do it it was great whenever I moved from my previous position now to have

MongoDB the amount of travel was due to be less and now it's none you know it's uh with with

COVID well I've I'm in Miami now and that's that looks like where I'm going to be staying for

for the stable future but um we're finally starting to see a lot of these conferences

start to move online as well so a lot of the conferences I've kind of been I've just come

out of a period of very hecticness because a lot of the conferences I applied for in like

you know december january that um were supposed to happen in you know may kind of time then they

push themselves back to like the last kind of quarter or last two quarters of the year so

suddenly had this glut of conferences come in in september october i think my my peak last week was

five in one week so do you do what i do with the conference talk which is like basically leave it

till the week before when you and start to panic and get it written all in panic mode or do you do

are you more professional about it and able to get them done it depends uh yeah so i i've i have

kind of some talks that i've given a few times before so they're they're easier because they're

um obviously ones already presented and the talks already to get to go normally just needs to be

updated slightly for any changes that have happened to libraries or or in the community

for new talks i tend to start working on them really once they kind of get accepted but more

a very abstract way you know i'll just start mulling it over and kind of thinking about it

but i won't start normally writing it until maybe like two weeks before and then i i have this

horrible habit of like two nights before the night before going i hit all my slides and then like

yeah pulling an all-nighter redoing every single slide um which i'm not able to do as much now

because what i've started doing for the online talks is i actually use a teleprompter so okay

for people who are not aware it's you know like a piece of one-way glass like a mirror fronted which

is uh has my a tablet being reflected on it with the essentially the script of what i want to say

and then minded behind that is a camera so i'm reading off this mirror but looking directly into

the camera which i think makes a a huge difference for for talks because i'm not like looking down at

my laptop or you know it's really difficult to give a talk and look directly into the camera

because then I need to look down at my speaker notes or I need to look at other things whereas

this way I'm looking directly into the camera the whole time but it does mean I then have to have

the entire script written for what I want to say which I've never done before normally I have my

slides and I have a few bullet points my speaker notes and you know beyond that I just kind of wing

it so now what I end up having to do is do that record myself winging it essentially and then go

back and write the script based on what I've said so there's a lot more prep required now for talks

than there was whenever i was just traveling fine i've seen a few um photos on twitter you all seem

to have uh these kind of professional recording studios now with like sounded sounded sides and

green screens and you know all the gear yeah pretty much that's the only the department i'm in

in miami at the moment it's you know very hip and modern and it's you know the unfinished concrete

ceilings and very tall ceilings all the rest which means it echoes which you'll probably hear in my

microphone at the minute I actually have two soundproofing panels either side of me trying

to cut some of it down and I'm using a lapel mic to try and cut some of it down but even with that

it's still very very echoey so I've got some of that stuff that obviously you're trying to work

in your lighting and then you've got so like I use a DSLR for a cam link for the actual recording and

yeah there's a lot of a lot of new equipment you need to think about which we didn't have to before

but in you know I kind of feel bad expensing all of this in the company but they're not paying to

fly me half year end the world at the same time so right it's probably cheaper yeah on the topic

of mongo could you just quickly make the case relational versus non-relational and you know

there are some cases where mongo with django might make sense yeah and then and then also

from there like how how are we going to use it with django because it does you say it does it's

not the first thing that comes to mind but you know regularly there'll be a message to the user

group or the forum or the django developers you know how do i use yeah how do i use django in

Mongo? Honestly, it is something that's come up quite a bit. I've seen emails for it when I've

been on the DSF and things as well. And it's something I'm pushing for internally at Mongo.

Historically, we've probably had more developer evangelists than advocates. The difference I see

is with an advocate, you're as well as being kind of the voice of the community or the voice of the

company in the community, you're also the voice of the community in the company. So I spend time

advocating for, you know, uh, increasing or providing some support to, to add, uh,

support from MongoDB to Django, obviously the difficulty of it being is, um, it's

not something we can just fund, you know, we can't just give a bunch of money to

the DSF, um, to pay for it.

You know, there's, there's restrictions there, um, on, because of DSF's nonprofit

status that they can't essentially do like guided work like that, you know, we

can't pay and then our company can't pay the DSF, um, and for a particular piece

of work you know we could make a donation but you that donation has to be just the dsf as a whole

you can then prescribe what they should use that donation for um and also they're not allowed to

fund software development right so the fellows role it's interesting that we're the fellows

maris and i we're community managers not there to write we're not there to write software yeah

and even without that the the separation between the dsf board and the technical board

um is there for a reason and it means that we can't influence the direction of

of Django's development anyway.

The DSF's role is not there to steer

the technical direction of Django,

it's more there to look after the actual foundation itself

and protect the usage of the trademark

and ensure that the fellows get paid

and that we have funding

and that whenever groups like Django Girls

or et cetera apply for grants

that we can meet those requirements,

or sorry, we can meet those requests.

You know, it's not there to steer

the actual technical direction of Django.

So it's something that like I declared

whenever I was obviously joining DSF

that I did work for MongoDB,

I did work for a software company,

but there's no conflict

because I don't have any say

in the technical direction anyway.

So it's an interesting position to be in because I'm obviously a big fan of Django and it's

a big part of my life and a big part of the community, but in the company I work for,

you can't currently use in Django in any meaningful way.

There's a couple of different packages that have tried to add Django support.

The Django is probably the most popular one, which I think actually unfortunately doesn't

work with Django 3, needs to be updated for that.

I am hopeful with the changes coming down the line with Django async, that it's going

to be easier for me to make the case going forward that we should be applying a lot of

our own kind of time internally to help and support Django, rather than looking at something

that we could fund in the community, looking at something that we should be releasing ourselves

and supporting ourselves.

Like, MongoDB has two different Python drivers

and already a, what do you call it, an ODM.

So instead of an ORM, we have an ODM.

That's like the ORM, but...

Yeah, so an object document mapper

rather than object relations mapper.

So, like, we already have an ODM that's in Python.

We have Modr, which is our async driver for Python,

and we have PyMongo, which is probably the one

most people are um more aware of so we already have quite good python support but the way that

they're looking at the moment it's like well if django wants support for uh mongo then they can

use the existing odm and integrate that and it's or it's not what i'm saying it's like well no we

should be trying to support that ourselves you know that's if we want to if we want to have that

first party support in in django then that's something that we need to step up and provide

we can't just you know throw money at it or or wait for the community to do it i mean from a

user's point of view i guess what you want right is you want your django models to just kind of

i've already got my django models i've already got my django project i want to just be able to save

them to mongodb i don't you know so i i kind of want a back end but then there's going to be

limitations in the the mismatch between the the relational model where you've got annotations and

joins and i don't know what and then the document model yeah it's obviously a different way of

thinking about it as well between kind of the document model and relation model um honestly

Honestly, the big ones people keep putting out as the advantage of document model is

that it's schemaless, so you wouldn't need something like migrations, you would just

make your changes to your models and that would be fine, it would just work.

Honestly, that's an advantage and sometimes a disadvantage, really, for me, where I think

MongoDB shines is in things like its aggregation pipeline.

So the aggregation pipeline allows you to apply basically data transformations in stages.

So you might have like a match stage, which is essentially like a select.

So find me all documents that match this query.

But you might have a project stage, which will allow you then to make modifications

to each document.

So you can go, okay, well, I only want these four fields, or actually I want this field,

but I don't want the value of it, I want the length of it, or I want to sum these two

fields together and add this as an attribute onto my document. And you can do incredible

things with aggregation pipelines. So I'm using it at the minute. I did this talk at

DjangoCon Europe where I spider like 10 million domains and then was trying to, just for the

headers of what they returned, and then was interrogating that dataset to try and find

interesting things that were returned. And being able to use aggregation pipelines, I

could do things like I could... So I had a dictionary of all the headers. I could convert

that dictionary into an array where each item in the array was a dictionary in itself with

a key, with a KV or K being the dictionary key from the headers dictionary and V being

the original value. So I can start querying on that and I could do things like, you know,

work out what the average number of headers were that were being returned per site or

or what the largest header returned was,

or I could run regexes against the keys

or the values to look for.

Okay, so let's pull out what the oldest PHP version

being used is, or let's look for lots of places

had these like powered by or expired by,

so let's try and group those together.

And all of that I can do in a single query

using like an aggregation pipeline.

So is that dynamic?

Is like that's calculated in real time?

because the process you're describing

sounds very much like a MapReduce process,

but you've done that batch offline,

whereas Mongo's doing it on demand, so to speak.

It's doing it on demand.

You can also create views out of it.

So essentially, it looks like a regular collection

that you can inquiry,

but that's coming out of a pipeline.

And again, that's live as well.

So it's really the aggregation pipelines

is the bit where I think MongoDB really adds value.

So just swapping out, like, the Django ORM for a MongoDB ODM,

you're not going to get the major advantage.

You know, just being able to have your data models there

and query them as per normal,

that's not where you're going to see any big difference.

Okay, it's going to probably be a bit easier for you

not to worry about migrations,

and I personally find it easier to think of things as objects.

So I have an object-oriented programming, I have my class,

and I have my different attributes on that class, etc.

And I don't need to worry about transforming that

into a relational structure to put it in my database.

I just send that to my database.

That kind of maps better for me.

Honestly, it took a while for it to click, but then once it did,

I don't need to worry about data structures anymore.

It's just like, it's an object, and it's an object in my database,

and it's an object in my Python code, and it's just an object,

and I can just use it, see it everywhere.

So those are key advantages, but you don't really get the full power until you also have support for the aggregation pipeline.

And how you would get that into Django or what the interface of that would look like, that's the bit where I think needs a lot of careful consideration.

So what web frameworks or what areas currently use MongoDB a lot?

I mean, because naively, I would assume JavaScript.

I worked with Meteor.js, which I think is still around, which we use Mongo.

But what's the lay of the land in terms of who's using MongoDB

in terms of frameworks and languages?

If you had to, you know, draw a big pie.

My sense is it's JavaScript,

but I don't really know if that's an accurate one.

No, that's pretty accurate.

Like the mean stack is still huge, you know, so.

Yeah, mean stack.

And then anything that's like unstructured, huge, you know, I don't know.

I don't know if the NSA is using Mongo,

but they're using non-relational to make the haystack.

Yeah, it kind of makes sense a lot in JavaScript

because they're so used to just passing about these JSON.

And essentially, MongoDB I see isn't JSON, it's BSON.

So it's slightly different, but for all intents and purposes,

we can, yeah, it's like JSON with more types, essentially.

So for JavaScript developers, it kind of makes sense to them.

It's just a JSON everywhere.

They just send a JSON to their database, it sends JSON back,

And they don't need to worry again about like doing any kind of transformations on objects and stuff.

So it is really popular there.

I can't really talk an awful lot about individual clients with like clearing it with the departments first.

But what we're also seeing a lot of use in at the moment is we purchased Realm about a year ago.

So Realm is an on-device database designed for mobile.

It's like SQLite, but not relational.

Yeah, but it's, again, it's an object store,

but it stores live objects.

So you don't really need to worry about

kind of reading or writing from the database.

You just get your object

and then you can use that in your code

and it manages the read and writes for you

in the background.

But it also does is manages sync.

So you can set it up

so it'll automatically sync to our cloud service.

And it'll do things like automatic conflict resolution for you as well.

So if you have multiple devices that are all connected to or all using the same Realm database and one of them goes offline for a while, whenever it comes back online, then it'll automatically sync the data and resolve any conflicts that have happened since it's been offline.

That's pretty nice.

Yeah, it's, it's, there's been a lot of work in getting it kind of integrated with kind of our Atlas, which is our cloud server. And it's a team that I'm on. And so it's a lot of fun to work with, to be honest. We, for Hacktoberfest, which is coming up, we're working with a group called WildAid. And we built an app for them, which essentially WildAid is like wildlife conservation.

And one of the areas that they do is they do a lot of ship inspections.

So they'll board ships to ensure that they're not breaching any fishing licenses or any of that kind of stuff.

They're not in any kind of breaches of wildlife regulations.

So whenever they're on board the ships, they don't really have internet connections, as you can imagine.

They could be out in the middle of the ocean.

So before, what they were doing was they had paper forms that they would fill out whenever they boarded the ship.

then these people forms would come back to the main office normally very wet and smudged everything

else and they would try their best to collate the information into like a database now they have

just a android device that take on the ship they fill in the form they can also pull down any of

the information for the area before they go out so if they are expecting to see particular vessels or

they know vessels in the area they can get the most recent information about it and any any

previous boarding reports and have that on their local device do the boarding take their photographs

make their notes etc and then as soon as they get a connection then that's automatically synced up

with the the main database so that's all open source um we actually built it in conjunction

with them as part of mongodb world which happened a few months ago it's now kind of they're they're

using it like in the wild as it were um there's more development needs done on it obviously

see there's additional features as there always are so that's how we're making it our hacktoberfest

kind of project and looking for contributors there so yeah we'll have a link to that that's

that whole realm of online except occasionally off is so interesting when you when you think

about it like because my my brother-in-law worked at a startup that was doing order delivery for

restaurants where they have a device they literally go in the cooler which is just like

a steel box yeah and they like he had to i think they're using flask for that they had to jump

through all these hoops to deal with yeah just quitting out even if they have wi-fi because

they're going in this huge steel cage for orders um so just one example of many where you know

yeah that would solve a lot of problems to have a it's a pretty common one that that stock

coalition thing because a lot of everybody said it's like if they're going into freezers or they're

going into big metal warehouses is like they're just busy walking into a giant faraday cage

you know yeah yeah exactly exactly i remember one one company i used to work for

um an edtech company we also were doing a uh demos it was a video streaming company you know

so they produced their own videos that would be streamed in classrooms and schools and it was all

django based and uh we had a big booth at a really yeah um still is as well it's it was what's it

it's twig twig so twig world i think is the twigworld.medicare.com's url but they they do

these short instructional videos for schools kind of like three minute long you know videos on

science biology chemistry physics that kind of idea so it's a video streaming and quality of

streaming lack of buffering was a big thing for us to be able to show at this conference but as

anybody's been to any big conference knows you can never rely on conference internet so the way

we got around yeah doing the kind of online offline there was um we literally had a little

server sitting on our booth at the conference that um was just serving up the media files so

anything large you know so like all the video files image files pdfs all that kind of stuff

and then we had a dns local dns that would resolve to that box for any requests

so all of our demo machines they would be running the web application online but all of the large

media which was served behind like a static dot twig world or videos dot twig world whatever it

was would be coming from the local network but also meant as well we were then able to offer

a you know an open access point that other people coming to the booth could join

and then if they wanted to watch it on their phones they got the exact same experience you

know they were still getting these videos essentially like offline um at the conference

Uh, that's awesome.

Yeah.

Anyone who's had to go, I mean, I worked at an ed tech company and went to some

demos and it would have been nice to have had that, that's just such a smart idea.

I guess you do it enough times and you're like, we need a better solution here for.

Yeah.

We did the year before we had essentially rebuilt the entire stack on like local

machines and just brought that all down with us.

So, and this, this was kind of just containerization.

was really only just starting. So with some of the bits that were starting to move across,

other bits weren't. And it was a bit of a nightmare, honestly, to try and get this

deployed onto local machines and ensure that all of the URLs were getting read correctly and stuff.

And yeah, we spent the entire time kind of being a little bit on edge. So next year we were like,

okay, we don't want to bring our entire stack down. We're pretty confident that the conference

internet can handle at least fetching the html from you know the actual application code we just

don't want to stream videos and that was our our solution was like okay well let's take the videos

offline and keep everything else just running on the regular servers and it worked pretty well to

be honest so can i cut back to mongo if say i okay i want to get i'm listening to the episode i think

i want to give mongo a go what's my best way of um hosting it because is it a cloud service like

mongo's that hosted should i spin up my own um because let me let me give you back i i haven't

used mongo for quite a few years but when we did a few years ago no no doubt it's all changed

then we had trouble we it was difficult to maintain and difficult to scale and

it was hard work so what would you advise i don't know a is no doubt changed from then this was six

years ago seven years ago um but what would you say if i'm going to give it a go and i want to

give it a play what should i spin up so the first thing i'd say is don't do app get install

like do not install it via like app get or any of the package managers they are all

so far behind um i think some of them are still running like two two and we're now in four four

you know it's it's ridiculous how far some of them behind in that and that's normally the the

issues that we come across as people are still running really really old versions of mongodb

you know and and yeah mongodb like any technology had some teething problems at the start which

are now all being sorted.

You know, we still get people coming up,

oh, we don't have transactions.

It's like, yeah, we do.

It's fully ACID compliant.

You know, yes, we didn't have transactions

five years ago.

We have them now.

Or that MongoDB is insecure.

It's like, no, MongoDB,

until you set up your, like, access controls,

until you set up your username, et cetera,

your admin user,

you can only access it from local host.

You know, it binds local host

and you cannot access it from anywhere else.

So you need to actually go in and configure it yourself

to make it public.

The default security settings are a lot stricter

than what they used to be.

And these are all things that if you're installing

older versions of MongoDB, of course,

you're still going to get the old issues that came with it.

So my first advice would be don't install it from AppGate.

Go and actually get the release directly from us.

The other thing is watch Mark's talk.

So Mark Smith, Judy2k, most people know him as on Twitter, he's done a talk recently,

I think it was at EuroPython, I'm not 100% sure, on misconceptions about MongoDB.

So he runs through a lot of this kind of stuff about where you should install it from, misconceptions

people can have, and yeah, it gives a good kind of like...

For people even like ourselves, whenever we join MongoDB, we might not be completely up

up to date of it. And you'll have these similar conceptions you have, Carlton, about what

MongoDB is or was, and that clears up a lot of them. So I'd probably watch that first

just to get some information on it.

If I'm setting up for myself now, I just use Atlas. So Atlas is MongoDB's cloud hosted

managed service. It's got a free tier, like a kind of free forever tier. You can deploy

it to AWS as you're somewhere else I think you know so you can try and keep your latency down

by deploying it as close to where the rest of your infrastructure is as possible but it's you know

it's so simple to set up it's like button clicks you go through my number one advice for databases

is get a managed database right yeah the best money you'll ever spend it's like don't don't

spend your time and energy doing that unless that's your profession like you know a hundred

percent you know and so the free tier is 99 of the stuff that i do all uses it the only thing that

i've i've had to um use any of the paid tiers for is that the project somewhere earlier with

django con europe so i was gonna say does 10 million data sets fit into free 10 million

doesn't quite um just sheerly because of the size to be honest so i had to um upgrade the size in

not one i think it's costing like maybe like nine bucks a month or something like that so uh that's

still it's i think what i i'm trying to remember back to the my meteor mongo days i think their

thing was called atlas 2 or something similar because when i when i worked i remember when i

worked to your point about switching to a non-relational database and just different

frameworks when i worked on meteor i'd already done django for a couple years and i did that

for a year and a half and then back to Django. And it's, yeah, hosting databases was just boom,

built in. They had all the auth stuff was built in. It's kind of amazing just seeing a different

spectrum of the customization where you could be. And at the time, like I taught a class at a

college with Meteor and we got basically total beginners to build websites in a month, like

really dynamic websites. So it's really, and I think by looking back, I think the biggest thing

was, it was just one click, like install Meteor and had Mongo in there and everything just kind

of worked. And then it was a hosted solution. And this was, yeah, five years ago. Like I wish when

I think of Django and what trips people up, beginners, you know, databases and deployment

is a big one. There isn't, you know, there's Heroku, there's Python anywhere, there's a bunch

of things. But for a true beginner, it's still quite a bit more challenging than JavaScript in

a Mongo solution. Yeah. No, it's interesting just to play around a different sandbox and see the

choices they make and be like wow that has some advantages i mean maybe different things down the

line but starting out um yeah it'd be amazing if django had a hosted solution that just one

clicked in i mean and you really don't is the thing right because you default to local settings

you gotta update static and there's a whole bunch of things and and that has benefits but it there

is a little cost for for beginners yeah django has got a heck of a lot better though i remember

first starting out on it and you would do

your kind of pip install

and your Django admin start project and then

you would try and run the server

and like none of your

CSS would load you know you go to admin page

and just like just raw HTML you know

you'd have to go and configure first of all it wouldn't

serve any static files so you'd have to have like

cherry pie or something running

to serve the static files for you

and there was no like collect static

at that stage either and yeah

it's come a long way

and become a lot easier so the work that's been done is phenomenal but yeah it's still deploying

django is still uh difficult as well you know it's i said there's no kind of one click um deployment

heroku used to have like their django heroku package but even that's being deprecated now

um yeah i think yeah i mean uh kenneth had set that up and that's deprecated i mean heroku is

still so that's the one i teach in my books just because they have a free tier and um

if i tell you exactly what to do you can do it in five six steps but if you try and navigate

their docs i don't think there's a lot of older things out there and people get tripped up

i mean one thing and there's also environment variables which is another thing which you need

for heroku and most platforms uh most platforms as a service you should use anyways that's another

one that really for beginners though there are a number of packages that make that a little bit

easier and that was a big thing for me with 3.1 updates i included my beginners book is showing

how to use environment variables which i was a little uneasy about but it's it makes settings

files you know if you're gonna do any sort of actual deployment that isn't wildly insecure you

have to use environment variables a hundred percent and it's actually a talk i gave recently

at um python australia was like on how to yeah the secrets one we have a link to that yeah um and

it's yeah it's it's so easy just to hard code a secret like even even experienced developers do

you know you're it's just a proof of concept or it's just a really quick hack and you're not quite

sure that you're going to keep this in here or you know you don't want to go through adding it

to your key management store or whatever else so you just hard code it for a second and you're

definitely going to take it out before you push your code up you know it's and then whatever

happens you forget about it or you you go away to make coffee you come back your muscle memory

kicks in you do a git push and suddenly your aws keys or whatever key is now up on github and

publicly available you know or even yeah i mean i was gonna say github now has they'll notify you

they've like generally they'll they have i think they acquired a company that will sort of tell

you hey there's a secret key hanging out in your um repo and um i remember like early github you

could just search for like secret keys and stuff and now you can't do like a broad search so they

they sort of they help you a little bit but doesn't solve the problem if someone has access

to your github repo you know there's 100 engineers yeah also right for security you don't you don't

want them floating around yeah 100 you know especially even if you then remember and delete

it once still in your git history you know you've got to assume as soon as that's pushed up it's

being compromised like um yeah that was actually that was another thing in my my professionals

book i'd sort of walk through updating the secret key in uh the 3.0 version but then yeah it's still

in the source code so um there's a couple there's a couple ways to do it to generate i mean because

it's any 50 plus character string and you can use python has a secrets module um there's at least

one or two websites which will generate a django secret key um but yeah totally i'm guilty of it

and actually for me like i have repos of my stuff up there teaching where i have the secret key in

and we swap it out and github regularly pings me being like you've got secret keys which is nice

that they do but a little annoying that i guess i get unsubscribed yeah everyone does it to your

point on the talk and i like that that was a great talk too oh it happens to everybody eventually

that's you you also have to kind of then plan for it too you need to know what what is my disaster

recovery for this you know it is going to happen at some stage so you need to know how do i rotate

keys you know how do we revoke keys what's the process for that you know how do we then go about

reissuing them it's it's like any other kind of disaster recovery you need to you need you hope

that doesn't happen but you need to plan for it to happen um no but it will as well because

like you want to i don't think that's an awesome point like because you like even your database

credentials just because you kept them secret doesn't mean you don't want to change them every

so often right because you know just people leave the company or whatever else you know it's yeah

you know you're going to have to return them eventually um it is it is actually something

that's quite nice with some of the realm stuff as well is it also can have like manage your users

for you so you don't really have an admin user as such that can be exposed and so your users

register their own accounts and then well it's one way you can access it you can set it so

register you can be registered on accounts and then you can set like on databases or sorry on

collections or even down to like field level what the access rights are that honor on that are so

can go okay well users can read all data that's in this this collection but they can only write

or edit their own data you know you can set that at a database level so you need to worry about

that that kind of business logic in your code you know so instead of like going okay i'm going to do

a a find and um only give me back the documents that this person that the author id is equal to

their user id you can just go okay give me all documents and i'm logged in as this user and it

will automatically go okay well this user only has access to these documents and just return notes

you know so it takes out that that kind of like business logic even to worry about it it's all

done at the the kind of service level which is really nice um you still don't want to display

an edit ui for you know if i haven't got edit permissions i don't want the edit ui presented

right so you still kind of need to deal with that but yeah well yeah yeah you still deal with that

um kind of side of it as well but it just means that then you don't have any kind of administration

like admin credentials to expose which means you can then move it into the client side as well

don't even need a server to run your queries i was just checking there still is it was a jango

login as which you can third-party package where um but i think it's more for users as opposed to

admins or devs but that's incredibly useful if you're trying to you know debug someone says i'm

getting this issue you can just log into your your site literally as them yeah and see it that's one

if you have if you're if you're a super user you you do like your username as and then like the

your customer's username and then your password and it authenticates you as the customer yeah

i used to use a lot yeah it's it's only got 230 stars i think this is the one but it's a

yeah incredibly useful there's a similar similar package called a django hijack as well which does

a oh that's the other one yeah and they both keep it on does hijack keep like logs as well of like

when people have logged in as other people i believe it yeah i know login as the does or did

because that was a big thing you wanted to make sure that people weren't abusing it like login

in as a potential user.

So it had like an access log of like who it's logged in

as like which user and when they did it.

Yeah, you need that sort of thing.

We'll put links to both in the notes

and we should check that out.

They're both, that functionality is incredibly useful.

Yeah, for debugging like customer issues,

it was indispensable to be honest.

Yeah, I know.

Cause otherwise you're just like, you know,

trying to tell them how to take a screenshot video

and yada, yada, yada, and it's just, they can't do it.

Yeah.

So I wanted to ask you,

as we're coming up on time, you're, I think, so I first saw you and came to know about you from

your pip install talk. And it's like the left pad problem from 2016, which fantastic talk. I wonder

if you could talk about that. And I think you're, you're planning an update to it, right? Cause it

was a couple of years ago. Yeah. Yeah. So I think I did that first time at like PyCon UK. Um, and it

was just after there'd been that issue in with NPM where, um, a maintainer had removed a package

called left pad which in itself was a very small package and just allowed you to pad a string but

was a dependency of so many other packages that it took down hundreds of thousands of ci processes

around the world as they attempted to install this package that no longer existed so the talk really

kind of was looking at you know how you can really secure your installation process and your

your dependencies and things like the obvious stuff of around like pending dependencies but

also how you would run your your own local g shop so your own local package index and

a bunch of other stuff and honestly most of what i've discussed in that talk is still relevant now

you know not an awful lot has changed some of the we've kind of had since then there was a move

towards you know pipenv and different packet like package managers have kind of came and went during

that time but the the core of it you know pinning your packages checking package hashes to ensure

they've not been modified from the the version that you're installing how to run like your own

local repository all of that is still the same now as it was back then but yeah i'd love to expand

upon it. I think there's an awful lot around the Python ecosystem that can be quite difficult for

people coming into it to start off with. For your very early users, they really don't need to know

an awful lot. People, okay, we need to know virtual environments, you need to know about

pyenv, you need to know about environment variables, and you need to know a bunch of other

stuff. And it's like, well, no, really, to start off with, you just need to have a Python executable

and write some Python in a file and call the file, you know. That's your very basic hello world,

let's get people interested, they can immediately see, you know, I put this input in, I get this

output out, you know, it's that instant gratification, get them hooked on programming.

But once you kind of move into the more, you know, kind of intermediate level then where you're like,

okay well I want to keep, I've got several projects I'm working on, I want to keep my

dependencies separate i might have a legacy project that's on a different version of python

than what i'm currently using for my own more recent projects or you know i want to run jupyter

labs but i don't want to have a virtual environment for that it's all things then you're going okay

well now we're going to need like virtual environments that's fine that's python 3

we've got those already but then you're looking at pie and so you've got like multiple python

versions then you probably want to have like a virtual environment wrapper to make it a little

bit easier to work with then you're looking at pipx so you can run like jupyter lab so if i

happen to install it and it's like okay well i've got all this stuff how do we then someone tells

you about poetry and yeah it's like there's a bunch of new stuff in there um i have an

like a python local python development environment i'm pretty happy with it's taken me a couple years

to get there but i've now got like everything set up that i i kind of like and i because i do a lot

to work with obviously different Python versions and different

packages and things.

If needs be, I can spin

up a version of 2.7.

But isn't

that the problem right there? Is that everyone

has come to their own

local thing that works. So when a newbie says

how do I just

duplicate that? A, there's

many different ways to do it. 100%.

And B, you're talking about path

variables and it's non-trivial.

Yeah, so like any of

these things, then I will make sure

my information's out there as well let's give people all the choice um no it's very very much

true like everybody does have their own way of doing it and like mine won't be won't work for

everybody i hope it works for some people um it's honestly pretty simplified i've i've really

i went down the road of like having poetry and having multiple different uh kind of editors and

plugins and everything else and now really I have my virtual environments I have pyenv and I have

pipx and that covers like 99% of my cases you know even the pyenv is probably a little bit of

overkill to be honest I think the only time recently I've had to install an older version

of python was actually to do a node installation where one of the packages was assuming you had

python 2 installed instead of python 3 it was still using the old print statement so the which

was interesting that i needed to have a particular version of python installed in order to do an npm

install because obviously web programming is not confusing enough for beginners well i i was setting

up a windows machine recently and had to install node.js and i went i went to the website and it

went i'll just use the lts and i went okay i'll download the lts and i'm after installing it it's

like you need python 2 i'm like what don't do this to me it really was the easiest way it was like

okay i could i can try and take these dependencies i reckon you know that people were posting like

fixes in that you could go and modify the the module yourself in order to get the work with

three because it really was a very small change you know they were just calling the print statement

in the python 2 way instead of like calling the print function python 3 um but it was like well

i have pyenv here you know i can just install 2.7 you know make that the the python interpreter for

this particular shell session through my installation and then you know not have to

use it again job's done i i think that well there's so there's online hosted python things

you can use like if you're teaching you can use replit you can use glitch um and i believe that

vs code is working towards this if they don't are having this where you know in five years people

look at us and go well you just log in and it's sort of a virtualized it's like docker in the

cloud so you don't even have to deal with this nonsense i think i'm not exactly sure where vs

code is at with this but in terms of beginners like i think they're pretty close in terms of

having it all you know like your own docker spin up so you can just download you know use vs code

and then everything is in the cloud you don't have to worry about versions um that's sort of

like game set match that solves a lot of these problems i mean it doesn't solve the fundamental

ones in python per se but if it's hosted you know sort of like docker for newbies and i think i'm

pretty sure vs code is working on that if they don't already have yeah no it's super like the

github's got this code spaces thing which is on yeah github too i managed to get but that's all

part of the same deal right and well yeah i guess yeah but it's super it's it really is it like it

fires up you know a containerized environment you just open up vs code and you do this remote

open this remote thing and it opens your vs code and it's just working and yeah you know if you've

got decent internet it's you know it's indistinguishable from you know local development

it's probably a way to have it work offline too if it flips out for you from a business perspective

when you're containerizing things there's much more of a cost to hosting than you know javascript

or static files so it's the type of thing that as a startup scales with virtualized things you know

their costs are going to keep going up it's a little bit easier for microsoft to eat the cost

of that long term i think glitch has even explicitly said they're i think it's still

basically all free they're gonna grow it out and then do something with it but but anyways there's

there's some additional costs in the back end but yeah those things come down it's a smart move for

them because if you've got like your containerized development environment you're you've built your

app etc and then you know it's okay well click this button to deploy to azure well no exactly

exactly that exactly that i mean you can see all the pieces lining up well so we're at um 50 odd

minutes is there anything you want to mention that we haven't anything else you want to plug

personally or professionally um all i would say is is obviously the combination of mongodb and

django is is something i'm very passionate about myself i would love to hear other people would

like to see that as well. The more people I have kind of asking for it and the better

business case I can bring to my employers, that's always will be helpful. So if you would

love to use MongoDB with Django or if you're already using MongoDB for something else alongside

Django and you'd like to see an integration between the two or you have a particular use

case that you think would be compelling, then please do reach out. You can ever contact

me via the DSF by email or you can contact me on Twitter. My DMs are open. I would love to hear

what people's kind of, what their wants are, what their use cases are, you know, anything I can help

to build this case to bring MongoDB to Django would be really useful. Well, and that's something I

think we, the DSF board, if when we do the survey, hopefully next year, we could have some sort of

questions around what people want um i guess in a way separate from jenga developers i know there's

that's the main place where things are happening but more of a generic

i don't know area to ask these kind of questions i don't know does that make sense carlton or is

that crossing no it does it does make sense it's the the sort of i whenever anyone mentions new

features i get this massive sort of ice thing down the back of my neck because i'm like

how on earth is this ever going to happen yet yes but you know over time we seem to keep rolling

out new features yeah i wouldn't say it's something that would be actually in django itself

you know i would still see it as something that would be a package that you installed like channels

you know we we have the ability to do that now that the kind of database engines and stuff

are easier to swap out so yeah i'm again it's it's not even something that that a company could

try and do because that then the final decision about what would make it into django is up the

technical board you know so absolutely there needs to be a very compelling reason for them to to want

to add it into the core and i i don't think that's going to be there not for a very long time but um

i would love to see it supported as a package that could be installed alongside django but all all

all features begin as third-party packages you know like yeah what unless there's an absolute

the reason why it can't it must yeah south migrations was a third-party package

you know um i still have my t-shirt with the uh from the django uh version released with when

so i've got uh merged in with all the birds flying away it's a nice shirt i like that shirt

yeah good good good i think for me just on the map the monger point i think if like the querying is

obviously different from the orms querying but if we could somehow take the model classes and then

lead that map to a document model somehow yeah then you you know then you would query the document

model but if you didn't have to duplicate that's the trouble when you switch like you know if you

the same with i know maybe you put your models into elastic search you have to create a elastic

search document which then and it's like it's kind of really similar it's like can i get could

i somehow automate this yeah this from my django model to the mongo document model so that i didn't

have to type that out twice that would be amazing and didn't have to update it each time you know

it's typing it once is fine but oh i've made an edit here now i need to make a match and edit over

there that's a pain yeah i honestly i kind of see it as working very similar to to like the q object

you know so you've got your your orm or in this case it'd be like your um odm and that's going to

be how you interact your data like 99 of the time yeah but then if you want to do something you know

uh more bespoke or more complex then it'll be like an mq object you know which you can then use to

create your aggregation pipeline or you know run any of the very specific kind of mongodb

type queries you know it's um you want it as far as possible to be something that you can just swap

into a project and if you're not doing anything you know weird or wonderful then it just works

as the regular Django RM would. But if you want to use some of the features of MongoDB,

then you're then dropping into a different kind of interface to that. And you're very aware that

this is now MongoDB specific. I will not be able to move this project easily from MongoDB

back to PostgreSQL or to SQLite or to whatever else. For some reason, if you're writing raw SQL

queries and you're maybe using some of the features of postgres that's not supported by

sqlite you know so you know you can't just easily use that in development than postgres in production

but it has to be that for me anyway there has to be that kind of separation you know so the people

are are very aware of when they're doing something that is specific to that particular database yes

yeah yeah absolutely absolutely cool super agreed excited well aaron we'll link to ways to contact

you uh in the notes but you're very accessible on twitter and other platforms thank you for coming

on thank you for having me it's been a lot of fun i've enjoyed it and everyone we are at chat

jango on twitter jango chat.com and we'll see you the next how would you say it carlton the next

fortnight we'll see you in a fortnight yeah in a fort a fortnight yeah okay we'll see you in a

fortnight. Bye-bye. Bye-bye. Bye folks.