Transcript: MongoDB - Mark Smith

Hi, welcome to another episode of Django Chats, a podcast on the Django web framework. I'm

Carlton Gibson, joined as ever by Will Vincent.

Hi, Carlton.

Hello. Hello, Will. Sorry, I would say hello, but I thought you were going to jump in without

me there. And then this week we're joined with Mark. Mark Smith, who's otherwise known

as JudyCoot2K for reasons we won't go into. But Mark, thank you for coming on. Lovely

to see you.

Thanks for having me. It's great to be here.

That's good. Right. So who are you? That's what we always ask. And, you know, what's

your stuff? What's your backstory?

Who am I? I'm an imposter. Like, why am I here?

Yeah, yeah.

Well, why do you want to be on your podcast when you work for MongoDB? Let's start with

that.

Well, there we go.

Yeah, yeah, yeah.

I have been a Django developer. So, you know, I spoke at the last DjangoCon Europe. I'm

a co-chair of this year's DjangoCon Europe, which is being hosted in Edinburgh in May,

may 29th to june the 2nd um so that's kind of primarily my link to django um and one of the

things that i'm i'm kind of hosting conversation around is kind of how you should use mongodb with

django and what we can do at mongodb to kind of make django work better with mongodb for the

people that want to do that so let's dive into that because that's quite an interesting question

because like i would imagine so if i'm going to sit down today and i'm not going to ask you i'm

just going to go right django view layer django view layer and then i'm going to go mongodb whatever

interface you give me you know the you know where you do the queries in there but i'm not going to

have much integration with say for instance the rm or you know any of those yeah yeah you're going

to miss a lot i think if you do that i think that's that's the big tension here is that django

has a heavy model, as it were. There's a lot going on in the model there. And that is highly

intertwined with the relational model. There's no real getting away from it. The Django model

is a SQL generator at the end of the day. And MongoDB, we have a SQL interface, but it's really

designed for analytics. It's not really designed for sort of production use in a transactional

application. So they just don't really quite fit. And there have been a few attempts to build

libraries to bridge that gap there's jongo which sort of still exists but i don't think has had

any releases in a while um and there was something before which i forget the name of it was jango

no rel or something like oh really was it called i don't remember that one well maybe i don't

remember it there was one which was built originally around um google app engine and

their data store but i think there were back ends for various of the non-relational i don't know

Yeah, and that doesn't really work because they don't generally, you know, they don't have the same structure or the same query language or anything like that.

So it's a nice idea, but you're going to be building kind of several query generators that don't work.

You know, there's only going to be a subset of features there that actually overlap.

So there's only going to be a certain amount of interface you can really expose.

So let me ask you a naive question, right?

So if I ran into a DjangoCon and my boss tells me we're using MongoDB and Django, I don't know why, I'm just a Django developer.

I Google Django and MongoDB, and I get this Django integration with MongoDB page, which is well laid out.

But there's three options.

It's not clear to me which, like, to connect.

Like, what would you say?

What if I ran to you in the hallway and said, help me, my boss is crazy.

I just need to prototype something to get, you know, him or her off my back.

What do I do?

Well, firstly, it's not crazy.

It's not actually a crazy idea.

Okay, fair enough.

Sorry for the bias there.

Django has so much functionality.

No, no, no.

but it has so much functionality django you definitely want to use django right i'm a big

big fan of django um but the the big model of kind of um building a django app at high level

is to use apps which are sort of big vertical slices including everything from views to sort

of business logic to through to the model layer um so those aren't really going to work with

mongodb very well there are these adapters and i think the page on mongodb.com talks about it

does talk about jongo i think is the second one um it also talks about mongo engine which is what

we call an odm uh which is actually a bit like what the django model layer um is it's it's an

adapter between python classes and and in this case of mongo mongodb database um but it's not

django compatible so you can't just use all the models that you're getting with the various apps

that you're sort of building your your application with um they're not going to store save in

MongoDB but like you can use those apps on top of a relational database like Postgres and you can

then use MongoDB alongside that so you're not it's not going to be tightly integrated with Django but

you get everything that you like with Django like authentication Django rest framework perhaps like

you know you all those components but you also get this database that sits alongside that has

this flexible document model that allows you to embed hierarchical data all in the same place

and then allows you to kind of drill down into that data

to query it and bring it back.

So if I was going to do this,

I would think just like April or I,

I'd go for the Mongo ODM and use that.

That would be my go-to option

because I think surely that's the most supported,

that's the one with all the features, that's the...

Yeah, you know, I wouldn't even use that.

I would go straight for the Python driver.

That is first, in fairness.

That's number one and it says like, you know, standard Python.

Python. Oh, does it? Oh yeah. Yeah. PyMongo. I've got the page up now. So yeah, that's exactly what

I would use. It's going to hand you back. Each document is just going to be a dict containing

Python core types and lists and whatever. So it's just like a, it's relatively low level.

And then I might use something to map those data structures into classes sort of separately to

MongoDB. The problem is, right, if you use an ODM, they don't expose all of the functionality

of MongoDB. We've got this really powerful query language that allows you to do things like

aggregations. It allows you to, the document structure is quite flexible. So you can store

documents that have no relationship to each other all in the same place. So they don't have the same

structure. So trying to map those into classes on the client side, just like that's going to force

you to store the same schema in a collection and it's not going to give you that flexibility of

having um sort of different versions of a schema in a collection or slightly different structure

like if you've got a metadata section for example each of those documents is going to have a

different structure um and uh yeah so i think sort of working starting with an odm sort of puts the

gives you the wrong mindset gives you the idea that every every document in a collection is

going to have the same schema and it gives you the idea that only the the sort of functions that

the odm exposes into the query language are the way those are the only way you can look up documents

um and neither of those things are true so so would i then because cloud atlas is the the hosted

hosting provider that mongo has would i then i guess i would put the mongo piece on there and i

could have my django in postgres on you know my post heroku option if i'm just you know spinning

something up how would i yeah how would i do that um so you're still going to have to host jango

yourself um we don't we we and it's mongodb atlas i think cloud atlas is a book it is i thought i

saw is that by david yeah it's quite a yeah well it not it is it is atlas in the cloud but yes i

misspoke um so yeah that that is our that's how we make money from mongodb right mongodb is open

source you can install it on your own machines if you want to but it's distributed by default you

need three nodes if you want full redundancy it's a pain to host yourself and most people actually

want to build their application so instead they ask us to host mongodb for them we can save some

money by it on you know in development environments and things like that you can you can use what's

a shared tier so that's a bit like a vm right so we we host the three nodes but we're hosting

databases for um uh various different companies on the same set of three notes so it just saves

some costs when you're starting out as you scale up you're going to want a dedicated instance so

you have you understand your performance a bit better and um and things like that so uh yeah it

so it sort of scales with you um so that's where you keep your data um and then yeah i would the

other thing is it's hosting on the cloud provider of your choice so we in atlas you can spin up a

database on um three ec2 instances um on aws or you can host it on google or uh microsoft um

that's a bit like crunchy with oscars to you sorry anyways no no we had craig i'm into it

you don't have craig kirsten's on and yeah you host was that what you're going to say carlton

i'm sorry no i was going to ask a really sort of technical question there if so i've if i'm on

abr i'm saying i've got a virtual private cloud which is firewalled from the um internet can i

deploy inside my vpc or is it kind of no i've i've actually i've forgotten i've got a a brain

freeze um there's a terminology for kind of opening up like to a vpc like sort of connecting

two vpcs together and we allow you can do that but you can't actually host it inside your own

okay no but that's equivalent right because it's firewalled as you can only communicate to

instances from that particular vpc yeah yeah okay absolutely well um atlas reminds me and you might

appreciate this that back i think 2015 16 i spent a good two years on a startup we used meteor which

was a uh framework with mongo db that was going to make all its money uh well it was vc founded

vc funded open source framework so that so they were like we'll just make money on hosting and

then atlas came in among other things and it was kind of like yeah i think it's still around but

But it was this magical experience back then because one line, like I taught a university course using it, one line, and it just installed Mongo and everything locally.

You could deploy it.

It had authentication.

It was like incredibly magical.

I remember getting some shirts from their headquarters in San Francisco.

But they were, you know, yeah, hosting is the way to make money.

I mean, someone should do a Django hosting product, Carlton.

I remember back seven years, I saw Atlas came out and I was just like, okay, they're totally toast now, unfortunately, because it was very cool tech.

Yeah, I mean, that's it.

It's always a shame when something like that sort of goes away.

I mean, I think it's still around, but no one's using it.

But it was the hotness for a while.

Yeah.

There was Mlabs as well that offered MongoDB hosting.

So maybe slightly less innovative, but they did production level sort of cloud hosting of MongoDB and we acquired them, which was incredibly useful to us.

because we were scaling up Atlas really fast at that point.

And so it was really helpful to have some people

with the experience of kind of hosting those clusters

and building those interfaces on top of them.

So while we're on Mongo, I just, like, I saw you give a talk,

or I didn't see you give a talk,

I saw the talk go past on social media,

and I was like, oh, I should watch this.

But it was about, like, everything you know about Mongo

or something is wrong.

Because, you know, I used Mongo a long time ago,

and we hosted it ourselves, and we had nothing but trouble,

and we lost data at times and things like that.

But that was obviously a long time ago.

And then you gave this talk about modern Mongo.

So just in case there's folks like me,

old kudgers like me who are listening to the talk.

I wrote that talk for me.

I wrote that talk because those were the misconceptions

that I had originally before I kind of got up to date

with MongoDB.

So yeah, you mentioned losing data.

That's one of the things that I talk about in that talk.

It made me item two or something.

And it people did lose data, but they lost data because they the default for your you've got your three nodes that you're storing data in and you're actually talking to one of them and then it streams the data to the secondary nodes.

So that's how you keep your redundancy and allow zero time upgrades and things like that.

But the right consistency was one by default.

So you would get a response saying, yeah, we've got your data as soon as that first machine had it.

And it wouldn't have journaled it or anything like that.

So it really only had it in memory.

And then if that node crashed, then your cluster would stay up because you'd switch over to one of the secondaries.

But that data would never have been streamed to any of the secondaries.

So we've recommended for absolutely years that you use the right consistency of majority, which means that however many nodes you've now got over half of them are now up to date with your new data before you get an acknowledgement that that data has been stored.

And that also journals the data as well.

That just like magically switches on the journal flag at the same time.

So, yeah, that's not the case anymore.

So now I think we changed it two years ago.

The default is now majority.

So by default, MongoDB is now slightly slower than it was before because it's now making sure that that data is stored across multiple nodes before it goes.

Yeah, your data is safe now.

So, yeah, you know, I have this slightly trite phrase, which is MongoDB is too easy because people don't read the documentation.

They don't need to.

They just pick it up and they can immediately start storing their data in it.

They don't need to worry about how they model the data for efficiently looking it up again.

And they often they never learn.

And so a lot of my job really is taking these people who go, yeah, I just saw I just get my JSON and I just store it in a database.

And it's great.

And it's like, well, it's great now, but it's not going to be great when you've got lots of data or lots of users because that's going to be really slow.

And that's that's like the, you know, the quick take on non-relational is, you know, it doesn't scale.

And so, but that's coming from, you know, if you're just dumping your JSON to begin

with, how could it scale?

Yeah.

And we see that all the time.

So yeah, it's a, it's a different way of modeling your data and there's a different

set of skills there.

So much as optimizing a relational database is a set of skills that you either learn over

time or you read up on it's yeah, we, we have documentation on the MongoDB website about

various different sort of common problems that you see and, and how you can model your

data to get around those problems.

So that was the ultimate thing.

The end of everything you know about MongoDB is wrong.

The conclusion was MongoDB isn't actually easy to use.

I don't really like to say that it's not easy to use,

but MongoDB, like any database,

especially any big general purpose database,

just has a lot to learn.

And you should go on that journey.

You shouldn't just finish with crud

because then the database isn't really going to serve your needs.

It's no different with Postgres, right?

you get to a certain size above what you what you've got in local development all of a sudden

it matters what indexes you've got and it matters you know that you thought about them and that and

looked at what was slow and optimized and it's going to be the same absolutely it's going to be

the same well i know we just dove into it i'm i'm curious since this first time i've met you like

yeah how did you get into coding like you know could and for listeners you know what's what's

do you have a formal education carlton and i do not you know what you know no no i don't i'm a

university dropout. Um, I was listening to your podcast from a couple of weeks ago with David

Kramer, um, who's a friend of mine. And that was how I found out that he dropped out as well

through the podcast. Um, so yeah, I, uh, my, my dad bought a BBC micro, uh, when I was nine. Um,

so that was my first introduction to programming, um, basically copying programs out of books,

uh, and then getting absolutely infuriated when they didn't work, uh, which was sometimes down

to a typo in the books, but you know, as often as not, like I wasn't a very good typist at nine.

Um, and so that was often my own fault.

Uh, like, no, I want to tell you a story when I was doing such a very, very beginning on, I

don't know, some old Amsha and I was typing it out and then all of a sudden it just sort of stopped.

It wouldn't let me, um, type anymore because I'd reached the end of the character buffer.

Cause instead of opening it, instead of opening a text file and entering a text

file and i was just typing it at the line it was like no you can only enter 256 characters because

that that was like my first sort of i'm a beginner of absolute brick wall barrier which i had to

get some advice about getting over and it was difficult to get advice in those days you know

no internet and i mean i grew up in dubai as well so it was like we didn't even really have very

good bookshops i'd have to wait till we went on holiday to the uk and then go through them go

ran the bookshops in the uk so yeah it was life was harder back then and so we should do a sketch

the yorkshireman sketch or something like that yeah you are um so yeah like fast forward i did

i studied computer science um at high school um and found that very rewarding that for whatever

reason my school really focused on multimedia stuff that was a big buzzword at the time like

pre-internet so it did lots of graphical things and really at one point wanted to be an animator

rather than the programmer but I also like making things interactive which which is where the

programming really came in went to university dropped out of university got a job as a java

programmer it wasn't quite that straightforward but you know that's what happens and spent many

years as a java programmer there was one I was the CTO of a company called E Street in London

at one point um it was very big in in london and also of all the major uk cities um and spent a lot

of time building etl systems because we we gathered data on kind of shops and nightclubs and bars

in london and then we had to get it into the database so i built lots of etl scripts to

translate the data and put it into mysql um initially in pearl um and and actually loved

like it's such an expressive language but it it's always it's got that problem where you come back

to it a day later and you're in a different headspace and so suddenly the code doesn't

make sense anymore because you you model directly what was in your head the previous day in into

characters my favorite description of that was right only programming language yes yes

um and uh yeah so at some point i started looking around for another language because i realized

java was a bad language for etl because it's just there was just too much to the type system got in

the way there was a lot of typing involved it was i needed sort of quick and dirty stuff to

munch data um and so i looked at pearl i looked at python which was like pretty new at the time

this was the late 90s um and as soon as i came across the fact it was indented i went no i'm

going to check out this ruby thing instead right good and it was super early days and all the

documentation for ruby was in um japanese uh except for um ruby in a nutshell from o'reilly

so so i got myself a copy of that and it was just too slow it was before i think they've got a jit

now um but it was it was really slow at that point in time and so i sort of had to revisit python

and get over the whole significant indentation thing and it took two days you know two days

later i'm already kind of um sticking in an if if uh if line if expression if statement isn't

if statement isn't it and then indenting the code that i'd already written underneath it to put it

inside that if block and go oh this is really elegant actually it's like i don't need to go

and put that curly brace at the bottom or whatever it's um so yeah that was there's one of those

points of just sort of instant bias that i had just like i'm not i'm not going with the indented

language um so yeah but after that i was still mainly coding in java in my day job and um that

company um went bankrupt and i moved into another started doing life sciences stuff for a few years

also java based couldn't persuade any company that i worked for to actually invest in python

like switch over i would i would build prototypes in my own time and bring them into work and it's

like, look how much better this is in Django than it is in Spring. And no, yeah, it was too big a

cost, right? Like rewriting things that exist is too big. So I carried on as a Java developer for

many years. And there were no Python jobs, right? They just didn't exist. They certainly weren't

advertised. And then one day I saw a contract for a Python developer being advertised and I'd seen

a few go past and i just thought you know i could do that i could actually switch from being a java

programmer to doing my hobby as a full-time job so i went for the interview um got the job um

and on my first day being shown around the company um i was introduced to somebody in the corridor

oh as this is this is mark our new python expert it was like my first day as a python developer

and the person responded with um oh that's good i'm glad we're investing in the python stuff

because they were doing a lot of matlab at the time uh and they said would you be interested

in training all of our matlab developers in python so like day one of the job i agreed

to to retrain all the software developers in python well you're still on trial you know that

first three months it's you've got to agree to everything no but that isn't that what like you

know being an expert in quotes is is like you're like i don't know anything but you look around

you're like i seem to know more than they do so i guess i'm the expert in the land of the blind

the one-eyed man is king um so yeah i ran a three-day training course um retrained a lot

of their developers they dropped matlab um so it was a big success but i did a lot of learning in

a very short space of time um you know it's a real crucible for uh for sort of learning python

um i set up the edinburgh python user group at the same time um and that's that's been a big

a big thing for me like over time just uh learning new skills like i was very much a heads down

developer um and running that group and occasionally giving a talk or introducing a speaker or

something like that um and just learning people's names like that was a thing that i decided i was

going to do from the from day one was like every every time we get somebody new i'm going to

introduce myself to them and i'm going to learn the name so that when they come back they feel

welcome um and that was just that was a really big thing for me and actually really tough like

learning people's names is really hard um so uh yeah i i then got hired away to skyscanner i built

their first django site which was called a skyscanner is a flight meta search engine you

may have used it it's um it it searches all the different airlines directly you often by scraping

their sites um to get you kind of the cheapest deal um on a flight from one place to another

and they were trying to branch out but they wanted to get into ski holidays so people would put

together their own ski holiday including like flights and accommodation and transport and so

we built this thing called snow scanner in django because they were they they were very much a c

sharp shop but they wanted to they wanted the benefits of writing python stuff because they

thought it was faster um and so we kind of proved that actually uh in a few weeks uh sadly nobody

ever bought any ski holidays so the site isn't there anymore it's uh yeah it was it was a big

failure commercially but technically very successful it's just perfectly built

so what you said that uh you found uh the type system getting in the way with java which leads

me to one of the other things i wanted to talk to you about was you've recently been programming

more in rust and i wanted to just pick your mind on rust i wanted i wanted to get up to the to today

then we can talk about stubs and types yeah i i moved from skyscanner to fan duel um oh yeah we'll

we'll yeah you may have heard of them um i know behind the scenes yeah i know yeah a friend of

one of my daughter's dad is the c was the ceo is the ceo of that or the other one here in boston

and yeah oh if it's boston then it's their competitor draftings okay sorry um but yeah

fanduel is a weird one because it's a it was founded in edinburgh so founded in scotland

but i didn't all of the business is done in the u.s and they have a head office in new york now

um but uh yeah it's actually one of three companies that i've worked for that have been

kind of split between edinburgh and new york for some reason specifically that like those two cities

um but that was really fun i learned a lot about building scalable python software there um and

then one day i this advert came up on a local tech mailing list for a developer advocacy role because

i'm a developer advocate these days rather than a software software developer sort of pure software

developer um and it just listed it was a terrible job advert because it had sort of a bullet point

list of sort of 24 different skills that they wanted you to have um everything from public

speaking to uh writing blog posts documentation writing code in multiple languages ideally and

it just went on and on and on but any normal person would have been completely put off by this

and instead i'm reading it thinking this just describes me it's like they've just listed every

single thing that i enjoy doing um so that was my big i moved to vonage or nexmo they were acquired

by Vonage um at that point and that was my my big switch to developer advocacy and then I got I got

hired away to MongoDB a few years ago just at the start of COVID which is a terrible time to join a

new company as a developer advocate well yeah because the job is flying around the world giving

talks at conferences playing part right that's a common misconception

so it's i i um i sort of somewhat someone recently wrote that uh developer advocacy is a function

not a role so it's the idea being that a single person can't do all of advocacy so it does involve

flying to conferences and speaking um and being part of the developer community but really our

job is to act as developers on the border of in mongodb's case in on the border of the company

and the developers who use our product.

So we act as customers to our engineers internally,

and we act as sort of technical representatives of MongoDB

to people outside the company, so in the community.

And that does take, that is, there's a lot to that.

So some of it is going to conferences and speaking,

just to get to know people, basically.

Speaking at a conference kind of gives you permission

to be part of that community

and then have conversations with people.

I always think those one-on-one conversations

or conversations in small groups are the advocacy bit,

like the actual public speaking from a stage.

I don't consider that really to be advocacy.

It's like the key that sort of unlocks the door as it were.

But yeah, I do, I write documentation.

I write blog posts.

I act as a sort of consultant internally.

I'm acting as a consultant to the team

that write our keynotes at the moment.

So it's a big multi-month project to write our keynote for a big conference that we're hosting in New York later in the year, probably.

And they need to make sure the technical language is correct.

And they also kind of want to know what developers find exciting about MongoDB.

So it's kind of my job to build that part of the story that we're going to tell.

So, again, that's a new skill set for me.

So that's quite exciting.

Yeah, sorry. The original question was, wasn't that difficult doing developer advocacy when I couldn't fly anywhere?

Well, certainly the flying around bit sounds difficult.

So, yeah, that was the big bit that changed. I did a lot more stuff online.

I gave a few talks over Zoom, which, I mean, I got a lot better at, but, and I bought this nice microphone that I'm speaking into now, but it's so thankless.

i i tend to write slightly silly talks like talks that have a bit of a punchline or something like

that and telling a joke over zoom where you can't see a single other face is that it's like laughing

at your own jokes you sort of you pause hoping that other people are sniggering at the joke

you've just told but you have no idea i still have no idea well it's a little bit it's a little bit

like writing like um you gotta you know you gotta put some things in there to keep yourself sane

Yeah, that's true.

I mean, as a side note, one of the things Carlton has introduced me to is the London Review of Books, which I just love, love, love.

Because they just, you know, we're on the, I don't know if we know if we can repeat the line, but we are flying out to DjangoCon US.

And there was just this offhand remark about, you know, who knows what passes for Oscar worthy contender these days.

But if I won't say it, but, you know, it was just this very U.K., very straight laced, disgusting, vulgar thing that, you know, the writer just slipped in for themselves.

And now Carlton and I just trade quotes from the each piece because there's a couple that, you know, the writer just has to put in for themselves.

So, you know, it's valid.

I am another person that Carlton has got into the London Review of Books.

I feel part of an elite club now.

Listeners should definitely subscribe.

yeah it's tough when you don't get a um response for it so uh yeah i mean that was that was the

main difference and i went slightly slightly mad with the lack of social interaction i'm a highly

social person and um just just generally it was a very difficult time um so it's nice to be able

to get out into the community um or in the case of jangakon europe have the community come to me

that's yeah it's a nice plus of hosting it's that's the upside to all of the work that's

involved with uh with chairing a conference but i mean you so go on let's talk about um

jungle new york so it's in it's at the end of may you said yeah 29th of may um running for five we

got three days of the main conference track with the with talks and workshops uh and then two days

of sprints afterwards uh it's hosted in the assembly rooms which is this amazing old um

assembly hall um in the center of the city it's beautiful like these amazing um uh chandeliers and

things like that inside um we've got a great sponsors hall like usually the sponsors get

these dark oppressive spaces with no daylight um and given that that's something i have done

for a living is sort of stand-up booths it's like made sure that we've got a good room with lots of

light so now this will put the sponsors in the dungeon stuff no exactly exactly and uh yeah i've

been one of those dungeon creatures before um and just this amazing hall um i can't you were there

when i announced uh janguk on europe and showed the photo of the big hall um that we just just

want to fill with people basically so that's the cfp is open now i should point that out cfp is

open uh until i think the 26th of february so we've got got basically a month and who who should

submit a proposal for a talk very tempted to just say everybody good it's always important to say

like it doesn't matter what level you're of skill you have with django uh it it we need introductory

level talks for the people who come to the conference who aren't yet django experts because

that you need that whole level set of levels those steps stepping stones to expertise um so that

everybody can feel included in the conference.

So very keen to get all different sets of sort of levels of skill in Django,

but also any kind of talk that would be of interest to a Django community.

So if there's talks about like some sort of technique for managing your time

or getting over stage fright or, you know, whatever it might be,

like we just want to see those kinds of talks.

We're just going to fill it with a whole bunch of different talks

that will be useful to Django developers.

Yeah, I mean, one of the big things I always say about Django Con and the lineup is that it's not just technical talks. Technical talks are a big chunk and a big part of it. But I mean, I remember my first Django in Europe in Florence, and it wasn't the technical talk. I could probably dig out the, oh yeah, that technical talk if I really thought about it. But the ones that really stayed with me were the community talks or the ones on, you know, non-technical topics like, you know, cognitive biases. There was one that was just, you know.

Those are the ones that really change your life, aren't they?

Those are the ones that make you think differently.

And that's really important.

So, yeah, very keen to get those kinds of talks in.

So if anybody who's listening to this has a talk like that or has seen a talk like that and they think it would be a good fit, like get that speak.

to get in touch with us

because we would love to have that talk at DjangoCon.

Well, this will be in the notes,

but I believe in 2020,

you gave a talk on like how to get on the stage,

giving a talk,

which lays out the whole process for people.

Yeah, ironically, that was the first year

it was run remote.

So I gave a talk called

how to get on this stage over Zoom

from my living room.

Yeah, but no, that's a very good resource though.

I think, you know,

good for someone who hasn't done it before,

just laying out the process

and yeah, people should watch that.

We actually have a link to that video from the CFP page, which I feel slightly guilty about because it's like linking to my own talk on the DjangoCon website.

But the previous organisers of DjangoCon put that video in there and I thought, you know what?

Yeah, it's a nice, it's a helpful introduction to kind of how you put a talk together, how you even apply to a conference to give a talk.

I had this amazing experience this year in Porto where I got my badge for the conference,

walked inside and they had some stickers to decorate your badge with.

So I was just kind of looking at these stickers on the table and there was a guy on the other

side of the table who was looking at me a bit awkwardly, sort of smiling at me.

It seemed friendly, but it was a little bit unnerving.

And I was in full on like, I find, like anybody, I find socialising at conferences slightly

awkward but i've been to enough now where i can kind of force myself to overcome it so i stuck

out my hand and said hi i'm mark who are you and and he said oh i actually know who you are um i

the reason i'm here is because i watched your video on how to get on this stage and i wrote my

first talk or i applied to give my first talk here and i'm going to be giving a talk tomorrow and

it's all down to your video thank you very much and it was like oh that's amazing that's that's

why you give those talks because you know in some way they change people's lives he could be a

developer advocate in five years well i think i think people should should take that next step

and reach out to someone if they see a talk i mean the very first interaction carlton i had

he gave a talk on how to grow old gracely's developer and i i believe i just cold emailed

you i i do this all the time like because i've given talks and so i know nobody emails you about

it right but it's it's like it's so nice you know so i'd emailed him and then at django con us my

very first conference he was there and like there was a pre-existing connection you know so like

here's a pro tip if someone gives a talk they're open to you emailing them you know like if they

didn't want to be contacted they wouldn't give a talk yeah that's very true i uh i love it i don't

very often get feedback on the talks but when i do it's just yeah it makes it all worthwhile um

Because it's a time-consuming process, putting together a talk and then practicing it.

Wait, I want to get to Russ, but can I just ask, I was trying to explain this to my wife,

what is the etiquette around? So if you're giving one to two talks a month, is it like one talk per

calendar year? Is it a 12-month rotating cycle? How does that work? Because obviously you can't

give a new talk every time like what's the like yeah that's true well firstly i don't like there's

there's two waves of conferences in the year one's a kind of late spring and the other one's the end

of the summer um and that's when i'll be doing like we have in the past done kind of one to two

talks a month um the rest of the year is actually much quieter fortunately everything kind of dies

down around the wintertime usually. But yeah, I tend to write a new... Before I started professionally

speaking, I used to write one talk per year and give it once, which is an incredible amount of

work to do to then get on a stage once. But I gave the talk because I was part of the Python

community and felt I had something to say. And I kind of follow a similar pattern now. So I tend

to write a new talk when i have something to say currently putting together a talk on exception

handling because i think i have some useful views on that um and but i and when i don't feel i have

a fresh enough talk to give then i just won't be applying to cfps because nobody wants to see me

give the same talk at yet another conference or to come back and give a very similar talk the next

year or something like that so it's it's really fortunately i'm in the position where my employer

doesn't want me to burn my bridges with the community

by giving bad, sort of low quality talks.

And so they allow me to sort of work on these things

at my own pace.

I have colleagues who write new talks all the time

and they're amazing at it,

but they obviously have more inspiration

or maybe more experience than I do at this kind of thing.

So yeah, we're all different,

everybody in developer relations.

I'm very much like it's getting blood from a stone.

It's like squeezing it out.

it's really hard it's like you know i can write one but that's it like you know there's going to

be 12 months before another one might appear like i think i usually write two or three new talks a

year um and then i'll usually give those talks maybe four or five times over the course of about

um two years three years um there's a there's a there's a talk that i wrote a few years ago

called stupid things i've done with python um which is a bunch of hacky tricks with python

um told with a straight face and um i it keeps getting accepted for conferences and i i sort

of submit it as a as a black sheep kind of like i'll submit two serious talks and then the stupid

python talk and they keep accepting the stupid python talk and it's now got to the point where

i can't give that talk anymore because it's just it's you know it's been recorded it's on youtube

um that is one of the talks that people give me feedback on um usually complimentary like it was

funny not useful but it was funny oh no well i like how you said i i think i i saw that i think

one of the more recent ones you've you've given a bunch and you said like this is just the data

model docs page by the way if you actually read it uh yeah yeah that is actually that's part of

that talk um i i love the python data model it's just it was a revelation to me that you could kind

of um lift up the lid on python and start rummaging around inside and just sort of put things back in

a slightly different order um so can i ask because i love it as well and i love um you know just

making an object respond to the dictionary interface so that it can just go in you know

it's a type of your own custom class but so all of a sudden it's it's interchangeable with the

dictionary and then yeah one of the things that sort of come out with the introduction of type

hinting is that so all of a sudden it's it's like you can't pass that object anymore because it got

ruled out as you know unacceptable and yeah what's your thought there because it's that wasn't a

question no emotes with me yeah like yeah what's going on um i yeah i think um when you abuse an

interface like that, putting my serious face on, then you're going to get unexpected behavior as

things kind of get tightened up. You know, if you're using something for something it's not

meant to, then you're not going to be taken into consideration when somebody goes, you know what,

we should make this safer in some way, like by locking down the types that can be passed in or

something like that. So it's really, yeah, it's like, I find it fun when you can stick a sort of

proxy around something so you've got a dictionary but you say wants it to be case insensitive so

you stick a wrapper around a dict and then you just make everything lowercase when it goes in

but you can still make it look like a normal dict which is like really cool actually because

there's lots of languages you can't do that with um but then when you start using i'm trying to

say oh i mean i've used the subtraction operator to make a uh suffix that was my big joke in uh

stupid python tricks was i've got this suffix of ish and you can you can subtract it from true

so you get true hyphen ish but it's actually a minus and so it's using dunder sub behind the

scenes um to return a kind of another object called a truish that's uh that does fuzzy matching

on other types uh and it turns out it's quite fun and in somebody suggested that that could

actually be useful in um testing for example so you can have a sort of a fuzzy number um

yeah so if you've got 2.5 ish that'll match 2.51 and 2.49 or something like that and it's um you

know it's i i'm not convinced it's actually a good idea but it's it's a fun thing it's a fun

thing that you can do it's a weird i i show that as an almost an aside after the main sort of trick

in the talk and at that one there's always somebody in the audience going

quickly writing notes yeah and this is the problem with like teaching these tricks with a straight

face is at some point somebody's going to take you seriously instead of going oh this is terrible

with their head in their hands it's um yeah it's a negative probably well that that reminds me i

heard this this weekend um so this is a joke one of my kids told me like what do you call a fish

with no i's fish so when you're saying that i was like i can't anyway um rust let's have we're

coming up yeah we're 40 minutes in like so the last couple of years you've been programming less

in python and more in rust you've yeah yeah that's so how's that's fair how would it how

no no what's what's your take on rust and why is it so hard i don't know why it's so hard why can't

we have nice things carlton okay i've got a take on why it's so hard this is because it doesn't

pretend to work it doesn't pretend it's working right yeah that's a really good really good

summary um did you will it was so we were part of a like a rust chat group for django developers at

one point weren't we carlton it's like did you bring that up then it sounds familiar we were

reading no i put it on twitter one time i was i was programming um i was writing something in rust

And it was a, was it some front-end-y thing that was going to get wrapped up in WISM and it was going to work in the browser.

And it was like, it took me an age to get it going.

But once I got it going, it literally just worked first time.

And I put something up saying, Rust is harder to, Rust is easier than JavaScript or something about this.

And the thought was, and everyone's like, no, it's not, don't be ridiculous.

But it doesn't pretend to work.

it doesn't say it's working and then all of a sudden it's you know undefined error it's like no

no no no still no still no it's hard to get things to compile but when they do they probably work

it certainly cuts down on the amount of test code you need i think because you can test the

functionality rather than all those edge cases like somebody puts in the wrong type or um yeah

or even passes null in you know those those kinds of problems go away which is really nice um i

guess in the question given this a python podcast is um what do you think the lessons we might draw

from rust into python might be so in some ways we already are learning the same kinds of lessons

very much that kind of not pretending to work thing um so one of the big differences between

python 2 and python 3 was obviously the string handling like how you convert from unicode strings

to byte strings um and you could get something like a a byte string and then encode it which

decoded it then re-encoded it again which was really silly um i the reason i'm mentioning this

is because i was part of the big effort at fanduel to take a large python code base from python 2 to

python 3 um and there was a lot of resistance to originally uh and eventually i took one of the

smaller services we had just to demonstrate like this is how you port something to python 3

and i did i ported it to python 3 and instantly came up with a whole bunch of bugs uh in the code

because suddenly it wasn't pretending to work anymore like a whole bunch of places where it was

um expecting unicode strings and not getting them but just carrying on anyway and there were so

there were that we could have had some fantastic bugs with weird characters and things like that

um so i think that's one thing and i think um type hinting is another one sort of moving in

the same direction adding type hints to python i know it's a controversial topic um i i think

that ability to kind of when you know what a type what type something is to be able to sort of lock

that down or at the very least annotate it and test against it um is a really big plus and you

start to see that yeah sometimes i might want python's full dynamic behavior i might want an

any being passed around but other times i might actually want i might actually benefit from sort

of locking things down and sort of moving a little bit away from duck typing and more towards like

you're meant to pass one of these in and if you don't then it's a bug um so yeah i i was i went

um head over heels from java's static type system into python's dynamic type system absolutely loved

it just a wash with all sorts of anonymous types flying around you know it's like it but i loved

it because it's like i can put whatever i like in this list i don't need to declare what type it is

they don't need to be a subclass of each other or anything like that the thing i've learned from

Rust is that you can have a static type system that's also really powerful. So as well as

locking things down or enabling things sort of broadly, I'm trying to think of a good example.

One of the things I found absolutely astounding with Rust was that there's this sort of global

implementation of, I might get this the wrong way around, I think it's a global implementation of

the from method. And it basically says anything that knows how to convert itself using into

into something else that destination type now gets a from method that will just call that into

on the do you see what i mean it's like you sort of it's really difficult to describe without

something on the screen um but it's like you just get this this blanket implementation

of from on your destination type um that you've only actually implemented into on your uh source

type have i got that the wrong way around i hate describing things like this the point being

that that's done once at the highest level trait and then yeah it's never it doesn't have to be

re-implemented all the way down and certainly in java um and as far as i'm aware c++ you just have

to go into every one of those types and just say and now i'm yeah implement this implement this one

line of code each time or use a macro of some kind or something like that um macros is one area of

Rust that I haven't really got to grips with yet. I really should. The stupid Python tricks was the

way that I learned Python originally, in depth. And I need to get back that mentality with Rust

and start writing stupid Rust tricks. Not necessarily like breaking Rust, but just like,

how would I implement this core type and things like that and start really digging under the hood.

And so I've found it, one of the reasons I found Rust hard to learn, apart from the fact it is

just a tricky language is because it's been a while since I properly learned a new language

that's especially a really different language and I've got that inner perfectionist all the time

going you must understand this completely before you move on and it took me a long time to start

going okay I've got that working now I can move on to the next thing and it kind of you know build a

broad understanding and then start to dive down into the depths I'm just not just not very good

at that anymore well it's tough to remain playful as you learn a lot because you just see where the

bodies are buried and you know you almost want to be like is that you know what's the tipping point

of naive versus well i'm just gonna you know see what works yes yeah very true they feel like a lot

of edge cases in rust as well just occasionally you'll learn something that works in one context

and not in another um i don't know if that's just me um but yeah it's been a journey i've now got to

the point where I feel I reach for Rust now with certain types of problem instead

of Python, and I'm just like, I know I can write this in Rust and it won't take me

too long.

The tooling is really good.

the it's really oh that's another thing python could really learn from rust is distribution i

know packaging is a tricky topic but it's like i also i can compile a rust program for different

platforms and just distribute the binary um and again there's there's been so many

attempts to do that in python um for different platforms or sort of limited sets of platforms

and things um and it's such an important thing for client-side software and i mean at the end

of the day service server software is client-side software to your ops people as well isn't it so

it just just makes getting stuff installed easier somebody told me a while ago that that containers

solved this problem but i i fundamentally disagree with that given given how many command line tools

and things like that there are out there in the world i think in that packaging thing there's also

something about the size of the the thing the binary you create it's not just in disk space so

it's five megabytes or 550k or you know 500 it doesn't matter because disk is kind of cheap but

then when you run it if it's 60 megabytes of ram in memory yeah persistent for doing almost nothing

that's where i think python's got a bit of a problem in that you can't run lots and lots of

little agent servers agent services just sitting there on a you know a vps with half a gigabyte of

ram you've like two of those and you've used a fifth of it it's like no no no no it has to be

it has to be small so i think that's where yeah the the native the truly native you know single

binary thing comes in um anyway that's cool the thing i'm interested in is the pi three the way

people are writing pi zero three the the way people are writing modules in rust and then

wrapping them in a small um foreign function interface layer which then you can then import

it as your python module so i mean that's um from the time i started using python and people would

say well it's too slow and you go well it isn't too slow but if it is you take the you do the

profiling you take the core thing that's taking the time and then you re-implement that in c

and that's always been relatively easy whether you use a wrapper generator like swig which i'm sure

doesn't exist anymore but yeah this was c types and c types and cffi yeah this gave you yeah this

was pre ffi and everything and it used to it used to build the boilerplate for you um so there was

a certain amount of boilerplate with any module and it was just a command line tool that just

basically used to generate everything you needed to fill in your c function that you know to just

and then that would automatically be exposed and the types would be automatically translated

between python types and uh and the types you'd expect in c um it was uh it was good like swig

used to produce these really big modules um i think you know i think it died a death when um

am i thinking of c types wait oh sorry my mind's gone blank c types is core python isn't it um i'm

there's there there is a tool for kind of um annotating python code so that then compiles

it down to a c module directly lxml uses it um and i've forgotten the name of it and i think

i think the most successful sort of c modules that are now exposed to python they generally

use that approach i can't remember what it is scython scython oh yeah yeah yeah that's right

yeah yeah without the p it's like c and python what would that be called

you can see why i was getting confused with c types though it's like wait am i talking about

the wrong thing um anyway cool c types is fun there was a talk at europython a few years ago

where somebody was um writing python code to open up c types to talk to the python dll and then was

changing some of the low level things so two was no longer equal to two or something on those lines

it was absolutely crazy oh yeah i saw the boston python user group i saw someone do exactly that

you know just being like nothing is anything and like it was like it was just sort of like wow

nothing yeah it was playing around with with c and python and it was i can't say i followed all

of it but i saw the result and i was like wow okay so where are we now yeah you can't trust anything

or anybody yeah yeah i think that's that's the ultimate lesson yeah pretends to work um i did

have i know we're coming up on time i think um maybe a actually a talk idea for you which maybe

you've done but i would love to see a talk that is like when you should use mongo with jango where

you take a standard you know not instagram clone but something where someone will just naively

build it and show how hey here's an example of where Mongo really shines and like how you should

use it because I think like for me I don't have the knowledge of when would I reach for Mongo

like you know because it's more of a scaling question but if like you could tee up in people's

minds this is what's going to kill you um that would be interesting and maybe that exists but

I think I actually submitted that talk idea to DjangoCon Europe last year um and they accepted

my other talk we gotta talk to the organizers this year and see uh see see what's going on

i'm in charge of the cfp this year so i could totally abuse that position

well i just i just suggested it so it's a new you know it wasn't your idea

um yeah no thank you it's i i you can tell i think that's a good idea i think we came up

with it separately and it is it's yeah it's a good set of questions i struggle um with writing

mongodb talks because i don't want them to come across as a pitch um i yeah it's it's very it i

love mongodb but um i'm also aware that if i stand up on stage for half an hour and keep saying the

name mongodb again and again then it's going to start to sound a bit like a pitch so i tend to

tend to talk more abstractly about architectures or um building certain types of application and

mongodb just happens to be in the in the background i mean it's really a scaling talk i mean we i feel

Like Carlton and I often try to push guests on, like scaling is interesting, right?

Because at a certain point, you prototype anything, but, you know, will it scale?

And to the extent you can have someone who's been in the trenches, who's just like, I can

tell you right now what's going to go wrong.

I mean, you know, you listen to the David Kramer sentry talk.

That's sort of, they're trying to get the tooling at some point to be like, you know,

we're all doing the same thing over and over and over again, whether you realize it or

not.

And if somebody or some AI can tell you, hey, like there's an N plus one problem, you might

want to know about that or like hey like maybe a non-relational would really help you out here

yeah um and they i mean david's been through the trenches and that one as well it's uh that

they've gone through multiple sort of scaling stages um because at one point they were storing

everything in postgres and then uh they they they hit a point where they had to then re-architect

all the way the way they stored all of their data because it just couldn't scale anymore

um and i think they've done that twice maybe three times i mean probably in an ongoing like

you know uh evolution now for um for keeping keeping track of their scaling problems but

that's a really nice problem to have when it's you know you're having to scale because you've

got loads of paying customers um that's a pretty good situation to be in um but yeah i mean it is

one of them mongodb is web scale right it's uh it's built it's built in i i have a t-shirt somewhere

with that written on the front um the other slogan we haven't had in the show yet is stash the hash

stash the hash

anyway carry on you've got what you've got but it's got sharding built in which is you know and

but there are some limitations with that you need to be careful about how you shard your data like

what you choose is a sharding key and that's true of i think any sharding solution as well so maybe

that something that could be abstracted out into a talk it's just like how do you scale when you

can't store all your data on a single machine or a single single cluster um i'll have to give that

some more thought really appreciate the input it's always nice to have somebody suggest ideas for

talks it's always easier to project on someone else your own problems you know but one of my

most successful talks about python packaging um was literally just me sitting next to somebody

in a sprint somebody much more experienced than i was and he was just tearing his hair out over

trying to package something and he said why won't somebody just tell me how to do this

i thought i could write that talk so i wrote an opinionated guide and uh yeah that's been hugely

successful i got my job because of that my manager loved that talk it solved a particular packaging

problem that he had and so um yeah he was very open to me joining mongodb uh yeah when when

the well there you go so when when do you reach for mongo like that's that's what i want to know

When do I reach for them?

No, no, sorry.

The DjangoCon talk, when to reach, you know.

When to reach for them, okay.

Or whatever.

Carlton, do you have your talk all set for May?

I mean, it's coming up.

I know you're so British.

You pretend that you haven't worked on it at all

and like you've done it six months in advance.

I've got my top secret proposal idea on the brew.

It's simmering away there.

So I've got till the end of February

to get the abstract nice and tight.

I've got the idea whether I get accepted or I don't know whether it's a good idea, even if I do, it's yet to be seen.

Awesome.

I look forward to seeing the proposal.

I look forward to seeing lots of proposals from many people.

It's early days.

We only opened a few days ago.

So it's like there's a few proposals in there, which is nice, but we need more.

I believe I put it in the Django newsletter that came out Friday as we record.

I'll keep putting it in there, but I'm pretty sure I put it in.

I mean, clearly didn't drive that many traffic, but...

We had some, but all CFPs are the same.

I've been through this before.

It's like, yeah, they all come in on the last couple of days.

So I'm not too worried.

I may be too calm.

I'm quite a relaxed person generally.

We'll be fine.

Everything will be fine.

One more just question on talks and talk writing is DjangoCon Europe,

at least I don't, maybe Django News,

they have this policy of asking to see the slides first

for a kind of code of conduct review,

which I'm totally supportive of.

But as a speaker, right, busy working on my talk,

I'm also like, if the slides existed,

I'd already have given the talk.

DjangoCon US definitely didn't have that last time.

We do have that at DjangoCon Europe.

So just from a, you know, experienced hand point of view,

how do you advise coping with that sort of stress of,

ah, I've got to give these talks for these slides,

but the slides are also a work in progress as we speak.

send in whatever you can right it's it's written in i mean i would call it the small print but it's

no smaller than any of the other it's like you know we don't expect them to be ready we just

kind of want to see the outline of the talk um and uh yeah it's uh it's it's a formality more

than anything else but i think it's also a sign of how seriously jangakon europe at least takes

i don't mean to compare with jangakon us it's at least a sign of how seriously we take the code of

conduct and ensuring that it's an inclusive environment for everybody yeah and we've just

one of the many checks we've said this a billion times on the show but like one of the things that

makes a jango con sort of special is you know the atmosphere and the environment which is you know

built upon the foundation of the code of conducts and its enforcement it's no point having a code

of conduct and they're not doing anything about it it's absolutely absolutely i mean people won't

be able to see this on the podcast but it's like i was nodding furiously it's the reason um i should

point out like i'm i'm a co-chair of uh janga con europe because my wife is the other co-chair so

it's like they're probably the most dangerous thing we've done for our relationship over 20 years

um is is to take on this thing together but um it's the reason we both wanted to do it is because

it's just one of the warmest friendliest like nicest environments that that we visit and i go

to conferences for a living to some degree um so when the opportunity came up to bring that

community to a city that we love like i just absolutely love living in edinburgh um that

you know we couldn't really turn it down although i really would have liked to i never wanted to be

a co-chair of a conference um but if i did it would be this conference it's going to be awesome

it's already looking really really good i'll have to um i was gonna say jeff triplett has

for the us one has put out a list of talks i'd like to see that he does sort of annually i maybe

i probably have enough ideas i'd put one out for people i mean because the other one if anyone's

listening i'm fascinated with the idea of starter projects um and as ever like where do you draw the

line so for django like i'm like the problem is it's only kind of an independent person or

consultant who's in a position where they're doing this all the time most people just drop into a new

you know an existing code base but like what are you know what are the i mean i have a forum post

like on third-party packages which you know top 10 that you want but like what can we all agree

on like where does it all become you know where does it all diffuse i feel like we can keep pushing

that out in terms of like okay right well we need you know django all author you know there's a

whole bunch of things like i'm always curious where that line is because that keeps moving

but um because i'm thinking i think about this in terms of i have this django x starter project

which is intentionally extremely basic, but it's designed to sit between that and cookie cutter.

And I get PRs that are pretty regularly like, add this, add this, add this. And I'm like, well,

I use that and I would, but I'm not as confident. I'm deliberately keeping it. I feel like there's

room above it, but before it becomes a custom subjective thing. So I'm always curious how

people do that but very people very few people are in a position to do that is the challenge i think

yes yeah that's true i um this that was just an idea for anybody who wants to take it up is that

right yeah yeah i should put it up we we have a list of ideas on the janguk on europe website as

well in fact it's one of my bits of advice when i gave that talk about getting on the stage it's

like go and look at the cfp guide page and they will usually have a list of topics that they want

to see talks on um so uh it's a good start those are pre-approved right it's like that's what the

organizers already thought of that they'd like to see see talks about so uh anybody who's a new

speaker looking for an idea like definitely why mark smith is great why i wasn't allowed to put

that in the page right okay well anyway if you are listening at home you got any doubts just send

just you know just put in your talk idea because whatever you're excited to talk about other people

will be excited to listen to and it's as simple as that it's you you know just just submit that's

absolutely true and um it doesn't matter how nervous you are when you give the talk there

will be a round of applause at the end and people will come up and tell you how much they enjoyed

the talk um even if they have to pick out one thing from the middle that they found valuable

it's like they will come up and tell you and it's a great way to meet new people at a conference so

that's uh yeah i can't recommend it highly enough i was not a natural public speaker but um after

giving my first talk it was like oh i'm going to continue doing this um because i didn't feel like

that before i got on stage but after i got off the stage i was like yeah i have to do this again

yeah well yeah nobody's there to take you down a peg every i think everyone there appreciates

how much work it is how difficult it is and so yeah nobody nobody's there to tear you down

yeah oh we look forward to seeing you in edinburgh in uh in may all right anything else is there any

last minute things you want to say before we sign off mark it's your last chance i had a feeling

you're going to ask me that and i'd been saving up like encouraging people to submit a talk okay

and i just did it so it's like no i have nothing

all right well thank you thank you for coming on it's real pleasure to meet you i've i've know of

your work and i've seen your talks but first time i've got a chance to connect so yeah it's been

awesome i've really enjoyed it thank you thank you so as ever jango chat.com we are on master

mastodon now um and we'll see everyone next time bye bye bye