Transcript: Official Django MongoDB Backend - Jib Adegunloye

Hi, welcome to another episode of Django Chat, podcast on the Django web framework.

I'm Carlton Gibson, joined as ever by Will Vinson.

Hello, Will.

Hello, Will.

Hi Jib, how are you?

I'm doing great.

I'm really excited you've come on, you've got some really exciting news.

Um, because Mongo just released uh the preview of a new back database backend for MongoDB for Django, right?

Um we just did our public preview or beta of uh our release called the Django MongoDB backend, and it quite literally is what it sounds like.

Um a MongoDB backend for Django.

I'm I'm gonna pause us there on talking about that, because we're gonna nerd out on that a lot.

So before we get there, I want to know I want to know a little bit about you and you know who you are and you know how how how come you get to be building this thing.

So, you know, do tell us.

Um, so I guess I'd start at one, you know, the the beginning of my sort of software engineering journey, journey.

Um, I you know came into sort of CS as an undergrad and over in 2014.

Um I'd always known I'd like CS and I was actually really interested in interested in database systems.

Um, so a large bulk of my uh CS education was actually based in SQL databases.

So Right, of course, right.

And from trying to build something of my own, it literally in the bike setting, bike shedding conversation with uh with my ragtag team of friends, they said, Hey, can we use this thing called MongoDB?

And I was like, what is it?

And they said no sequel.

And I was like, absolutely not.

I've I've literally spent occasionally but eventually uh eventually I actually got battered down and uh I went ahead and I used MongoDB.

And I will say I am now like a MongoDB evangelist.

Um so really um I started like once once I graduated, started seeing more and more.

Um and so when I got to my first job over at Meta, I started seeing more NoSQL and and more pieces like that.

Uh and you know, after kind of doing the gamut around the the tech scene, I'd seen an opening for the very thing that you know had kind of like been the defining factor of me leaving sort of academia and going into you know the business side of things.

And there was a position open to be a a senior python dev over at MongoDB, and I was like, yeah, absolutely.

Like you will see me there.

Is that what you're saying?

Okay, super.

Were you working on databases as well, but or no?

Um so I first started off at what was formerly known at the time as Oculus Research.

It's now Facebook ARVR or I think reality labs.

Um and so I did a lot of the stuff around the what's currently seen as like the headsets.

Um and then um from there I had transferred over to what they you know termed the the the blue app to Facebook proper, um, where I did uh quite a bit of machine learning ranking work for the groups, Facebook groups um activity feed.

And then from there I decided to do yet another pivot, and I worked uh deeply in the traffic organization, specifically uh dealing with live stream uh video network protocols.

Um so on across the Facebook or a meta family suite of apps.

So Instagram, um, Facebook, uh uh Oculus even.

Uh so um if it had live, we we were we were very, very much in tune.

Um and like I said, uh I started out in a life of databases.

I found myself in testing infrastructure, then machine learning, uh, and then network protocols, and I was like, let's go back home.

Let's let's go back to

that already.

I'm giving a keynote on uh Django and data science, machine learning.

So I'm fascinated to hear from people who've actually done it in a professional capacity because I'm certainly new to it.

I I know the web stuff a little bit, but it definitely feels like two separate realms.

Like people who do data science don't really do web and vice versa.

Does that match at all with your experience?

Um, I'll say that the um like one thing that I was introduced to was truly the vastness of computer science by jumping into this machine learning space.

When I had done the transition to the team, I had almost openly said, I have no experience yet.

Um and so um, and I thought, hey, maybe I would have experience given, you know, I understand scaling large-scale databases and things like that.

Um, there are definitely some notes that you can pick up, like uh in terms of like how do you ingest all this data to feed it into um to feed it into a system that that's getting trained or that is um using the information as training data uh in order to generate informed decisions.

So there's definitely an aspect of if you know a lot of really complex SQL, yeah, that's a great place to get started, right?

But I'd say the big thing that um I'd learned like when dealing with applied machine learning um was understanding that you know the the components you learn about in theories, right?

Uh the the things about hypotheses um and um building uh nodes and things such as that, they all kind of map to a larger thing, which is the larger, like like what is the genuine machine learning pipeline?

What are the machine learning componentry that stack up in this pipeline and actually give me the result?

Because even um, I'm not sure if you're familiar with the current generative AI things, those similar like sort of principles kind of map where um the AI or the MM, the machine learning, to be specific, machine learning, because generative AI and machine learning are truly like they have they have kind of like uh are forked in the road now.

Um, but yeah, the machine learning aspect is well, what part do I need to apply my machine learning functionality to to then pass it back off to you know, like the sort of what I like to call the bread and butter, the meat and potatoes um engineering aspects of still parsing, processing, and generating output for the information.

And so what that looks like sometimes is hey, I've got 1.7 billion users, right?

Um, and I want to suggest them a group.

Um, what can I look at?

Well, I know that they've got some, um, maybe they've got some terms that they use that are public asset access content, and I can do uh basically a unsupervised um learning where I just kind of match up commonly place together keywords.

The output of that are like tuples of just like two to three sort of keyword pairs or word pairings, um, and you actually can't make any sense of that.

Like you, but you now know that people talk about the word dog and chicken a lot in a post, right?

Like that's the thing that the uns the clustering algorithm told you.

It's like dogs and chicken, they're doing something there, right?

So now we have that raw new piece of information, and we're like, okay, so we know that we want to look for posts that say dogs and chicken.

So we then run it through another algorithm.

Well, we then actually run it through some SQL where we're we're querying for inf posts or information that are specifically using both the terms dog and chicken simultaneously, and then we feed it through yet another machine learning um algorithm to uh understand.

Well, okay, of these phrases that we've now isolated, how can we use it to understand uh or like piece out sentences that actually make sense?

And then finally we piece out the sentences that make sense, and then we feed it back into our usual like ranking algorithm of what's getting the most engagement.

We we score that on against engagement, and then finally, boom, you've got the output you expected.

Two out of four of those stages were machine learning.

The other two stages um were just again, like us kind of making sense of of the of the machine learning application.

And so it's that fine-tuning where you build a machine learning um like uh machine learning algorithm um and applying it and just building a sense of confidence around it in that model, um, that becomes the crux, at least of my app sort of like applic uh application of it.

That's one of the few times it counts.

So I don't want to get too off track, but uh that's all like super interesting and um yeah, we could we could talk about that the whole time, but we're gonna talk about other things.

So, Carlton, go ahead.

Well, okay, go on then.

So tell us um not meta, sorry, your mongo db.

Tell us about what you do at Duongo, so you're senior, senior software engineer there.

And if I were to describe my role right now is sort of on paper, clear cut.

Uh I deal with the like our um I work on the uh database experiences Python team, specifically also known as the uh drivers python team or python drivers team.

And what a driver is, is essentially um the the like the library or set of mechanisms that allow a user, a developer namely, to easily connect to their Mongo, like their database instance.

So in this case, the the like our our mainstay, our core language or our core library is PyMongo, which is the Mongo the the Python driver for MongoDB, right?

Or the MongoDB driver in Python, yeah.

And so uh the mainstay thing we focus on and we make we maintain is that library.

There are a uh several subsidiary libraries that we either contribute to heavily or maintain, um, some of those being Mongo Engine, which we contribute to, um, or uh an example of one we maintain ourselves, PyMongo Arrow.

Um, and then additionally, there are several others, especially like in the AI ML space, like uh Langchain.

We've had several contributions there and have you know sort of built a relationship and rapport.

Uh and then we've also got the web frameworks that we've done our best to integrate with, namely, you know, things like Fast API, Flask, and now um uh uh Django.

So we try our best as a team to integrate there.

I wanna I've gotta I've gotta ask, because hang on, querying Mongo is nothing at all like using the Django RM.

So uh hang on, what's going on here?

How have you how how on earth are you are you integrating with the Django ORM?

So um one thing I've been saying um and I'm I'm actually eager to get you know like uh checked on it, is been when I look at Django, when I've when I've viewed Django, I've also always been like, oh, this is a SQL like framework.

Like that's what it is.

But when you start pulling or piecing apart the layers under the hood, you realize that the way the system works is yes, there's the the Django query set API, like that framework piece, the component tree that allows you to actually issue queries to the database.

At one point, at some point, that gets converted to explicit SQL, like structured query language, and then that gets fed to the database.

What you realize is okay, so there's a set of API calls that actually generate the SQL.

So technically, if I just remove the parts that generate the SQL and replace that with MQL, the Mongo query language, it should work, right?

Um and like I said, I

got Jim.

We'll just do the Mongo query language in there.

No problem.

Go on, carry on.

So yeah.

Um forgive my SQL um here, but essentially you'd be doing select star from you know select whatever from specifying the database table where starts with the letter Z.

That's what it would be structured like in SQL, right?

So for us, when you do that lookup call, instead of calling, you know, basically like this conversion of as SQL, we have embedded a callback that mirrors that same behavior where instead of calling what as SQL, we're calling as MQL.

And this as MQL um will uh is essentially um doing uh taking that step of where it's seeing the starts with and it's making our manga db analog.

We actually do have uh a predicate that lets you regex match against the the beginning of a string.

And so um that's that's like the one mini component.

But the thing that there enraps it is we've created our own customized compiler because what uh Django is actually speaking to is a SQL compiler.

And so we've made our customized SQL compiler with an embedded Mongo query generator that at the same steps where you would call these that SQL where clause to generate um to generate your SQL, we call we call that where and it starts generating what is the syntactic equivalent of a WHERE in uh MQL.

And then um and then uh the the stuff that's wrapping it, similar to how it's wrapped in Django, where I was talking about select star from, that's that's kind of standard information that you get from having the model exist.

You already know what you're selecting from, that's the model you defined, and you already know what your columns are because you define them in the fields.

So if you know you want X fields, we already know that by virtue of having the model.

So when we it's very easy for us to now construct our MQL equivalent and say, all right, we don't have a select star from, but we do our match statement on starts with, and then we do what's known as a project statement to say, actually just give me these specific fields that I specify.

And so the dollar projected MQL looks like works in a very similar way to select to you know to select star from, right?

Implemented that as well.

And so what we what this led to, or what this leads to is an almost completely obfuscated layer of MQL that a Django developer does not need to know in order to get work done.

Right?

I mean, based around the um the Google App Engine um data store originally, I can't remember what it what it was called, but it it and it kind of it kind of worked, but only only kind of.

Whereas you've gone a step further, right?

You've got pretty much all of it works now, right?

And so to even and it's great you brought up the Google App Engine, um, because I would say that one, that was a product of almost 10 years ago now.

Yeah, yeah, right.

At that time, MongoDB um was, I believe, operating on a version uh less than server version 4.

We are currently trailblazing on MongoDB server version 8.

We have a wealth more of uh of querying predicates that um do mirror uh a lot of the things and expectations over in SQLand.

But also, like we we too have built uh a unique set of things that uh have made us a significantly more powerful, uh significantly more powerful database than um the implementation strategies of 10 years ago.

And then moreover, um those implementation strategies were using uh basically a different paradigm.

Right now, um we're leveraging the unique power of the MongoDB aggregation pipeline.

Um I'm not sure folks are familiar, but just to walk through the MongoDB aggregation pipeline, um the it is uh it's not necessarily new, but it's MongoDB's um sort of premier service in creating very, very intuitive and complex uh queries that can really um stand uh you know production level um work.

And so how it works is there are you've you've heard me say the term predicate, right?

So they're essentially these um aggregation operators and this aggregation pipeline, and I want you to imagine like an assembly line of of instruction, or like an assembly line the traditional assembly line right and so the MongoDB aggregation pipeline works kind of like a traditional assembly line where each predicate is its own isolated augmentation onto the collection or the database that you're working against and so you can quite literally intuit what's going

to happen next because it's procedural right so um if I were like using that starts with Z example right I know let's say I want to find something super complex like um I want to find every movie that made over two billion dollars in the last 12 years right you could spend a bunch of time trying to think about how do I do this in a very in increasingly nested structure or you could say first let me match on every movie um right the I'm I want to

group every movie by all of its total sales great that's one step now that I've grouped every movie by every uh each one of its total sales I want to now do a greater than operation on two billion like based on this do are they greater than two billion great now I've found all the ones greater than two billion and now at let's say I want to do something like rank them third procedural step all of these isolated buckets that can can be swapped around interchanged um and tested in real time.

So um which actually uh jumps into my third point about why this is different I hate to kind of you know uh to put to put it out there so plainly but in the past solutions you know MongoDB may have been a passive member or we may have you know helped influence the solution but in today's iteration in today's version MongoDB has put their full force behind it.

We've got dedicated engineers working on it as well um we've uh enlisted uh the consulting power of Tim Graham a former Django fellow um so all of our decisions are couched in both Django specialization and MongoDB specialization so there will be times where we truly like as a team come together butt heads and have a real discussion about like okay in this in part of the implementation are we more Django

here or are we more MongoDB or can we move up?

Right and because we're now taking it from a from two from deeply specialized angles we're able to say more concretely whether or not this is a decision that stand that will stand the test of time because we know we we've got somebody here um two people in fact that can speak very very acutely to what a typical Django developer looks for.

And then we got employees who you know deal with manga to me every day.

And so I think even in that relationship we've we've formed something richer um and and we folded it into our into our quarterly plans our yearly plans it is it is something that we we're committed to continually iterating on.

I think like the the trouble with third party backends historically has been well who's maintaining them who's developing them where's the engineering time to make sure they keep up with the changes in the in the ORM as Django Django evolves.

And like for Mongo to allocate that in engineering time to it is that okay that gives us a you know some some degree of confidence that it will continue to evolve compared to say other you know other backends from companies that we won't bother naming it to that stuff.

Yes.

So in terms of that there's definitely overlap because we can use it but in terms of how you we view or we've treated each one I would say that um uh from from my experience like the Flask and the Fast API they are they're sort of closer in in wheelhouse um based on their implementation um you know flask being sort of like the least opinionated as we know like hey you just want to start something up here you go right fast API having a bit of an op

inion um the specifically the the uh Sebastian's template generator is a little more opinionated because it helps you scaffold things but in terms of like letting you kind of pick and choose your own adventure fast API still allows that leverage.

Django very very on the other end it's

Or or disagree with me please if that's not the case.

o view and then all of a sudden you're bringing in you know the the the the library for whichever you know whichever um no SQL database you're using and it's like well hang on this just doesn't feel like Django exactly the whole point with integrating the ORM is it will just feel like Django still.

I want uh solid administrator management I want solid session management I want to be able to know that when I build a form it's all automatically like being able to like take the information and then write it to the database in such a way that I'm not thinking about this like 24-7.

In the prior solutions they're still gonna be a somewhat non-trivial now and that's not to their their detriment it's just their

but to yeah but to answer the question in the present right there are a dirge of MongoDB specific things that we are aching to get in.

We're just trying to get it in the right way.

So one example is in uh in a traditional like uh SQL structure you're looking at the usage of foreign keys that's that's second nature right um for us we also support that ability like it's it's like if you want to use your foreign keys go ahead but we understand that what makes MongoDB performant is moving away from that paradigm and using um what we call embedded documents.

In this library they're known as embedded models um and they're viewed as embedded model fields the power of MongoDB is that you can have a document with tons, like like that hundreds, thousands of sub-documents in them, and then query against that.

Because this is not something that's really been done um in SQL.

Like I know there's HSTore, there's there's sort of like sub-objects and things like that, but um to the degree in which they're done in MongoDB, it's a little bit different.

And so we've been working on ways to get this nested embedded model structure to be airtight, um, but also work very intuitively in the Django ecosystem.

And so that's one area that we're we're actually barreling down right now, and and and we are like absolutely set on making sure that that is a fluid experience for folks.

Um, there are other things that you know um that we're that we are um we've we run into corner case situations on.

Usually if it's like just uh an issue around like SQL, then we just we we ignore it.

Like, well, I can't support these SQL functions, it's okay.

Um But yeah, I'd say like a lot of the like we we're we're running into things where we want to improve more and iterate and introduce more MOLDB specific aspects.

And I'd say that the like as we are personally going through making sure those work, um, we're also hoping that during this like beta phase, people tell us what isn't working um or what they find to be weird so we can immediately kind of capture those.

Yeah.

That's buttondown.com.

Email software for developers like you.

There are hundreds of email marketing software services out there, and they all pretty much offer the same thing.

Collect and clean addresses, send out broadcasts or drip campaigns, get analytics so you can see what's resonating and what's not.

Buttonown is designed to hook into the tools that you already care about, everything from static site generators like Jekyll or Hugo to payment platforms like Stripe and Memberful.

You can hook your site up to button down with just a form element or a simple rest call.

Write emails and markdown and then get on with the actual work you're supposed to do.

New customers can save 50% off their first year with Button Down using the coupon code Django, and if you email support, they'll white glove migrate your existing subscribers and archives for free.

I know Async is you know a new frontier in Django, and I believe there isn't really Mongo support for that yet.

Is that correct?

Um we're also in our driver, we are also currently like um improving like our asynchronous asynchronous functionality support.

So simultaneously with this, you'd see like an and like an like an improved and richer version of uh asynchronous functionality both by the baseline driver and also and and we hope it it complements uh Django as well.

So um we aim to have support, uh, but for now we we've we figured it's it's just based it's great to get that that sort of bread and butter synchronous um uh ability uh good to go.

Um and as we you know push towards gender availability where uh more production use cases are leveraged, we we we want to be confident about saying, hey, we can also support your asynchronous case as well.

Okay.

You know, what was the bit where what was the moment in the project where you're like, oh, oh, this isn't gonna work.

So I think uh there's there's actually so many of these.

Um one uh I'll say I you know, let me start off with what was easy.

So what was actually shockingly easy to me uh was um doing that initial replacement of um like the MQL with SQL.

That was actually shockingly easy.

It was like, hey, here are the lookups, here are the functions that the lookups will generate instead, right?

Um it's quite literally a dictionary.

Yeah.

Where things got very interesting was when we had to start um supporting things like annotations, um or supporting um our version of grouping, uh like like um, yeah, like supporting group by in a MongoDB uh specific way.

Because like I said, we have a very procedural, um like procedurally generated uh SQL statement.

I mean um MQL.

Uh it's an array of it's an array of Python objects, if you if you want to think about it like that.

And so one big thing is in order for it to be self-referential about something that happened, let's say at step number four, we need to capture metadata now.

Um, and then make sure that the metadata hasn't been mutated too much from step number four to step number seven um to ensure that the output is the same.

And then another challenging one was MongoDB at the database layer handles nulls differently, right?

So where Django would expect uh a null value, MongoDB is like, bro, we don't have a value at all.

Like we like we're we're not we're not behold into that.

And so um we would run into those uh almost quirks where it's like, okay, what do we need to do?

How do we get smart about this?

Um, and how do we like really get things working?

And then I think honestly, personally, personal to me, one of the biggest um necessary headaches, tests.

Wow, I was gonna ask you about tests.

Yeah, go, go, go.

MongoDB does not use, like traditionally, our primary key is an object ID, which is a um let me not get the bit encoded, but basically it's it's like a specialized uh unique ID um constructed in Bison.

Um and whenever you submit, let's say, in this case, uh in like let's say you submit a document or a row, um, we don't supply that object ID usually for creation because when it reads like like because we'll automatically generate an object ID and then submit it.

And so what would happen in the test is the test case would expect hey, um, this object that's coming back should have the ID of one, two, three, or four, and instead we'd have the object ID of one a three seven three five, like, ah crap, right?

And so we had to we had to basically fork the test to then override the natural test structure um to uh allow us to just use object IDs or um to or to just like uh kind of like uh fake use uh like a different ID implementation.

And the thing is, this sounds again hand-wavy and easy solution, but when you when there's hundreds, thousands of tests that run into this like minute issue, and you're trying to debug whether or not, hey, is it this thing that we've identified or something completely different, or is it something we can even solve at all?

It is something that you have to quite literally look at each test and deduce yourself.

And it's like I think it was for me, it was it felt like the most daunting thing because I to look at such a a uh a tenured databases um or tenured frameworks test cases and say, like, okay, we're gonna try for this, this, this, this, this, this, this, this, this, this, this, this, right?

And then to kind of present that to the team and then have you know, Tim, who extremely well informed, like, actually, no, not this one, not this one, not this one, not this one.

And like, it's it's just so it's it it felt like is this ever actually going to end?

Um, and thankfully it did.

Like, we we we managed to triage every single test, figure out the ones that don't necessarily matter for the sake of what we're doing, and the ones that we absolutely want to make sure matter based on this forg.

But I think it comes out to the number of like 82 test suites with well, you know, well, into the hundreds in terms of tests.

But once everything went green, that was honestly like Christmas Day for me.

Yeah, I can imagine.

One, we keep a uh because um you know Django's got its branching as like 5.1, 5.2, 4.2, um, we've got our individual branches, like forked branches that will keep up with whatever changes they manifest per those specific branches, and clearly, like for main, like we we also do our best test against that.

And so, in terms of like how we keep that up, is hey, we check with 5.1.

Are we in alignment with the 5.1 tests?

Are we aligned with 5.1 changes?

Great.

Are we aligned with 5 0 changes?

5 0 tests, great.

Um, with main, we see something's been added.

Is this is this something that we want to get ahead of now?

Yes.

And so we we we have even like testing coming in in later bits just to uh continually make sure that um we are up to date, and so we again triage the issue to understand whether or not it's something we need to tackle now, if it's something we can just skip, um, and and things of that nature.

It does all the time.

And and so that it's that bit, it's the the kind of like, yeah, we need to keep bringing in the new tests, keep bringing in the new features as they're developed, you know, to the extent that it matches the the the data model.

Yeah, yeah.

I was just saying, I just realized this is I think we've this is now the third episode we've done related to MongoDB, because we've had two developer advocates before.

We had Mark Smith, who's based in the UK uh two years ago, I think.

And then we had Aaron Bassett before that, who was on the Django, I don't think is at MongoDB anymore, but he was on the Django board with me.

So um it's the rich vein of discussion.

Yeah.

I think there was a third, there's a there was a it wasn't anything official, I don't think.

I've got one more question.

Migrations.

How does that look?

No SQL schema list.

Like what how do how did migrations fit in?

Do I I just divide create them as usual, run them as usual, and all this magic?

So one, yes, I'll say this.

Um next question, Carlton.

Yeah, yeah, yeah.

Obviously.

Come on, why are you

So again, MongoDB schemaless.

I will say that you know, having a lack of a schema, one, does not mean that you don't need to have a schema.

I as a as a person who's done a lot of database work, please always, as you're like as you're codifying your work, establish a schema because you don't want developers or anybody kind of saying, oh, there's no schema, I'm just gonna throw in a random key every now and again.

And then you can see that.

later down, later down the road.

Yeah.

Um right now we've chosen not to enforce a schema.

Again, we're in beta.

Um because most people understand the rapid prototyping and development of MongoDB.

Uh and like m enforcing the schema at this stage just feels like um, you know, we we may uh feel like we may be jumping the gun, right?

Um but the beauty of we we still recognize the beauty that comes from having something like migrations.

And one, now that it's codified what your schema is, regardless of whether or not it is or isn't enforced at the database layer.

Um, and then two, um when you make alterations, migrations will also capture those alterations and make them um as necessary, supplying, and uh even go so far as you know, supplying whatever the necessary default is when it when it comes down to you.

So, like in terms of migrations, it works TM.

It's just it works.

It works, uh, it's just not enforcing schema, but that does not remove it, the level of importance we understood and found about it.

right there directly, and then go right back to your Django development, and it will it'll propagate because you're using the same, like the same client configuration of the connection.

And so um if you if if somebody does want to enforce schema, we've documented how you can go do that, grab the client, use the $jon schema validation, and you're good to go.

I'm like, you know, I'm I'm I'm just itching to go and go and play now.

Thank you.

You know, I mean I know there's a whole lot of press that's just come out, like I went we've touched upon a number of things.

Are there any anything we haven't mentioned that specifically you're excited about or think people should know about with this this new driver?

Um so first, um I think one of the biggest things that uh I'm really proud we put out uh in in this iteration is uh it's called uh raw aggregate.

Now, I understand there is there's there's literally a predicate for aggregation in SQL.

This is raw aggregate is um is is hearkening towards MongoDB's aggregation pipeline.

So if you remember the dot raw uh in Django, which allows you to pass a string that has a structured SQL in it, we've made our own analog, but it's not just a string.

Um it's a list of, like I said, Python dictionaries, which in this list of Python dictionaries, you can quite literally construct a normal MongoDB aggregation pipeline.

Uh and what it'll do is similar to how the dot raw works, it will give you a a uh uh a Django query set object.

Like you will get a query set object back from a query you've done basically executing raw uh MQL.

Um and I think that's been that's a really powerful thing because nobody no no other implementation has done that.

No other implementation has really looked through the API for all of its richness and said, hey, this is something that um if this is something that's that's that would be really powerful to use, especially at this stage.

And the reason, even more so why it's powerful, is MongoDB's got things that come natively, right?

We've got vector search, we've got um our full text search capabilities, we've got our geospatial query, and we've got even more interesting like um pred like search predicates, like graph lookup and things like that.

And by using that raw aggregate framework, you're almost you're immediately able to interface with that and then still get Django like you'd expect, and use it very seamlessly or or or very well in the flow.

Like in the future, we definitely want to improve upon this and like you know have uh potentially like some you know integr more integrated solution of search and geospatial.

Um, but for now it's it's a great way if you want to like do more advanced things that you know or you understand MongoDB can do really well, you can.

It's like one of the complaints about an ORM, a multi-database ORM in particular, is that it's kind of ends up being the lowest common denominator, is the criticism, is because you don't get the advanced features of database A, and you don't get the advanced features of database B, and you don't get the advanced features of database D.

So to be able to break or to have made space to say, you know what, if you're using Mongo, you can still reach out for those advanced features.

That's a nice, a nice addition.

Of course you can, you can always get the driver, you can always do the things, but if it's nicely exposed in the API, that makes it all the more enticing, right?

And um I want to say, you know, very succinctly, yes, we do.

Um, we actually have introduced this uh operator called $LOKUP.

It's been something that's existed in MongoDB for some time.

And um without getting too in the weeds, that is that is our joint.

You can execute a left joint, a right join, an outer joint through the dollar lookup operation.

Um now, my big sort of cautioning point is uh like I'd said earlier, we are not uh like we don't the that third normal form and and and leveraging that, that is not what makes MongoDB MongoDB.

So for for now, um please go ahead, use it, uh create a foreign key, create a one-to-uh one to many map, one-to-many field, a many to many field.

Um, but uh really uh uh in the in the in the coming months, in the coming weeks, like look forward to seeing um the things like nested embedded models, and we will also provide documentation on how to transform that sort of traditional foreign key usage to uh a representation in MongoDB that leverages that the that more traditional document structure that that we as at MongoDB know I'm not.

Ultimately the um SQL model and the the document model aren't the relational document models aren't the same, right?

And and uh and this is the say the third piece that I want to touch on um is the third-party integrations, right?

Third party libraries.

Yes, so like I said, like like we've chatted about, we're trying to make this work with Django, period.

And so part of Django's power is all of the libraries that have come out.

Things like Wagtail, Django All Off, Django Rest framework, um, Django Filter.

We are currently doing the work to make sure those are just as seamless as we're making this ORM, like as we're making this framework integration to be.

So um that that is that is on our work and that is on our roadmap um for the months to come.

And if there is a third-party library, a third-party framework uh with Django that uh as a developer you find important, we are all ears, we are listening, we are open, um, and we're like and and we we'd love to know sooner rather than later um what those are so we can evaluate whether or not we can support them because we we'd most likely want to.

Um and so like to conclude those three points, it's like we're still growing, we're still evolving, we're still learning.

Um I I say this, yeah.

I say this as the sort of lead engineer on this project.

We are here for real, like we are fully invested um and and with open ears, and we are a team of enthusiastic Django developers writing MongoDB, like code for MongoDB.

So don't ever think you can't just you know check out the MongoDB community forum or post uh something in the Django forums, uh, or even post something on Reddit and and at MongoDB and let us know.

We we've got folks just just looking, waiting, checking, because we want to do this right and we want to do this permanently.

That's kind of um that's that's kind of like the the recurring motif for that.

So that the you know that you're here saying that Mongo's here for the medium and long term.

That's you know that's a really nice thing to hear.

So I I think this is I think this is really great, I mean, selfishly for Django, right, to have a powerful and OS equal story because people have used Mongo over the years, um, but it's been you know, there hasn't been that all-in support the way there is now.

And so both, obviously, hopefully Django's clearly makes sense for Mongo, but I mean just for Django to be broader in terms of Python web all-encompassing, yeah, it opens up so many areas that you know people one of the big reasons why people use Flask is because they want NoSQL and they feel like they can't do it in Django.

Like that I would say that's a top, top three, top five reason I hear.

Um, so that would be pretty powerful for Django if if that shifts a bit.

So I've been doing a project, um, uh, like I said, that the project that I started off on MongoDB sometime back uh with my friends, and we were like, okay, we've been using Flask, um, this is not like a production level site.

Let's actually um, you know, get some authentication and paneling that we can stand behind.

First thing we looked towards is like, okay, what does it take to move to Django?

And we ultimately had to pass because the upkeep for maintaining both MongoDB and Django at that time was not something that we as sort of ragtag group of developers uh um felt like handling at the time.

And so this really feels like a full circle moment for me, um, as well as those same friends who I am currently messaging, like, guys, look what I've done.

No, I mean, and again, thinking about what I just said, I think it's definitely a top three reason why people choose Flask.

Like, so why would someone choose Flask?

They would say it's more lightweight and you can get started on it faster.

There's been a bunch of work, not not the least, Carlton's given talks.

There's these nano Django repos and stuff showing that, you know, hey, like you don't have to use all the batteries of Django right off the bat.

No SQL support is a huge one.

Yeah.

Um I can't think what the third is, but you know, to leverage what what we have in this community of the third-party packages of the forum, of all these things, like um, I I can't speak as much to Fast API, but but Flask, the whole point is it's a tool in part that you use with other things, whereas Django is an all-in-one code and community in a way that you know Flask is is not.

And that's not to take anything away from Flask.

It's just they're different, they're different things, different tools for different jobs.

There's also a like you like when you're implementing something for Django, you're implementing something for like clearly a community.

It was very like my first time at a Django Con, and I was like, I was, you know, grading ear to ear just chatting with everyone.

Yeah, and I really, really look forward to going to another, like, um, and and sort of being engaged because I understand it's not just it's not just code that's being written.

It's like the impact to the communities, um, the like the the workflows and how it's you know influencing so many things and how even engaging with the community can help you know inform uh decisions itself.

So I like I I will say like it's it's a it's a full package deal, and and that's what I do really love about Django.

You know, there is a difference apparently between the EU and Europe, especially when we were in um Scotland two years ago.

So I uh I'm attuned to that because I was corrected on it several times.

So just I'll just mention that to you.

It's it's DjangoCon Europe, not EU.

Thank you so much.

I think these things are important in these positions.

Over the less it less important, but you know.

No, thank you.

Um uh great, great.

We often ask people, um, but I think it's it'd be really nice to ask you this this question because you've you've come to Django as somebody implementing an you know no SQL backend for the ORM.

If you had a magic wand and you could fix one thing in Django, what would it be if you know if Django could make one change?

What would what would your magic wand fix be?

Magic wand.

Or new feature or change.

Um let's see.

Magic wand feature fixing for Django.

Oh, um, I would say that uh I would really like um.

Sorry if we sorry if we sorry if we sprung it on you.

I mean, it's something we've gotten in the habit recently of asking guests um because yeah, it's a tough one.

Right.

And I think that's off putting to to people.

So that that seems like something that could be done.

I'd say in that same vein, yeah.

Um I think uh when we were like creating our quick start for um for the Django project I was like how quick can I make this quick start and and I feel like um yeah there were there were several things that if there was a if there was a way to even um basically go from project to first app in one command line argument that would be a uh that would be beautiful because right now like you start proje

ct after you start project you start app and then and then you you then go in you edit the app and then you also have to edit the main line to link URLs.

So I there we go I found my answer.

URL linking behavior

So it's a start project and then that's already an app that you would then carry on to create your view and things.

He's been working on that a lot over the last couple years.

There's also I maintain a package called Lithium now that was Django X that's sort of it doesn't have full deployment yet but it has like just gets you started like you know gives you Django alloth gives you crispy forms gives you a couple things so you can just get going I would check that out it's basically a simpler um cookie cutter Django, whatever that is called so that's been around for a while.

Uh but if there's a you know eventually if there's a dep and there's an officially supported way to do it I think it would get over that idea in people's head that Django is like this beast that can't be tamed when you know it's customizable but we can have a an easier path if you want to just kick going with something.

Yes Carlton

wn start project.

Yeah.

There isn't really a big next place.

So um Mongo seems very well placed just for that to slip in and also because yeah like if we can just do away with the NoSQL doesn't work on Django story that's that's such a huge win.

Like let's just swap it.

Oh yeah you can you can put that to the board we'll

The the fastest response we'll give is if you provide um feedback on the MongoDB community forums or filing an issue or Jira ticket um because that's that's that'll get right to developers almost immediately.

Also definitely um like yeah definitely uh if the Django form is the is is the next sort of comfortable line I I'd say go there.

But yeah the MongoDB community forum and um our issues and help section um providing a Jira ticket.

Well thank you this is uh this this was a pretty amazing uh chat I think so and and timely I'm sorry that it's gonna wait two weeks uh to come out but that's okay we'll you know the other things will come out on the on the news the Django news newsletter and other things and then people will be uh will have this to look forward to to have more explanation.