Transcript: Building a Django API Framework Faster than FastAPI - Farhan Ali Raza

Whether it's scaling an application, deriving insights from data, or getting results from AI, Six Feet Up helps you move forward faster.

See what's possible at sixfeetup.com.

I'm Carlton Gibson, joined as ever by Will Vincent.

Hello, Will.

Hello, Will.

Hi Fahin, thank you for coming on the show.

I have been seeing you like for like uh months and like I'm for I have been following Carlton for years, so like it's an honor to be here.

So come on, Farhan, let's talk about um template partials and um the Google Summer Code project, because that was the big thing, and I've mentioned it a few times on the show, but you were on you were on the sort of coalface doing the the hard work there.

So how how is that for you?

Describe the summer.

Uh I did not like uh think that it will be like that hard, but like I was I was just uh when I was applying, I was just thinking like uh it is gonna like it's just a package, I'm just gonna copy-paste things.

So but like when doing like I find out like it was not just copy pasting.

So it was amazing uh experience.

Um I learned a lot of things.

It was great.

But that's pretty pretty impressive that your first Google Summer Code is like a top three feature in a major release, I would say.

Carlton, what about for you?

You've mentored other people.

Was there a moment when you panicked?

When is there any moment where you were like, oh no, this is going wrong?

Um in when we were handling render time and compile time things.

So I I I did not like I did not even th think about that uh when applying.

So uh I was thinking like this is something else like uh there is kind of there is a kind of thought that kind of you have to think before making changes and stuff like that.

So that was um the whole process was cool, like I enjoyed the whole process.

There was no panic per se, but like it was just uh a surprise there.

I mean, so the way I've described it when we talked about it on the show is that without your help, I would never have been able to get it in.

Not because I couldn't physically do it, but I just didn't have the time, I didn't have the capacity.

It really does need like somebody dedicate, you know, a 12-week project does need someone to dedicate a good chunk of time to make actually make it happen because there's a lot of things which were the third-party package was okay, but not up to Django standards.

And like there there are so many uh kind of what can we say, like a side quests that happen when we are merging something into Django.

Uh the thing that I remember is the um like uh we uh merged the like that thing like to find partials, uh the regix solution that was merged into Django template partials and then that was merged.

So we thought the regex was the solution, and we I think uh there was five or six reviews for that regex that like we have to fix the regex part, rex part, rex part, and in like in the next PR we removed the trage.

So many side quests that happen.

But tell us a bit more about yourself.

What did like you were you are you a student?

Were you a student?

What are you up to now?

I uh now I'm like just uh freelancing and stuff, like par working part-time, like that.

Uh before that I I was a pre-med student, so I was not like a computer science student, but like it uh it like there is I don't want to go that deep into it, but like uh there is a process uh in like Pakistan called MCAT.

So there is a like a competitive exam for medical admissions, uh like MBBS student admissions.

So uh that was the goal when I was doing pre-med, but uh I discovered kind of programming in my like 11th grade.

So there is uh like it's different.

I don't know like what is uh in Spain or America, but like it's first 11th grade, 12th grade, and after that you start your bachelor's degree.

So I discovered programming in like 11th grade.

I was just um um like I I downloaded a book had first like HTML CSS.

That then I discovered HTML CSS and like I thought that was very interesting.

So like like it was lucky like um uh the like I could not clear the the competitive like exam and but I could become like um uh uh of a doctor of pharmacy or something like that.

So I thought I don't want to do medical at all.

I want to do computer science, so I switched the whole thing around.

How is it taught in Pakistan?

Do you use Python?

Is it Java or what is the formal or what are the languages that are used?

Like every university has uh different way of teaching.

Like uh most commonly I think uh they start with C.

Like uh in first semester there is introduction to programming that is C.

And after that, Python comes at like I think fourth or fifth semester.

After that, like Python and stuff.

Because in the US they don't.

It's like an elective course usually.

So because they uh in like smaller universities they don't have like a lot of teachers, so they just enforce uh even elective subjects.

So um there is a one development uh web development course.

So I I remember I remember like we did HTML and CSS in 2021, I think.

Like HTML and CSS just in six months of that semester.

That's not um that's the case also in some US universities, so it really depends.

Um sometimes the web development course is designed for people who are not computer science majors.

Actually, often it's not, so that's why it'll be HTML, CSS, a little bit of JavaScript.

But um I there's not that many places that have a, you know, Django course or a spring Java course.

Um for some reason it seems seems important, right?

Because everyone gets trained and then goes to work on the web.

But I guess there's a databases course, but there's really there's even you know, even MIT, uh just down the street for me, they have a missing semester.

Uh it's like a January term where they teach like how to use terminal, how to use GitHub, how to do web development, because in the core MIT curriculum, you're not gonna take web development unless you do it on your own, right?

So it's everywhere.

Uh like I I I learned my Python from that course.

Did you actually this is the the David Malin course, did you work through the whole thing?

Because like famously people start, but then it's a lot of work to do on your own, and it just sort of keeps ramping up the uh demands.

So I just uh like watched the stuff that I wanted to learn.

There um there was I think a Django video for one hour, uh and like the Python video there is uh for one hour.

So I just saw like what I wanted to learn from that.

I did not like follow the whole course, I just like uh learned the small parts that I wanted to learn from that.

But that was very like uh very good uh course.

Uh can you you know I mean I've got a whole uh bullet point the list of projects to mention, but can you talk us through your your open source work around Django for the last few minutes?

I would say besides the fact Carlton knows you, when I look and see who's doing really interesting stuff, it's like you again and again and again and again.

So it's not just with Carlton.

I'm having discussions with people, we're sitting back going, who's this mad scientist doing all these crazy things?

So it's just so you know, it's really exciting to see someone smart with new ideas just doing stuff because so it's it's exciting for the community that you're doing this.

So because uh I I was uh there was a like side project I was building that uh required web circuits or like uh a server like SSing server sentiments.

So there was a fear of fast API and Django channels.

So I thought I should build a new framework to solve that.

So I built uh

So I just wanted to have something fast that is built for Django, so and then I I I saw Robin.

There is a framework called Robin.

It is also like Rust and Python mixture, so I thought like it should be possible to like build something that uh handles uh request in Rust and uh does uh like business logic and Django part of the things.

So that uh was it like gives me like inf inspiration to like uh go all in.

I I I I I started that just an ex as an experiment, so I wanted to see if like that was possible or not.

But when I like tried to implement it, it its performance was very good, so I I thought like I can spend like months after that to like make it something better.

It's one of my pet peeves that on the show that people talk about fast, but they never give any numbers.

And you've been you've been laying down benchmarks from day one, which is what makes it so exciting.

So can you give us some idea of the performance, the throughput you get you're seeing on Django Bolt?

Uh so like I and there there is like a whole ready discussion when I um like launched Django Vault on that.

So everybody was saying these benchmarks are like uh just uh just wrong, like that this does not approve anything, prove anything and stuff like that.

So like my whole point was like when I am building something, I will have to use something to measure something, like like what I will do, like a build a whole application and deploy it on a VPS to measure.

I I cannot do that.

I will have to measure like small JSON, small database queries, small like stuff, so that I know like if I make a change, it makes a positive impact on my like base results.

So what I am getting in like if we use one worker, so one worker in Django Vault is uh one active thread, like Actis is a Rust framework that I use, and that one active thread is uh bound with one like Python event loop.

So active handle the Rust part and Python event loop like uh uh kind of uh uh like uh execute the whole Python thing.

So that makes one worker.

So that one worker I get about 27,000 RPS um on like simple if like uh if we return a simple hello world JSON.

So if we do like a database query, so that like the ORM becomes kind of the bottleneck for that, so that like uh the performance that becomes that is still faster if we use SQL Alchemy with SQLite.

So I have not measured with PostgreSQL or like faster uh ORMs with fast API, but I have measured with like SQL Alchemy, so that I was getting about uh 5000 nearly I think 5000 RPS uh with like uh I think there is one query for 10 records that reads.

the question.

Yeah.

Like if we use some fast uh serialization and like uh yeah yeah, I I have forgotten the actual numbers, but like it I think it was 1000 RPS uh on Django, like raw Django, and like I think uh I got about like uh three four above four thousand, I think five thousand RPS in Django bolt.

I mean the first the first thing I saw here was you did um you did a project called Django Rapid, which was using diff which was just Python-based vanilla vanilla Django, but then changing out the serializer layer.

And you can were comparing REST framework serializers to, I don't know, um uh Pydantic serializers to message spec serializers.

Um and what was interesting, I thought there was that you were getting fast API like um throughputs just with basically with Django.

Yep.

It like I think um it will uh like if we have because Django ORMs or fast API ORMs are async, so if we have a lot of queries and uh like a lot of business logic, then the performance I think uh the performance difference be will increase, I think.

And um but like if we compare simple read queries, the performance difference is nearly equal.

Like the the per difference is not very stark.

You mentioned you mentioned async ORM then today, I think, or yesterday.

You put out um a kind of uh turbo ORM package, which is um an async query manager for the Django RM, right?

Talk to us about that.

So he he he wanted to measure that with like the async backend that uh Frey created.

So I he also like I think he does not said like um uh another I forgot his name.

Uh he said like uh you should be able to use that async backend with uh Django bolt and measure like what performance difference you get.

So I when I tried to use it because it is just uh uh async backend, so we can just use raw SQL.

So I thought like it will not be that hard.

Just uh I because in my in my mind I was thinking I am gonna overload some like execution of Django um queries uh and like uh use that ASIC backend and just create it.

But like when I tried to create it, like that was a whole another process, it took me like four or five hours of work uh to just like uh create something working.

Because it worked like in uh in one request it worked fine, like it loaded the thing.

But when I wanted to benchmark it with the bombardier, the like I was getting um like connection exhaustion errors so I had to fix uh polling issues and stuff like that and I did not know like the full it it's it's a naivety of like a person so uh I because I did not know how the full ORM and database things work I did not even think about like how the connection manag

ement works and how the stuff like that things work.

So in my naivety I started and after that I have to solve the whole connection things and stuff like that.

Because uh in a Django handle the sync version of Django handles the connection in a very different way.

It like uh I think threads context was and stuff like that.

So it like I think it uh when you do not use polling it closes after request.

I don't know for for fully uh is uh if that the case but I could not find anything uh for async that we like we were discussing before this podcast and so I will try the async signal thing on that.

You're like yeah I'm gonna take this on I'm gonna do it and then you discover the massive locker and you're like right I'm gonna fix that as well and you just keep going.

So I mean for me looking through the the the the Rust front end for the the request response plan I think that's a really exciting um uh development diff looking at the different serializers we talked about that on the show before but like rest framework serializers are kind of last generation and there's catters there's pydantic there's message spot there's all these um much quicker options out there and to so to demonstrate those with Django is like a real service to the community it's like yes look this is how you do it and look how much faster

they are wow and then for me the the um async properly it fully asynchronous the ORM is like the last piece of the async story you know if we go all the way down to the database connector then you know whatever you want to do with async whether you want to use it or not it's it's the story's then at that point finished so like you're really hitting kind of all the all the tick boxes of the the things I'd love to see.

Maybe it's an application that won't scale or an AI initiative that just isn't delivering but that's where Six Feet Up comes in.

With deep expertise in Python Django and AI they solve hard software problems modernize platforms and get teams to market faster.

Their impact speaks for itself automating healthcare pipelines for hospitals helping NASA explore Pluto building severe weather prediction tools and applying AI to connect farmers with vital crop data.

When the stakes are high and the problems are hard six feet up is the partner that delivers see what's possible at six feet up the partner that delivers

kind of stuff so uh that was the whole kind of process if like we if we have ORM that is full async that will make the ORM very fast because there is like a sync to async part and the whole stuff it is it it I when I was using uh I was trying to handle sync in Django bolt um the performance drop for sync to async part

like becomes very like stark at that if you are getting like some even if you don't use the ORM the performance difference uh if you are getting 22000 rps and you want to execute the synchronous views you like um create uh like a wrapper using sync to async so on the above the performance difference drops like 22 to drag like uh 101200

0 so there is like a 50% drop there because of the thread part of the whole thing so if we have the whole ORM async I think that will be that will I think um be the performance difference will become I think as talk like a 50% or something like that.

Is that where the I mean because obviously there's a cost to um spin up the extra thread but is it is it the fact that the the Python threads aren't able to um execute in parallel um that's causing a lot of the blockage there?

Do you know?

I mean you may not know you may not have been able to profile.

So actis is a one worker and then the gill part.

So if we have something that execute using thread it kind of blocks a gill for some reason.

So this is also a gill part and also a latency part of like uh spinning up a new thread for handling that request I think.

Like it's for some reason uh I think it is not kind of ready or something the performance difference like decreases and perform it like decreases a lot like not like normal decrease it decreases a lot.

I tried it with like I think fast API V and like Django port the performance difference decreases a lot.

I think there is some issues or like I was not able to and also because um the PyO3 it does not like it has different like for it is a it has a different method of handling uh the non-threading python.

So I was not using the whole uh things that it provides to handle like non-threading python.

So I think that also causes an issue.

So I was not able to like properly benchmark that

Okay that's I I think all that's really interesting.

Will jump in before we disappear down the uh the Yeah

like for email marketing people come in and validate their email like is this is a valid email this is a invalid email this is a catch-all email and uh uh that like people take that result and do email marketing with that.

So he knew that he it was his idea and stuff.

So I was just the like kind of CTO part of it.

So I I I was gonna build and like we have like a partnership kind of thing there.

So that we

That's Carlton's setup right now at his starting

So what what's your process?

How do you think about them

When like I started there was no LLM, but I still I did not enjoy typing the code.

So LLM part solves that part for me.

So I use cloud code um so it depends on like what I am trying to do.

So I usually start with plan mode uh uh for cloud code.

Like if I am for example, like if we uh discuss like Django Turbo.

So what I I will do with that is I I will start with plan mode, it does not produce any code with that plan mode, it just uh produces a markdown file.

So I will go, I will discuss the whole plan with it.

So if I wanted to build like a async ORM, so I will discuss uh like because I did not know.

So I I I tell him like this is the repository, the Django backend, and this is uh uh what I want to build.

It like researches the whole thing, and uh I usually have like a cloned Django repository in in the project folder.

So I say um like uh this is the Django repository, like tell me like how can I achieve that thing.

So it like goes all in and like at the end it produces a plan like what we want to achieve.

I read that and if like that makes sense or if I want to change something, because like once you enter, like if you accept the plan and you want to change something, you they forget that like they have added a function or class, and they are gonna just like create a new thing, so you will have a lot of code that that has no like purpose in there.

So that is kind of an annoying part.

So I usually spend a lot of time planning and stuff.

Um I I research if like um I I'm not sure if like the LLM is telling something, so I I will go and uh th like read my code myself and like tell it like you are thinking it is wrong or you're like uh your hypothesis is wrong, yeah, stuff like that.

So uh that is kind of the whole process.

So after that, like testing is a whole another thing with LLMs.

You like it is I think it is impossible to produce like correct test with LLMs because they think like and you are like they don't think like we humans do, so like um uh their tests are like made up things.

They are gonna test what like they are gonna say it is working, but I can see it is not working.

So the testing part is the whole another thing.

I I usually like tell it to like test, but if you remove that git commit, the test should fail.

Like, so that I followed from Django.

So uh uh like Django, I think uh um uh I forgot her name.

Natalia Natalia Natalia, yeah.

He said, like if you create a test, the test should fail without the fix.

So I I took that thing from that, so I I like put tell LLM to like test should fail, but it still produces a lot of bogus tests.

So that is a very it

When I will like I I will say like create a failing test for this case, it will create a test that is passing, but in the like in the in the test it will have a if condition like it is failing, but it will pass the test for some reason.

So they are very afraid of like failing tests.

So I think their OCD kind of personality they have, they don't want to have a failing test at all.

That's really interesting.

You're you're I think you're a bit I I know you're a better user of them than I am, but you have to plan with them in advance.

And I mean, so one of my colleagues, uh Paul Everett, he's been doing a bunch on spec driven development.

And so basically what you described, like having talking through, planning things out, and I I suppose you're too young to know, but the joke is you could never get a human, like a senior software architect, to write specs for something.

But now that the LLM will do it for you, everyone suddenly is excited about specs, whereas they never were excited about specs before.

So it's forcing maybe good behavior on humans.

So there's an irony in there for those of us who are a little bit older.

Um, but I completely agree.

That's yeah, I I wonder if there's if we're gonna have different because they're so eager to write code, so eager to pass things.

I don't know if it's a filter or it's a different retraining, but it feels like they do need to be harsher, at least in a coding context, right?

It would I wonder, you know, yeah, to can the companies get there?

Can we get there with better prompts?

Is can some third-party package come in with a whole thing of like, you know, how do we get there?

Because I agree, it's a big problem, especially around Tus.

And uh I I tweeted like uh some months some like a few weeks ago at Claude.

Like it for some reason it is like very eager to write code, like it is very eager to produce a hypothesis, even if like it is a wrong hypothesis, like it is very eager to uh say, Oh, I found it.

But it it it it like it the whole in whole conversation, like it will be say two and three times I found it, but it never found it, like it it just say I found it, but like it it will create a like wrong hypothesis and like start working on it, and after that it will oh I did this thing wrong, I but it will forgo to revert to the changes that it made because of that wrong hypothesis.

So there is a lot of code that is like uh just uh uh like unowned code, so that is just a junk that kind of leaves there.

Um I know from at at JetBrains and our our AI AI teams work, a lot of times the benchmark is user-accepted completion rate.

So they want their they want you to say, oh, that looks good and move on.

And so I suspect that writing more code or writing things that pass even though it's not good is better than them.

Like they're not incentivized to say, don't do that, right?

I think you know that sounds simple, but I really do think it's it's almost like I can see like we say, oh, what's our you know, completion rate, user acceptance rate, whatever, for Juni versus Claud Code versus Copilot.

Like that's that's the benchmark that internal AI teams are using, which would lead to that behavior.

Like even if that feature is not complete or not working, it is just gonna pass the test which is working.

Hey, like come congratulations.

Like if you get a point for adding code, you should get two points for removing code, right?

I want that.

I have not tried it uh like very significantly, but it is kind of does that.

Like it it is gonna see the commit that you made for the solution, it is gonna go that go over all the changes and it is it is gonna simplify the code and like uh try to remove if like it thinks it is it does not belong.

I think that is a step in the right direction.

I think the difference that I found I find in a human code and an LM code it's uh also like a viral uh uh tweet.

The human code is beautiful, like everything that it produces, it looks like it is like it was designed to do that thing.

The LLM code is like a kid coding, it is just gonna solve the thing, like it is not gonna see like if I'm putting that function there, why I'm putting there, like it should belong to that file or not.

The a lot of comments that it adds.

So like beautification of the beautiful code, like uh if you think like a coding as an art, like that is human.

And if you think like uh NLM coding that is like um uh kid coding, like it is gonna solve the task, it like it it it is gonna solve like solve the issue, but like uh the code will be everywhere and like stuff like that.

Um the difference you we can see if like we see a professional code base like Django and a solution provided by an LLM.

So the difference is very clear there.

Um, that code simplifying thing from Claude, I know I think it was only four or five days ago that that came out, but that's that's really interesting.

So, my question is how do you stay informed about all this?

I mean, because Carlton and I like we we pass tips on like RSS feed readers because we love to see like blog posts.

And I feel like we're multiple generations back.

Like, are you on so like what are your news sources to stay up to date on?

That is that is a very good source.

And like on I think Twitter part of social media is very eager for wipe coding.

Mastodone is like 180 degrees of that.

They don't want like AI in their code.

So that's

Uh he also did uh like poll.

Uh the Mastodon people and the Twitter people, the poll is like uh a 180 degree.

Like Twitter, there is like ten less than 10% of code, like LLM write and in Mastodon, like there is like few um I think larger than 50% code I write myself.

So there is a difference.

There is like the extreme kind of like uh two things.

I I kind of believe that like both uh like any extreme I don't know the word for that, like any extreme what is the word like that?

Um like any extreme thought a person have, like uh any extreme opinion, sorry.

Any any extreme opinion a person have, like extreme opinions are never right.

So the truth will always be like some something in like in between.

So if like coding changes, it will be like kind of 50% human, 50% LMs and stuff like that.

It seems to me they're less good when you're trying to push the boundaries into something that's not directly in the training data that's not that's new, right?

And you seem to be doing a lot of things that are new.

So how do you how how do you find I mean does it is it that the planning mode, spending enough time in the planning mode helps to pin it down?

I ask for like if something is not working, I ask for like options, like give me options to solve this thing.

So it is gonna say like it is gonna be an uh an extreme option there.

It is gonna be like something that like that I think like can work.

So I I will say try that.

That is also I think a very good point for LLM because we can try a lot of options because the the price of code is very cheap that way.

So if like that thing does not work, I can I can just like uh discard the those whole changes.

So I try different options.

Uh the it there is uh there is like two extreme.

Uh one time you are gonna say like LLM is oh my LLM this solution was very intelligent and sometime it is gonna be stupider than a dog.

The solution is very like stupid.

It thought something else and produced something else.

So there is like extreme cases there.

So I think we can never like predict LLM stuff that makes them dangerous, kind of that way.

So there will always be human for coding.

I think they will be less human, but they will always be human somewhere.

But so for me, like if I don't type something, this is my biggest problem with LLMs.

If I don't type it, it's not I'm missing a pathway in my brain.

So I'll have LLM code.

And it part of it is I didn't fully think it through, but I think a lot of it is I just didn't type it.

Um and so you're making me wonder maybe that's just a step leap that the newer generation of people, you know, won't have that problem.

Like they'll just be like, oh, this this this is normal, this is how I I do things.

Um because I don't know, Carlton, is that the case for you?

For me, if I don't physically type it, it doesn't quite get in there, but but that's because that's how I learned it, learned how to code originally.

You were gonna sweep.

Like I don't understand it fully.

Uh so the difference kind of becomes you have to be a good reader of the code.

If you like read it and some like and you will have to because sometimes like because I I was new to Rust, so sometimes it produces uh like keywords or something like that that I don't understand.

So there is like a skill to have, like uh, you should have a kind of a way to like you want to learn.

So if it uses RW log, you don't know rw log, you should ask it to like what is rw log.

If you just become a like uh um just enter enter enter like everything except like it just works and stuff like that.

So that um you will not never like understand like what it it is doing.

There is a thing like if you code it yourself, you understand it better.

When I was working for JSOC, I I was very aware of that thing.

So I usually if I I I I used LLM, I used it for uh like small commits.

Uh there is a mode in cloud code where you like you don't accept changes, it it just shows you that commit.

So if uh if like I uh see that commit, I will say like this this is not Django standard.

So rewrite it this way, or like if I wanted to do it myself, I will I I'm gonna rewrite it that way.

So if you want to understand the code, it is uh uh like um uh a requirement that um you will have to read it and if you type it, then like you will be a better understander than uh just a simple person that uses LLM to just push everything.

They might be able to produce something very quickly, but you know, m code is more maintenance than it is production.

And then at the other end, I'm I'm as much as it might pay me just to think, I I don't think we can handcraft every character anymore because of the echo just the economics of it aren't gonna make any sense.

And so between those two extremes, that the the it's gotta settle somewhere, and maybe there are a range of of sustainable positions that that an individual developer could take up.

But I mean how how you find you know how are you finding in your freelance work that you have to you have to balance these things?

So if like a person comes, most of the people come with a like kind of an MVP of the project.

So I want earth, I want um like stuff like that, that is already there, like solved kind of things.

So LLM is very good at those.

Like if you uh uh tell it to do earth in Django, it will like does it like 100% correctly, kind of so in freelance work if like problem is something interesting or something complicated, that kind of like uh uh a recent project that I am doing, it is a like uh software for ships.

Like um uh there is uh uh like a task is assigned to people on ships like uh like ocean ships, kind of that.

So um it does not get it, like it does not understand it, like if a task is like the the whole thing is that it does not understand world like AI companies say it does, like it does not.

So if like it it creates a task, it like makes wrong hypothesis about the real world, stuff like that.

So that way you have to guide it, like you like uh if a delivery comes, uh a person have to accept that delivery.

It is gonna put the all the buttons on the top to accept delivery, accept, accept, accept.

I uh I I I saw that because I built that.

It's like if a delivery comes, I have to see the pro product and like uh receive that delivery because that's how the real world works.

But it created like every button on the top to accept delivery and it like it asks nothing, like uh like what is the delivery ID, what is the like there is a whole process, there is a difference the whole like how the real world works and how LLM thinks it works because I think the this this is the whole context problem also because if it is working on a small solution, it kind of forgets a lot of how the other things work or something like that.

So it it has a narrow focus kind of that way.

So there is uh difference if if a like a problem is uh kind of real, like so we I will have to like explicitly say that like uh you don't want to do that, you you have to do this like in this way.

So that's the kind of the difference.

Like, is there a point where it um and actually that's uh I'm gonna pose you my philosophical questions to get your take.

Uh are you familiar with the term model collapse?

Have you seen this idea out there?

Um I can describe it.

It's it's basically that um as the data goes from being human generated to being computer synthetic data uh generated.

Um there's there was an academic paper out of Oxford uh 2024 showing that over time if you train on synthetic data, the LLMs get worse, right?

So in my mind, I think of it as we know the underlying data is kind of getting worse.

Like it's AI stuff, even as the models are somehow getting better.

So, but where does that all end up?

It feels to me like maybe the whole foundation is crumbling beneath us.

Um but I'm curious what your take is on that.

I feel like you're much more in tune with these tools.

Uh so

Uh it is also becoming a race for the benchmarks that they have set up.

Like it is like they want to optimize for those benchmarks.

So if like you compare a GPT 5 point something to cloud code, even if like cloud code is better, but like GPT is gonna be like uh higher on the benchmark or some something like that.

And uh the data problem is real, like that is also the I think that data problem and the architecture problem because like you can say it is AGI, but it is just a probabilistic like um uh prediction of the next word.

You can say it is more intelligent, but it will never be, it is just a prediction of the next word.

But like the like people who are like um more intelligent than me, but like they're gonna say like your word, uh your brain also predict the next word, but it does not like uh we cannot explain it like what our brain does, but it does not predict the next word.

Because if I like if I walk on a like an icy road or something, I know I'm gonna fall because I have like fallen when I was four or something.

So the context of that 20 years is there with me.

So LLM the like I agree with that uh your statement like 100% the uh like uh artificial data part and like the optimizing for the benchmark part, like it is gonna be it is never gonna be like um more intelligent than uh like this is my opinion.

Uh it is not gonna be more intelligent than human in like general tasks.

You can train it for a specific task in specific environment and specific like in a box, it is gonna be more intelligent than a human, but if you break that box and uh everything is gonna fall like stuff like that.

Carlton, you look like

You talked about lack of ideas of causation, lack of um decent uh of proper m memory, lack of world models.

These are the stuff look like the the criticisms, and the the LLM architecture has gone a long way and they are truly amazing what they can do.

But you know, uh if nothing substantive changes, then it looks okay.

That's a very clever trick and it's a very useful trick, but it's not AGI as you know was promised to us by now by Lord Order.

Um I do I think you you're right what you were saying though about the the benchmarks.

I mean, this is something that uh uh JetBrains, again, I'm just gonna read it out.

They have a developer productivity AI arena, DP AI.

So they're trying to do an an open standard because right now there are these kind of closed standards for the major models, and it just leads to, you know, optimizing for the standard, right?

When it's like, well, who set the standard?

What is the standard, right?

But it's like as complex and intelligent as these things are, it's sort of like we have we all have narrow incentives as humans and as AIs.

I think okay, they're interesting, you know, but I don't feel that my w um career is under threat.

You're sort of at the other end of your career.

How do you feel with about them?

So if I am like if I am trying to get a job in a small city like where I live, but um I like there is a fear, like in short, there is a fear, like uh the whole thing is gonna change and uh uh they uh like there will be less jobs, but the uh the thing that is like I am hope

ful, uh like because like I like to learn and like to know stuff, so only those people who kind of have that uh like there are some people who are gonna do computer science just for the job.

So there is a difference between uh like a programmer who wants to do is for like because like they love programming, and there is a difference because he like someone said it to like computer science is a good like job or good pay and stuff like that.

So I think there will be less programmers, but we don't know because the costs now are very optimized and stuff like that.

Uh we don't know how much the LLM actually costs because like not everybody is gonna be able to like uh use them if like uh something like not a Doomer, but like something the bubble crash or something like that.

So uh if the cost is gonna be the same or something like that.

So there is a fear in like normal computer science student, like there there isn't gonna be no jobs or less jobs.

Is that fair to say, or what is your current employment status?

Uh now I am just uh doing freelance stuff.

Uh I'm open to freelance contract or job.

Anything works for me.

Having worked with Fahan over Google Summer Code and seen him in action since he is, you know, he's a good he's a good find.

You should snap him up.

Thank you.

I guess if somebody does want to get in touch, is I guess email on your personal site, uh, LinkedIn.

What what is the best way for someone to contact you?

There is also email on the LinkedIn.

So like both options are fine, LinkedIn or email.

I haven't asked you about you know static typing and your thoughts and whether in an LLM world everything is pushed that way, but um I guess I don't know.

Is there's I I always like to ask, uh wait Carlton, are we doing are we doing book reviews?

Oh I'm

Wow.

Um it's Django REPL, which um

Built on is it built it PyDiere?

Pyrodite, okay.

Okay, so forhand, I've really, really enjoyed chatting with you and working with you, you know, over the last year.

And it I'm so excited for all the things you're doing um in the Django ecosystem.

So thanks for coming on the show.

Thank you for inviting me.

Bye bye.

Bye bye.

From scaling applications to simplifying data complexity and unlocking AI outcomes, they help you move forward faster.

See what's possible at sixfeetup.com.