Transcript: AI in the Real World - Marlene Mhangami & Tim Allen

More in their services later in the show.

I'm Carlton Gibson, joined us by Will Vincent.

Hello, Will.

Hey Carlton.

Hello, Will.

Can you give them a quick intro?

Yeah.

Hi, everyone.

Um, but yeah, my name's Marlene.

Uh I'm a senior developer advocate.

Uh like Carlton said, I currently work as at Microsoft.

I focus on Python and AI.

So I'm on the Python on Azure team.

Um, so I'm doing a lot with AI right now.

I think most of the Python code I'm writing, including the Django stuff, is is AI.

So yeah, that's that's me.

I'm also very involved in the Python community.

I was on the board, the PSAF board for a couple of years.

Um, was one of the co-founders of PyCon Africa, so I love that.

Um, so love the Python community, love Django as well.

So yeah.

And Tim, you're a you're a Wharton, right?

Wharton Business School.

Um, we actually work with Harvard Stanford MIT more than Wharton faculty on my actual job.

The platform we run, um, we have several petabytes of data, of finance data that we store that uh the business schools across the world come to our platform to do academic research on.

Um, so it's a pretty cool job.

I mean, I'm lucky that I get to build cool things and work on solving interesting problems.

So um I still get to write a fair amount of code too, which is uh which is pretty nice.

I mean, how do you manage to stay involved?

Um that's one thing I've never understood about the tech industry fully, is so often the career path for your top engineering talent, which has been the rarest, hardest to find talent over the past couple of decades, has been, oh, you want to progress in your career, become a mediocre manager and do what you really don't want to do.

So it never made it no triangle,

You rise, you rise to your level of incompetence, and there you stay.

That's how it that's how that's how I've heard it phrased.

Well, I do want to mention, so this this came about in part because you both gave excellent talks at DjangoCon US, which Carlton, sorry you weren't at.

I think this is a DjangoCon US, yeah, sure.

Oh, very nice.

But broadly speaking, there was discussion about AI, and I know Carlton, we're not making this an AI podcast, but both of you had slightly different takes on AI.

Marlene, you gave a keynote.

Tim, you gave a talk that touched upon it.

Um I guess I'll just start with a question, which is in your day-to-day jobs, how are each of you using AI?

You know, not the marketing type, but like actually like day-to-day.

Uh either one of you.

Um and yeah, like Royal, you mentioned, we I gave a talk at DjangoCon about

Yep.

And uh was a little nervous because it's like AI, not everyone loves it.

But uh definitely I would say in my own work, I I tend to use AI quite a lot.

Um right now, I would say the primary way that I'm using AI uh uh in two different ways.

So I think with developer advocacy, I'm doing a lot of writing, for example.

And so typically what I'll do is I'll have, you know, if there's a topic that I'm supposed to write about, I'll have AI go and get all of the documents, or I'll I'll paste in some documents and then have it summarize the documents to me.

And then I'll like use that as a starting point for a draft blog or something like that.

Um, another thing is that I'm writing a lot of code.

So right now, for example, I'm maintaining the Azure integrations in Langchain, and I'm doing a bunch of Python code for that.

And um and I I don't yet feel comfortable having AI like write a full PR for me to for me to merge into those libraries.

But what I do have AI do is, for example, run linting for me.

So I don't know why whenever I do linting, there's always like something that tends to be missed.

Um so I have something where it just automatically runs the linting for me, um, which I think is quite nice.

And I also have it sometimes if I'm having trouble with a bug, have it figure out where in the code the bug is, and then I use that to as a starting point as well to debug.

So yeah, those are the primary ways I would say I'm using, I'm using VS Code and Copilot, by the way.

That's a plug for Microsoft.

But yeah, that's what I'm doing.

much every day.

So I still write a fair amount of code, but I found as I become a more and more senior engineer, I'm the first principal engineer in the history of the university.

So this sort of new career path um was sort of developed with me in mind, which was, you know, a really big honor.

But I find the actual time I'm pressing buttons writing code is getting to be less and less.

And these days I also find that my best PRs often delete more code than they create.

And I like to say, you know, while the LLM is very useful for those, you know, the 10 to 15% of the time I'm actually pushing buttons in the IDE, um, the most value my employer actually gets from me isn't Monday through Friday, nine to five.

It's when I'm doing two things one, sleeping, and two, showering.

Because that's where I come up with the solutions to the really hard problems I work on.

You know, there's

Or I get out of the shower and the solution to that bug is just there.

Um, another part of my job that I really enjoy on my day-to-day activities is working on projects with junior engineers.

And sometimes an LLM can be involved in moving that along.

So, you know, when you're pairing up with somebody, you're not sitting there, so you know, one person isn't sitting there for 15 minutes while the other person is looking through the documentation, um, trying to find that thing you know is there somewhere, but you can't quite find it because every documentation setup is slightly different.

So um, I mean, that's another thing.

Teaching is something that is that I found is still always fulfilling that I never get bored of.

So um, you know, having an LLM integrated in that does make it a bit more efficient, but I don't think it's any kind of magic wand for my day-to-day activities.

Um, and yeah, that's sort of a summary of how how it's, you know, it's another tool in my in my toolbox.

I mean, I know you play with everything.

I like to my my usage is very much uh what I call the Stack Overflow use case.

Where I, you know, I found last year or so where I would previously have gone to Stack Overflow, I'm happy to type it into Claude or type it into Copilot, and the answers are for me about as good.

They're they're sort of like, yeah, okay.

Um, and probably better than Stack Overflow, because otherwise I wouldn't have replaced them in that I don't have to spend quite so long reading the number of threads.

I get a concise good answer straight away.

And so for me that they're super.

What I'm not doing with them is writing code.

Um I might I might you know ask for an explanation.

How does this work in JavaScript?

Or how does this work in playwright?

And it'll tell me, and then okay, I'm still gonna write the code myself though.

No.

Um I've got I what do I use for autocomplete?

I use um still use um what they call LSP-based um uh autocompletion, which is um uh deterministic and correct and um uh perfect for me, in fact.

Um like the next line suggestion stuff that comes up, I'm like, I don't want that.

I just I literally don't want that.

Um as well, here's here's something you've got to remember.

I have um snippets for every um code construction.

So if I'm writing a loop, I have a snippet for the loop with placeholders that I tab between and things like that.

I'm never typing for variable name in loop name, you know, I I'm tabbing between um the placeholders very rapidly.

Um but that's totally pre-LLM, it's totally static, it's yeah, you know, so so the the the the use case for autocomplete just doesn't come up for me um in my use.

You know, for years, people have talked about getting into flow as a programmer and hitting that flow state.

And I found that when the LLM sort of interjects itself, it's jarring, it gets in the way, it gets in it gets in the way of my thought process.

So I too, I like having the chat agent on the side, but for the autocomplete or the think ahead type stuff it tries to do, I just find that really um impedes my progress and uh, you know, breaks chain of thought.

I mean, and I think we're all sort of coalescing around, and this is one thing that came out of DjangoCon US is it's a it's a tool.

It's a tool that has benefits.

I don't think anyone feels like we're immediately going to be replaced.

Um, and it's imperfect for sure, but I also I was thinking, you know, the the old days of Stack Overflow and hunting around for a Google blog post and just being stuck, stuck.

It's not that that was perfect either.

Um so, you know, I guess like the three of you, I use it a lot for discovery, for research.

I also find the autocomplete aggressive and also just annoying.

Um, but I'm using, yeah, the chat interface all the time for research or to like hunt down a bug.

And increasingly I'm using agents on the command line, um, but more for boilerplate greenfield stuff.

And still I haven't quite, I still don't fully sit back.

And even when I sit back, it's almost like I I sort of miss the days of like fully doing compiled code.

But when you sit there and just like it's worrying away, like what is that feeling?

Like I sort of I just get like existential waiting three seconds, 20 seconds, and then I have to evaluate.

Like, I'd I'd like to be a little more leaning in.

So for myself, it's much more of like a research tool that's a great research tool.

Um, but the fully agentic spinning up six parallel agents and you know, having a coffee and coming back and it's done.

I've never quite had that.

Sorry, Mali.

You know, once

Yeah.

But yeah, it's well, but I think, I mean, the thing I've come back to is, I mean, again, we're all in the Python space.

So if you're doing Python stuff and Django stuff, you get almost as good a result as you're gonna get because Python and Django is so mature, so well documented, right?

If you're using a newer programming language or a smaller programming language or framework, it doesn't work anywhere as well.

So like this is tip of the spear, what we're all doing.

Um and even then, you know, on the one hand, I'm like, oh, it's like I still have to code, but it's also this amazing research discovery thing and finding bugs.

I mean, I remember spending hours, if not days, on some little thing that I and I couldn't find the right post.

And now the LLM, even if it doesn't get it right right away, it'll, you know, I mean like try harder, you know, and it'll try harder and find stuff, right?

It just feels like a speeding up of that, but not a total replacement of code.

And yeah, and I still I still like elegant code.

These things don't write elegant code, even if you have rules or guidelines, like it's incredibly verbose and you know, it just vomits up code rather than like concise code.

I've even tried, you know, write it in the style of you know, Carlton Gibson, like go crawl his GitHub repos, and you know, that helps a little bit, but it's still not a total replacement for you, Carlton.

Cowboy coding, right?

You put your cowboy hat on when you code Carlton.

Go on more.

And so I've been kind of looking at the comments on co- on like VS Codes, uh like social medias and things like that.

And one of the number one complaints people have uh on social media is that they just feel like the autocomplete is not fast enough, or it's not like they come like a lot of people compare it to cursor and and say that they think that you know, in VS Code it's too slow or it's not doing enough, and they just want to tab, tab, tab, sort of is the vibe.

And so it's so interesting to me that actually all three of you think that the auto complete is usually too aggressive or it's um there's too much going on there.

And I think it also shows kind of there's like a bit of a discrepancy in terms of like people's comfort levels with with this, in terms of I think certain groups are like pushing the boundaries and want to be like really at the edge there.

And even Will, you mentioned not wanting to leave agents.

And I was talking to Armin, who's like the creator of Flask, and he was like, Here's

the 90%, right?

And and he's he's a great programmer.

And then on the other hand, you know, you I I also, for example, even when I was get first getting started with the autocomplete, felt like it was too aggressive.

But over time, I think I just started to get used to it and that kind of wore off.

So I'm I'm now kind of using it quite a lot.

Um, but yeah, I think the discrepancy there sometimes is so interesting to me and and kind of different.

Um, so yeah.

Like he has a custom something or other for YOLO mode where he just like lets it,

you know.

So you have you he's spent a lot of time to fine-tune this.

And I think I think that's part of it is not just playing around but fine-tuning it.

Because if if you just try to one-shot something, of course it's gonna be.

I mean, this is the main thing I have.

When people say, like, oh, it didn't work, whatever tool they're using, Juni, Claude, you know, copilot.

It's like if you just ask it a simple one-sentence prompt, how could you get something good, right?

It's like the whole the joke is like you kind of have to use it to write a spec.

You'd never write a spec for a fellow human, right, Tim and your team.

But if the machine will do it for you, you'll write the spec.

So I do feel like you need to kind of like play with it more and to get to a point where you can fully evaluate it.

But that said, I still am shocked that people, and there are some like like Armin, who knows how to write code, is fully comfortable and happy and sped up um, you know, doing that.

So yeah, it makes me feel like, what what what what's going on?

What's happening?

Yeah, wait, wait, wait,

where's the

is a

Um you know, I still prefer chat mode to agent mode.

Agent mode feels far too much like autopilot to me.

And when I've gone down that rabbit hole, I've ended up with sort of a maelstrom of nonsense a couple times.

Yeah.

Um, and you know, studies have started to show that um if you use LLMs too extensively, it does add a lot of technical debt.

I recently, so I haven't owned a car in a dozen years.

I just recently bought one, and um, the state of software on cars is absolutely miserable.

Um CarPlay.

CarPlay.

CarPlay is actually an

Because for me, every line of code I write, I consider reliability.

Every line of code that I put out is a potential security flaw.

So when I purchased my new car, I actually did research on what vehicle I could buy that had the least lines of code.

And I ended up with a 2022 Toyota Corolla with only one screen.

It doesn't have blinking lights all over the place, it doesn't have sensors all over the place.

There was an interesting study that came out of Ford.

Um, Ford has 150 different software vendors that write the software together for their vehicles that they try to put together into one working package.

And of course, car repairs for software are just as common as car repairs for anything that's actually wrong with the engine now.

Yeah.

And I think we're gonna see this problem continue to grow over the years.

You know, we're starting to see people whose actual job is fixing your LLM coded mess.

Like if people are marketing themselves as I will come in and fix your VOD-coded nonsense.

They will, they likely will.

Like he's he's you know, he's figured out how to make it work.

And I think it's a combination of knowing, of being used, to your point, Tim, of being used to reviewing PRs.

So you're kind of in that manager, whatever, higher level mindset, anyways.

But um it's just a it's just a it's not a paradigm shift, but it's a shift to go from just evaluating someone else's thoughts to like thinking the thoughts yourself and then writing them.

Yeah.

I try not to be all calcified about it, right?

Like I use the chat and the research every day for two years, but the agents and I still play with it, and I'll use it for like fun projects and it mostly works, but then I just get, you know, then the debt builds up or I I've lost control of a mental model of what's happening.

And um, you know, but it's weird because you can, you can, you know, we anthropomorphize these models, right?

You can sort of treat it like a person, right?

Like it'll do something and then say, like, are you sure?

Be like, okay, imagine that like I don't trust you.

Like you can do all these sort

of tricks, tricks, and it

To to be like, you know, how sure are you?

You know, and and it they're getting better about being a little less uh syncophatic, um, but they're still, you know, they're not, they're not to your point, Tim, they're not gonna like take out code as much, right?

They're they're more like they wanna they want to generate stuff for you.

Hacksoft is your development partner beyond code.

From custom software development to consulting, team augmentation, or opening an office in Bulgaria, they're ready to take your project to the next level.

And this is one of the things that I mentioned as well in my talk is that I think the best way to I agree completely that the technical debt thing is is is a thing.

And um, and I think more people are struggling with their code.

They have vibe coded and not knowing what to do with that.

Um, and I really do think that's a that's a real issue.

My my thoughts there, or what has worked for me when I'm using uh like agent mode to help me with coding tasks is I really use it in a modular way.

So I don't have it generate unless it's something like a completely green field, where it's like a prototype or something like that, or I, you know, it's just creating the out the structure of it or something.

Usually what I will do is I'll give it one file and I'll say, This is the issue with this file.

I want XYZ done in this file, or I see this bug in this file.

Here's all of this context.

And if you treat it in a modular way, where you know exactly what the goal is for this problem, and you give it all the context that it needs, I actually think you can be super helpful in those cases when it's a modular uh case.

Um, when you're just generating these larger, bigger projects and just having it go off, I would say those are a little bit harder to control, and that's where I think the the technical debt comes in.

But I I think there's other ways to use it.

I I primarily use agent mode, and I found that for me, agent mode is is is better because the model usually has access to the logs and and things like that and the whole structure a bit better when it's in agent mode, but I will restrict it and say, even though you have all of this, only you know, write code, change this file, or or do things like that.

So giving it restrictions.

Um and but but usually I will I will put it in the chat.

So usually I will give it the context that it needs.

And I found that that's that that works for me.

Um because I'm not usually trying to refactor the entire code base, at least not in the problems that I'm I'm working on.

Um for me, it's usually adding a feature or debugging an issue one at a time.

And so that's where I I would use it.

Um, I started to think about what are the biggest productivity boosts.

So I started writing code when I was six years old.

So I've been writing code for a long time.

I was trying to think of what are the biggest productivity boosts I've seen as a developer over, you know, over four decades.

And you know, Stack Overflow sort of coming about in the 2000s and 2010s was definitely a big boost to my productivity.

Having sort of

the the amalgamation, the coming together of all the development knowledge on a single source that I could rely on was.

But I think the biggest productivity boost I ever got as a developer was Windows 98.

Yes.

That was truly a game changer for my development productivity, let me tell you.

Having like browser and Emacs up at the same time, I'm dating myself, but that was in danger.

And I think that the thing that these tools are amazing for is you know, if if I remember when I was learning, got stuck, I'm missing a semicolon or something, and I'm at home by myself, and I had

nobody to

One of these machines would have spotted that and told me and gone, look, this isn't working.

Why isn't this working?

It's not working because, and that would have been an amazing boost.

I think you know, again, Stack Overflow, a lot of people learnt by copying and pasting from Stack Overflow, and there was all this kind of programmer superiority about, oh, you mustn't copy and paste from Stack Overflow, but a whole generation of people learnt by copying and pasting from Stack Overflow.

Well, nothing wrong with that.

More power to them.

And I think the same here from LLMs.

I think if you're a junior by yourself, you haven't, you know, who hasn't got a support there, who can a mentor there who can brainstorm it with you, to have the LLM go, here's the problem, that's going to let you go forward faster.

I think for me, that use case is very interesting.

We can open up an awful lot of doors with these, I don't know, what are they?

Uh magic lanterns that speak, you know, speak kind of the truth.

So yeah, like days, days on a semicolon, and maybe newer IDEs would fix it, but that feeling of just helplessness, like that's that's gone.

Um I think

I'm like, you know, why isn't my why aren't my dick why aren't my keys for my object being right?

Oh, because they're key, literal keys, and you need a evaluated key.

Oh, putting it.

Uh thank you, LLM, for solving my problems.

Yeah, it's not just about being a beginning.

Yes.

Yeah.

And I was there with my own devices and a book of Borland Turbo Pascal.

When I found that semicolon after three hours, I'd learned a lot more about my code base.

I'd learned a lot more general knowledge from going through the book and learning about, you know, how you should write code and how to avoid these problems in the future.

I'd learned a lot about debugging and the feeling of accomplishment you get after struggling like that.

Sheena O'Connell said something brilliant at last year's Django Con when she was teaching people during the tutorial during sprints.

And that was never steal someone's struggles from them.

Because that is the best way that people learn and how they come to love programming is that feeling of accomplishment after the struggle.

That journey is an important part of becoming a senior software engineer.

And uh I I think that part of that is is being taken away from a generation of people, and part that I truly love.

Yeah.

Yeah.

I just want to

pin that because the

I can say all that.

The the real reason why is I'm worried about losing my strength.

I'm worried about going to the gym and getting the machine to do the weight, the weights for me.

And you know, what happens in six months' time when I haven't really been coding the same way?

Have I still got the same the same strength?

So that's the flip side of the hard learning, is by doing the reps every day coding, you you keep your skills sharp.

Are we

situation where the entire developer industry is something like trying to find COBOL programmers now?

Yeah.

And and and if I'm being honest, I think that's my primary concern about LLMs in AI.

Um, my primary concern is about it is for junior developers and and building that that muscle.

And I actually was it's so interesting because I had a conversation, I was reading something recently where um, yeah, where there's been some studies done about how LLMs tend to select for or it's biased towards senior engineers, where even we talked about AMIN earlier and even the intuition needed to review what the agent has done, ETC, to know what to change, what not to change, is something that is

biased towards senior engineers that already have the experience.

And so if junior engineers don't have that struggle that Tim is talking about to be able to understand their code, how do they actually get to the point where they know how to interact even there with an LLM in a way that's productive?

I think that's a that's yeah, that's actually an industry-level concern that I think we need to be worried about as an industry.

Um, how do we actually solve that problem?

I do think that there needs to be some spaces where there's a joint collaboration.

So I think LOMs can be good, like Carlton said, for potentially like personalizing education and helping to make um it easier to ask questions.

So maybe if a junior developer doesn't know how to solve a specific task and maybe you give them room to struggle.

So I think we need to be thinking, rethinking how do we use what we have, these tools, to be able to create these environments where the junior engineers can still grow, can still learn, but we are preparing them for the future that is inevitably coming with the with the LLMs in general.

So I would say here on this point, I do, I agree.

I'm it's something I'm concerned about.

And I think we need to be thinking about this.

Um yeah.

I think if if the if the chat UI had like a kind of timer where you had to spend 10 minutes writing the prompts, right?

So if you've spent 10 actual minutes writing the prompt, then you can press enter and then send it to the LLM.

But if you haven't done that, then you have to spend the 10 minutes because that makes you think it's really that's a lot of rubber.

These LLMs are kind of ghosts of like where did the stuff come from?

And you know, not everyone has ex has access to you know ping Carlton when they're stuck on a bug, but it's it is I don't know if it's magic or what it is.

Like I was just watching Andre Skarpathy just had a a long interview on um was it Darkesh Partel's vodcast and some really interesting takeaways.

And and he was saying something I've thought about.

Like he's like, the internet is such garbage.

Like it's not, you know, it's not New York Times articles, like it's just complete garbage.

There's no, it doesn't make any sense how something can come out of this garbage.

And so on the one hand, like what's left there that has any like good meaning.

So stack overflow, like that, like let's take that as an example, right?

So you ask a question and then humans come in and give responses and vote up or down.

There's no, and it's not perfect, but it's pretty good.

Like there's nothing like that with these LLMs, and yet somehow they sort of kind of do it.

But like you can't trust it.

And also, why would any of us go on Stack Overflow now and try to get credibility or write an issue, do a good response, write a book, you know?

Like the new Django survey will be out by the time this comes out, you know.

People aren't reading books, they're not even reading blog posts, you know.

Like, look at like Adam Johnson.

If you just read Adam Johnson blog posts, you would be like a senior dev, but you know, what's he's just doing it because he wants to, right?

Like there's no

Like Stack Overflow is dead.

I used to get, you know, 30 points a day for my various responses out there.

Yeah.

I now get, you know, 30 points a month or something like that for upvotes to my old responses.

What happens if things change and there's no central model for the LLMs to steal their training from?

Um and you know, I'm looking at right now, all of these LLMs are being operated at a big loss.

So here we go.

I made $4 billion last year and spent $9 billion to make that $4 billion.

This is not sustainable and it's not gonna last forever.

And, you know, I referenced AI a bit in my talk.

A lot of my senior thesis written in the mid-1990s was about the dream of AI.

I am not a hater, but I've been around enough hype cycles and I've seen what these big tech companies have done to the internet and done to search engines, and shittifying their own products, that I truly think, you know, the same patterns that social media and big tech have brought us over the past few decades are sort of being repeated on steroids.

And they have not yet insidified the LLM.

What happens when they want to keep you on the LLM longer, like Google has, and starts give intentionally giving you worse results?

Some of the worst companies and the worst people in the world are the ones pulling these levers while genuflecting to the current US administration.

This is worrisome to me.

I can see the same exact pattern of what has happened to Google, to search engines, to social media, to them lobbying and not having any concerns for our children's mental health or futures, being amped up on steroids.

And I see the same sort of people.

I see I see Sam Altman and I see Mark Zuckerberg.

You know, um, I don't see much of a difference in the morals or the discussions coming out of them.

I see move fast and break things.

I see little concern for the mental health of people, for humanity or the future, and that worries me.

Um so I think we're currently at the peak of performance we're gonna see because the companies are gonna start intentionally inshidifying them to put profit over pulling people in.

Um, because that's a pattern we've seen over and over and over over the past couple of decades.

And I don't think, you know, LLMs are an exception to that.

Um I do think LLMs are not explored enough in terms of the good potential that could come out of them.

Do I think there's loads of bad things that are gonna come out of them yet?

I think we are seeing that already now.

I'm not sure if you saw even like, I don't know, Sam Altman shared some updates recently that are concerning.

Um that I think as well the intention is to continue to keep people using Chat GPT, for example.

And um, and you can look for that online.

I won't talk about it.

Um, but I think that um I also think that technology is always going to be this sort of double-edged sword where you have very good things and very bad things coming out of it.

So, you know, when I was at Django Con, I talked about how we haven't really explored to the extent that we could these open source models, these small language models that have huge potential to transform education.

Um, you know, I talked about the Django girls curriculum and how we could do something like Django Girls Offline because there's so many parts in Africa, for example, that are being left behind, and there's this growing digital divide that is consistently growing, and is and no one is doing anything about it, and it's not changing.

And like people just don't have the resources there to bridge that gap.

And the thing that has helped to sort of close this gap over time, technology is a huge part of that.

So, you know, I grew up in Zimbabwe, and a big part of why I learned how to code in Python is because I had access to an internet cafe where I could go and I could connect with people online who are like writing Django code and and who can teach me stuff.

And and that's because of technology.

And those advances, yes, can absolutely have huge ramifications, negative ramificate ramifications.

But also, you know, I think about the parts in Zimbabwe, for example, that don't have access to stable internet connection or internet is super expensive.

And thinking to myself, can I imagine a world where we bring in small language models that can teach people when they don't have an internet connection?

And what that means if someone now has access to all this information is huge.

And then also potentially very positive.

So that's my perspective.

I would agree to an extent, but also don't want to forget the other side as well.

I think I'd you know really positive.

So I saw that a similar article to you, Tim, was about um the economics of all this and the the sort of latest, well whether it's latest or or recent figures, they're spending three dollars to get one.

Well, that's not you know, it's three dollars of spending to to make one in revenue.

Well, that's not sustainable and nobody's going it seems like nobody's gonna pay three times as much for the same service because you can run a local model, which on your on, you know, you know, I've got a five-year-old laptop that runs a model that I can do 90% of what I do with the you know, the clauds and the chat GPTs or the co-pilots of the world.

It seems those local models might be, you know, the way forward.

You if you can, you know, there's an internet in a box project, which I think is, you know, has you you download Wikipedia, you download your PyPI mirror, you've got everything you need to run workshops without an internet connection.

Well, if they hadn't a local LLM on them as well, then you've got the complete the complete set.

Um I just I wonder what you think about local models and

you know

how that affects the um the economics of these these frontier the the anthropics and the um open AIs of the world.

And uh I remember Hitchhiker's Guide to the Galaxy.

They build the computer deep thought to find the answer to life, the universe, and everything, the ultimate question of the universe, and it comes back with uh the answer being 42.

Um but what new use is really is there in that?

And um, I think you're really onto something because it's been shown, you know, studies have started to come out that show that the smaller the scope, the higher the accuracy and utility of a language model.

And uh, you know, I could be completely wrong, but I think the far more useful future that you're um speaking about here could be small focused, tightly scoped language models.

Um, you know, not just RAG, but the entire language model being contained to a very specific scope.

I mean, imagine if our coding LLMs didn't have billions of unused parameter pathways for when someone wanted to make a cat picture, but was just focused really on, you know, the stack overflow training set.

You know, it would be a lot more efficient, it would be a lot more environmentally friendly to run.

It could run locally.

Um, and you know, maybe dividing these things up, having a thousand different models instead of one big God machine is the way to go because then it also can be run locally.

You know, maybe the race shouldn't be to a trillion parameters, maybe it should be to a million parameters.

Um, yeah, it's gonna be interesting to see.

That's you know, small focused, that's the term I've seen bandied around.

Um, but I I Carlton, I do want to can we mention the the comment you shared with me yesterday about the art heist at the Louvre in the context of all the

Because there is, I I keep coming back to the underlying there there.

Like what how how even if the technology keeps improving, how could it have good responses if the underlying content is garbage?

And as all the economic underpinnings for creating good content on Stack Overflow, on a book, on a blog post, even something as prosaic as like I want to buy, I'm going hiking, I want to buy a backpack.

In the old web, you know, you could try to rank for like top backpacks and do a ton of research and have affiliate links.

And that would justify hours and hours of time to do well.

Well, why would you do that now?

Right.

Especially as like Tim, you're mentioning, these all these LLMs are now gonna be adding in, you know, e-commerce ads, right?

Like I I do sort of wonder if this is the glory days of, you know, if this is like Google in, you know, 2000 or something, right?

Because they're actually trying to give you the right response.

But because of the underlying economics, we know that they're gonna be uh doing the things you alluded to, Marlene.

Like they're gonna be turning it in, they're they're gonna be inshidifying it.

Um, and it's gonna be about engagement um 1000%, right?

Like there's that me another meme is the um connector core, like the power adapter plugged into itself.

Like that

does seem

to be uh yeah, accurate.

I don't know, I'm not sure.

I think I think it's hard because at the same time one of so there's there's definitely opposing views in the space in terms of like on one hand as well, we need to kind of be thinking about the future and how do we get to how do we get to the best possible technology we can create for the future?

And do I think that I think the issue is that the frontier labs are at least framing themselves as having the solution to get to that future.

So a lot of people have talked about AGI.

That's a big thing people are hoping we get to.

And so um, you know, when we look at OpenAI, for example, they were the pioneers for ChatGPT, they've been the ones that have been pushing the frontier.

You know, Anthropic has now come on the scene and is also pushing the frontier there with LLMs.

And and and we've seen that scaling these LLMs has helped to an extent.

And if the people who have the most knowledge at the moment, we're assuming the people that have the most most knowledge about this space are working for you know these frontier labs, um do they not kind of owe it to us to kind of explore to the full extent how good these LLMs can get so that potentially maybe the LLMs could generate really good results on their own?

And I know there's a lot of talk right now on reinforcement learning and and and having these LLMs learn by themselves and improve themselves over time.

I don't know if that's a practical future.

Do we think that this is a practical future, that that's something that is actually going to happen, or do we think this is all going to be driven by economic incentives and there's no incentive almost at all to reach this kind of AGI um future is is my question.

Whereas imploring people not to buy into the AI hype.

You know, some people took that to think that, you know, I was a technology hater or something like that.

It's quite the opposite.

You know, the thesis of that talk was meant to be that this insane hype cycle, which is the biggest technology hype cycle I've seen in my over four decades in tech, is actively preventing us from finding how to use these algorithms to improve the human condition.

Because I do think there is a there though.

There's an undeniable, um, there's an undeniable something there in these algorithms, in these algorithmic advances we've made over the past few decades.

Um, you know, we see it in better prediction models and tracking hurricanes and stuff like that.

But, you know, trying to make these into some kind of magic wand, I think, is is actively dangerous.

I think it uh, you know, I've watched open AI go from being supposedly a nonprofit that was supposed to improve the future to humanity to now, you know, creating interactive sex bots that I think Marlene was referring to.

That's true.

They just couldn't, they didn't have the salaries to compete with Google.

And so what better way than to say we're an academic lab?

So I I think I don't think they ever meant it.

I mean, he there's internal chats like a week in with between Musk and all the rest saying, well, as soon as we hit scale, we can just discard this.

Wow.

Oh my god.

That's we've had the discussion.

Yes.

Yeah.

They don't think they're they're anywhere club.

If if I could switch gears slightly, because we're coming a little bit on time, I did want to ask a question around um how can Django be an if not an AI first framework, not miss out on this wave that Fast API is riding?

Um Carlton and I have discussed this.

I'm curious, Tim and Marlene, if if either of you have thoughts on, right?

I mean, Fast API is completely ascendant for a number of reasons.

And how do we how do we how does Django latch onto that and not be left behind?

I I I mean, in my opinion, I do think that as a community uh in the Django community, we we just need to be, I think, more open to AI.

Um, do I think AI has some very toxic things potentially associated with it?

Yes, absolutely.

Um, but at the same time, I think there's lots of good that we can add uh that AI adds.

And so, you know, I mentioned potentially a lot of Django just generally as it's structured is fantastic for um doing modular programming, for example, and and helping people who are vibe coding applications.

Django's a potentially really good framework for that.

Um, and I think doing things to actively like interact with that AI community is something that I think can grow the Django um ecosystem in terms of AI.

So I mentioned um I mentioned creating potentially an agents.md file that will people can go ahead and put in their code and have co-pilot or whatever Juni, whichever uh assistant they want to use, be able to create a Django app for them, but that's following the guidelines that this agents.md file creates.

So really creating these uh centralized resources that we're also using to kind of steward where we want the industry to be going.

And even as as as Django developers, I think these conversations we're having right now are really helpful.

So, you know, how helping Django developers have some guidance in terms of how they should use AI or or how we think we should be approach approaching the community.

Um and I forget, why am I forgetting his name?

Corey is is is fantastic, and he has a really good video he made on vibe coding with Django that I think has like some fantastic principles on approaching it uh, you know, in terms of modularity and things like that.

Um so more resources about AI and Django is what I would personally love to see.

Yeah.

And on the second day, Sage Abdullah and Tom Usher, uh two of the core team, gave a really good talk about how Wagtail is going to handle um AI in the future.

And I really like the path that Wagtail is going down.

You know, so many of the corporate entities out there um are forcing AI into all their products, forcing it down our throats, um, raising prices after initially giving it for free.

You know, we've seen it over and over and over again with Microsoft putting Copilot everywhere.

There are like 18,000 different versions of Copilot coming from every angle.

No offense, Barlene.

I know it's that's

Um, you know, all of the big tech companies are making it sort of mandatory.

They're not giving people an opt-in option.

Wagtail is taking the opposite approach.

Wagtail has made a commitment that there will never be any AI for the Wagtail core, but that the Wagtail team is making a secondary package for anybody who is interested and wants to opt into it called Wagtail AI.

So you can pip install Wagtail if you just want the core Wagtail, AI free.

But if you want some of the AI features, you can pip install Wagtail AI.

Um, so the talk given during Wagtail Space, I think it's available on YouTube now, is called AI and Wagtail, Responsible Innovation for Content Editors.

And a couple of the commitments Wagtail has made is again that it won't ever be forced into core, but this is a blessed under-the-wagtail organization's umbrella package that is being developed secondarily to Wagtail for people who want those AI features.

Um, it also said we'll provide, you know, we'll provide a clear picture of our AI vision, have it publicly stated, and we will always avoid any kind of vendor lock-in.

It also gives sort of a practical knowledge of what's available today in the Wagtail AI in this talk.

So it's something that people can look at right now.

I think that's a pretty good model.

So, you know, I consider myself incredibly lucky.

We've had Django code in production for a decade now.

I've also been sober for a decade now.

So it's caused me to sort of reflect on how lucky I am to be part of these amazing communities, my recovery community, my work colleagues, and then the Russian doll of Wagtail within Django within Python.

You know, I've met some of the most amazing people of my life through these communities.

And uh I think that having the right people in place here to have this sort of moral and ethical conversation about the right ways to do it to allow people to opt in without forcing it down our throats.

Um, I think Wagtail is really sort of leading the way here.

And as a as being sort of the smaller Russian doll within Django and Python, I think it's a good place where we can look where on a slightly smaller scale than Django or a much smaller scale than Python, maybe Wagtail is sort of leading the way in something we can look to as Django on a way to sort of address it head on.

Um, but without being the first ones to dip our toe in the pool, it's it might be somewhat easier to do when you're in the content management space than a bigger web framework space.

But I think there's some really good ideas there.

Yeah.

Yeah.

I would just add my my talk was related to this idea of I think there's a sense that Django and Fast API just on the underlying technologies is an either or when practically speaking it's both.

Um, you can, you can and probably should use both.

And um I see a whole new generation of Python developers starting with machine learning and pure Python, and then FastAPI comes along and it's it's an endpoint, and they think that's the end of the web.

And so I I see a gap there, an education gap around, you know, when does Django slide in?

Because people think, well, do I even need Django?

And part of that is just if you read, you know, Reddit or Hacker News, you only see posts from OpenAI or Anthropic.

You see massive, massive scale as opposed to, you know, me, a couple of people trying to incorporate LLMs into a workflow.

And then Django's Django's there.

But I think we in the Django community need to tell that story a little bit better.

Um, one of the things in my talk was showing how you can hook Django into a local LLM and have a chat bot because like it works, you know, it works pretty fast, actually, too.

You know, so like we're not running our own frontier models here.

So yeah, Fast API has its uses and for sure, but like just clarifying like I'd love to give a talk next year on kind of choose your own web framework, you know, like choose your own adventure, and just break down like Flask is great here, Fast API is great here, Django is great here, and just sort of, I guess, redefine what those boundaries are because I think it's still a little fuzzy, uh especially to a newcomer, you know.

It used to be Flask versus Django, and now it's really fast API and like, do I even need Django?

You know?

So that's

I've sort of the same feeling of fast API, but I've got those nice, comfortable batteries that I'm familiar with of Django and the Django ORM.

So I've got all like my webby stuff, but then it gives me sort of that fast API, you know, quick, easy API access, ultimately flexible.

So Django Ninja has been one of the tools I've really enjoyed working with, sort of in the LLM space, tying it into Django.

But they don't know anything about the Django ORM.

And I think there's a nice way where we can you know handle um restricted field queries handle pre prefetch related automatic prefetch related uh and have modern serialization and the speed effects from there.

So I'm I'm working on that.

I'm hoping for end of year around that kind of period to have something to show

you.

Okay um

you didn't say one I did okay okay well

like your demos were so good.

And it's just it's an absolutely wonderful book.

It's uh I I needed something optimistic and uplifting given the state of the world and and it definitely hit the spot for that.

So I will definitely recommend uh Sasha Sagans for small creatures such as we and I also wanted to mention just two projects um Wagtail Nest and Djangons I think people are probably pretty familiar with um I've moved some of my projects over to Django Commons and Wagtail Nest and I found them to be a huge help with avoiding developer burnout.

Just having other people to pick up the slack during those months when you are doing something like I don't know planning a wedding and now have a 13 year old in your life it's been uh it's been very handy.

So if you're looking for a um a group to get together to avoid sort of that single developer on project burnout um both Wagtail Nest for the Wagtail community and Django Commons for Django projects um has been absolutely wonderful.

I wanted to mention a project um uh Django Keel by uh Sayyom uh Kurana who's goes under the tagline on GitHub of Curious Learner it's another uh kind of production production ready template for your Django projects um Sayum is top notch he's I'm really quality developer I'm really keen to see what his opinions are and what options he's had it looks like it goes into a lot more depth than most of your your startup projects he's got things about observability he's got things about async background tasks he's got front end stor

y back end story he's got proper docs on uh read the docs it looks really good so I'm excited by that um that's Jack

opportunity.

Um for a book I wanted to mention um the Web Accessibility Cookbook this is by um Manuel uh Matuzovich which is uh and it's a it's just a kind of how-to go it goes through like I don't know everything you need to know about your website so you can create accessible HTML from the the get go rather than trying to bolt it on afterwards um I really really really solid learning in that book I I I really recommend it um

ok and then Marlene if you you have your thoughts you can go so I've been reading Apple in China which came out early this year.

Tim you'd love this one if you haven't read it uh but it's you know basically how Apple thought it was uh it moved tens of billions of dollars every year into China and really developed the infrastructure in China and thought it was getting the better end of the deal and then it turns out uh China was so I think it's missed day it's interesting just day to day the executives at Apple really did never step back and think why are they being so why is China being so accommodating?

Why are they allowing us to do all these things?

And you know Apple thinking short term, China thinking long term in a way and the amount of investment Apple alone put into China just dwarfs any federal plans and everything else we have here.

So I think it's yeah it's really really eye-opening really kind of scary um but really well researched so I'd recommend that book.

So Marlene do you have something you want to add?

for the Django project.

I think Will

And then I went off like on a tangent to watch the YouTube video.

And and anyway in that uh YouTube video Corey like talks about like principles for vibe coding and then he also shares like a project that he created um a Django project he created through this vibe coding process.

So I was going to recommend that I cannot find the repo at the moment but hopefully I will find it after and I will link that so that it can be linked somewhere for for people to help.

But a great person to follow I would say is Corey has a YouTube channel with with great thoughts there.

And then for a book recommendation I I I don't know I'll get kind of do like a half a two half recommendations.

The first is um there's not a book but it's Simon Willison's blog um which I think is super is really good if you want someone who's a realist, someone who's connected to Django of course um I think he has fantastic thoughts about AI and and generally with industry is kidding and has very kind of realist perspectives there.

But that's like a that's like not a book.

So uh a a fun book kind of fun to recommend for reading is I've been reading a book called uh Children of God and it's like a sci-fi book.

Um is that fun though?

in a British sense.

Yeah it

she's so good sorry

And I was just like, I do not know how someone from back in the day could have this kind of foresight to imagine that it would be like this.

Um, so that book is called Children of Gob.

It's a fiction book, can recommend it.

I still remember because it's it's uh Father Sandoz.

There's a so he's he's this linguist who speaks like 13 languages, and there's a line that's still haunts me where he

I think it's

He's he has a line about you know, uh, you know, fluent in 13 languages, and he couldn't find a word to express a phrase or a feeling, and that utter frustration of language failing oneself.

Um, thank

you for mentioning her.

She's so good.

She's written a ton,

she's done

Oh, deep cut.

Yeah, she's she's awesome.

Really good book.

Can recommend, yeah, some cool, some cool pieces for sure about language and and about AI as well.

So I think it's

Kind of

Carlton's good.

Carlton's good.

I I've

realized it's my job to recognize life.

Yeah.

So I'm like, okay, I'll I'll I'll be I'll be the tech book person.

Yeah.

100%.

I want to assuage that fear a bit.

When you look at the history, whether it's farm work or uh I mean the computer being introduced in the 1980s, it eliminated 3.5 million jobs, but it created over 19 million jobs.

So while some people were displaced, it ultimately led to more work.

I remember 20 years ago, headlines saying that half of the employment of the healthcare industry was going to be replaced by technology.

And instead, over the next decade, it doubled.

So I want to leave anybody who's struggling to look for work right now.

I think that's got a lot more to do with sort of political forces right now and uncertainty than it does anything AI.

Um, you know, Mark Benioff, CEO of Salesforce, has been gleefully been celebrating laying off 10,000 of his own employees.

But they hired 20,000 people in one year during the pandemic.

Is it more likely that he is actually using AI to replace these people right now?

Or is it more likely he's covering up for a stupid hiring bitch that his company made?

So if you're looking for work, I know it's tough.

This is the worst job economy I've seen for tech in many years.

I want to send out my sympathies to anybody who's looking for work right now.

It's incredibly hard.

But I also want to give a ray of hope that I don't think that AI is going to take all of the technology jobs in the future.

And uh I think there will be hope and it will revent.

So I just wanted to send that out to anybody out there because I know it's tough.

And I would say that I do still think that the core skills matter.

And um, you know, right now, for example, even at Microsoft, we had like loads of layoffs as well.

Um, not great.

But at the same time, there's like I have no idea why I laughed when I was saying that because it's such a serious issue.

I'm so sorry.

Um, but like I think the issue is that we had like lots of layoffs, but at the same time, the company is hiring as well in in a lot, and there's so many open positions right now, and a lot of those positions are kind of aligned to AI.

So I would encourage people as well not to be afraid of AI.

I think there's a lot of negative things that it's led to, but I think if we can combine the skills that we have right now with that knowledge, I do think that um there are opportunities there for us.

Um, and I really hope that as an industry as well, we can just work together to to hopefully shape the direction of where the industry is headed for for the better.

So yeah, that's what I will say.

Nice.

Well, Tim and Marlene, thank you so much for coming on, for continuing this conversation.

That's really kind of the point of this podcast is to do that, to have the conversations and to share them.

So thank you for making the time.

Thanks so

Bye bye.

Learn more about their services in the link in the description.