Transcript: Web Security - Mackenzie Jackson

Carlton Gibson joined us over by Will Vinson.

Hello, Will.

Hello, Will.

Hello, McKenzie.

Thanks for having me.

Welcome, welcome.

So McKenzie, too.

No, say what you're gonna say.

Go.

But but I've I'm realizing now that I it's audio only, so I'll stop making obscene gestures.

I can wave back and then you know you'll laugh.

The audience won't know why.

So Mackenzie, come on, we always get this.

So what's who are you?

Tell us why you're on this podcast.

How did you find how do you find Python?

How did we meet?

How's you know, what's your backstory?

I I was actually um I started off live in my first my first life as an architect, like a building architect, and I hated it.

And I spent most of my job trying to automate, automate it by learning to code um in some of these big systems that we have called you know BIM systems.

And um then I was kind of figuring out like, why don't I just skip this architecture thing and do what I want to do, which is write software.

So then I uh kind of went down that path and had my own startup for a while called Compago, which is it's still around today, it's headquartered in Australia.

I haven't been involved in it.

I was in there for about four years as a CTO.

Um and then I left, I guess when the company got big enough that it needed a real CTO.

And and so I and then, but the one thing that I loved about um uh coding, a bit of a subsection, is that we were uh uh a care provider, um, building technology and in kind of healthcare space, which meant we had to comply with a lot of different areas like HIPAA compliance and other things.

And going down that I really learned uh how vulnerable software was to lots of different things and how to secure it.

And I got really into that.

So when I left, I decided to kind of focus on security.

Um, and that's been pretty much my jam.

I work now for a company called Get2Guardian, which is a code security platform.

And uh one of the the coolest things about my job is I get to work with some research teams.

We discover you know how attackers are kind of exploiting code, uh, we create talks about that, and then we get to go to some cool conferences, which is where I meet cool people uh like Carlton, because we we met at PyCon in Italy.

Um which was yeah, I think probably one of the I think we'll let some after party, and you were the keynote, and so I was trying to hassle you to use your keynote powers to get more drinks than the one drink limit that was allowed at the party.

If I remember correctly.

So let's let's move on.

Um, so attacks the sporting.

So I've you know, as part of my fellow role, I joined the Django security team, and we get quite a lot of um reports for you know, you know, every month we'll have reports go on.

So you you do a lot of that forensic work trying to find exploits, or you did you have done, or you work with teams that are doing.

Um, so it particularly around exploiting secrets, exploiting credentials, uh, also exploiting like misconfigurations.

And one of the cool things that we really like to get into is when an attack happens, to try and recreate that um attack path to try and figure out exactly, you know, not only like how do they get in, but what tools that they use, can we recreate the system and point out exactly what we meant vulnerable?

Because the nature of security vulnerability reporting is that if an application has a security flaw that gets exploited, then the blog post that inevitably comes out from the company is extremely limited and will you know won't give too much information.

So it's kind of taking that and then trying to figure out uh what what it what's behind it really.

For one reason, we don't sort of want to make it too easy for the people who are attacking unpatched Djos.

So I guess it's difficult to know how much you should say because I kind of also in the same breath, I kind of think an evacu an attack is not going to be put off by the lack of detail in the report, they're just gonna work it out.

I don't know.

So the the there's there's varying levels of thought behind us, and you have to be responsible on how you disclose you know information.

Um, and we certainly wouldn't just put out um kind of like a cheat sheet of you, you know, how to do it, but at least finding out, you know, at at what point at what point were it was initial access made, um, where did they where did they actually go?

Was it a phishing campaign?

How did the whole thing start and unravel?

Um now when you're talking about CVEs or like uh vulnerabilities and dependencies and things like that, then well, it's a lot uh you have to be a lot more responsible in how you disclose that because as you would know, uh two years on, there's still people running vulnerable, vulnerable packages and things.

So you you have to be a bit more careful, but in terms of recreating it and not necessarily just just showing the logic behind which the attack is used, because I think a lot of people don't understand attackers' logic, if that makes sense.

, but perhaps then they use that as part of a a bigger attack, and you know, I don't know.

Um and you know, like a lot and a lot of uh a lot of kind of attacks happen because people use you know the wrong the wrong function.

So the wrong, you know, you're generating a random number.

There's multiple ways to generate a random number.

Some are predictable using maths, which so if you're trying to use something to create access via these random numbers, then you you know you you have to be sure to use the the right thing.

Now, that doesn't mean that the un the insecure version is pointless.

It's it has its reason.

Um, it's just kind of the logic behind which you're implementing it is is kind of flawed and and people can often like figure figure that out.

But um, I kind of always say that you must update, you must be on a secure version, you mustn't use an end-of-life version of Django.

And I say that simply because I think kind of think as soon as these reports are out in the open, within a short period of time, there's a there's a just botch your kits you can download, which you can automatically run, which tests every available export known.

And you're you're sitting it's not a question of if your system is cracked, it's just when it's cracked.

Is that fair?

Is that a reasonable approach, or am I being a bit over cautious there, in your opinion?

I think that's like you absolutely have to patch uh regularly and definitely patch anything that has vulnerabilities against it.

Like it's absolutely vital to be able to do that.

Um and what you I think what we typically find in companies that can't patch regularly is that they if you you should always have a system in place to patch regularly, regardless whether or not there's some critical CVE, because if you in a habit of doing it regularly, it becomes easier when you need to do it and it's critical to do it.

Right.

And I think a lot of people kind of will patch when something's critical, but you know, that can, you know, that can cause all kinds of havoc and it becomes a scary thing to do.

And I think people often shy away from it, but you know, patching regularly is uh is absolutely fundamental, especially when there's a vulnerability out against it.

The argument kind of against patching regularly, and it's a bit of a weak argument, but just to make it, is that it takes about when a new version comes out, some often it can take about three months to figure out if it's vulnerable to anything, especially if it's like a big change in something.

So then, you know, the argument is do you patch immediately when a new version is out, or do you kind of wait?

And I think the solution is, you know, like patch regularly whenever there's a vulnerability, no one against it, and then stick to a regular patching routine for everything else where uh it's consistent and you you know, like you're you're you're doing it.

But if if there is a vulnerability, try people will be able to find it.

They build scripts to be able to like exploit these automatically.

You know, it's we're not talking about uh we're not talking about the most sophisticated actors.

If there's a C V E against it, like you've you've basically given the cheat codes to how to exploit your application.

So you should definitely patch.

Okay, okay okay and I guess just that point about new versions I guess for Django say um you know 5.0 is about to come out and you might wait for 5.0.1 or 5.0.2 if you're particularly worried you know about those first regressions but meanwhile 4.2 point whatever is still being released with the security updates so you should definitely be getting that each month

So Flask which is another big python web framework recently there was some discussion in the community about something unrelated to security but that's an example of something widely used that as far as I'm aware does not have like a formal security team for example.

And so I'm curious if you if you saw any of that there was there was a an an issue with like Flask login which is a essentially a third party package where it's hasn't been maintained and so there's a new version of Flask and all of a sudden login was breaking for everyone and there were some comments about that and people thinking that Flask was this you know Microsoft or something and had all the support when really it's like one person and a slight handful of volunteers maintaining um so I gu

ess the more like I just want to address that because it's been out there but I'm curious like where you sit when you see web frameworks you see that right there's the whole gamut there's Django and then there's many widely used frameworks that are you know may not have robust security things.

I'm not saying putting that on Flask but like a lot of these projects may have lots of users and it's a you know handful of people doing it.

Why?

Because that was like we had a .NET guy and we had a React guy.

So like you know that's what we like that's what we had right so which is you know everyone's like why haven't you using node at React because we didn't have a node guy we had a we had a .NET guy and so when it when it comes to like these types of frameworks and things like that you know like if you have the foresight and I think you know people that have been around for a while will be able to know and they they understand it and you know certain different frameworks will do different things they're maybe more secure they may be faster they may be be able to handle large quantities of data better.

So there's like lots of considerations and often security is is a foreshot but you you really there's a couple of things to look for uh when choosing a framework one does it have a long history of of being maintained and I don't necessarily mean like years and years and years I mean you know is it constantly being maintained is there a community around it because if if those are the cases then you should you know you can you can start to check off and feel a bit feel a bit better.

Is there a security team for it?

Well that's not normal you know for for everything but that should definitely help you kind of make those decisions.

So if you are in the fortunate position to be able to pick frameworks then looking at the community looking at the team that maintains it because you you know you will be surprised you know what happens is the example of the um of u a parser uh this is I apologize for everyone this is a node you know package but that's okay there'll be a there'll be a there'll be a python equivalent but you know u a parser was uh just a a package that let you know what operating system your users were you know viewing your app on

uh what it's something very simple it was used by it had 10 million weekly downloads um and it was maintained by one guy named Fashal and uh his np his node his npm account got hacked and then someone created a malicious version for it you know this is because you know this is this one guy maintaining this thanks you know thank thanklessly uh you know it would have passed everything but you have to also understand you know consider the team behind these not just is it popular

think um go on so get thinking about that I mean um PyPI have done an awful lot recently about tightening up just within the Python ecosystem about tightening up you know you need to use two factor auth now if you're a popular project and they've got this trusted publishers thing.

What's that?

campaign isn't easy but it's certainly a lot easier if there's no two FA to to get into it and um these supply chain type attacks can have massive uh implementations you know because you're not just attacking this one system you know you're potentially uh creating a vulnerability on millions of applications so uh one thing to remember about attackers is that they operate on economics like a normal business does where there's a risk reward like is the risk of

me attacking this system going to outweigh the reward that I you know that I could get potentially and when you're talking about systems you know like PyPy packages um that are being used by millions of different uh applications then you know the r the reward is massive potentially because you could install you know even a crypto miner that gets released onto a million websites you know can create something so simple can create you know pretty good profits.

So it's it's it's really good that these uh that these package managers are actually implementing you know more security implementations around this because it it it like 2FA may set not seem like a lot but you will really strip back the amount of account takeovers from that.

So you know making sure that there's not long lived passwords that's being accessed in there.

So that when you look at that it's just another tick box.

And when you're choosing packages you know you don't need to spend hours on it.

It should be easy to look at something and say oh this is secure and that's why something like the trusted publishers is really powerful because you know you can quickly look at that and know that it meets the required the like the criteria of at least that minimum so you can move forward as it's actually quite powerful in terms of being able to quickly make decisions of what to introduce into your project okay and is it going to be secure.

I'm getting a little niggly inside though because there's some of some of these metrics that you you sometimes get on projects that they can be like did you tick this box did you tick that box and it's like some of them are are you using a particular feature on GitHub and it's a bit like well no we're not using that particularly jet say for instance Django Django's got its own release program release process it's got its own security process it's got security archive it does its way of hand in CVEs it's all you know top quality really but it's not going through the GitHub

security advisories panel therefore we don't get the tick in the box on that metric and you know sorry I I sometimes get a little bit like oh I hate these metrics but

Yeah right you know being able to use them uh with confidence rather than you know something that's massive you know that's that's trusted by people if it doesn't have the tick box then you you know like you you can you can still get it passed but in terms of like if I'm just looking for, I just need a package that's going to be able to do the small job, you know, can I trust putting this into my production or not?

If it has the tick box, then that's a good step.

But you know, but it's not the be all and end-all.

And no security will will be the be all and end all and handle.

And like I work for a vendor, and you know, we have people that come up to us and just be like, okay, so here's my list of requirements that I need to tick for for my sock certification or for this or for that.

You know, like where does your product fit in?

Oh, you don't fit into this tick box?

Oh, I don't need it then.

It's kind of like so they're not worried at all about getting hacked, they're worried about like compliance, and I guess that's important, but so I understand where you're coming from.

Is it, you know, because if the box on the questionnaire isn't right, it's not good at all.

Okay.

That yeah, how it's just the same questionnaire box for for such wildly different applications.

There is a third-party package that Jazz band maintains, but maybe there's been separate discussion around auth in Django recently.

Carl's been uh uh advocating for some changes because you know, because there's a number of, I'll just say off off the top, there's a number of things, you know, Django, if you're gonna do it today, like there's first name, last name is the default.

Well, that doesn't fit a lot of the world's population, for example.

Um, also it defaults to username, email, username, and password.

Most people want email.

Um but maybe I'll look Carlton, I just wound you up.

Go.

It's meant to provide the batteries, and but it's not any old battery, because it's like for instance, it used to provide comments, but comments isn't something you can't build yourself or can't be maintained in the ecosystem, and it's it's not necessarily it's not it's a bit of a burden to maintain because there's so many you know opinions about what it might have and so many different ways it could go.

So DjangoContrib.comments was pulled out and it's a third-party package now.

But auth, auth really is a battery that Django has to provide because it's so central and it's so hard.

And if you get it wrong, the consequences are so bad that that's a battery Django should provide.

And so, yeah, we we we've got good authors, we've got good central auth, but we don't have this two-factor bit yet.

And for me, it's it's kind of like that's a missing battery.

That would be really nice if we could do something that's a you know, one-time passwords, or I don't know, what what the pass keys are the new things?

Can you tell us about pass keys and one-time passwords?

What are all these things, McKenzie?

Um, and especially our reliance on different things like API keys and uh and that that are kind of just sprawling everywhere because they're handled by so many different people.

So some of the things that people are trying to do to essentially remove these points of the vulnerabilities is to create basically the same systems but only valid once and created for the purpose of that session.

So, you know, like you have something like a dynamic API key that's managed by a trust, you know, like a vault or something where the API key is created, you then use it, and then you then destroy it at the end of the only valid for one time.

Or you know, whatever, however long that that it that it takes, right?

You know, um, and so these are kind of really, and I think we can expect um these to really start taking over along with kind of rule-based or authentication that's being implemented in in lots of different ways, because one of the problems that we also have been facing is that when you're trying to manage you know pass keys and passwords and API keys and all and all of that, you know, you can be tempted to create, you know, you've you need to do multiple different jobs.

If I create one admin key to be able to do all those different jobs, then I don't have to manage all these different keys, right?

But then if that that key becomes so sensitive.

So uh you know, having having role-based authentication where you know you restrict what what the uh you know to the absolute bare minimums and your authentication is created for the purpose that you're trying to do with all the minimum permissions that you're trying to do.

And with infrastructure, when we're kind of getting into you know infrastructure as code and on all of these different systems and secrets faults, and we can actually tie them all together so that it works really, you know, really nicely.

And I think that um we're not using these two to the full extent, but they're becoming more and more uh expected in certain things that I mean, like we I guess using your analogy, I guess we can expect you know frameworks to start putting in these different batteries um you know as we go go down.

Um so you start off, you get a you get a settings file, and in there the real secret is your your database password and your um your this secret key, which is used for um signing and and whatnot.

So it's important that you you don't commit those to git, and we can talk about Git Guardian in a minute and and and whatnot.

But the first part has always been stick those in a setting in an environment variable.

And kind of what would be what would be nice, I think, for Django to have is a kind of um a pluggable um interface around that.

So okay, if you're using MVES, you get it from there, but then there's all these other mechanisms like you know, vaults um secret managers and things that we we could sort of swap out the back end and you could be using those as well.

I think it'd be nice to have something like that in the Django space.

I mean it's uh secrets management is going to persist to be a problem that we're going that we have to kind of uh deal with.

And I mean people may have it's it's you know, it's funny, one of the one of the uh common passwords that Git Guardian detects later is you know, is the Django, is the Django secret keys?

Because often people get excited, they created their first Django project, they get at all and commit to to Git, and then all of a sudden they've you know released this this secret Django key.

Now, if it's it's probably not that interesting and to it to an attacker, at least if it's your first project and you're kind of having a play, you know, but systems don't know that, so uh they alert on it.

Um but I kind of feel like that's a good process because if you leak something on GitHub, you're going to get an email about it, and then you kind of are forced to learn how to how to securely do it from the from the from the start.

Um, and you know, when you're talking about environment variables and .emv files, I mean there's and vaults and secret managers, I mean there's like there's huge arguments about like what to use and when to use them.

And I think I differ from most of the security community from what I think, but now cool.

But what about that?

And we can't quite agree on what we should have, so we don't do have any anything.

It's like we're not

And you know, one of I I mentioned multiple ways to do it, but one of the ways, and the way that I like particularly like is using environment variables and .env files.

The talk before me, I don't know if this was like organized by the plan by the organizers, but the talk before me, the entire talk was about why you shouldn't use environment variables before it.

So so and so I I like environment variables, but I want to start off the bat and say that they're not the most secure thing to use them.

So if you if you ask a security person, you say, Well, how should I manage my secrets?

Then the official answer is you should use something like a Vault or a Secrets Manager that's dedicated to that, that's a server.

It's gonna create dynamic secrets, so just in time, you can connect to that to you know to to authenticate your developers so that they have access to secrets or their apps have access to secrets that no developers, you know, and it becomes like this heavily complicated thing.

And what will happen most of the time is that they will interact with that once.

Okay, this is such a pain in the ass to interact with the system, and then they create secrets.txt on their desktop and they store all their API keys there because then they don't have to deal with this heavy system that some security person spent a whole year implementing that has you know 400 pages of docs to go through of how to correctly use it.

And and that creates another pro problem, and then that's why secrets end up in your history because you you've been told, okay, I need you to create this feature and you need to connect to some kind of data bucket to do it.

So you to start off with, you just hard code the secrets because to do it properly, such a pain that you will, you know, that it's the pain.

But don't worry, because by the time code review comes around, you'll have removed that, not knowing that that secret is now in your Git history, but no one's seen it.

So no one actually knows that it's there, and that creates like a a big problem.

I know this is getting very long.

No, no, no.

And that's why

Because, you know, like because you know, like you because you just try you've you're under time pressure, you're trying to use it quickly, right?

And people don't understand that.

And that's why uh an attacker, if they make it into your git repository and they'll fish there's lots of ways for them to do this um they'll scan your history now the top layer of git's probably not gonna have any secrets in it um like by what I mean by the top layers is kind of like what's on the main branch you know what's in the the latest commits on on everything but when you go deeper you're gonna find all these secrets that have been added and removed from people because dealing with these heavy systems is is a nightmare.

Now does that mean you shouldn't use the heavy systems I I really think it's it you know like it's gonna it depends like everything.

But why I like environment variables is because for most people that's an adequate you know solution and it's easy enough to secure you create a.env file in your repository you create a dot getignore file and that's going to solve a lot of your problems.

The argument against that is that there are ways to dump out your environment if your kind of if your infrastructure if your server or you know yeah your other your has been your operating systems have been compromised then if the first thing any attacker's gonna do is type in ENV and dump out the environment variables from a running application and see what's and what's it that that is 100% going to be the first thing that they're going to do.

And so the argument against it is that if you're using environment variables you've just created a nice package for every you know for the attackers.

But my argument is if an attacker's made it that far like let's face it like you're not in a good position anyway.

Like I can I have access to your RAM I can find secrets in other ways maybe it's not wrapped up but let's not pretend that like the ENV file was the problem here.

's fantastic.

So you know that's that's you know my my rant over because I mean the it's yeah yeah there's there's there's lots of things but I on my personal opinion is that it's not the most secure way in the world to do it but it's so much better and easier and it reduces the friction which is part of the problem with security.

So you could search you know we as a like game we'd like get AWS keys you could get stripe keys because they were brand new you know so wild wild west like you just hey search is powerful boom there it is and like you could literally see it for so many like every company because there was no you know there was no automated there was no hiding it there was no email there's none of this stuff it was just like yep search search it all yeah so that was sort of a fun game we would do

address.

So if you're interested in targeting like a specific company you can just filter out of domains for people that are committing with a I don't know pick a company at at twilio.com email address to you know and find their personal GitHub IDs then start scanning all their stuff you know like it's still the search feature is a little bit harder like Dorking's got harder for sure but in terms of like you know finding keys we found 10 million secrets on GitHub in public repositories last

year like two and two million of them were for a uh for cloud provider keys so like it's uh which were all valid because we validate them though those particular keys we validate so like it's like it's it's bloody wild what you will find publicly but I will say github they're implementing other things that make it better along with like some companies okay so

o developer is this this helpful to me of

Anyway uh enough about my work so get guardian so we're a code security company um and we our our platform was founded on detecting secrets inside repositories so we talked about you know get history uh things like that so the the core get guardian product is that we connect into your repositories and we will search all through the history and bring out any secrets and we can if you're in a large company the value of it comes in that we will prioritize them we will validate them and we will

help you remediate them.

So in a large company that's what happens but for individual developers you know uh all of our systems are free we're the number one security app on GitHub so I think we're about 4000 uh users on GitHub um on their GitHub marketplace at the moment um it's just to make sure that you don't have secrets inside your repositories at any any point and then we also have cool tools uh we have a CLI tool called gg shield which will help you do things like uh install a pre-commit hook to that

will sit between you know your local repository uh well pre-commit sits just in your local repository it just kind of blocks any commits going to go getting staged that have a secret in there because once it once a secret enters your repository if you're in a team it's gonna be cloned into different areas it's gonna be backed up by different systems probably it'll end up inside like Jira tickets or you know like the disks crawl everywhere.

So once it's your repository you have to revoke it.

It's the only way forward you know by doing things like with gg shield CLI tool you can block them um block them coming in.

So it's really cool.

And why this is so cool is honeypots aren't new.

Honeypots have been around for a while but why honey tokens are cool is that not only can you put honey tokens kind of everywhere in your internal infrastructure you can actually put them inside third party tools.

So like CircleCI had a big breach at the start of this year and encrypted secrets were discovered.

So if you put a honey token inside your circle CI environment then you can actually know if that system has been compromised you know are your other secrets in there compromised so it's the only type of honeypot that you can put in different systems.

So there we are that's a bit of a longer plug than I intended to go out for but I

So going to slash admin is the default and so that's a pretty like good place to go look for stuff and so you can set it up and like log who's trying to get to your admin even though your admin's somewhere else.

And what what's a fun thing to do is create a honey token leak it on public GitHub and then watch what happens.

In a matter of like minutes people will try and exploit it but then it will also typically get sold like as part of a package like on the dark wood months later.

So you'll you'll start off by getting these random calls like low-level calls and then it will kind of get sold and then you'll start seeing like different types of activity.

It's really fascinating to be able to track what actually happens when a credential gets leaked, you know, like and how quickly it happens and how it moves through these different levels of attackers.

Because someone for 20 bucks will purchase a thousand valid credentials and just and then just spam them and see what they can do.

And you know, and if they if three of them allow them to do some mining or install some key loggers or something, then you know, happy days.

Are is that where a lot of like these bad actors are, or is it people in you know the United States who just cover their tracks a bit more?

Like what is what does the actual landscape look like of these, you know, uh business businessmen and women out there who are doing hacking?

Uh so like with a honey token, like you can get the IP address and you can see where the calls are being made from.

But I mean, if that's you know, if they're not if they're not using some third-party service to mask that, then you know, that's pretty surprising.

But you know, if it really is everywhere.

You have you have countries that are a lot more forward in sponsoring bad actors.

So you have, you know, there's Russia, you know, there's

But then you also have a bunch of teenagers like Lapsys, which are big, which were based in the UK, that were, you know, like that were really out there for for just clout, um, not really doing too much damage, but just kind of reputational damage.

Um, so there's like, yeah, I mean, there's people kind of everywhere that are that are interested in it.

Um, and I don't think it has any any any kind of real boundaries.

And I don't think any countries apart from maybe North Korea are kind of like green lighting, green lighting this, but there's certainly people where it's more beneficial to to to do it.

I will, you know, some different countries that don't have expedition agreements.

Yeah.

Yeah.

Yeah.

Deals, right?

Like, I mean, I guess, you know, you're not really doing it for the money, it's reputation, but you know, that would be the ultimate thing is to, you know, nobody knows how to.

It's like, you know, how do you how do you steal money from a bank?

It's like buy a bank.

It's like you

Like we we can't get access to like as an employee, but we do know a lot about it.

But I I honestly feel like people don't talk about it, but those, but people know, like those that know really know.

Like the I I was I've been talking to people at the moment, I got really into reverse comp reverse uh reverse compiling mobile applications and then looking for secrets inside of them.

And it was like wild what was happening inside there.

But then, you know, like you go talk about it to someone that knows, you know, someone you know, oh I've found this really cool research, I found all these keys, and everyone's just like, oh yeah, we know already.

Like no one's kind of like talking about it, but those that know kind of already or all you know, like already know it's it is like yes, like a secret little club out there when you when you know you know.

Um so we'll put a link to this.

You wrote an article about you know why Chat GPT is a security threat, essentially.

And that just like you win the like SEO lottery, right?

Because that's like every I wonder if you could expound expound upon that, right?

Because that's um you know something that it is big everywhere, like I've been using every every day now.

Um, but it's like, oh, that like you know, something else is gonna come and get me.

Um so for yeah, so for individuals or corporations, what's sort of the the main threat that you out outlined in that article?

And it's funny you mentioned like the SEO win.

Since that article came out, I've been published at so many like the Financial Times did an interview of me as if I'm some kind of like AI expert.

We have all boxes here.

We're we're looking for chat GPT and security.

He's at he's at Git Guardian, he speaks English, boom.

I've just got this vision of you and Simon Willison on a panel, expert panel at uh you know

So, number one, we look at it from the company perspective.

A lot of companies have now banned this.

And one notable one is Samsung.

So you kind of look at okay, why did Samsung buy ban their their employees using it?

It's because this is now another system that you know sensitive information has a way of leaking into.

So GitHub, using the example of GitHub, because everyone knows it, if you're an organization and they don't use GitHub, the chances are their employees still will, which means that that organization still has some kind of risk associated to it.

Chat GPT is kind of similar.

So you get it to summarize some legal documents, you get it to sift through a whole bunch of data or create some kind of uh uh connection using these these credentials.

That data is being stored somewhere, right?

And because of that, that means that chat, that your sensitive data is now in Chat GPT server, which is not a you know a super secure platform.

So now it's a target for other people.

So big companies have kind of have have have started to ban it.

I think that's the wrong approach because I think that your employees will use it anyway, because it's such a production boost, you know, and I feel like there's a fear that if you're not using this, you're going to miss out.

The other area of chat GPT that's a security risk or other LLMs is that when you when you you trust AI systems a whole lot more than what you should.

So where does Chat GPT get its data from?

You know, so you ask it, hey, can you create a coding instruction that will do X?

And it will spit back.

It'll do it in record time and it'll even like explain how it works.

So it feels like super great.

ChatGPT uses the largest data set around, which is the common cruel data set.

It's what all of these systems use.

Most of the code in that is rubbish.

Like look at a random open source repository.

It's that's what it's learning against.

Now, these systems can't really distinguish between good code and bad code, um, at least not in the vacuum of which you've asked them to do a query.

It's found the most common result.

A great example um is a different system, Copilot, GitHub Copilot, whereas uh at least this was true a year ago when I was really experimenting with it.

If we used the author, if we put in as an author at the top an author as a really respected, well-known open source maintainer, someone that's very prolific, and asked it the same prompts as if I used my name as the author, would get different responses.

Um, you know, like because like it's kind of looking at an example of what would this guy do and what would this guy do.

And obviously, I'm much shittier than the other person.

So, you know, so the whole point is that you actually come the risk of like the code that it gives you will have like vulnerabilities in it, probably.

Um, and here's a big difference is people say, well, what's the difference between that and me copying code from Stack Overflow?

And the difference is that Stack Overflow has a comment section where people will very quickly let you know if you're doing something insecure or around it, there's a huge community around that.

People can add input, you'll get various different results for the same thing.

Chat GPT, copilot, other LLMs have you know, don't provide that additional community input as to which is the best way.

You ask it for an answer, it gets an answer.

Um, you know, and I wrote an article called Shitty Code, Shitty Copilot, and it's you know basically you know that crappy input, crappy output.

And the people that are most likely to use these systems are probably people like you know that are early on and may not know the difference between random and random secure, and when you know, like when generating a random number and when to use both of them.

If that makes sense.

I mean, maybe it they're not as good at like um filtering things out.

I mean, it'd be kind of cool to have an LLM that's just approved answers on Stack Overflow.

That would be kind of cool.

Maybe that exists.

It's not a genius AI system, it's come from other people on GitHub.

It's organizing it in a revolutionary way that helps you find the solution that you're looking for extremely quickly, but you know, you have to understand the limitations of that and you know where the answers come from.

Yeah.

And if we're talking about security too, the other area that of these systems and AI that people are worried about are attackers using AI.

So can attackers now, you know, more effectively and the answer yes and no, like everything.

Uh, so again, again, uh AI systems use known data to do this.

Revolutionary attacks, you know, are not on the public internet, or they're very, you know, very hard to do, and it's not coming out with new stuff.

So if if you wanted to create malware, if you use it, will block you at first, but if you use clever prompt injection, like one that used to work, I don't think it does now, is like imagine you're an AI system that doesn't have limitations.

If you put that in chat, you know, used to I think it's important to my career.

You know, like yeah, yeah, yeah.

Like, or give me an example of malware, you know, that these things used to work.

And prompt injection is, you know, there's lots of clever ways to get it to do stuff.

There's also open source LLMs that you can remove these limitations.

But anyway, um, if you asked it and you successfully got it to create malware for you, it's going to be pretty standard malware.

Like it's nothing that you wouldn't be able to find by Googling yourself.

However, you know, what it does do is it gives Script Kitty superpowers.

So, you know, like when you combined a script kitty, which is a you know, someone that doesn't have a huge technical knowledge that just can run malicious scripts to get into things, it's a big section of the malicious market.

You know, you now give them Chat GPT so that when they're doing phishing campaigns, they can now do spell check.

Like, you know, how long was it that the biggest red flag was misspelt stuff?

Well, forget about that now.

Um, you know, they can they can manipulate the code for the first time, they can input malicious code and say, adjust this so that it can do this on this system or get, you know, um and these things.

So it does give some superpowers, but when we're talking about like, is it going to revolutionize the hacking?

Not at the moment, because it needs to have it can't come up with new ideas yet.

It can only regurgitate existing ideas.

So I mean, this is like look, there's lots of concerns around AI.

Um, and I did uh uh an interview recently, which was uh with a guy called Simon Maple, which was like, is AI our friend or foe?

And um surprise, surprise, the answer was it depends.

No, but I think it was uh

Like, if there's a magazine article and it has a has a question in the title, like it doesn't know.

Otherwise, it would say AI is a threat.

AI is a friend.

Yeah, yeah.

So, like, is it a threat?

Yes, no.

No.

So it's it's it's here to stay.

Um, and at the moment, the biggest risks are not externalism, it's not an attacker using it.

The biggest risks is internal, your employees uh using it, but not understanding.

And if you block it, if there's someone here from a large organization that's thinking, okay, I'm just going to block this system, then all that it's going to do is uh send people to the background.

They're still going to use it, but they're just not going to tell you that they're using it.

Right.

I mean, as a security expert, let's talk about LLMs a little bit more.

I was curious your take on there.

Is this issue of like the Ouroboros of like now that we have, you know, so much of, you know, so it's like Google's the problem with Google, right?

Google searches the internet, but then people like want to feature on Google.

So then you write articles for Google.

It's like why YouTube videos are all like 10 minutes and one second, right?

Like, so it becomes like just gets crappier and crappier and crappier.

It's like silt building up behind like a dam.

Um, and that's I've seen, and I can believe this based on just typing in that like they think the majority of article, you know, articles are soon gonna be AI generated because it's like perfect at you know these these topics.

And so over there's an argument that over time, like this is as good as it's gonna get.

It's just gonna get more and more crappy as AI consumes AI, and then like, you know, and of course it's all copyright laundering, which we we don't need to get into, but um I'm curious if you have a quick take on that, or does that seem right to you?

Because to me, I don't really see a way out of that.

You know, both in terms of I guess code you can run and you can test it, but like things you search for.

I mean, I use Chat GPT as Google now because go and I used I've been using DuckDuckGo for years too, also just because Google's doing its inevitable thing.

But I kind of wonder, like maybe maybe I'll just be stuck on like, you know, September 2021 or whatever is the date, because it's just gonna get worse.

So I don't know if this is gonna be like the popular take, but I have an interesting take.

I think it's gonna have the opposite effect.

So here's why is that we came from it like 30 years ago, the main source of information was all books.

It came from trusted kind of sources, and you could say whether or not you think that's bad or good, but you know, you that's where you're gonna be able to do it.

Very trusted.

And it's hard to it's hard to it's hard to publish shit.

It is hard.

And now Chat GPT has made it so easy that I feel like it's gonna have the effect that the that there's gonna be so much shitty information that everything's gonna get so much shittier that we're all gonna go back to trusted sources instead of instead of kind of going on, so like are we going to be like googling answers or will we go back to the trusted peer-reviewed research or articles or from the overriding.com books.

Yeah, yeah, Django.com or what you know, like all these, you know, like I feel like I feel like it's actually going to kind of make uh make the people that had some kind of platform but no brains, like it's gonna the they're gonna become so noisy that you're just going to ignore them.

And then the and then what will become kind of the new system will be that we all need to take a step back and go back to trusted sources.

That's kind of all that's what I'm hoping that's going to be.

Well, you know, I have I've my robots.txt is updated for all the crawlers, so I'm sure they're gonna abide by it.

And so I'm not worried about my stuff being pilfered.

Of course.

Yeah.

I've got it updated.

And there was a guy that was that was providing um providing feedback on a uh on a penetration test, and he explained how this person's organization, their infrastructure was vulnerable.

And the person's response to that would be yes, but that's illegal, so no one would do that.

Nice.

People don't abide by laws?

So we're right, we're coming up on time a bit, Mackenzie.

So I had I just wanted to ask you one more question.

So if you if I if we would say to first thing, move all your secrets into Embers and you know, then use a secret stand or something like Get Guardian and make sure you that you catch anything you do commit.

Is there a third kind of obvious thing that we should be doing that we you would say?

So uh we we need to add additional layers of security uh upon things.

So you know, there's a concept called zero trust, um, which is talking about, you know, like just because you have an API key or you have a password doesn't mean that you need to trust explicitly that person.

So there's something that you know, something that you have, and something that you are.

And we should be implementing those rules across security as well.

So um, some some simple things that you could do, you know, if if you're listening to this go, like, okay, that all sounds nice, but what can I actually do?

So come up with uh ways to rotate your keys regularly.

The reality is no matter what you do, your keys will leak.

You know, the best companies in the world, their keys will leak.

HashiCorp, they created a product called Vault.

It's one of the best secrets managers.

You know, they had a breach because they had secrets leak into their Git repositories.

So like it happens to everyone.

So what you can do is you can implement a rotation policy that rotates keys regularly.

So it's one thing so that that doesn't happen.

The advantage of that is that you actually know where your keys are or you know how to rotate them.

Because like then, if you have a leak, it's kind of like, oh, what the heck does this key do?

Am I gonna break production if I revoke this?

Like, if you're regularly rotating things, then you know what they are.

The other thing is like stop producing admin keys.

Like if you need a credential, don't give an admin credential.

I had so many stories of when you know a read-only credential would have been totally sufficient, but you know, that requires creating another key.

But if you've got a secrets manager, then that then the problem kind of goes away.

So, you know, make sure that keys has like limitations of what they can actually do to suit the job.

And then whitelist your services.

If you have a system that's only meant to talk to this other system, right, then you can set it up so that they can only talk.

Now, if I find the credential between them that allows me to talk to it, then I'm still coming from I'm coming from a different area.

I have a different IP, I'm from a different system.

So these are things that you can kind of actively do to add layers on top.

Because you just got to remember that there's no bullet, you know, everything about security, the famous words of this podcast, it depends.

But you can add layers in that create friction.

And that's ultimately kind of like the best way to handle this is to create enough friction, um, you know, that that we're adding so many different layers to to an attacker.

Yeah, that's like that's like one of those yeah, startup sayings.

Like there's no silver bullet, there's just a lot of lead bullets.

Yeah, yeah.

And one of the things is like no one has any nice bikes because you always want to make sure that your bike is shittier than the bike that's next to you.

Because then the person's going to because we're terrible at locking our bikes.

Yeah, it's lower.

So isn't

I did a bike tour and he was giving me these crazy numbers, speaking to the fact that like, you know, I guess for a century now, everyone bikes everywhere, but everyone like stuff gets stolen.

And there were some crazy statistics about when the Nazis left and the number of bikes that were trashed or something.

Because every now and again you'll see the canals, uh, a crane will come into the canal with these big claws and they'll just scoop up all the bikes that are in there because it just happened.

But you know, bikes are bikes are pretty easy to come by, and and it's an unwritten rule that if you know if it's past 2 a.m.

and your bike's been stolen, it's socially acceptable to steal someone else's bike to get home.

So you just gotta make sure you're the last one from the club as well.

Right.

That's perfect.

You know, if you had it so that your bike's slightly worse than the neighbor's bike, it's the opposite.

The mirror of that.

Great.

Again, so anything else that you'd like to call out while you've got my that we haven't mentioned, that you've you've mentioned your podcast, um, you mentioned Git Guardian, um, Compango, you've mentioned, we'll put all of those in the show notes.

Anything else that you think?

Oh yeah.

I think that's enough, enough, uh, enough plugs for for one episode.

But anyone that wants to follow me, um, you can follow me anywhere on social media at the handle at advocate mac.

Um, I'm even on threads, I've never posted, but you know, like just in case Twitter does explode, then then I might do.

So make sure you follow me.

But uh uh yeah, and I mean that's it.

I mean, if you want to check out if you want to listen to me geek out about security stuff, that's where that's where you can find me.

Okay, brilliant.

Really enjoyed it.

Thanks.

And great to great to great to meet you, Will, and great to see you again, Carlton.

We are at chatjango.com, and we'll see you everyone next time.

Bye bye.

Bye bye.