Transcript: Authentication - José Padilla
Hola, buenos días.
Soy Carlton Gibson and siempre my amigo Will Vincent.
Hola Will.
Hola Carlton.
Hola, Jose.
Hola, como estamos.
Let's do the rest in English.
Yep.
Jose, uh, we're thrilled to have you on.
You and I have met a couple of times, and I think I first became aware of you through your work with uh JWT, so Django Rest framework.
So what's the do you have a quick background on obviously Carlton and I know uh about your work in Django, but what's the quick take on your involvement in the Django community?
Um so I spent some time building um a product called Blimp.
Um we were a small team building a project management software.
And our I remember our first MVP was actually built in Flask and like MongoDB.
And after um failing doing that right, we switched back to Django.
It's like probably like Django 1.2.
Um and so I I spent uh at least like around five years working with Python and Django.
Um so I had a chance to um kind of help maintain Django's framework and kind of pick up helping maintaining other libraries like PyJWT.
Um yeah, and like through that I I've worked on random other bits and pieces.
Um and by far Python keeps being my favorite um community.
Yeah, well, I uh we'll link in the show notes, but I I remember seeing you gave, was it 2014 to DjangoCon, you gave a talk on JWTs, which I think was your first time at a Django conference, is that right?
That's right, yeah.
Um and um I I think that went well.
I I didn't actually give that talk anywhere else, probably like in another Python meetup, um, but not in a conference.
Um what I most liked about that talk was how at that time visually explaining what a JSON Word token is, um I I think I I I did a good job at that.
Um kind of tying that up to authentication with like Django Rest framework and Django.
You know, I have to say, Jose, because you've been you were working on Django Rest framework uh for just ages and ages and years.
We were there and you were an absolute machine maintaining the framework, you know.
Ever ever I mean, maybe you're doing it part of your work, you know, so you had the time, but you were there, you know, making the fixes, doing the updates, you know, absolute rock for years.
Thank you, thank you.
That that means a lot coming from you, definitely.
Um and and I don't like looking back, I don't like I think what I um contributed the most at that time was kind of the story around third-party extensions, um, and you know, how that story fits into Django's framework and Django and you know the whole uh Python um community as a whole.
Um but yeah, it was like it was definitely not part of my job.
Um this is a free time kind of thing.
Okay.
Um but uh but I definitely was glad I was able to help out with with things like that.
Yeah, no, and you you would you know absolute storework.
But that's an interesting point you made about the third-party extensions, because it's something that in Django Res Framework we were really um um keen on and still are keen on because we have very limited time and limited um bandwidth to maintain stuff.
And so someone comes along with a new feature idea and it's like, well, first of all, can we put that into a third-party package that that that contributor can maintain themselves?
Because that makes that keeps the the core maintainable, right?
Yep.
So what was what was Blimp?
What was the quick take on that what that service was?
So we um we were kind of three people working on that project, and we got together one day and we were just thinking of like things we could possibly build a product around.
Um we had all worked in um kind of digital agencies and marketing firms and things like that, where mostly they were using Basecamp, like first person or Basecamp at that point.
Um so we had like really strong opinions about how project management should look like and should work like, especially in that um creative um environment, uh small teams.
Um so we had we we proposed a structured kind of process through product management.
Um so we we we we had a really nice like niche.
We you know, we were we were targeting small teams like ours, three, four, five people.
Um most work was around the actual tasks.
Um so we you know Basecamp had um messaging and people would just do product management via messaging, which is via email.
Um so we we kind of forced you to use our proposed flow.
Um and then you know, at some point Trello came up, Asana came up, um, and you know they were way bigger than us.
Um and and eventually like it it it made sense for us to just shut it down and focus on other things.
Right, okay.
What was the run there?
Because like that was you know, five-year period, you say?
Yeah, like uh at least five, six.
That's a good run, right?
For a startup that you've you know, a small team that you're self-running your own.
We didn't take money.
It was completely bootstrapped.
Um we you know got together and made it happen.
Um we learned a lot through the way, you know, through the whole period.
Um and there were definitely like really good opportunities for other things to pop up.
Um, one of them being uh this side project I still maintain called file previews.
Yeah, I wanted to ask about or you and I have spoken about it, but I think it's a really interesting project.
So what's the quick take on it?
And then we can dive into the tech stack, because that's that's interesting.
So we we were building um in uh at Blimp, we had like a file section, and you could upload files, right?
Share them, um, and you could upload provisions of files.
Um and once we designed the whole thing, we noticed that we were kind of missing pretty uh thumbnails for files.
So we you know if you upload a JPEG or a PNG, that would that's easy.
If you auto if you upload like a Word document, that's a little bit harder.
If you upload a Photoshop file or Illustrator file, that becomes harder.
Um so we we kind of built this um pipeline where you could upload basically any kind of file, and we were we would just output some uh PNGs.
Um and that was harder than it sounds.
Um so we like from the first um from the get-go, we we kind of built that separate to Blimp as as a serve uh separate service.
Um and we like after we built it and we were using it, we noticed that we could just put a landing page on it, put a pay button in it, and other people might buy it.
And surprisingly, it it outlived Blimp as a product.
Um it's been going on.
Um and I kind of took over it um after we we decided to shut down Blimp, the product.
That's great, because that's like the original microservice, right?
That's yeah, right, yeah.
Five years before the the book, five years before the the meme, it's like, yeah, it's microservice.
And so what's the what's the tech stack?
What how much can you tell us about the architecture of it?
Um so yeah, so it's it's built with Django, obviously.
Um it it it does have a simple API, so that's Django's framework.
Um you know, we use Celery um pretty extensively.
Um we are using Redis as our broker, Postgres as our database, um we host our workers on pretty uh hefty EC2 instances.
Um and our API is actually still in Heroku.
Um so yeah, it's it's it gives me an opportunity.
I I don't work with Python on my day-to-day job um for a while now, and it gives you an opportunity to actually be um in tune with you know what's going on with Python and Django, um, but it also gives me a chance to kind of um polish my scaling skills.
So we have uh you know, this is pretty CPU intensive work, um, and we have uh pretty interesting traffic patterns.
So it it you know I I have to think about how we scale this, how we improve our availability, um how you know we I make sure we have monitoring and observability up to par.
Yeah, because I can imagine that you could and maybe do get uh a big client just comes in and crushes you with uh yeah.
I mean, I guess so.
How does the traffic look?
Is is that common that out of the blue you'll just get nailed with a lot of stuff and then otherwise it's flat, or is there any consistency there?
So yeah, so
we you know file previews has customers around the world um and different kind of traffic patterns.
So we we have people doing like bulk imports of just files that they have.
Um so sometimes we'll get you know 5,000 requests um one after the other to generate um previews for different kinds of files.
And uh you know, the I guess part of the challenge is you know, you'd handle that with scaling your instances so you you have more instances and you can kind of you know process the queue faster.
Um but things like larger files.
So you know, we we have customers that sometimes upload a PDF file that is like 20,000 pixels by 30,000 pixels, right?
Um and it's kind of like a PD, it's like a giant PDF.
Um so you know, those are one of those could be, you know, could hold up a queue for a longer time.
Um so like you know, we need to kind of be observing our limits, time limits, memory limits, view limits, um, file size limits.
Um would
this be a good example of um a use case for serverless, like you know, a lambda or a functions-based approach, or uh you know, because that kind of looks like a the perfect example, right?
You've got these discrete jobs that can be done one by one.
Does it would it work in that environment?
Or um
so maybe so yes.
So that I'm hoping to be able to try that.
Um Lambda, uh like AWS Lambda, um they have some limits to the you know the size of of what you can actually install there.
Um so you know, we even though our workers run in a Docker container, you could imagine that Docker container is just like a VM.
Um so it'll be interesting to kind of split that into um to be able to just fit into whatever the constraint for Lambda is.
Um, but it should definitely be possible.
Um at this point, you know, um this is kind of my side project and I haven't had the time.
I I I I tried I tried porting a story.
I tried porting a specific part of the pipeline, and it just was so like it had an additional complexity like with testing and just like kind of how we would compile different dependencies that would fit into that particular environment.
Um but yeah, it it it definitely seems like the thing that you could probably kind of model in that service less like architecture.
Right, okay, interesting.
But it's not you know, so there's two parts.
One yes, but the other no is that hang on, like you know, the VM does it gives me a lot here.
I've got this whole structure.
Yeah, we I mean like I I spent a lot of time kind of tuning you know what we have to a particular infrastructure model, um, and that has solved our immediate scaling issues, so I am not you know touching that much of it now.
Can I ask, currently you're working at Alt Zero, is that right?
Yes, I do.
I started working at Alt Zero as an engineer in August.
Okay, so in an earlier episode we were talking about authentication, and I I sort of said, you're you know, I gave Alt Zero Go and it didn't really fit in with Django, and so you know I probably wouldn't use it, and I don't know what's the use case.
So here, can I ask you to put your Auth Zero hat on and tell us the all zero the Auth Zero story for Django?
Yeah, you got to Yeah, you're wearing the sweatshirt.
Can you can you tell so if I'm if I I'm gonna use Auth Zero in my Django project or I'm gonna consider it, can you just give us the set the sales page?
What what what does it give me as a Django developer?
Okay, so so kind of going back to that episode where you know you were talking about authentication and authorization, you kind of defined those, and I think you did a pretty good job.
Um and I guess one question I as a developer of like in the small side projects using Django, I I often question myself like why would I use auth0 versus something like Django all auth, right?
Um and if we kind of think about Django's um batteries included philosophy, you know, Django does a really great work at not letting developers screw up authentication, right?
We we uh there's a configurable password hatching system with really great defaults.
So you don't have to think about using Bigcrypt, or do I use like MD5?
Or you know, um it's really hard to even store like plain text passwords.
Um and then there's this other like whole other things that third-party third-party packages kind of help with, like password strength checks and throttling, um, authenticating against like third-party identity providers like Facebook or Google.
Um, but you, the developer, you still need to think about those security-related things.
You need to kind of know that you're looking for throttling of your login endpoints, and you need to install a package after you find it, you need to configure, you need to maintain that, and whatever other um kind of dependencies that that might have, you might need to have a Redis server or something else, right?
Um so off zero provides authentication and authorization as a service.
We give developers and companies building blocks to secure their applications without having to become security experts.
Um, you know, we provide unified device agnostic login experience.
So it doesn't matter if you're working on a web app or a mobile app, you can still provide this unified login experience that looks like one single thing, right?
Um we provide like many social identity providers.
Um you can add custom ones too, brute force protection, breach password detection, um, you know, single sign-on, passwordless login, multi-factor off, user management, and you know, these are all the things that you could probably find.
If something like Django or Django All app can't provide, you can go and find possibly a third-party package that somebody built.
You then need to actually assess if they're you know well maintained, if they're well built, if they're secure.
Um, and then we go back to that you know security expert thing.
Um offSier also does something really cool, um, which uh they they allow you to write custom JavaScript code to customize any part of the authentication and authorization flow.
So, for example, if you wanted to um enrich user profiles from data from like ClearBit, um if you wanted to reuse information from you know another database or service, um if you wanted to, I don't know, notify another system of like real-time logins, you can do that.
Um so yeah, so it goes back to like is you can you the Django provides you login out of the box, and you you know, you you could figure out the whole template situation and so yourself.
And you know, going back to like one of the recent episodes where we're talking about patterns, um, and you after you've built you know one, two apps that have login, you'll notice the patterns and you'll maybe have your own project templates and you know, um, but then this other like little thousands of other things around authentication, authorization, and security.
Um, you either need to be aware of, be proactive, or you'll have to be free active after a possible incident.
Um so either you know you kind of delegate all those really important security and trust aspects of your application to somebody like Office Europe, or you know, you take that on, and you need to be aware of you know what taking that on means.
Okay, that's so that that's a super answer.
And so what um one thing you you mentioned at the beginning of that, so two thoughts that came up.
One one was you you mentioned about the side project versus the big you know, you know, it's a side project, yeah, totally.
I can see why you you take it on, but ult zero sh sh surely pitches itself for big projects too, right?
Oh yeah.
It's not just for the developer of site.
So yeah,
so so it's it's not only for you know somebody building a web app and a mobile app.
Um we are an identity platform, right?
So if you're building a single sign-on for your whole like internal infrastructure services on a large enterprise setting, auth zero would still you know work for you.
Fine.
And okay.
And then when the second thought was when I tried this a long time ago, it was when Auth Zero first came out, and I I gave it a go and I gave it a spin.
I didn't really find that it integrated with Django in the way I wanted to.
It would do the JWT client-side authentication.
I could see how that would give me the mobile app authentication.
That was all super.
But what I really wanted was in my view, request.user to return an instance.
So is there now a nice story about integrating with the Django project?
With like remote user, but remote user backend?
Yep.
Yep, um, so we we have I feel like one of the things that attracted me to Auth Zero a long time ago um was their content as a marketing effort.
Um and you know, they have really great guides.
Um so there's I was just checking out the getting started guide with Django.
Um, and since Auth Zero actually provides an auth um sorry, an ol auth2 um API, you can use something like Django Social Auth or Django all auth, and you would use auth zero as an identity, right?
And then you would handle in auth zero, you would handle Facebook logins and whatever.
Um so I I you know part of the the the uh quick start guide I was looking at does use remote user backend on the remote user middleware.
Um and it you know it it it really fits into the whole Django lingo.
Okay, that sounds that sounds super.
So I think it can you um in the show notes we can put a link to those getting started guys with Django, yeah.
Definitely.
Well what I wanted to ask you too.
So I I saw on Twitter, I think just yesterday, there there was just um the North Bay Python conference, and uh uh Guido uh tweeted this thing about um Jacob Kaplan Moss, uh Django co-creator, saying, Stop using passwords, use login with Google, Facebook.
Um quote, Google's security team is better than yours.
I have some I'm honestly my jaw dropped when I saw that, um, but I'm curious what your thoughts are as much more of a expert in this realm than I am.
I just don't even know where they're coming from with that.
Uh you know, I mean Facebook was storing stuff in clear text.
Uh I mean I can see auth zero, but to just hand it over to you know one of the couple monopolies, to me, I just can't um I don't know how I'm I'm I I feel like I'm missing something bigger than
if this is just a side weekend project, then you know, absolutely, the more you can outsource the best.
But if this is if you know if your user database is a core strategic asset of your business, then you can't just hand it over to a third party.
So there's that there's that point which is important.
And yeah, our Facebook, that's the claw, because every time I turn on the news, it's
it's so crufty and creepy.
I mean, Google's not much better.
I mean, that's I actually I mean, so uh with uh Django All Auth, which is fantastic, I I think I still have the you know top Google spot for a tutorial on that.
And I actually deliberately don't I think there's one I maybe I showed Gmail, but I don't show Facebook because well here I go, never working at Facebook.
I think Facebook is pretty awful, so I don't feel like doing their job for them.
Um but we I've we got off on a little bit of a rant.
So so make the case for auth zero as opposed to like Google Facebook.
If someone just says, well, I'm just delegating my auth in either case, why is that actually incorrect?
That's interesting.
So so like I think one thing to notice is that you can use all auth, I mean, sorry, auth zero without um integrating any like social identity providers, right?
You don't need to use Facebook or Google or you know the other 300 providers that we might have.
Um you can use like email and password against the auth zero database, right?
Um yeah, it's not like trust is the name of the game.
Um it's especially when you're handling such an important path for an application as authentication is and as authorization is.
Yeah, no, I mean it's a it's a deep topic.
I mean, what one part of this which I've seen with some services recently is this login by email where there is no password.
Every time I log in, I just put in my email address and I get a one-time link.
You know, that's I can see how that's more secure.
It's maybe a little annoying, but I'm seeing that more and more.
Um But as well, you've got to bear in mind as well that a lot of people struggle with passwords.
Um and if you can send them a link where they get a button and they can click it, that's or you know, tap it these days.
That's actually a much better user flow.
So it's it's arguably more secure, probably is.
And you know, assuming the email's not compromised, but the email's the sort of golden key to all of this, isn't it?
Um yeah, so so so that's like passwordless login, and that's something that Autol allows you to do pretty easily.
Um, and you know, we we since we also provide like um MFA, so you can do like multi-factor authentication, yeah, you can use like um like the Google authenticator or auth or or or if you have to, well, you can use SMS.
And I like I think now that we're starting to see 2FA become like something normal.
It's like I I wouldn't say it's not like I I can't get my parents to use like 2FA.
And it so then you you can't like make that a hard requirement, right?
Um so I can see like passwordless login not depending on your target um as a UX endpoint, it it you know it it might be it might actually like hurt your user experience.
Um like I can't imagine my mom, you know, putting in her email, if she remembers which email to use, and then like why do I have to go to my email?
And like I found the email now what?
Oh, there's a link.
Um so like again, it's like it depends on your target.
I can see now that you know there's a lot of players kind of working to improve the whole like user story for security.
Like if you think about security keys, like hardware keys, um passwordless login, um, MFA and things like that.
Um so I'm uh I'm hoping that in the you know coming years we'll have a better story for all of this.
I think Django needs a better story as well.
We don't have the two-factor rules built in yet, and we don't even it's not we haven't yet got the even like the hooks where you would add that really.
There are a couple of third-party solutions, but they're not, you know, they they need to be brought into Django, I think.
It's part of the batteries that now that you need to be able to plug into two-factor.
Yeah.
I I I recently um uh so I love the you know Django admin and I never turn it off.
Um so for file previews I've had to kind of work around things like pagination, because like you know, at our rate, most things don't work in the Django admin.
But like once you get through that her you know hurdle, it's like okay, so now how do I hide the admin login?
Um because I already have like this whole other login flow that I want to use and and and kind of funnel all traffic, if you know, if it's for our regular users or for our admin super users, users.
Um and yeah, that's it's been a little bit clunky on that side.
The way the way I do that, which is
I don't quite understand that I was just gonna ask, so are you giving admin access to some users can see their their previews in there?
It sounds like there's three levels.
No, it's just just me at that point.
Okay, so so what I do in that environment in that um situation is I use Nginx to lock it down to localhost and then just um SSH into the box so that it's and connect via a ton of
yeah, and then if you're using Heroku, that becomes like a harder.
Oh yeah, okay.
Okay.
Heroku's a different but like the ri the reason I do that is because then it's not exposed on a you know URL and
Yep, and like you know, uh most logins out there don't even have uh freight limiting or throttling, so you can do like credential stopping attacks and you would probably you don't even notice.
Yeah.
Yeah.
Uh well this whole I mean I wonder just from the for authors from the marketing aspect, you know, all of this stuff sounds the more you learn about the web, the more you go, oh my god, how scary it is.
But it's you know, but a beginner just you know wants it to work and has trouble with that.
So I I wonder I maybe you at what point does someone go, Wow, I really need to switch over to auth zero.
Is it they already get burned or they're at a big company where the login is complicated?
Um I I just sort of it feels like you almost need to get burned or be in a large enough, complicated enough organization before you go, Oh wow, maybe I want some help with this.
Yep, it that's that's it's I I think right now it's it's kind of you know reactive.
You've had a bad experience, you've had a leak, you've had, you know, security vulnerabilities being reported to you.
Um and I think one of the other like great things that Autreo does well is again, they this is probably a marketing strategy, um, is you know, putting out content that is relevant on security topics and best practices, and of and kind of like teaching the general audience about security.
And hoping that we're gonna start seeing people being more proactive about this and you know, not having to wait to actually be on a bad position after an incident.
Um, and then you know having to figure, oh, maybe we should have use an identity provider, um, and you know, three years in with millions of users, that might be a harder thing.
But still possible.
Right.
No, I think they have the fantastic guides on I remember uh just JWTs when I was learning about that.
And I mean it's similar to what DigitalOcean has done, I think, to differentiate themselves.
They have fantastic content because, you know, spinning stuff up is scary for beginners.
Though it does seem to me, just from a business aspect, that at the end of the day, you know, beginners are kind of whatever.
It's all about big clients.
I mean, this is what I have to keep reminding myself because I'm I mean, even Heroku, like I I I have Heroku in my books, and I mean almost every day I get some sort of question around Heroku and or Sengrid, you know, Twilio.
And I'm just like, why can't they be bothered to make this you know friendly?
And it's because they don't I I think it's because they don't really care, right?
I mean, a couple newcomers isn't the same as having really good uh you know docs as opposed to tutorials that an experienced developer at a large company can just dive in and use.
Yeah, I that's the thesis I have now.
I don't know.
I I can't speak to you know, bot zero or or twilio or gitalation tough fact, but um having worked on foul previews, I you know, after some time I I've decided to cater to a specific target of users.
Um mostly because I don't have the bandwidth to deal with a lot of like um incoming like incoming beginner level leads, right?
At the end of the day, they'll they'll unfortunately they'll require um a little bit more time um and kind of like not not hand holding, but you know, explaining what a webhook is and and things like that.
Um and in my experience, kind of converting that, you know, all that invested time and resources into converting that specific user to a paying customer, you know, the ROI is way less.
Um yeah.
So I I'm kind of vague in some parts.
Um I'm kind of vague in like some of the pricing, um, vague in kind of like some of the examples that we put out.
Um and uh for me that has worked.
Um so I I can't definitely speak to like
I think that's the a standard story from SaaS businesses, right?
It's the the big the what the few big enterprise clients, the the one where you're off the end of the pricing thing and it's call us for a quote, that's where all the profit comes from.
Yeah.
Yeah.
Well, I mean I'm slow coming to this realization, but it sort of makes sense because I've always I've always wondered.
Um well, so what so what is the stack at at Ot Zero?
Because you mentioned you're not using Python there.
What what sort of technologies are you using day to day?
So it's moly it's mostly um Node.js.
Um like from the ground up main core services been built in Node.js, and we've you know we have tons of people that are very experienced in how to make Node scale.
Um there are some services that are built with like Java and Go.
There's some Python, I think, but it's like for some like ops related things.
Um yeah, as we switch over to async.
Just do a total rewrite, no big deal.
Yeah, I'm I'm currently um so so when I joined, I joined the kind of the platform domain.
So we are like the you know backbone of trust for you know our customers, but we uh kind of enable other developers in off zero to you know be able to be as productive as they can be.
Um so I'm working on our untrusted code execution platform.
Um so we basically all this code that our customers can write in JavaScript to extend like different flows of authentication authorization.
Um we basically execute their code um and inject the results in different parts of the flow.
That's a that
um and yeah, and all that.
That's a great problem though.
Like you because hey, give give give me some code.
I'm not gonna know what it is, and I'm gonna run it on my computer.
Like that's uh that's that's a tricky problem, no?
Yeah, doing that at scale seems hard.
Um, but we we're I'm really excited.
Like it's I it's I I've been here for I don't know, a couple of months now, and I've learned so much from like so many smart people.
Um and you know, uh coming from a small kind of consulting and small products, um, the experience I've kind of gained, you know, doing things at this scale is amazing.
Um and that's you know kind of why I wanted to switch after all these years, you know, doing the entrepreneur thing.
Yeah.
No, that's that's great.
Well, and are they and this may be an ignorant question, but so it's node, so are they using uh you know, for the web stuff, is it express?
I mean I know the web part is just a teeny piece of it, but what is beyond node, what what is the the stack?
I mean, I know it's hard to define in such a big organization.
There's some express and there's some happy.
Um happy.js.
And happy is a um a kind of batteries included um web framework or API framework, right?
It's got it it provides things like Joy for the um validation and the back end, that the authentication backends, these are you don't have to go looking for them in micro.
It's definitely a a little a little more than express, way less than Django.
Um yeah, and that's that's um but I think that there's like work on standardization around you know which one should we use when.
Um that's all
I know.
Well I know that um uh Brave Browser, which actually I use, which is uh Brendan Ike um involved with, yeah, they're big on the the happy train.
Yep, and stamp of a Oth Zero also um actually uh they sponsor a lot of open source projects um around authentication, identity, you know.
Um so uh before I even joined Auth Zero, they reached out to me one day and they you know asked me if I wanted kind of them to sponsor the PyJWC library, and that's what that's amazing.
And I, you know, learned that they do that with many other libraries.
Um so they're actually, you know, we we use these libraries internally, and even if we don't use them, the fact that there are other developers working on this, um, they care about that.
It's really good.
Yeah, well, I like that they're doing the positive education approach as opposed to the fear approach, probably because fear is you can't fear something you don't understand.
Because certainly, you know, as a parent, I look at all the ads, they're all like your kid might die, or you're a bad parent.
Like they're pretty unsubtle about it for you know a car seat or a car.
Um so I appreciate them having that approach.
I did want to ask, uh we're coming up on time.
Um, I think you just gave a talk at Pygotham, right?
On uh which will is that out yet?
Can we link to that?
Yes, uh, I can definitely add that to the show notes.
Um so
Yeah, can you talk about that?
So yeah,
so I it's it's a it's called Python Government and Contracts.
Um it's it's a talk at Gave I think September at Pygotham.
And actually my partner in crime, Roylan Irisari, he gave a version of that talk at Northway Python.
Um that recording is also out.
Um so it you know, after Hurik and Maria passed through Puerto Rico, um we you know, I was not in Puerto Rico at that time.
Um and you know, I spent weeks, if not months, without hearing from my parents.
Um, you know, due to infrastructure issues, you know, that power was out, uh cell phone antennas were out, um water, you know, was out.
Um and so that that kind of leads to some uh kind of questionable contracts that were awarded to companies to kind of you know supposedly rebuild infrastructure in Parti Rico and kind of help after the destruction.
Um the FEMA was involved in some of that.
Um and that leads to um I kind of learning about the so the office of the controller in Partico, they put out the contracts that government agencies award to other contractors.
And kind of sometime after that, uh I had some questions as a civilian.
I am you know I am not an investigative journalist or anything like that, but I I did have some questions about you know how money was being spent in particular agencies.
Um and I kind of wanted to see you know relations between you know different contractors doing business as other you know companies.
Um and uh so I basically built a project um together with Roiland um where we kind of took the data, we we we kind of pulled and downloaded all this data from the office of the controller's website.
Um, we downloaded the actual PDF documents for the contract, and we're kind of extracting text from you know kind of things that I've learned doing fulfill reviews.
Um, we're extracting that text, we're indexing um that data so now it can be searched and correlated.
Um so this is kind of like a you know civic hacking project that I'm very passionate about.
I've been working on it for on and off for almost a year.
Um, and finally this year we decided to kind of talk about it um because we thought it would be an interesting um story about you know kind of how we went about it.
It this is open source, it's built Django, Django framework, Postgres, um our front end is like Next.js.
Um and yeah, so I'm I'm definitely very very excited to keep talking about this.
We we're we're at a point where we're you know talking about how we can be more transparent about you know how this is financially sustainable.
Um it does cost some money to run, um not that much.
So we you know we want to be transparent about that and see if we need help from you know others.
Um and it I I'm finally like excited to see where this goes.
Um, you know, I I want to empower other people like me to kind of know what questions to make and kind of be able to visualize and and at the end of the day just hold accountable whoever needs to be held accountable.
These kind of things you always see this you know, you know, billions of dollars being budgeted for rebuilding, and yet somehow on the grounds that you know the stuff wasn't rebuilt, and well, where's that money gone?
And it it all it takes is the light to be shone on where that money goes, and it will be spent in the right places, whereas when there's darkness, it goes into people's pockets.
So that's super.
I mean and and and and this would seem this would seem like like the the reason we talk about Hurricane Midi and and how that leads up to this, and and even after you know two plus years after that happening, is that you know, recently the FBI arrested a FEMA um employee and the CEO of one of the companies involved in one of these contracts.
So this is still relevant today, and there's ongoing investigations to multiple, you know, uh uh some politician has a daughter working for you know ten thousand dollars a month in some like unrelated thing in the government.
And now we're starting to see this thing's like popping up, and it's like, yes, we're fighting against like corruption.
Super Well, do you do you know if any of these uh federal agencies are using your work to help in their investigations or do they have the their own tools?
How does it work?
So I so I don't think so.
Um at the end of the day, this the actual contracts they are available via the office of the controller.
Um I do know that you know there is like some um I don't know, journalists kind of using this tool instead of the controllers tool.
Um just because it's it sucks less.
Um and it's you know, you you get some insights, um even at the simplest level, you get some insights of spending over time, um, and just doing you know, correlation between, you know, I don't know, um a contractor being entered into a system with a typo.
And you can still kind of search through that and see, oh, you know, these two contractors are the same entity, um, and they've been awarded this amount of money, and you can see the actual PDF there without having to download it.
Um at the end, I wouldn't have wanted to build something like this.
I would have wanted the tool put out by the Officer Controller to be this.
Um I'm happy to have done it um and and you know, keep like I spent a lot of time like you know kind of figuring out how to gather this data.
Um it's definitely not open data.
It's it's you know, it's on their site somehow, and I managed to just download it very slowly and and make it happen.
Um so like if now I can like empower, I don't know, some data scientists to build cool visualizations on on something else that I don't know, like I don't know, spending by a particular geo gro geographic location by fiscal year or something like that.
They can now do that without having to spend all this time and effort into actually gathering the data.
Super.
Well, that's fantastic.
I mean, because I uh to make this about me, I spent two years of my life on a project with school data in the United States where every state has uh all this reporting data on these various tests that everyone has to take, third, eighth grade, high school, all these types of things.
And I probably just got a t teeny glimpse of what you had where it's just a nightmare getting this information.
I mean, first of all, you know, some states had it online.
You could just download the CSV files, you know, like Massachusetts, California.
Some states uh, you know, it was like a PDF.
I think it was I think it was Kansas who said I had to file a Freedom of Information Act request to get a CD with the data.
Uh, you know, and it's just I I so I guess it's a combination of nobody has the ability and impetus to actually pull all these things together, and you realize how futile all these government agencies are where they're just checking their box and it's like no one it doesn't even occur to anyone to use it for these broader, like, well, what if we wanted to analyze the data?
What if we wanted to use it for something?
They spend millions and tens of millions collecting it and then it just goes away.
Deeply, more deeply sinister than that is that there are institutional biases against making the data openly available because people in with in positions of power have a vested interest in keeping it quiet.
And so it's important that people like Jose do the civil society stuff to bring it out into the light and call them to account.
That's just that's how democracy is maintained.
That's how I'm I'm welling up.
It's yeah, well, and that's one thing I I love about Django is it's I think unusual.
It's roots in journalism and investigative journalism, uh, you know, from the beginning, right?
I mean, uh Jacob worked at 18F, I mean Simon's on a fellowship at Stanford on this.
I mean, I kind of love that you know, non-pure tech background of uh of Django, whereas you said you you wanted an easy way to display this data and the Django part didn't stand in your way, it was the collection and all the rest.
Yeah, I mean I I keep going back to you know Django because I hate reinventing the wheel.
I just want to get stuck.
And so so it's it's you know, I have a project template that kind of has some of my preference picked into it, and I feel so comfortable.
Yeah, no, I I use that.
I it's one of my favorites.
I'm always following the update.
Wait, you don't use mine, Carl?
No, yours uh Yours is fine, but I use Jose's, okay?
Sorry.
Um competing over free tools, yeah.
Yeah, so so it's it you know it goes back to not having to reinvent the wheel all the time.
Like I just want to make things and do it fast and you know, kind of get out of the door as fast as possible.
And you know, Django and Python are the things to do that.
Um as we wrap up, are any plugs you want to make for uh so you have a personal site, um jpedia.com, which will link to anything around side projects or auth0 you want to mention as we head out the door?
So I mean authori is always hiring, um, so I'll I'll plug that link um in the show notes.
Um you know, I'll I'll also link to that uh Pygotham talk.
Um it's something I'm really passionate about.
Um I also set up my kind of sponsorship profile on GitHub, so I could maybe link to that.
Um I I'm hoping to kind of get back in track with um some of the projects that I've I'm working on that I know I've kind of over time let you know kind of sit away and and be unmaintained.
Um and I kind of feel really bad about that, um, because you know people actually use them.
Um so I'm hoping to you know start spending some more of my free time kind of getting them back to date, especially like dropping to Python 2 support.
Right, but let that it's let's just let me just say I just have to say that you mustn't feel guilty because the time that you give to open source is phenomenal and it has been phenomenal over a massively suspended period period of time.
You've got a life, you've got family, you've got work, you've got uh uh other commitments.
The guilt is is not necessary.
Your your work is uh stands on its own.
Thank you.
Great.
Well, again, I'm so pleased we could have you on.
Um my pleasure.
Great thrill for me to meet you in person, you know, a year ago.
Um so yeah.
Well sure links to everything.
Thanks again.
Thank you for having me.
Ciao.
Ciao.