Transcript: Star Wars and Django

Hi, welcome to another episode of Django Chat. Are we Fortnite-y? I don't know how often

we do it. A podcast on the Django web framework. I'm Carlton Gibson, joined us by Will Vincent.

Hello, Will.

Hi, Carlton.

Hello, Will. And today we've got with us James Spadafora and Andre Prado from Industrial

Light & Magic, who are going to talk to us about all things of how Django is used in

a wonderful world of audiovisual and such. So thanks for coming on. This is a really

exciting to to have you on yeah thank you thank you for having us fine so let's kick off like

hang on industrial like magic you do really cool films um so how on earth is django involved in

that could how how'd you get into programming what's your backstory how you know and how does

django play a role in what you do you want to go first andre you go sure i can go first uh so i

actually graduated in computer engineering i started working the hardware industry and then

And I've changed industries a couple of times and ended up in Visual Effects almost by accident.

And then it's been like seven years now since I've been to Visual Effects.

And, you know, a lot of people don't know anything about Visual Effects.

Like, I didn't.

But we actually have this massive infrastructure to render frames, right?

We have, like, petabytes of storage, thousands of CPUs in a farm to do batch processing.

We have a bunch of services.

We have many data centers.

And they are all there to produce these great pictures that we see on the screen and that we don't even know what they are, how they are made.

Yeah.

Can I ask, when you say you've got these massive data centers, are they like in the basement in the office or are they in the cloud somewhere?

Because that's a lot of compute.

It's a lot of compute.

Yeah.

No, we mostly have our own data centers, and we have our own connections between the different data centers.

So we have data centers in Sydney, San Francisco, Vancouver, Singapore, London, basically everywhere we have an office.

We've also been using the cloud a little bit recently, but cloud ends up being very expensive for us.

Like our workstation has like, you know, 200 gigs of RAM and 64 cores to be able to open those heavy scenes and those, you know, do some rendering.

So you can imagine how the cloud can be very expensive

when you're paying for those sort of machines

and very high-end GPUs, too.

You don't get that on the AWS free plan, then?

Not at all.

Well, Carlton and I, as a side note,

both got the new MacBooks with the A1 Max chip.

Yeah, but now I feel inadequate hearing about 200.

I've downloaded Blender.

I've downloaded Blender.

So I'm almost there, right?

that's you are one step ahead of me uh yeah i basically come from you know more recently more

devops back-end uh systems engineering and i actually don't have a lot of the knowledge in

the proper you know visual effects and you know i've been in the industry for a while i help

people with workflows and whatnot but uh i think yeah james has been much closer uh to the product

than I have. So maybe that's a good segue for you, Ispad. Yeah, sure. Yeah, so I graduated

from Savannah College of Art and Design with a Master's of Visual Effects. And I started at ILM

right out of the school, working as what we call a technical assistant, which is a way of saying I

worked from midnight to 10am watching these renders on these massive machines that Andre was talking

about. I then followed the support path role, helping artists troubleshoot issues with their

scenes until eventually I moved into the production tools team where we develop tools for interacting

with data such as what shots need to be done, what tasks need to be done, generating media,

those sorts of things. Can I ask what do you study in that degree? Because I know,

Andre, you gave a fantastic PyCon talk in the show notes on sort of the pipeline.

And, you know, James, I can see Fluent Python on your bookshelf behind you.

So I know we're going to talk about Python.

But what does that curriculum look like?

Because I have no idea what that would entail.

Sure.

Yeah.

So it was a mix between art and programming.

So we learned, you know, how to use visual effects software, create images and render them.

Naturally, I kind of gravitated more towards the technical concepts, which is how I ended up in the position I'm in now.

We did things such as learning to write shaders, which are programs that determine how light interacts with surfaces, as well as just, you know, compositing, putting images together, all kinds of things.

Now, would that be like C++ versus Python?

Because I know, Andre, just from your talk, you mentioned that a lot of the engines themselves are written in C++, but the glue that puts them together is typically in Python.

Is that correct, James?

That's true.

We did do a little bit of C++ in that program, if I remember correctly.

But we never got into writing full-fledged renderers.

It's more like a computer science thing.

You can think of it as like the program was learning to use the end result of those tools.

if that makes sense yeah i think to give him to give an overview like you can imagine i

think the majority of code in visual effects is python by a large degree and all those renders

and c plus plus you know simulations and whatever needs to be super fast renewed in c plus plus

offers a python api so basically every tool that we use uh we use you you have like a python binding

a python api that you can import into your environment and you can interact with the scene

in real time via Python.

Let's say, you know, you're making a, I don't know,

a space battle and you have a spaceship.

Then through Python, you can, you know,

create 10 spaceships instead using an API

and rotate them and change, you know,

their texture or what have you.

All via Python APIs.

Very similar to how you do import requests

or what have you, but you actually import the DCC.

We call DCC Digital Content Creator APIs.

yeah we're just importing the millennium falcon yeah well in your talk andre you had you know

import maya and you had some examples of that um so is that so what is i so i well i have to jump

in my i told my wife i was interviewing you all and she's a huge mandalorian fan and she was like

what about that huge screen they use to film it right the the volume i think just to like skip

ahead which i believe uses unreal engine which is like a video game for the background and then

would that still use python to move things around because i've seen videos where they can

you know right there change things yeah i think interesting you mentioned that because we were

just starting a new what we call stagecraft studio in vancouver and spad has actually been part of

setting that up uh but uh yes definitely it's not as far as i'm aware we haven't been using

in real as much we have our own render engine that we uh wrote in house uh for reasons and uh

yeah i don't think i'm the most qualified person to talk about that but for sure there is python

there uh we run our own uh uh software that we wrote in house and there is python bindings for

all the different things i do think a lot of the really heavy lifting is done in c++ though we have

some genius engineers working on that side of things i don't know if you have anything to add

bud no i think you you summed it up pretty well um we we have a lot of engineers writing some great

tools uh for interacting with the screens um we also have a inner we've created interfaces like

on i don't know how much in detail i can go but this is a technical podcast interfaces yeah i met

like legally oh well that too we can edit it out if that's it yeah no one listens it's fine well

we can we can cut this if it doesn't work right so we've developed like interfaces on the um the

ipad uh that lets yeah that's on the video i saw i saw that on the volume is it there's it's from

2020 it was showing someone with the ipad and saying they were you know moving it around and

showing the screen moving so that's out there correct yeah all right well that's good then i

can go into detail on that um yeah so typically we'll have someone on set with the ipad and what

they'll do is they'll manipulate the light in the scene so in typical film production if you want to

change the lighting you would need to like move your lights around but with the volume what we're

able to do is with the ipad create these squares of light up on the walls uh to illuminate the

scene and that allows the director to get much faster feedback um and all the tooling around

that i think is i believe is written in python to kind of stream that information over to our

proprietary software because it's all leds right it's the 21 foot screen and then also the ceiling

right so there's no actual lights right it's just leds everywhere pretty much i i'm actually not

sure okay to be honest how much um practical lighting versus the led wall but i know that

we do use the led wall a lot to manipulate the light in real time right as we're working and

a suit of armor is pretty bad for for that sorry carlton i'm just so geeking out right isn't that

also last point isn't that also true that like you know lighting is super hard but then you have a

reflective surface like that really magnifies more than skin right the issues absolutely i mean uh

i don't think we could get as good looking results without the volume um because the reflections in

real time on the mandalorian suit of armor like that that's all from the wall um other in the

old days we probably would have used a green screen it would have been kind of tough to do

that so yeah and i think it's also worth noting that we've been using the wall for a lot of

projects and not just mandalorian but for example the last batman movie used uh some of our

technologies and yeah we have a lot of things in production using that which is super cool it's

like reinventing filmmaking to some degree absolutely carlton you you say well no it's

quite funny because um james and andre are for listeners at home james and andre are in vancouver

and will's in boston i'm in spain so like they've got a nice little chat and i keep trying to butt

in but there's this nice latency so i can't ever get it it's fine you know will you just carry on

um what i wanted to ask about was the um the the change over the last say decade um or you know

or longer in terms of because you've used lots of phrases like real time and you're talking about

having an ipad in hand and being able to drag components around and that's and that's just

raw processing power right that's that's that stuff that wouldn't have been possible you know

five ten years ago or even you know can you talk about the change in the technology and

um let me give you what one example was that i saw a line from the pixar people about

rendering um toy story 4 and they were able to render it basically in real time as they would

as they were producing it versus when they did toy story originally 20 years previously

you know it would take several hours to render a single frame yeah i think it's yeah for sure

technology has evolved a lot in the last decade uh but i do think movies keep getting more complex

too and we keep having higher resolutions so it's almost like it doesn't matter how much it evolves

artists on computer graphics are always using everything we've got so yeah it's like we

definitely do a lot more microsoft word always takes seven seconds to open right no matter what

okay adrian what's like getting an extra bin in the kitchen you just end up with more rubbish

and just the amount of visual effects show that is used nowadays right basically every single

show you watch on netflix or amazon prime or what have you has an immense amount of vfx shots

probably more than movies than had like five or ten years ago even though you know you don't even

notice it uh it's out there making for a very realistic experience so i want to talk about

the pipeline but maybe we could just jump to where does django fit in all this right like

what let's start let's start with that so right so you have this whole c++ things glued together

with python you've got massive rendering and databases where's where's the the django small

bit and all that i think that's a great question uh i think you know i'll give a little bit of a

Historic perspective plus, you know, and SPAD can get into the details.

That works for you, SPAD.

Cool.

But I think, yeah, historically, you know,

Visual Effects was just a bunch of Python scripts on a big shared folder.

And then you add that folder to your Python path

and you version those Python scripts.

I think that defines, you know, a Visual Effects pipeline to some degree.

And I'm talking about, you know, millions of lines of code.

So it's not like some Python script.

is actually like terabytes of Python scripts.

But I would say that with things getting more complex,

we have immutable data center, doing more shots,

we needed to scale that to some degree.

And I think a good way of scaling it is, you know,

decoupling a little bit of the client side dependencies

from the business logic behind the service.

And that's where servers start making an appearance

in visual effects, I believe.

And also, you know, some people want some fancy web UIs

done on React instead of like Qt

or, you know, some more desktop-based apps,

which also a server helps with.

So I think that transition has been happening

in the visual effects industry

a little bit slowly towards services.

And I think, you know, historically at ILM,

We are using, each group has a lot of

independency to, they are independent

to choose their own tools and frameworks.

And we end up with a lot of different teams

picking different tools.

For example, a team would use Tornado,

a team would use Flask,

and it made it super hard to be cohesive, I feel,

how we deal with authentication,

how we deal with databases.

So we sort of standardized on using Django

to building some of those services

that abstract the business logics away.

We set some common libraries that we use,

like Django REST, for example, and other things,

like how we connect with LDAP and et cetera.

We just made the process of developing a service more streamlined

that allowed us to have Ansible plays or Docker files

that are standard across the teams and et cetera.

To some specific examples on how Django is used,

maybe, Spad, you can go ahead.

Yeah, sure.

Actually, if you don't mind, I just want to expand a little bit on what you were saying about the scripts in the pipeline and whatnot.

So before I was on this team, I was on another team that worked as kind of the go-between between the projects we were working on and the engineering teams.

And we have some of the smartest artists in the world working at ILM.

And as a result, they like to write Python code.

So managing all the scripts written by all these different people across all these different teams has always been quite a challenge for us.

And also that move from Qt, because all of our third-party software, Maya, Nuke, was all written in Qt.

So that's what we were using for a long time before we made the shift to using WebUIs and whatnot.

To get back to the specifics of the Django stuff,

one area we're using Django

is to deal with subscription management.

So for an artist,

the typical workflow is they'll do some piece of work

and write out some media,

and then they will submit that media for review

for either the director or their supervisors.

And we have a lot of artists submitting a lot of media,

and trying to route that media to the right people

is a bit of a challenge

because you have, we have like five different sites all working on different projects with

different people. So we developed a service that lets users subscribe to different tasks based on,

you know, what the artist is doing. So for example, if I'm a director, I could say,

I only want to see lighting media from San Francisco. So they can go into this web UI

and edit these filters and every time the artist submits media it goes to the correct people based

on the filters that they've created yeah filtering and organizing stuff is really hard even there's

a site jango packages.org that jeff tripla has taken over that um you're just going through

grids and opting into things and i can't even imagine how complex it is to actually do that

at the scale you're you're talking about i mean can you you know because with code you can use

version control and git like how do you track changes on a image or audio file is there a way

or what what's the process for that well we typically don't check track uh changes on like

the specific files um what we do is we track submissions um so if the artist uh they'll

submit something and they'll get some notes that are attached to that version the artist will then

act on those notes and submit a new version and we keep track of all those versions associated with

like a task that needs to be done yeah i think in general terms visual effects companies usually

have this thing called an asset system which is the very much you know in the core of the company

and basically whenever someone published a new version of a model a simulation a rigging that's

a new asset and then you basically track some metadata with the asset a bit like s3 and you

have like versions for the assets have dependencies between assets so let's say i'm compositing some

scene and the model changes or whatever then i get a notification like hey there's a more up-to-date

version of the iron man model uh you know do you want to update it or what have you and uh yeah

there's a bunch of things like that that we have to tracking data across the pipeline is super

important as you know as i've mentioned we have a lot of storage and those assets are massive like

a texture can have like terabytes uh you know and it's it can be very yeah cumbersome to keep the

disk space uh tidy as we have many shows and artists and assets being produced well carlton

we always talk about you know it's something internet scale and like i think we need a new

vocabulary for vfx scale because it's just but it's so good because you've watched these over

the last year or so watch these you know new generation of macs come out and they're kind

massively overpowered for web development you know like you know there's just more more megs of ram

and more gpus than you really need but then when you get into doing you know actual visual effects

well yeah that's where these sort of superpower computer systems really do yeah we definitely have

have scenes that you know just to open it you need like you know 200 gigs of ram just to open the

seen yeah several questions one i mean one again and i'm just cribbing so much from your talk

andre because i watched it a couple times it's such a good introductory talk um thank you one

of the things you mentioned because i think you gave the talk in 2019 and you mentioned that python

2 was dominant in 2018 and there's gonna be a switch to python 3 we certainly have lived through

that switch in the django world is that is that switch complete or like what version of python

are you using, or is it all over the place?

Because it's kind of all over the place

in the Django world, but it's pretty much

all Python 3.

That's a great

topic.

I think the DCCs

like Maya, Nuke, and

other software, the visual effects

artists use, they ship with a Python

interpreter, and that Python

interpreter has still

been the standard PIN 27.

There are some of them that are in Python 3

nowadays or you know I feel the industry is slowly moving towards Python 3 we've certainly spent

huge effort on migrating to Python 3 recently a lot of those scripts and as I've mentioned we

are talking about millions and millions of lines of code from Python to migrate so it's very hard

to justify the business need for it I feel because you know ILM has been around for 40 years

and we have a lot of old code lying around

and, you know, but they are working

in the business function.

Right, so end of life two years ago doesn't matter.

You know, it's still, as Carlton and I joke,

every code is like just good enough to not break.

Yeah, I will say though,

that with the new versions of those tools

being Python 3 only,

that puts pressure on companies to move to Python 3.

All of our service side code

or most of it is written in Python 3, I would say.

When you don't have a dependency in the runtime of the artist

software, which is Python 2, we can do Python 3.

So really, we're trying to make all of our code

compatible with 2 and 3.

All new services start on 3.

But yeah, I think we are winning that battle.

Slowly, it's bad.

I don't know if there's anything you want to add.

No, I think we're definitely winning that battle.

We just had a big push across our teams

to convert all that artist code over from Python 2

Python 3 compatible, and it's given us an opportunity to do a lot of house cleaning,

which has been great, like archiving stuff that isn't used anymore. And it's been an interesting

exercise to figure out what's used and what's not. I think in hindsight, it would be great to have

more analytics on a lot of stuff. So I think that's a great thing to build in if you have a

lot of software to maintain. On this topic, Carlton, as a fellow, maybe just for listeners,

What was the two-to-three transition like for Django?

Because smaller scale, but still a pretty massive amount of work.

It lasted forever.

I mean, like, you know, I was maintaining packages like Django Filter, Django REST Framework for the whole, you know, over that period.

And you were just supporting 2.7, you know, and then 3.2, 3.3, 3.4 as it started coming out.

And you were running the test suite against each of them.

And it was a real, you know, at the end, it got to be a real.

pain it was like oh we do we really still supporting 2.7 here and then i remember when

it was like no actually it's the end of life now we're starting to drop it and

it was like ah what a wait it's so liberating yeah yeah it really was and they they got it where

you had you had this nice world where you could write library code which was both 2.7 python 2

compatible and python 3.0 compatible and it sort of went on it was like yeah this is doable this

is no problem but when you could delete all those objects you know it's important from objects and

when i can't remember the things and to get rid of six this you know you you weren't able to call

the string functions you had to use the the six compatibility shim and it was like and when all

of that dropped away it was just like oh like a breath of fresh air it really was um but you still

you know you break something which is python 2 compatible and you still get issue reports opened

as like, oh, you've dropped Python 2.

And it's like, well, Python 2's been end of life

for a couple of years,

so I don't feel too bad about dropping it.

But clearly there's still lots of people using it.

It's still really out there.

And then what are you and Marius,

what versions of Python are you testing for Django now?

Like 3.5?

Because I know you have to-

No, no.

So Django 2.2 is 3.6 through till, I don't know, 3.9 or something.

And then Python 3, oh, Django 3 is 3.8 up, I think.

Much smaller space to cover than, as you say, the visual effects area.

Yeah, shipping with two.

Well, we've got like 30,000 tests and so many lines of code.

We're not talking terabytes of code.

I mean, so okay, here's a good question.

Tests, right.

Because so I'm a digital artist and I write up a little Python script to manipulate my 3D scene I've got in front of me.

It works.

And, you know, I can do exactly what I want and I've bashed it together.

But I haven't written any tests for that ever.

Is that?

Yeah.

Yeah, that's pretty common.

So how do you address that?

So usually we use artists for their scripts.

Do it live.

Do it live, yeah.

Do it live.

Yeah, so what we found with a lot of the artist scripts is you kind of have to have an understanding of the software to test it.

So this was a good exercise for our team because a lot of them hadn't touched artist tools, so they got to learn a little bit in that regard.

Ideally, I mean, all of our software that we write on our team has tests for the most part.

And that's super important to us and something we want to continue doing going forward because it just makes things so much more maintainable.

an upgradable um yeah for for artists scripts um it's a little hard to enforce because we have a

bit of a wild west um of scripts built into our pipeline where anyone can commit anything

and use it um i don't know if you want to expand on that andre yeah it feels a little bit chaotic

and it is but it kind of works we have a software management system that we wrote in house a little

bit like, you know, pip in virtual env, but it's our own implementation, and that basically tracks

artist software so that you can version up, version down, you can easily roll back, like rolling back

software for us is like a one-second operation. You basically change the same link to point to

the previous version. So at the end of the day, we do release, you know, hundreds of times a day

those scripts and software, so we release software quite often, and we make it easy to roll back.

And for the RT software in particular, the Python 3 migration,

in our software management system, we added a check

that it basically tries to compile the software to Python 3.

And if it fails to compile, that means you cannot release it anymore

because it's not Python 3 compatible.

So we enforced at that level.

We didn't do the unit test or whatever

because it's just not doable, I think, for our problem right now,

our problem space.

But we did enforce that it at least compiles with both Python 3 and 2.

so that you can make sure that it is compatible.

And if not, you have to do a dash dash force,

Some people still do it, but, you know, at least we track everything.

We have, like, a web UI.

Yeah, sudo dash dash force, you know, make it.

Well, that's something like, you know, testing in our realm, Carlton, right?

I mean, I've often asked you for advice because there's, at one extreme, is test-driven development, do the test first, 100% test coverage.

And then there's also, you know, happy path programming, test it, and then when something breaks, write another test.

That's more of the iterative approach, I think, Carlton, that you've advised to me, at least, as well on this.

it depends as well like you know to test even in the quite let's say um

mundane realm of web development let's call it where you're testing a web ui that's hard to test

it really is oh yeah you spend a lot of time writing you know ui tests and they're really

flaky and they fall apart yeah integration tests they're yeah to try and do that with a sort of

you know a graphical editor like blender or whatever i don't know what the artist used but

kind of you know 3d environment tool that i can't imagine it's even feasible to get off the ground

testing that i just can't imagine there are tests it's even possible yeah i think one thing should

that spad mentioned is that we have a lot of people writing code that are not trained software

developers or don't care about software developers uh software i hope they care about software

developers but not about software uh you know at the end of the day our business is to ship

uh pure pictures for the image for the movie it's not to ship software too right so i think

that there is a trade-off there and uh yeah usually we just need to know how to manage it

you know if it's some very critical service like the asset system you better expect it has a great

test coverage because you can bring down the whole studio globally if you you know release a bug and

you know marvel is not going to change the release date of a movie because we can't render it or

whatever so uh you know some very core piece of the software i think to have the love and care

they deserve but it's more scripted side of things which is a lot of the software we have to maintain

uh yeah it doesn't need or follow the same uh scrutiny maybe that's the word i mean i think

i think the phrase you've hit there is it doesn't need it like it like this is a this is a script

i'm using to manipulate a 3d scene that i've got in front of me in a in a gui program i don't need

it to be you know unit tested in the same way it's it's how it's it's a tool to help me what

with what i'm doing not you know call bit of my business logic that whereas the django app you

were describing where you're doing the subscription tracking to asset changes that sounds sounds like

the sort of thing that would be susceptible to you know yeah much more rigorous testing

of course those script tends to evolve and then people start using each other's script and then

they build a library around these you know fast four or three years that random script is now a

core part of the business yeah yeah it's like it's like the mythical excel macro that you know no one

the whole the whole multi-billion dollar business runs on this one excel sheet

well i had a question about infrastructure which this is just like the layman's version but i think

it's somewhat famous that uh pixar with toy story 2 right they lost it except some art like producer

was on maternity leave and had a copy on her computer um i'm just curious for you as professionals

what what does that story say to you because i hopefully that's still not possible i mean i guess

you couldn't store a movie on a computer but like how do you how do you test or not cover someone

like you know nuking everything right because that's a pretty crazy story i think they actually

did thought they didn't have it for a number of days and they're so we had a bug a couple of years

ago when we started using with docker and someone ran sudo docker rm you know to remove the container

after it executes and then it mounted an nfs share that had a show and then it deleted parts

of the show but we could catch up it we caught up it on time that must be like five years ago i think

was a bug in docker itself uh but uh yeah we do have you know a big big nfs storage and we have

policies for backup so i do think we are covered there uh but you know some of our data centers

use tapes for backups it is a lot of storage uh but yeah i don't know we have software to do the

operations like an artist who don't you know as artists don't have root and they won't be able to

or MRF, a show partition, for example.

They have to use some tools to do this cleanup and et cetera

that check their permissions and whatnot.

Everything's tracked, or almost everything is tracked.

Spad, I don't know if you want to add to it.

No, I think you covered it pretty well.

I mean, the tape backups are definitely an interesting area.

Fun fact about the tape machine, that's actually the inspiration from,

I don't know if you guys have ever seen Rogue One.

the uh the movie do you remember the giant machine that took the tape out

and she looks at it in the movie do you know what i'm talking about yeah that's actually

inspired by our tape machine fantastic basement is that rogue one behind you or i mean that's

not star wars oh no that's the last one episode nine rise of skywalker yeah it's my first credit

on the star wars movie uh i think james you signed here somewhere yeah yeah you got you got

everybody up there to sign it yeah yeah yeah uh so that was my first star wars at ilm and uh yeah

we did some pretty amazing work there on the visual effects side so that was a nice inside

joke though james on the um getting that in there on the process for the visual effects folks just

just a little nerd out moment it must be quite awesome to be credited on the star wars film

yeah totally they're for eternity yeah definitely so how does how does um how does someone get into

this unless if they don't have a master's degree i mean andre you said you found your way in i mean

so let's say someone knows python what like what advice would you give them or what you know how

do you go the technical route into yeah well then what do you what do you look for how do you screen

like what what's what kind of so i think visual effects has a lot of different departments uh for

example spad mentioned he started as a technical assistant which is a ta you don't really need a

lot of advanced python or visual effects to get there you mostly need to script stuff that's

usually the entry level of the industry where you you know you're going to be tasked with ingesting

some meta some tapes of some data from the client into our pipeline watching some render jobs seeing

if they're having issues and bugs taking too long and then you start scripting some stuff to make

your job easier and eventually you make your way through the pipeline as a pipeline td which is a

pipeline technical director or in core pipeline our group or in rnd there's usually a little bit

of mentorship there and depends where you want to go some technical assistance assistants end up

being uh going to editorial or being an artist but i think that's the entry level however if you do

have some experience with software visual effects is always hiring uh people and if you don't have

visual effects experience that's totally fine i'll say just go to like ilm vfx careers you can

just put on google we are hiring right now across all levels uh to join our team uh if you know if

You have the traditional backend, full stack experience.

I'm sure it's going to be a huge asset to our team.

And we can teach you the specifics of visual effects.

We don't expect you to know that.

Yeah, hopefully that answers the question.

Spad, do you want to add something?

Yeah, I can expand on that.

ILM has a wonderful culture of fostering talent and empowering people.

You know, as Andre mentioned, you know, I started out working midnight to 10 and I the most Python experience I had was writing scripts, you know, that monitor the farm and fast forward seven years.

And, you know, I'm writing Django services, deploying with Ansible and Docker.

All of this was learned through ILM.

So we are always looking for people at all levels.

And we love growing with from within.

So apply.

Yeah, that's great to have training.

mean even at megacorps for traditional software it's pretty rare to actually have it from what

i've heard but sorry carlton go ahead well i was just going to say there's nothing there's nothing

visual effects specific about um you know deploying with ansible say you know doing devops

you know spinning up a new server provisioning it these kind of things there's nothing you know

fx about that so you don't know you don't have to be an fx artist to do that stuff

totally i do think it helps like if you have you know you know how a visual effects pipeline

actually operates or you can talk with artists and understand their pain points like it is helpful

right but it's something you can learn uh in the job and uh you know we have product owners for

some of our products internal products that will be that interface with artists and technology

um yeah you got to talk with a lot of people we're very collaborative so yeah if you are

interested in this domain i can say it's pretty fun uh you know i wasn't even aware it existed

and i absolutely yeah had a blast working on this domain or have it's not your boring

crud app that a lot of devs complain about oh yeah all the web devs after a couple years it's

Kind of like the same thing.

So on the career thing, because James, so you've been at ILM your whole career, but Andre, you started off in hardware.

You've been at a couple other visual effects places.

I guess I'm just curious.

So in traditional, you know, in web software, people kind of bounce around a lot.

It sounds like maybe at ILM, it's not so much the case.

But you, Andre, because you came in a different route, you've made that transition.

So I guess I'm curious, what causes someone to move around, right?

Is it new challenges?

is it new learning environments or what's i mean i assume salary as well like what what's give us

a sense of that world and the career tracks um if you're not from the inside well no if you if

you're not inside ilm so you've like how did you first get into vfx and then you've you have been

it like when you gave your talk you were at a different effects house at method studios yeah

i was there for i think a year and a half at the time uh yeah i don't know i think i was just

looking for a job i was working on a startup at the time in london called open desk and we were

having some troubles uh financially and i started looking for a for a job and uh you know i think

i was posted on stack overflow careers and i was like what i can use my django and docker and python

to make marvel movies that's insane so like for me it was like an instant apply right and i just

wanted yeah to apply the knowledge i had as a software engineer uh into the visual effects

industry and for me it was just mind-blowing that i was able to do it and you know when i joined

method studios was a little bit of a shock because i think visual effects doesn't have a lot of people

from outside the visual effects world building software and like it just felt like you know

they're not following the best practices and etc but they were very open for it and i think you

know it did a very it was really awesome and rewarding uh to push the company forward in the

software engineering realm and i think the same at ilm too you know when i joined i feel there

there was a lot of great stuff in place but there was a lot of great opportunities for improvement

too and i think that was the challenge how do we bring this you know iconic 40 years old company

to the future with kubernetes cloud and all the buzzwords and we we did it right we are running

workloads in kubernetes and the cloud and it's just for me has been amazingly rewarding uh being

able to contribute to the companies while making visual effects right like our product is actually

something that i watch at night when i go to cinema and i enjoy it and i can say hey i saw

this before or i know who worked on this in the credits right i think it's so you're actually

reading you know as the credits go past it's always always staggers me the number of people

that are involved in making you know even a half hour program on you know whatever like but a full

movie it's just like hundreds and hundreds of people small subset of the people like usually

not everybody gets credit it's a little bit of uh you know the client offers a certain space and

then you put the people who contribute the most and whatnot there's a lot of uncredited people

too that are also a crucial part of those movies it's an army it takes an army so what would be

so you know people who use django but are somewhat outside what would what would improve things for

you if Django, like async, like if you could wave a wand and add something to Django or to a web

thing, what specific to your workflows would be helpful? I feel like we're still trying to figure

out how to handle the UI side of things. You know, we build a lot of our front ends in React

and then use, you know, Django REST framework for interacting with that. We don't use Django

templates too much, you know, I feel like there's

some areas that we want to explore on that front.

Like I heard you guys talk about HTML

um a couple episodes ago um htm or htmx htmx so i think that's so i think that's just one area we

need to explore a bit more i think from my side when i see visual effects people trying to get

into services is can be a little bit intimidating i think you know the deployment for example oh do

i need unicorn i need to know nginx i need to like i feel like flask for example makes it very easy

to have like a 10-line script that does the one function

that people need it to do.

And I really love how Django is batteries included

and people don't need to think about that.

But almost like I want, I would love if Django

could also have like a simple mode or something like that

that makes it, you know, I think maybe in the quick start

slash tutorial of the Django page,

you could have another one that is like, you know,

single page application with Django

or single Python script application with Django

because you do have that in the doc somewhere.

yeah i don't know i think for that yeah well that two things one is that we we we django could and

should improve the official thing and the second is carlton gave a keynote demonstrating this and

i have a package showing actually you know instead of six lines it's what 15 lines to to do with

django so it's possible it's just not typically used in that way so it's more of a marketing

problem i think than anything else there's two there's two templates i'd like to create so that

when you do start project you can do these and they'd have to be just put on github i just haven't

got around to actually putting them on github and then you could point the the start project

command to them but one is single file and the other is single folder because you can that sort

of sync when you start app or start project sorry you can just turn it into an app by adding a

models file and adding the the name of it to installed apps and it turns into a fully featured

app but you haven't got then the project folder and then the app folder next to it which is like

there seems like an awful lot of files for just one model that i want to create like why is there

some why are there so many files here like and obviously as the project scales that starts to

you know that starts to pay dividends but i think there's a lot of room for these kind of two because

the other the other example that i that makes me think about this is people doing data science with

pandas and you know numpy and all the rest and they've just they got their data set they got

their thing they just want to quickly wham it online and they end up using fast which is great

but from a django perspective why aren't you just using django well it's because i

there's too many files like ah we should have a better story there i think uh yeah there's a bit

of analysis parallels even like how do i name stuff every time i start django it's like should

They named that project the same as they have.

It's like I always get into that rabbit hole.

Yeah, one thing we started doing on our team

was setting up a cookie cutter template.

And I'm hoping that crosses to all the other teams

so that way our projects are standardized

and everything looks the same.

That does help as well

because you get people able to move across teams

and work on a different project.

And because it looks the same,

they don't spend that half a day going,

ah, where is everything?

Yeah, we've recently had a discussion about starter projects in Django, because there's a couple that are paid now that usually add billing and stuff. But there's Cookie Cutter, and then I have one, Django X. And I think Django itself, if there was a starter project that could be deployed, because part of the problem is, you know, it's local development in Django. And it's quite the, since I teach this, it's quite the leap to get to deploying something.

and that loses a lot of people so even if it was something that was just a switch you could toggle

um that had some basic stuff that you could deploy the um what is it the polls app carlton you know

like there is a there is probably a way to to do that um or certainly to have something where it's

just environment variables and it just switches debug secret key and um allowed hosts which gets

you most of the way there yeah and it's it's interesting too on the low in the barrier of

entry right i think django does a great job at saying like you know hey in production change

these hey this is not safe for production but then when you have people just starting off they're a

bit scared and intimidated like oh my god i'm doing my local dev but none of this is safe for

production so like what am i doing where do i go from here and i think maybe just saying the

consequences too and like it is okay if your use case you know it's simple you're disconnected from

the internet you're doing some local dive in a data center as a you know data scientist or visual

effects artist like you know we are not touching the internet from our data center so a lot of it

you know matters but doesn't really matter as much as if we were open to the whole world with

our app we're really trying just to you know develop a microservice to help with this workflow

thing here uh that could just be a python script in a server that iscp and you know right the system

did it start i don't know i feel there's a lot of potential to simplifying uh and not yeah yeah

there's a you know there's the docs versus tutorials uh dilemma which is that django has

docs tutorials are separate what you know people want tutorials it would be nice to have django

tutorials but they're a little bit more opinionated than the docs so yeah but also there's that

problem with open source projects in that people people think of a project like django and they

think of it like an organization like industrial light magic and it's anything but it's like half

a dozen people giving their spare time and you know it's it's not it's not a corporation it's

not a thing that has capacity to do everything so it's a question of what gets done in the time

and it's interesting but yeah for what it's worth we don't also so yeah

it's that bit is that what's the business case thing as well right right i do think for visual

effects and machine learning and data science as well and uh you know django has started recently

or you know maybe i haven't followed much sorry but uh all the async stuff and doing long running

processes for example we usually use django with salary and setting that up and the infrastructure

can be a little bit too much if there was like a simple thing too that you know maybe bake all of

that in and you know not performing the salary or scalable but at least it you could run some

async stuff and some sort of queues i think you know fast api has a class that does that i think

tornado has something that does it too because it's just a sync io and flask granted i haven't

been following that too much in the django side it might already be there but i think that yeah

no i mean starlet the the framework which underbinds underlies um fast api it allows

these background thread background tasks where you can just sort of spin it off and go and what

it doesn't have is like the retries and the you know task acknowledgements and all those kind of

extra things which is when you need a proper cue and yes celery is the big beast in the room and

it's hard to it's old and skate gnarly and whatnot but it's super powerful if you can get it

it's a difficult one it's a difficult one there's been many attempts at a simple cues um but yes

that's another one that's always sort of on the list as a as a goodie james can i ask with the

um the permissions on the web guis do you how complex do you all get with your permissions

are you using like groups in django are you doing at the api level like how are you

setting that up if you know yeah so um typically like our django apps they um they'll interact

act with some of our internal apis um and they usually handle a lot of the authentication um

it doesn't as andre mentioned like a lot of our tools don't touch the internet um and we're a very

open studio so um i think it depends on the application itself um like for example that

subscription management uh tool i was talking about our permissions on that are open so users

can log into other users and mess with their settings but you know no one does that because

it's a internal company and you know we don't hire any trolls that go in and mess with those so

and having that open atmosphere in our tools um makes it easier for us to debug things so we can

have somebody on our production team help a supervisor and just log into their account and

see what's going on if that makes sense carlton i'm trying to remember there what's there's a

really nice third-party package in Django where you can uh log in as any user uh yeah yeah Django

hijack is the one I would use if I um that is a great all the time someone says it doesn't work

suddenly thinking it sounds bad but yeah that labels you an admin a staff user to log in as

another account via a button in the admin and then they can just sort of log out of that and

And that can be really important because you've got a customer who's struggling and they're like, this is what I'm seeing.

And you're like, but it works for me.

And to be able to step, you know, with their permission, obviously, step into their account and step through the things and see what's going on.

That can just, that can solve it because you're like, no, this doesn't happen.

This is fine.

This works.

Silly customer.

And until you actually do it and then you're like, oh, there is a bug.

You know, they've got an accent over one of the letters in their name.

And of course, that crashes the app because, you know, we're Anglo-Saxon programs.

We never considered that, you know, any number of problems can come up.

But yeah, that kind of process.

I like what you were talking about there, James, about how it's an internal tool and the way you can balance what your threat model is not, you know, hackers on the Internet.

The threat model is, I don't know, we spill coffee on the server.

That's, you know, a silly example.

But, you know, you can grade, you know, what level of protection you put on by what's realistically something that your application needs to handle.

Yeah, absolutely.

We do have some internal checks to keep people from seeing things they're not supposed to because we do compartmentalize, you know, some projects we're working on.

But for the most part, it's pretty open and that makes debugging very easy.

The software management system we have that allows symlinking also makes it very easy to see what the user is seeing.

So if, for example, an artist is working on their own script and they're like, it's not working, can you help me?

It's very easy just to symlink it from their home dir to my home dir and pick it up.

And then that's like live coding, right?

So you make some edits, it immediately appears back on there.

That's super.

I mean, you know, the IDE developers are only just getting to that now.

yeah yeah we were looking at um vs code uh was it live share a little bit but um since we're

air gapped it doesn't quite work yeah the jet brains folks have very similar um coded me i

think there's this cool we're close on time are there any bits we didn't get in i think just to

stress that we are hiring we have a lot of openings if any of this interests you please

apply uh in find out apply anyways yeah sure all recruiters will get back to you and figure

something out well surely this audience is yours and you're like you can use your django skills to

build like you know marvel films yeah no more no more boring cred apps work on star wars yeah

everybody apply we'll put the link for that in the show notes yeah no we will we will for sure

well thank you both so much for taking the time and thank you i think andre for i think we

connected on twitter someone had put out i forget a tweet about django in the real world and you

said hey we're using it over here and that's how this started so yeah of course thank you for

connecting too yeah no it's been awesome yeah it was a what's the name of the books bad uh which

one the django for will will's book yeah oh yeah django for beginners professionals and apis yeah

those books were huge help so thank you for that i'm very happy to hear that well you should get

you should offer an ilm discount yeah yeah you know they want they want bulk i know i need to

no it is it is actually there are i know for a number of companies that for new recruits you

know my books and some others you know just to onboard people who know python but i have no idea

how to do Django.

There's still not that many resources

actually on it.

So yeah, if any companies want to do that,

hit me up.

I think if you want to get someone up to speed

and like, you know, sit down, read these,

that's the way to go.

You know, your books are absolutely the reasons.

Yeah, I'll plug Adam Johnson's books too

are fantastic when you get to the next.

Sure, but they're next.

They're follow up.

It's a DXN, Speed Up Your Django Test.

And there's other materials,

but um generally people write books kind of more at the level they are at um whereas to write

beginner stuff is less interesting as you progress in your own journey so yeah i just finished his

book it was great yeah anything he writes recommend um again thank you both so much for coming on um

we are at jango chat.com chat jango on twitter we'll have notes to everything in the show notes

so see everyone next time bye-bye thank you bye-bye thank you bye-bye thank you guys

Thank you.