Transcript: PyCharm - Paul Everitt & Aleksei Kniazev

hi welcome to another episode of django chat it's a fortnightly podcast on the django web framework

i'm carlton gibson joined as ever by will vincent hello will hi carlton hello will and this week

we've got paul and alexei from um jetbrains joining us who're going to talk everything

pie charm and django and all such things hi guys thanks for coming on hello hello hey thanks for

having this um thanks for coming on let's kick off pycharm right jetbrains is one of the big um

big sponsors of django cons and and big in the django community tell us about um pycharm what

is it and why should we be using it first and foremost we're an ide integrated development

environment we are not necessarily text editor we try and take all the things that you do and bring

them into one consistent user experience that's all supported by us so that you have one throat

to choke i can't really say that in a podcast so when it goes wrong we know who to come and talk

yes exactly right yeah and uh it's funny since the beginning of pie charm which was a long time ago

10 years ago uh it was positioned as the python and django ide so the django support is mature

it's evolved it stayed current because people like alexei put a lot of time into thinking about it

okay and and before we just dive into the features can i ask a bit more about jetbrains

because i go to the jetbrain site and i see pycharm's just one of the options there

What else does Jeff Raines do beyond PyCharm is what he's trying to say.

Well, we have IDs for most of the major languages.

Maybe the biggest one is IntelliJ IDEA.

That is an ID for Java and Kotlin.

We also have IDs for, well, Python, Go, JavaScript, as well as plugins for Scala and Rust.

We also have back-end services, TeamCity, things like that.

And the point that Alexei is making is it's all built on top of this IDE platform.

It's written in Java, but all the integrations are written with the native language participating.

And so what that means is when we have killer Git support in the platform, everybody gets it.

Okay.

So you don't need a Git client as well.

So it's literally everything.

It's as you say, it's one throat to choke.

That's good.

Okay.

Well, yeah.

Well, we want to talk about the Django specific features,

but maybe if you could just fill out the batteries included,

because I know you have database support.

So for example, like I use Visual Studio Code primarily,

but I've also used PyCharm.

What are the, I know there's a bunch more that PyCharm does.

Like just what are the high points of what, you know,

beyond an IDE that it provides as opposed to Visual Studio Code or Atom?

Right.

So we have PyCharm Community Edition, which is free and open source, but it's aimed at Python.

PyCharm Professional Edition gets you full stack.

We bundle WebStorm, which is our ID for JavaScript, HTML, and CSS.

We bundle DataGrip, which is our ID for SQL and databases, some remote development stuff, some things like coverage, etc.

And with PyCharm Professional and kind of the whole full-stack thing,

we say full-stack from a programming language perspective,

but we also mean all the activities, running, debugging, profiling, coverage, VCS, database,

all kinds of things.

Virtual environments, right.

Yes, that's a very good point, trying to ease the onboarding.

Well, because I know there's academic licenses, because I think three years ago, I visited a local university that was using my book.

And I remember asking them, like, how is the setup?

Because I know that's challenging for people.

And they all said, well, we're using PyCharm.

So it wasn't an issue.

And that was the first time, actually, I was like, oh, I didn't realize the full featured nature of it.

I was like, wow, yeah, it's a lot simpler than rolling it yourself.

Indeed.

And you mentioned academic.

It's also free for open source as well.

and we have kind of startup discounts.

You know, we're still a human-sized company,

and so we kind of act,

we treat people the way we'd want to be treated.

Yeah, well, and we should mention the,

so Django has done the last couple of years

an annual promotion with JetBrains Pie Charm

that's going to start in April,

which is about when this comes out.

And that's the biggest corporate donor

to the DSF by far.

Yeah, we're proud of that. That started a good number of years ago. I think it might have started

I think five.

Yeah, like right when I joined the company five years, the previous PyCharm product marketing

manager, Dimitri Filipov, is the one who started that. And it really has raised a good amount of

money. And some things fit people's brains. Some people don't fit people's brains. This seems to

fit people's brains. Yeah. I mean, just in terms of specifics, it's generally about a quarter of

the DSF budget. Um, so Carlton's one of the fellows that's most of the money goes to Carlton

and Marius, who's the other fellow. Um, and it's generally around a quarter. So it's anywhere

between 30 to 60,000 a year, the last couple of years. So very material. And just cranking up

again, what next month? Yeah. So it's going to start the month of April. We're working with

roberto um and what what's the details of the of the offer it's like what 25

50 well it's the same as before so it's um 30 off with the proceeds going to the dsf

for the month of april if you go through the link fantastic did i say that right paul

yeah and uh to be to do the kind of full honesty perspective how in the heck can you

pie charm giveaway something for free well we're a subscription and so we hope to see the upsides

in the future years if people are happy with the product but it is true 100 goes to dsf yeah and

that's partly why the dsf has done it and it's the only corporate sponsorship of the type that

the dsf does in part because the dsf exists kind of to exist not to make money so as long as we

cover our costs for hosting and fellows and conferences that's that's the goal of the

organization but anyways enough about the dsf let's talk about features let's talk about features so

can i tell you my my sort of top favorite ones from pycharm my my absolute favorite one which

is closely related to my second favorite one my absolute favorite ones there's a little down the

left hand side there's a little structure um button and you you click it and up comes this

sort of class diagram of all your you know your nasty model layout and it's like that's fantastic

that's what i've been looking for for years and you can move around it and you can and it's even

got a mini map so you know when you've got far too many models and they're all joined up in

horrible ways you can click around the mini map to navigate that's my favorite feature my second

favorite feature is in the gutter um down by the side of the editor you've got this little button

about um for and you click it and it shows you all the subclasses and all the superclasses of

the class you're dealing with so i don't know you've got my custom http redirect response nasty

thing and you're like oh wait i need to see what the the base information you just click that and

it's like oh here's all the superclasses you're like that's the one i'm after and you're there

and that so those are my that's my favorite and my second favorite feature of pycharm

that's just an opportunity for you to know how to spin off for 20 minutes and tell us all your

favorite bits well yeah that's that's really hard to overestimate the importance of uh being able to

easily navigate all over your code base. We are very proud of how it's actually done in PyCharm,

and what I really enjoy is that it can be done not only through just a Python code,

but just navigating to your templates from your view to the view function from the template and

to the variables that were passed to your template that's one of my favorite right so if i've if i've

got extends base extends you know skeleton extends yeah exactly and these magic strings and config

files or route declarations and things like that essentially we've taught the magical django

strings and conventions to be symbols as if they were python and once that's done

all of our ide machinery kicks in so that's where you get the autocomplete for model options and

and refactor rename and navigation and all that stuff find usages okay yeah autocomplete is

actually a whole whole story in itself because uh well part of it you get from just uh just a

python code but due to django's nature that you can't always have this information from simply

python that's that's where django specific knowledge gets handy if you if we know your

models if we read your code and understood it we can give you like autocomplete for foreign keys

for example that are not in the in the current model but might be somewhere somewhere else and

in many cases the opposite of autocomplete which is squiggly's telling you you got it wrong or you

changed the end it didn't change this end yeah that's the thing i mean sorry go ahead carlton

well i was going to ask how and so this is something that's come on um over the last couple

of years in the vs code world with um with the um what they've got the pylance extension here which

uses their um the pipe the pyrite type checker and how are you able to say oh you know we think

we're better in this regard or we think we're better in that regard like how would i compare

the the autocomplete from the two processes because you've got a lot more like you've you've

got 10 years of building the thing yeah true uh it's hard it's hard to say for sure um i still

think that we're somewhat ahead in this regard and uh well we try to evolve of course and

having uh 10 years of uh django support uh in the in the id it sometimes it's it takes

some work to how to say it better well to make it work with the new features okay and uh we

we adapt and evolve and in fact the imminent upcoming release about this had basic yeah

had a basically a full-time pie charm person lana she worked on we think we're great at

autocomplete and lana's like actually and so like total rethinking of the experience for

typing a couple of characters and getting a deeply nested import generated for you

which is pretty badass because i hate stopping what i'm doing and going and typing my imports

just hate that. But just in general, it's fun to see the rise in smart editors. It's fun to see

the rise in Python providing more information to help smart editors. Whether it's fun to see

machine learning giving suggestions for you is, I'm an old dude. Get off my lawn, I know what

color me a little skeptical on this one but the ability to not just give you a list of completions

but a short list of correct completions i mean this is a challenge with search i i gave a talk

at django con the last one two years ago on search and the problem is everyone assumes google is the

baseline and google is a pretty high bar so like it's like when people complain about search i'm

like why don't you try building your own search and then you'll start to see you know exactly as

you said it's like teeing up the right one is is quite challenging but but you know it when you

see it right back in the day i had when i was doing cms stuff i was i converted a customer to

use postgres full text indexing and i thought this is gonna be great it's transactional and

they're like no it sucks great it does search it just doesn't do it astonishingly yeah can i

ask about um so databases this is something i've been really impressed by pycharm the integration

there so i um and specifically docker so like in one of my books i use docker because it is

challenging to set up postgres in particular on mac versus windows and people get that wrong all

the time can you talk a little bit more about the database integrations in pycharm specific to django

uh well for database integration we actually bundle data grip which is our ad that is specific

for databases and um yeah so this is bundled in uh by charm professional together with james support

and once we uh have the connection to your database once we know where it is it's actually

uh so we can provide your suggestions for for example for your column names we can uh

highlight to SQL queries and stuff like that.

Yeah, it's incredibly powerful.

It's pretty astonishing.

You can be typing a SQL string or, as Alexei said, working on a model.

You can navigate to the column.

You can sit in your Python code and rename your column,

and it will go and actually change your database.

You can execute a SQL query right in the middle of your Python.

It'll open up a panel, a console with the query results.

And maybe, Will, your question was, what can it talk to?

It's JDBC.

So, you know, most anything that has a JDBC driver, which now includes MongoDB, we can provide this experience with.

I wonder, do you do, I haven't looked into this bit, but what's the sort of integration between generating a query set and then, you know, being able to map to the SQL that you're generating and think about that?

Because that's one of the hard bits where people are like, you know, if I annotate this, what do I get?

I mean, you've got secret sauce there.

I don't know.

Do you mean like watching what the ORM is generating?

Yeah.

Yeah.

Well, I mean, because you can always, you can, so you can get query set explain, which was new in, I don't know, three point something or other.

But it's one of the hard bits to, you know, when you get into advanced queries, it's one of the bits that people...

struggle and you have to spend a bit of time you have to spend a bit of thinking about how do i

craft my query set exactly to get the the sql that i'm after well colton how do you look at it right

now so outside of pycharm what's your way to visualize it okay so outside of pycharm i got i

get in a shell and i go query set and i start annotating and i go bam explain and i see or i

go query dot query and i get the the generated um sql string and it's fine yeah it works but it's

not but it's not smooth it's it's a bit carlton let me ask you a question do you think that string

that's being generated is all the information available static from static analysis or does

it require some kind of runtime thing i'll bet it is available from static analysis yeah yeah

because you um like the the the sql compiler is you know it doesn't actually hit the database

particularly it's the parameters that you need but like you know when you once you've got the model

defined once you start filtering and you add on the columns or you add the annotation

the actual compiling because i know alexi came onto this podcast thinking you know i'd really

like to add a new to do on my feature list wouldn't it be cool if you could do alt enter

and convert a SQL ORM query to a SQL string?

Yeah, sorry, I'm kind of spaced out.

I'm thinking how we can implement this.

Okay, so it'll be out by the time the podcast is live.

Okay, because that's one of my bug bins, right?

It's one of the more difficult bits of Django

is when you're crafting these complicated ORM operations

and you've got a nested subquery

and then there's a window expression

and exactly, you know, if I add in that rank function,

what's it going to start producing?

It's like to see that live, that would be, you know,

that would really change the game, I think,

to sort of be able to iteratively build your query set calls.

That's a pretty good idea.

That sounds like a hackathon idea, Alexei.

We have internal hackathon contests.

I think what Carlton's getting at is, to me,

at least, I mean, query sets is sort of really what you do at an intermediate advanced level

in Django. That's really kind of where most of the focus is on, is figuring out how to optimize them,

how to craft them. Kind of the rest of it, I would say, is not as advanced. I don't know.

That's my take. Models are models. Views, templates, URLs, it's a little bit of whatever.

But the query set is kind of where it's all at once you get beyond a prototype, I would say.

But I like sweeping generalizations, so.

Yeah, well, I mean, it's true, right?

It's true.

Well, we're in the business, or I'm an advocate, developer advocate,

so I'm definitely in the business of sweeping unfalsifiable generalizations.

There you go.

We certainly, as a product, need to understand who we are for,

who we're not for, what parts we have covered,

what parts we need to be more competitive at.

Well, I don't think anyone's doing a good job on the SQL front, so.

Sure. And so that could be a good space for us because targeting people like you, Carlton, where it's like, okay, the easy stuff is easy. How about giving me some help on the hard stuff?

Yeah. I mean, it's kind of bifurcated, right? There's the beginners for whom, if they have access to PyCharm, which I guess they do if they're a student, it's solved so many problems. And then there's that middle ground. And then there's the Carltons who are like, yeah, these advanced features. I suspect those are the two barbells of where you kind of focus. Is that accurate?

Indeed, and the Python developer survey that came out recently showed that there's a question about how long have you been using Python?

And the answers were like, I don't know, under six months or a year or something like that.

There are a lot of newer people, yeah.

Yeah, it looks like 150% of the people found Python yesterday.

Yeah, I was surprised by that.

It's shocking.

It almost made me not discount the results, but realize how many new people there are in the space.

Because there's a lot of other questions about your, you know, preferences as a professional.

It's, you know, it's a, whatever, a third new people.

You know, we actually, that Python survey, Django did our first survey in years last year, really modeled off of that Python survey.

We're going to do it again this year.

So actually, if you have any suggestions on what we ask around IDEs, let me know.

Okay, sure.

Because we'd love to, because we don't track anything really about Django.

So the survey is a way for the, you know, fellows and the technical people to kind of know outside of Django cons and message boards what we should focus on.

If you'd like to, let's set up a call and get you with our research team that can talk about the mistakes they've made over the years with the survey and what they've had to change to make it statistically sound and any other tips that they could give.

Okay, that'd be great.

Yeah, we can do that offline because we had we had thousands of responses and it was literally a Google Doc I set up.

Yeah, so we could do better.

So I'll ask this question back to Alexei

in regards to kind of that dumbbell thing

that Will was just talking about.

In looking forward to the rest of your year

when it comes to Django,

do you think your activities will be more about beginners

or more about advanced?

It's hard to really pick a side here, because what I'm planning for the Django,

well, part of what I'm planning is actually aimed more toward beginners,

and part of it is, well, should be useful for both categories equally, I guess.

So we plan to do some work enhancing the possibility to view Django documentation from inside the PyCharm.

We already have this possibility and I really like this feature, but it's kind of obscure because you have to enable it specifically in settings and we don't promote it much yet because there is some work to be done.

But when it's finished, I think that it's going to be pretty good.

Because we have the ability to go to your class hierarchy

or to a symbol like in your settings file

and check the documentation that comes straight

from the Django documentation website

from inside the PyCharm.

And it doesn't matter where you do it,

if you do it for your views or for a string

in your settings file.

you should be able to build that locally right because and for the particular version of django

in the in the in the virtual environment so i'm running you know on this particular project i'm

running 2.2 point whatever i've i've got it yes i can pull the the documentation for 2.2 point

whatever and i can have that demonstrate i mean so i use dash which is i think just an awesome

thing and i have you know i have it open all the time for documentation um but one thing it doesn't

do is allow me to version by project right i've got what he's always got the latest one but

sometimes i'm not on the latest one i need to see and so i have okay i have to go to the website and

okay it's not the end of the world to have to go to the website but it's not you know right there

and then so if you could pull it locally and build you know rumry sphinx and yeah so the dream is to

have it not only adapted to your version but to the specific uh element that you're searching for

to save you like at least some seconds of looking through the through the page yeah i mean yeah if

you index if you can index properly and pull out all the those annotations that get you right i

know i'm looking at http response where's the docs for that yeah yeah so that's the advanced

feature right because you know beginners they're too scared to look at the docs they don't do that

they you know yeah that's kind of well that that's my thing is that docs and tutorials are different

things by design. So there's, they go there to be confused and then they come to my educational

stuff. But I think that is valid, right? I mean, cause I've thought a lot about this, like, should

I, what is the difference between the two? And the difference is docs are designed for, you know,

Carlton has a question or I have a question and we go and see, oh, this is how it's defined as

opposed to a tutorial or a book that says, let me start from the beginning and go through everything.

and that's not what you need if you're you know more advanced and it's also so much work for me

it's like hard enough to do my things let alone that you know the django docs are thousands and

thousands of pages so so there's a place for both i guess is where i've come out after a number of

years of doing this and so alexi what was the what was the other thing what was it because you have

definitely two things you're going to tell us yeah uh the other thing is actually a pretty cool thing

that comes to us from the platform.

So sometimes it happens that we have an idea directly

tied to Django.

Sometimes it happens so that amazing people that

work on the platform level have a cool idea

to implement across all IDEs.

And in IntelliJ IDEA, we got this thing

called endpoints, which is basically your URL endpoints.

And we have this for Django, like for your URL patterns,

but this is so much more powerful

when we can know your actual endpoints

where your resources are located.

We can do navigation.

We can do auto-completion and not just in your Django code,

but like in your template, in your JavaScript code, anywhere.

We can build diagrams of how they interact with each other.

so i'm really excited about it so i'm thinking i'm trying to build say um a dictionary for a

request body for an api endpoint and i need to do the request body and that will be kind of

auto-completable for me uh well the the resource um the actual address of uh of your view view for

example will be uh indexed and will be available to you across your project and maybe even across

different projects that you have opened in the same frame so for example you have two applications

that need to talk to each other we can uh since we already tied your uh like templates and python

code to each other why not tie a different application that are meant to talk to each other

so like microservices kinds of things or a client server example yeah yeah that's amazing to see

like a video demonstration of that because i i think i know what you're saying but i'm sure you

have stuff in the works to i'm exactly like you it kind of bounced off me until i saw a demo i'm

like are you kidding me how do you know that how do you know how to complete that yeah i mean i

think i know what you're saying but i'd love to see it just to okay yeah well actually that's it

Go ahead, Carl.

I was going to say, what sort of timeline?

That's sort of this year, you think?

Well, I'm hoping to finish it this year.

It's shipping in the platform for the Java side.

It's that not only does each IDE need to wire it up,

you've got to wire it up to Swagger OpenAPI,

or you've got to wire it up to Django,

or you've got to wire it up to some GraphQL thing or something.

Okay, fantastic.

Alexei, is that right?

yeah so can i go on can i cut um cut bands slightly because i wanted to talk a little bit

about um so from the ide developer perspective talk about um some of the features of python

and then their applications django particularly typing right because django is really dynamic

by its nature and it's really hard for us to throw in um type hints where you're just going

to end up having any all over the place because yeah and at that point it's it's not worth the

you know if you've got a nice new thing it's tight and it's you know you know exactly the

types and type into beautiful but something like django it's much more difficult what are you

but yeah obviously those give you more static information which enables you to make the ide

better so you know riff off that how where's the ground and where would you like to see django go

and what do you think is possible and um well that's that's true that for django it's harder

because there's so much more dynamic stuff going on.

There's keyword arguments,

there are attributes that are not really declared but added later.

So it's really hard to express statically.

Some of it might not even be expressible statically.

But we've got a very good type checker that works off of Python,

And then it has the ability for us to plug into that with our specific knowledge of Django and say, like, I see you have a filtering function here.

Probably this is related to this model.

Let's have autocomplete here.

Let's have keyboard arguments here.

So it's always easier to do for stuff that is statically expressible, but I understand it's not always the case, and we try to do our best for Django where we can.

There is this project that is stubs for Django.

Yes.

Which is pretty great.

And I'm hoping to one day use it for our own analysis

to kind of enhance it.

But so far, it's been kind of hard

because of the dynamic nature of Django

and because the stubs project is tied directly to MyPy

and has its own MyPy plugin.

That was a Google Summer of Code project for Django last year.

So because some of it is still done dynamically,

we kind of have to figure out a way around it on our side.

But I'm still hopeful.

I mean, there's one example.

In fact, you mentioned that Summer of...

I'm sorry.

No, go on, Paul.

Go on, Paul.

You do.

You mentioned that Summer of Code.

If I remember correctly, one of the first...

The motivations for the DSF campaign with PyCharm, the fundraiser, was the money was going to go partly towards Fellows and partly towards the Static Time.

typing uh work yeah no this was years ago no it still comes out it comes up on the main list and

it didn't you know didn't jet brains didn't they fund some work on it and it's like yes but how is

it to be done but it's hard it's you know we have this sort of idea that hey there is a chunk of

money in theory available if we can do this but it's just not possible well it's not it's not

that it's just not possible but it's a massive undertaking and you know maybe we could smuggle

in some bits in the internals where it's more tightly constrained and then you know slowly

do something but it's interesting for me i spent a lot of time in the front end world as well

i do big tutorials on react and typescript and test driven development and watching typescript

happen yeah it's been fascinating to me if you thought that the python people enjoyed loosey

goosey javascript's like hold my beer and yet typescript is now really popular in the front

end world but in the beginning i would read the typescript roadmap and i'd be like okay first

i don't understand any of those words they're using these type theory words that i don't

understand and second i would think okay that's for angular that's for angular angular is the

only one that has that problem angular is the only one that has that problem and gradually they've

expanded expanded the scope of their type system to be able to allow things like angular and large

scale react things to give a fantastic developer experience statically yes static analysis and the

world of python just doesn't quite seem to be in love with static analysis the way that some of

these javascript people have yeah i think that's true i mean you know there are some big advocates

you know called python core developers are very big advocates but equally though

you know people are very big in the python world who are you know vote quite vocally

against going too far down the road um i think there's one thing from the peps which is

um important which is that type annotations were always meant to be optional even by convention

sure even by convention yeah and there's a danger that we get to the you know sometimes when

maintaining django there's pressure to put type annotations in where it's like but they're meant

to be optional and you're not feeling that as a maintainer you're feeling like no they're compulsory

and you're falling behind the times by not putting them in it's like oh hang on the peps say they're

optional even by convention so i don't know it's curious let me ask the two of you as you do these

podcasts as you go out and talk watch what's happening and the zeitgeist yeah the world of

django is what percentage on board with using my pie adding type annotations

maybe 30 i i guess like and i don't mean the core team i mean just in the wild i think people are

sprinkling them in um i think using the using the type checker is not as prevalent as it can be

because if you're not what's the point of an annotation if you're not going to use the type

checker well the ide gives you better order complete that's why you know i i'll just

i think people aren't using them too much but again you get um you get these quite because of

the way Django is written you get these kind of horrible annotations you know like I was writing

some yesterday to see if I could get autocomplete to work so I've got you know it was in a pytest

fixture and I was returning factory function from the pytest fiction and I wanted autocomplete and

he wasn't giving it to me so I spent you know 10 minutes 15 minutes writing this annotation

I still didn't get autocomplete and I was like but it's horrible so I'm deleting it

um and I don't know why he's nodding I think you know this this problem with making autocomplete

yeah but and then yeah absolutely you know i i program in elm and i program in swift where you

these things aren't optional and where you've got an ugly annotation you create an alias and then

you just use the alias and then it's slightly prettier and there are there are things which

i do without thinking these other languages which i'm kind of reluctant to do in python i think i

need to just get over that so i don't know about in general i think it's coming python in general

is is increasing it but i mean django is maybe not the right example because it's got we've got

this big history and indeed and this big like the core of the orm you type you try type into that

like yeah sure um it'll come i don't know i i don't see it as you know with beginners

intermediate i don't obviously see it as much because they don't feel the need for it um

i would echo what carlton said when i talk to professionals it's pretty split some people

are really into it and some people are very wary of it so it does help you reason about your code

If you take the time to think, what's the type that's being returned here?

Oh, well, this is a callable that takes these parameters and returns this.

And in fact, it's got a generic variable because I'm using it different times.

And if I genericize it, I can create one version of this instead of four.

And all of a sudden, you've refactored your code because you spent a little bit of time

thinking about the types.

And it's like, ha-ha, that's a win.

I mean, it's a bit of a religion.

You sort of need to have the problem, spend the time, come to the solution.

and then you're proselytizing for it.

But it's hard to say

what percentage of Django people

that applies to.

Well, so I have a question for the...

Go ahead, Paul.

No, no.

I stomped on your segue there.

Go ahead.

Well, I was going to ask

for the two of you.

So there's so many features

in PyCharm for Django.

How do you think about

prioritizing them

and when do you readjust

the top three, top five, right?

Because I assume

the first question is well what kind of programmer are you are you a student are you a professional

um so i guess you know you keep adding features but you still can only hit someone with like three

to five killer features what is that process like um because i imagine that's a challenge

uh that's that is actually a big challenge but um for one thing we have an open issue tracker

so it's always possible for users that have some have a need for some feature to come up to us and

say hey i would like to have this and we always look at what our users want we uh talk to people

at conferences we try to find out what the workflows are uh try to think about how we can

actually enhance it not just by because maybe there is a better way that would be

that would include this feature in it but would actually deliver more so we have a voting system

in our issue tracker so that's that's basically it so we we talk to people we encourage people

to talk to us about what they want

and try to stay up to date with what's going on.

And how do you take stuff out, right?

Because you don't want to be, sorry to interrupt,

you don't want to be AWS with 300 config options, right?

And that's always the challenge is you keep adding stuff.

It's really hard to take something away.

But if you don't, then it's a bloated mess for a new person.

That's true.

I mean, this is indirect to anything.

I agree.

It's been a challenge for us,

and I would argue that it's still a challenge in some areas.

We still have some support for some things

that we would probably prefer to take out,

and I hope we would in the near future.

Usually when we remove stuff from PyCharm,

we extract it into a separate plugin,

And open source it if it wasn't already open source so that people who, yeah, so that people who still would prefer to use it can still do it.

And if I can add to that about how decisions are made about what to do, before each release cycle, there's a two-headed planning meeting, one a retrospective to look back at the previous release cycle, and then a planning the upcoming release cycle.

And they actually do produce a list of top tickets with votes.

And that is an important input to deciding what gets done.

But one of the things we're learning is that we have a lot of features.

We have a lot of features.

We could probably go and announce new feature in 2021.1, this thing we implemented five years ago.

yeah and people would probably it would probably probably could get away with that

so that's what you were saying alexi there's like so many hidden killer features but you can only

front page a couple to people exactly so we're going through a big effort in the company to

surface functionality to make it so you don't have to know it's there it presents itself to you

in a subtle non-irritating way at the right moment like clippy she didn't say clippy

yeah well carlton that's i mean so you know dealing with deprecations and stuff that's something

you know jango and you spend a lot of time on trying to get right yeah i mean you can't just

break stuff i mean you can't if you're jango you can't just as much as you want to right at night

you're just like i'd love to kill this thing there are bits yeah i take like the whole of um

django.core.mail i just like you know control delete that and that could be gone and people

find their own solution but we can't just we can't take that whole of django as much as i'd

like to so okay that that lives and we maintain it and we we keep it and we make a change there's a

the deprecation policy and the deprecation period and all of those things if you're a mature product

you you've got to you know you've got to maintain that um i mean you know so this has come up in the

typing conversation to just cut back there is we could make the types tighter in general you know

it's meant to be a a string you get here we could say yeah it should be a string and then drop the

or any but we can't do that because there are people relying on the fact that you know dark

typing works and blah blah so okay that won't change um i guess the same for if i was developing

an ide what would i think i mean sometimes i still use an editor right so my other thing that i use

is bb edit though i have it open all the time i use it for any number of reasons but one is it's

it's just an editor and sometimes you just want an editor um i don't know how do you

How do you maximize that kind of efficiency zone where you've got the features and you've, but you're not, you know, you're not like flying a 747.

I mean, it's like a mental thing too, because I use a simple text editor.

I mean, it's sort of like a, you know, a five-star chef wants to have a cheeseburger sometimes, right?

Like you just need to balance it out.

but i think that i mean with django thinking about extra features i mean we have a very robust

third-party package system and that whole process of sometimes things can leave jank what is it

something recently major left okay but contrib form form tools was taken out and um there was

the dot browser thing or the data browser thing that was taken out there you know there's no

reason why you can't extract um a package as a plug-in that's a great idea you know make it

It's still available if people want to use it.

They can maintain it.

They can help maintain it.

They can use it, but you're not forcing it on everybody.

So perhaps we could make the core mail a third-party package.

That's a genius idea.

I'm doing the Mr. Burns now.

Yeah, but it is that tension, right, for any project.

And I think the important thing is almost having the outlets,

so having a way to deal with it instead of just saying no all the time

or or bloating out a project um because certainly django django itself like carlton you and marius

and everyone really tries hard to keep it narrow yeah you know it's like one of the ones i get all

the time is why can't templates do more or why can't there be a front end thing and you know

it's a logical question for a beginner i think um but it's also it's already big enough yeah we

can't yeah we can't maintain it at that point so the other thing we do is we say you you want a

feature, build a third party package, and if it gets uptake, then maybe we can bring

it into core.

So you export features to plugins.

You also can say, I think I want my own feature, and you're like, no, we're not adding that.

Can I write a plugin for PyCharm?

Yeah, of course.

Of course.

And there are lots of plugins for all our IDEs for PyCharm as well.

Recently we had a poetry support added by a third party plugin.

and in that case we worked with the person to add a couple of extension points

or to refine our existing extension points so that poetry could be included okay and

poetry is pretty hot hot right now like i get i think the biggest personally the biggest thing

for django beginners is installing python and setting up a virtual environment yeah because

Same for us.

Right?

It's just there's so many ways to do it.

There's still a lack of universality.

You know, PitBamf project has an interesting history.

You know, I still use it, but I debate it.

Ask Alexei what he's been working on for the past six months for PyCharm.

What have you been working on for the last six months?

Oh, my God.

Well, I've been trying to refine our whole story around packaging

because we have a way to install packages from inside PyCharm

that is located in settings, like your interpreter.

And the thing is, no one in our own team used it

to install packages in PyCharm.

So that's a sure sign that something's not going right.

So, yeah, so we started rethinking this whole story.

So we extracted the support for installing packages

into a separate thing we call tool windows,

like the thing you get for Git or for your terminal.

We extracted it there with the possibility

to browse the documentation for it.

And we plan to do much more to...

to tie it to your environment description,

like for your requirements.txt file

to your PyProject tom or whatever you're using

so that if you install package via PyCharm,

you can just not care

whether it's properly added

into your environment description.

What should Python do, right?

The two of you, Paul and Alexei,

you know, if you could make the PSF,

What's the silver bullet to solve virtual environments?

If you just had to pick one, what would you tell them to do?

I've been told by Lukasz Longa that I'm wrong.

So I will go ahead and suggest the failure approach.

There's that pep push by Kushal Das about having a Dunder pi packages Dunder that's like node modules.

So that you don't even think of a virtual environment.

There is no concept of that.

And in a tweet, Lukasz explained why I'm wrong that edge cases and backwards compatibility and stuff like that.

But my gosh, all of these, as a developer advocate, all I do is tutorials.

And they all start with the same thing.

Like, here's 50 hours of janitorial work that I wish I didn't have to say.

well and the problem is they they google around and then they have four different versions of

python because they download it directly or then they did homebrew and this and that and then their

path is messed up and then i have to explain path you know and then they're on linux even though

they're beginner and as much as i want in his head like i've just spent six months dealing with you

know i think it's i i don't we don't want to blame the blame the victim right i think i feel like

there is a solution and and maybe it's there's a 90 case and then there is a 10 case that if you

have these issues you go elsewhere but i mean uh russell keith mcgee you know his keynote at pycon

two two years ago black swans you know one of the two things was we got to solve this problem

or we're going to lose the javascript or you know web assembly i agree and if there was going to be

a python 4 that had a breakage and there was only one breakage that could be allowed this is the one

i would break uh we don't have to support easy install from 2005 you know some obscure edge case

there needs to be one story right i don't you know ultimately it doesn't matter if it's this

tool or that tool or the other tool but there needs to be one story and everyone needs to be

using the same story and all the tutorials need to have the same thing and the learn python book

needs to say this is the thing you have to learn and for node it's an npm install right and although

they've got yarn now so you know perhaps it's well it's like sort of the i mean it's basically

the same thing but i think that's just the case is that i mean so i have a site install python

3.com and i actually own install python 4.com that gets quite a bit of traffic because a beginner

says I don't care I want it to work

and so as soon as we

start saying the conversation we

just had we've lost them right it's like

okay just get something working and then you can learn

why it's hard but it's really probably not hard

if you're just starting out

I'm curious

Alexi's answer to this question

it's like a personal project like a

greenfield you know you wake up

cup of coffee new

Django project how do you get things installed

honestly

I would just use PyCharm

I would just give it to PyCharm, like, do this thing for me.

Okay, just out of interest, what does PyCharm do?

Does it use VM or does it?

So you can select an option to create a virtual environment for you

or use the one that you created before it.

And it would actually remember your choice

and use it for your future projects.

It would remember the location of the, for example, .vnv folder that you suggested and use it for your feature project.

It would detect the one if you're opening your project with the virtual environment.

So, yeah, of course, lots of thinking is going into how we can simplify the project setup.

We've added recently the possibility to install Python from inside the PyCharm.

if you're opening pycharm you don't have a python installed you can do this uh and it would look

virtually the same as setting up your environment you can just hit next several times and boom you

got a python there and is that is that um can i can i manage multiple pythons uh with within

within one project carlton no well yeah okay so this is my this is my goal right and i haven't

and you know it's a basic for base project i'll use vm and i'll just use vm and you know that

that's it you know and i'm happy with that and pip install with requirements file and i don't

want any of these other things because they just they're more trouble than they're worth right

but the but then i need to test against python 3.5 python 3.6 python 3.7 python 3.8 3.9 i need

to test jenga 111 2.0 you know i need to test all these different combinations um so i use pyenv to

get the multiple python versions installed and then i use tox for to define this nice matrix

for all the things i just run tox and that's about as close as as it can be and what i haven't found

is a kind of way of handling that smoothly in the development environment so i want to quickly spin

up a python 3.6 django 3.0 you know i want to install that one and and tox have in have come

up with a devm flag where you can use it to you can say use this environment to create a development

environment it's close but it's still not perfect and that would be somewhere where i would expect

an ide to step up and say look i can manage this matrix and i can help you create all the different

ones and i can activate this one and i can be 100 you can be 100 clear of which one on the matrix is

activated now that's where i think an ide would pay value over a tech said you know where the

text editor is not going to do that because it's out of bounds but that's the sort of extra feature

that i'm thinking oh does pycharm do that because if it does then it's you know really i mean not

necessarily yet but that's what that's my sort of holy grail that i haven't found and you know

pipenv had different it had a dev thing but it just wasn't enough it wasn't flexible enough it

was you know yeah so um the thing is it's it's hard to both maintain several environments

and have a consistent development environment set up.

So, for example, if you're, for sure,

there's a possibility to have several interpreters

set up for your project.

You can easily switch between different ones.

No problem about that.

But what if your requirements file is not the same

as the state of this interpreter?

Should we suggest to install it everywhere there?

Won't we mess up something in your next environment?

So, yeah, it's a really hard question.

It's the one that we're constantly asking.

And I think we're trying to bridge this gap.

What you described is a very good case.

We're thinking about it.

And I think we'll continue to do small steps in that direction.

I'm hoping we arrive there.

Well, let me just give you one more nugget.

And then I'll leave it with you to ponder on over the next 6, 8, 6, 12, 18 months.

But the one more thing as I want is,

okay, this is all going great

and I've got my environment set up and all the rest,

but now I've got a really hard problem to debug.

And I want to install various extra debugging tools

that I just want to use for now, right?

I don't want them in the virtual environment all the time,

but I really need to solve this problem now.

So pip install this, pip install that,

pip install the other, fix my debugging problem,

but then my virtual environment is nuked

and I have to rebuild it.

And in which version, which, you know,

so the ability to layer on a sort of extra layer

of debug tools just for the moment that would be a kind of you know like cherry on the top anyway

i leave that with you to ponder because that's these are the things i end up struggling with

that's a that's the thing that i'm actually working on at the moment the one i talked about

the whole story around packaging i'll i'll add it to the list oh sure now it's interesting both

of the things carlton mentioned there's a lot of i don't want to say churn because that would be

negative it's actually positive churn in pep land about solving individual parts of the problem

they learned their lesson 10 years ago when they tried to do some big bang effort and it got

rejected i don't remember any peps about either one of these two problems yeah don't try i don't

aware you would record it i'm not right i think that yeah i mean there is there is a lot of change

now i mean so i think um pyproject.toml has just sort of finally been given the the blessing that's

going to be the way forward and a couple of years it's been like well do we want to add this to

django is this where it is i mean we just had setup.config and we moved all the stuff from

setup.py to setup.config and now we're going to move it to pyproject.toml is that going to be

permanent or i think that will be permanent now i hope and like brett cannon's doing a lot of work

on peps to take really tiny allegedly non-controversial things get them encoded into

that universe and then everyone complains yeah no poor brett i find yeah it's thankless work

python's such a nice community and yet every every time the pet pi ideas and the python ideas in the

pep world it's really quite brutal and yeah i i stayed clear of it for this reason so are there

any any last points you want to um either of you make that we haven't touched upon as we wrap things

up i mean so where so people listening who say oh i've got a great idea or i agree with that i guess

we can put a link to the issue tracker um for pie charm is that the best way to to reach out to

either of you yeah of course uh go to our issue tracker create an issue or upload the one if it

already exists leave us a comment describe your use case the more information we have

the more thing we can provide yeah and there's a check and we'll have a link obviously sorry go

ahead i was going to say there's a generous trial period and whatnot because i for ages i looked at

the pie charm website a few times i was like should i download this and then i was a bit scared by the

form and in the end i clicked the download button it was quite easy and they just booted up and

wasn't a problem you know i mean it's almost you know too easy because when i like when i teach

people it's so tempting just be like we'll just use pycharm and all these things are solved but

because not everyone uses it i have to kind of go the the raw way but certainly i think

that's starting to change and certainly for students you know when i go into universities

it's overwhelmingly PyCharm

over anything else

in part because you know they can use it

for free but because it does have all these

features and then you know sometimes

I'm not as familiar with all the power features

so I have to sit there and go huh I didn't

know that like that's pretty nice

so it's

educational for me as well

that's normal I do this all

the time

I do this all the time because we have

so many features that we constantly

to discover new ones yeah and i i manage the uh help manage the twitter and so i see someone

pointing a tip out to someone else and they say pie charm so it gets in our inbox and i'm like

oh really i didn't know that how did i not know that it's my job same thing happens to me with

django you know people say well of course you must know this because you wrote books on it

but you know i just discovered this thing and you know like yesterday it happened i was like

Well, I didn't know that about Jenga.

You know, people talk about imposter syndrome.

I get paid to, like, be an expert for PyCharm.

And I know maybe 2% of PyCharm.

And I wake up each day, and it changes another 1%.

So don't talk to me about imposter syndrome.

My imposter syndrome can beat up your imposter syndrome.

Well, you know, Carlton shouldn't have any.

But, you know, for me, conquering the imposter syndrome is when you get to a point where

people ask you questions and you're like, I don't know. And then nobody knows. And so it's just

knowing, being comfortable with not knowing, but having the confidence to say, well, I could figure

it out. I think it comes about when you are a beginner and you don't know, and you feel like

you can't figure it out. So when you've ingrained that process of Googling and you have enough

context to use the Django docs completely, I think that's when you overcome it. But when you

feel like you hit a roadblock that's where the imposter syndrome smacks people i think i don't i

i think you're right but i think it applies at every level because you know i'm you know i work

on jango all day and you know i have the code base open all day and yet i'll go to an issue on the

issue track and it'll be some nasty in the depths of the orm and there'll be two or three of the

major orm contributors there and i'll just be like oh i'm not hanging out in this club this is too

what don't you know that's my point when people start when people start asking you questions and

you're like i don't know you know that's when you're like well i guess i'm at the level of an

expert because nobody knows i'm not i think it's easy to feel intimidated in any at any level you

are and that you don't have you don't have the knowledge that it's like oh well but you know

if push comes to shove okay i can get in there and i can get my hands dirty as well but that that

feeling of doubt still lingers that feeling of like yeah well hopefully that's something we do

with the podcast is having people on who are experts and everyone says the same thing so

people listening know that this is just what it feels like you know it's like we just get

comfortable with feeling stupid and you almost you know it's like i almost get bored now if i

know how to do something i'm like shouldn't this be a little harder you know because the muscle is

so built up anyways well we'll have links to everything um obviously you can go down download

pycharm in april check out the django project site for the free trial that benefits the dsf

um and yeah thank you both for coming on and sharing about pycharm really exciting thanks

for having us thanks all right we are at chat django on twitter and django chat.com we'll see

everyone next time join us next time