 Welcome back everyone. Now it's time to welcome our next speaker Christian Morira for this so Christian is a developer and an astrophysicist. Hi Christian. Are you joining from? I am joining from Berlin. Nice So Christian has previously has worked on the QT project and works at the QT company He's a developer and an astrophysicist and today he's going to talk about internals or CPython So I can't wait to listen to your talk sounds really interesting over to you now and good luck Cool, I will just set my timer now just to control myself not going longer than expected So thank you very much for the introduction and my name is Christian As you as you heard I am currently writing code of the QT company One of the projects that I work with is the QT for Python project, which is the official Python bindings Maybe you heard the pie side So that's why I'm so close to CPython development. I need to deal with a lot of internals just to maybe to be able to expose Qt to Python users. So yeah The the talk is called learn CPython by breaking it because it's usually the approach that I prefer to learn new things I am not really I know that for some people books works well For me, I am really lazy on that regard. So I prefer to just break staff and learn why that happened I'm really lucky that the previous two talks covered topics that I was really looking forward to cover in this talk too But I didn't I couldn't because of the time constraints, of course So I am really happy. Hopefully you watch them. If not, I invite you to watch them afterwards and So yeah, thank you very much for the organization to put in this together Hopefully next year we will see each other in person. So let's start So I know that nowadays The the fear of missing out is quite large among people related to Programming because we have new libraries languages and everything's popping up everywhere And this might be kind of tricky just to catch up. So I know that some of you will say, yeah CPython internals No, it's really not my thing. So in any case, I know it can be scary, but don't worry about it So I put this picture here, I am not sure if maybe in the chat you can see if you can recognize this thing Maybe when you're a kid, you kind of like got one of these things So I put it here because when I was a kid, usually I had a couple of those because that's what got to get when you break your maybe your toy car and You get to this motor inside and then you can connect the battery with it and it spins and stuff like that So for me was really interesting since the early age and This was kind of like my way of learning things is the beginning like breaking something trying to see how it works And then it started to enjoy and do other things So of course, I will love to jump directly into show you some code of a strange behavior But we need to take a look at a couple of things first just to be sure and just for you to have all the necessary Tools even though this talk is for intermediates Level I said that if you are the moment are listening are you are too afraid because you don't know see is totally fine You can say around I usually put intermediate because when I say see language some people get scared and this will not be scary I promise So first of all See Python maybe some of you thought man, then is this a typo or something? And I'm pretty sure if you watch the produce talk you kind of know what is coming on the next slide So here we have for the sorry six different terminals I will give you a couple of seconds. You can check them out all out Can you see what is the difference? Maybe you will see that here? We have like four symbols here We have five symbols, but the code itself is the same in all of those terminals So if I go to the next slide You will see that only the last one is the Python that we are more used to it They will divide them that we know so you have their Micro-Python, PyPy, Jython, which is the general implementation Rust Python piston and Python itself. So in previous talks you already got like we have different implementations of Python, right? As I said already, we have Micro-Python, Rust, Jython in Java, PyPy in Python and Python We have all those like startless Python, R and Python, Pgion and Cpython is the main one So that's why usually we say when we are talking about Python in a more colloquial way We usually refer to the interpreted that is defined by the Cpython code So what is the Cpython code? If you go to github slash Python Cpython You will see there that you have the official repository of course that will have all the developments So when you get it you will get the latest and the greatest But inside there are many directories took me some time to kind of understand what was going on there But luckily there are many good talks out there tutorials and explanations that can you can give you a glimpse of what is going on there So this is overall the structure that I got before preparing the talk You see a couple of directories there and a couple of files But if I may I said that the most critical ones so you can maybe look into it are the of course documentation The old interpretive header files the grammar that maybe you saw since last version do as a nice improvement there It's thanks to pep617 Then you have the standard library in pure Python because of course some bits of pythons are written in Python Then you have the modules. There is a standard library in C And then you have objects which has all the built-in types This is one of the most interesting things for me at least because I was really curious to know how does Python list works dictionary sets and everything Of course, there is also a directory called Python that has all the cpython runtime So you will see all the maybe there are the functions that are important if you want to try to build Things with cpython directly like you saw in the previous talk So if you want to build your own Python, of course I am saying and not installing it with double click or your package manager or whatever just build your own things You can easily do it By cloning the repository, of course first Then you can go inside the directory and run the configure command if you're not used to these things Don't worry about it. It's only this slide, but this is more like unix focus on how things are being built And then you do make just to start compiling Then you can specify if you have a lot of processors there and finally you will have your python interpreter So you can see it there as you see here Is the 311 because it's the alpha is what I got when I was building this thing So, yeah, we need c because we're talking about cpython. So I promise it would be super brief and super simple So let's go a step by step side to side things that you can have in python c I know this may be a silly example But I just want to you to to give an example so you are not afraid of looking at these things So let's go first comments. You see different ways of writing it Way of importing modules in c usually you include kind of header files or kind of modules You can define a function there. There's a simple function add to do parameters Then you kind of have a main function in python, but in c it's mandatory You really need this main function there So I was trying to write something roughly similar in python I know that some of you don't like to have this structure, but it doesn't matter Then you have some types right you have first a string letting c is a little bit weird and integer and float You can call your functions there and do a print. I don't like this way of printing in python But it's just to make you the analogous to what is c So you will say at the moment. Yeah, but I mean that's really complicated But is it I mean we we saw today in the the keynote how important and difficult types are the same thing But if we have type here's type annotations You see that the code itself. It can't look more familiar to it. So it's nothing to To worry about it. I mean, I'm pretty sure that if you can understand the code on the left You will be able to understand code like you see on the right Okay, but maybe you're saying like yeah, but python is way simpler forget about safe Is it python always really simple? I mean Maybe on the chat you can see what does this code does Maybe you don't know. I don't know you can let me know afterwards in the in the break-up room Or whether this code does for a given number. Yes, you see it could be a little bit complicated This is valid python code, but anyway, so let's bear with me The most important thing you learn see or at least a little bit of it. So you will be able to understand other things Now we want to start Digging into and try to see if we can break some stuff there So don't be afraid It would be really cool to do it and the first thing that I want to tell you is that Sometimes we we kind of I'm not saying that python is about project But we believe of projects that we use so they are a really super Strict and things that work properly all the time and they have like no further or strange comments So what I wanted to show you is that briefly if you go to the c python code And we look for a couple of things inside the repository You will see that we also find nice things. So what if you find the word to do you see there's a lot of to do still And in the whole code So you have things like yeah, this does not run things like that So you should go there and maybe you look for a fix me it will also So many things that yeah, I don't know if this is working. Oh, this is not correct. See this is wrong And of course you can go even further and try to look for Sad face events that people sometimes don't know or they don't know how to bypass a certain thing So what I want to say here is not saying like oh look at how bad it is No, it's like python is a project like that Like the ones we write so don't be afraid of it Okay, so now we are going to see python code, but it will be brief. I promise you Um, okay So first of all we'll use a tool if you're in in linux you can use gdv You can use your mac ldv and windows. I really don't know if you don't know what it is Thing you can believe you can think about it's like a crystal ball that does some crazy things the allergy to see through code so one of the first things that I wanted to show you is that if we do something like Like this for example, I told you already don't worry about this thing that there is this always this main function So we can even say it's okay Let's break the main and then let's write python and then what something was find it This is inside python. I just see here. So we have an int mate So it's a similar thing that we saw but there is another function there called By by it's made. Okay. Let's break there and try to continue further Uh, if we go there we still see okay, there is another function there cool But there is another thing there by main main off. Okay, so let's try to break there Try to go there and maybe we can see there is And still there are other main fun. Okay. So this method might work. Of course, you can break in different parts But it's not really necessary at the moment But with this tool you can be able to go inside and see what is going on under the hood I know this might be maybe not too obvious But uh, just bear with me that this is something really useful at some point You will like to see and start to see where to break things Another option which is really interesting is the the nice this module So let's consider this is small the same function that we have there Maybe you have seen in many talks and when they run this on this yet, they guess this weird thing is at it That's by code. It's totally fine But something interesting there is that you see that you have two variables that somehow are load And then there is this thing called binary at binary at so then now maybe you can go back to the code And look for binary at binary at We will check that's this thing out and go here There it is As you can see, this is c code. Don't be scared But this is quite simple since Python, of course works with this kind of a stack So you will get the two elements that you have there Right and left and then you will perform some some we we all learn now for example that first we check if there are a Unicode so there are strings. Maybe Do some special concatenations because you of course, you know that string has two things And here there's a special function called by number add So see you already know like if for example, you want to alter the mechanism of the plus sign You can do some things here and you're going to start breaking this code out about So it's really interesting. Another interesting thing here is that you see if you're familiar with Switch cases. This is doing things for different options. If you go up, we have the modulo there You go a little bit up her floor divide and many other things there So if we go to the line 547 we see that this whole process is inside Something quite similar to a while true So you can see this is running all the time and there is there errors and we are evaluating code and stuff like that All these mechanisms which is inside a c eval c file It belongs to something that is called the main loop And the the main loop I think that on this line You will see the evaluation of this frame With this kind of the equivalent of like what you are trying to do there. I think that even if we look for main loop Yeah, so we have some see we have there where it is specified where the main loop is located So this is what is running with python is running evaluating everything that we have there okay Just so many things that were pi object before in the talk before I was really lucky that And previous speakers show the interaction with pi objects and stuff But I just want to show you what this structure looks like it's in c python So this is a simple structure Maybe you are familiar with data classes is something, you know that holds a couple of values You can look at it as an enum, but it's not the same but still It's a kind of like something that holds a couple of values quite simple, right? So the thing is that here you have something it's called up type and when I when I went inside Okay, what is inside is up type This is all the code that is inside out type I'm pretty sure you're gonna even read it, but the important thing here is there are many cases and situations and stuff regarding Like what is going on with python Here you can see there are many things to specify for numbers for sequences for mappings, etc, etc So maybe now you're thinking okay, okay, whatever. I am really unsure for this. I quit Uh, you don't need to learn them all the thing is quite simple If you need to implement a type that this looks like an int you implement the methods and the things that are required for ends For this same things that require friends and for strings same thing So, uh, I just want you to to to get your head around of py object as a big box that you can feel However, you want depending on your needs. Okay so a couple of things that I was trying out for this talk is that Um, I first I would like to talk about I will just open my python interpreter here Of course, we're running python 4 the euro python version. I hope that all your view are also using it and with sets the problem that I have is that Every time that I was trying to okay. Yeah, let's have a simple set here And I was trying to add a new element. I hope that you can see right. Yeah You have it there. I was always trying to then to add another Set that is of course there is a problem because that nothing has to do So I never managed to remember what was the name of the function to include another set So I really would love to have I don't know something something like I don't know add another set Okay, sorry about it. Yes, I forgot that I need to of course be more gentle add another set, please Yes, and that we have it so we can do things like to just for you to remember even if the name is longer I mean it works for me, right? I kind of remember what was the thing if you're wondering what was the name of the function is called update Never came to my mind. I'm very sorry about it So this is something that bothers me about the set, but just you can see you can you can break some stuff there and fix it so Something else that is also quite interesting. It's our lists that of course There are many modules inside sorry methods inside the list We can see a couple of them and I'm pretty sure that some of you don't know it So let's create now the list again quite simple list Let's say that we want to append a new number. So, you know I want to append number three. You have it there. How do you? Pre-pend an element. We don't have the prependment, right? Because if I do prepend or something so maybe some of you in the chat would be say, okay You do the insert, but maybe you can do also the same with writing the word append backwards But then if you do that, then you can of course start your Your number there. I don't know. Maybe it may be useful. Maybe we can submit a patch about it There is something else and that list has that list has behaved like lists like sorry less tax So you have something that is called pop Right, so you see that with this you remove and return the night and blah, blah, blah, blah So if I do now a pop The list kind of Changed right So, but I don't know about you, but I'm not really about a lot of pop. So I prefer to use rock So if you try to get the the help of this it's kind of similar But they say that of course is is rocking. So at least maybe I can try to rock And of course I am rocking out Then the number two And then of course it maybe that's not your cup of tea and then you want to add all things that You want you maybe want to I don't know salsa out the list If you do some salsa this salsa the list in a salsa steps order and return none So maybe see if this is my list and I do salsa You can have you don't know some salsa mix there You know, but these these are things that are kind of like nice that you can have in the Let's try to create another list Another example So let's say that if you want to work with colors, right? Maybe you want to do like something with red or we want to do something with As a color. Oh, sorry my camera. I think that is What? Uh, I'm sorry about it. I was not Okay, I really don't know what to do help Okay, let's forget about this example then I think that is too strange to to have these kind of things Uh Okay So, uh, then of course That's what the things that you can maybe play around with the silly things at the with the list Something else that we can kind of look at we have a couple of minutes till we have seven minutes It's that things that in the future you you you can come and kind of count on it so, uh, there are many things that have been discussing already, uh with The conference which are the next things that are coming in in python light Something else that I found interesting is that, uh Well, Pablo was talking about, you know, this really useful way of implement. I mean the this pet 657 How to get you to point directly where the or the problems are, right? and I noticed that the community that on another side of the spectrum is is, uh Using a lot of our tool for visual studio code. I don't use it But I have seen that a lot of people really like visual studio code pets So I was thinking, hmm. What if we kind of mix these ideas? So I was thinking what if If you know, I mean, you know that you have a bad list This is kind of the examples that people is showing And then maybe we can do something crazy and uh, I don't know Uh Call trying to call this thing And maybe we can revive our friend clippy and then we can make it I don't know more like a pets related to it If you don't like clippy, maybe have some past from us And maybe you want to do something different and when we are doing python and you have some I don't know some weird error. Maybe you want to have something more friendly Like, uh, I don't know. Maybe I'll take out that, uh, you know, it's kind of telling you like When it's going what is going on there? So you can kind of mix these two ideas, right? So yeah, this is zero division error, but uh, But do you know what happened when we Type zero divided by zero in python or at least in the newest version I'll think that it's quite a bit tricky thing here going on But uh, yeah, I think that let's let's not go there. I think that it's too scary to to to work with. Let's go back to it So yeah, since we are finishing, uh, now I was noticing a lot of people that uh, they were struggling a little bit with Hey, all the backgrounds of the frame. We are there were some discussions and some material at least from the speaker But did you know that python was conference ready at least in the newest versions? Oh, if you go here and you try to for example to write a help with the euro python Uh, you see that with this at least option you can prepare your webcam for your python conference So I really don't know if we can try to do it a little python Yeah, as you can see now from python we can control our camera feed and everything As you can see dance here that of course there is the euro python logo and also my nickname So yeah, that that will be the main things and maybe you thought already like yeah, they are made details This is really not for me There are other stuff here that are really important likely it was discussed in the previous talk The first one is the reference counting thing. This is how pi objects of the lips in python It's usually kind of the a simple idea right every time you use something You increase a counter and when this counter reaches zero then that object it does stop Stop living there in the python. So that's why it's get garbage collected It's a really interesting topic But it really requires you to have a full talk or maybe a couple of workshops Something also really interesting that you see many people talking about a global interpret block And then this also is really interesting for you to go there and try to learn by yourselves And something else that also is likely for us. It was addressed in the previous talk It's the module on site creation, uh, even though Dealing with all these five objects things might be a little bit tricky I really encourage you to start now to clone the repository start to modify the like a couple of things that I already told you And maybe you can start to to check it out. So to say don't worry about it You can go slowly and also embrace the python. It's nothing too serious It's something that you can still copy and paste here there All the little jokes that of course I was sharing during this talk I will share them later on make sure that maybe to to follow me on twitter or on github I will be posting this all the material so you can replicate all the silly things that I was doing today At your own pace So, yeah, I think that that would be it. I think I will still have a couple of minutes that they have I am not sure if We have some questions already that maybe we can invest this time to if not We can also kind of like look into the things that I was the implementation of the course that I was showing Yeah, it was it was really mind-blowing one word to describe this talk is brilliant You fall for me. You you sent me a message saying that my camera was working Okay, right, so we have a few questions. I'll read them out for you First thing everyone is talking about what what was it with python fool? And of course as I said, I mean there were many jokes here It is just the branch that I just manually changed you will see it later on when I share the repository is You can change one variable and then you can get of course whatever adventures you have there as you see here It's even the the euro python version, right? So you see it there Well, my screen is not but you can see euro python version. So this is this is a silly joke Okay, nice Okay, uh, what's the average compile time on a normal machine? Is it minutes? Is it ours or something else? Well, uh, let's try it out So, uh, let's create a new build. I will build I will go To the c python code that I have I will just try and configure here and This is just configuring And then of course for the compilation process. I think that it will be maybe less than a minute, but I have a rise in five. So I have 10 CPU cores there This is not taking too much time. There it is. So if we try to build now It shouldn't take a lot of time, but we still have we are even inside the 25 minutes So I think that people will be fine too to just wait and the meantime I can show you like I have, uh Yeah 12 processors to build this thing and I think that I will read because this is the last thing and this was running the config and Yeah So let's say that it will be like two minutes. Maybe at least on this machine of good I know I had in my old laptops and stuff that It takes a lot of time, but he shouldn't be much We can leave running this thing. Now there is so see like kind of like one minutes less than two minutes Okay, nice Uh, we have another question, which is kind of interesting one So did you also go under the hood for other python interpreters like rust python or rotten python? Can you tell no, not really? Yeah, I mean I have been I wanted to to contribute at some point to to rust python because I when it started I thought it was really cool But I didn't know rust at the moment and learning any language It was the same kind of issue that I was mentioning at the beginning of my talk The only thing that I have some Brief knowledge is that stackless python, which is something that maybe not many people know But they can't remove the whole concept of a stack in python. So all these things that you see pops and stuff It's really interesting project. I have the like to to work with the author in the past. So yeah But I said the c python is is quite the simplest one Do there are other much and maybe this is something a good opportunity there are other projects Maybe you can check the python us notes from their conference. There is something called hpy People can look maybe and I see other implementations Nice, I hope that answers the question So, yeah Really, we enjoyed a lot And I still see people posting questions around So if anybody has any more questions, please feel free to jump on to the breakout room. Yeah We can reach out. I mean I am see my array or on twitter You can write me whatever and I will maybe now go to the To the breakout room and maybe people want to shoot Just that's the last question. I'll just show I just got on if you are playing around with it You'll need to recompile for every update you make to the code or It's only a little thing. So let's say if we do something like This will take less than a minute. I promise If you go to see python update this object And for example here, it's all the silly things that I was doing with OBS during the talk So maybe we can remove the The fake append Method that you saw there. So be me PA I can remove it from here So I think that this should work. But in any case, so You can see there it only build that data. So it's only what I touch Of course, if you you modify the object, which is the main thing from python You will need to more build other things with as you saw is it's quite simple right Okay Well, I suppose There are no more questions yet. So Let's wrap it. So really it was a pleasure listening to you And yeah, thanks once again. Thank you very much for the organization and the effort. Bye Bye Okay, so I have two announcements to make So lightning talks are going to start in 10 minutes from now in the optima track. So please stay tuned And also a gentle reminder for all of you The sprints are starting tomorrow and there is still a chance For you to find more developers and collaborators for your open source project and also to educate people about them So please feel free to register and list your project on the sprints registration page The link has been already shared on the metrics channel Enjoy the rest of the conference. Goodbye