 So hi, thank you for for coming on Francisco Fernandez here. You have my tour handle if you want to Asmiss something during the talk or whatever. Also, you have some questions during the talk don't doubt about interacting me So, okay, let's start today. I'm gonna talk about a garbage collection on Python Here we have the agenda of the this today talk in first place. I'll do an introduction Include the motivation why I think this talk could be interesting for a Python developer some known problem some manual memory management some definitions on garbage collection from academia that I think is also quite interesting to do know about and Also every solution has some trade-offs. So we will explore them Later on we will see how see Python implements a garbage collection We need to know something little a little bit about the simplification. So There will be a little bit about see all good. See Then I'll explain the algorithm itself and finally the cycle detector and if I have enough time, I'll try to Do a high overview of how pipe I approach garbage collection. So let's start Okay, what's what's the motivation knowing more about garbage collection or memory management? We don't need to take care most of the time out memory in Python. We are some kind of lucky most of the time we are working on a business logic layer because probably most of us we are Developers not Python core developers. So we don't do Native extension. So we use a lot of Extractions that the language give to us without knowing more For example, we use a lot dictionaries and we don't know and we don't need to take care about how dictionaries are implemented or the layer between the operating system and the standard library and also memory is another abstraction that we have and Sometimes it's useful to know about So we'll know more today. I hope that you want to know more about how memory is managed in Python Okay, but managing memory manual is is hard. How many of you are familiar with C or C++ place race? So, you know, probably, you know how difficult sometimes is Managing memory manually. There is a lot of problems that I tried to list here What the most known is memory leaks that basically is that We allocate memory for some resource than after using it. We don't have to allocate it We don't we do that most of the times by accident. No purpose Another problem that we have is the Omnistip Let's say for example, like in this snippet of code. This is some kind of library This library allocates memory on the hip and it returns a pointer Okay, everything fine, but who is responsible on freeing this resource with the library to do I do the free free this resource This leads to the problem of double freeze. What happened if I free the resource and again the library Somehow had the pointer and finally tried to free. We end up with a segmentation fault in the best of the cases Another problem that you can have is dangling pointers For example, this is another snippet of code. We are allocating here variable on the stack and returning the reference and once we are out of the scope this address is not valid anymore. So again, we will end up with strange behaviors Segmentation fault will be the best of the cases because you know that something is wrong Values that are not correct is the difficult one Some languages Propose some solutions or sort path some patterns for example C++. They try to apply as much as possible right pattern they also introduce some Some tools on a standard library like unique pointer and serve pointer And also there are another approaches like in Rust that they try to make language safe on compile time There's a lot of solutions. I don't know if in C you have much tools to deal with that But we cannot have garbage collection in every scenario There are some scenarios where it's mandatory to have full control of memory One of them could be embedded systems where we have very limited resources and we need to carefully Deal with memory also for example, we have Very demanding applications like a video game can be where for example, we need to Know very carefully how the memory layout is to try to avoid cash misses and so on and also applications that need the termination because most of the algorithms as far as I know Garbage collection. We don't know for sure when our resources is the allocated. So there are scenarios where garbage collection works and other But what is all this about the garbage collection probably most of you know what is but I would like to introduce you a little bit about history Here we have a picture of Mr. John McCarthy playing chess with a computer And he was kind of the inventor of garbage collection in this paper from the 1960s The first mention to garbage collection on records a function of symbolic expressions and their computation my machine And it's kind of funny because He described the whole process He described mark and sweep algorithm there and as a food note He says, okay, I've been naming this process garbage collection But I know that academia is a bit picky and I want to use finally Academia uses garbage collection. So kind of funny So what's the formal definition of garbage collection garbage collection is basically? Automatic memory management while the emulator runs it's routinely allocates memory from the heap If more memory than available is needed the collector reclaims and use memory and retouch it to the heap This sounds too formal, but it's quite simple the emulator You have there the definition, but basically is our business logic The heap is where the memory is allocated and the collector is their garbage collection Algorithm itself in our case Python virtual machine is the one that takes care about collecting annals and annals resources If you want to dig deeper into this topic more in the academic side, this is a quite good book It's quite extensive, but It's not so so formal. So I recommend you to take a look but as I told you before Every solution has some trade-offs. Nothing is perfect. So if we Decide we are a language implementators and we decide to have a garbage collector language We have to know that probably We'll need more resources we will have performant impacts because The algorithm has to run so we are losing CPU cycles there and on some Algorithms, we don't know for sure when resources will be free. Okay So now we will start looking into how C Python implements a garbage collection It implements a reference counting algorithm, okay But before we need to know a little bit about how is implemented in C This is the main object that is exposed mostly for us as a user all the time This is a pi object that is basically a strap with two two members. One of them is Pi size t that basically is for us an integer Name of ref count, okay This bit member holds the number of incoming references that our object has so for example if this is an instance B and Object A as a reference to to be the country's one because it has one incoming reference And the other member is just a pointer to the type. This allows us to have Dynamic typing, okay So let's see a simple example For the purpose of this talk that I want to be Educational we will only take care about instances. We know that Foo class is also an object, but for simplifying we only deal with instances, okay So how does this look? Graphically, okay main is the main module and It holds two references one for food another one for my list So those counters that I told you before are both one, okay Because only main is referencing food on my list But let's make this more interesting. Let's append food to my list now Foo will have two references. We can see graphically like this, okay Now foo has both main and also my list referencing him That's that's happened, but who takes care about this counter presumably is the CPython implementation so here we have a snippet of the Standard library the only subject this function what it does is given the list self. It happens the object V in our case Foo on my list, okay And take a look in into this line by in reference V This is the point where we are incrementing by one the counter because now the List self in this case has a reference to to be okay in this line it sets in the last position the element How does this pie ink ref looks like like in more a lot of places in CPython is a ugly macro Basically what it does is it does a custom op and Increment by one the ref count Easy But what happened when an object is not referenceated by another one anymore? well Presumably the counter will be decremented by one so we can We end up with this case in the same picture as before because now My list on index zero that before had foo inside now has none Now we end up again with the same picture because my list is not holding a reference to foo anymore Okay, so foo again as a Overfound of one and how does it look like on? CPython level again. This is part of the list object.c file that implements the delist CPython Again, we have very much like the inverse operation in ref on the whole item That basically is the one that we are not interested anymore. What happens there in that macro? basically The counter is decremented by one, but it also happened one check there If this counter is equal to zero we know for sure that no other element No other object in the system running is interested in that object anymore. So we can safely the alloc that that Object this is what happened here Does the check here? If if there is greater than is greater than zero, okay? That's some text Other case it the allocated, okay? It's clear till now And well, let's let's reproduce this case in our little example now We say to the module. Okay. I'm not interested in foo anymore delete all the references there, okay? so Ref count reaches value zero. So we Deallocated well we the garbage collector. Okay, and the interesting thing is that once we know for sure that No other object reference it foo we can deallocate. We don't need to wait till collection like in other algorithms, okay? collection phase But Do you notice any problem that this approach can have any clue? probably most of you know what what happens but Cycles what happens with cycles this approach don't work for for cycles, okay? Like this now let's imagine the foo has a reference to list and we create a cycle foo has a reference to my list and my list to foo like this Okay, now both objects live in main module. So no problem. They are there But what happened if? We don't take care anymore about foo on my list We end up with a cycle and those objects are not Deallocated because Both objects has a reference count of one and it's not possible that anyone reference it again because we don't have Reference on our system we delete from the module main So we end up with a memory leak and I told you before that one of the problems with Manual memory management is memory leaks and we also have here memory leaks So what was the pro of having? Garage collection. Well, we are lucky and in CPython implementation They implemented the cycle detector Well, this is another topology that we can have more complex cycles this is a Those are very very simple examples, but in the real world they are more complex but yeah What what happened if? If we have a fake cycle We'll explore how CPython implement the algorithm to check if there is cycles to do that garbage collectors to know more or less All the objects that are allocated on on the runtime and this is keep track on this On this double link list. This is a note of a double link list That lives in GC module with this this signature and for performance reasons and there are three generations on implementation because Doing a whole scan or looking for cycles on the only been objects is quite costly So what they do is they divide into three three generations from the youngest one to the oldest ones and We do a step-by-step. So we don't need to check for the whole the whole living objects. Okay and Well, this is some some see tricks to This is a union. Well, don't don't take them Worry about it. He's just some some details so Let's represent Our example like the one before We have those three generations and we have a full on my list both of them probably you can see in the back Both of them has the value one on GC riffs riffs And what's what's the key idea of the cycle detection algorithm on? On C Python the key idea is that We know the object that belong to a generation So what we can do is extract all the internal references on the on that generation and those objects with Number of reference bigger than zero. They have references from other generation objects. I Don't know if I explain But or but the idea is is that we We keep track only on We subtract all the reference internal references in in the in the same generation I'll do a step-by-step trying to explain better to do that types on internally on on C Python implementation has this method that abstract how to traverse all the references on Objects so for example, I implemented this as a Python to do it more clear So for example dict extract traversing all the references basically as iterating over the key and value and applying some function a List basically iterate over The all elements and apply a function. This is a utility that we have for For these kind of problems like cycle detection. Okay, so those tools help to To implement algorithm, okay So, let's start. Let's think that we only have full on my list on on my Generations here. Okay to simplify things as much as possible So I start iterating over the first element that in this case is the one marking red food What I do is using These functions depending on the type I iterate over all the references that these objects have Okay, in this case, it's only my list and I apply one function this function What it does basically is check that the number of references is greater than zero and decrement by one So with this way we are breaking the cycles. Okay, because all the cycle all the internal references I decrement are decremented. So if we have a case like like this one We are breaking it. We are marking us unreachable. So this is the first step Who only has a reference to my list decrement by one now the reference is zero. Okay The next step we have my list as the next object in in the generation zero and again we decrement by one So now we know that those objects could be unreachable outside of my of my generation Okay, the next step is moving those objects to a new list called unreachable and One step that we have to take care is we need to check On objects that are reachable all the references because there can be cases Where some object get a value of zero, but there is some kind of topology like one cycle like The one that we were seeing before and two other references this end up having zero on those two objects But they are still reachable by these two objects. So we have to check for that also After that we move to an unreachable list Those objects that are reachable are moved to the next generation because they can leave more But I prepare like a little demo in Britain in Python before I did this using gdb, but was too much. So Let's Let's take a look. I have time I think Can you see the code over there? Yeah Well, I tried to represent like more or less the implementation on Python to make it easier to understand Do you remember this total link list? This is one of the represent the nodes on how the garbage collector take Tracks all the allocated objects, okay These are some helper methods that implement the traversal methods to know how to iterate over the the references one list Some examples we will run an example now. Okay, and here we have the Garbage collector object. Okay, it has in this case two generations With two thresholds that I didn't mention before but we don't do as Cycle detection each time that we allocate an object. There is some threshold that they they studied and If we allocate more objects than that threshold and cycle detection cycle is is fire. Okay Here we have this logic Let's say that I want to instantiate some class what I do is I Append to my first generation this object with on this Link list and I check this threshold. Okay. Is my generation greater than the threshold? Yes, I Collect I do a collection How does how is this collect cycle? Okay, we Get some references for be more comfortable programming The first step is we need to update the references because on the nodes this node Don't have the most updated reference count because as I told you before we don't run the cycle detection All the time so it can have a value that is older. Okay, so the first step is We iterate over all the generation that we're interested in and we update The reference count with the pie of that one. Okay The next step is this one of breaking cycles just subtracting one on the internal references. Okay, so Let's take a look how this What we do is again we iterate over the whole list of this generation and using this helper method that abstracts how to iterate over all the references we pass a function as an argument. Okay And this function basically what this does is it checks that the reference are greater than zero and decrement by one This is a helper method. Don't worry about it So we end up with the After this step, I will run one example. Oops. Do you know how to evaluate all the cells? I'm pretty new Sorry We'll do like a step-by-step See it clear. Okay, we allocate one list. We are trying to replicate Our example we have a list and an instance of an object and we we are creating a cycle. Okay We allocate We'll explore line Manually, I don't know what's happening demo Okay, so after this step on our example, we will get Those two objects with a count of zero So the next step is just check those edge cases where we can get some object with a count of zero, but Some other object can't reference it them. So what we do is In first place we use again TP traverse and we use another callback That basically what it does is if we find One object that has a count zero. This is a false false positive as an reachable. So we need to Keep track that he's still reachable In other case, we should just check that Refs are greater than zero And after that we have those two list the unreachable object and the young ones So we know that These ones can be delete But it's not so simple so we move the reachable object in this generation to the older one and I'll explain later what we need to do with week references and finalizers and we finally delete all the All the garbage. This is there are some edge cases that I'll try to explain now, but You understand more or less the algorithm should I explain something more or it's clear Okay, but as I told you before there are some problems with this approach We need to take care about finalizers. I mean underscore underscore deal. There are two kinds of finalizers ones are kind of legacy Those cannot be delegated safely. So we end up having memory leaks if we have a Legacy finalizers We deal with underscore underscore deal. It also has to take care a lot of edge cases, so please don't use finalizers and Finally, we have to take care also about week references What it does is check one the ones that are reachable and execute the callback that they have associated So what makes the emoji happy that is incremental this algorithm is incremental So as it works it frees memory. We don't need to wait to a cycle to recover memory And I'm always worried because as you can see the tech detecting cycles is kind of hard Not algorithm itself, but there is a lot of edge cases that I didn't mention that is kind of complex and even we also have some size overhead on On objects that this OV ref count and as a personal opinion also in CPython garbage collection is to to couple couple to the model So that we have a counter in in the main object of the interpreter. So it's not so so we cannot change the model easily so That's a sign note And I have little time now Like five minutes. I'll try to do our view of how pipe I handle garbage collection and memory so Probably most of you are familiar with by pie. It's an alternative implementation of Python is written in our Python and Regarding to garbage collection It has some points that are quite interesting One is that it's kind of agnostic to the garbage collection algorithm so during translation time that Is how do you get the interpreter at the end you can If I'm not wrong you can change the model of garbage collection the algorithm that you do want to use So this allow by pay developers to experiment over time With different approaches that I think is is a good good thing not to be tied to one one implementation and the algorithm by default nowadays is in the minimak and let's Let's explore a little bit about by pie one of the things that by pay developers notice is that We allocate a lot. Well, we interpreter allocate a lot of objects during With a short lifetime. So for example, we have some some examples here like for comprehension. We are Allocating objects that will leave very very shortly even when we are Calling a method on a class we are creating a new object a bounded method With a very very short life. So those objects will be Allocated in a performant way. Okay, so they end up with this kind of memory model. They divided into two areas one for the young objects this one that leaves early and Down the young area is divided into two areas the nursery that as a fix size another area where all the objects that don't fit in In the raw model of area go there and also some arenas to To hold the objects that are older than the young ones Sorry So how does it work? It performs What they call minor and major collection minor collection is performed on on the young area So it's fast because we don't need to transfer to the the old life objects and Objects are moved only once to to the old area And also major collection is done incrementally at the end they implement kind of mark and sweep algorithm that Well, this is are the faces that they implemented in first place they do As can face then mark this week Finally they execute finalizer because we have the same problems Here we have some some schema how Mark and sweep works on in general, this is a very very is the same the same schemas in the paper of McCarthy basically what we do we know for sure the root of the main objects and we Traverse the graph of the of memory And we mark on this traversal which ones are reachable Once we know which ones are reachable we treat over the whole The whole list and we sweep with the allocate the ones that were not mark Yeah, that's it quite quite simple and elegant mark and sweep can collect cycles because we basically are doing a Graph traversal but type implementation can be more complex to understand and see Python because At least for me and on full collection We stop the world. I mean that we cannot as well as in when we are detecting cycles and see Python We cannot perform other operations So Thanks for the Explanation of these algorithms what I would be interested in as a Python developer is to anything I should take care about in my code so that I Enable the garbage collector to do his work most efficiently try to avoid the underscore underscore deal methods for example because There are a lot of edge cases that interpreter has to take care of so That's one off We pipe are you say there is a Possibility of the stop the world issue on garbage collection is Is there a way to In future implementation or theoretically to Circumvent the circumstance For example splitting memory and garbage collecting different They do that approach So the stops as far as I know they're not so long But it could be interesting having Again the JVN where you have parallel strategies and so on What about threads all the you cannot have Parallel code running because they are global inter-terlock as far as I know But I don't get your question Can you elaborate more or if I run more than one thread and each thread allocates and then it has to be garbage collected is the garbage collectors Specific for thread Can you force the garbage collector to run or is it free also to ignore it if you can if you call it Can you repeat the question? I didn't hear you Can you force the garbage collector to run and if it does is it free to ignore it? That you can force a cycle on GC module At least on CPython you can import GC you can force a collection a cycle detection because On reference counting resources are Free when when they are not used anymore you can force a cycle detection But you have an API to manage that collector. Could an object be resurrected by its finalizer. I Didn't hear you can you repeat the question can an object be resurrected by its finalizer Yeah Those that's one of the cases that you have to handle. Yeah That's why it's not good practice too is the garbage collector the one that collect cycles gathering any statistics can I Inspect and see some statistics. Yeah, you have through this API on GC module you can You can check as far as I can remember number of Ritual objects Well, you have Documented somewhere but it It it Check for statistics is called the time so important you see you can you can access to some of them Yeah, definitely Does it run with a certain frequency or How often or on CPython it depends on this threshold So it's time that an object is allocated is appended to the jungle generation list and once we Reach this threshold a cycle detection cycle Algorithm is done. Last question. We'll have time for more So, do you know about some user friendly tools to detect the memory leaks before you deploy your application? I think there are some but I don't know on the worst case you can use Valkyrie But not so useful friendly and you have to know the internals But probably there are some that I'm not aware of sorry great presentation. Thank you very much everyone