 Okay, let's bring Gaurav to the stage. All right, let's shoot it out. Yeah Good morning I think it's a good evening for you. Yes, you said it's It's it's late it's late. Good evening. I would say units. It's 11 30 or almost 12 at And 12 p.m. And look at look at you know, how fresh he is so so that's an I'm gonna come to rally and ask you What do you eat for dinner? Oh, yeah, well because of your freshness because of the freshness you're still very fresh I'm a turkey. I'm a steak and wine person. So If you come you will be We'll be more than happy to have you but if you're a vegetarian we can find something for you as well Thanks, that's an All right, Gaurav the stage is yours. Thank you for coming and everybody including us are super excited to See what you brought us Yes, and and Gaurav Gaurav used to do all of these things Gaurav is Is is one of the influencers I would say on linkedin and youtube He produced lots of great content around around, you know system architectures and java and algorithms So so Gaurav you are already a popular star on the internet and we are glad to have you on on the show Thank you. Thank you Gaurav. Thank you for making this possible. Uh, it's taken a lot of A lot of work to make this possible. So thank you both Karan and Edson. Okay, so let's then start today what I'll be talking about is The evolution of garbage collection algorithms Specifically in java, but the good news is this is going to be a general topic. I'll just share my screen Okay Here it is Right, so my screen is now visible so The topic name is the evolution of garbage collection algorithms in java But like I said, most of these concepts are actually used in multiple languages. So Apart from python, which has its own way of dealing with garbage These concepts are used by go these concepts are used by a lot of the other languages In fact, some variations of python also use The algorithms that we're going to be talking about Okay The agenda of this talk is that you understand what garbage collection Is what these algorithms do so in case you have some sort of An issue that you're dealing with in production You should be able to reason your way out Thinking about why I did the garbage collection algorithm take too much time or you know, how can I optimize my process so that I am Nice to the garbage collector, which is always running in the background It's like a big brother. They know you when you are and they they are always in the background so Also while talking about this we are going to be thinking about what are the problems that are faced by a garbage collector and how they are solved and then finally the gc evolution this is specific to java, but we you know try to make it as generic as possible Okay, who am I I am the founder of interview ready. I have been working with java for the past Eight years. So I like the language quite a bit In fact, most of the content that I've created on youtube when it needs some sort of a program. I always code in java So that's the reason why I like working with java also In the companies that I've worked in whether it's a direct i or Morgan Stanley most of the code that I've written is in java Okay, the first problem with garbage collection When you need to collect garbage before that you need to actually identify what garbage is so There is one specific algorithm for this called the tricolor algorithm You can think of this as a graph algorithm Okay There's three types of objects in your memory one is objects that you're currently using The second is objects that you are unsure of You know whether using them or not using and the third one is objects that you're not using at all so useful Unsure useless Okay, and the way we find which objects are useless which objects are garbage is by Traversing this graph By using a stack. So when you use a stack it becomes a depth first search Okay, so let's say our root node is a And we have a in the stack Then we find b because it has a link to a so you are doing some sort of adjacency searching C comes in c does not have any outgoing edge. So c is marked as used Okay, and similarly we run through the entire stack But you'll notice that at any point we did not touch f and g So the nodes which are not touched after the completion of the graph run are considered as garbage Which makes sense because if you can't point to it, you can't use it if you can't use it, it's garbage So get rid of it and you're freed up memory okay There's a few problems With this approach. There's no problem with the approach rather. There's a few problems with garbage collection in general The first thing is when you're doing this the program is not running The program has stopped the garbage collector comes stops your program Finds out all the garbage collects it And while this is happening all the requests that you're getting in your server are not being You know handling maybe you are running some sort of background process. That's also stopped everything stops when the garbage collector runs Right because it needs its own space if you are running your program while the garbage collector is running in parallel Maybe you make some extra pointers here, which makes this algorithm not work You can think of some scenarios where you add a new object to a black object And there might be a failure. So we look into that later in attempts to make it parallel But so this is a slow process The the garbage is being collected, but we want to make it faster. So there's three ways to do this The first one is to try to identify certain patterns in garbage collection itself the The idea of garbage collection is that you want to take only useless memory So can you find where users memory usually exists and optimize there? The second one is to split the graph into regions Right if this is what we do whether it is sharding whether it is Taking a recursive approach any any kind of large problem is broken into sub problems. That's a computer science way of handling things As you can see the first one is a bit domain specific though And the third one is run them in parallel if you have the number of resources or threads or you know If you can run things concurrently or in parallel usually the process speeds up Latency goes down While the garbage collector runs for one second the next second you can actually respond to application requests And then again run the garbage collector. So you're time slicing and you're running things in Concurrently Does anyone have any questions till now? What we have done till now is just identified garbage and Got rid of the white objects This looks great Let's go for step one the easiest way the greedy way is to identify patterns and change our program according to what The reality is right. We find places where we can optimize and then Optimize only those places to get the maximum returns What's one observation that we can have here when it comes to garbage collection? The fact is just like human beings if an object has stayed alive for some time You can expect it to stay alive for a long time if a person lives for five years Then the chance of them living till 70 is is quite high But if the person hasn't made it till one year, you're not sure Okay, that's a high infant mortality rate is one way to put it It's the same thing with young objects Usually a function needs some objects the objects are created after sometime the objects are collected their garbage But if some objects stay for a long time, there's a threshold period Then you know that once they are in this phase They're probably going to last for a very long time Till this point and then you can see very old objects are no longer relevant So sometimes, you know, that's one way that At this phase, maybe you can keep another garbage collecting line But the idea is simple you have young objects and old objects young people tend to die quickly Therefore you should run your garbage collector more frequently in the young object region Okay, uh, that is the generational hypothesis How do we use this? We can put a we can put a region of young generation role generation. So That's what we have done here You're seeing there are some extra reasons. We'll talk about them later. Just ignore perm gen because that's like permanent objects More importantly, we have the old generation in blue, which we are ignoring because the rate at which you're running old generation garbage collection is less Also Guys Hello, I'm sorry. I don't know if I said hi. Thank you so much for the kind words on the comments Uh, and I'll just get you once we are done with the young generation role generation fiasco Okay, uh, so we are interested in the green region because we are usually collecting objects from there, right? And you can see the green is split into three regions Eden comes from the bible It basically means that just baby objects just the start Of the like when objects are just initialized you put them in the Eden region Then you have survivors you can think of these people who have survived one garbage collection cycle or two garbage collection cycles all right So in the start, let's say you have six objects which initialized. This was a function you're running six of them survived Or rather six of them are initialized, but only two of them survived So one and five are survivors. Everything else was useless. You pick it up as garbage What do you do now? Well, the survivors should be put in the survivor region Why am I doing this because eventually I want to promote all objects into the old region the useful people Into the old region so that I don't have to continuously check them and waste my time checking whether this object is alive or not So I put them in survivor one Now some more objects are initialized some of them a lot of them die rather uh and object number 11 Is surviving So I could put object number 11 in survivor one But I don't do that because if I try to do that then I'll have to You know find the right place for object 11 in survivor one, which is its region If I have two survivor regions, what happens is I can easily copy all the objects in survivor one And paste them in one place in survivor two the live objects And uh object 11 can just go and slide on to the remaining objects So this is efficient And I can clear Eden and this can go on happening so you know Yeah, eventually what happens is like An object survives for quite a bit of time. You can tell that dark green right now So object one and five are deserving of being sent to the next region. You can now go to the old region Now object one and five will be assumed to live for a long time. Therefore. I won't check them continuously Whether they're garbage or not Okay, Rishikesh Devkar has an interesting doubt. What is the frequency of gc run in the old generation? um so right now You can okay. So firstly that is configurable. Uh, the second thing is As you'll see the old generation was usually a stop the world garbage collection. Uh, we used to Oh no, the young generation has stopped the world garbage collection, but you'll see that the old generation also will find some optimizations uh Frequency run of old generation is Less than that in the young generation. Uh, how much less? Uh, it can be configured Okay, uh, you can make it 10x less than the the number of Garbage collections that you know in the young region. Uh, it depends a lot on this graph if I Go back here on this graph So it's a standard value in java Uh, and I don't know the exact value But uh, looking at this graph, I am sure you know, you can integrate the area of the left hand side and integrate the area of the right hand side And then get a probability But yeah, that's a that's a good question Uh, just to keep in mind you don't want to run your garbage collection process too often If you run it too often, then your application is not running. You're not serving requests But if you run it too late, then maybe your application will run out of memory And the only way for your application to run is to stop everything and run the garbage collector Okay, as we'll see the garbage collector should ideally run concurrently We are now promoting objects to the old region. This is good news But the old objects still need some scanning. Okay, and why is this the case? Well, take the green region as the even region or as the young region the blue one as the old one We can't simply kill all objects two three five and six Why because they may have references From the old region you're seeing that old object two is pointing to three and five here So collecting them is a mistake because old object two is still pointing to them And if you refer old object two dot three dot println, then it's going to throw an exception Okay, so you need the pointers from the old object to the new object and the problem is When you do this, uh, you are effectively scanning the old region also Right lots of objects which you don't want to scan because they are rarely garbage collected But you have to basically scan the entire memory just to see the pointers to the young region So it kind of defeats our purpose What we instead want to do is that we want to Note down all the pointers from the old region to the young region Immediately when they are being made right a hash table or a bloom filter You know some way to tell that this object is marked as alive So the idea here is uh is interesting Every time an old object is pointing to a new object you note it down how You use something called port injection Okay in java, uh, you you might have heard of dot class files So dot class files are assumed to run on any kind of hardware. It just depends on the interpreter uh That that is what we are taking advantage of just in time basically injects some code Every time you say that uh old object two is pointing to three The moment you say that you're pointing to a young generation object There is some code which runs a if check if old object in old region and three in young region then Add code. What is that ad code? Well go to the hash map and say that old object two is pointing to three therefore you cannot garbage collect it So like a boolean value you're which is going to market as true three is alive Uh, but technically what happens is you're actually using something called a card table Very similar to what we were talking about on a hash map or a bloom filter. Uh, the card table says that Old object two is an object that you have to look into okay, uh Quick recap. This might be going a little difficult. So When you have the young region Having dead objects you cannot collect them because some of them may be being pointed to by the old region So they're technically alive The old region now is split into regions split into sections And every section has a Boolean yes or no pointed to it. Okay one or zero If it is marked as one you have to go and check for pointers in the young region from there So you're seeing three here is a is a section of the old region Which has been marked as red means go through all the objects over there and check their pointers to the young region The way we did this the way we mark three is by actually having a right value This is uh This is an interesting concept. Let me know if you guys have any doubts on that Okay, shishu m is asking Did you ever need to play nice to the garbage collector when you were writing code because the gc was not happy with your code During your office days in terektai or some other companies The strange thing is we were not even coding in java then we were using elastic search which has been written in java and what happened is There was a lot of so elastic search when you're sending a lot of data what it does is it Updates its tables So I think they call it segments um but The the table updation was so fast Right because we were sending data in bulk and in in uh hundreds of mbs on gbs. So the The elastic search Was finding it short. I mean finding uh its capacity less Basically with so many updates it needs to make a lot of updates to its own table, uh, which results in a lack of memory So it was basically exploding its table. Um, and then what was happening is it was running a garbage collector Uh, so the garbage collector was trying to find memory because it it was short on memory To the point that it started doing stop the world garbage collection Stop the world garbage collection basically means that nothing is going to run right now Everything that we have talked about till now is stop the world garbage collection we are going to look at how we can speed it up but The idea is that you know if you're running stop the world garbage collection none of your requests are being met So our service used to go down So this was happening in uber. We were using elastic search And we learned the hard way that the garbage collector if nothing else then the garbage collector will provide rate limiting for you It's I think what happened is we started thinking about how to optimize garbage collection But unfortunately, we should have thought about how to optimize our own program. So we were not nice to it Uh, that was one scenario. Yeah Ruchikesh asks does the object have some kind of flag that maintains the number of gc that object has survived in the young Region before moving to the old region. Yes, there is a counter Uh, there's a counter which tells that how many times this object has actually survived In the young region. So in the edin You basically haven't survived anything. Uh, survive region one. Yes, you have a counter survive region two. Okay, two survivals And then depends on what you want to configure it to but after a certain period of time, you're promoted to the old region Okay, this looks good Pankaj agarwal edin young object stops everything old object does not need to stop everyone Yes Yes, the edin or let's say the young generation garbage collection is the stop the world process There are certain garbage collectors. I think c4 is one which does young generation also in parallel or concurrently But the young generation is usually small And there's a lot of garbage. So usually this is a very fast process. The old generation is usually big So doing garbage collection there, uh with optimizations makes sense Okay Ajay kumar, I think we cannot control the frequency of gc run in the young old generation depends a lot on factors Including the jbm architecture. That's true. Uh gc is triggered when jbm identifies that young old generation is almost full We can configure at what percentage we want to trigger the gc, which again is not guaranteed. Absolutely. Yes, that is true Uh, the percentage that you're talking about has recently been it's not very recent also It's a g1 gc. Uh, since then we have started doing that. So it makes a lot of sense To to configure it. But yeah, like you said, uh, there's no guarantee that okay 60 percent garbage collectors going to run It's more like a indicator to the garbage collector that if you know, you want to at 60 percent, please do the garbage collection Okay, so let's head to the next part. Uh You know, which is that the generation hypothesis may not be true. Uh, can you tell me one place where the generation hypothesis Is false where young objects Usually tend to survive and old objects tend to die It's a very, uh standard engineering thing that we use especially on top of databases or on top of expensive queries Where the younger objects tend to survive Because they're being used right now and the older objects tend to die Because they're no longer relevant any any application that you can think about which does this we usually You know to speed queries up to uh To reduce network calls. Okay, I'll let you guys know it's a cache Okay, uh, the cache is a place where you Yeah, LRU scenarios Rishikesh. That's absolutely correct. So Avinaba also has it right LRU is like least recently used so it's one policy of a cache But in a cache usually there's something like an expiry time, which is Which basically sets a cap you can't get older than this You're going to die at this point. So that's pretty weird for a young and old generation garbage collector Okay, the second thing is that older generation objects tend to die actually really quickly In a cache because they are no longer relevant so, uh Well, the garbage collection hypothesis fails here. That's one example. The other thing is Q When you have let's say a doubly linguist. So you have a front pointer Some of these let's say the one two three are old objects So initially you pushed them so it makes sense for them to go into the older region uh Now three is pointing to four But let's say your front pointer went ahead. Okay, and your front pointer is now pointing to five The problem is the old generation three object is pointing to four and because old generation is You know collected rarely it means that three is continuing pointing to four The the card table still has that entry. So four is still alive and you can't collect it This is called nepotism where dead objects or old objects promote The younger objects into the old generation. You can see that four is still alive After a few gc cycles four is going to be promoted to the old region Okay, even though three is dead the reason three is not being collected like we said is because the old generation is not collected often enough So three promotes four which is dead. Let's say five also dies four eventually promotes five and so there's this uh Sludge of dead bodies which are being pushed into the old generation and finally collected in the single soup of the old generation garbage collection so here the Generation hypothesis is again a bad idea You can think about apache Kafka which handles this it handles it by using uh, you know pages. So it There's two ways to handle this one of them is to remove the generation hypothesis the other one that Kafka I think uses is Uh, it uses Pages of memory which it can get rid of in one go So you can have a look at that. I'll add that in the in the link After this presentation, I'll just share that with you guys Concurrency, uh, ask any engineer. How do you speed up process up and they say let's paralyze it Usually, you know domain specific things have been taken care of now We are looking for engineering ways to speed things up concurrency paralyzation. These are things that we want to do Yes, the nation now wants to know who's at the head of the cube uh nepotism so okay Concurrency is awesome. You know, you can take A process and you can slice it. Uh, and then once you slice that task You can actually give it to different threads and they can do that job for you uh the and You know, we can take this graph slice into pieces and give it to different threads Which can find garbage and not find garbage. Okay, um apart from the algorithm complexity of splitting a graph the problem here is that We might have something like this you go to a mark it as you know, uh alive b c Okay, everything is going fine c is marked as totally alive black because There's no outer node and here's the interesting thing Let's say that application wakes up Okay, we are going for concurrency right like like we uh want the garbage collector and the application to run concurrently Which means that at any point in time in that code execution The application can wake up or the garbage collector can wake up and start doing its job without being aware that the other person is actually running So the application makes up here. Um It makes a new object f Then it makes a pointer from c to f And then it kicks out the pointer from d to f. Okay, this is the application doing its job So the object graph can be manipulated in any way by the application uh Now the garbage collector wakes up Look at the stack when the garbage collector wakes up b's and on top of the stack right left Opposite b's on top of the stack D now sees that d is alive Uh marks it d sees that is alive and all nodes are marked except f Which is crazy right because the application clearly said that uh i'm creating this new object f And c is pointing to it which is a live object So f should have been marked, but it's not and now it's killed if you call f dot function I'm going to throw another pointer exception or some sort of an exception. So illegal memory address exception. Let's see Okay, that's the problem when you're running things concurrently you might be killing objects Which are not being pointed to uh by any of the unmarked objects In your algorithm. So if a black object is pointing to something a new object Well, black is already taken care of so your graph is not going to traverse through the black node again right, uh The way we deal with this again is code injection The problem of losing live objects through code injection can be solved. Let's see how when you're pointing uh from From c to f A black object is pointing to a white object. That's the only time this problem happens If a gray one is pointing to a white object, nothing happens because you're going to go through that node, right? You can see a and b which are gray are in the stack So you're going to go through that node and make sure that all uh untouched objects are picked up It's only when black points to white Okay, so What is one possible solution? One possible solution is to respect the wave front Uh, here, I'll just share my screen Oh, okay, I'm nearly out of time. So Okay, um The basic ideas I'll never allow this to happen. Uh, whenever a black object start pointing to a white object Again, my interpreter is going to see if object equal to black and pointed object equal to white Run the if condition if condition is basically set the object as gray Okay, so f is marked as gray gc wakes up Uh, and you know now f is on top of the stack Market is black back. Okay, like the entire sub graph of f is going to be taken care of And then eventually things work fine. So you don't lose any object. I'll just check current is it Is it time should I I mean we can I can share the presentation So I think uh, yeah, I mean we can spend still spend like two or two three minutes, I guess Okay, okay, so, um, I'll just quickly run through the third part because it was talked about As g1 gc, uh, the third thing which you do very often is sharding split into regions split into sub problems Basic ideas you want to partition the heap You want to mark regions as survivor or old region or young region You want to remember the pointers with them? So I'll I'll just show you an example, which is easier Look at that green is even SS survivor, uh, which is yellow and uh, oh blue is old. Okay Uh The way this garbage collector works is it works region by region It doesn't work on the entire garbage on the entire memory You know graph, uh, it splits it into regions Uh Regions are converted to a single region. So here you can see three survivors And a few other old regions are being converted to a single old region So some survivors are being promoted while some old regions are being compacted into a single old region That's a general idea of the g1 gc Yeah, yeah Okay, I'll just take a couple one doubt here Isn't it like this that when we when all the links to a particular object are not there even if in the old generation gets collected by the gc no no All the links to a particular object are not there even if it is in the old region gets collected by gc. Yes, uh, so I'm sorry. I read that wrong If it is a garbage object, it will be collected So the old region object also if it is garbage, then when you run this algorithm, it's going to give you I mean, it's going to pick up that garbage On generation hypothesis. This g id logic is added for concurrency. Yes Usually yes, but there are certain times when you want to add the g id logic for simplicity also um Deepak Anand ask And this uh asking can we talk about g1 gc? Yeah, the okay. What are the benefits of g1 gc? I'll tell you One is that it's incremental. You don't need to run it through the entire memory graph The other thing is that garbage collection can happen concurrently Which is awesome, uh, and it's easier so to speak because it's split into regions And it also has some advantages like compaction Right, uh, it's taking a lot of regions and compacting it into a single region. So, you know where the garbage or the objects are Okay The only thing remaining is actually to making that compaction process concurrent So those are the three garbage collectors. We have talked about the first one was a simple garbage collector. It's it's uh, it's basically It's not even concurrent right? It's the basic garbage collector. Um mark and sweep The second one is concurrent garbage collector concurrent mark and sweep um Which is here Allows concurrency the third one is g1 gc Which is pretty good. Uh, but there's a newer garbage collector called Shenandoah Or zjc Those are two different garbage collectors actually, but they are also pretty fast low latency slightly lower throughput also unfortunately, but That's a tradeoff From java, can you give how object can land to old generation objects? Yeah Like we said, if it survives for a few cycles of garbage collection cycles promoted to the old region That uh, pankajakarwal asked that question Uh, the example that we said is, you know, if you let's say have a object which survives for multiple, uh, gc cycles, let's say you create a static object Although I think that'll go in that that should be put in the old region regardless because the static object is Just there Okay, uh Thank you. Thank you. Beshikesh Yeah, we can see like a great feedback from from the questions and from the comments. Yeah, unfortunately I'm sure we could stay for another hour watching you present this amazing content But we have to move forward So, yeah, I would like to thank you so much. Thank you. Thank you