 10 seconds of my time in here, so I think I'm gonna get started. So, hello everybody. So the title of my talk is that one that I can read in the end. Tweaking will be a garbage collector, parameters for fun, speed, and profit. My part is the fun part, for my boss is really speed and profit, so it's how I got to convince him to give me time to work on that. So before I start, let me just tell a little bit of story here. So, who is here? All right, it's better? Okay, I'm gonna try to stay still in here. So, who is here for the very first time? All right, it's a pretty good audience, and that's something that's really amazing to me about RailsConf. Last year was my first RailsConf, and I had a really great time. I thought it was awesome conference, and after it finished, I decided that this year I wanted to come here as a speaker, and as a matter of fact, I'm here. So, if any of you guys that are here for the very first time, if you like the conference, and after that, you think it's really awesome, you're not alone, a lot of people is gonna think like that, but if you decided to come next year as a speaker, I can assure you that's totally possible, and if you have any questions about that, if you wanna talk, if you really get interested in that, let me know, I can share a lot of the road that took me here, and so with that out of the way, my name is Elio Colla. I've been doing software development for about 15 years. I spent about 10 years doing C, C++, Solaris, Linux environment before I switched to Ruby on Rails. It's been about five, six years that I've been working for Ruby on Rails, and I go by that thing on the internet, if you wanna find me. So all right, let's get started, and let's talk about the Ruby garbage collector. So throughout the presentation, I'm gonna go through some of these terms. GC, garbage collector, IGM DC is the restricted generation of garbage collector. The INC DC, it's the restricted incremental garbage collector. COW, it's copper and white technique, AB is a batch plan back tool, just a couple of terms. My talk is gonna follow these topics. So while I'm here talking about garbage collector, despite my little story at the beginning, something, a little bit about the history of how the Ruby garbage collector algorithm evolved through the Ruby releases, and some configuration parameters, how they evolved, and my approach into measuring tuning this thing. That's, these options are gonna be how I got to this talk, how I created all these, and how I learned about all these. And at the end, I hope I have some time to, for some questions and answers. All right, so why I'm here and why garbage collector? So also upon a time, there was a Rails app. There was one, two, three, four, five Rails app. And they all input their production, leaving, live and prosper. But we didn't have a lot of inside story on how they were behaving. So we, practically, the company decided to install one of these fancy, full of fence tracks, monitoring tools. And then that's how all these got started. We installed these monitoring tool and we start seeing, I got exposed to this information. And I was looking into those things and I was like, all right. On the left side, I have everything mostly blue. It only says GC runs. On the right side, I have these mostly yellow, a little bit of blue, and it shows minor images. So, hmm, what is that? I don't know. But it sounds like something interesting to find out. And I also saw these, on the left side, you can see that the average is 80 times. That's how many times the GC is running per hundred transaction and the right side is running 46. So I'm like, hmm, all right. The right side seems better. So why my left side app is not behaving as my right side app? So I wanna find that out. And up until that time, I have enough questions that got me going and got me interested in these. So I did a lot of research. I read a lot. I Googled everything I could find about Ruby Garbage Collector. And I had a really great time. I really, really have a great time learning all that. Was pretty awesome. I guess I was a little bit bored at work as well. But I decided that I wanted to share that because despite having found a lot of documentation, I didn't see a lot of people talking about the Ruby Garbage Collector. And finally, my talk got accepted, right? So if that didn't happen, I wouldn't be here. So I'm gonna tell, before I dive into how I approach these things, I'm just gonna give you guys a little glance on how the algorithms evolved through the Ruby releases. So let me ask a question before here. So how many of you guys here ever changed tune in Ruby Garbage Collector in production? All right, that is a handful, a dozen people. All right, that's interesting. You know, one of the meetups that I did in DC, I think one guy in the room, but it was a much smaller room anyway. So this part of the implementation, there's gonna be a lot of information. I don't expect you guys to follow everything, to get everything on your mind. I'm only gonna glance through these algorithms. I'm not gonna explain them in details. But if that's something that you get interested in, if you get curious about that, just let me know. There is a lot of documentation that I can point you out. You can Google around it as well and find that. But I have some reference that you guys can use and I can point that to you. So basically 1.8, the algorithm is simple market sweep. 1.3 is the lay sweep, 2.0 and the bitmap masking. The Ruby 2.1 is the gem DC. Ruby 2.2 come with the ink DC and symbol DC. So during my research, I came across a mom's good to blog and I didn't come across this blog at the beginning of my research. It was close to the end. It was after I read a lot about the Ruby garbage collector mostly about the Gen DC and Ink DC, the generation incremental garbage collector. That's where you find more documentation about it. And then at the time to the end of my research, I came across this blog and I am a visual person. And when I saw these and I saw how he concisely and visually expressed the difference between these different algorithms, it all clicked in my head. And he also gave me some perspective on how prior to 2.1 and 2.0 X, how those algorithms evolved. And so for the next couple of slides, I'm just gonna give you a couple of screenshots from his blog and how he showed that in a visual way. It should be used to digest even though there's a lot of behind the scenes. So 1.8 is simple marking sweep. So basically you have a mark phase and sweep phase during the time that is to stop the road. That means that your code's not running because the garbage collector is running. And it marks a bunch of objects and then the ones that are not being referenced, they got swept and then your code resumes. 1.9.3 is the latest sweep. The main difference here is the mark phase is still the same but the sweep it's doing lazily. So as the objects are required, they are released and they are swapped for you to use. And that's the 1.9.3 latest sweep. 2.0 bitmap masking. This is just about memory management. It's improved memory management for our gems and applications that do fork and do those kind of things. And also the mark phase will be written to be non-recursive. That's what's going there. But the main logic about the marking sweep phase remain the same. And if you plug a 2.0 application in one of these fancy full-off fancy charts monitoring tool, that's what you see. You see GC runs. This is from a simple application that I have on my machine. Ruby 2.1. Here's where the juice starts. So generation of garbage collector. The idea here, the objects die young. You probably heard about that. And if an object survives a GC run, it gets promoted to old generation objects. So next time it's run, a minor GC, it doesn't go through all the objects, only the young objects. So you traverse a smaller list of objects and so we spend less time. And that's kind of the whole idea of the generation of DC. Again, there is a lot more into that. I'm just gonna go into these to give you guys some basics. The 2.2 has the ink GC and the symbol GC. Symbol GC, hey, just get collected now. So if you're a symbol lover, you don't run the risk of a DOS attack crashing your app anymore. So the symbols are gonna get collected. And the incremental DC. So the incremental DC is all about shorting the pause times. As you can see, it's just a replacement of a long mark pause is replaced by a few small mark faces. And from the implementer, Koishi Asasada, which I think is here in the room. This is not a silver bullet. It's not a guarantee that's gonna increase your throughput and improve your web response time. So it was all about replacing a long pause to a small pause. And if you plug a 2.1 and 2.2 application and also 2.3 into one of these full of fancy charts monitor into that's what you see. You see major and minor. The major should run less often than the minor. And that's how it looks. So a couple of configuration parameters. The same way that the algorithm evolved and got more complex and faster. Also the configuration parameters they evolved. And growing number and complexity. In the 2.0 we have these three configuration parameters. Maloc limit, hip means lots and framing. This is all about how many slots you allocate during your application startup and how many slots needs to be free. The minimum number of slots needs to be free after I just see runs. So it's gonna release a couple of memory but that is not that amount of free slots. It's just gonna allocate more for your application to use. And Ruby 2.0 X we have these three parameters. As you grew to 2.1 and above we have now 11 configuration parameters. There is some documentation explanation about all these parameters. I'm not gonna go into details here because you probably gonna leave the room if I do that. So but basically those are the parameters. Hip needs lots, hip free is lots. Hip growth factor, hip growth max lots. Hip old generation object limit factor. This is a pretty cool one. I kinda like it. So now you have these other three sets of configuration parameters that are related to a young and old generation objects. Limit, limit, maloc, limit max, maloc, limit growth factor and these things. I didn't poke around all of them. I kinda changed most of them and tested when I was doing all these work. But at the end, I didn't end up changing all of them. All of them had D4 values. So if you don't change, there's gonna be a D4 experience. But sometimes that is not your best option. If you look at the Ruby source code at gc.c5 and I have the URL for the references in there, that's what you're gonna see. On the 2.1 X and above you have these long C commented section that lists all the configuration parameters, some documentation. And if you see there, if you can read that, the first one, they need slots, it's initial allocations slots. The one that I thought was cool was the gc hip old object limit factor, which is do a full gc when the number of old objects is more than r times n. Where r is this factor and n is the number of old objects just after last full gc. First of all, these description here, when I read that, I was like, holy shit, that's pretty cool. So as I said, I worked for almost a decade with C and C++. So going back to these and reading these stuff and compiling the Ruby source code and making that compiled version of Ruby be used on my Rails app, it was pretty awesome to me. Before that, I never really went back to C and C++. So that kind of helped me have some fun during this process. And at the end, if you're wondering, that's how you actually test that. So if you wanna change some of these configuration parameters and test that in your application, in your laptop, that's what you do. Those are all environment variables on your Linux machine. So if you export that variable within some value and then do Rails as, then when the Ruby VM starts and the garbage card gets set up, it's gonna pick up those variables from the environment and it's gonna create the memories accordingly to that. If you wanna change them, you stop Rails, see export these values for different values and we start it again. So for testing your laptop, that's pretty much what you do. So with that kind of, I finished all these stuff that I wanted to talk the glance to you. If those algorithms, details about that is something that you get curious and it occurs to learn more about it. I can point you guys to some documentation, but also I strongly recommend you guys check out Eric Weisstein talk, Ruby Conf last year. He gave a talk on garbage collector as well, but his approach was explaining more into details how all these algorithms work. So I was here, I saw his talk and I watched his talk after I'd been to all these, learning about all these stuff. So watching him explain these algorithms was pretty cool. I really liked it. I really felt like I was back in college and watching my CS teacher explaining algorithms. It was pretty cool, I liked it. And as a matter of fact, I don't like everything, but the things that I'm talking about, probably there's a reason for that. And so I think his talk was pretty cool, I really liked it. So now let's go back here to where everything started. So as I mentioned, I got exposed to these things that you've seen. The left side you have the GC runs and the right side you have my major and minor and those average on top 80 versus 40, it kind of bugged me. I was like, hmm, that sounds like there is some here, something here for me to improve. But I wasn't sure whether those numbers were normal. Maybe I'm in a normal scenario. During my research, I couldn't find any documentation or anything that gave me a clue that all right, if you're running on average from zero to 15, you are awesome. If you're from 15 to 25, you're great. 25 to 75 is okay. And above 75, you're really bad. I couldn't find anything you'll get to any knowledge about that. So I wasn't sure whether my 46 was great or was really bad. Maybe tomorrow I'm gonna get a call overnight saying that my API is down, the sun is down and I'm gonna have to wake up in the middle of the night to fix this application and try to find out why the machine is dying. I wasn't sure and that was one of the reasons that why we set up, we installed these monitoring to run our application because we wanna get a more in-tail and more practically measure and see how healthy our applications were. So I went on research mode. That's what got into me all these. I read, I studied the source code. I compiled the Ruby source code. I did all of that and as I mentioned, I haven't really got great time. And after a lot of research, and I realized, all right, that Ruby 2.0x, I'm not gonna worry about that. That's gonna get into upgrade path. It's gonna go to 2.1.2.2 and then the 1.2.2, all right. Let's find out what I'm gonna do. But that application 2.1.x, as I mentioned, I didn't know whether 46 was normal. Maybe 46 is super great. I mean, a really good spot. Don't need to worry about these, but I wasn't sure I needed to find out. So what did I do? All right, and then I talked to myself. All right, let me run all these on my laptop, do some tests and do some load tests, some basic load tests. I'm not gonna go too crazy, not spend too much time. And maybe I'm gonna find out something. Maybe that's gonna give me some clue to whether 40, it's good or it's bad, or maybe I'm gonna realize that I'm just crazy and go do something else. So I did that. I did a lot of tests. And at the end, that's what happened. And still on my laptop, despite everything that I do, the GC was running 3.99 on average. I was like, hmm, all right. So it's not like what I'm seeing in my production apps. My production apps are running 40 times out of, on average, 40 times out of 100 transactions. And then what did I do? A couple more days go by, testing, imaging, analyzing data, and doing the process of what I realized is all right. So the tests that I'm doing on my laptop is not the same as my AWS servers. Not the same architecture, hydro architecture of my production environment. Maybe the tests that I'm doing on my staging environment, which is shared between a bunch of applications, has the similar architecture. So maybe those tests are gonna be better than what I do on my laptop. I don't have the same amount of data. In production, I have a freaking amount of data. Huge amount of data. I didn't have that neither on my laptop nor on my staging environment. In my AB calls, maybe they're not reflecting exactly how my endpoints have been used for my users. So then I kind of got to the point, right. I'm not gonna try too much to get to, to simulate my production environment, how much pressure I'm put on GCU and production in my testing environment because I may never get there. And so what did I do? I slapped on it. Say, all right, I'm doing too many tests. I have too much stuff in my head and I needed to go do something else. And my boss was bugging me about that other feature that I wasn't working on. And then during those meetings, after those meetings, and a couple hours hang out in the meeting room and I tried to talk to some of my teammates about the thing that I was trying to do and try to get a sense of, right, maybe they are gonna say, dude, you're crazy, forget about these. Or maybe they're gonna say, oh yeah, it makes sense. All right, what are you saying? Your theory kind of makes sense. So have you thought about these? Have you thought about that? Have you seen these or that? So I tried to talk to them and explain to them what I was doing because maybe that's gonna help clear my mind and give me an idea on what I'm gonna do next. And what really helped me after that was really to plan what I was gonna do. Because up to this point, I have a lot of information about how these algorithms work, the parameters that I can change, some of the parameters that I change and work and doesn't work, and it makes a difference, it doesn't make any difference. So I've done a lot of tests and I had a lot of stuff. And planning what I was gonna do next and focus on that, it kinda helped me a lot to set myself on a path where I was able to find something that I thought was cool. So, and then after that, either these or that's gonna happen. Either you're gonna see something that you weren't seeing before. And it was there, right in my face. And I wasn't seeing that. So on my dev environment, the same fancy monitoring tool was showing that my object allocation per transaction is on a range of 13,000 objects, while in my production it was 94,000. It's like, oh, all right, that might be why the GC is not running as crazy as it is in production because my backend doesn't have a lot in my testing environment. I don't have the same amount of data, and so I'm manipulating less objects and I'm speeding out the JSON response much smaller than what it's doing. Maybe my AB calls is not actually trying to get as much data as my real users. And then when I did that, what happened? I got to something that was very similar to what I was seeing on my production environment. And that was still on my testing environment. So, and then I got to a point where, all right, I have an environment in here that simulates the exact same GC pressure, the same pressures in the GC garbage collector, that now I can control, I can test, I can replicate, and if I make any change, I can find out whether that's gonna improve or not, if that's gonna release the pressures on the garbage collector or not. So I was pretty happy about that. If that doesn't happen, if you can't find a scenario in your testing environment that kind of replicates in some level of similarity what you have in your production environment, then you establish that baseline, and then you work on that, and then you see that if you can make change on that, it's gonna improve or not. But if you can't replicate kind of, if you don't have a similar environment what you see in your production environment, then you kind of have a stronger case to make to whether the change that are gonna do in that is gonna have the same effect in your production environment. But anyway, so I did got that, I did get into that actually, both on my laptop and on the staging environment when I added more data into my database and when I changed a little bit of how my AB calls were. And after I got to that, then it's about change, measure, and analyze. Change, measure, and analyze. Do that over and over again. Don't do that when you're hungry because it doesn't work. Do that early in the morning only. And analyze what you're reading. Try to read your scene. Remember, I spent a lot of days looking to those graphics and I didn't see for a long time, I didn't see that my dev environment, I was allocating 13,000 objects in my production environment was 94. So it was there, black and white, maybe kind of grayish and white, but I didn't see that. So you have to analyze, especially if you generate a lot of data, you have to take your time to let your brain to digest all that information. And one thing that really helped me is some of the numbers that I was getting during my tests, I literally wrote down in a post it near my laptop and I kept really, kept looking into that from time to time, right? These use case, these test scenario, I got these numbers, these other tests, I got these and I literally wrote down in there. And that kind of helped me to try to extract some knowledge from there and get some clue out of that. So, and also the other thing is that's something that I always do is if you're testing these things and trying to extract knowledge from the tests that you're doing, try to do one change at a time because then you don't have to worry about, oh, which change I did affect that. So if you do one at a time, it's kind of a little painful to use, but it's a good thing. Again, this is not an advice to you. This is just something that worked for me in this particular scenario. You probably have all the different ways to test and to approach some of these things, but this is how I felt comfortable and how I was able to validate myself and the things I was doing while I was doing and learning about these. So, again, either these or that will happen. That's what I mentioned. If you find something, in my case, I did find a couple of configuration programs that improve my response time and I was really happy about that. If that really happens to you in something that you're doing, investigating, document it nice, share with your team, make a small presentation inside your company about that, make a small talk out of it, and maybe next year you're here, Ray Osconf speaking about it. And if you do that, especially in the company that I was working at time, after all that work that I've done was kind of pretty easy for me to deploy these stuff into production. And if you don't find anything, then you still document something. If you went through a lot of time, if you learned, if you tried a lot of different things and you still didn't work out, you still couldn't find anything, document the same way because maybe there's a lot of people in the world trying to do the same thing before you or alongside you and that if you share that information, maybe you're gonna save a lot of people's time or maybe you're gonna get some feedback from the community on what else you should do to solve the problem you're trying to solve. And so that's how I went by all these stuff and now from the next slides, they're all gonna be a couple of charts that's very easy for easy. And if you guys don't wanna do some of those things, you can piggyback on some of these images and you can send that to your boss. I'm pretty sure your boss are gonna like these charts and maybe that helps you to convince them to let you poke around some of these stuff. At the end, these were the parameters that I changed. I'm sorry. At the end, these were the parameters that I changed that I made all the way to production. Hapiness in each slot, freeze slot, maloc limit and limit max. I'm not sure if all of them actually, one or two of these combined was the one that resulted into the improvements but these are the ones that after a long, a lot of testing, those are the ones that I decided to make into production. So now I'm just gonna show you guys a little bit of some comparison. And some charts, some nice charts. At least I think they're nice. So I'm gonna compare Ruby 2.0X, 2.0.0 with 2.1.7 before any tuning, before experience. Imagine that you have an app with Ruby 2.0.0 and you migrate that Ruby version to 2.1.7. That's what you should see. We expect to see an application. Then I'm gonna compare Ruby 2.0, the same app with Ruby 2.1.7 before and after tuning these configuration parameters. And then it's gonna be the time of 2.2.3 before and after and 2.1.7 and 2.2.3 after the tuning just to find out how they compare and how those algorithms play a role into spending less time doing garbage collector. So as you can see here on the left side I can see on the left side is Ruby 2.0X and on the right side here it's Ruby 2.1.7 and that's how everything started. I had two apps with two versions of Ruby and in these cases the same application when I run and I do some load testing with Ruby 2.0.0 and the same application after upgrading Ruby version to 2.1.7 and out of the gate before experience the average of GC runs dropped from 80 to 40. That's what you get just by upgrading 2.0 to 2.1X. And so here now is a different metric. Again on top is 2.0.0, on the bottom is 2.1.7 and this metric is the time spent in garbage collector percentage of your wall clock time. I'm pretty sure somewhere there's a really nice explanation of wall clock time really means but that's a metric that they show up. If that's something that you're curious to just I have some more documentation about that as well. But basically by changing the Ruby it drops from 18% average to 3% average. Just remember this is an average of a percentage of a metric based on a percentage. So I have to keep our expectations regarding to this one. But this is a more easier metric web response time that this one is kind of very easy to digest. And if you can see on top the dark brown one it's the time doing garbage collector. And you can see that on top you pretty much spend close to 40 milliseconds per transaction doing garbage collector. That's 40 milliseconds of your web response time. And if you change from your application from 2.0.0 to 2.1X that drops you well below 20 milliseconds. So you're getting there close to 25, 30 milliseconds. And in this case you can see that the web response time goes from 156 to 133 milliseconds. So you're totally improving your web response time. And so two on seven before and after. Here you can also see on the left side is before on the right side is after. And the average of runs goes down from 40 to 24. These data is from my sample application that I run to get these charts to put in this presentation. What I really got into my real application when I did all these, we went down from 40 something to 12, 13, 14. Was some improvements a little bit better than what you're seeing here. And again the same time spent on garbage collector on top two on seven before, at the bottom two on seven after. And you see that the time spent on garbage collector drops from 3.28 to 2.38. So a small drop is a percentage metric but it's a drop of close to 30%. And in this case the web response time drops from 133 to 129. That's real gain in your application. Ruby two to three before and after. Then it's very similar to what we see on the two on seven. Remember that when I was showing and explaining the algorithm for the Ruby two to X that's where the incremental garbage collector that I introduced, but that wasn't about improving throughput or web response time, it was all about shortening pause times and it wasn't guaranteed that the incremental you see would improve your throughput and web response time. And then we're gonna see that when I compare the two on seven before, two on seven after and two to three after. And so two to three before and after very similar to two on seven before and after. The time spent in GC also drops from 238 to 1.4, a smaller drop in this case. Web response time during this test, it dropped from 137 to 130 milliseconds. And this is two on seven before and after. Here you see that the two to X is actually still a little bit worse than the two to zero X. That's probably just a little bit of noise in my test. You see that on the right side, that the chart kind of goes up and down. So probably I was doing something else with my laptop and was trying to test these scenarios. What I really got into the real app in my testing environment, when I tested two on seven after comparing to two to three after, I consistently got an improvement of about five, four, five, seven milliseconds of web response time. So, but again, the two to X, the two to X algorithm incrementally see is not supposed, it's not guaranteed that it's gonna give you a better throughput in response time. Not always, it's not guaranteed, but I kind of felt like because my results on those were consistently a little bit better, I mean five, six milliseconds, maybe it's just noise, but when it's consistently getting the same results, test after test, run after run after run, I kind of felt that my application was in a sweet spot where the change for the incremental garbage collector did help me, did help me, the application, right? And so this is two on X, so in this scenario I'm just comparing two on X before and after, and two on X, two on seven after, and two to three after. Oh my God, so many numbers. So it's dropped from 238 to comparing those two, one is 238% and the other one is 1.4. Here the web response time, it barely changed. This case is pretty much noise, most likely. And that's with that, I kind of conclude what I wanted to talk to you guys here today. If you have any questions, I'm more than happy to answer them. Thank you. So the question was if this comes with more memory usage, yes. So you're just telling the garbage collector to allocate more memory. In my case that was happening, I increased the heap and then I have more memory for my transactions to allocate 9,000 something objects and so the garbage collector runs less often because there is more memory available for him to run. Yeah, so the question is trading memory for time? Yeah, exactly, so since I have more memory and I find it in the machines, that's what I was doing. I was giving the Ruby VM more memory so it has more room to operate and so you need to run the garbage collector less often. What tool? So the question is what tool I use to measure the garbage collector? I use New Relic and so when we install New Relic in our application, we start seeing all these things. So I didn't use, actually I didn't use New Relic to test New Relic is how I was getting these results but what I used to do my load test, I just use a partial benchmark tool, I was just putting a small little load in my application to get the garbage collector to run as fast as with the same pressure that I was getting my production. So, all right, so the question was if, so these charts that I have are indeed from a simple application, it's not my real application but what I get on my real application and so one of the applications in the staging environment I got about 10% web response time increase and in production those numbers are a little smaller, production I have way more cash in the staging environment. These was one of the APIs, in some other APIs there was one that I couldn't see any change in production. I saw some changes in the staging environment five, six to 10% improvement in the staging environment in production was the same. And after the end of all these, pretty much all APIs are using Ruby 2.2x with these configuration parameters. There was actually one of the APIs that I use pretty much the same parameters for more than one application because they have the same range of object allocation. It was around 9,000 and but there was another application that I had to change because it has a memory footprint a little bit different from the others so we use different set of parameters. Okay, so the question was the before and after values. The after values are those that I put in here. The before is the default experience. Oh yeah, okay, so I didn't. Yeah, sorry, I didn't actually put the numbers in here. I can send you some of those. Yeah, I can send you some of the samples that I use for these ones. And the before was what is in the default version of Ruby. In the source code in some of these reference here you can find the default and I can point that to you. Maybe you can make a slide available. Yeah, yeah I can do that. Yeah, I can do that. I can definitely do that. Put the default values in the ones I use for these. Yeah, that's a pretty good question. So the question was how did I change these parameters if I did it systematically or if I used them or if I changed to anything. At the beginning I was, is that okay? Yeah, yeah. So at the beginning I was learning and testing and change all of them. After reading some of these parameters you kind of, some of them they go along together. So if you change one you have to change the other. But because in the 21x we have 11 parameters and not all of them are very easy for you to digest and change and to monitor the change I didn't really use. I kind of got, after doing a lot of tests I kind of get an idea on the ones I want to change. But the interesting point about the question is that is actually after I went through all these I found some documentation about not everywhere is affordable for you to test, right? So if you're a physicist working with our hydrons supercollar that you can test a lot. You have to be very careful with the tests that you're doing. How, which parameters are gonna use. And I was actually really curious on trying to investigate some of those algorithms and see if from all those parameters if there is a better, a sweeter combination that's gonna fit better for my application. Never really get time to do that. So the answer is at the end I really used some of the parameters that I felt was, it helped doing my tests. You see, I have a 90,000 something object allocation. So if I give memory to the Ruby machine to allocate the memory, if I give enough memory to the Ruby VM to allocate those objects without having to run the GC all the time then at least for my application it's gonna take care of the GC it's not gonna be necessary to run that often. And when I got to that point I was satisfied with the value that I was getting for those parameters. I think you're absolutely correct. I got to, I presented this talk in a conference in Europe and I met a guy in there who was a physicist and two developer. Huh, oh my gosh. He, oh my gosh. So and he actually told me about Taguchi tables, you know, talking about something. Throwing one at the top of my head but he was going to that. He's the one that mentioned about these examples where not in all industries you can test it's affordable to test as many options you have. You have to be, you have to study very well which parameters you're gonna change and then you run your test because running the test is really costly. For me, here running these tests changes this parameter is not costly. I can't afford doing that. But I'm pretty sure there is a better combination of parameters that would make it better. So yeah, so the question was if I should suggest to change the default value for these parameters and the answer is no and that we shouldn't do that because like you were saying these parameters are based on what is your memory allocation. So if your application allocates 10,000 objects per transaction, the default value is all good for you. Maybe your app will never gonna be in a point where you need these parameters to be tuned. So like they said in one of the other presentations maybe 99% of the Rails apps in production does not need to change these. But maybe you're in the 1% and I actually worry. One of our apps were a couple. All right guys, thank you so much.