 Hello my DevNation friends from everywhere in the world. Thank you for coming for another DevNation tech talk and Today well right now. We have a storm going through here in Cary, North Carolina I hope you're all safe and of course we have this still have the pandemic going on but no worries We have a very special guest today Christine flood is going to be talking everything about Java and containers and I'm sure you will be amazed by the content and Without any other thirdly Christine. This stage is yours. Welcome Hi everybody. I am here in Massachusetts where the weather is gorgeous, but we'll see how it goes I I submitted the abstract to this talk Java and containers. What's there to think about? But as I was writing the talk afterwards, I think a better title might be adventures with containers for JVM hackers I am not directly involved in Container effort, but I am on the open JDK team and I've spent some time playing with containers And I have spent a lot of my career dealing with JVM performance So first a little bit of history I kind of the evolution of Duke when I first started working with Java It was back in the 1990s and the thing we were most concerned of We wanted to be able to download bytecode from the web and run it safely on whatever machine people were using heterogeneous architectures so everything was about bytecode verification and making sure that that it was safe and Then in the 2000s everything was about making sure that Java was fast Right, you could use all the memory on the machine. You could use all the processors on the machine The only question was who could get you know Vlana mark or whatever who could run it the fastest and Then in the 2010s it was all about smooth Duke I'm pretty pleased with these pictures. Oh, and by the way If you have any questions feel free to post them on YouTube. I like to be interrupted I like to have a dialogue if at all possible So smooth Duke This came the era of quality of service guarantees and folks wanted to be able to promise response times in under 10 milliseconds and so all of our JVM work was focused on Making sure that garbage collection pauses were really short making sure that that things could get processed quickly But now it's 2020 and What people want is to fit as many Java virtual machines inside of containers on A physical box as possible So what we're optimizing for now is We want a smaller footprint we want faster startup and We want smaller image size So I'm distinguishing between the footprint and image size because the image size is the thing you download To start running and the footprint in this context is how big the JVM is when it's running your program So when I agreed to give this talk, I thought I'd be talking old school and about What kind of things you can do with open JDK to make your JVM smaller? things that you can do so for example if You're running in a container and you only have one thread You might want to think about using serial GC. It's not as well maintained. It's not the default But it was you know, it's been there since the very early days and it's meant to share a single thread with your Java program You can adjust the GC heap parameters saying oh don't grow the heap so fast Let's all or let's shrink the heap faster and try and get it back down You can adjust how much time you're spending in garbage collection So the default is that you're going to spend 1% of your time doing GC Maybe you want 25% of your time doing GC and keep your Your run size smaller Adaptive size policy weight there are folks who have found that this helped them a lot It's a simply scale from zero to a hundred if you adjust it up Then it's going to it's sort of like a knob. This is okay. Let the GC do more perturbations to trying and have it exactly where Your program needs it in terms of memory space Okay, so there's some more command line parameters There's native memory tracking So if you use these options Native memory tracking summary and print nnt statistics You can see the difference between what you asked your program to use and In terms of a heap size and the memory. It's actually using So there are things that your Java program needs to allocate like they a card table for the garbage collector or Other sorts of things that the JVM needs to run that aren't in the heap and This will give you the ability to look and see what how much memory all those things are taking in case you want to shrink that Each Java thread has a stack So you can decrease the Java stack size if you may all are Then that's less memory that your program will use and Finally, I put this at the bottom if you want to see what the default is for all these operation all these parameters you can print out all of the flags and you can see what you're running with and how you might want to adjust them okay, so and on the previous slide you said that well the default for the stack size is Yes 1024 yeah, 1024 from your experience. What is and a common recommendation for the stack size on most production workloads Well, I as I put here it was 228 K is something that seems to work It seems like it's a good place to start if you can run if you set the sex size stack size to 228 K and Your program still runs. You're good. Maybe you can even dial it down further I'm Doesn't mean we'll get stackover flow errors if it doesn't work your program will fail with a stack over with yes, and so that's just a place to start and you can tune it and Again, I am sort of shooting in the dark here because I have no idea what your program is and how deep your function calls go so It's just a parameter for you to play with and give yourself a little wiggle room because just because it ran in a certain Sack size this time Doesn't mean that it will always run in that stack size So the goal is not to get stack size to the absolute lowest it will ever run in But just to get it small enough so that You're confident that your program will always run But you can still save some memory because 1024 is probably Bigger than you need Awesome. Thank you okay, so I Really, I'm supposed to be talking about containers and that was all old school just giving you How to what some parameters are that you can play with in the JVM to Make things smaller, but you're really running in a container and you can still run with those parameters This is just showing you how to run Java inside of a container I have a program that I'll talk about in a little while called test random and you can just add the class file to your container and then you call it via a command and You can play with all those parameters into a container and see how they affect your program now my experience has been that Things don't always behave in a container the way that I expect them to so it to Play around with this and look at PS and see what your container is doing. I Have set up a Java Running Java with like a two gig deep size and when I look at the process the The RSS is not that big So the kernel is smart enough to say well as long as you haven't looked at this area or touched this area I'm not going to give you that memory yet So it's it's it's an interesting dynamic and something fun to play with So we recommend that you use podman here And so you can if you take the contents in the first part of the slide and put them in a docker file You can build that and then you can one The thing that you so I added this thing here called X show system settings so setting system That gives you some interesting information Especially if you're running with C groups V2 so you can see what? There's of the container are and this is interesting from a Java perspective and I'll get into that a little later But you can set these things so you can say that you only get sort of half of a CPU or That even though you're running on an eight processor system. You only get one processor So my test random program. This is my favorite program I just and when I need a Java program to play with it's what I do it It creates an array with the number of bins set to the first parameter and then it generates as many random numbers As a second parameter and it puts them in bins So it's just a tester for a random number generator. You would expect things to be approximately equal at the end of it, so What can we do with our containers to make them better? The first thing is checkpoint restore now I've been working on checkpoint restore from inside of Java for a long time but that is That doesn't work inside of containers right now right now. You have to checkpoint the whole container So it's pretty straightforward It copies the entire container to a disk In the middle so you're running your Java program you're halfway through you can copy it all out to disk and Then later on you can bring it back in so this can be really really efficient If what your container is doing is what if what your Java application does is it does a lot of class initialization It does a lot of warming up of the jet. Yeah, it You know has a lot of GC garbage sitting around If you can catch it at the right point You can checkpoint it out to disk and then later on when somebody when a request comes in and you need it All you need to do is restore that image and it's very fast. It's within the order of milliseconds So if all of your concern is in startup time Checkpoint restore gives you an answer for that So we have our podman container. We're running it. It's really easy to do Checkpointing and restoring it's a simple command at the podman level So I don't think there's any way to make that even simpler All right, the one thing that I do have to let people know I've tried to make all of my examples runnable and if you're running Fedora 31 it's using C groups V2 and Checkpointing isn't working. So I had to run some Oops Christine seems seems that we lost your audio I'm sorry It seems that we lost your audio for a bit. Can you hear me now and now We can't hear you but we lost the screen share Okay Let me see what I can do. No worries. That's what happens when we're trying to do some things live And if you're watching this, I hope you're seeing a lot of useful information and yes I'm getting a lot of new information and actually Christine. I can see your screen now and I have a question for you Okay, you you've shown this podman commands for a checkpoint and restore just in case somebody isn't Using Fedora Linux or Linux at all. Can we do a checkpoint and restore using docker for example? Oh Okay The checkpoint restore folks have done the work for podman and They've done the work for Linux So if you're running Windows, there's no hope If you are running If you are running docker, I know that I have seen it work and I have played with it I Am not certain that it is currently maintained. I Would have to look into that and if if you want me to look into that send me an email I'm chf at red hat and I will go and double check Okay, thank you All right, so another way to reduce the size of your image or to to make faster startup and Lower memory is to use a native image So I've created I have two ways of using substrate VM now. I'm gonna be honest with you guys. We're all nerds here Up until a week ago. I never used substrate VM, but I couldn't give a talk like this without talking about so if I Believe that we call it mandrel instead of red hat the Sunlabs project was called substrate VM Basically what it does is it gives you the ability to turn your Java program Into a simple Linux executable So if you look at what I've got set up here I'm bringing in an image. So I'm bringing in the the corkis mandrel image and I can use the native image command to generate an executable which is test random that out and then I can Run that image Which will run that command now? This is not the most efficient way to do it my buddies that are on the corkis team are saying that it's Inefficient to generate the executable and run it in the same Docker Container that you probably want to generate it in one and then you can run it over and over again in a smaller one I'm just showing you what worked for me. If you're an old Java hacker and you want to play with it This is a pretty simple way To get it to run You could also rather than Doing all of the Docker file stuff to get it to run You can run it on the command line and I have put a command line here that shows you How to do that I'm not going to dive into this because it's horribly complicated but and I think the Docker file simpler, but if you have questions I can help but it it it's worth trying it because It's pretty cool to be able to run your your Java program without all of the JVM Now I do want to say is that If you are Doing anything like dynamic class loading, so if you don't have a closed world when you start Then you can't use substrate VM The only limitation is that you have to know all the classes that You might use When you build the executable We have a question from Alexander What's the difference between substrate VM and grow VM? same thing Same thing I should have called it grow VM. Actually, I'm just an old It was substrate VM and then it became grow VM so My bad. I Still think of it as substrate VM Anyway, where was I? When it was a labs project at Sun Labs, it was substrate VM and now that it's out in the world for running grow They've renamed it as grow VM Okay, so the second command down here will run Will generate test random three dot out and then you can just execute that as a command line or you can put it in a container and execute it there so Just for grins. This is a little bit controversial and I will admit that but just for grins. I ran a native image which is using grow VM and I ran the same program using open JDK and For this particular application, which is just allocating an array generating a ton or however many of this big number is over here of Random numbers and putting them in array. The performance is roughly comparable The elapsed time is slightly smaller for Open JDK, but the user perceived time is smaller for the native image and the space is It's really on it's in the noise with the differences But I want to I want to say from the beginning when I set up to do this talk I was playing with my favorite program right I am Virtually certain that if you ran a program that did a lot of class loading or a lot of Manipulation of data that was bigger than ants that this difference would be striking in the native image would be smaller and faster I want to bring the podman ps command to your attention When your containers are running you can look and see what they're doing here. You can see that I'm running, you know one of the tests random dot out which is the the native image and one of them is running the JVM the Java command and So it's a it's one way that you can look at What's going on inside? Okay, so there are some questions that I didn't that I promised I was going to address in this talk and I didn't So I want to just talk a little about about what that means And the reason I want to do that is because I don't know what this means and so I want to start people thinking about it What happens if you try to run open JDK on a partial share of a processor, right? When we started out we talked about how Java crew to be able to take over all To be able to run a parallel garbage collection that used all the processors to have separate threads for compilers and Clean up and whatever so if you go when you run your JVM on a partial share of a processor I don't know that all those things are going to work. Well, and I I had hoped to be able to do some runs and to show you But now I'm just going to ask you to think about it What does it mean that you have half of one of the processors on your machine and you're running this program? That was meant to have all these threads if you just run the default on the machine. It's it runs a parallel garbage collector that Doesn't make any sense if you've only got a single processor or a partial share of a processor What so I want people to start thinking or to start talking about what we can do for open JDK? If that's what we want if we want to keep pushing open JDK to work well in containers Make it run well on a single processor And I gave a quick talk. I always give a quick talk, but if you have any questions Feel free to email me. I love to interact with customers. I want to make sure that the things That I'm working on other things Right now I'm focused on doing checkpoint restore from inside of Java so all out and do it But if there's things in the container world that you want to see happen Let me know and I'll see what I can do Are there any questions? Okay, it sounds like there aren't any questions. So I guess we're gonna end a little bit early I want to thank you for your time And I appreciate your attention. Oh, sorry. Sorry. Sorry Christine My bad. I was a mute and thank you Marcus Was now it was a human technical issue or not that technical it seems I Have a question from Lucas like if we're working with a thread pool Is there a way for me to limit the execution of the thread pool like a total execution time? Yes Let me see. I had that side. I meant to put that slide up and maybe it didn't make it in Okay, I didn't make it if you go and look at the pod man Run command. There are ways to say I want to only use This CPU set these three processors or I only want to use This amount of the thread quota I had very much wanted to give Some examples of that in this talk and I just ran out of time. I apologize If you guys want me to give another talk where I go into the details of what does it mean to set the you know Somewhat ludicrous bounds on the container that your Java program is running in and what happens I'm happy to do that Awesome. Let me see Frank Kelly has a very interesting question here Christine would you say it's a bad idea to run Java in production Allocating only one bCPU to the container. I think that um Let me take a minute. I don't think that's a problem. I think that Java can run well in one CPU. I think you need to take a little bit of time to play with some options um, who run well and Open JDK can run. Well, the growl VM can certainly run well in one processor It's not what Java Evolved into in the early 2000s, but it can still handle it. Well um, I Have not done a whole lot of experiments and I'd like to with figuring out how to make Java run Well with limited resources Like you said one one processor or a share of a processor Okay, and another question from guy doom. Dumas. Dumas People still configure memory size with power of two units instead of like round numbers It made sense with bare metal processes, but inside the JVM is there again I personally configure round figures like a thousand instead of a thousand twenty four Is there any difference? Oh Okay, so inside the JVM if you're talking about your heap size It really doesn't matter if you've got a region-based collector like G1 or Shenandoah or ZGC It's gonna break the heap up into a Particular chunk size and whether you have you know, 16 chunks or 18 chunks. It really doesn't matter or Maybe more more likely, you know 2,000 chunks or 2048 chunks. It doesn't matter No, there's no reason There's something tidy about powers of two But I don't think there's a performance difference. I have not seen a performance difference. I never worry about it No, good to know. I didn't know about it either And I think the last question here on the chat is from Lucas again Like I work with Docker and I have many containers running in my cluster in production probably How do you deal with the competition between those containers? Is there anything we can do or each container is independent for each other? Okay, so here we're going off into my ivory tower there has been a paper and If you send me an email, I will find it for you there There has been work done on getting JVM's to cooperate so for example when I talked about how you can have a Parallel garbage collector so you can have a Java program that's single threaded and then every time it goes to garbage collection It uses eight processors, right? And so if you want all of your JVM's to sort of coordinate and agree and say, okay Well, we're only going to garbage collect You know only one of us can garbage collect at a time for example There was a paper by Jeremy Singer on How to get JVM's to do that and to coordinate those things But it's all academic at this point. There is nothing that I know of in production in the real world to tell JVM's to coordinate when they do I I Highly multi-threaded work awesome Well, we already on the top of the hour now and if you have any further questions I would ask you while Christine requests as you send in her an email at CHF at redhead.com Yes, and Yes, so please so Yes, Thomas, please send an email to Christine She'll be more than happy to to reply to you and if you're watching this Thank you very much for staying with us until now. It was a super interesting talk and Christine The DevNation team Thanks you again very much And if you ever have any other interesting subjects, we'll be more than happy to have you again Thanks, everyone Hey, bye and see you next week on the next upcoming DevNation Tech Talk