 Good morning. Good afternoon and good evening to those joining us from all parts of the globe. Thank you for joining this Java container Optimization for the cloud webinar. My name is Danny Bauer And I'm a member of the red hat partner connect team that is hosting this event before we begin Go over a few housekeeping items the webinar is being recorded So after the webinar you'll receive the recorded link that you can forward along to your co-workers If you have any questions during the presentation, please submit them in the bright talk question field Then we'll try to answer them at the end or during the presentation as they come available as well There will be three separate poll questions displayed during the webinar for you to submit your responses They will be displayed one at a time and you can find the presentation slides and other resources available For downloading see the attachments tab underneath the window So now I'd like to introduce introduce you to today's speaker Severin a gay wolf Severin wolf is going to be taking over now. He's a red hat principal software engineer. I'll leave it with you Thanks so much Severin Thank you, Danny Hi, and welcome to this talk on Java container optimization for the cloud I am sibling give all the principal software engineer at red hat I work on red hats JDK team working on upstream open JDK and mandrel a Gralium distribution specifically for carcass use So Deploying to containers in the cloud. Is it actually different in our experience It is why when you're deploying to containers in the cloud the application runs as pit one and You usually have one application per container That usually means that you have smaller JVMs overall by small we mean to less than two cores and less than two gigabytes of memory and It the environment is also more dynamic. You are deploying to the cloud you're deploying with a resource limit in place and your resource limits might change during the deployment lifecycle and The nodes underneath that actually run your containers might be different throughout the deployment lifecycle of your containers as well So overall a more dynamic environment than the usual on-premise deployment scenario then when it comes to GC tuning you might want to optimize for a different metric for example in the public cloud you have to pay attention to the cost associated with it when you're you're paying for memory use and CPU use and that means you probably want to tune your Garbage collector in a way that saves as much memory as you want as he as he yes He can then you have to think about the base image which you are operating system. Are you running the container? Includes part of the operating system. How do you get updates for those? Dependencies part in part in the container. What JDK am I running? Does the JDK that I'm I'm running get security updates? to some extent you also need to think about your host or nodes in your Cloud cluster that you deploy to for example the underlying Linux kernel might make a difference as we'll we'll see later the Linux pseudo file system for example might be different on the on different nodes the scheduler Besides based on the resources that you give your containers on which nodes to schedule your containers on So overall we think it's different and the goal of this talk today Should be for you to be aware of the common gotchas when deploying to the cloud And to containers in the cloud Java containers in the cloud and make informed decisions So how am I going to structure this talk first? We will talk about red heads open JDK images and Then we'll move on to open JDK container awareness. What it means why it's useful then we'll conclude with deploying to runtime images and Compiling your cork is based applications to native and what benefits that has So red hats open JDK containers You don't use the default open JDK image on Docker hub to you If you are you might be running an unsupported Upwarding system and unsupported JDK build. It's a different build per tag So you might get a different vendor depending on the tag overall that image is deprecated Please be mindful of that several Communities upstream have run into that problem. It's a deprecated image the image is only Available still because it facilitates testing of early access builds of the latest JDK version development version Currently to JDK 21 The full details are linked in this blog post by Tim Ellison if we want to have a look So this brings us to our to the red hat open JDK containers They're not yet on Docker hub, but they're available on the on the red hat wretches 3 available and the the UBI Eula and that's you can freely read or distribute those and If you want to have the full support for them you can so you can use them for your upstream communities if you want and There those containers are built With the open shift cloud deployment in mind Those are builder images where you which you can use to deploy your source Your application source code to the cloud using that image without even touching a Docker file right now UBI 8 and UBI 9 are supported the version listed and And those images also include certain Optimizations already done that help you prevent certain gotchas that we will show in in the later demos So how do you tune our containers? There are very many Environment variables that you can use to that you can set in For those containers which affect build time or runtime for example, we will show the Java opts append environment variable in a demo later on which is useful to Set additional property something something else you need to set for your application and if you want to switch your garbage collector then to see container options is a useful environment variable to use for for our containers and the full documentation is linked here and You can the those link to JDK 11 To the to the JDK 11 documentation, but the JDK 17 documentation is forthcoming and very similar that brings us to the next section of This talk which is open JDK container awareness What does it mean? By open JDK container awareness We mean that the upstream open JDK has code in it that when deployed to a container with resource limits it detects it and then runs as if It was deployed on a physical system with those constraints So in in a way the code upstream is modeled around that concept It got modeled around the Kubernetes use case where you have a container Image that has the JDK inside that container image gets a resource resource limit applied to By using a deployment config for example in open shift that in turn results in specific settings on the Linux C Groups pseudophiles system and Those settings will are then being used by the open JDK runtime to figure out what its limits are they're turned on by default and you can turn it off using the presented switch using xx minus use container support and When you're deploying to open shift for example as mentioned before the the Open JDK reads the Linux pseudophiles system and that file system tells it what the quota is and What's what's ahead is it first figures out what Linux C Group version is in use actually So for rel 8 the default C Groups version is version 1 for rel 9 the default version is C Groups version 2 and the open JDK code handles that and figures it out and looks at the files and then that results in Open JDK resizing its internal structures as if it has the physical memory as specified by the zero quota So how would you be able to observe what open JDK thinks? That the its limits are as we'll see in a in an upcoming demo There is a VM info J command That's very handy to to use and inspect on your live deployments And it shows the C Groups in the init C Groups section what the container limits are And it's basically the same information as presented in its Aero log when you get crashes and on the JDK side There is a launch switch called X show setting system which shows the container limits as well And when you're deploying in a container the operating system MX beam is has been made container aware And will present the container limits as well instead of the physical host it runs on So what do you do when container detection goes wrong? For example your your deployment config is has a bug and the the The resources get not do not populate to open JDK You can turn on tracing for debugging your deployments And there are version specific switches that you can use to see a variable both output which files it looks at what uh underlying C Groups version it detects and um and so on those switches are unlock diagnostic VM options print container info And for JDK 11 It's xlog os plus container Equals trace those are very handy Should you really run into trouble on on your container deployments using open JDK? so we'll also see It's useful for your containers to turn on basic very unintrusive tracing for the garbage collector our algorithm in your In your deployment logs. It's useful to know what GC algorithm open JDK actually selects Why we will see that later in a in a demo as well because it's not given that that The the default collector gets selected That brings us to the first demo Where we look Build everything together. It's using the open JDK Builder image based on ubi 9. It deploys the sample application that's hosted on github Deploys it to open shift without touching a docker file. And then uh, we'll see a common gotcha for When when uh deploying it Throughout our demo, we will be showing the same corpus based application deployed to open shift It's using the ubi 9 based open JDK 17 image The main piece of source code of the sample application is shown here It's using rest and points and the hello greeting and point Serializes the response as json Taking a single path parameter When rendered in a browser the application looks like this When given an input it'll render the response from the api endpoint that got taken from the json response Moving on to the first demo will show a memory config which isn't lending itself nicely to change container resource limits The application got deployed as source build from the memory config bad branch of the git repository Where our application sources are hosted the configuration uses The sy environment capability which sets environment variables on the deployed pod after the build This configuration sets minimum heap size and equal to maximum heap size and uses some other flags pasted application at startup via java opts append Looking at the deployed pod on open shift We see the same java opts append environment variable showing up as we'd expect Now let's look at the configured container resource limits We see cpu requests limits being set as well as memory requests limits Open JDK container awareness code takes cpu limits and memory limits into account When inspecting the running jvm with vm info j command We see in the cgroup section of the output its version of what it thinks the container limits are As expected cpu quota and period recognized as specified Similarly the hard memory limit recognized as one gigabyte Let's look at the memory footprint of this deployed pod It shows current rss size as well as currently configured memory requests orange dotted line and currently configured hard limit blue dotted line We see the pod uses just above 560 megabytes of memory Now what will happen if we decreased the hard memory limit to 512 megabytes of memory Once the pod tries to get redeployed we immediately see it being killed due to an out of memory condition So how could we correct such a situation? As shown next we see the same app deployed again from the source build branch of memory config good Comparing to the previous configuration we see that nothing changed but our java opt to pen values In this case we'll opt for a more elastic memory configuration Removing hard coded maximum and minimum heap sizes values and avoiding always pre-touch Setting max RAM fraction instead knowing that the builder image sets max RAM fraction to 80 percent for us Looking at the resource limits config of this deployment We see that we start out with a one gigabyte hard memory limit With 512 megabytes requests as before Limits are again being recognized at the pod deployment level of open JDK Now if we change the memory limit of this deployment to 256 megabyte The deployment succeeds and we no longer see out of memory kills Looking at the deployed container we see the updated memory limit detected at 256 megabytes Nice we've just decreased the net RSS footprint of this app from more than 500 megabytes To less than 100 megabyte of memory by adjusting some deployment parameters To verify the application still works as expected So what are our key takeaways? For this application we've seen that it it actually matters what jvm flags you pass on for your deployments It's actually fairly common for some older deployments to set The maximum heap size equals to the minimal heap size But it might not be what you want when deploying to the cloud also be aware of the always pre-touch flag and I also recommend using elastic memory configurations for your heap size instead of hard coding it using absolute values because you can have Container awareness take care of figuring out what the total Container memory is and then base your heap size on that limit and that adjusts As you adjust your container resource limits Hey Severin, I want to jump in there's a question that came in that I think is appropriate to talk about now You talked about how to edit and configure some of the resource limits If someone had asked a question Specifically about if you have any guidance around cpu resource limits They had an example scenario where maybe there's issues with cpu detection And and being single threaded when set to under a thousand core Could you talk about that? Yeah, I think we'll we'll touch on on that on the on an upcoming demo So for example, well, I don't want to give away too much from the demo, but it depends on the resource limits that you set to what What the what open JDK like if you're going For your cpu quotas like millicores if you're going below A thousand millicores then then that's that's one core you the jvm will will use A single thread and think it can only use a single thread. There is no way around that If you if you want to turn it off you can turn it off but it would then run as if you Run on the host system and you might want to configure Active processor count, which is a jvm flag that can that you can use For explicitly setting the values that you want to so Let's let's get back to this question after after the talk and and see if there are still some some Questions with regards to the cpu limit specification if that sounds all right Sounds good. Thanks Okay, that brings us to the next section of this Talk, let's talk about uh deploying to runtime images and compiling your Quarkus based application to native So so far we have seen the builder image and working its magic to go from source code application source code to a deployed container on open shift So it clearly targets developer experience by getting Fast feedback cycle just to push to your git repository and get it deployed on open shift automatically And how would you be able to use that? Using the open jdk runtime image So the builder images can are full fat. They contain maven. They have a java compiler as part of it They contain sy build scripts to do the magic to to deploy the container without touching a docker file and The builder the runtime images only includes java and and the run javascript and the the The convention we use for the runtime images is to have the minus runtime suffix appended to the builder images and The the goal here is to have smaller or slimmer deployments Slimmer in size less dependencies installed less attack surface And that's what customers have asked for and and we'll show how to use the builder image Because it's not clear how to go from your application source code to a deployment based on the runtime image with an automatic setup If you want to recreate that setup, there is a link here and also in the attachment sections of this webinar First it also uses the builder image and the application source code and compiles that and produces an intermediate image And that includes maven and the the secondary step is a multi-stage docker build that Transplants the the binaries produced by the builder image from from the intermediate image to a Open jdk runtime based image that is used at deploy time Which we will now show in the next demo In the next demo, we show two things One deploying the application from source and then using an inline docker build to deploy to the ubi 9 runtime image two when doing so we show that Garbage collection algorithm selection is an aspect that one should take into account Looking at the open shift build configuration We see the two stage builds as well as the docker file of the inline docker build Since in this scenario we deploy using the runtime image We don't expect maven to be present on the deploying container Which was in fact available when the application was built Looking at the resource config of this deployment We have arranged for two full cores and two gigabytes of memory Being available to the deploying container Looking at the locks of the deploying pod. We see Via xlack gc config that g1 gc is being used This deployment meets open jdk's internal server class requirement Now what happens if we decreased our resource allocations for deploying containers again In this example, we decrease the memory allocation from two gigabytes to one gigabyte the pod deploy successfully Though interestingly Serial gc gets selected This is due to heuristics and open jdk which uses g1 the default for jdk 11 plus deployments Only for server class machines This precondition no longer holds with the updated deployment Finally, if we change the deployment config to no longer override the dev opts environment variable Which we've artificially used for this demo to show gc algorithm behavior We get the default specified in the inline docker file of our run runtime image deployment That inline docker file configuration specifies parallel gc and thus we see it being used At redhat we've had good experience for small jdms with the provided config in this inline docker file It gives good memory footprint behavior by not having too adverse effects on cpu consumption Okay, so i'm not sure Going back to the previous question We've seen that open jdk internally has this concept of a server class machine And depending how you you if you're running a plain open jdk image or our image or Whatever you're specifying it might depend on the resource limits what what garbage collector it selects by default the rule of thumb is is if if greater than Two cpu cores and greater than two gigabytes of memory then the default garbage collector gets selected That's g1 for jdk 11 plus and parallel for jdk eight otherwise you get zero collector and The way to get around this is to explicitly specify the garbage collector that you want and We've seen as part of this demo that you can Deploy to the runtime image using an automatic setup Going from source code to deploying on a runtime image-based Deployment a small caveat might be that you depending on your inline docker file that you might have to do some some Manual juggling with the cl cli options of of your java command depending on if you're using the run script or not Okay, that brings us to the final section, which is quarkus quarkus is a quarkus's mission. It's a java framework right with developer joy in mind and quarkus has has the mission of Making java again the preferred framework for kubernetes native Deployments and it brings a lot of extensions with it if you want to talk to a database or serialize json or Do rest endpoints all that good stuff is all included with parkers But the but the great thing about it is that you can run it in jvm mode and also aot compile your java quarkus based java application to native mode And why would you want that because it gives superior startup time as we'll we'll see in the this final demo The final demo demonstrates the difference in application startup time between jvm mode versus aot compile to native mode Looking at the pod logs for jvm mode deployment We see that the application is ready in about 1.3 seconds In jvm mode the deploying pod has java installed Comparing this to a pod that deploys the application to aot compiled native mode java is not there Only the native binary running the application is Looking at the pod logs for native mode We see that the application boots up in about three milliseconds very nice If we inspect the memory footprint of the native app We get we are still at less than a hundred megabyte memory consumption with a current limit of one gigabyte decreasing the hard memory limit to 256 megabytes Similar to our good memory config in the first demo we see the application still deploying fine The app booting up in milliseconds and the application is still working as we'd expect So this is a demo which basically shows yeah When using quarkus and java you can actually have comparable startup time again To go applications you can have very fast startup time ideal for newly developed microservices that yet you might have and Going native is included when using the quarkus framework the extensions that the quarkus comes with are tested with it and You can flip you can flip a switch and compile to native when you're ready Okay, that brings us to the end of this talk and we are we have been talking about the red hats open JDK containers and why you shouldn't be just using any random open JDK image and Consider the security update cycle for example for open JDK. There are at least four security updates a year you Might want to have a rolling update for your containers on the cloud And that's all possible using the shift and source build approach. Then we have talked about Open JDK container awareness and why it's important. It's important because you want to have Open JDK kind of take care of its limits and and operate within those limits as specified in your container deployments and don't go beyond those limits and get them out of memory killed or Starved and the final final part of this talk was going to runtime images what There are common gotchas using the garbage collector There is quarkus a great framework for newly developed apps and with superior startup time and the The key takeaways that I would like to have for you to take away is That you should think about the base image that you're using It's part basically you're shipping part of an operating system for every up you app you're shipping and don't randomly use any base image Do specify container resource limits requests and limits first it helps the scheduler for placing your containers on on a certain note and second it Basically is the the root of your good Java container-based deployment for container detection doing its its thing and figuring out what its actual physical memory should be And and sizing its internal structures accordingly Don't rely on default container resource limit configuration and open shift. There might not be any Use elastic memory configurations, which is such as max ramp percentage in your java containers for example when deploying To a container or when running open JDK on a physical system the default for open JDK is to use about 25% of the available Memory for for its own operation for its heap size So when you're when you're deploying to a container and you have only have a single application in there You probably don't want to have it only have 25 of the container limit used for Heap size you probably want to want to use it 50 percent or 80 percent and the rest for a native memory So use the max ramp percentage the the red heads open JDK images have those optimizations built in and Don't set the the heap sizes in absolute terms try to refrain from Setting maximum heap size equal minimal heap size if you're deploying to a public cloud and and actually want to spend less memory You might be surprised what's possible To run your application sufficiently using fewer bytes of memory then Don't use many applications in the same container The containers are usually optimized for a single application and otherwise they would compete with each other which with each other and To employ limit ranges on open shift projects, for example, the limit ranges is a default for uh specifying container resource limits and deploy Container resource limits. So if you specify a limit range you get Automatically a container resource limit applied if if you don't do that Then consider using quarkus for a new application. It it can have a very fast startup time if compiled to native We've we've also had Good experience with just running in jvm mode many customers use it Just for jvm mode because parkers using quarkus alone Makes their applications boot up faster Okay, that concludes this talk there's a slide with all the references if you want to have a look and this brings us to the Question and answer section of this webinar and I hope you thank you Yep Thanks so much severin. Um Well, thanks. We've got a bunch of questions that came in Go ahead. Jeff Sorry about that. My audio is a little slow One of the questions I thought was pertinent to the last topic was How does cryostat or java flight recorder work for? I think it was talking about the native images. I'm assuming Oh There is work being done by our team that actually Like there is support in in brawl vm for flight recording in events and it's it's being Expanded so expect that to for for you to work in an upcoming release of Of brawl vm and cryostat All right, another one is how often our red hat open jdk container images updated. Is there a specific update schedule? Yes, uh, basically upstream open jdk does batched security updates. There are quarterly every Every january july april july and october there is a security update And you can expect for the red hat packages or the images to get updates on at least those times of the year and You there there might be features or other updates Based on the base os So you can expect at least four updates, but they're they may be more frequent all right another question is What things can customers do to test the validity of their java workloads that are deployed in the containers? And would the migration toolkit play a part of this? That's a good question. I'm not very familiar with what you mean by validity It's uh, I might have to defer that question to the to the uh migrate mta team that's not part of our group but uh, there is there are containers for EAP and um There should be the same the same tooling available for Running your EAP workloads in the cloud and the basic the basic underneath open jdk workings are the same you need to be careful of um that the container Is running with a resource limit You might not run with with with it and it might run on a host depending on the on a big host small host um Your application behaves different so That's all I can say. I'm sorry. Not not super familiar with The EAP workloads and mta. Okay Um next one. This is interesting. How would you manage your memory limits in the case of a multi pod deployment? Uh, if you're if you're doing multi pod deployment the same the same optimizations apply so you're you're you optimize your pod Like you have multiple pods deployed in that that comprise your full application and they talk together say using open shift services and But each single container or pod should have Its resources specified as as it needs it. So You can have the you need to figure out what what your specific pod does What the what your needs are for for for each specific pod and then specify the limits for them and and together You can specify those limits as as your multi multi pod deployment Using for example a preconfigured Open shift template that that's that that boots up your whole application. You're all your pods Take a look at the demo That should work for many pods. You can scale that up as you like Um One thing is someone had asked is is there a way to recreate the demo that you did uh in open shift? Yes, there is uh, I think I have Uh, they're in the attachment sections of this webinar There are links to the to the demo sources And that those have instructions How if if you have a open shift cluster You run those commands and you should be able to recreate Those demos on your cluster and play with it All right, and last one this is might be a tough one to to finish on What advice would you give to a customer who? loves the stability that EAP and VMs has but is unsure what that would look like as you move to containerize Running containers on the AP i'm assuming that means running Containers on the AP on open shift is I'm kind of reading between the lines there Um, that's a tough one. Yeah, you got any ideas on that? Yeah, it's a good question. The the first question I would have is Why would you think that an EAP deployment on on on the cloud or in a container is not stable? So that that would be the first thing I would look into um Theoretically if you if you take your on-premise deployment and have an equivalent Uh deployment in in your cloud it should behave the same if you're if you're uh EAP pot is is configured correctly Uh, sometimes it's it's small things that make a huge difference But overall I that's where I'd start Okay, I think that's pretty much it for the questions that I can see Severin Quick comments from my end. I'm going to be closing the last poll Also appreciate if you could vote and submit your feedback for any other ideas that we can Maybe perhaps talk on for future webinars Other than that, um, I wanted to thank everybody for joining today Thank you so much, uh Severin for your presentation. I really appreciate your time And thank you to all of our partners who joined here today. We look forward to seeing you again Thank you. Bye. Bye