 All right, let's get started. Okay, so, hello everyone. Thank you for coming, our session, our KEPR update. So, I'm Chatshi Chiba from Navi Research, and today we're not gonna talk about KEPR and KEPR itself, and also there are some project updates on the deep dive of the recent topics. So, all right, so let me get first for the, briefly introduce some of its introduction and the motivation and also the what's going on and the KEPR deployment over the last couple of months or last years and what's going on next. So, all right, so before jumping to the KEPR, so, you know, all know about the sustainability is a top interest, the top concern about your companies, right, and this kind of the sustainability topics is rapidly growing and focusing on the many organizations and the, yeah, you know, the CTO, CEO, everyone want to introduce the sustainability future in their product and that's behind that, right? So, why they want to know about to focus on the sustainability is that there are many reasons, is there are many kind of the climate changes happen in the world and then so we want to reduce the global emissions and the gas, right? So, that's the requirement for the government and the companies and what they want to track and all of the energies, consumptions and carbon footprint reductions, right? So, that's behind one of the reason, right? So, but the almost data centers is consumed or many watts, right? If you can imagine the 200 and the 255, 500 kilowatt hours electricity cities is, I cannot imagine, but it takes a very huge resources in the data centers. I heard it's coming almost same of the Oswald years, the day usage, right? So, really big usage. At the same time, it's also the data centers faster data center using for these energies, right? It's time-order computer energy experiments is real. We know many GPUs running on the data centers and the many AI workloads consume data center energies especially for the GPUs, chat TPs and creating some large market models and consumes many GPUs all over days or weeks or maybe they're over the month continuously. That makes a lot of the energy consumption in the data centers and also as well as the influence workloads also consumes. So, the growing demand of the resources, computing resources and AI workloads, inference training, that makes a lot of the power pressure to the data center, right? So, to solve that, we have two options. One option is creating a new hardware to reduce the energy consumption as much as possible. That's the one key thing to solve the, to tackle other energy problem. The other thing is the software, right? The software, we can reduce or optimize our workloads and decide that the wearer is good for the sustainability energy produced to the carbon footprint. That's the reason why we want to develop the K-Pro. So, let's move on to the tower sustainability computing topics on cloud. So, I just defined four things to optimize the DRIs, the sustainability computing here. So, it's not only for the Sentinel-B topics. It's common in many workloads, performance of donations and on the cloud. So, four steps, the complications, assess, optimization and automation. Quantification, is there capturing and tracing over the matrix of the power usage or the workloads and the gathering and put together and that's the same, the quantification. That's the main part of the K-Pro, right? The correcting the power matrix and the gathering. Then assess. Assess means you're just identifying the hotspot and taking the next actions for the workloads or users. That's kind of part of the assess. And the optimization is based on the assessment result. We can try to consider where we should put all of these workloads. Maybe they're another zones or another countries based on the carbon emissions, reductions or power resource usage. Where it comes from, the power, right? And the energy sort of energies or some other power sources. That's kind of the scheduling or workload dependent topics is happened in here. And after that, we will do that in the automation. We are the cloud natives. We are in the cloud native cons, so that's the way to the how to make it happen automatically reconciling loop that I defined. We defined the four steps for the making sure that something we're computing. So to do that, we have still required several things. How to create the workloads, energy, containers, running workloads. That's the basic questions. And this kind of the stuff is always happened. But we want to correct the matrix. But we do want to put much overhead, the desk container. So we want to reduce the overhead as much as possible. That's the next questions. And if we have, yeah, it's bare metal machines. We have the power data source. We can correct the directory that the power usage is, but what if we cannot see the power source in the cloud? It is common in the cloud environment. It's running on the VM, doesn't export anything to the VM. So to how to do that, how to deal with what did that, all of that things happen. And yeah, and the four things is the question is, I need for the wearer to come from the power consumption, divide it. So you're using the 100% of the system. It's definitely is coming, you are both. But the cloud or open-source Kubernetes running various users running on the, sharing the same nodes, so right. So dividing the wider for the users or containers is also important topics for that. So KEPRA, KEPRA is, so to solve that, we are developing the KEPRA. KEPRA is now in a CNC and sandbox projects being the last June and getting a lot of interest in recent trees and the many features, right. So that's, but just said or the focus on some part of the future of the KEPRA. We are collecting metrics from process, containers, container-level metrics, C-group, constant resource usages, and also the gathering the power resource usage from GPU, CPU, DRAM, and also nodes, and also power metrics from the GPU, CPUs, and nodes, and so on. That then put together and reporting. And it's also running on Kubernetes and also the OCP, open-source as well, and the bare metal, blames, whatever, and we have the standalone mode so we can run just installing the KEPRA into the red as a standalone mode. And also that power, some part of the libraries depend on their architectures. So we have to also support X86 and Linux Z also supported. I'm still in the next features, but we are thinking about that. Okay, so to reduce the overhead, we have never empowered the EBPF features than to reduce overhead. And to seek or provide the blames power metrics, we are preparing the power models in the server to expose and train the models and based on the past histories. That's features that we are providing right now in the KEPRA. So, yeah, actually we have many road masks in the, yeah, let me briefly introduce the recent update on the histories of the project KEPRA. KEPRA itself is born in the early February or end of the year 2021. Originally Red Hat guys signed up to this project and at the same almost same time that we joined this project and to accelerate the develop on the making a lot of research work for this project. Just we just highlight some of the key features we provided at the later talk, the make sure that some training model and preparing the model servers and also how to identify the idle power, right? And what else, anything? Yeah, it's also being the CNC Thunderbox project, right? And now on the next open seat, we will release the KEPRA as a technology preview. That's the real, so yeah, that's really good. Our KEPRA project. So that is kind of the recent history, the road mark of the KEPRAs. And we also provide a lot of the ecosystem of the KEPRA rely on the KEPRA, so to again, realize this configuration that says automation. So some parts, for example, exporter model servers and the database, those are the configurations of the basic future of KEPRA, right? And also the KEPRA dashboards or a graph on dashboard we provide and the operator is automatically setting everything and the craving or peaks, that's kind of what the style is for the optimization based on the KEPRA metrics to allocate or to decide the right places for your walk-throughs in or on your current Kubernetes cluster. So that's, yeah, we are growing our ecosystem of KEPRA. So let me just jump into a lot of a few demos. First is happen inside of the open seat of the KEPRA. Might be a little bit small, right? Just, yeah, we can provide any ham or operators, so easy to set in after KEPRA into your environment. Just click and install and setting up the small KEPRA custom resources. It's a very huge option. We have many options, but we can customize first what you want, turning off or enable over there some future of the KEPRA. And after gathering some metrics or power metrics, we also have the integrated open seat dashboard to show the actual usage of the work clothes and power usage of the work clothes, right? And I can see some spike or, yeah. And also one more things, although I've been preparing some graphon dashboard to show us what's happened in your clusters. As you can see, we have there some meters or the carbon emission, it's just a demo. So we have them can change the coefficients based on your some environment or requirements, right? But we can calculate, dynamically calculate it or your carbon mission as well, and also show where it's come from. And we have some most pain or heavy usage or other GPUs or CPUs or packages or memory, some of them. All staff is we are correcting and showing the kind of the dashboard right now. All right, that's all staff. I want to talk about some stats, first phase of the KEPRA and the next. Marcel, you want to go next? Hello, I'm working, I'm Marcelo Moral from IBM Research also, and I'm going to continue the explanation here. So we have been working on KEPRA as Chiba Sun mentioned for the last almost two years already. So we have increased, implemented different features, fixed a lot of bugs, and KEPRA it's becoming stable and getting a lot of attention. And just to say here, we support basically two use case, one it's on-prem, when KEPRA is running on bare metal nodes, another one it's when it's running on top of virtual machines. The difference here are some bare metal nodes that doesn't expose real-time power consumption. So when we are in the bare metal, we have hardware sensors and we can collect the real-time power metrics from some specific resource. For example, CPU power, memory power, or GPU power or the node power. On VMs, typically it's not exposed right now on public cloud providers, maybe in the future it will be exported that so we can use. The only power metric that we can collect typically right now is the GPU because the GPU we have passed through and we fully have access to the GPU so we can access the GPU utilization. But CPU and memory it's not exposed so we need to use some power model to predict that. Which I'm going to detail a little bit but power model is created through regressions of data that it's collected from a bare metal node. Okay, so this is the KEPRA architecture just to give more information how things work. So KEPRA estimates the power consumption for the process. So we collected the resource utilization of each process. For example, the CPU utilization, the memory utilization, the GPU utilization, and we collected the power consumption of the resource, the CPU power consumption. And we use the premise that it's general we use it in all research papers, actually this premise that 10% of the CPU, if a process is using 10% of the CPU, 10% of the energy consumption of the CPU, it's associated to this process. So it's as simple as that. Of course, CPU utilization can be determined by different metrics, isn't it? So we have instruction, cache, many things but the assumption is general assumptions thinks about the process using the percentage of the resource, the power, it's the same percentage linear relationship by that with the energy consumption, the power consumption. And by doing that, so we use EBPF to collect resource utilization metrics, both CPU time and hardware counters if we are on bare metal nodes. And we also collect some, for example, page cache hits to understand some memory operations so that we can also create the power models if it's needed or just use the hardware counters when we are on-prem and associate that to the energy consumption. As we were mentioned, like multiple times, we collect power metrics from the bare metal nodes from different APIs. For example, Intel has the RAPO API exposed CPU and D-RUN power consumption. NVIDIA GPUs also export that and normally the motherboard has sensors and we can use a CPI API or IPMI also. It's also commonly used to collect the node power consumption from this motherboard sensor. If we are on virtual machine, it means we need to use a trained power model that it's generated by regression. So we run a set of workloads in a bare metal node and by collecting this data, we perform some regression algorithms to create a power model. Okay, this is just like a simple way to see how CAPRA is deployed in different use case. As I was mentioning, bare metal, it has directly access to the sensors in the hardware. On VMs, it typically only has access to the GPU power consumption. So it needs to train power models to estimate the energy consumption of other resource. Yes, sorry, click in the wrong direction. And there is a third deployment approach which is some pass-through. So for example, if you are in the private cloud, maybe in the future public clouds can also support something similar to that. But it's CAPRA can run on a bare metal, estimate the energy consumption of the VMs using the real-time power metrics from the sensors and expose that within the VM. And in this case, there are another CAPRA that it's running on top of the Kubernetes that is running on top of VMs can have access to this power metrics and then associate that to process containers and pods later one. So this is the third one is again, so it's the user has on-prem environment that it's creating virtual machines. So private cloud can have this use case and maybe in the future cloud providers can also move to this. Okay, so now it's just some updates from the project. We have updates related to the bare metal deployment that I was showing before. So during this year, we have enabled the concept of idle power which is well-accepted during the research community. Idle power is basically the power that is consumed in the node without any resource utilization. So if nothing's running there, there is still some power that is being consumed there in the machine. The dynamic power is the power that is related to the load. So it's dynamically increased according to the load. So it means we need to, and according to the GHG protocol, we need to divide the constant power consumption which means as the idle power which is constant, okay? Divide the constant idle power based on the size of the applications to be a fair distribution of this idle power. So that's why it's another motivation to separate these two different powers, so the idle and the dynamic. And not going to have too much details here, but basically if the idle power will be constant, it's very hard to see here, but considering the blue, it's the colors, right? So considering the blue parts here, so there is like a minimum power consumption that happens to the node to resource and the dynamic parts. So this minimum power consumption is the idle one that we separate. They are different to separate that. Right now we estimate the minimum power, so we watch the power consumption from a period and get the minimum power, but we are going to improve that later to also estimate the idle power using some regression analysis so that we can say, okay, for zero resource utilization, this is the power consumption of this resource. And then some, yeah, so the bare metal node also, we have the Kepler standalone support for now. It is specifically important for, as I was mentioned before, if we have this VM pass through, so we want to run Kepler standalone and collect the VM power consumption, expose that. Also, we have been discussing that through the community. There are some use case that for the edge computing with some edge device not running Kubernetes, Kubelec demon there, so it needs to have some standalone demon to expose the energy consumption on those devices. So the standalone comes from that motivation. And also, we have been developed that to collect the power consumption of process. Then we can aggregate that to containers, pod and VMs based on IDs of this resource, of this instance. Okay, so for the VM, when we are on the VM environment, the power model that I mentioned to you is trained in the power models. And to estimate the CPU power, it's more directly, so we can have the CPU utilization like CPU time and then do estimations. Memory is more complex, isn't it? So memory allocation doesn't mean memory operation. So application can allocate a few gigabytes there, but maybe it's not using read and writes on that. So it's more challenging to have a power model for memory. And memory is, has a lot of, we know that it's complex because it has virtual memory, it has cache. So memory, it's very complex to analyze. However, we can have some metrics to give some hints of memory operation, especially if the application is doing call through the operation system. For example, it's using virtual memory. So applications that are using virtual memory, we can have the page cache heat to a BPF matrix and associate this memory utilization. And this is one metric that we are enabling on Kepler. For performance optimization that we have done inside Kepler, okay? We have identified through the comments that collecting C group metrics had a lot of overhead. So a BPF metrics, a BPF is designed to run only on the kernel space, not user space, because of that, it's also very efficient, low overhead, so it's easier to collect metrics. C group open files, so it has, in the user space, so it has a lot of overhead to collect C group metrics. And in this case, there was a use case for real-time kernels, for example, that was complex to reading C group metrics because it has overhead. So we are using container D library to collect C group metrics and we compare that run C libraries for C group version two has lower overhead and then we actually are using two different libraries for different, you know, if it's C group version one, container D is better if C group version two run C is better for performance. And then Kepler is doing support in that right now. However, we modified everything as I mentioned, collecting C, even though with the optimization, collecting C group metrics is still like has overhead and we can rely on EBPF metrics. And for that, again, we support different use case, but for the main general use case, we can disable the collection of C group metrics if the system can support, you know, all the EBPF metrics collection. And in this case, we can also reduce a lot the overhead so we can see here by creating here many Kepler replicas here, the total CPU utilization it's much lower when we disable some C group metrics and which actually it's good for lower footprint for Kepler. The other thing that happens is how often Prometheus collecting metrics. So the default configuration for Prometheus is collecting metrics every 30 seconds. By design, we have implemented Kepler in the beginning for lower and latter for that Prometheus was collecting every three seconds, the metrics from Kepler. And again, so after discussing also with the Prometheus community, they were recommending to increase the scraping interval to reduce the overhead in Prometheus and overhead in Kepler because every time that Prometheus is communicating with Kepler also is generating CPU utilization, isn't it? So we also change the default 30 seconds and we have done also performance analysis to see if it's important to change that or not. So and the overhead when we increase their 30 seconds it's also much lower. For the future works, we are right now Kepler expose Prometheus metrics, but we have some work going on to expose open metrics for open telemetry, especially for edge use case that we were was describing before because edge typically it's relying more on open metrics right now. And also open metrics, it has trace information so it gives more flexibility and more information in general for monitoring things. And we are going to, so to even further reduce the Kepler overhead, so it's already low but the idea is you have as low as possible, isn't it? So there is a possibility also to introduce the idea of we are already implementing that on Kepler, it's working in progress, the EBPF is sampling rate so EBPF is collecting metrics, but we can skip some metrics, you know, it's, let's say during the context switch, it's collecting thousands of informations per second and it's keep some of the collections. It's typically fine, you know, but it's an again. So by having sampling rate, we also need to measure the impact of that in the current so it's already ongoing work inside Kepler that. The VM pass through scenario that I mentioned before and the idle power that I through regression that I also commented before. It's our next, there are much more things going on so I just listed this few, these things here that are considering important to mention. So Kepler, it's a growing community, there are many companies now contributed to Kepler open source. Those names here are the ones that create pull requests and on the open source. There are more names, no more people that contributed to design ideas to discussion in our community sections. Their names are not here, so sorry if I'm missing them, some people here and this is just to illustrate all the companies that are contributing to the Kepler code. Yeah, finished, so questions. Has there been any performance benchmarks between say running Kepler on bare metal node then tossing some VMs on there, putting Kepler on top of the VMs on the same bare metal node and then seeing sort of the margin of accuracy there? Yeah, yeah, so we have different ways to validate the occurrence of the energy estimation. So on bare metal where we can have the energy consumption of the, measure the real-time energy consumption, we can have some delta of some scenarios, what I mean is we have one scenario run one application, then another scenario run the same application twice. So the incremental power is the power of this application that it was added to the node, okay? So by having this delta power, the estimation that we are doing with Kepler should be something close to that. So this is the first way. For virtual machine, what we can do is we have the power model that we are recreated and then by having the data of this, for this application that is running on the bare metal, it has this dynamic power. When we are running the virtual machine, what's the dynamic power that is estimated there? And then we compare that. Question about, like, do you have any plans to also add this matrix that you are gathering using Kepler to inject into some sort of scheduler extenders for Kubernetes to do like power, like workload consolidation, for example? Right, so we export Kepler matrix to Prometheus and Kubernetes can get metrics from Prometheus. So there are some scheduler plug-ins that can be doing that. So the project that we mentioned, PIX, is actually doing that. So it's adding a plug-in to the Kubernetes scheduler, getting the power matrix from Kepler and doing some scheduling decision based on the power consumption. So one of the slides you're talking about, the eventual ability to run Kepler like on public clouds, right, and expose the metrics. A little skeptical of how long that would take, and I think that the trading model, I'm just curious if there's been any interest from the cloud providers to help trade those models for you. So then that way you don't need to do it on, you get kind of as close to their bare metal as possible. Yeah, that's a good point. So what I'm saying is we thought about two different scenarios. One is, as you mentioned, so it's actually the main part that we create a power model and the cloud providers can provide that or some companies can help actually Kepler. We have this concept that we ask the community to help with that. So different companies could also, they have like bare metal nodes. If they have bare metal nodes, they have something. They could train power models and make it public. All the power models that we create in Kepler, it's public. We have a GitHub repository that is power model database and it's there, so everyone can use that. So because the power model is dependent of the CPU architecture and the CPU model. Different CPU model has different power consumption. So we ask the community to help with that, yeah. Interesting, thank you for that. So on supported architectures 390 S390X, is that main for it? Yeah, exactly. It's IBM's machine, yeah. Awesome, thank you. Thank you. We still have one minute. If someone wants to comment or ask something. Okay, so in any case, I ask the community, again, Kepler is a growing community. I ask people to join our Slack channel. It's the CNCF Kepler project, the Slack channel. We also have bi-weekly meetings. Everyone that wants to comment something or if has some doubt or some interesting use case to show us, to ask us, it's very welcome to contact us through Slack or in our bi-weekly meeting. Yeah, thank you everyone.