 So it's really a pleasure to meet everyone here and we have something exciting to share with you So first of all we want to share with you what we are what's the idea of these projects is about and it's not really a something That's a you know sat in stone. It's really a methodology and In some is hopefully some inspiration for your projects for your methodology to develop So we really want to have accountability in In a software sequel system, that's what how people how much energy you use in your software software and hardware information system and how can you make a Adjustments to your workloads to your infrastructure and to your administration We present the problems and presents also Give you some solutions. That's what we hope can give you ideas for your future Experimentation the architecture is really based on the sense of ecosystem Although these are capra is not in CSF as we hope this is we'll get you to the way very soon Are you using a number of our technologies look as the base of this fund? I just a solution with some other type of technologies And we will briefly talk about capital capital is the project we gave you as the key knows It's meant to be help you to manage the software workload Level energy consumption We present to you the analytics that we are correlates with the workloads facing and energy reporting from capra to figure out How to identify your API level energy consumption? You will we'll give you a demo and we will give you some of the ideas what we plan to do in the future So if you are looking at the big picture on the energy consumption as in general has been Tremendous the number itself may not make a lot of sense if you're just reading them right now But you can see that's ICT in general using anywhere between 1% to 10% data center and ICT and we are between 1% to 10% of the total Electricity consumed around the globe and if you are looking at the recent strength That's picture could be even bigger in the future as well In my opinion, that's going to be two things. That's making the growth higher Why is that's the wildest network the 5G? You know, even though 5G has been rolling over the years, but if you're looking at the Over the last generation 5G over 4G the 5G is three times denser and three times more power Expensive than 4G each of the device. So in total, that's a 10 times more energy used in 5G than 4G The other one is the AI and the base coins and crypto cryptocurrency AI has been known for a long time But most recently if you are looking at how just GPT is changing the dynamics even search engines like in Google and Microsoft It's going to integrate a trust GPT like work or workloads So according to a certain benchmark if you are integrates AI into the search engine The workloads we are using five times as much computers as single search So that's what means lots of explosion in energy consumption in the near enough long-term future So if you are not doing anything right now Then that's going to be a catastrophic future. That's we're going to face And if you are putting in the numbers in perspective our industry In just in data center Industry is comparable the carbon footprint is comparable Then our line in general and I imagine that's is how big they are the flies have to go around the world And how frequency you have to use it But if you are posting the carbon footprint together, they're comparable and our industry is actually growing much faster So that's really gives us a huge urgency to take actions right now And how we're going to do that our beliefs as accountability is going to drive in force To make the energy efficiency and in the future if we are not taking into accountability If you do not know the metrics if you do not based on these metrics take actions Then nothing will happen in the first place If you just say I'm a green data center by what's metrics do you mean is green if you say I'm a green software Infrastructure by what means and how do you mean to be green in practice? so we want to bring some of the metrics and some of the measurements into place and I we want to bring break down the workloads as different levels such that When you see your workloads when you see the API you consume you will have some ideas How much energy how much carbon the API and the software is associated with so that is something we hopefully can provide you Some of inspirations from this presentation So the problems we identified consist of the three of them one of us what base level How do you identify the workload energy consumption and how do you correlate the workload energy consumption with carbon? So that's sort of fundamental questions. We want to ask in the project capital So as we are studying the keynote's capital is a software Power meter is using a number of technologies including machine learning and ebpf to help you and the end user as well as the software Developers to guess idea of how much energy the software the workloads uses on your runtime system Is that so by certain open public methodology that has been used in the research community for many years the second one is For your operation, how can you trace end to end? We use an S3 in this example This is very popular API and everybody uses so by using this API We hopefully can give you some ideas if you are using your own API. How do you do those of tracing? So as this API does consist of lots of entities So we using Rook in this solution to help you to switch from and to form the ingestion of the API to the processing and to the Final destination how well the API will traverse and we think that's what they are tracing We are correlates. That's a log in message with capitalist or power message because ideas of API level how much energy used by the service Thank you. Okay. So now we know what we want to achieve You know what our goals are and we'll have to take a little bit a small look at how we achieve that will come to technologies We're having our architecture. So as humans said our example is with Cloud storage. So we're using Rook to set up the the self cluster And the other technologies we use is Kepler as women mentioned This is what measures the energy consumption of the different pods in our system and open hermitry in Yeager This these are the technology that we're using for the end-to-end tracing Eventually everything finals up to Prometheus. This is where we do the calculation and we show the outcomes now We're gonna drill down a little bit into the different architectures So Rook and Seth as I said Rook is the Kubernetes operator to deploy Seth. Seth is an on-prem Storage defined software defined storage solution distributed storage solution and we're gonna focus on it has different APIs Block and and file we're gonna focus on the object API, which is S3 compatible So if you look at what's going on there, there is a Rook operator It deploys the system or it deploys the system. It deployed the front-end, which is the S3 interface Those are called the Redis gateways or the RGWs and it deploys also the back-ends Those are actually the drivers that take take the data and writes them into disk. Those are called the OSDs So in such a system, you would have multiple of the front-ends and multiple of the back-ends and In front of the front-end usually is to have some kind of a load balancer. So when a user uploads their data or downloads the data from the From the storage system, they can pick any of the entry points any of the RGWs The same user can use there's no affinity can use whatever RGW that they want and the user writes the data to their buckets This also is most often sharded and written to multiple OSDs or to multiple disk for of course for Parallelism and performance. So if we look about the if we look at the logical Entity, which is the user then it is pretty much all over the place So the the actual processes that takes takes care of the user are many processes Of many types both the front-end RGWs and the back-end OSDs The other system that we're using also a CNCF project is open telemetry in Jaeger so open telemetry is a framework for for tracing and In many cases it does instrumentation for the technologies our technology is done in the SEF technology both the RGW and OSD are written in C++ So we had to do the instrumentation ourselves not only that we also did the serialization and the serializations of the traces over our Rados interface that this is what sends from the RGW to the OSD and eventually to the disk. So we'll get the end-to-end Trace because we serialize the trace into our own messaging so the Jaeger operator Injects the Jaeger agents into the pods as the sidecars and The open telemetry SDKs speaks to them And they're sending all the information the Jaeger operator also creates the all-in-one Pod that we used in the demo, which is a Jaeger pod and the agents Send the information into the this the collector in the pod. There's also a query interface in the pod They're also UI We're gonna use the query interface in the pod to fetch the information later on and do the processing That would give us the results that we're looking for Now for the next technology, which is Kappler I'm gonna return that back to you Thank you of all So Kappler is really a based on a number of scientific research We mix the technology the research of affordable in software technologies so people can consume is very easily So Kappler as I if you are looking at the layered model, we collect the information using eBPF as a very bottom So the information will consist of the software as to the hardware level So the hardware we're using the hardware counters are including the CPU instruction the CPU cycles the cache misses And potentially we're going to add a more So these are the information that's in the scientific research as proofs to have strong correlation with the Software activities that has been running in the system with this information. We also collect the software information Including the software as an operating system level the CPU run time the CPU run time These memory usage and things like that. Why do we need to have both hardware and software information? It's because we want to have our capital to be able to tell the energy use both by Biomedical environments as well as the virtual machine environments when you are running on biometals We use in the hardware counters to tell you how much energy used by the workloads When you are running on virtual machines as in most of the public clouds or private clouds You do not have access to the hardware counters that we usually use in the software counters as And different models to tell you how much energy used in your own environments The nice thing is that is regardless where your workloads runs You always have a model to match and that's model will just help you to identify the energy used by the workloads without a lot of No change of configurations or just change of the settings in your workloads Once we get this information the next thing we do is to try to correlate How much energy this is the you know starts the metrics from the process of C group and we find out that the identity of the C groups in the user place our user space representation in a user space We do not the user space use the the share instead of the in 64 as in the kernel So find out that's sure and they are using that shard in Quarry the accumulates API so we can get to the part name a container name and the container name space From this level we can report everything that's has has happened to this container to this part and then we create the Prometheus metrics to exports the energy information associated with that process or containers or part So that is how things works in this way. What's is missing here? It does the model how the model is created. So We create a model. This is a little bit chicken chicken egg We need the model to tell the energy consumption by processes or by containers But so we create the model using the same metrics as we collect as the process level But we aggregate everything as the server level as the node level and then from that's no-level information We associate space the node level energy. That's we read from the CPU So for x86, you have rebel the runtime power level or on certain army Implementation you have hardware sensors to help you to get the energy information from the CPU package And then we build out these models using linear regression for simplicity because the linear regression is not hard to swing and very Efficient to calculate. We also support nonlinear models For example, new neural network as well as the machine learning model such as sg boosts, which we are given better accuracy But that's also come as an expense as the difference time because it also takes a lot of CPU cycles to calculate But we give these options anyway. So if you want to have a simple model Low overhead using linear if you have want to have a more accurate model But you can afford more CPU cycles using a nonlinear model. So that's opportunity. That's a choice is all totally up to you So once this information gets, you know feeding to the machine learning models trainer We call it the capital model trainer We can create a model and that's model is uploaded to our github and we download from github and packages in the capital container And we use that model Just to show you what we can capital can do this is as already as the promissive API The This side of the This side of the curry is as a container level as the name space level So we aggregate all the paths in that container and show under due the rates because capital Only collects the aggregates Energy are in joules. So if you divide the time by time step, that's the power in watts So you see over time the watts as difference name space Is all more or less constant because we do not run a lot of workload at the moment The highest one The purple one. I think that's is a rook And then that's it's about 80 watts. And by the way, this is running on mini cube on virtual machine So that's just to show that we can run both in virtual machines as well as our models And you can easily replicate these environments in your home environments On this side of the The diagram this is in the pot. It's in the pot. So if you look at the query on the top That's what we are just look at the container name space looks self So in this name space, we can break everything down as a part level So you can see which which part consumes more energy than the other. Um, I think it has a I'm kind of Kind of blind at the moment, but I think the highest Energy consuming part is 16 watts at the moment And for that I will show the best thing from you all Thank you. I mean Okay, so now that we have all the mechanics in place. I'm going to show how we actually do the calculations So on on on one side, you see the tracers. So tracers are consisted from multiple spans Spans are like units of work inside the trace the end-to-end trace Um, and they have different attributes So a span will have the pod that the unit of work was executed executed on It will have the duration and it can have tags And that's really where the kind of special thing is happening because Um, for example, the entry point the reddest gateway. No knows the user the osd's whatever writes to the disc Doesn't have any idea of what the user or bucket is But we're going to stamp the the span that knows the user with a tag that tells the user name and As the as the trace traverses the different processes and the different steps Then it will going to accumulate more spans more information about the duration of the operation each and every pod And one of them or some of them will also have these extra tags that would allow us to later on correlate the whole thing This is really a simplified Vision of what a span is because span could have hierarchies You can have a sub span and then when you calculate the duration You have to make sure at which level you calculate the duration because you don't want to count twice But for just for simplicity in those cases like the spans come just one after the other So we're fratching all the spans from the from the Yeager query interface and then We can create we will group them by first by pod and user Which means that for each user on each pod we can account the duration of all the spans Of this user Even span that doesn't know what the user is because the trace does know what the user is So this is this is how it works So we do this initial group by as I said of pod and user Then we also do another group by only of pod And so you would know how much overall duration of all the traces of all the users of this pod Took and from those two Tables you can calculate the percentage Of a user in a pod during of course the lifetime of the tests that we're running So in these cases we just summarized that everything per pod And just did the simple division to get the percentage of each user From the pod during the lifetime This is the big table on the left now from Kepler We're getting a simpler table just each pod just like the previous slide from huamin Each pod has its energy consumption during the period of the test And we see that that table on the upper right hand side and from these two tables We can combine the information together now. How do we do that? So for each of the percentages that we calculated for the The user in a pod we would go to the matching pod and took the energy And we'd assume that the proportion of the duration that of the traces and spans of the user in this pod Matches more or less the proportion of energy that the user consumed So we know the energy per pod. So now we can know the energy per user per pod And if we want to know the overall energy of the user we'll just summarize all the energy Of this user across all the pods. So at the bottom right hand You would see a table that gives you a user with their summaries And the overall energy that the user used So that's how the math work and this is why we need the end-to-end tracing because without end-to-end tracing We cannot do the group by by user because we don't have this information available in all of the processes This might be a little little difficult to ingest in in a very short Amount of time You're welcome to ask me question later on or I would recommend Just look at the python code that does that That would make that much simpler. I think And now we're going to try a live demo Thank you. Okay These are working Is it working? Okay, okay. So first, let's see what's going on on the system that we're running So Those are the pods running there. You can see that the Kepler part that does the The calculation of the of the energy the whole bunch of prometheus Pods they're doing the permissive stuff In the in the monitoring namespace you would see in the ability namespace There's a jager operator and that what spins up the all-in-one part of jager, which is the second one You'd also see the whole bunch of of rook pods So the important ones for us is that the bottom one is the self rgw This is the entry point the s3 Interface of rook and we have two osd's osd1 and osd0 that match the two disks that I have there on the system So these are the pods that we're working on the rest of the the rest of the pods is kind of overhead or idle time and it was a great talk yesterday about Doing Optimization of energy on bare metal and one of the interesting thing that they said there is that the the idle time overhead is big And we would see that here as well Okay, so and now we're going to run the demo Okay, so the first thing i'm going to create three buckets I was talking about users but bucket and users in this case It's pretty much the same each user has one bucket just for my simplicity I'm going to create all the files i'm going to upload ahead of time The reason i'm doing that is to first of all I don't want to measure the energy consumption of the creation of the file because this is like outside of the system The second is i want to upload everything in parallel so i can load the system as much as i can Because if the system is low not loaded and it's all idle overhead The first thing i do i fetch from primitius the Kepler counts The reason i do that Is because those are running counters, so i have to fetch them before the test Now i'm actually doing the test i'm uploading the three The three buckets i have a light bucket a medium bucket and a heavy bucket and The object sizes are kind of random and 10 goes to the light bucket 30 go to the medium bucket And the remaining 60 goes to the heavy bucket So i took a snapshot of the Kepler stats from primitius now i'm running the test It's running And once it's done i'll have to take another snapshot from the From from primitius for the Kepler stats, so i'll do the diff So i'll have the energy consumption during the test and then i'll take that Take the traces kind of calculate everything that i showed in the previous in a previous slide in in my python code and See how much energy everything anything took Okay, let's give it a couple more seconds Okay, so i fetched the The Kepler stats from primitius Okay, i'm fetching the traces that might take a little longer. There are quite a bit of traces here And women will talk about it later because there's also the question of How much energy the tracing and all that stuff is taking and let's see what pans out Okay, so whatever it says internal. This is like overhead You would see that the heavy bucket took about 300 joule The bucket checker is again kind of overhead. This is something internal to rook That kind of checks the health of the system. By the way, i'm running with rook 110 in rook 111 they removed that so Saved on energy you can take off 130 joule from from there the light bucket Take uh, well the smallest amount at the minimum bucket takes something in the middle You would see here that the bucket checker which is just overhead takes the same amount as the medium one So it's good that they removed that And this is pretty much the the output of the Of our system and end of the demo Thank you. Well, so that's very uh Straight forward way to visualize what is going on and this is a really good plc So from there, uh, we can discuss a lot of potentials. First thing is um, you don't want to turn on tracing your production, right? So how and also tracing has a lot of overhead So in practice, I think it's a reasonable to think that's um, You can taste hung on some sampling from time to time just give you some sense what is going on So what we are going to next is to use the sampling to build up the model that's well similar that's well Guess the so we will build out two models one from the space in the other one is from Sampling think about non-niche distillation So sampling will often learn from the spacing the actual energy consumption But with Much much fewer data points. So that much much less overhead So in that way we hope we can give you still give you accurate Estimates without incurring a lot of overhead. So this is going to be an experiment to the next The second is um, as you see here is that that's also of a automation A lot of a manual process. We grab something from Kepler. We calculate something Inside of those demo scripts Hopefully in the next demo in a future work, we can automate the whole thing So when the system is running, we automatically get the couple of metrics and that's where matching up the Bucking's activities. Hopefully that's we are giving the overall Visualization as the end. We don't have to waste you know from in a batch manner That's lots of things can be optimized so you see It's definitely something we can reduce And then we can also visualize the how much energy used by overhead in the system For example, the the self-checker if you can use it in the same method to identify some other things in your software stack That's going to be great And eventually I think more or less in the future. We are going to have some carbon oriented policy carbon limiting similar to race limiting carbon neutral similar to you know performance or qo s things like that So people were using their service in a responsible way and in a sustainable way Without having to worry about what they were over budgeting their carbon credits or over consume their you know Their budget in the power and carbon. So hopefully that is something we can also support in the future Let's answer our presentation and we open two questions Yes, I think a protocol is you have to move over there the speaker over there I was wondering if you can correlate or add correlation to Other resource consumption like networking for instance and also how much does it itself Consume like can you actually figure out how eb how much energy ebpf consumes? Thank you. That's a very good question. So the idea is The question is how much energy we can how much overhead and how much energy we can measure So let's just go to one one thing at a time. So ebpf the overhead is slow It's a very low But when I say low that is a very subjective with you identify certain things we can optimize But in all in Transparency when we run capital we see capital only use two percent of the cpu And that can be optimized as well because most of the capital overhead comes from permissions We can use you know more efficient to permit those clients api in the future The other thing we do not Measure is the ebpf itself how much overhead in ebpf itself So a nice thing with that is our nearest kernel keep progressing is supporting more Advanced usage of ebpf. So we start we Right now is small. We can still optimize in the future So the system yourself or when we collect information and do the correlation That's obviously puts a lot of pressure on the promethys and hopefully we can have more Efficient Aggregation that's so a permission can support or any other analytic system can support So we do not have to use a lot of aggregation Accreation in the future