 Okay, so the next one is by a tennis and he will be talking us about environmentally sustainable AI via powerware batch scheduling Thank you very much. I hope everybody hears me Yeah, my name is Satan as Satan as of today. I'm going to present a teaser of a joint work with Daniel Wilson and Christopher catapult From from the Boston University And I'm from Intel so the work is about sustainable AI through power aware batch scheduling Actually, I'm based in Europe and as some of you might know a Big issue what we have in Europe are the power cost which explodes and the first kind of Indication like that's a problem if you will look also on AI workloads. This becomes quite interesting So nowadays in 2023 there was Reboar from Schneider electric about the power consumption what we have today in the data center So we see around 54 gigawatts of power consumed in the data center and around 8% actually going in AI workloads Projecting out to 2028 this will increase. So we will see Maybe 15 to 20 percent going to AI workloads in terms of power consumption, so we speak about 20 gigawatt and Yeah, so that's quite a lot and in this talk. We are focusing on AI training Yes, this is a very energy Intensive workloads very compute intensive workload. So it's the one which Consumes a lot of power out of those then data centers Actually, the workload is classical HPC workload. So what of the techniques which we saw in and applied in HPC hold also here Let's look a little bit deeper. So AI workloads also can behave very interestingly So there are such kind of curves describing basically how on the AI workload Can behave if you Play with power cap basically if you run the workload with less power So you will get a slowdown or than certain conditions But the interesting message here that those slowdowns can be described with those non-linear curves And you can do some optimizations basically in your data center so we thought about putting that in a batch scheduling framework and Usually what we could see in a batch scheduling framework You have the admin view where an admin wants to prescribe Global cluster power limit and then you have the user who can ask for a certain power limit For his job or a slowdown. So let's say the user is okay with 10% slowdown he can specify that and basically the job of the Cueing mechanism is to fulfill these conditions So we propose an architecture where we have a demon set running our kind of power Powerware service we we call it geopm service which can speak with Modern GPUs through specific APIs to do power tuning and Additionally, we have jobs which are specifying Bad jobs for AI workloads the bad job consists of two containers We have a side container which issues a Message to our geopm service with the power cap and the actual application and then on the side We have the present previously presented framework Q Basically to do the batch scheduling in Kubernetes We have the MPI operator to run MPI together with this kind of parallel training jobs So we are able to configure power limits on the cluster level through through the Q configuration mechanisms We can do sweeps of jobs which allows us to build a model a power model for for a specific workload and then a user can select Let's say workload slowdown and execute the job based on that parameter This is a little bit in a cartoon how this can look like so you can have a queue where users submit three jobs each with different power cap 1500 thousand two thousand and basically the the admin configured to the queue The cluster to be limited to five kilowatts. So a four job will get Will be suspended until until a job finishes and we have a few more jobs to do And we have enough power to execute that that job Right. So it's closing remarks. So we developed a New approach for power capping of AI workloads in in Kubernetes This is very simply integrated in in the queue batch frame framework We use a new feature of Kubernetes 1.28 The unit container sidecars, which allows us basically to do prologue and epilogue scripts To configure power caps for the nodes and if you're interested to learn more about the full approach Come and see us on Thursday from 11 a.m. Where we will have 30 minutes presentation about that. Thank you Thank you Again, we have a time for one question. There you go while we switch laptops I would suggest that we start switching though just to speed up but go ahead. Yeah, so can you hear me? Yeah, just a quick question in a world where resources are scarce saying in your particular scenario They're obviously running a training job and they're running that on GPUs. You know given the DC GM might another way of looking at this be Asking users to maybe the flops requirement or something like that. So if you lower the power You you you still hold on to those resources that maybe someone else could have landed on I'm just wondering in another spin if you asked for the the flops or something like that Then you could maybe place them on a different GPU and then hold that GPU for someone that needs more just as a thought Only because you know while powers scarce GPUs are scarce as well now. So I'm just wondering how you unblock that just a quick question I was thinking maybe there was previously a concept Presented in queue about sharing between different cues there there is was this concept for cohorts Where you could do actually stealing You can steal power from another cohort or still flops from another cohort It's maybe quite interesting to combine the two words Thank you Awesome, let's thank you after a second