 Welcome, everybody, to this session on building carbon awareness with Kata. I hope everybody has had an awesome week of Cloud Native Goodness and ate some really good deep dish pizzas, because I know I did. You guys all made it. This is the last session on the last day of KubeCon. So, woohoo, you guys are troopers. Yeah. For those of you that don't know me, my name is Paul Yu. I am a developer advocate at Microsoft. And I'm here to talk to you about sustainability and how you can add carbon awareness to your applications without making any application code changes all with using Kata. And I know many companies have some very, very aggressive sustainability goals. Microsoft for one, we're aiming to go carbon negative by the year 2030. And in this era of AI, where we are now relying on very powerful GPU machines to run our models and whatnot, we need to do everything we can to save as much and reduce as much carbon emission as possible. And so what we'll talk about today is the importance of being energy conscious and how you can build sustainable applications using green software principles as outlined by the Green Software Foundation, otherwise known as GSF. I'll also demo two open source tools that you can actually use today and test this all out in your Kubernetes cluster, whether it's local, on-prem, or any cloud. And so I'll demo two tools. One is the Kubernetes Carbon Intensity Exporter and the other will be the Carbon Aware Kata Operator. So let's dig in. But before we dig in, let's talk a little bit about being energy conscious. And here's where I want to take a trip down memory lane and maybe this story might resonate with you. So let's go back in time. You're a kid. It's nighttime. You're in your room, lights are on, TV's on. You're watching a show, playing video games, whatever. You're thirsty. So you get up, head over to the kitchen, and get a drink. Your parent walks by the room, notices everything's on, lights are on, TV's on, everything's on, and begins to shout. Turn off that light switch when you're not in the room. Has that happened to you? It happened to me a lot, and I got yelled at a lot, right? So your parents, they were being energy conscious. But let's be real, they were being money conscious, right? Because saving and reducing energy has a lot of benefits, including saving money. And of course, being sustainable for our environment. Now, over time, we just learn, we're just conditioned. Turn the lights off. When we leave the room, turn the lights off. Otherwise, we're going to yell that. And we start to adapt that behavior as we grow. And now I'm the one that's yelling at my children, my daughter, when she leaves the bathroom and the lights on. I'm yelling at her. Turn that light off, right? And these are just things that we do naturally, especially doing chores around the house. Back when I had to do my own laundry, I didn't own a washer and dryer, so I had to go to the laundromat. I would never go to the laundromat in the middle of the day because it was just always too busy. There wasn't enough washers and dryers. And so I would always go there early in the morning at the crack of dawn, right when they open, so I could get in, get out. And I'm there when there's less demand on washers and dryers. Same deal with my own electric vehicle. And I need to charge that vehicle. I don't charge it when I get home. I don't charge it between the hours of 4 and 8 because it's more advantageous for you to charge it, let's say overnight, like hours of 9 p.m. to 4 a.m. or whatnot. So the point of that is basically to say, over time we've been conditioned to do certain things at certain times because it's more advantageous to us. We do things when there is less demand on the resources that we're trying to use. And so why can't we take some of these concepts and apply them when we're building our applications, building our applications in our data centers? Why aren't we looking at the most optimal way for applications to reduce energy consumption? You see, we need to change that, and we can change that as long as we start to look through the lens of green software principles by GSF. And green software principles will actually help you to be conscious of the energy that you're using within your applications, the hardware that you're choosing, the ways the apps are written, and the way they are run. So let's take a look at that. These are the three pillars of the green software principles that the Green Software Foundation is outlined for us. Now, this is an emerging discipline, and it's still a little bit new, but it's pretty easy to understand, right? The first one is all about energy efficiency. Pretty simple. Use less electricity, right? Consume the least amount possible. Remember your parents yelling at you? Yeah, same deal. But this time, Mother Nature's yelling at you or Captain Planet, if you remember that cartoon, Captain Planet is yelling at you. So turn those servers off when you're not using them is the message. But consuming less energy is not enough. We also need to be hardware-efficient, using or selecting hardware that has the least amount of embodied carbon as possible. But that's not always possible. Actually, sometimes it's out of our control, but what's in your control is the amount of hardware that you're using, the way you scale your applications. And you also have the option to choose, let's say, OOS architecture like ARM64 that just generally uses less power. The third principle, and this is the principle that we'll be focusing on for the rest of the talk, is carbon awareness. And this is essentially a principle or a mindset that your application does more when the carbon intensity or the energy is clean and your application does less when the energy is dirty. Now, what do I mean by that? Well, you see, not all electricity is produced the same way. Generally, an electricity grid provider will tend to source its electricity from renewables like wind, solar, hydroelectric. But that's not always possible in every single geographic location. And when they can't you rely on renewables, then they have to resort to burning fossil fuels. Renewables are great because they emit very, very little carbon into the atmosphere as you're generating that. This dirty energy, it emits a lot of carbon, and this is a period what's known as high carbon intensity. And so the general idea of being carbon aware is to simply do less when the carbon intensity is high. Now, let's see why it's important that we do less when the carbon intensity is high. There's this thing called the Software Carbon Intensity Specification which is currently being worked on by the Green Software Foundation and it's going through the standardization process now so that hopefully all developers that are interested in this space will be able to use this as a guideline. Now, why is it important to lower carbon intensity? Well, if you can see here, it's a pretty important factor of this formula. And this formula actually is on the same pillars that we just talked about. Energy efficiency, hardware efficiency, and carbon awareness. And you can see here, just by reducing your demand on dirty electricity, you are effectively lowering I in this equation, which is a multiplier. And so you can see how this becomes a major step towards sustainability. And there's a couple of ways that you can actually lower I. The first way you can lower I is through a concept called demand shifting. If in your region, carbon intensity is really high, you can look to shift that workload to a region where carbon intensity is low. Now, that's not trivial and it may not be possible all the time. So the other option that you have is to actually put a pause on that workload. So you can actually figure out that period of high carbon intensity and say, I'm not going to run it right now, my job could wait. And so I'll wait it out and wait for the carbon intensity to drop a little bit. So demand shifting is a concept of moving your workload through space or time. The other thing that you can do is this with what's known as demand shaping. This is where you are raising your demand for electricity when carbon intensity is low and lowering your demand for electricity when carbon intensity is high. Basically, it's like a governor, right? You're raising or lowering the limit based on that. And this is where the carbon and where a CADA operator operates. Let's take a look at the architecture of this operator. So this is a pretty simplified diagram. You may have remembered this from KubeCon in Amsterdam. Basically, it gives you a general idea of how this operator works. Step number zero, you need to install the operator. Actually, step number zero, you need the workload. You need CADA, because obviously this operates on top of CADA, and you need a scaler in place. Now, this thing just comes over the top of CADA and you implement this operator and you need to deploy the CRD to act as a governor for your CADA scaler, right? But this alone won't do the job. You need carbon intensity data to determine whether or not you want to raise or lower the limit. This is where the second operator, the Kubernetes carbon intensity exporter comes into play. And this is a really cool project because it's looking to solve that problem of getting electricity grid emission data from various providers, let's say, from dot time or electricity maps, and it's looking to bring that into your Kubernetes cluster in a standardized way, in a standardized data location, right? So this operator, the exporter, actually leverages the carbon aware SDK, which is also an open source project brought to you by the Green Software Foundation. So kudos to them for that. The exporter effectively acts as a wrapper for the various APIs. So you go sign up for either watt time or electricity maps, get an API key, install that operator into your cluster, and then it'll start to pull that data out and store everything in a config map. Once you have the CADA operator in place, the exporter in place, and you have your data, you now have the ability to deploy this CRD and you configure the CRD and you start to set the thresholds in what you think is high carbon intensity based on your workload requirements, your region that you're in, because they kind of vary between regions. So rather than this operator defining what's high, medium, and low in terms of intensity, we let you, the human operator, determine that. And let's see how that's done. What we'll do is we'll take a quick tour of some Go code. The operator is actually written using CubeBuilder. So shout out to the CubeBuilder folks because they made it pretty easy to write. This is a simplified version of the Go struct that basically controls the entire CRD. What we have here is we follow a similar format to what Cata did where they target their deployments or whatnot. And so here this is effectively like a meta operator. And so it is acting on top of your existing Cata-scaled objects or Cata-scaled jobs. And so you just point that out. The next thing that you need to do is you need to define where that data source is. And so we already have a config map and we would just point to that. If you do not want to sign up for any of the Electricity Grid Emission Provider APIs, no problem. You can actually mock data into your cluster and just test it out and see how it works. The next thing that you would want to take close attention to is this thing called the Carbon Intensity Config. This is where you define what's high, medium, and low, high, medium, and low, and then you define the threshold. So let's take a look at that next. This struct is pretty simple. All you do is you set the upper limit on what that threshold would be. And you can define any number of these thresholds so you can create as many buckets as you want. So you set, let's say, high is 500 for you. So you set that to the threshold to 500 and then you set the max replicas to a low number, I don't know, like 10 replicas that you want to ultimately scale out to. And that's the way you configure that. Pretty simple, just two properties. The next thing that I want to call out is this thing called Eco Mode Off. Some of you in your vehicles, you may have a little green button with a plant on it. That's like the Eco Mode Off. It's like the kill switch for the governor, we should say. Right? And there's a couple of different ways you can actually turn off this carbonware scaler because you might just need to get your workload done. You probably don't want to throttle your workload all the time. There may be instances where you want to override that. And so what you would do is first define when this Eco Mode is off, how many replicas do you want to ultimately scale out to? Then you need to schedule when this thing should turn off or configure it. There's a couple of different ways to do that. You could run a custom schedule, which effectively is like starting end times, right? So let's say tomorrow through Saturday, I want it off. You can schedule that just passing the date and you're good. You can also turn this off based on a recurring schedule. This accepts cron syntax. And let's say I don't want Eco Mode off every day between, I don't know, 2 a.m. to 4 a.m. You can schedule it that way to not be in Eco Mode. The other thing that you can do is you can actually specify how long is too long. Let's just say you don't want to throttle your workload for a very long amount of time. And so you can say, if my carbon intensity is 500, which we defined as high, for like 8 hours, 16 hour blocks, well, the carbon aware cater operator can look at that and say, okay, you defined this to be too long and we're going to disable that so you can get your workload done. So that's pretty much it. And this is ultimately what the CRD looks like. So we have here the carbon intensity forecast data source. We're just pointing to the config map. It's in the cube system namespace and that's the config map name there, carbon intensity. Here is where you define your carbon intensity thresholds. I only have two here, but you can define as many buckets as you need. The final thing here is the Eco Mode off. We have an example that has max replicas. So we want to go back out to the default of 100 because cater defaults to 100 max replicas. And then we have custom schedules. So on November 9 during this time block, it'll just be off. And then we also have crime syntax in there as well. So at 11 p.m. on Thursdays and Fridays, your eco mode is off. So that's pretty much it from a design perspective. Let's see how this thing works. So let me break out of this presentation and jump over into my terminal. And so I'm going to do this live and I'm going to be at the mercy of the demo gods and conference Wi-Fi. Hopefully the Wi-Fi isn't being hammered right now. So what I'm going to do is I'm actually going to create a brand new kind cluster because remember I said this thing works on any Kubernetes cluster. So we'll just wait for the cluster to create. Now some of these might take a little bit longer than I normally expect because it needs to download containers. So the cluster is in place. First thing I wanted to do is step zero, right? Deploy the operator. And so I will deploy the operator and it's literally just a YAML file to deploy that. So this thing will download the container and install the CRDs into the cluster. So we're good there. The next thing I want to do is I want to install the Kubernetes carbon intensity exporter and you can actually do that using a Helm chart. So I'll do a Helm install and within the Helm install command you'll see here that you are passing in the Wattime username and Wattime password. Now this could be Wattime or it could be electricity maps. The other thing that you need to pass in is the region that you're in. And grid emissions data, they have their own region codes with their own geolocations but by default the Carbonware SDK kind of maps to Azure regions but you can send in a mapping file of your own to map to whatever cloud provider or whatever geolocation you need. So I'll go ahead and install that in my cluster. So great, I have the Cata operator installed in my cluster and then I have the exporter in my cluster. I'm missing a workload. I don't have that yet because remember this is a fresh cluster. So I'll go and deploy this sample deployment and it's just a sample, it's really simple. The whole point is to just show you the Cata operator not necessarily this workload but it's just a Redis cache and a cron job that pushes words into this Redis list and then a workload that processes off the Redis list so it's not terribly fancy. The next thing I need is a, hang on a second, I need the scaled object but I can't deploy the scaled object because I don't have Cata yet installed and so what I'll do is I will actually use Helm again to install Cata into my cluster and this is pretty easy. And so we should have Cata installed in our cluster and now at this point I should be able to deploy my scaled object. There we go. So now we have Cata operator, the exporter, we have a sample workload and we are scaling that sample workload based on Cata and what we'll do here is we can actually take a look at the scaled object and you can see here that the max replicas is 100. Let's check to see how the exporter is working and we'll just take a look at the pods. Okay, cool, it is running. Let's see, if it's running then I should have a config map. So I'll do this little query here where I am grabbing the config map called carbon intensity in my cube system namespace and awesome, I have 288 records and I can see here that my maximum forecast because this forecast goes out about a day or two and the operator runs every 12 hours to refresh the config map and so you can see like based on the forecast that I retrieved there's a max forecast of 440 and some change and a min forecast of 349 and some change. In addition to kind of like the summary there is raw data inside of the config map and so that's actually in there as a immutable, well the config map is immutable but it's in there as binary data. So if you ran that query you can see all the individual carbon intensity locations for that. So we see we have these weird region codes but remember they're mapped to cloud regions as well and then you can see here that they have a duration and a specific value. Okay, so great. We are ready to actually deploy the CRD which will actually raise or lower that I guess the threshold in how much demand we're asking for on electricity but before I do that remember we had to configure the thresholds but I don't have all the values with me just yet. What I'll do is I'll actually run some math in a bash terminal to get these figures for me and so I have the minimum and maximum and then what I did here is basically I'm just calculating how big my buckets should be so I have high, medium and low and so I'm just dividing it by three and just very simple math so that kind of gives me an idea on what the low threshold should be, what the medium threshold should be and what high should be. Okay, so 380, 410 and 440. Okay, help me out and sometimes I forget what these values are so if I do forget let's go fill that in. So here what we're doing is we're basically cycling through our high, medium and low buckets and we're defining what that max replica should be based on that threshold so remember low intensity, do more. High intensity, do less. Right, so it should be an inverse. So 366, that's not quite right. 349, no, no, 380, 410 and 440 is what I need so let's go ahead and update those values. 380, 410, thank you. Thank you, thank you, you are awesome. Okay, so these are my thresholds and if I am at 440 for eight hours then eco mode will be off and then same deal like in the example eco mode will be off at the 11th hour on Thursdays and Fridays. Alright, there we go. We just deployed the CRD and now it is acting upon the carbon intensity data that's in our Kubernetes cluster and it's basically acting as a governor and reducing demand based on what the current forecast is. So now, if I take a look at my logs what you should see here is that the operator is actually using the carbon intensity that's in the config map and where is it at? Oh, look at this. The current forecast is 430, that's high for us and so we want to set that to a max replica of 10 and then there we go. The max replica has been lowered and if I do a scaled object again you can see that it actually operated on top of the scaled object and reduced the demand. Okay, so we're at 10. Now, looking at logs is cool if you're like a command line junkie and you just want to look at everything through logs but we want to visualize this, right? So what you can do is CubeBuilder has the ability to install the Prometheus operator and so you can actually look at all this stuff using the Prometheus and Grafana stack and so what I'll do is I will go ahead and deploy this operator and then I will also deploy a service monitor and then I will actually deploy the Prometheus and Grafana stack next. So back that up and then a whole bunch of stuff just flies by and you know, it's like magic we have monitoring and observability. That's why I love Cloud Native. So what we need to do is we need to wait for some pods to come online. So let's see monitoring, wait and depending on how Wi-Fi is because I tried this in my hotel Wi-Fi wasn't great so it was taking a little bit of time. So we'll give this a little bit but if it doesn't come up in time I actually have all this running in my Azure Kubernetes service cluster and we can see what the data looks like. In fact, it might be even better just to see what it looks like in the in a long-running cluster because I literally just created this right now. So why don't we just go ahead and do that. I will flip over to a Grafana dashboard and you can see here oh and I forgot to call out there is a very, very simple dashboard inside of the Carbon WorkHata operator repo. It's nothing fancy. This is just what it looks like. It's either you want to look at logs or you want to look at a pretty graph and so what this is telling you this green line at the top is actually showing you the carbon intensity over a period of time and this blue line is showing you what that default Mach replica was before Carbon WorkHata was in place. What this yellow line does it actually shows you over time how we are raising or lowering the demand of electricity and you can see here some inverse effects. So over here on the left carbon intensity was high and therefore you can see down here in yellow we lower that bar. You can see as it progressed through the day it must have been at medium because now we're at 60 Mach replicas and then you can see a big drop here in carbon intensity and we actually burst it a little bit higher than the default Machs because that's what we configured it to do. We set it to 110. Then you can see that over here you're probably wondering what's going on here because carbon intensity was at a steady state but I'm not sure if you can see it here this was the 11th hour EcoMood was off so we just ran at whatever the default Mach replica was at that time and then over this long period of time it was steady state high carbon intensity and we kind of limited down to 10 Mach replicas over that period of time. So that's basically it. You can literally add carbon awareness reduce carbon emissions within your software without making any application code changes. Now there is a caveat in that it really depends on your workload and stuff like this is great for batch workloads stuff that's not really time sensitive. So I would highly suggest that you give it a try. One thing I'll call out and it'll be in the presentation that I upload is I have all this documented so if you want to try this at home you are more than welcome to head over to the repo and literally if you have Docker and Kine and QCT on those tools you're good to go and try it out. So with that I would say if I go back to my slides here I gotta catch up I would say next steps like I mentioned this is an emerging field but it's super interesting because like I mentioned before everything we can to reduce our carbon emissions because it's just the right thing to do, right? I would invite everybody that's here to join in in this discussion it's in the CataCorp project and we're hoping to add this not as an operator as a meta operator for Cata but adding it into the Cata project itself. So I had some use cases if you have others please join in on the discussion and let's help design this for the final state I do want to call out that if you want to learn more about green software there is a event going on next Thursday I believe called decarbonized software so if you click that you can gain access and register for that it's a virtual event about green software practitioner information go to that website there and if you want to learn more about AKS's position on sustainability there's a link there as well I want to give a quick shout out to all the folks that actually helped build this project I'm here standing alone but I certainly did not do this alone there's a bunch of people here that actually contributed to this project and I just want to make sure that they have their recognition so I would say Captain Planet the power is yours that QR code will take you straight to the Github repo and you can go ahead and give it a try and that's all I have today I just want to say thank you for your time and attention and special shout out to all the US military veterans out there because it is Veterans Day this weekend so thank you so much