 So, welcome to the CNCF Technical Advisory Group for Environmental Sustainability, basically update. So, this is Niki. Hello. I'm a software engineer at Grafana Labs, and I co-chair the Green Reviews Working Group in the TAG. And I'm Marla Weston. I'm over at Intel. I'm a Cloud Software Architect. I'm one of the chairs, co-chairs for the CNCF Environmental Sustainability Tag. We have another chair in our room, Leo, right there, we've, so here's our mission statement. So the TAG's goal is to advocate for, develop, support, and help evaluate environmental sustainability initiatives in cloud native technologies in the ecosystem. In the TAG, we identify ways for service providers to reduce their consumption and carbon footprint through cloud native tooling, providers and vendors of tooling. So here's some links. Links, we have the TAG Environmental Communication, so there's a home, GitHub issues, calendar, mailing list, Slack channels, there's probably should be one later. And we have four base projects right now. We have the Landscape, which is a big document, we'll go into that later, the CNCF Project Green Reviews, which Nicky will cover, our comms group, which is Community Advocacy and Outreach, and then blogs and demos. All of this can be found on the TAG's website or through the repository as well. So there's LandscapeDoc, I mostly run this thing. The point is to create a comprehensive set of projects, organizations and research, targeting sustainability and cloud native systems and put it in one place. We really welcome contributions to the stock, it's still in preliminary stages. I was talking with a service mesh person yesterday, because we don't have anything on service meshes and what solutions use less power than other solutions. There's a lot that can be added. There's also a lot of tooling there, so if you're starting from nowhere, from scratch and you want to know how to reduce the power footprint of your cluster or if you're using cloud providers, how you're doing that, you can also read through that doc. But it's still version 1.2 or something like that, so we really would welcome more contributions in that space. And then there's working group comms. The motivation is to promote open source sustainability projects and still remain authentic and true to open source openness and transparency. And we basically, in that group, organize communications and market efforts through the ecosystem. This is being run by two people, Cara Dahlia and Patricia Cahill. The objectives are to monitor sustainability events. We try to get out web pages before big conferences, so if you know of one, then we will add a website, collaborate with other open source projects, and out of scope, we don't market products or solutions for vendors and we don't market for other community groups. It's a little outside of what we can do. We have three deliverables, which are templates for contributions for blogs and white papers, publishing social media events, posts and calendars, and planning in person and virtual events. And how to get involved. They meet for half an hour once a week, and you'll forgive me, I don't remember what the time is. We can look that up. The meeting is included in our calendar, and new members are always welcome. If you can't make the meeting, that doesn't mean you can't contribute. That just means you'll contribute through the SOC channel. Yeah, so we do have a calendar that you can subscribe to. You can add it to your own calendar, and that's all in the repo, in the read me. Next, I'd like to present the working group Green Reviews, that I co-chair together with Christina Devochko, and some of the goals of the Green Reviews working group is to help CNCF projects and the maintainers of this project to review their sustainability footprint. So it's really still very new to evaluate the carbon footprints of cloud-native technology, so we're aiming to help facilitate this by looking at the projects that are in the CNCF. So we want to create a pipeline for gathering these green metrics, and I'll delve more into what green metrics are in a minute. And we want to provide resources, reviews, and guidelines to the CNCF project maintainers to improve the efficiency of the software. We have a charter that you can find that goes more into our goals and deliverables. I wanted to mention that something that's out of scope is creating new tooling from scratch. By that, I mean we are building get-up-action workflows, we are building cluster configuration, but we are not building tooling from scratch. We are trying to use existing tooling where possible. So what are we building? We are gathering everything in the design document to start. We have a draft, and we will share the slides afterwards so you can find them. But there's a design document. I invite you to read the charter and then go through the design document and make any suggestions or questions, etc., or take ownership of different parts of the pipeline of that is of interest to you. Some of the processes are similar to the tag security assessment. If you're a CNCF project maintainer, that might be familiar to you since it's part of the maturity model. So to go from sandbox incubating graduation, like the different steps in the maturity model, you have to go through certain tasks, and the tag security assessment is one of them, and we are trying to see if we can add something around sustainability there. So what else are we doing? We're building on infrastructure that is donated that I'll go into more depth in a second, in the test pipeline and metrics gathering and assessment. So there's quite a lot, and we really want to invite you to learn and contribute if you'd like to. So the first part, looking at the end goal is to gather metrics that are related to sustainability. And so we're looking at the software carbon intensity specification as a leading principle, but we're still assessing how much of it we can implement and how that looks in a cloud-native environment. The SCI is a specification developed by the Green Software Foundation, and it's currently in the process of becoming an ISO standard. There is a guide that you can look into for more information about it, and I really like this diagram by ThoughtWorks that shows what the SCI is like. So there's an energy component, so the carbon emitted per kilowatt hour of energy, that you multiply into a carbon equivalent with a carbon coefficient, or there's many different ways of achieving this, depending on how much information you have about where your software is running. And there is another component that is about embodied carbon emissions, so the software that is running in the data center, for example, it takes power to run the infrastructure, there is also carbon emitted when this hardware is manufactured, so that is called embedded carbon, and that's also something very difficult to do in a cloud-native ecosystem, but in a cloud-native environment, but it is something that we are creating this kind of playground environment to ask those questions and try to answer them as best we can and identify gaps, and we do follow some of the guidance from the Green Software Foundation for best practices on how to do that. And the last part of the SCI is R, which is the functional unit, so it's how your application scales, so you want to look at all these metrics and evaluate how the application scales to have a way of putting those software boundaries for what you're trying to measure. And one quick note, Tammy over here in the audience is part of the GSF, so if you have more questions, go talk to her. So what we're trying to build with the pipeline, what we're doing at the moment, we have infrastructure, which is dominated by Equinix, so we're using Equinix Metal and a CNCF community, it's actually, the Equinix Metal is donated to the CNCF and we are working with the CNCF to use that. We now have project-wide access in Equinix Metal and we're using that infrastructure. We're looking at cluster provisioning, so we have different, we're trying to find which tool is best to use for use case, and for the cluster configuration, we do want to use GitOps as much as possible to benefit from the access control of having pull requests for external contributors to be able to contribute, and access control is one of the questions that we are also looking at. How can we build on infrastructure in the open and with a community of external contributors? There's a lot of things that come up, like how are we going to use a personal access token for flux when we're all external contributors. If you have ideas for these things, I'm sure we can, you know, we have a pool of very smart people in the audience and I would love to hear your feedback and suggestions. And for the sustainability assessment pipeline, we are looking first at manual tests that we will then automate using a GitHub Action workflow, pretty fairly straightforward. One thing I wanted to mention also is that the first CNCF project that we're working with is Falco, which is a security tool, and that is EBPF-based as well, which is interesting given that we're using Kepler, which is an energy monitoring tool, which is also EBPF-based. So that's interesting. We do, it is a little, like the learning curve might be a little bit high when we have new tools such as Falco, but we are working with the maintainers of the project to see, okay, this is the working group's responsibility, how much will the maintainers take on, who writes the load tests, what kind of demo workload we're working with to evaluate the CNCF project, and how do we deploy the tool, et cetera. So there's great questions. It's really interesting if you'd like to gain experience in a certain area, it can also be a way to develop those skills and learn. So some of the next steps, we are looking at scaling our processes and scaling our team sustainably. I think we can all agree as open source contributors, it can be, you know, sometimes the most important part is to focus on the contributors and make sure that people have the tools and resources to succeed and that we are making sure there are no blockers. And with that, I mean, we are looking for more contributors so that we can scale sustainably as a team to make this project a success. Some things we're looking at, you know, the test cadence. So do we test per release? Do we test as part of the maturity model? And for the load test as well, should we use K6 or something else? And how are we going to do the schema for this test? One of the ideas is to maybe export some of this data to DevStats, which is some, it's actually a Grafana dashboard that is maintained by the CNCF that contains information and metrics about different CNCF projects. For example, there is a rating of the security, there's a security metric for projects and we're thinking of doing something similar for sustainability potential in the future. We're looking for the next CNCF project that we will take on after FALCO. So we do invite CNCF project maintainers who are interested to open an issue in the tag repo so that we can, you know, maybe look at that project next. And some other things that we'd like to do is trial tools potentially that are related to sustainability or patterns. We have quite a lot of items in the roadmap, but there have been suggestions like, oh, maybe you should use the green software foundation's carbon CI, which is a static analysis tool, which you can use to get metrics about your software as part of CI step. So that way you can get some metrics immediately in the pull request, for example, about how you're doing or how much the feature you're introducing or the fix or whatever, it's changing the sustainability of the tool that you're building. And another idea was to measure the power consumption of networking components, which is a big question mark as well. So please do get involved. We have meetings that alternate with the tag regular meetings. So every other Wednesday, every second and fourth Wednesday of the month, we have a Slack channel, and we do encourage asynchronous collaboration because we do know the time of the meeting might not work for everyone, so we do encourage people to work on the repo or on Slack. Yeah, and so we have issues on the GitHub board and the design doc. So the last current project kind of within our tag is blogs and demos. We try to have demos of projects minimally monthly at our standard meeting, and these are usually pretty interesting. We've had Kepler do one, we've had a couple, Scafandra do one, we've had a few others, and they're pretty informative. So if you have a project that you're working on that you are interested in sharing, especially around sustainability, I've spoken with both Istio and LinkerD, and they have sustainability improvements, but it would be better to have some sort of measurement and some understanding in a demo. Please just contact one of the chairs or leads to get added to the agenda. That's this person or myself, Leo, and we do have other people on the channel. We also do blogs, you can open an Issue, here's where you open an Issue. Once you open an Issue, you usually link to a Google document, whatever that looks like, and we're happy to do commentary, and then you'll open a PR, and after the PR you get feedback and then we publish it onto our website, so our website's gotten getting more and more content as time goes on. We were a little bit sad because we had a meetup yesterday, but we didn't have anyone really show up. We had, what, five people? We had five people. Yeah. And it wasn't well advertised, so we kind of wanted to use the rest of the session so people can go and have conversations with each other on what they're working on, what they care about. You're also welcome to ask questions of us before we start that. Do you have any other questions? So yeah, so one thing, just a logistical thing, are these slides available from the SCED website? I mean, you had a lot of interesting links to go to different resources, so I don't know if you guys could post that later, that'd be great. Yeah, I'm kind of interested in this topic because we're trying to understand the carbon impact of our scientific workloads. Analysis related to the CERN project or Particle Physics Organization, we have data centers around the world and we've got some of them, I mean, most of them are, I would say, traditional HPC, but we are increasingly introducing Kubernetes as a platform in particular for analysis. So yeah, I would like to understand what kind of tools are available, what the right framework is for that and how to sort of understand this because it can help us with understanding our decisions going forward for what we need to invest in and to educate our communities to what their impact is on the environment for the science that they're doing. Yeah, so if you follow the AIML, it's a similar path with HPC, the types of computations you're doing, while they can be different, the type of traffic you're talking about is similar. The thing that's still lacking in the tooling is being able to measure the networking. So that's really where you want to start. And so my request is at this point to start talking with Service Mesh because I think the Service Mesh, if you can get them to measure, the same measurements are going to be applicable also for the whole system. With HPC, there was also, are you familiar with Jim Larrows? He started a whole sustainability movement within HPC. You should look him up, he was Sandy in National Laboratories, but he started a whole sustainability thing within HPC. And the types of measurements they did there, I think we need to translate because there's also some interesting things because you have checkpointing with HPC, right? However, you have to restart the whole thing if one of your notes crashes. And there was a concept of a mean time to failure. And so if you lowered the power, but your mean time to failure was at the point when your workload hadn't finished yet because of the power lowering, then it actually consumed more energy. So we need to be fairly conscious when we start stepping into those spaces. As far as measurement, if you're only doing CPU and memory, you can use Kepler. My concern with Kepler, it's a great project, is that I don't know, like with any tracing mechanism, right, how much is it consuming versus how much is it running. So it may be useful to run it on the subset of your workloads and then get metrics and then do calculations according to that up and down, right, depending on how often you're running those workloads. Does that help at all? And you're also welcome to show up at our meetings and help with the networking because the networking is a big problem still. I wanted to add to that with Kepler and EBPF-based energy monitoring. EBPF can emit a lot of metrics. So it's probably best, as Marlo said, to use it on the small subsets, very targeted action rather than using it to monitor your entire infrastructure. So that's just my personal experience is best to start small and evaluate how much Kepler itself is consuming. And you have to be careful of EBPF. There is a known issue that when you're doing a high-performance networking, it can impact your networking because it's kind of consuming the best networking paths that are available. So right now, I would recommend do off-the-box measurement, make a change and then do off-the-box measurement, or use Kepler to try to figure out individual cores, can you tune the cores and then go back. I'm also happy to talk to you afterwards. I have other tooling, too. Hi. Can you hear me? Yep. I guess I'll... I'm short, too. Yeah, I got it. So I'm sorry about the meetup. I would have loved to come. I didn't know about this. And I can relate because I've been in open-source community building and if you do an event and nobody shows up, it's not good. Yeah. Fun. So my question and my apologies, if I've missed it, you might have covered in the talk earlier. Where is this project in maybe mandating all the Kubernetes projects to have this kind of scan and maybe meet certain goals? Are we there yet? We're not there. So Green Reviews is where we're starting to figure out how to measure. I don't know if we're going to end up mandating because different projects have different goals, right? So if you're doing a high-performance thing and you don't care about power because you're just running it for three minutes, that's one thing. But if you're running it over and over again for three minutes, then maybe you care. So maybe parameters can be built for different projects like that. Yeah. We're still evaluating the cadence and how, you know, if it's optional part of a maturity model or so we haven't, we're not close to mandating it yet. And I would say looking at our roadmap until after KubeCon EU, we might have more, hopefully we have a working end-to-end flow and maybe the second project in the pipeline. But we're still far from adding it to mandating it. Yeah. Okay. And I'm interested in joining and contributing more. I'll find you guys after the talk. Great. Thank you. Thank you. Hi. I found Taiwan and I just had a kind of native sustainability week recently and got good feedback from Taiwan. The discussion on ISO 14064 has been very hitting recently in my country. In the future, the tech or working group has plan to certify with ISO or standard or have any kind of cooperation. We're working with ISO, the SCI, the software carbon intensity specification, which is from the Green Software Foundation. So that's the ISO standard that we're using as a leading principle. We're not yet a stage where we can fully implement it, but it is leading the metrics that we're trying to gather in the Green Reviews Working Group. Creating an ISO standard is very time-consuming, takes a very, very long time. So I don't think that would be something that the tag does, but maybe the Green Software Foundation has more information and Tammy can answer your questions. Okay. Thanks. Hi. Olivier Tardieu, IBM Research. We're doing a lot of work in this space, but I'm a little bit curious to understand what you mean by measuring the carbon footprint of a project, because it depends so much on what you're using the project for, how you're using the project. So could you explain a bit more how you generalize these two different kinds of projects? Yeah. So the SCI is trying to achieve this. The software carbon intensity specification is kind of, is guiding this question of how do we measure the sustainability or the carbon footprint of our software. So I would recommend the SCI guide as something to look into. And that's what the tag is using as our green metric. And so it combines energy, combines carbon coefficients, carbon metrics. Yeah, it will probably get more complicated too with time, because when you start looking at where your workloads are, so if you are somewhere where you have hydropower, that's going to be less of it, and it's cold, so you don't have to do a lot of cooling. That's going to be less of a carbon footprint than if you're somewhere warm that doesn't have solar, right? So it really depends on where your workload is running to some degree, and there's a lot of discussions about where you place your workload. So what region can you move them according to region, or is there a performance requirement that you have to be co-located to your user? And there's information that we will have to backfill probably later on, because we're using this as a model, but there's information we know we don't have yet, like the networking components. So we know that the information we have right now is not complete. It doesn't paint the complete picture of the energy consumption of our software. Embedded carbon is very hard to measure in the public cloud for users. So we know that we're trying to do the most that we can with what we have available right now, and to highlight the gaps. Yeah? Hello, folks. I'm unfortunately not smart enough to figure out how much power my software is using, but one thing I'm good at is bringing people together. So I wanted to have a quick chat about, you know, just to show, like, a project I've been working on lately. The tag environment sustainability is already aware of it. I've been organizing trains for the next Cube Comparise. The idea is to bring people to Cube Comparise in a way that is sustainable, fun, and efficient. And what we've been trying to do pretty much is from different cities in Europe, from different capitals in Europe, we would like to rent a wagon of a train and have a mini pre-conference pretty much. So we would rent a whole wagon. We would call it a Cube train from London, Amsterdam, Zurich is already in place and selling tickets. And yeah, have a mini pre-conference, a hackathon, lightning talks, and things like that. The idea is to bring as many people as possible to the next Cube Comparise in the most sustainable way. So if you're interested in participating, if you're interested in organizing one, if you're just enthusiastic about the project, please come and talk to us, to me or to the other folks over at the tag environment sustainability, it would be more than happy to help. Could you mention your name also so people know how to find you? Cubetrain.io is the official website. If you write to any email over there, somebody will get it. Also, your name? Andrea Giardini. But this is very difficult to spell and people are not going to find me. So just come up here, I will leave you a business card, or you can take a picture of my name. Yeah, so Cubetrain channel? Cubetrain.io. And Cubetrain.io. I know people that would like to organize that, so I can put in contact. Definitely. Yes. Absolutely. The train options from one continent to another are limited, so it's mainly... For people who are not... If you think about renting a boat, outside Europe is a little bit more complicated. So I can tell you that right now, Cubetrain from Zurich is already reserved and taking reservations, so you can already book it today. I think like London and Amsterdam are two very easy wins, they're nearby. There is a lot of interest from the German community as well, Austrian, Swiss and Italian. So really, let's say that the initiative has outgrown me by a lot, but if you're interested in participating, I don't want to take too much time, you can find me here for the next 15 minutes or so. Yeah, awesome. Thank you. Yeah, thanks. That's a great call-out. Hi. Hello. I have... I will phrase my question as an assumption and I will just ask you to validate or invalidate that assumption. My assumption is that the largest public cloud providers or all public cloud providers do not really publicize a lot about how they consume energy. Is that a true assumption and how important is that to this overall effort? Yeah, there's a few different levels to that, so where to start? The public cloud providers, what they do share. Data information is available to us. There's the PUE of data centers that I think they do share some information about that. They do share... They do have the, for example, AWS has the customer carbon footprint tool, which is a tool available for free. It's globally accessible in an account, so yep, I mean, it's unique billing access actually. That's a top-level carbon metric that there's also a lag of three months, so it's not... It's more for long-term reporting. It's not really useful for engineers. To answer your question about why they don't provide energy metrics, and this is my understanding as a lay person, yeah, that's awesome. I did not know this, so to repeat this, it's Microsoft and Google have come to an agreement to publish Kepler metrics from an API to expose energy metrics to users by adding Kepler to their infrastructure. I think the Green Software Foundation had some work around this as well, if I'm not mistaken. I think from what I've asked this question to folks, the response I've received is that it's mainly security and the accounting per customer is very challenging, but it's great to see that actually this is changing and there are public cloud providers who are going to provide this. The regulation is also a very important aspect of this. There's the EU-CSDR, the Corporate Sustainability Reporting Directive, which was implemented a year and a half ago, I believe, so it takes 18 months for regulation to become law in the EU, and that means by January 24 companies will have to report on their carbon footprint for 2025, and that's publicly traded companies that are larger than a certain size. So I want to make sure we distinguish the difference between power use, which is what Kepler measures, and actual cost of the power at the time. So that's something that I would like to see more publicized. I know some energy companies actually tell you what the current carbon cost is of power, but I don't see that pushed all the way through to the user from the cloud provider yet. I think one of the, my main concern for this is one of the things we're working on is starting to, obviously I think probably a lot of people here, in the past year we've started to really focus more on when we run an internal developer platform, and we present the UX of here's how you're configuring your workloads, showing, okay, here's the dollar cost of the resources that you're consuming, given economic conditions and things like that. So I'm doing a step further in saying, here's the, not only is it better if you go run this workload, especially if it's like a cron workload or not really region sensitive, if you go run it in this region instead of this region for AWS, you'll save this much money. I think doing the same thing with carbon or energy usage or carbon footprint overall would be a really nice thing to be able to do, but it sounds like that's a very hard problem. If we could convince the cloud providers to tie the carbon cost to the cost of your workload, that would probably solve it. You have to go talk people into stuff, right? Okay, I'll work on that. Thank you. Hello, my name is Brandon. I work for a company called OpenGov, and we're very cost conscious, and so we've been working a lot with a tool called OpenCost, which I believe is a CNCF project now. And I didn't do a whole lot of research on this, it's kind of a random idea I had. I look on their issue tracker, and they're interested in integrating with, well, I mean, they didn't use the SCI framework specifically, but they're interested in integrating carbon emissions into the tool, so that might be a good collaboration to make, and I would be happy to help contribute to that in whatever way I can because I've been working with OpenCost quite a bit, so yeah, my name, yeah, Brandon Hybe on GitHub if you want to collaborate on that. That's a great idea, let's talk after. I'd like to make one call out from the previous question also. I think that as customers, we do have some say in putting, you know, these questions out there for suppliers to ask about, you know, sustainability initiatives and what matters to us as engineers. So I just wanted to make that call out that by, you know, making these questions, these requests and to down the supply chain, it can also be a way to see some progress. That's why it's important. This is again on the lines of like, how do we increase the awareness for this working group and what do we do, right? I heard in the keynote and that's why I came here. It's something that is so fundamental that we all should be worried about it, but we don't have enough conversations. So what would you suggest as some, you know, like conversation starters that you yourself do when you're meeting like people that you don't know or like, you know, just start that. I'm not sure I got the question. Sorry. What are some of the conversation starters to get people interested in this topic? The easy way with companies is you tie it to money because ultimately at the end of the day we care about how much money we're spending in systems. So when, you know, we talk about HPC, a lot of those are run by research labs and they have limited funding every year and they don't want to spend it on power, right? Same thing with data centers. They don't want to be spending their money on power. They want to pass the profits along. So if you tie it to money and that includes the carbon credits that sometimes companies have to purchase to greenwash their actual work, right, then they will get more involved. Awesome. And if that doesn't work, the regulations that are coming can be, you know, fill the gaps where, you know, profit-related initiatives don't, you know, or suggestions don't work. Then also customer demand, that's, so that's, if you can find, if you can tie what you're working on to end user, that obviously would help a lot as well. Awesome. As a follow-up to that one, from end users, are there any end users that are already part of this working group? Yeah, the end user working group is involved to some degree. We do talk to them. So we should continue talking. Awesome. Thank you. And we are out of time, surprisingly. So feel free to catch us after the session. Yeah. Thank you so much.