 All right. My name is Ian Coldwater. I am co-chair of Kubernetes SIG security. This is SIG security growing together our maintainer track talk. If you are in here for a different talk, you are in the wrong room. And that's cool. We're happy to hang out with you anyway. What is SIG security? SIG security is a Kubernetes SIG that takes a community building approach to improving Kubernetes security based on the principle of people coming together and consent. We work to improve security for the project itself as well as our end users by cultivating a space where security-minded contributors of all experience levels can get together, bring their energy and ideas, and work together across SIGs and internally to help make Kubernetes more secure. A lot of our work is done in our subprojects. We've got four of them right now. We've got SIG security docs, which works on improving the documentation content and creating new content so that people can get good guidance on how to use Kubernetes securely. We've got SIG security tooling, which develops and maintains security features for internal Kubernetes development. We've got security self-assessments, which empowers projects under the Kubernetes banner to improve their own security by teaching them how to threat model for themselves. And we've got SIG security audit, which is responsible for the periodic third-party external audits of Kubernetes security that happened every once in a while. We'll be hearing from all of these subprojects in a minute. All of these things help keep Kubernetes security improving over time. So first off, we're going to be hearing from SIG security tooling. Hi, everyone. Hi, Mai. So I'm very glad to say that I've been involved in SIG security for a few years now. And today I will be representing SIG security tooling. Thank you. So SIG security tooling improves Kubernetes security by writing codes, mainly, and we maintain mostly two projects. So the first one is the auto-refreshing CVE feed. So this is like an official list of Kubernetes CVE from the website for Kubernetes. And those have been very helpful to help people maintaining cluster to keep them secured in the past. So now it's in beta stage. It's going to graduate into stable. And the next project is like the scanning of the Kubernetes release artifact. So basically, we scan the Kubernetes container image and the binaries to find if there are any known issues that we in them. So I wanted to mention that the SIG security tooling is meeting on a bi-weekly basis. Usually we alternate between a working session, working on the two projects I just mentioned, and learning sessions. So learning sessions, usually we invite people to speak about the tools that matters for them. And you can yourself, sorry, request a learning session by opening an issue on the SIG security repository. And you have a bunch of existing learning sessions that you can watch the replay for. So the SIG security tooling now is looking for people. So these are a few image on the left side. You have like the scanning, the pro job for the scanning of the container image. And on the right side, the official CV feed thingy. So we are looking for volunteers like in the next month will be, we will have like some exciting news coming. So I think it's the right time to join. So if you don't need any particular experience, you can join. Everyone is welcome. And I would say the easier thing to join is like reach the SIG security and SIG security tooling Slack channel. We will be announcing a shadow program. So you will be able to learn under like SIG security tooling lead and try with us. So now to SIG security self assessments. Thank you. So next up is SIG security self assessments. SIG security self assessments came from some new contributor coming and seeing a need, which was projects under the Kubernetes banner, such as cluster API. Often aren't in scope of the external security audit, although you'll be hearing more about this in a minute and also need their stuff audited or need to be able to better understand their own security posture or threat models in order to be able to submit to an audit in the future and have a better posture and a better idea of where they're at. So what SIG security self assessment does is empower those projects under the Kubernetes banner to be able to do those threat models and assess their security posture themselves. We take people who are experienced in doing threat modeling and partner with people who maybe are less experienced and then we all work through it together and everybody therefore can figure out how to do it on their own. This model is based off of tag securities previous self assessments program, which was for projects under the CNC F manner, but projects under Kubernetes, such as cluster API are actually out of scope for tag security. So we brought that in and started doing that for ourselves. SIG security self assessments meets every other Tuesday. And if you have a project that you are maintaining that you want to participate in the self assessment process for, we have a way to do that. You can submit an issue on GitHub. Currently, the self assessment that we have going on is at CD with brand new SIG at CD. They are working with us to assess their own threat model and security posture. Past assessments have include cluster API and the vSphere CSI driver. Again, if you have any projects that you're maintaining that you want to help that you want participating in this process, come holler at us, come to a meeting, come to our Slack. We'd love to help. Next up, we've got SIG security audit. I'm Tabitha Sable. I'm the other co-chair and I am delighted to get to brag about what my friends in the SIG security third party audit sub project have been up to. As you might guess from the name, the third party audit sub project focuses on running the process to do periodic third party professional security audits of the Kubernetes code base. A wonderful thing about Kubernetes is that within our broad dev community, there are a fair number of people who have a lot of interest in security. There is, you know, the possibility to come and talk to SIG security folks and others and to improve that knowledge that folks have on their own. And in order to help to ensure that that is continuing to improve things over time, we get a second opinion from outside. The most recent audit report was published last spring in 2023. And so the sort of post learning phase of that audit is wrapping up. Many of those results have been fixed. Others are in the planning stage. And so now it's the time to start thinking about the next audit. So the planning for the next audit is beginning right now and this is the perfect time to become involved with that. So right now the folks from the audit sub project are working on rescoping the audits to be able to do more frequent, smaller, more focused audits so that the firms can on board to the scope of what they're doing more rapidly and so that we can get more of the parts of the project through the process more quickly. And so as part of that, they are happy to work with folks who might be interested in contributing to the audit process by, you know, working with the vendors and so on. And also seeking Kubernetes sub projects that want to be included in the scope of the next audit. So if you have a particular feature area or sub project in Kubernetes, or if you would like to participate in this upcoming audit process, you can reach out to the security third party audit sub project lead who is Ray Lejano. Name is spelled here on the slide on Kubernetes Slack or attend one of their public meetings. Public meetings are currently being rescheduled to be more accessible to folks in European time zones. So if you join the mailing list, which will be linked at the end, you will get the invite for the new time when it comes out. So I'll now hand off to hear about docs. So I'm Rory and I help out in the SIG security docs project. We've got a couple of areas that we try to help out with around Kubernetes. The first one is that we help with looking at what documentation there is on the Kubernetes website and how it is racked up for security. And we look at areas where perhaps guidance might get outdated and things that need to be updated. I mean, one of the things anyone who uses Kubernetes will have come across is that things change. New features come along, old features go away. And it's really important that the docs stay up to date so that people are getting the right kind of information. We also do another thing around this, which is we notice areas where something could use focus or could use extra documentation. I'll talk a little bit about one of those later on in the presentation, but where people might think I would like to know more about this and maybe other people would too, we help people come along and actually develop that into some kind of document. There's a couple of areas that we are looking at at the moment. We are looking currently at the hardening guide. So we originally started this off as a single project and then realized hardening Kubernetes is a really large topic. No one person is going to write this entire thing. So we broke it down. And at the moment we have got the authentication section completed. That's up on the docs side. And we have two in review. So they've been drafted and they're being reviewed. And actually I'd like to thank Ashish and Anshuman who have been working on that because it is kind of quite a big undertaking and something that we, you know, it's difficult to get it all done. We're also doing a couple of updates on the website at the moment. And Tim Bannister, who's in SIGDocs as well, has been doing a lot of work on that. So if it's something you're interested in, then that's definitely something we can get involved in. We have some stuff looking forward. The hardening guide has more sections. If you have a specific interest in a part of Kubernetes security and you think, I know a lot about this, I would like to write it down. Come and help. There's lots of sections to do. Alternatively, if you think, you know, I don't know a lot about this topic, but I would like to. This is a great opportunity for you to learn more because there's lots of people in the community you can talk to. If you start drafting something like this, who will help you and give you information and point you in the right direction? Personally, from my own experience, writing docs is one of the best ways to learn about something because you will start writing it and you will think, oh, I don't know this as well as I thought I did. But writing the documentation 100% gives you the opportunity to learn that. So if you're interested in learning more about security, genuinely, I recommend docs as a place to come. We are also looking at security checklist for developers. We have one for administrators, so this is like a kind of short checklist. People can go through quickly and get some ideas of things to do, but obviously we have a lot of developers who work in Kubernetes as well. And so the next step we want is to say, let's have a checklist that's targeted that audience. So again, if that is something that you're interested in, something maybe you've had experience of, you know, as a developer, you thought, hang on, I don't really know what to do with this thing. It would be great if there were some official Kubernetes docs that told me that this is a great opportunity. In terms of the meetings, we have meetings monthly at the moment, first Thursday of the month at 2pm Eastern. So if you'd like to come along, that's definitely a good place to do it. We also have a Slack channel for security docs, and that is where they're 24x7. If you have a question about docs or you see something on the website and think, hang on, that's not quite right, and it's security related, please do come and tell us. And we're happy to try and help and see if we can fix that. So next up, sick chair stories. So one of the things we wanted to focus on in this talk besides report backs from sub projects is talking about who we are as sick security, what we do and why we do what we do, because that's something that we don't always talk about explicitly. So sick security started out as the third party audit working group for the first audit several years ago now. There was a working group that got together and for those who are less familiar with Kubernetes governance structures, working groups are temporary. They are supposed to be there to do one thing and then disband. The thing is that we actually needed, as it turns out, more than one security audit over time, and so that was going to need to be a thing that would need to continue. And so having a single working group that disbanded and then starting the same working group again didn't necessarily make a whole lot of sense. And then I think the folks who were involved in that working group were like, and also we don't really have a clearing house for security within Kubernetes. Historically there was no sick security. I think the idea was that security was everybody's responsibility, which it is, but when there were new contributors who wanted to get involved in improving security or folks who were security minded who wanted to get together, there wasn't really a spot for that because Cygoth had things that they were doing specifically around off. So it was figured that what needed to happen was that there needed to be a SIG that would be a more static thing than a temporary working group that would also provide a clearing house and a space for security minded folks of all experience levels to be able to get together and come up with their ideas for improving Kubernetes security and help make them happen. So working group audit became SIG security. And when that happened, we also figured that there needed to be a community focus because it wasn't just about the work. It's also about the people because what is the work at the end of the day, if not the people. So we provide a place for those people to get together. And so when we started doing that, we were like, okay, how can we intentionally create a space cultivating so that it is as community minded as possible and as welcoming as possible? How can we create the most inclusive space that is welcoming to contributors of all experience levels from the newbies noob to the grayest gray beard? How can we get everyone who is interested in doing this in one place get together, make them all feel welcome and help them make things happen together? So we figured we would make these processes and put them in place and everything SIG security done is done on purpose. In every meeting, we have everybody introduce ourselves every time so that we're not scaring away the one new person who shows up if only one new person shows up by being like, okay, hi new person introduce yourself and then not introduce ourselves because that's not very welcoming. We wanted to make sure that everybody knew that they were welcome and that there was a place for them. So we intentionally make sure to have good first issues from contributors. We are really proud of bringing a lot of new contributors in and helping them grow as leaders. SIG security doesn't have a membership roster. SIG security is made of whoever shows up. If you show up to a meeting, you are a member of SIG security and if you have an idea, we can help you make that happen. Or we can help make that happen with you. We have had examples of new contributors who have come to SIG security meetings before never having been to a Kubernetes meeting with an idea and say, you know what? I want to lead this project and now their sub project leads or becoming shadow chairs or things like that. You can just come do that here because we are designed as a place on purpose where anybody can come and make it happen if they have an idea. And another thing that we do is as well as welcome new contributors as well as experienced ones, provide a place for people to gather, bring their ideas and energy. We also do a lot of collaboration across SIGs because the fact is that Kubernetes is a big project with a lot of code in it and a lot of areas. So if somebody has an idea that reflects somebody else's code base, which is often the case because SIG security mostly doesn't host code outside of tooling, then we help facilitate those SIGs getting together and figuring out how to make that work happen together. My co-chair Tavi here has a really good story about that. Yeah, I'll be happy to share that story. So the story here is the story of the deprecation and removal of pod security policy and the creation of pod security admission. And this story starts back shortly after the earth cooled in the early days of Kubernetes. There was clearly going to be a need for some sort of policy enforcement within Kubernetes and pod security policy was created. It was very flexible, it was very powerful and its API depended on the details of the API of the rest of Kubernetes. So for example, when new features were added to the pod spec, new features needed to be added to pod security policy to go along with them. That aged approximately the way all of the software engineers in this room would guess that it would. So by, you know, the 116, 117 timeframe, it was pretty clear to the maintainers of PSP in SIG auth that there was going to need to be a replacement for pod security policy. This was very difficult to settle upon. So for several years, they would periodically take a stab at asking around, you know, what, what shall we do to replace pod security policy and meeting with difficulty. You know, there were a lot of conversations and there were very few conclusions. There was an interesting talk I think in KubeCon in San Diego from some folks from SIG auth talking about difficulties in pod security policy. So our involvement in this started with the last of those attempts. One of the maintainers, Tim Allclear, said, now is the time we are going to find a solution to this. And he advertised pretty widely at this particular meeting of SIG auth, we are going to, we are going to begin a process that will end with PSP being replaced. And he talked about it on Twitter, you know, contacted people in his personal network and so on. And so a lot of folks who were interested came to that SIG auth meeting. And there was quite a lot of the sort of discussion that had happened in the past. And at some point someone opened her big mouth and said, we can solve this. We will write a proposal that will be suitable for replacing pod security policy. That's not how it went. But it turns out that it was a very good start because with that we were able to start to lead that by example. So Ian and I with input from a lot of other folks and our own backgrounds running Kubernetes in production as end users started to dream about what would it look like if it were suitable for. Our needs and our understandings. And so we started working, you know, in our secret little Google doc and in, you know, Slack chats and whatever on what our replacement would look like. And also separately Tim and several of his colleagues began working on a proposal in the same sort of way. And we kept in touch periodically. And because there was so much community interest about this. It also became a recurring topic in both the SIG off and SIG security meetings. They were on alternate weeks. So if you wanted to keep talking about PSP, you just come to the next meeting on the next week. After a few of those rounds, we had a proposal and Tim, Tim and his colleagues had a proposal and we brought them together to a dedicated meeting for this project. We invited everyone to look at them. We talked about what the benefits of each were, what the shortcomings of each were, and it became clear that the right choice was neither of them. But because each of those was specific enough that we could have implemented it, it was specific enough that folks could praise them or criticize them in really actionable ways. And so those discussions led to a pretty quick consensus around a different plan that combined the strengths of both of the other previous community-driven plans. And that is the one that became the cap. And so by the time the cap process for pod security admission began, we were arguing about the details about exactly how implementing PSA would work best, you know, with the backwards compatibility concerns and all the various things. But the fundamental agreement about what the shape of it should be like had been already built. So we were able to set an example, gather the community and combine our experience. That is a nice lead into one of the first things that Mahe was working on. So I'll hand it to him. Thank you. So, yeah, I'm going to tell you the story about this thing. So basically I started working as a security researcher and learned Kubernetes as Rory mentioned through writing docs basically. I wanted to learn more, just started writing docs and I was helped by all the sick docs people and all the sick security people. So I saw this blog post by Tabi and it was exactly about what she mentioned, the deprecation and removal of the pod security policy. So I got interested in that and I was thinking like maybe I should try to learn about how this thing came to be because I arrived in the project very late and I didn't really know. So I did some archaeology and ended up like writing this blog post. Many people from sick security and sick docs encouraged me to write this thing and like from the whole research, I discovered this very obscure admission plugin thingy. So apparently it was added like at the same time as the pod security policy and this thing is called security context deny or CSD9 in short. And the idea of that was like to restrict the number of fields of the pod security context. So clearly like these days, this thing was clearly obsolete and I was thinking maybe we should either update it or either remove it or duplicate it. So with the help of sick security again, I finally opened like this issue, deprecation and removal of the security context deny admission plugin. And it was quite a ride because like I had to talk to many people, actually many people got involved and I ended up with a kind of a list of item on what to do. So if you feel lost, like it's okay, people will give you directions and it will be fine. So like actually a few months ago, we finally merged like the last thingy, the actual removal of the admission called the plugin. And yeah, now I love to say that I mostly had a very negative impact on Kubernetes because I mostly removed code. Yeah. So now up to Rory. Awesome. So yeah, and actually just to finish off my story, if any of you have to use the CIS benchmark, thanks to his work, we have just removed that from the CIS benchmark. So it won't turn up in your compliance checklist anymore once the next version comes out. I had a very quick story about something we did in security docs to add in to some of the stories we've been telling. And it was around a problem that we identified, or not problem, an opportunity we identified to improve people's understanding of admission controller risks. So many people, in fact, I think probably most people who are using Kubernetes in production will be using admission controllers for security. And they're great and they can really improve the security of your environments. But also they are quite powerful. And there are risks that come with adding new powerful tools into your cluster. And what we decided to do as a group was essentially try to brainstorm that idea, essentially work through it and say, what are all the things that we could think of ways in which this could go wrong? And how could someone at a high level mitigate them? And this turned into the threat model you can see up there. Essentially we created a threat model and we created a document which goes with that, which goes into some more detail about how you can use those and how you can fix some of those issues. From talking to people, both users, but also people who write admission controller software, they found that quite useful because it was a neutral view of potential risks and something they could work to and look at their project and say, how are we addressing all of these things? So it was kind of an opportunity to say, if you got something where perhaps you've been through a problem in your environment that you think, hey, this is more generally applicable. We ran into this security problem and you would like to make that a wider story. SIGDOX security is an opportunity for you to do that. It's not just something which has to be a page on the website. We also do things like this, which are addressing specific problems people identify. The other one, even if you've not been all the way through it and come up to a solution, but if you have something where you've encountered and think, I don't think this is the guidance on this is clear. There's something in security that I'm trying to do or some challenge I'm facing in using Kubernetes securely come along to us and give us the idea. And if you want to run that, we will help you do that. If you would like maybe to give it up to community to say, does anyone else know what you could do here? We can do that as well, but there's a wide variety of things. And the more voices we get involved in this, the more things we're going to find. We don't know all of the things that are causing people problems. If people come and tell us, then we've got a better idea of where to kind of focus and put our efforts. And with that, she'll hand over to Ian. So we've been trying to tell you these stories in an effort to emphasize that everybody has something to teach and something to learn. Everybody's perspective and everybody's input is valuable. You are brand new and you are reading the docs and you're like, that's confusing. I don't get it. What does that even mean? That is valuable input because we are not serving you well by giving you information that's confusing. If you are really experienced and you're like, you know what? I don't think this one goes deep enough. I really want to like bring my technical expertise into this. Awesome. We love that. Everybody, every single one of you has something to contribute here. So as I said earlier, SIG Security is made up of who shows up and the ideas and energy that they bring. So we would love to see all of you bring your ideas and your energy to SIG Security. You can come join a meeting. We have them every other Thursday at 9 a.m. Pacific, which it's daylight savings right now. So do the math because it's wrong next week and we would love to see you there. We just had one this Thursday, so the next one will be in two weeks. And whatever it is, if you have an idea of something in Kubernetes that could be improved in some way, if you have an idea of something that seems really interesting or weird that you want to learn about, something that you want to make happen, a really cool hack you heard about that you want to share, whatever it is, we would love to see you. So come on out. We have SIG Security and Kubernetes Slack if you want to come meet us on Slack or just come and show up. To get calendar invites for our meetings, you can join the mailing list with that lovely QR code up there. And yeah, we would love to have you. Thank you all so much for coming. We really appreciate you being here. And one last thing. Thank you all so much. And we actually have a couple of minutes for questions, I think, if anybody has any of them. Hi, I'm Shweta. My question is, sorry if I missed it because I entered room a little late. Where do I see what is already released by this group? Well, one place where you can see what sort of artifacts are committed, you could go to K slash SIG Security on GitHub, you know, so github.com slash Kubernetes slash SIG dash security. And there's some of our process documentation. There are some of the reports that have come out of work been done by the by some of the sub projects to to review some of what has been going on. There are previous SIG security maintainer track talks from various Kube cons. And we take live meeting notes together as a group. And so you can also just look and see, you know, what have been the topics on the last few, you know, on the last few sessions. The document there is linked from the read me. So if you go to github.com slash Kubernetes slash SIG dash security, then you can get the links to all those things. Or you just joined Slack and say, you know, what's going on and where's the links? Looks like opportunity to consolidate it. Yeah, will be in that. Because SIG security is what's called a horizontal SIG, we don't own code as much as many of the other SIGs do. We are more of a place for people to get together and and make things happen in collaboration with other SIGs who do own code. There is not as much code living in our repo as there might be other SIGs because a lot of our work is done horizontally across those SIGs. But we do make a lot of things happen. So we'd love to see you sometime show. Thank you. Anybody else have any other questions in the great room for more. Do you have any direct connections with tag security or how much do you all work together? We sure do. One of our sub project leads for tooling Pushkar who is not here because visas is the co-chair of tag security. So we have a lot of overlap. We have done projects in collaboration with one another and inspired by one another. Our self-assessments project was inspired by tag security similar self-assessments projects and we do a lot of work in collaboration with one another. And with the recent change, I mean maybe a year or so ago for tags to be called tags, we are no longer so often mistaken for one another, but we are still in fact friends. Literally. So there were two SIG securities. There was CNCF SIG security and Kubernetes SIG security, which is why historically they had better SEO than we did. So historically we have very strongly insisted on being called Kubernetes SIG security to differentiate the two. It is no longer entirely necessary. We just keep doing it. It just feels good now. But that did actually get changed because of us. But yeah, they're great. They're good people. They have a fantastic raccoon logo. And you know hash collisions are always a problem. Anything else. All right. Thank you all so much for coming. We really appreciate you taking time out of your day today. It's great to see you. Cheers.