 Okay. Hello everyone. Good afternoon and thank you for coming this session. I assume I'm probably on the one who's holding you from ending the day. Anybody have a session after? Perfect. So my name is Ariel. I work today in Cisco in the emerging technology innovation team. I was part of the Portshift. Portrait was acquired by Cisco. We're developing cloud-native security tools for Kubernetes for Istio. And now we're doing it in Cisco. Before that I was the head of the serverless security in Aqua and even before that I work in Checkpoint. Working in some open-source contributions, contributor and maintainer in Kube Clarity. We try to encourage everyone to test and use something which is very useful for whoever's using Kubernetes in production. It's handling all types of software issues from vulnerabilities to signing, S-bomb and everything so you can check and validate it in your clusters. It's a very interesting approach at giving it an accurate snapshot. Never mind the topic of this talk. Also working in the CNCF in different top projects and some contributes to the Mitra attack metrics for containers. Now before I start, just a quick poll with not a lot of people. How many of you here in the audience handle deployment of containers on a daily basis? Raise your hand. Perfect. Thank you. How many of you are hardening the settings of the containers which you deploy? A smaller part. Have you heard of used pod security profiles? Yes. Okay. Perfect. Great. So you're in the right place. So a little bit about the basics of pod security. So we all know what pods are. Each pod has a security context that controls the runtime settings of the pod. Now those settings are very powerful. We all scan our code for vulnerabilities and we really get worried even if we have like a medium or a high vulnerability that's completely irrelevant to the code that we write. But sometimes we don't really look at the security settings. Security settings are defining what privileges the pod has, what capabilities it extracts from the operating system, what volumes it can mount, what access it can have to the file system of the host. So those are really powerful settings. Now the defaults are very permissive. Okay. But if you can, but they can be modified, they can harden and a hardening them can really eliminate, you know, a lot of security pitfalls. It can really eliminate the need to use external tools or governance just by using it correctly. You can improve significantly the level of your clusters. Now PSP or pod security profiles. So we're already designed to control those, the security context of these pods. And the topic of this talk is to talk about, a little bit about it, what it is, why they're deprecated, what they are, what can be done instead of it and how we can really enhance and automate the entire thing. So another little bit look of how does the PSP structure was supposed to be something which I took from the AWS EKS environment. So pod security starts with policy. As you can see in the policy, you can define the spec. You can see this is like an example of a very permissive spec. You can see the containers can run as privilege and it allow privilege escalation. It can use any capabilities from the operating system. It can use it can mount any volume. And even if you look all the way down to the bottom, it can, it doesn't allow, I mean it can do right, has right access to the file system of the host. Now in order to apply those policies, you need to take one extra step and you need to create a role. Okay. And in this role, you need to, you know, define the policies and use them and then create a role binding and then you are set and you're ready to go with the PSP. So PSP was a great initiative. If you want to take a look at an example, so this is an example of the MitreTech MitreTech framework for containers. So every cell here represents a real attack, which was detected in the wild, meaning it's a real attack that happened and can be identified. And by, you know, using the security context or hardening configured, you know, correctly, you can eliminate, you know, 30, 40% of those attacks just by, you know, doing simple hardening, simple automation, you can really bring, you know, reduce dramatically that tech surface of your cluster. So the potential is great. The impact is big. It's really useful. And then one of the questions that you want to ask yourself, if this is really true, then why PSP is deprecated? Okay. Why? We know that it's in Kubernetes 1.21, 1.21, they started the deprecation, probably completely removed from the code in 1.25. And the question is why? If it's such a huge benefit. And I think this is a classical case of great intent, but I would say a challenging implementation. You know, when you really use PSP, and I don't know if many of you have had a good positive experience or not, it creates some challenges. So for example, if you're adding a PSP, why the cluster is active, it's already running and you want to add PSP, you can result, you know, without paying attention to the policies, you can mistakenly like, you know, deny all pods entry and you're going to stop like deploying any pods in your cluster. There is no auditing. So if there is a failure because of the PSP, you don't know that, you know that there is a failure, but you don't know why, what caused this failure, which of course makes it much harder to debug. When you create a PSP policy and you think, okay, great, I'm done, but you didn't update the authorization, the role, you know, the role, you know, the role-based access control, then you didn't do anything. So I think, you know, it's really great intent, you know, it was really a good thing to do, but the implementation was a bit challenging. So yes, very sorry, PSP is deprecated, but no worries, there is something instead. So instead of PSP, there's a new security model. It uses similar concept, but slightly different implementation. So the new model simplified the governance of the security context. It creates three default pod security standards that enable three, you know, different levels of pod deployment. We'll touch in a minute, watch each level provide. But I think the most important thing is that users can customize them. Those are just recommendations. You can, of course, go ahead and customize it and create something which really, you know, tune to what you need, what you think is needed, what you think is important. The enforcement is much simpler instead of using role-based access control, using admission controller. And this gives you, as I'll show you, really high level of flexibility and really a very good level of, you know, enhancements. And, you know, it follows the basic Kubernetes policies model that you define. The more the policies are decoupled from enforcement, you define policies in one place, you enforce them with other things. And this is a great way forward. So yeah, we have enhancements. Perfect. Let's talk a little bit about, you know, the standards. So the standards are accumulative, privilege. Those are the defaults, right? You can, of course, as I said, create your own and customize it, something that fits your needs. But the privilege is all the default settings. Everything is allowed. Go ahead, break everything, do whatever you want. Let's party. The second one is baseline, which is minimal restrictions, pretty much. Don't run as privileged. Don't run as a root. No privilege escalation. And the upper one is restricted, which is like implementing the security best practices. Sorry. So starting by Kubernetes version 1 to 22, we're having those three standards which define, you know, the security settings of port deployments. And those policies are cumulative and they start from very highly permissive into highly restrictive. Now, again, if you ask me, even when you do some things in Kubernetes, you still need some marketing, because calling restricted to the policy which really provides you the security best practices is probably not the right way to do it. If you would ask my recommendation, I would say, use by default the restricted environment. I don't think it's heavily restricted. It pretty much gives you, it's follow all the security best practices of, you know, what capabilities you want the port to inherit from the operating system, what volumes you can mount. It doesn't try to just privilege, not running as a root, and not allow you to escalate privileges, which I don't think is a bad thing. It recommended, you know, read only writes the host file system. So in general, my recommendation is, you know, use the restricted profile. Then if you have exceptions, use the exceptions. Again, I think that when nobody wants to be restricted, so I think probably would use a better name for that. But never mind. That's what it is. I think it's definitely a good recommended, you know, profile to use. Okay. Next, let's talk about admission controllers. How many people have experience with admission controllers? Great. Okay. So what is admission controller? Just like, you know, in a very short and brief way, any API request in Kubernetes to the API server has certain stages. It passes before it reached the API server and being executed, right? So like, you know, things like, you know, authentication and schema validation, many, sure, I can do it. Those are important steps. But I think another very important critical step is to use the admission controller. Admission controller allow you to using a web book and allow you to really pretty much validate or modify at the request based on pretty fine policies or pretty fine ones. So for example, you can create a validation web book that any, you know, poll that is asked to be deployed. If it, for example, have critical vulnerabilities, you know, it will not be able to, you might be able to deploy it because every time that, you know, we want to execute a build command, we will check the validation web book, can check and see if it meets the set of policies. So using admission controller is a great way. Using it for security can really provide you an agentless approach. You don't need to deploy agents, you don't deploy site-side containers. You can really use admission controller and govern your entire state of your cluster. Great. Now two quick things. Validate, you know, admission controller has two modes, has the mutating web book, okay, which allow you to really mutate and fix, update, change, modify the request. And it has the validating web book that you can just check if it, yes, no, if it meets or doesn't meet, you know, the need. You can use both, you can use just one, okay, but the idea that if there is an API request that doesn't reject it by any of those controllers, then the entire request is rejected and is not moved to the API server and consequently is not being executed in your clusters. It's great. Now, if you do want to use admission controllers, you can create your own, you can create your own customized resource, but you can use what the CNCF provides. So there are two projects in CNCF. One is the open policy agent, one is Kiverno. Each one of them has an admission controller that you can use for your purposes. So the open policy agent, those are not familiar, so it's just, you know, small introduction. It's lightweight general purpose policy engine. You can write policies using declarative language or regu, okay. And then you can use the gatekeeper as admission controller to enforce those policies. So it's a mature project. It's graduated four years, tons of stars. It can be used. You can see here an example of a policy in regu that eliminate or prevent containers from running as privileged. Of course, you can take it one level deeper and say, okay, it's not running as privileged. Just use one of those standards, either the default or the one which I customize. Okay. So that's a great way to make enforcement to the security state that you want. Kiverno is more interesting. I'm not saying more interesting, I would maybe be caveat. It's another alternative to use declarative policies. Here the policies are not written in regu. It's more like a CRDs. So, you know, it could be like, you know, very simple to ever write CRDs to relay to those policies. And here in Kiverno, you also have an admission controller that can mutate and validate requests. It's a slightly younger project. It's still in stand box. 2.5,000 stars. It's a good start. Okay, great. So we talked about the new model. We know what the security standards are, what admission controller is, what admission controller options we have. Let's take a deep look in implementation. So when we really want to say the model is a great initiative, definitely an accelerate adoption of security hardening. But there are a few items that would love to hear what you think about. I think can be slightly enhanced if you really want to reach the good level of security. One thing is that the current model is restricted to namespace level implementation. So implement a security standard on a full namespace. Okay. And the same security standard applies to the entire namespace. Ideally, you probably would want to have some granularity level, right? You would like to apply a bit different standards to different pods, so you can have like each service or based on the service risk, apply or use different standards. So this is, you know, one option. Another thing which in my opinion can be slightly challenged is the fact that it only uses the validating web book. So we have a binary decision, either yes or no. Is it good or is it bad? If it's good, fine, we can move ahead. If it's bad, we reject it and we go back to give the users, go back and go and fix the failure. And here, one other thing that thought can make it slightly more interesting is two small enhancements. So for example, we can use a hybrid mode of admission controllers, right? So if using just the validating web book, we need to remember that admission controller also has the mutating web book. And if we use both options, for example, instead of just making this decision, which is yes, no, is it okay, is it bad? We can also add the option of fix it. So for example, let's assume that you want to apply a certain standard as a basic. Okay, saying this is, that's the basic security setting. We want all the pods to run in our cluster. Now, if you're using the mutating web book, if by mistake, someone, you know, make a deployment, you know, without remembering what you think is the best security method to use, or just because you use a help chart or use something that you found intended or just downloaded from any repository, then your mutating web book can not just tell you, okay, sorry, you failed, but can fix it for him because this is what you think should be done, right? So the mutating web book give us the capabilities, not just giving you a yes, no question, are you compatible or not, but also modifying it and changing it based on what you think should be the right result. So this is, I think, in my opinion, and a good enhancement to the current model, you know, we'll have to hear what you think about it. We think it's useful. Another thing that can help us to do more customization or more granularity in the usage of the standards is using something like a pod ID, something like if you're familiar with Spiffy, or if you're familiar with Kubernetes labels, okay? And then when you create, you know, a policy to enforce the standard, don't enforce it on a namespace, but only enforce it on your selected approach. So this way, if you have a service which is more sensitive, you can apply to him a more, you know, restricted security standard. If you have something which is no less sensitive, less important, more closely couple, you can enforce a more flexible and more permissive security standard. So having those capabilities, granularity allow you to get, you know, better fine-tuned policies and again, achieve your results of being secure without restricting a thing that doesn't have to be restricted. So those are, I think, two enhancements that we can, you know, enhance the current model. I want to have just, you know, one word about automation. So we all love automate and we really want to automate and then you ask yourself, okay, but how does what you show here with the granularity and everything can do with automation? So here just sharing something, you know, from our user experience, and again, we'd love to hear what you think about it, is a lot of times, you know, we want developers to own their response, the security responsibility of their service, right? You create a new service, you know the best, what will be the best way or what will be the best settings for it, how you want it to run, you know, what privileges you need, what capabilities are required. And here, you know, using GitOps, in this case, you know, we use the Argo CD, but doesn't necessarily have to be Argo, you can use it, you know, with any GitOps tool, then you can ask your developers when they're committing a code, right? Committing a code which will then go to deployment also to create the security settings or to create the policy that this service should run with. And then, okay, when you create this rule, okay, the defined will be the desired profile that would be, you know, for this service, you can upload it to the Git. Now, when using GitOps, right, the policy rules will be automatically added to your admission, to your engine or to your policy engine which runs in the cluster when you sync your, you know, your Git. So when the Argo CD, for example, sync your Git repo, it will not just push the update of the new code, but it will also push the new policy rule. So in this way, you can really allow developers for them, you know, to go ahead and when they commit code, also commit the security setting of it. And when they're using GitOps, it automatically will apply. No one needs to go and make the fine tuning or the specific policy if you do want to make, you don't have to, but if you do want to make specific policy for this deployment model. And this is something that, again, that can really provide a very good way for you to automate the security and use your developers to pretty much define what's required for them, and you can use it back in your deployment. So if I want to summarize everything, hardening pod runtime configuration has now a new and a friendlier model. So I really encourage, you know, everyone, you know, to go ahead and use it. The pod security standards and the admission controller replace the PSP and the RBAC. So the model is much better, is more suitable for Kubernetes. It's much more friendlier and easier to use. It's highly customized. So you can both customize the standards to meet your needs. You can use not just the recommended or the set of validating admission controller. Each of those options, whether using Keverno or using a gatekeeper or you have your own, you can also have the ability to not just validate, but also to mutate. And this unit can really save you a lot of time. And if you're using declarative policy engine and a building admission controller, you can really automate the entire model, as I showed, you know, using GitOps and something that can really save you the burden. Shiftless leftover developers allow them to have a more precise definition. Thank you very much. If you have anything, feedback, comments, would love to hear from you. If you have any questions, we have some time for questions, I believe. Yes. Okay, so I'll just repeat the question. So the question was, if you're using GitOps and using automatic policy, the Git stopped being the source of truth. And I think not because in the Git, you have the policy, you have what you define or the state that you want to run. Now, it's reflected in the cluster. Okay, so the only way is just like, you know, instead of, you know, having the burden of doing it afterwards, you can do it when you make the deployment. And still, the policy is in the gate with the code, and you have this single source of truth. Now, if I understand you correctly, what you're saying is that that's correct. But in one gate, I have part and in another gate repo, I have a different part, right. But I think, again, you're correctly distributed, but you don't lose the gate as your source of truth. You just have the relevant part there. What you can do is, you know, try to define to yourself, at least for the machine controller, what the machine controller is being used with any Git. So you can pretty much restrict it and have a more distributed approach or at least adapt your deployment to that. But yes. But again, I think you still maintain your concept of Git. Okay, okay, okay, okay, understood. Okay, understood. So you're saying what happened if the policy that I have is overrun by another policy, which is the cluster, and I gave the mutating web book to the, to the, to change and modify that. Okay, that's a good point. I think this is maybe a case that you write, and you can use just a validating web book to avoid this scenario. I think the mutating web will give you some kind of flexibility because many of the mistakes they see are mistakes that are not dying deliberately. I think because someone forgot or didn't know or just use, you know, something that you found in a Helmcharp repo because you want to write by himself. And then if you would know, you probably would do it. And in this case, mutating web book can fix it. You're correct. If you want to keep the GitOps more, they need to just use the validating web book. But I think again, it will save you a lot of time to change back. Yeah. Absolutely. I think it's a good comment. Anyone else? So if not, thank you very much. I think I save you five minutes.