 Okay, let's get started. Hello, everyone. Thank you very much for joining us for today's CNCF webinar, Admission Controllers, one part of your Kubernetes security and governance toolkit. I'm Jerry Fallon and I will be moderating today's webinar. And we would like to welcome our presenters today, who John Patel, Cloud Architect at Palo Alto Networks and Robert Haynes, Cloud Security Evangelist at Palo Alto Networks. Just a few housekeeping items before we get started. During the webinar, you are not able to talk as an attendee. There is a Q&A box at the bottom of your screen, so please feel free to drop your questions in there and we'll get to as many as we can at the end. This is an official webinar of the CNCF and as such is subject to the CNCF Code of Conduct. So please do not add anything to the chat or questions that would be in violation of the Code of Conduct. Please be respectful of your fellow participants and presenters. And please also note that the recording and slides will be posted later today to the CNCF webinar page at cncf.io slash webinars. With that, I'll hand it over to Robert and Gujian for today's presentation. Thanks very much, Gerald. Good morning, everybody. And thanks for joining us today. So just quickly, a quick intro, I'll do myself and then I'll let Gujian say a couple of words. Robert Haynes, I work as a Cloud Security Evangelist at Palo Alto Networks. My role is to get out there and talk about all things to do with security, especially cloud and cloud native technology security. Gujian, tell them about yourself. Hey, everyone, my name is Kanjan. I'm a senior cloud architect here at Palo Alto Networks. And I have been working on Kubernetes and containers for the last five years as a developer and architect at many open source projects, including Kubernetes and CNI. So just if anyone's in any doubt, he's the smart one of the pairing this morning. So as we said, this is what we're going to talk about today. We're going to try and take about 45, 50 minutes, maybe slightly less to go through this. We'll have a bit of time for Q&A at the end, depending on how many tangents I get up on to during our presentation this morning. I'm going to give a real quick, short security overview of my view of Kubernetes security from like 10,000 feet. Kanjan's going to go into the details of OPA and the Rigo language. Then we're going to take a look at some practical admission control policies, things you should think about doing and that you can use admission control for. And then we've got some examples and a demo to go through at the end. So I'm a fairly high level, I take a fairly high level view of things about what the attack vectors are going to be in any kind of environment. And when you look at Kubernetes, there's really kind of like four or five main ways that someone's going to try and compromise or misuse a Kubernetes cluster based on the architecture of Kubernetes itself. At the very beginning, you've got to think about the container. We're not going to go into a ton about container security today. That's a whole other webinar about what goes in there. But if you think about what's inside a container, it's got code, it's got libraries that you've included. So software supply chain attacks you need to think about. Then you need to think about how the containers are running. Is it running as a privileged container? Is it mounting the cost file system? Is it sharing network space? What can it connect to? Is it storing secrets in there? All these things are important container security because that runs on the node. And on the node also runs things like console. So there's a great story that someone posted some medium article actually. And it's like, how did crypto miners end up on our production Kubernetes clusters? And essentially, they had a workload monitoring console. This wasn't the main Kubernetes clusters with third party console, been left open to the outside world, didn't have any authentication and could run arbitrary commands on running containers because of how it was set up. So that was a great vector for someone to come in and relatively easy compromise a container. And also because the container was running with privileges and with file system mounts, they were able to then compromise the node as well. You know, everything that happens in coincide of Kubernetes, every configuration change goes via the API server. So protecting your API server is important against misuse and against denial of service. And obviously, SCD is again, a kind of key component and you can compromise, so file walling it, you can take it down. So protecting it from DDoS is important. And that kind of covers the real big chunks of Kubernetes vulnerabilities. So we obviously need to be able to work out how we're gonna mitigate those. And there's all this technology which we don't have time to talk about in one webinar, but they're all relatively important things that you should be thinking about. You know, cloud native firewalls, scanning your container images. There's tons of tools out there for looking at what's inside a container image and comparing it to CVEs. Runtime defense, you know, container, cloud native firewalls, application layer firewalls. How do we manage consoles? Again, cloud style firewalls, access control into there, filing your node, having runtime defense on your node. Your node is still a Linux VM or a Linux physical box, depending on where you're running it. So you need to think about how you protect that actual node running in itself as if it was just a standard server you weren't running Kubernetes on. You need to think about that. And you need to think about admission control in that space. API, misuse, there's RBAC in Kubernetes. RBAC everywhere is always a, and you know, infrastructure, I am in general, is kind of always a complicated tack, you know, NCD, firewall, encrypt, limit access. Today we're going to talk a lot about admission controllers. Those are the kind of key things that we want to discuss today because they have a really powerful role to play in essentially limiting the effects of any kind of compromise. I think, again, a fairly simplistic view of these things. I always think of them as like a firewall or an application like a firewall for your control plane in Kubernetes. So the admission control process lets you control what events, well, not what events hit your API, but what your API does with those events when they come in. So, you know, we'll go into that in way more detail in a few minutes. So, this is where the delicate ballet act of me handing over the screen to Jan happens because he knows all about OPA or Open Policy Agent in far more detail than I do. So he's the right person to talk through it. So if you're ready, I'm going to stop sharing and you can start sharing. All right, thanks Robert. No worries. I will also turn on my camera, so it looks like there's a few people. All right, so thanks Robert for that introduction about different security attack vectors and what are some of the mitigation strategies. Now, as the webinar title says, we're going to look at admission controller and as one part of that security toolkit and specific, we'll look at Open Policy Agent, which is a CNCF open source project. So what happens typically is you have your client which is typically either a cube cuddle or a helm or tariff from what are used to create Kubernetes objects. That request goes to the API server and then API server will make it happen. There's like an action associated with it with an admission controller or OPPA in this case. There's, we're adding an additional step. When the request gets to the API server, there's a query sent to OPPA and it will check against all the policies that you've created to see if this action is allowed. If this action on this resource is allowed by this user at this time, whatever you want. And it will give back a decision and then based on that, there's a yes or no on the action. So let's try to zoom into this a little bit. So how does an object creation work in Kubernetes? So here we'll try to look at it through an animation. So here's you sitting with a smiley face and you're using kubectl with your kubeconfig file to create a Kubernetes pod. So you type kubectl create and then you're creating a pod, right? That request goes to Kubernetes API server and these are all the, everything you see here in blue is the control plane components in Kubernetes. That is stored that object you're trying to create, it's stored in at city database. And then you get a success back. If there was an issue with your schema or if you didn't have the proper authentication, you'd get an error right here. The scheduler and controller manager are controllers that are always watching for new objects. They'll watch and notice this change. Scheduler will quickly assign a node to this pod and say this node seems to have some extra space. So schedule it here. So we'll look at that node, what happens on that node? You have all these data plane components that are present on every node, right? You have your kubelet, you have your CNI, Docker daemon or CRI daemon and kubetroxy. So what happens here is kubelet notices that it has a pod that is supposed to schedule. So kubelet will coordinate the pod creation. First, it will create a pod network namespace and notify Docker daemon or CRI daemon to create containers inside it, one or more containers. Then it will send an add command to CNI. CNI is container network interface that goes depending on which CNI you're using. It will create a network interface for the pod, assign an IP address to make sure this pod can talk to all the other pods in the cluster. And then kubelet will say its success, the pod has been scheduled. Kubetroxy notices that change and it will add that pod as an endpoint if there's a service using IPvS or IP tables. So this is a very high level overview of how an object is created. Now in this process, let's try to zoom into step one, two and three from when you create a Kubernetes object and then you get a success back. Which is number three. Inside the API server, there's a whole bunch of things that are going on. First thing is the HEP API handler. When you do a kubectl create, it's handling your API request. The next thing it goes through is authentication and authorization. It will make sure that you are who you say you are. That's, you know, that you're providing through kubectl here and authorization to show that you have the right credentials to do, sorry, you're authorized to make this change to work on this operation. That's something you define in Kubernetes RBAC, for example. After you pass that, there's mutating admission webhooks where you can have something where you want to insert something or mutate the object before it gets stored onto HCD. After that, there's schema validation to make sure that the schema you have specified is, sorry, the object you're creating is the correct schema for the API version you are using. This is where all your typos and wrong YAML syntax will be caught and rejected. After the schema. Yeah, Robert is very famous for restyping things. After that, there's admission, validating admission webhooks. Step one dot six and one dot seven. This is where OPPA comes in. After your object schema is validated, it's sent to OPPA or some external admission controller. It will review it and then return the response back. And if it says, okay, then it's stored onto HCD and then you get a success back in KubeKurl. So let's take a look at, let's zoom into that here. Step one point six and one point seven. What happens from your object to the admission controller, especially in case of OPPA? So you have your Kubernetes object YAML file and then through KubeKurl you're specifying what you're trying to do. Let's say create or update list some operation on the object. Within the API server, there is a request form called admission review request where the Kubernetes object is embedded into that request and operation that you're trying to perform using KubeKurl or whatever tool you're using. That's also embedded in that admission review request. That goes to your, that goes to OPPA in this case as a query in JSON format. OPPA is checking against all the policies you have created in advance that's using Rego policy language that we'll cover in a bit. And then it will see, it will check all the policies to see if this exact admission review request, this object and this operation is allowed to be performed. And then it will send back a decision as admission review response object back to the API server. And based on the response to get either stored onto a city or user gets an error saying you're not allowed to do this right now. So kind of a stupid analogy that I like to use for this process is, you know, when you go to nightclubs, I know we're not supposed to go there right now because of all this pandemic, but there's usually a bouncer who has a list of all the guests and there's a line every time a new person comes in, the bouncer will check their name against that list they have. So you can think of OPPA as that bouncer and all the Rego policies as that list and all the objects and everything you create, update, delete in Kubernetes. All those objects and operations are the people standing in line to be admitted to the nightclub. So anyway, that's my stupid analogy. So let's quickly take a look at the Rego language that we're using to create these policies. So some language basics, there are variables. You can assign some value to the variables using this syntax and these are not like, you know, you don't have to pre-define X is an integer type. You can just say X equals 42 and it will create and assign that value if it already doesn't exist to X. It can be a string or Boolean or a composite data type. Data type. Equality you can equate X equals equals 42 that returns a Boolean if it's true. Not equals, you can compare to strings, compare a variable with a number greater than equal to or greater than less than. There's also arrays in Rego where you have multiple objects and they can be of any type in an array. And looking up things is similar. You can have in an array, you can look it up by a specific index or you can look it up by a specific key in a key value pair. In this case, object foo that's looking up the foo index, whatever the value is there. Then if you have a nested object, which will most likely be the case, if you're using Rego for admission controller, you can use the dot notation to go down the hierarchy. We'll see that one in a bit more detail. It also comes with a bunch of built-in functions like starts with to check if a particular string starts with this substring, ends with contains, trim, split, and count. This is so you don't have to perform a lot of string operations manually. Iteration is slightly different from other programming languages if you're used to them. Let's say you have X equals an array of a couple of elements, right? You can say X of index, which is zero, one, two. Or if you wanna iterate through them and assign all those values, you can say Y equals, sorry, Y colon equals X in square bracket underscore. That means it will assign all those values A, B, and C to Y. You can also do a traditional for loop using the keyword sum, sum of I. And X of I, it will go through A, B, and C, zero, one, and two indexes. And double for loop sum of I and J. Now let's take a look at how a Kubernetes object will look like as an admission review object, right? You have a very simple Kubernetes object YAML file. This is a pod, name is MyApp, and container image is NGINX, and the name is NGINX frontend. So when it gets to OPPA, it will look like this, admission review JSON object. You have a larger field called input, and then you have admission review object. Then there's a request kind for pod type version and then in the object field, you have metadata which has name, spec, image, and notice this is a square bracket here and here. That means it's an array. And you can go, there's a sample rego playground link. By the way, if you're not familiar with it, the rego playground, it's pretty cool. You can evaluate the policies you write against different objects before you create and apply those to your admission controller. I was just about to jump in and say the same. The rego playground is super useful if you're trying to develop policies, depending on what environments you have to play with. I learned a ton just by talking most, messing around with that, totally worth it. And since we might have a little bit of time, let's click through it. Just wanna quickly show you how it works, right? There are three main sections here, input. You have your rego policy here and output. So in this predefined page, I have a sample object that's already there, right? It's a pod, admission review, and then there is a pod. And it has a label environment equals not dev dash 42. What this policy is saying here is check to make sure if make sure that the object that's being created has a label environment that starts with the dev. If we evaluate this, it says it's supposed to start with dev dash and not this, whatever you have. So what it's doing here is input here, which is this whole thing. Dot request, this is the request. Object, which is here, metadata, labels, and environment line 11 here. So it's assigning its value to the variable value. And if the value doesn't start, not start with dev dash, then it's false, right? It's going to return false for this predefined deny statement. That's why deny function, right? Let's get back to this one. So here's another sample, rego policy, where you have a function called match. And you can return a message with your allow or deny, whatever you want here. Input.request.operation. Now, operation is also included. I didn't show it in the previous slide, but you can also be very specific to which operation you want to apply this policy to create, update, put, delete, et cetera. If you don't specify that, it applies to all of them. The time you have to make sure this is there is if you're applying policies and you already have a bunch of resources created, you may not be able to delete them afterwards. If you don't have a specified operation. kind.kind is pod, resource is pods, and this one is checking against privileged containers. So input.request object spec containers. Now you will see there's an underscore. That means it will, if you have more than one containers, it will go through them and assign them, evaluate them, each of them one at a time. And security context and privilege. If that is true, then it will return true format with a message saying privileged pods tonight. Now, one gotcha here is init containers. In Kubernetes, pod you can have init containers that don't follow the same structure. And then those ones, they can escape this policy. A lot of people make that mistake. So make sure that is covered. All right, so how does a rego policy look like? You have this higher level policy and allow is a function, right? It's a rule in this policy. Within that rule, you have statements, individual statements that you're evaluating. And if you have multiple of those that return different Boolean values, they get, there's an end operation for them. So input.val equals 42. List zero equals care. If both of them are true, then allow returns a true. Same thing with the other allow. And between those rules, they get odd. So only one of them have to return true for the main allow to be true, right? And you can specify a default value for allow up here, which is false. You should always specify this as a best practice to false or whatever the negative value you want to assign. Now let's take a look at some security best practices for Kubernetes, right? The way I look at it is there's four main areas, integration, touch points for security. One is in the development phase, then you have your CI CD phase, then pre-deployment phase and runtime. So things you can do here is dev and DevOps checks is, you can have some checks built into IDE, get pre-commit checks to follow security, some of the security best practices. In the CI CD pipeline, you can have a whole bunch of IT governance policies. You can also use OPPA here in your CI CD pipeline to validate the Kubernetes manifests you are checking in against some compliance issues or whatnot. Admission checks, this is where the admission control comes in the picture, right? You can bypass CI CD, you can ignore IDE error messages. You can merge the code however you want, right? There's no enforcement point there. With admission controller, it makes sure whoever is creating this resource is following these policies. There is no, you cannot bypass it, right? Even if it's a Kubernetes high-level object creating another pod, it has to go through the same admission review process. This is where you can check for things like signed images, IT approved based images, you're following security best practices or you're making changes that allowed business hours, sorry, not during business hours. And runtime checks, similar policies, you just look for violations and those. So let's take a look at a couple of security best practices where you can use an admission controller for and then Robert will walk through each of them individually. So first one is only running your container from a trusted source. You don't wanna run containers from Docker Hub where anyone can push things. You wanna trusted registry, your company's registry. Not allowing dev latest master image tags, right? A lot of times you have your CI CD pipeline set up where every time a developer checks in a new code in let's say dev brands or master brands, which is called main branch now. A Docker image is built in the CI CD and it's pushed against those tags in the repository. So not even by mistake, you want to use those. Same thing with latest. All these tags don't tell you which version you're using, right? So preventing those that's good hygiene, not allowing privileged containers. This is kind of very dangerous. If you're running a privileged container, it has access to the host and all the other containers, file system, which process it's running, all the network calls, everything. So make sure there should be very few exceptions to this. Not mounting the host file system, like mounting ETC var of the host to the pod. That means it can modify ETC and var root of the host. And it's also not a best practice. If your pod dies, gets recreated somewhere else, it doesn't have the same local file system anymore. This one is important. Making sure a container file system is read only. This check alone will get you a very high score and meta-spoil check because your container file system is read only. That means you cannot, if there's anything, there's any malware, it cannot download anything. It cannot execute anything. It's a read-only file system. All you can do is read. And this is another good hygiene thing. Blocking services of Type Note port. You might have developers who just create a lot of, they might bypass and create public-facing services using Note port service. So those are some of them. I have a GitHub repo here, github.com.stats.gov, cloud-native security, where I list a whole bunch of these with risk or assign to them. Not all of them are applicable for admission controller, but you can go through them and see which ones you can apply and what tools you can use. So now I'll hand it over to Robert to go through these use cases in more detail and show us the rego files. Okay, thanks very much. So full disclosure, I am using the OPA implementation and admission control on Palo Alto's Prisma cloud, but it doesn't actually affect how you would run any of this. It's just the easiest way for me to do the demo. So let's just start. So let's have a look. Let's just cat, let's type in the right thing. Okay, here's my real simple piece of Kubernetes YAML, okay? It's just gonna start, all it does is start a pod running Nginx. It's pretty simple. So if I'm lucky, and I can type as we were discussing earlier, see I cannot type, and I cannot type even more in front of an audience. So let's run this. I think this is waking up. All right, okay. So my image, Nginx latest, fails to come from a trusted repository. So let's just take a look at like Gunjan's rules of container hygiene. So this is the rule that we're actually using. It's not actually using hoody.com as the image start, but this is basically enforcing that my image comes from a trusted repository. So I think if I cat, I think now I should clear between each ones. Now I'm pulling my image from my, this is my personal Google Container Image Repository. This could obviously be anywhere, anywhere you like. So maybe this will work for me. So, actually, yeah. Okay, so my image is tagged dev prod or latest, which are not allowed. So let's go back to here. Okay, so this is the rule that's kicking in here. Again, it's just picking up the image. Is the image got latest master or dev as a tag? And if it is, then I'm blocked. The way that we do the blocking in Prisma is anytime that this matches, it's an automatic negative, but you could change this to deny. It's not like a language specific thing. Okay, so I think if I took a three, okay. So I've now used the deploy tag, which is very imaginative of me. So hopefully now I'm using a trusted registry and I'm using a deploy tag. So let's just take a look now. Privilege escalation pod not allowed. Okay, so let's just click here. Okay, so here I'm looking at this thing here for allow privilege escalation. Now, obviously there's other ones we can do like runners route and a few other things you could throw in here, but it keeps them simple. I just picked on allow privilege escalation. So let's have a look. So maybe if I take a look at, okay. So now if you notice before, I had privilege that was equal to true and privilege escalation was true and I had it running as root. You could pick on any one of those things and you should probably pick on all three of these things. So now let's see, we've got trusted registry, deploy tag, no privilege escalation, running as a high number user. So hopefully, let's see if this gets me there. Okay, so this is, I see we're trying to mount the var var directory. Now there's a good argument you shouldn't mount file systems of the host at all. But here I'm just demonstrating that we can mount, we can mount sort of, we can control which mounts might be allowed. Okay, so I'm, you know, if it starts with the ETC, Etsy, if it starts with var, if it's just mount everything, then I'm gonna stop that. There are probably great arguments you could have about whether there's ever a good thing to mount a host file system or not. But here's just a demonstration of how we can do, and you could actually stop it if there was any kind of host path mention of there at all. So let's take a look at, let's look at number five. Okay, so now I'm just mounting the home directory into slash mnt slash home. No, we could say is that good or bad, but I've decided to allow it in my environment. So now I'm using privileged container, trusted registry rather, deploy tag, no privilege escalation and a host path mount I'm hopefully happy with. So let's try that. Let's clear the screen to make it slightly easier to see what's going on. So number five will be the charm. Okay, we've got to have a read only root for file system because in terms of rules of container hygiene, file system read only. So let's have a look down here. This is basically looking for, I get the concierge, oh, up a bit, not read only root file system in there. Okay, you've got to have, not only has it got to be there, it's got to be set to false. So there we go. So if I take a look at, okay, so now I have a trusted registry, deploy tag, file system mounts I'm comfortable with, no escalation and read only file system is set to true. So, and I've been successful. So basically this whole chain of stuff has forced me to apply a set of good hygiene to my pod creation. But there was one other rule, prevent node port services. Essentially, I really want all of my traffic to be going through proper ingress control, low balancer type objects. So I don't really want to create a node port because that potentially exposes my container to the outside world without any of the controls in my environment. So just flip back to here and let's just take a look at, I've imaginatively named this. This is just a real simple, you know, service node port selects for the app, my app listens on port 80, node port of three, three thousand and seven. So, and again, because the rule is kicked in and it's spotted that it's node port, I've been unable to do it. Now these are simple, simple rule sets. You know, you can build multiple, kind of detections in each one. You can make your message as long and as unpleasant as you like. It's easier in some ways, depending on your implementation, I think to keep rules separately rather than trying to combine them into one mega rule because it allows you to create, you know, different rules for different places. It's slightly more portable. I find dealing with large chunks of rule text that have multiple matching statements in it, slightly more complicated. So there we go with that. I personally like to do it this way, that there are definitely other ways you can put these rule sets together. So I think we'll take, actually let's take some questions, because there seem to be a bunch of questions that have come through in this last, this last environment. Probably I won't be to ask that answer that many of them, which is why I brought my cleverer friend with me. Hey, Robert, do you want to switch over to the resources slide? So- Yes, sure, so we can do that. If they want to take a note. This will, and these slides are obviously be available later on anyway, so you can get to these. The policies that we use today were in this GitHub repository, the first thing, there is a subdirectory of the up a rego policies called, I think CNCF webinar, you can look in there. There we have a blog on container security best practices, which is what we've been referencing today. The last two are specifically around how Prisma Cloud have implemented OPA support and how we've done Ingress control, how we've done admission control in there. That is quite specific to Prisma Cloud technologies. With that, what we've got sort of 10 minutes or so left to go to answer some questions. I don't know if how we want to deal with those, but we can take them one at a time or we can take a quick look through and see which is the most relevant to go. I can help facilitate the questions. No problems. Thank you both first of all for a wonderful presentation. As Robert said, we have about 10 minutes or so before we wrap up. So if you have any questions, please feel free to drop them in. We have several here, the first one here. If possible, can we go a bit deeper on how common policy repository, time state analysis and admission control are working together along with the CI CD pipeline that results in production deployment? My answer will be, won't be vendor neutral. I just want to put that there because my familiarity is with Prisma Cloud platform which has agents and components that can integrate into everything from IDE through source code repositories, through build service, through container registries. And then into actual runtime in production with admission control. So yes, we can do that. I think it makes, however you choose to do it whether you use a combination of open source tools and other stuff, it does make a lot of sense to do that. I think that good container hygiene policies, good container hygiene policy. And if you can try and enforce the same set of policies at every sensible step in your build pipeline, that to me makes complete sense. I think that the advantage that, and you should do it, you should make it so that as a developer and I'm using an IDE of choice like IntelliJ or whatever, I should be able to submit my yaml for evaluation at that point before I even commit it because I personally don't like to break builds and make mistakes. So anytime I can get a piece of technology to help me and show me where I'm going wrong, I'm usually fairly grateful, little frustrated sometimes, but fairly grateful. So I think having those checks and having those sanity checks at every possible stage that you can put in that's not, doesn't cause too much friction and trouble is a good idea. The advantage of doing things with admission control is that it's very hard to get around, it's very hard to avoid, it's impossible, well, impossible is the wrong word, it's very difficult to avoid the admission control process. You can try and write your way around the rules, but essentially once you've installed admission control and then open agent into Kubernetes, then every API code goes through it. So you can't fudge the build or anything like that. Okay, this next question here, is it possible to get all the violations at once instead of one at a time? You could certainly write a big rule that looked for every single exception and built a bigger message. But, and which I can see is better in lots of ways, but I have questions about maintainability and how easy it is. It depends who's doing them. You have on this webinar today, myself who actually has a background in CIS admin and security, and my esteemed colleague who's a developer and our brains clearly don't work the same half the time as we've discovered whilst putting this presentation together. And so for me, I would probably put up with extra pain and maybe slight lack of optimization in the responses for ease of maintenance and ease of comprehension. Because I'm not great when shown pages and pages of code, but other people may have, it kind of depends who's maintaining it and who's managing it, I would say. Yeah, I would say you can jam in multiple exceptions, sorry, multiple rule violations into the message. But it's, as Robert said, like I wouldn't jam all of them into the single, into one message because admission review response object only has one string. You can definitely append to that message. So you can have all the evaluations in separate rules, but you have one global variable message where you keep appending bullet points, styles saying violation one, two, three, four, and that's returned at the end. But maintainability point of view, you don't wanna have a ginormous rule that has all those checks in one. You still want them separate. You can just, sorry, prepend to the message in each of those rules. Okay, so our next question here, how do we manage exceptions? There are no exception handling in Drago. There are some cases where you can have error message errors, but those will be compile time errors. Hopefully what you're using for OPPA implementation or the OPPA server itself will reject the policy. If you have, let's say you already have created a variable and try to assign another value to it, it will throw an error message, but there are no exceptions. Can I apply a rule only to a specific namespace? You can check for it in the rego policy. You can say object, whatever, dot namespace equals equals this namespace. And then, because if you remember that slide, all those statements, they get end, logical end. So it will, rest of it will only apply if the namespace equals the specific namespace. Can you do math with rego rules, for example, to force on a container that memory limit? Yeah, if I understand it correctly, yes, you can do that. Memory dot, you can get values from memory dot limit and memory dot request. And then, those are variables. You can get those values, do the math on it and then make a decision based on that. How do you recommend to protect the control plane from attacks? XT API server, et cetera. Robert, you wanna take this? Yeah, so I think there's a, careful, I say I have a slight vendor take on this. But you can, there's a bunch of really good security out there in terms of like, when you can look up best practices that don't involve any third party tools. But essentially, limiting, I think it's important to limit which nodes can access at CD. So there's firewall, there's a firewalls you can put in, third party firewalls if you need to keep things away. There is mutual TLS between, between, you know, your Etsy D and communications and your API. Communication should be potentially be encrypted. We could go on, there are, I did hunt around and I found a number of really well-written blogs on this. So I encourage you to go and check those out rather than listen to me, Babylon. If OPA is outside, how is the context of the environment pass into the input to OPA? So even if it's inside the cluster, well, sorry, let's use that example. If it's outside the cluster in the admission review request, number one, it comes from a specific API server. But OPA doesn't have anything built in to distinguish where it's coming from or evaluate that as part of the policy. One sort of hack I could think of is if you remember that slide where we zoomed into what happens inside the API server, one of the step is mutating admission webhook where you can mutate, you can have a simple mutating admission webhook that just adds a label saying cluster equals cluster name and add that context to all the objects and before it gets to schema validation and ultimately to admission controller. So when the request gets to the admission controller, it has a label environment equals this or cluster name equals this. That's one hack I could think of off the top of my head. Do you recommend using Istio service mesh? Yeah, it depends. But if you're talking about in context of OPA, yes, it depends on your use case, right? Istio service mesh is a whole another topic for another talk, but there's an OPA plugin for Istio to extend your validation or admission controller for Istio objects and authenticate the request. Just about wraps up all the time that we have for today. I wanna thank you, John and Robert for a wonderful presentation and I also wanna thank everyone for taking time out of their day for joining us today. As I said before, today's recording and slides will be posted on the CNCF webinar page at cncf.io. Everyone, thank you again for your time today. Please take care, stay safe and we will see you all next time. Thank you. Thanks a lot.