 Hello, everyone. Welcome to the CNCF webinar on Kubernetes policy exceptions with Kivarno. Before we get started, a little introduction about myself. I'm Anusha Hegde. I'm the Technical Product Manager at Nermata. Nermata is the creator of Kivarno. Previously, I've worked at VMware. I've worked on vSphere SDKs and Tanzu Kubernetes Grid for Edge. I started my OSS journey back in 2020 or early 21 with cluster API. I was also the maintainer of cluster API provider for bring your own host. And recently, I'm also working with the policy working group. This is our agenda for today. I'm sure most of you all know what Kivarno is, but for the benefit of everyone, we'll have a quick overview of Kivarno, what our policies in Kubernetes, what is the need for policy exceptions, and how Kivarno enables policy exceptions. We'll also look at the demo of policy exceptions in Kivarno. And in the end, we'll also talk about what's next for Kivarno and policy exceptions. All right, let's get started. This is the basic architecture for Kivarno. Any Kubernetes API request goes through the API request lifecycle. First, you have the authentication and authorization. And then if any web hooks are registered, the API request is routed through these web hooks before persisting into HCD. There are two types of admission controls in Kubernetes. One is the mutating admission control. The other is validating admission control. Mutating admission control, as the name suggests, has the capability to mutate incoming API requests and validating admission control can allow request to persist into HCD or not. Kivarno acts as both mutating admission control and validating admission controls at the heart of it. It is a root processing engine. Since Kivarno is Kubernetes native, everything in Kivarno is governed by CRDs and controllers. All the yellow boxes you see around here are custom resource definitions. And Kivarno itself is split into multiple controllers that are responsible for different behaviors, such as we have a controller for policy reporting, a control for background scanning, and so on. Kivarno also provides comprehensive CLI to detect misconfigurations in your Kubernetes manifest as early as possible. You can use the CLI in your CI CD pipeline, which can help to detect misconfigurations much earlier in the deployment lifecycle. Before diving into how a Kivarno policy looks like, let's understand what are policies in Kubernetes? Policies in general are a set of rules that define how resources should be configured and behave in a cloud native environment. These policies ensure compliance, security, and efficiency. YAML is pretty much a standard format for writing Kubernetes manifest, and Kivarno also enables you to write policies and rules using YAML. Every policy in Kivarno is composed of one or more rules. Each rule has a match and exclude block. This is to filter on resources on which a certain policy and rule has to be applied. It can match and exclude on any of these. It could be resource kinds, names, labels, annotations, and so on. There are four different types of rules, validate, mutate, generate, and verify images. Validate based on some pattern matching in the rule either evaluates to true or false. Mutate, as the name suggests, is to mutate the incoming request. Generate helps you create resources on the fly based on certain triggers, and verify images is used for verifying image signatures and attestations. This is a sample cluster policy. The validation failure action is set to enforce. It means if the rule evaluation evaluates to false, then this particular request will be dropped and not persisted into HCD. There is another option possible called the audit mode. And next we have one rule in this policy. It is matching on resource kind pod. It follows the validate rule type. This is one of the four that we discussed. This rule checks that an image tag is required, but latest is not allowed. So here we match on the pod spec containers image and we see the exclamation stands for negation. Negation of star in regular expression indicates any image. So any image that ending in colon latest is not allowed. So if your pod tries to pull an image something like a genetics colon latest, that pod will be blocked by the admission control. This is a simple validate rule policy, but writing, mutate, generate, and verify images is also just as simple given as an extensive policy library. This policy library consists of close to 300 policies as of date. So please take a look. Now that we know what given policies are and how they're implemented, you might wonder that under certain special circumstances, you might want to deviate from a certain policy rule. We saw that match exclude block allows filtering of resources, but from this diagram, as you can see, match and exclude is part of the policy itself. Here we match on resource kind pod or we could exclude from certain resources, but we have to define that upfront at the time of writing this policy. Now this might become limiting if the policies are not directly editable. And it is indeed a security best practice to not allow edits to your policies. And oftentimes in organizations, policies are defined by a central security team and policies are authored by this team. And different application teams that are responsible for deployments will need different kinds of exclusions and exclusion for one team may not be applicable to some other team. Hence it becomes difficult to define all the exclusions in that central policy. Policy exceptions in Kivirno addresses all of these concerns because it is applied externally to a policy. This also leads to increased collaboration between teams. It is always a good practice to review a policy exception before deploying and acknowledge the risks involved. And exceptions are generally intended to be short lived. So you can provide temporary exceptions to policies by combining with yet another powerful feature of Kivirno called the cleanup policies, which allows the cleanup of resources in the cluster. And in this case, the policy exception itself is the resource. So how policy exceptions are implemented in Kivirno. Let's look at this flow diagram. Suppose say a validate policy exists in enforce mode and the webhook receives a violated resource without policy exception in place because this is a violating resource and the validate policies in enforce mode. This resource should be blocked. That is, it should not be allowed to persist into it city. But now what if a matching policy exception exists? If it exists, then the resources allowed, even though this is a violating resource. But if the policy exception did not exist, then as expected, this resource is blocked. This feature was introduced in Kivirno 1.9 release, like everything in Kivirno policy exception is a custom resource. It is a names-based custom resource. You have to specify the policy and rule name that meets exceptions. And you can also use match and exclude blocks to filter on resources. Policy exceptions are supported in background scanning as well. The effect is that the policy report now reports as a skip instead of a failure whenever policy exceptions are present. Now let's take a look at the demo. For this demo, I have a local client cluster. I have a single node client cluster that's ready to go. I've also installed Kivirno. I've installed Kivirno 1.9.5 release. Also, it is important to note that in the Kivirno deployment, you have to enable policy exception to true. By default, the setting is set to false. So if you want to use policy exceptions, you have to set this flag to true. Also, another important flag here is the exception namespace. So by default, this is set to Kivirno. It means you have to define all your policy exceptions in this namespace. Even though policy exceptions are defined in this namespace, it is applicable to accept resources in any other namespace. It is generally a good practice to define all your exceptions together in one namespace so that you're aware of what exceptions are presenting in your cluster. Now that we have our environment ready, let's look at our Kubernetes manifest. First, let's look at the policy itself. So here I have a cluster policy. This allow host namespaces. The validation failure action is set to enforce. The rule name is host namespaces. It matches on resource kind called and the rule type is validated. So here it matches on this pattern spec. It says if any of these fields that is host PID, host IPC or host network is present, it should be set to false. Either it should not be present or if present should be set to false and no other value. Let's apply this policy. Policy is created. Let's take a quick look. Yes. Now our disallow host namespaces is ready. Let's look at our deployment here. So here I have a deployment and in the spec, we see that host IPC is set to true. So this is violating the policy rule mentioned here. I have host IPC, but it is not set to false. It is set to true. Now if I try to apply this deployment, Kivarno should block this resource because it is a violating resource. Let's try that. So as you can see, error when creating deployment channel admission webhook has denied the request and why it has denied because these fields, if prisoners be answered or set to false, but we have a host IPC set to true. I get all the deployments. We see that it is not created. It is blocked by the admission controller. For folks new to Kivarno, you might wonder that we have a rule called autogen host namespaces. And also another point to note is in my cluster policy, my match resource kind is spot whereas the resources for deployment. Kivarno is intelligent enough to determine the higher order controllers for pod. So generally you don't create a pod yourself. It is usually via a higher order controller, like a deployment or a stateful set and so on. So Kivarno has the ability to generate rules for these higher order controllers, even though the policy states for matching the resource kind pod. So the rule name here was host namespaces and Kivarno automatically generates rules for higher order controllers like deployment and the rule name is called autogen host namespaces. And that is what we see here. Autogen host namespace, this rule has failed. So autogen host namespaces is applicable for deployment. So even though the policy was for a pod and the resources and deployment, the rule is still applicable. And now we'll see how policy exceptions can help us to bypass this rule and allow this deployment to be created. So let's look at our policy exception. Like everything in Kivarno is a custom resource, policy exception is also a custom resource called delta exception. The policy exception itself is defined in the Kivarno namespace. And now we'll see for what policies and rules this exception is applied. We have this exceptions block wherein we specify that this allow host namespaces is my policy name and for what rules host namespaces is what we originally defined in our policy and autogen host namespaces is what Kivarno generated for higher order controllers. It is by choice that Kivarno does not autogenrate policy exception for higher order controllers as well. It is because it is always a good practice to limit the scope of your policy exception. So you have to always explicitly state for what rules you want the exception for. And similarly in match resource kind, you don't just say pod here, you also say for which higher order controller you want this rule to be accepted. And then we say in this namespace delta and also the resource name starting with the important tool. Let's apply this exception. But this exception is created. We can take a look. So we have delta exception in the Kivarno namespace, which is ready to go. Now if I apply my deployment again, I should be able to do so because we already have a policy exception in place. Yes, our deployment got created. This is because even though the deployment spec was a violating resource, because of this policy exception is present, it allows you to bypass that violation and the deployment gets created. That's it for the demo. Thank you. Coming back to our presentation. So what we have learned today, we learned about policy exceptions and how they provide an external way to specify exclusions to policies and rules. While powerful, their use has to be carefully reviewed and limited in scope because we saw how it can admit violating resources. Combined with other features such as cleanup policies to ensure security while still being flexible. Get involved in our project. So we are on GitHub at Kivarno slash Kivarno. So if you like our project, do give us a star. We're on Kubernetes Slack and CNCF Slack on the channel called Kivarno. We have a thriving community of over 2000 people. We have a weekly contributor meeting and also the quick first issues are listed on the GitHub project. And these are some of the references I used in today's webinar. Thank you.