 Welcome to the webinar on securely provisioning cloud resources using Kivarno and Frostline. Before we get started, just a quick blurb about me. I'm Ritesh Patel, VP Products and co-founder at Nirmata. Nirmata is a company behind a popular open-source policy engine called Kivarno and Nirmata provides policy and management and governance solutions around Kivarno. In terms of my background, I have around 20 plus years experience in enterprise software, I've worked in Motorola, Nortel, Trapeze, Brocade and also have an MBA from Berkeley as well as a master's in computer science from Michigan State University. So let's jump in. So as you all know, Kubernetes has become that de facto orchestrator for containers and there's a lot of surveys talking about the adoption of Kubernetes. The CNCF survey mentions about 96% of enterprises have either started using Kubernetes or are evaluating it and what's also happening as Kubernetes becomes ubiquitous is that enterprises are building platforms on Kubernetes. So platform engineering has one of the top trends as indicated by Gartner where platform teams are leveraging Kubernetes as that orchestration layer along with other components, could be open source, could be proprietary components to build internal developer platforms and then developers in those organizations are consuming the internal platform and this really helps developers focus on their applications, focus on their workflows and not really have to deal with the operations and the nitty-gritty details of keeping the infrastructure up and running, configuring Kubernetes securely, things of that nature. So that's really what's a key trend as we see it. There's a lot of literature, a lot of content out there on why platform engineering the benefits but we won't get into that. We'll focus more on the topic on hand. So as we know and as we've heard, Kubernetes is really effective, really helpful in managing applications, containers at scale but it's also very complex to secure and scale the platform itself. There's a lot of different surveys that have shown that this increase in security issues as 93% of users have reported at least one security incident, a lot of it is related to misconfigurations because Kubernetes uses a declarative way of defining applications as well as the infrastructure and any misconfigurations could potentially open up an attack vector. So that's really why Kubernetes has become very difficult to scale Kubernetes, especially as the demand grows and as the infrastructure itself grows. So Kubernetes becomes more complex to secure as one scales the infrastructure. Another challenge or another issue is as misconfigurations are prevalent in Kubernetes and one of the top issues, the cost of finding these misconfigurations tends to be high if these misconfigurations are found late in your development cycle. There's research that shows that misconfigurations are orders of magnitude costly to fix in production instead of finding them early in your coding or testing cycle and that itself is a huge challenge and something that enterprises are looking to address. So in the next few slides, we'll talk about how some of these issues can be solved using a combination of open source projects like cross-plane and Q1. So cross-plane is an open source project. It's a Kubernetes native provisioning engine for cloud resources. So it allows you to provision cloud resources or it could be other applications. It's actually very extensible and enables infrastructure as code. So basically, it defines a set of custom resource definitions that these controllers monitor and then you can use Kubernetes native YAML to describe the resources and once the YAML is applied to the cluster, the controllers will provision those resources based on the configuration described in the YAML. The benefit of this approach is that you can leverage full power of Kubernetes like policies, namespaces, real role-based access controls and so on. It also allows you to, it can actually monitor state of your resources and it can reconcile those resources from the cloud. So that gives you a view into the health or the status of your resources. Another benefit of using cross-plane is creating abstraction. So you may have, if you want to deploy multiple resources, you can create an abstraction to deploy multiple cloud resources with a single signal abstracted YAML. And now because this is done in a Kubernetes native manner, it also allows for security and compliance to be enforced at the API level, at the Kubernetes level without exposing that complexity to developers. So it really enables that self-service approach that enterprises are looking for. And the community, the cross-plane community has, you know, providers for popular clouds like AWS, UCP, Azure, so on. So for enabling developer self-service using cross-plane, kind of the way typically it's done is by creating compositions, cross-plane compositions to abstract all infrastructure requirements that are needed to provision a particular, you know, service or a particular kind of resource in the cloud, right? So for example, if you want to provision, you know, a cluster in the cloud, you may need, you know, to provision virtual VPN, private networks, maybe you need to provision subnet, security groups, all of that can be complexity, can be abstracted away from the developer and the developer can just focus on, you know, providing some basic information for the resource and all of the details can be handled through composition. So that makes it really help useful for enabling developer self-service for cloud resources. So next, let's look at Kivarno. So Kivarno is a policy enforcement, policy engine, and it can enforce policies in Kubernetes clusters. So Kivarno is a CNCF incubating project and it runs as an admission controller in cluster. So Kivarno policies are written as Kubernetes native YAML and can be stored or, you know, in Git or can be applied directly. And, you know, when these policies are applied, depending on how the policies are configured, Kivarno can validate, mutate, or generate configurations based on policies. You can also, these policies typically are used to prevent misconfigurations to ensure, you know, compliance with the enterprise standards or you know, industry standards. And these policies can also be used to automate developer workflows. Several examples in the community of policies that can do automation, for example, when a new namespace is created, you know, you can generate other resources that may be needed for that namespace. In the community, there's tons of examples of policies, you know, right from part security policies to workload security, RBAC, multi-tenancy, etc. So next, let's look at the Kivarno architecture. So Kivarno is a, it runs as an admission controller. So it, you know, when it is deployed, webhooks are configured, you know, validating and mutating webhooks are configured so that Kivarno can receive every request that's made to the API server. And then Kivarno has a collection of controllers which then act on the API, on that request, depending on the policy. So, you know, there's a controller that can apply policies to that, to the API request. There's a background controller which can keep checking if any of the policies have been you know, violated, you know, by any of the existing resources. So there's a lot of kind of checks that Kivarno does to ensure that the, all the resources are meet the policy requirements. And if they don't, then Kivarno reports policy violations in the policy reports. You can also specify exceptions, policy exceptions, that's a new capability that was added to Kivarno recently. In case you want a certain resources, certain namespaces to be kind of, you know, excluded from these policies, policy exceptions can be used. But overall, the architecture is designed to handle the load that, you know, in very large clusters. So it's highly distributed, highly available, the separate controllers performing separate tasks. So if one controller, you know, has issues, it doesn't impact other controllers. And then the common reporting layer provides the ability to kind of get the reports in your cluster and then throw other tools, you know, make those reports available outside the cluster as well. So as you saw, both Kivarno and Crossplane are Kubernetes native tools. And one Crossplane is for infrastructure as code for provisioning resources, Kivarno is for policy as code. And there's several benefits of using Kubernetes native approaches in your platform, you know, for developer self-service, for, you know, automation, and so on, right? So one of the benefits is obviously they are, since they are Kubernetes native, they're using, they let the Kubernetes API machinery, a lot of the resources are defined as using CRDs. So as a Kubernetes user, the learning curve is very low, right? So you can use familiar tools, you can use Kubekartl, you can use Helm, you can use GitOps to deploy some of these YAMLs. Also, out of the box is integration with external projects. For example, there's endpoints for collecting Prometheus metrics. There is ability to use, you know, Argo CD for deploying some of these policies or other cloud resources. And then, you know, both Kivarno and Crossplane project have huge communities, as well as an ecosystem that helps if you need any help, you know, or run into any issues. So community can help with, you know, issues. Community also builds, you know, tools, adapters, providers, extensions around these core projects. And then there are commercial companies supporting these projects, like Nirmata for Kivarno, like a bond for Crossplane that can also help if needed, right? So there's a lot of benefits to using Kubernetes native approach when building developer platforms. So now let's focus on the use case that we were, we initially started the discussion with, right? So self-service cloud provisioning. So we'll have a demo of this use case a little bit later, but before that, let's look at how through using Crossplane developer can provision a cloud resource, right? So here I have a management cluster where Crossplane is installed. It's a Kubernetes cluster. You know, I can see the API server as well as the database. And then there's on the right, you have your, you know, cloud services. In this case, we look at AWS. And then the developer, let's say in this case, developer wants to create an S3 bucket. So the developer would specify the YAML to create the S3 bucket. We'll see what it looks like, but it's essentially a custom resource definition in the cluster configured by, created by Crossplane controllers. And then when the developer creates the S3 bucket, you know, the Crossplane controller and the AWS provider that's part of the Crossplane, one of the Crossplane components will actually detect the creation of the custom resource and it will trigger the creation of S3 bucket in the, using the AWS API, right? So that's kind of pretty straightforward. The developer applies the YAML and the controller, the Crossplane, the provider actually picks it up and creates the S3 bucket. It actually even, you know, periodically synchronizes with the configuration in the cloud and the developer will, you know, by looking at the custom resource, the developer would know if the bucket has been created or not, right? In case of any errors, the developer would know that, you know, would see the error in the CLI. So next, now the challenge with this is obviously, you know, any configuration, any cloud configuration has a lot of different knobs, a lot of different configuration parameters that have to be, you know, approved, checked, verified to ensure that these, you know, the configuration is not insecure. For example, the S3 bucket being created is not public. The S3 bucket, you know, information or any operation is being logged. There's a whole bunch of things that a platform engineer may want to do or there may be security requirements, you know, to be enforced. So for that, now this is where Kivorno would come in and help with enforcing some of these, or setting some of these guardrails as well as enforcing some policies, right? So in this case, now the kind of the model changes a little bit. You have a cluster, management cluster with crossplane and Kivorno both installed. And then the platform engineer can create these policies beforehand. We add these policies into the cluster, and then the Kivorno controllers will start monitoring the API server requests. And anytime any resources created that match the policies, it will apply the policies and allow or deny those requests depending on how the policies are configured, right? So once the policies are configured, now the developer workflow remains the same. But before the crossplane controller can act on those custom resources, in this case, creating the S3 bucket, Kivorno can intercept and apply the policies. And then if the policies, all the policies are successful, successfully applied, and there are no issues, then the request is completed, and then the crossplane controller can actually start processing the request and creating the S3 bucket. So that's kind of the flow. It's by adding these policies, now you have been able to set the guardrails and enforce various checks as needed in the developer workflow. And all of this without having developer having to learn anything, any policy violations will actually be available to the developer directly. They don't have to learn or know about Kivorno at all. So let's jump into the demo. So I have a cluster that is already configured. And it has Kivorno, it has crossplane already deployed. So before we start actually verifying the policies and seeing how it works, let's look at the policies and the configuration, right? So I have here, I have a set of policies for S3. And these policies are written in YAML, they operate on this bucket resource, which is a custom resource defined by the AWS provider, the crossplane AWS provider. And there's various checks, this policy checks for version, this policy checks if the S3 bucket is configured to access for public access. We have a check here for region, another check here for if notification is enabled, and then we have a check for logging. And then I have two different, you know, S3 buckets and I'll try to create one in USEs too, and another one in US West one. And we'll see how these, whether these get created and what happens, right? So let's go back to the command line. And I'm in the S3 policies repo to apply these policies, just going to apply all the policies in my folder. So it'll take a couple of seconds for these policies to be applied. Once the policies are applied, you can actually see these policies and make sure that they are ready. So here, all of these policies are ready. And you can see which some of them are in enforced mode, others are in audit mode. So now let me go to the resource folder and let me try to apply the policy that or the YAML to create S3 bucket in the East region. Actually, before I do that, let me show you what's running in the crossplane namespace. So here you can see the crossplane system namespace. We have the crossplane components and then we have a provider for the AWS cloud. And and if I do get CRDs and grab on crossplane, you will see all the CRDs that are installed in my cluster and a lot of all of these are most of these are related to the AWS provider, right? So these are created by AWS provider, these map to AWS resources. So for example, the one of the custom resource that is available here is the bucket resource. So we'll be creating that resource in the AWS in our AWS account. So let's let's go ahead and do that. All right. So this so when I try to create this resource, I get some violations, right? So because I'm trying to create this resource in US East, I have a policy which prevents me from doing that, which is this policy. And then there are some other policy violations that are also that are also triggered because my bucket is not configured correctly, right? The configuration for my bucket does not meet the policy requirements. So that's kind of how, you know, immediately developer gets feedback that, you know, they cannot configure this S3 bucket. So let's let's now try to configure the other S3 bucket in the US West region, right? So let's see if this works. So yeah, in this case, the S3 bucket got created. And if I do, if I check buckets, and I should see that my S3 bucket, the class frame S3 bucket is created. So maybe I just to verify, I can go to the AWS account and in my account, I can see my demo West S3 demo West bucket as created. So that kind of shows you now with this, the S3 bucket got created, it had, it pretty much had all the configuration that was correct. We can actually see if there are any policy violations. And in this case, we actually see that four of these passed, but then there was one audit violation which failed, which was around the email notifications being enabled. So it failed and it actually, but since it was set in audit mode, we could go ahead and create the S3 bucket, right? So if you enforce, policies in enforce mode will prevent the creation of the bucket. But if it's an audit mode, the resource gets created, but then you have a policy violation. And you can actually look at the policy violation in more detail by just using this command. So it tells you that this message tells you what happened and why it failed. Great, now the other aspect about using crossplain is you can also use crossplain to actually discover or find resources from your cloud account, right? So crossplain has the ability to adopt resources and you can do that. And in that case, and you can still apply policies, even though those resources are already created in your cloud account, you can apply policies and make sure that there are no policy violations, right? So what we'll do here is we'll go to our, we'll go to the elastic ECS account and just check in our ECS account. We have some task definitions and we'll adopt one of those task definitions and try and make sure that we are able to validate it and in case it has any, and check if it has any issues, right? So we have this ECS task definition created in my ECS account. So in order to do that, let's go back to the ECS folder and what we'll first do is we will apply these policies. So I have, so let's go look at the policies first. So here in the ECS account, for ECS, we have two policies to check if the image is coming from specific registry. And then we also have policy to check the task definition and to make sure that the CPU and memory are, you know, configured within the specified limits, right? So, and I know I've purposely kept the limit low so that we can, some policy violations can be triggered. So first, let's apply these policies. So if I do, I'll get C pull. I should see that I have now two policies and both of them are in audit mode because here we just want to detect we are not going to enforce anything because the resources are already created. So let's go to the resource folder. And here I'm going to just import the task, the task definition. So before we do that, let's look at the task definition YAML. So this is a very basic YAML. It just specifies the name of the task, the region, and only the required fields, right? So here we are going to just specify the required fields. And because it has this annotation called crossplane external name, it's actually going to just pull down the configuration of the task and add it in the custom resource. And also we've created configured the deletion policy is orphan, which means when this task definition is deleted from the cluster, it won't actually delete the class definition from the cloud. And this, this can be configured either way. But in this case, I've just done it to make sure that the original task definition is not deleted. So Google create minus F import task definition. So now let's look at the object itself. All right, so we have this task definition, and we can actually get more details. If you go to the YAML format, it's still synchronizing and it's getting additional information from the cluster. So this could take a few minutes. So let's maybe check again. Okay, so now we see all of the configuration for the task has been pulled down. There's a lot more information here. And so now let's see if there are any policy violations. All right, so we see two, we see one failure and one couple of errors. But in case of this failure, let's look at the failure. And this failure, this policy violation is for image check. So because the image for the task is not coming from the registries that we would like, this violation is triggered. And it says all the image images for this task definition should come from an authorized repository. So this is a way to actually apply policies, even to existing resources by using crossplane and Kverno, crossplane will actually synchronize those resources and then Kverno will apply those policies. So just to summarize, as you saw in the demo, you can use Kverno along with crossplane to enable secure self-service for developers. You can set the necessary guardrails when developers are provisioning cloud resources using crossplane. Kverno can provide the necessary policy enforcement, audit compliance for your cloud resources. And then the benefit of doing this proactively while provisioning these resources is you can pretty much prevent misconfigurations. You can overall improve security posture. And by preventing these misconfigurations and not having to deal with them later on in your production environments, you effectively save costs as well. So that's kind of the overall benefit. And this in the end leads to a much better developer experience without any back and forth without any friction that may typically be involved in such scenarios. So that's it for this webinar. Please give us some feedback, reach out to us if you have questions and would love to hear from you if this was helpful or if there are any other topics that you would like to hear from us. Thank you.