 All right, thanks everybody for coming. Most of you were probably at the keynote earlier, that if you weren't, there's some points in here that we're going to refer back to here, and we're going to do more of a deep dive on one of the concepts from that. So if you weren't there, my name is Brian Oliver. I'm a principal architect at ThoughtWorks. Some of my colleagues are here, actually. Kavi over there, Nick over here. And then some of the other concepts from this talk are an evolution of things I've worked on with Kingdon sitting here. So it's fun to do conferences over the years because you get to make more and more friends and work on more and more things. So pretty cool. So today we're talking about CBEs primarily and how we want to build a process for mediating them that creates more of a fluid, frictionless process for your development teams and makes security happy. So the first component of this is we're going to talk about CBE scanning. In the typical world, you might have sneak, you might have tenable or something else, and when you build an image, you generate a CBE report based on that image tied to like a git hash or something like that. And this is pretty standard. You can do a sneak, tenable, twist lock, et cetera, deploying that application into production after you've maybe blocked that in the pipeline. So the first problem statement that we want to talk about is a lot of tools like sneak give you the ability to auto update those images inside of your like say deployment files. Like say you have a deployment YAML file or something like that. Flux has an image reconciliation controller that allows you to auto upgrade. What those tools don't have, and also I should mention sneak has an automatic pull request tool that does similar where it can see like, oh, you've got a, you know, maybe in a vulnerable image in your, you know, whatever file, sneak will send you a pull request but that's usually YAML based. So I wrote an open source tool a while back that's been used at multiple enterprises now that will look not at just the actual built images but also take a look at the underlying dependencies. So the Docker file particularly is what we're gonna look at today where it will go and actually find like if you need to upgrade a version of your base image in a Docker file and push that change to your repo. This is really powerful because if your team or company has a policy of building golden images, then this allows you to auto upgrade those for all of your teams across your enterprise. And the way we built this, this is again something I collaborated on with Kingdom a year or two ago, where we create an image policy in Flux and we'll take a look at this, there's a demo, you don't have to remember everything. Get an image policy in Flux that tracks the versions of your base images and then uses that API to auto update the downstream teams Docker files. So to get a better idea of how this looks, we have say a set of image repositories maybe those are a Node, Rust or Java base image repo. We then might have a scanning tool like Sneak, Tenable, whatever. In that tool, we're gonna use those tools auto upgrade features to continuously upgrade our base images. And then as those images are upgraded, we're pushing them into our upstream Docker registry. This allows us to stay on top of vulnerabilities at the microservice repo level. So to give you kind of a more of a higher perspective on this, the way that this works within your sort of Kubernetes environment is we're going through that process of auto updating our base image repos. And then you'll see further there on the, I think that's your left, the microservice repos which are consuming those base images are then receiving updates to those base image versions within their repositories. So if you have like say Node 16 and that's vulnerable and we've just updated the golden image to be Node 17 we're gonna auto push that to that microservice repo. That's then gonna be reconciled through a Flux process like as you've probably seen in some of the talks today and deploy that application into your Kubernetes environment. Now the next thing we wanna do is tie that to what we talked about earlier today in keynote compliance of the point of change. So what I mean by that is not only do we wanna auto update our images but we also want to block deployments that are trying to deploy things that are not vulnerable or that are vulnerable. So not only are we providing a way to give those development teams away to auto upgrade but then we're also going to make security happy by then blocking at the point of change, the cluster. The ability to deploy anything that's vulnerable past maybe a policy. The way we're gonna tackle this is with an open policy agent which we're gonna use as an admission controller. This will interrogate say upstream CVE reports and then either block or allow that deployment based on the image version. So the way this works is we're gonna have a validating admission controller and this admission controller is written in Rego and for our case or for this demo example. And we're going to send the deployment into our Kubernetes API server. We're gonna validate that that version is the version that we want it to be. And just to sort of recap from compliance to the point of change, the way that this works is we're scanning for CVEs in those images and we're then creating CVE reports and then we're blocking those images on the Kubernetes side with the admission controller hooks. And we'll skip that. And then the last part to cover here is when we're referring to that point of change in the case of our Kubernetes. So we're taking the image update process that I talked about at the beginning here and then we're saying, okay, at the point of change, meaning at the Kubernetes boundary, we're then going to allow or deny certain deployments. So let's go to the fun part. And I put a crashing plane on there because demos never work, but you know, we'll see how it goes. So the first part, we're going to auto-update some Docker images. So let's first take a look at, I've got a basic stampo repo here. This is just a Rust application. It's like an echo bot. And in here, we're going to go and revert my testing of the demo earlier. So we're going to put it back to the vulnerable version. We'll just do that from the get UI because that's totally where I write all my code. Not really. We'll push that. Cool. So then now what we're going to do is we're going to flip over to our code here. And we're going to take a look at this. So what we're doing here is we have a Golang program that is talking to our Flux API deployment and we'll flip back to that in a second. I just want to scroll down here. There we go. So we're talking to a Flux CRD to get an image policy. So we're going to take a look at that real quick. So inside of our Kubernetes cluster, we have Flux deployed and we have an image policy here that is deployed there as well. What this image policy is doing is tracking two things. An image repository, which in this case is Rust. So just Docker upstream Rust, basically in our case. And then the policy is we need that version to be within say 1.59 and 2.0 for versus a sample. So what this does is when you go down to the Flux API, you've got a set of CRDs from the Flux API. And what these do, if we get a response back from our cluster, there we go, is give you a set of APIs that you can hit from the QB API server. So in our case, we're going to say, let's get that image policy that I just showed you from the repo and that's this guy here. So this is our image policy and what it is saying is for the Rust image policy, the version that you're supposed to be using right now is version 1.69.0 is the non-vulnerable version of Rust in our example case here. So then what we're going to do is we want to update that Docker image to use the more recent version of Rust or in this case, the version that our team is saying is the compliant version for you to deploy into our environments. So we're going to run that program I was showing you earlier. This is the plain crashing part. There we go, looks like it worked. All right, so we'll flip on over to our repo here. And if we go to sample repo and we take a look at our Docker file, it looks like it did work. So we got history, update file. Yeah, okay, so we've updated that to the non-vulnerable version of our Rust image that we want to use. Now for the purposes of this demo, we're doing that from that go run command that you just saw. I have under my handle a production-ready version of that tool, image-updater bot, that's written in TypeScript, using the same client go library just generated in TypeScript. But literally the same exact API calls. That is containerized and you can give it a config file to push all those updates to your Docker files on a iterative basis like every minute or hour or whatever you want. It's just much faster to show you that from the terminal. And then we started updating that tool to be in Golang. So the fact that we just demoed that on stage is a pretty good sign. All right, so for the second part, we want to talk about compliance at the point of change. And so for that, now that we can auto-update the versions of our images, we've given our developers access to receive those updates on the flyer in real time or very quickly, right? So now that we've given them that, we should be able to enforce some of those security requirements at the boundary of our cluster at this point because now there's almost no excuse. There's no reason why they shouldn't be upgrading those versions, it's being done for them, literally being pushed to their own repos for the brain. So now we can say, okay, our policies on the side of the cluster are, you must use the base image version within that range that we specified. So for that, we're gonna take a look at our open policy agent configs. So inside of that same cluster, we have a certain set of deployments. We're using a tool called Gatekeeper. And let me flip back to our slides for a second. So what this is doing is we're taking a look at our image versions. And in this case, for the example, we're just saying looking at a specific version of REST. And then the Gatekeeper is looking at an OPA policy or a Rego policy, checking that version and they're allowing or denying the deployment based on if it matches the policy from our Flux API. So if we look at Gatekeeper system, we've got a set of Gatekeeper controllers and these are going to be enforcing our audit policies. And then inside of our actual deployment or actually deployed to our cluster, we have a couple of things deployed there. So the first thing that we have is this constraint template. What this is doing is this is giving Gatekeeper a configuration that says these are the allowed images for my pod of deployment. So in this case, what we're doing is, let me move this over to the left here, there we go. A number of things. The first part that we're doing is we're taking the input event of the pod. So if you take a look at this sync.yaml file, when you set up Gatekeeper, you want to tell it to listen to certain events on the cluster so that boundary at the environment of the cluster, there are tons of events hitting that boundary constantly. We only care about certain ones. So in this case, we're listening to service, pod, ingress, namespace. And in the case of the demo that we're gonna look at today, we're primarily gonna look at the pod creation events. But you can do this with just about anything that you create. So if you create a service or if you create a deployment or you create an ingress or a namespace or anything like that, those are all events that Gatekeeper can parse and read through and make changes too, right? So our case, we are looking at this guy. So when we push a deployment to our cluster, the first thing that we're doing is we're taking the input of that event object, which is going through the Kubernetes events API. We're then parsing that object and getting the image URL out of the deployment spec. So if we go to our, like any deployment file basically, let's look at, yeah, same board repo. What we're looking at is this URL here. Like in any deployment file or pod spec, you're always gonna have an image URL, right? That you're adding to. So what we're doing in Gatekeeper is saying, give me the value of this explicit field and then we're gonna do something with it. So in this case, we are then using rego, which is a kind of tricky language to get used to, but as you get used to it, it gets better. We are grabbing the image version by just using a simple split command from that URL because those Docker URLs are always rightmost for that version number. And then what we were doing here is a semver comparison of version to 1.58.1. Now for the purposes of this demo, this part right here, 1.58.1, that is hard coded. What you would replace that with is a call to that flux API that we used earlier to update the image version to get whatever the minimum is. So you might be like, okay, log for shell or log for J came out and that's version 1.17 or 1.16. Well, then we're gonna mark that as, you need to be higher at least than that version. Otherwise you're gonna just completely block the deployment. So in this case, we're saying, okay, we need to be higher than rust version 1.58 and we're gonna apply the comparison to the version that is being trying to be deployed to the cluster. So that's this version variable right here. If that's not satisfied, then we're gonna return a response with this is an invalid image version and we're gonna give them a response that's helpful. So we're also gonna say what versions are allowed in that case. And the way that we do that is through a configuration file for Gatekeeper. So in this case, we're just saying the only version that's allowed is 1.69. In a production case, you might say there's five or six or seven other versions that are acceptable. In our case, we're just gonna show one. But you could easily make this a list or a range or whatever you want. All right, so now let's do some examples. Okay, so first let's make sure we don't have anything deployed. Nope, all right. And then let's do a bad version deployment first. So let's say, yeah, 58.0. I think that's our, yeah, 58.1, sorry. That's our insecure image. All right, so we're gonna apply that. And we're gonna get back a response from Gatekeeper here. So this is what I was talking about with that point of change deployment here. So even though we've gone through our pipelines, processes, we've built our images and all that kind of stuff, once we get to the boundary of the environment, we're like, oh, okay, we're actually gonna decline this deployment because the version that you're supposed to be using in your Docker file is version 1.69 and you're currently using version 1.58. So now we're gonna take a look at bad version YAML, 69.0, do the same thing. And now we're able to create bots. So now we're able to deploy our application to our server. All of the demos worked, which is the first on stage, so I'm pretty happy about that. All right, so we're gonna flip back to a second. So just to recap, what we've done here is two processes combined into one to create a frictionless development experience. The first one is we are auto-updating those base images and the golden images into our application repos of our developers. And if you wanna get more of a deep dive on that process, Kingdon and I gave a talk on that at GitOps days. It's about, it's like an hour long talk, so I won't go into all the details there. But you can check that out, that's on YouTube. And that's this process that's on the far right side with the image scan, bold and update. Then on the left side, we've taken compliance at the point of change and said, okay, now we're gonna have an OPA gatekeeper that's going to either block or allow certain images into our environment based on if it was vulnerable, not vulnerable image. We could extend this. So if we go back to our repo for a second. So Rego also has a HTTP package and a Kubernetes package. So what you would add here if you wanted to make this more robust is you might have an HTTP call out to sneak from this Rego right here that says, okay, give me the report of that image hash and then parse through that report. I mean, you can get those reports back in JSON if you have like a sneak account same with Twistlock or some of the other guys. Similarly, you could use the Kubernetes package to get the CRDs from the local cluster that you're on and query those and get that policy version that we were looking at earlier from the Flex CRD. And yeah, that's my talk. Questions? Back. So everybody comes out and for whatever reason the pod would, another one would spin up whatever and then it phase, phase starting because in the meantime, the pod became vulnerable. Like how can you handle that? That it won't bring down your, the rest of the system, so to say in that sense. Right, so that's why we combine these two things. So the, that image update process that we're doing, that's continuous. So you might have version 1.58 currently in production but because that image update process is continuous you should be configured to where you are always deploying to production whenever that image updater bot updates the images of your team's microservices and then get that to production which is gonna be allowed through by the policy because that matches what the policy is looking for. It's not gonna go and retroactively remove things because that would cause all kinds of chaos in production, but to your point that's why we combine those two things together so that we get that sort of unified experience. Yeah, questions? Cool. All right, thank you very much. Appreciate it. Thank you.