 Let's get started. So hey everybody, my name is Obkar Litter. I am a product manager at Tenable. At Tenable, we provide a cybersecurity platform that you can use to analyze your exposure and risk for all of your assets in one place. That includes, for example, risk-based or priority and risk-based vulnerability management on your workloads, configuration management or misconfigurations on your cloud resources, including Kubernetes and cloud-native resources. Then I said that twice. Vib application security, identity security, and a lot of other things. So all of that coming together in one place, because that's how you need to look at security, not in silos and one vulnerability at one resource level. I primarily focused on shift left and container security. So that's the area I plan, and that's why I'm here, to talk to you today about embedding policy as code in your GitOps and CI CD builds and lifecycle. I have this slide here on how amazing and awesome cloud-native is in Kubernetes, but I feel like I'm preaching to the choir here since you're all here. But I remember a story I went to CubeCon North America first time ever, and I was really convinced that everybody who speaks there or goes there as an attendee basically can code an operator in their sleep and they dream about CRDs. And when I got there, I was pleasantly surprised that I was wrong. People go there who are newcomers, beginners go there to learn things, and so that's why I left this in here, right? Because I know we're being live streamed and also this will go on replay later. So for folks who are new to all this, the point I want to make here is cloud-native infrastructure is fueling innovation. There are two main reasons for that. One is increased velocity and less process friction. These numbers tell the same story. This is coming from CNCF Survey 2020. There's another one in 2021, and these numbers go up slightly, right? So 92% of organizations using containers in production. I remember maybe like 10 years ago. I'm from the Bay Area, I was visiting or meeting with a friend in a cafe and he was so excited about containers and it was a brand new thing back then. I feel like now it's more of a commodity, everybody sort of uses it in their day-to-day development lives, right? So 92% using in production, 82% organizations using Kubernetes. Kubernetes is obviously a big leader or has a big lead in the orchestration race. And then 30% organizations using, quote-unquote, newer technologies like serverless in production as well. So cloud-native, apart from, I don't know if you've seen the cloud-native landscape, but you literally need a magnifying glass in order to see the little cards on that page, but apart from all of the tools and frameworks, cloud-native brings a change in culture as well as processes and mindset, right? That's more important to me because technology comes second when people come first. It provides extremely high-paced infrastructure, right? We all know this, you can set up and tear up clusters in a matter of minutes and all of the major cloud providers. That is not to say that the underneath architecture and technology is not complex, but they're able to hide that from the developers using simple-to-use interfaces, right? And lastly, I see a lot of companies building stuff on top of Kubernetes, but Kubernetes by itself is not something that developers necessarily can consume outrightly. So I've worked with serverless environments, for example, where it almost feels like I'm back in the Heroku Cloud Foundry world, where I have my local code and I do a CF push and it pushes my code. Under the theme that might be using Istio for service mesh, I think the one I used used Paquito for a source-to-image build, et cetera, but all of that, again, is hidden from the developer, giving them an easier way to manage runtimes and deploy at scale on the cloud, right? Sounds great, just like the wallpaper. Who remembers this? Anybody? Yeah? This is all, yeah. However, is this velocity leaving you vulnerable, right? This is the crux of the talk, so it's nice and awesome to move fast, but what are you missing out on? Specifically from the Kubernetes point of view and coming in from a developer lens because I was the developer for a while before I got into product management. Again, if you read the blogs and stuff, there's tons of ways to slice and dice at this information and put it into different categories. This is what I think, again, from a developer lens, where the security problems come in. And there's always a flip side to this, right? So developer-focused management, what I mean by that is developers, again, are able to spin up clusters, developers, DevOps, SREs and other technical teams in my experience are in charge or manage the clusters, right? Security in most organizations that I talk to and most customers I speak with, security still lies with the security teams and there's a big gap there, right? So with that disconnect, the best case scenario is you slow down your velocity because security teams will not allow you to push stuff to production that has a vulnerability. Worst case scenario is you expand your exposure service and attack service because you're letting things go into production, right? So that's number one, easy for developers, however, security takes the backseat and it's managed in my experience by somebody else. Second is default configurations are not secure, right? Before you get up and start throwing tomatoes at me, Kubernetes has a very good security system built in. My point here is it's not turned on by default in most cases and that's the problem, right? And there's a reason for that. Default configuration drivers are in kind of behavior, right? Kubernetes is built for speed, performance, scalability and that's why it has the defaults it has. And like I said, every level in that architecture in Kubernetes has the levers and control built in for security but in a lot of cases I see, developers go with what's given by default and that generally is not good enough. And the last one again from a developer point of view is privileged management. So this is again, if you think about you have your infrastructure layer, then you have your platform as a service and then you have your application and now you're building microservices on top of that. Deploying for example EKS and AWS, AWS has its own policies, user groups, et cetera and then Kubernetes comes in with its own RBAC system for authorization and so you have to be very careful and that introduces complexity that any change you make at the lower layer need to then propagate to the other layer as well because they both are tied to each other, right? So these three in my viewpoint is what makes Kubernetes difficult or security difficult in Kubernetes. And also if you talk to, I've talked to many customers if you read these white papers and things like that, right? Down in the weeds like what are the top concerns that security people have? So one is, am I able to define consistent policies across my teams, right? Second is once I've defined that am I then able to apply these policies across my many clusters as well as many cloud providers, right? Every customer I talk to usually there's a public cloud provider plus there is something on-prem that's going on, right? And you want to be able to apply the same policy consistently across. And the third thing is, how do I get my developers to honor the same policies on the very left on their laptops because that's where everything starts. So this is my sort of recommendation if you're new to security in Kubernetes, I would say look at these things first. So on the very left is Kubernetes misconfigurations. So you want to be able to create a single policy framework for governance and access control. So this is when we talk, this is how we talk about policy as code, right? You want to be able to say, hey, here are my set of policies and associated benchmarks and I need to apply this again consistently throughout that lifecycle, the development lifecycle. Examples could include for, you might have a policy that says that you're not allowed to run any root container in your cluster. That's an example or you might have a policy that says your Kubernetes API server cannot be publicly accessible, right? And these are all what we call configurations and a violation of such a policy would be considered a misconfiguration. Moving a little bit to the right, then you want to again apply the same policies in your CI-CD lifecycle, right? And that's when, let's say you have infrastructure as code, whenever a new commit comes in, a new PR is open, you again want to test against the same policies and disallow that PR if a policy is violated. But that's the misconfiguration side of things, right? Then you want to be able to check for vulnerabilities and images as well. I've worked with an open source project called, it's called Pira. They were based out of Spain, Barcelona. It was a great team to work with. And I remember us running, whatever comes default with Docker, that the vulnerability scanner. And then as long as everything looked mediumish, it was fine, right? So that's the developer cake on it. And that's why policies are important to enforce, to avoid that kind of behavior and forced strict policies. So again, very important to scan your images before they get into your runtime, obviously, because one CVE and one vulnerabilities propagated multiple times of your production on your runtime environment. And finally, once you have all of these three pieces in place, then it's really important to look at sort of the whole picture together. You want to look at your, how you apply your exclusions, can't say the word, how you remediate putting in automated processes to be able to do that, right? That's sort of the last step. Okay, I think I'm going really fast, which is good. I usually am slow. Okay, so just strictly looking at your CICD pipelines and security guard rails in that context. Policy as code is, again, a perfect way to do it. Policy as code, so if you know infrastructure as code, the benefits that that brought to the infrastructure layer in terms of repeatability, reducing human errors, the same thing is brought by policy as code through your regular policy files into your policies, right? The main power of policy as code, I've listed three years. So the first is low friction, right? Because it's policy as code, it can work with existing development environments, tools and frameworks, right? So if you're using your Jenkins or Travis or GitHub actions, whatever you're using in your pipeline, you can very easily embed a tool that'll check for violations in there. The second one is more secure by default. Since you are writing these policies even before you actually initialize your infrastructure, you bring that up, you're by default more secure because you're checking for all of that even before your infrastructure comes up. And then finally, increase security visibility because it's policy as code, you check the policy files back into your source, SEM. And so that way, you have visibility around what the different teams are doing and you can monitor it there, right? So very important. So if there's one thing that I keep repeating here that you should go away with is policy check, it doesn't just apply at one place, you need to apply the same policies consistently across that lifecycle all the way from very left on your developer machine to your CI pipelines to your CD pipelines. And then apply the same plus additional policies on your runtime containers, right? So once they have left sort of your world and gone into your production environment. Okay, so this is where Terescan comes in. It's an open source solution by Tenable. You can see it's available on GitHub. Lot of, driven by community, lot of interest as you can see here by your forks and stars. And I know that's not the only way to measure that but it's one way. It comes with 500 plus policies out of the box. It supports ISC engines like Kubernetes or sorry, ISC engines like Terraform, Helm charts, customized Dockerfile so you can scan all of those. Plus it also has benchmark support, specifically CIS. It leverages the open policy engine if you want to create your own custom policies or expand on the existing built-in policies. And I'll show you this in action and also tell you where you can go look at it or we have built a tool where you can test it without actually downloading it and doing it on your terminal. All right, so GitOps, like how many folks here actually practice GitOps frameworks? Okay, very cool. All right, so this is more of a simplified version of what a GitOps workflow might look like. So you have your engineer on the left pushing some code to your SEM, maybe that's GitHub. Once that code gets pushed, a good practice might be to run a GitHub action or a GitLab runner, whatever, to then build an image. You want to run all of your unit tests and then if everything goes well, you get all greens so you want to then push that image into an image repository, public private, right? Then at the same time, you trigger an agent like Argo CD, which is looking at your repository to start your GitOps process, right? So you've got a quote unquote good image sitting in your image repository, Argo CD says, oh, there's a new image I need to deploy. It then tells, it's got that information and it tells your runtime. So in this case, maybe it's Kubernetes. Hey, go get that image, do your thing. And this might be a simple, you know, kubectl set image kind of command. Kubernetes then goes post that image and it rules it out, you know, it takes care of all of that for you. So what I'm suggesting and recommending here with adding your security policies at different levels in this lifecycle is first you put in your Terescan checks and policy checks right on the developers machine, right? One way to do that is using pre-commit hooks or Git hooks. And so this is an example from the Terescan site. I have to say some, so I have not tried this exact code, but again, it should get my point across that you have a Terraform pre-commit hook which runs before every time a developer does a Git commit and it runs a certain set of policies that you have created or using the pre-built policies against the, in this case, it's the ISC false. You can see down here, it says your container is missing a liveness probe and also some of the resources in terms of memory is not set, right? So that's number one, right on the left, right on the left. The second one is on the CI side. So in your continuous integration, after you've done your linting, you've done your unit tests and whatnot, then you put in checks and balances there and this could, again, would differ based on what your code looks like for ISC, it would be a different set of policies for vulnerabilities would be a different set. A way to do that, shown here is with GitHub actions. So there's a TerraScan job, oh, yes. Okay, wait, yeah, this is correct. So there's a TerraScan job action which takes in certain parameters. So your repository, what kind of ISC you're checking against and it'll check out the code and run that and only pass the build if, and you can set that in the configuration because that might depend on your team, right? So you may want to be, you may be okay with a certain number of threshold in terms of low, medium, high and critical vulnerabilities. Okay, the third thing is to introduce the same set of policies or extend those policies in your CD pipeline. So this is where Argo CD comes in and syncs with your GitHub repository. You put in what is called a Argo CD webhook but it's a pre-think type of webhook. So every time before a sync happens, it'll check out the code. Of course, you pass it the right secrets in terms of your repo access, private repo access. And then you can see at the end here, there's a command that runs the TerraScan, that runs the TerraScan binary on your, in this case, the type of infrastructure we're saying is Kubernetes. So it might be Terraform files that it's running against or sorry, it might be Helm charts or customized files. So that's number three. And I have a, this is the demo that I was hoping would work. If not, I do have a video here. I'm gonna try it one more time. So let me see. And this is where I was having problems earlier. All right, let me go, I'm gonna cheat a little and go and try and grab the secret that has my password. Okay, so I'm in my Argo CD app now. And let's create, I've got two applications here. I can actually show this to you. Even if the demo doesn't work, this will be good to see. So I've got my, this is, by the way, this is public, I've forked it from my colleague. You can go to his or come in here. And this sets up Argo CD in MiniCube. It sets up the TerraScan job, connects it to the admission controller. So every time, and it creates that sync, the precinct webhook with Argo CD so every time you come in and it starts to sync, it'll create a new job with TerraScan and do the scan. That's what's going on here. The deployment file is what I was looking at before. So this does all of that. And the demo would show you two different apps. So there's a test app good. And to be really frank with you, this app is actually not, no, I want to show you the bad app first. So if I go to test app bad and open the deployment file, I was going to say to be really frank with you, this does not look that bad to me, right? So this is probably what I would do when I was starting with Kubernetes. I've got my Lightliness Pro, my Readiness Pro. Those are two things that people always add. What else? I've got some labels, which is always nice. And I've got my, I'm not hard coding anything. I've got my environment variable setup, right? So this is a pretty decent looking app. But if I, let's see, if I can get this to work, if not, again, we'll go back to the video. So I'm going to say, hey, create a new application. Let me see if I have the namespace created. So this should be an app. Nope. Okay, so let me go to repo and then I'm going to kubectl apply Apple. All right, so now kubectl get namespace. So I've got this app namespace created here. Okay, so in this one, test good app. I'm going to pick this as my project. Man will think it's okay. I'm going to leave the rest as defaults. Then if I go into my repository, let me grab that URL. Let's happy with that. And then it needs a path for my bad app, which I'll get from here. All right, so I've got my app done here. Let me create this. Oh, I'm missing this. So yeah, I want to deploy my existing cluster with namespace of app. And again, like in real world, you'll probably do this through YAML files and not using the UI, but let's just demo. Let's see, so I've got, let me zoom in a little bit. And let's do a sync and synchronize. So ideally if this demo worked, this would go in and this is the, oh geez, okay, I named the app good app, but it's the bad app. Not to cause any confusion here. Okay, so it's doing its thing. It's thinking. Okay, let me move over. I'm going to show you the video which does the same thing that I just did. But it'll show you. So it's a little bit blurry, but the important part is coming next. So again, this is my bad app. I'm doing the sync here that I was just trying to show you. And you'll see two jobs come up on the side as pods. And then I'll click into, and you can see me on the top right. So this is the real demo I did, which was working. All right, so I'm trying to get into the pod to see the logs. And you can see here, this is the output that comes out of Terescan. Right, so this is doing the scanning and looking at policy violations and outputting the severity. I have configured my demo to only fail if it finds high severity. If it finds the mediums are low, it won't fail. So again, moving a little bit forward here, just want to show you this guy at the bottom here. So this is kind of hard to see, but I can see there's low two, medium, 11, high one. Right, and because there's a high severity, it's found a high severity right before syncing and pushing bad vulnerability to production, it doesn't do that syncing, it stops. And that's that sort of broken heart that you saw up top. So the thing didn't happen, my cluster is still secure. And then moving to the other side, let me show you what the good app looks like. So if I go back to GitHub, let's go back one, back one, good app. So you can see I've done a couple of things differently here. So I've still got my liveliness probe and readiness probe, but now I've added the security context that says do not allow to run as rude, run only as a specific user, as a specific group. And by the way, down here I've also set some restrictions on the file system, so read only file system, et cetera. So this is configuration that my policies are checking against. So in this case, let's go back to the video again. So now when I run the same Argo CD sync operation on the good app here, so sync, sync manually, and it'll bring up the same pods again. So it's bringing up the pre-sync hook jobs. And it usually takes a couple of minutes for the logs to come back. All right, so you can see on top there's a green heart instead of the red one, so it's letting things through. And if you look down here, now I've got a high of zero, medium of five. So that's still the vulnerabilities, but it's met my threshold to pass this and let this go into the production environment. Okay, so just last couple of things here. So again, you can go here and download this and get it working locally on your mini-cube just to try it out, see how things work. You can also, if I were to open a new, this one, just wanna show you real quick, nope, not this. All right, so I've got one folder here. If I look at app, so again, this is a simple deployment. Right, that's something you might do. IngenX also looks very similar. Right here, I've got a couple of replicas, what image it should come from. And this one, I don't even have the liveliness and readiness probes here, so this is a really bad app. But I can say TerraScan scan. Let me show you the help real quick. So you can tell it other things too, right? So what kind of ISE things you're trying to go at, what kind of, if you have custom policies, you can tell it where they're coming from, et cetera. You also have skip rules, because that might be important. That's where the exclusion policies come. Our workflow comes in. But for us, we're just gonna leave everything the same and just say scan. And you'll see it comes back with, it's found some ISE code, and this stuff we are missing. So it's missing some resource level limits. Somewhere in here it'll also, so this is admission of root containers, is what we saw in the other one. It's missing the read-only permission on the file system. CPU limits are not set, et cetera, et cetera, et cetera. And it's got a couple of high ones as well, which is the root privilege policy. So let's go back to this. All right, and the last place where you should also put in your TeraScan policy evaluation and checks is on your runtime. So you've done it on the developer machine. You've done it in a CI pipeline. You've done it in your CD pipeline. But once your image has made it to your cluster, then TeraScan integrates with admission controllers. So you can write a hook there, which says, hey, at this point, you know what image you're trying to grab. So go test that image and see what vulnerabilities are in there before you let this thing become a resource in your cluster. That's the last place where you can check for the same plus more policies. So lastly, this is our security majority model. You can see everything we talked about today falls in the first step of the ladder, policy as code. So automated continuous assessment with your policies, custom policies, and also benchmarks that the policies associate with. Because at the end of the day, then at the end of the day, you wanna see which teams which are the compliance level at a cluster level, at a teams level, whatever that may look like for you. Then the next step is governance as code. So that's where you capture your, for example, your governance decisions like exceptions. That can also be codified, right? It should be codified for audit purposes, for example. Next is drift as code. So now that you've got a cluster running, you still wanna see, you know, any time there's a change in the cluster, which was not provisioned by you on the very left with your IAC files, et cetera, or your images. You wanna see drift between your running Kubernetes versus your resource IAC files. You wanna see just between your running containers and your images, et cetera, right? That's very important. Even with all of those policies in place. The next one is security as code. This is where you start looking at your breach path, your attack path, and not look at one resource at a time, but look at the whole picture, as well as you wanna see the impact that each resource has if it was to be exposed, right? And be vulnerable. And then last thing is remediation as code. That kind of, especially in the IAC world, if you do find policy violations, then you want to automate that process of at least the sort of the low-hanging fruits where you automatically do a pull request and fix that IAC violation, right? So I'm not sure where folks are with all of this, but this is what we consider as you go to the right. But what I showed you today was the very left and where I think is a good place to start. And then I don't know how much time we have left. I think we're almost there. But this is the last slide I wanna put up. There's a QR code if you wanna scan that. And I can show you the site as well. But this is TerraScan in action on Tenable's site. You can put in your own IAC code here. Again, the Dockerfile, Helm, Customize, whatever that may be. And you can see what comes up in terms of vulnerabilities. Right, and I'll actually bring that up real quick. That's the last thing. All right, so this one here. So it gives you sample files. So since we just talked about Docker, I'll pull that up. So this is a simple Dockerfile. And you can click on scan. It'll tell you the, it says there's one medium violation here. You can click on that to get more details as well. Right, and you can do the same thing with let's say your Kubernetes YAML files. So it supports different formats as well. Here's some of the stuff we saw before as well. So it's good to try and just see where you're at, right? And again, it's completely open source. I encourage you to join the community. If you see problems, raise issues, contribute if you like. And I think that is it for me. Yeah, I think that's it for me. Thank you very much for listening.