 Hey, KubeCon. It's Mike Foster here, a technical marketing engineer at StackRocks. And today I want to take you through a Kubernetes security checklist using the MITRE ATT&CK framework. All right, so I have 15 minutes to get through a lot with you guys. So I'm going to start off by talking about the Kubernetes ATT&CK vectors. And this is specifically through the lens of the MITRE ATT&CK framework. Then I'm going to get into some Kubernetes security controls. We're going to demo a few just simple techniques that you can use to secure a cluster. And then lastly, I'm going to talk about how we apply those techniques in a way that makes sense for the developers and the admins and the organization as a whole, how we can kind of balance security risks and still keep developers speed up. Now let's kick it off by reviewing the original MITRE ATT&CK framework. So in 2013, MITRE came out with this framework. And it has since been expanded on. It now encompasses over 11 different tactics and over 290 techniques for exploitation of systems. Now this is a comprehensive list that was detailed based off of real-world examples and has been added to over time. Since Kubernetes 1.0, we've seen Kubernetes-specific exploits. And we found it useful to put this into a matrix that we can update over time and that you can follow along and make sure that you have all your bases covered when working with Kubernetes. Here you can see the Kubernetes ATT&CK matrix. And it's nine different tactics across the top. These tactics are the same as the 2013 version. However, the 40 different techniques underneath them are different. And I can guarantee you that they will be added to in the future. Let's start off by discussing the first tactic and that's initial access. And this tactic is referencing the attacker's objective of getting access to the Kubernetes cluster, either through compromising a component of the control plane or from a Kubernetes resource itself, like an image or some sort of container vulnerability that you have imported into the cluster. Next, we're moving to execution and focusing on the container at runtime. Now we're looking at an attacker trying to achieve their objectives, either by getting access to a container or having some sort of remote access where we're downloading and executing scripts inside the container at runtime. The next tactic is persistence. And this is the case in which an attacker wants to maintain a presence within the cluster. So they're either going to use scripts and mount them into containers or they might leverage a Kubernetes functionality like cron jobs to have some sort of reoccurring process within the cluster. The fourth tactic is privilege escalation. And this encompasses any technique that enables an attacker to gain additional privileges that can be used to take on more actions within the cluster or from a wider scope of resources. This can include running a privilege container, taking advantage of rules within the cluster that the administrator forgot to lock down or getting access to cloud resources. The fifth tactic is defense evasion. And it's a group of techniques that are focused on concealing adversary actions intended to avoid detection. So this could be deleting evidence of an attacker's presence or deleting evidence of how access to a resource was gained. Moving on to the sixth tactic, we have credential access. And this is activities where the hacker is stealing sensitive credentials such as application secrets, passwords, tokens, anything that can be used to be leveraged into accessing users or accounts. And this could be anything from applications, cluster resources, even sometimes cloud resources. Next, we have discovery and this tactic really revolves around a hacker gaining access and exploring Kubernetes through mechanisms like the Kubernetes API server or the Kubelet API. And even some things like the Kubernetes dashboard in which they could get access and then pull the information from your Kubernetes cluster out. Next, we have lateral movement. Following a breach an attacker might try to move throughout the environment and gain access to other resources, including other containers, nodes, or even other cloud resources. And we want to minimize that as much as possible. Lastly, we have impact. And this can be seen as a hacker's ultimate goal. It's aimed at disrupting or destroying resources and activity within the target environment. Now, with those vectors of attack in mind, what are some simple things that you can do to make sure your cluster is secure? It starts by upgrading to the newest version. We want to take advantage of all that the community has to offer the different patches and functionality they bring to the table. Make sure that you are taking advantage of that and upgrading to a new stable version. Next, when we are creating our images, we want to make sure that our images are as lightweight as possible with no packages that don't need to be there. And it also helps to implement some sort of image scanner into your CI process, your build process for those images. Moving on, we have namespaces. And since namespace is that core concept that just encompasses all of our applications, we want to make sure that those are segmented in a way that makes sense for our organization and that we can build reasonable policies on top of that. Speaking of those reasonable policies, RBAC is the main one here. Make sure that your RBAC is enabled in your cluster and that we are actually scoping our resources accordingly. We want to make sure that resources don't have the ability to create cluster-wide resources if they don't need them and that other people or if you especially if you have a multi-tenant cluster that people don't have access to namespaces and resources, they shouldn't. Now moving on from RBAC, we have network policies. Make sure that you have a software-defined network that supports network policies. If you're using a cloud provider, sometimes you have to actually enable network policies, but we do recommend having them. If they're not, you could, if they're not enabled, especially in a cloud provider, and you go and try to implement network policy, you might find that they won't work. So again, make sure that network policies are enabled and you have a container network interface that supports them. Lastly, logging, monitoring, metrics, tracing, you need to make sense of your cluster, you need to figure out what's going on with your microservices and your various applications. In order to make educated decisions for vulnerabilities and where to allocate time, we need to know what's going on. So again, make sure that you are getting the accurate information and that you have the right monitoring, logging, and metrics set up. And that way, we can relay information to the developers, they can relay information to us, and we can make more accurate decisions about how to assess our cluster. Now, I've talked a lot, so I wanna get into some real actual use cases, and there's three that I wanna outline in this talk, and that's the read-only root file system, some Linux capabilities, and network policies. I think these are the three core concepts that I think you should leverage within reason for your developers, and then I wanna talk a little bit about how we can implement them in a way that makes sense. Now, starting off with the read-only root file system, what we wanna do here is we wanna make sure that hackers don't have the ability to download or use any malicious code internally in the container that shouldn't be there. So for keeping our images slim and we're making sure that there's no packages that shouldn't be there, really one of the exploits is somebody accesses our container and then uses it to download and run some malicious code. Now, I'm gonna give you an example of the struts API deployment in which right now is configured with the ability to read-write to its file system. So we have it up and running now, and what we're gonna do is we're going to run the exploit, where we're gonna download and run a crypto miner. So as you can see, the container was able to download and untar the miner D binary and currently is running inside of our API server. Now, that's not what we want, obviously. So let's create the API server as a read-only root file system and try to execute the same code. So let's update this with a replace. So we've reconfigured our deployment and we'll rerun the exploit. And as you can see, we weren't able to actually download and untar the miner D binary. So this is just something as simple as making all of your default for all of your pods as a read-only root file system and then adjusting accordingly as developers need access and need to be able to write within the container. Now, you can actually create a read-only root file system and still be able to save code in the container. What you should do is use a volume and empty directory and then write to it instead of writing actually to the internal file system within the cluster. Just the fact that you're doing it alone will confuse a lot of would-be hackers since they tend to use exploits that are well-known. So things like always being able to write to temp since temp is normally writable. The fact that you are mounting an empty directory volume into your container and then using that for read-write will throw off a lot of would-be hackers. The next functionality that I wanna highlight is the Linux capability functionality. And really what Linux capabilities are here to do is to split that root superpower into a series of smaller capabilities. And you've probably run across these in your work is things like CapChown, CapNetRAW. So CapChown used by Chown or CapNetRAW used by Ping. And these are just examples of system calls that can be limited. We can shut them all down or we can limit it to the scopes of what our applications need to access. Now in this example, what we're gonna do is we're just gonna drop all. And then we're gonna see that the container is not able to actually download the binary and execute it in the cluster. So as you can see, it's running right now. And if we run the same exploit where we're trying to download and untar the minor D binary, we'll see that we were able to download it. However, as soon as we want to change the ownership of the binary, we have to exit the container. So last concept that I wanna talk about is network policies. And you can think of network policies like L3, L4, firewall rules with some Kubernetes-specific concepts layered in there. What we're gonna be doing is we're gonna be applying a default-deny network policy. Now, what's interesting about this network policy is it's not really intuitive. We're just really saying we're giving it a name of default-deny, matching to the namespace in the pod selector and matching labels just like we normally would with a Kubernetes service. And we're saying we're limiting a policy type of egress. However, this policy is going to shut down all communication tuned from the pod altogether. And that's because Kubernetes has a restrictive permission system with network policies where everything is opened by default when you create a cluster. And as soon as you apply a policy, everything except for the policy that you've applied is shut down. So since we haven't actually specified anything underneath this policy type for a specific port or protocol, all communication tuned from that pod is going to be shut down. And that means I'm going to redeploy my first struts of vulnerable deployment. And what we're gonna do is we're going to then add the network policy for this deployment and again, run that exploit that we had before. Now, what we should see is we should see a W get function call and it should, it's basically just hanging in the background. It's not able to actually perform its function and eventually it's going to time out. So to summarize, we went through a read-only-root file system, outline Linux capabilities and demonstrated a basic network policy, all that can help secure some basic workloads in your cluster. Again, it's up to you in order to interpret how to use them and that leads into our next section of how do we actually implement these policies in a way that makes sense for our developers. We've gone over a lot in these last couple of minutes so I wanna take some time to talk about some ways that you can actually integrate this in a way that makes sense at your workplace. The first step is to shift things onto the developers and please don't hate me for this but the developers, you guys are gonna know what you need in terms of resources, the networks, communication that your containers are gonna have and any persistence or any storage that the containers need. So there has to be a communication in terms of what makes sense. Secondly, build things into your pipeline as get as many security checks into your pipeline as possible, whether it's image scanning, something like a configuration check as part of that pipeline. Automating will save you a lot of time and the faster that you can automate and do things the safer you actually are because you're constantly building checks into your pipeline and it's harder for hackers to accommodate for the speed at which you develop. Lastly, after we have established that communication and we've set up checks in our pipeline, we wanna use a principle of least privilege when applying these policies across our teams. Keep that communication up when a developer needs something else, we can relax that policy or enforce more policy as different services are needed. But regardless, again, try to keep that policy standard as minimal as possible to reduce that service area of attack. Thanks for watching, I got so many shout outs to give. Follow me on Twitter, LinkedIn and my GitHub account where I'll be breaking down the certified Kubernetes security specialist. Check out KubeLinter, it's an open source security tool for enabling policy requirements as part of your CI build pipeline. Come join my meetups. We talk about cloud native, Kubernetes and general security applications policies, what's new, what's hot. Lastly, if you wanna reach me personally, I'll be at the booth during KubeCon, come say hi, bring your questions with you. You can always email me DM me on Twitter or LinkedIn or check me out at the Kubernetes Slack channel where I'm a part of security and security tooling. So come join the community and I look forward to hearing from you. Thanks again.