 Hello everyone, let's welcome our next speaker in security of the room on FOSDM, it's Lukash Rabbets and he's talk about Selinux's container and times. Let's welcome Lukash. So can you hear me fine? Okay, before we start with the presentation, I have one question for you. How many of you know Selinux? So many people, okay, that's great. So let's start with the introduction. My name is Lukash Rabbets. I'm a Selinux evangelist at Red Hat. Some people already call me that. I'm a member of security technologies team at Red Hat and I'm a rail and Fedora contributor to following packages, which is Selinux policy, X-Gest, Udica and Netlabel tools. Here is my blog post. We're very interested in some CVEs and how Selinux mitigate them. You can find some interesting blog post and GitHub link and also Twitter. So why we are here? I'll use a nice example of one of my colleagues from last week and in one OpenShift cluster, the Selinux is enforced by default. On this cluster, there are almost 200 pods running more than 600 containers and 134 containers run as privileged. In this case, I mean there is no Selinux isolation for them and 134 is so many containers, right? So privileged containers are really scary and today I would like to teach you how you can generate a Selinux policy for every container. So quick introduction to Selinux. I make you Selinux experts in less than five minutes, hopefully. So, Selinux, you probably know it, is implementation of mandatory access control, but we can say it's a process isolation technology for mitigate processes, for to mitigate attacks via a privileged escalation. One of the most important thing you need to know are labels, right? In this case, there are two examples, container underscore T and container file underscore T. These labels are assigned to processes or to system resources. A system resource could be directory, file, sim link, socket, and many more. And these labels are assigned to processes and system resources by Selinux security policy. So how labels look like in reality, they are stored related to system resources, they are stored in extended attributes of the file system. For example, X2, X3, and X4. There are two ways how you can see the Selinux label of some specific file. In this case, it's slash hc slash password. And as you can see, the whole Selinux label is what is written in red. But in this case, we are interested only about the type, which is the third part of the label, which is password underscore file underscore T. So for processes, we can use parameter capital Z for PS command. And if you want to grab some label of container, you can see that in the example, there are three containers and each container has a same Selinux type, which is container underscore T, which is here. Selinux security policy described the interaction between the processes and system resources, right? And we are using for the interaction allow rules. And this is the example of the allow rule. The first is keyword allow, that's quite clear. The second thing saying that any process labeled as container underscore T can read, open, and get attribute of files labeled as container underscore file underscore T. And what is important to say also that everything is denied by default, so you need to have explicitly allow rule inside the Selinux policy, which is loaded to kernel to allow some Cisco. And basically that's it. You are now a Selinux experts and that's all you need to know for generating Selinux policies for containers, of course. Okay, so now we will talk about generic Selinux policy. For each container without some custom Selinux policy, we use the default one, which is container underscore T. This policy protects the host system from the container processes. And processes or containers with this Selinux policy can only read or execute files in slash user. And these container processes can only read to, sorry, only write to container files. As I show you, process type, that's quite simple. That's container with container underscore T, a Selinux label, and file is every file and directory on the host file system labeled as container underscore file underscore T. So this is really, really nice slide. I really like it. These are the least, just few of them, few of CVEs of container runtimes when there was a possibility that the process inside the container could escape from container to the host file system, and in several cases gets rude access. Selinux blocked all of them. The latest one is from February 2019, when a Selinux completely blocked this type of attack. Yep, I say that. Okay, so what about containers attacking each other? For this, we have also solution and it's multi-category security, which is part of multi-level security. So this is the way how it works with MCS. We have two containers, both of them has same Selinux label, which is container underscore T, but the categories are different. First one has categories C1 and C2. Second one, category C2 and C3. In the middle, there are three files with same Selinux type, which is container underscore file underscore T. And as we can see, following container can access this file because C1 and C2 is subset of C1 and C2. What is interesting that a file in the middle doesn't have any category and empty set is also subset of C1 and C2 and same situation for C2 and C3. So this is how we are isolating containers between each other. This is the same slide, but arrow showing what is not allowed. So C1 and C2 is not subset of C2 and C3. So you can ask how you can label some directory as container underscore file underscore T. Container Runtimes has a really nice feature. It's for sure supported in Podman and I'm pretty sure that it's also in Docker that if you're starting some specific container, you can buy them out, for example, in the example slash var slash leap slash MariaDB and you put parameter capital Z. So container runtime engine will be relabel on the host system, this directory to container underscore file underscore T, also with the same categories like running container has. That's because of the capital Z here. If you want to have two containers and you want to share the same directory, you can put small Z, which means the engine just relabeled this directory as container underscore file underscore T with no specific categories. So of course, this solution has several problems. The first one is that one generic default as Selenux policy for containers is too strict for certain cases. For example, Fedora civil project needs container to read and write user home directories. Second one, project flu and D needs to able to read logs stored in slash log slash var on the host system. There is another case when default Selenux policy is too loose in certain cases. There is no Selenux network controls, which means that container can bind on any port and there is no Selenux container, there is no Selenux control for capabilities, which means that a running container has all Linux capabilities. So we are able also to solve this. So what's the current situation? What can you do with MCS and generic Selenux policy? Let's say we want to start container with flu and D project and we want to bind out slash var slash log from the host system and we use capital Z as a parameter. It's here, okay? And this is not good. Actually, it's pretty bad because all other confined services on the host system will not be able to write or read slash var slash log because the label will be container file underscore T instead of var underscore log underscore T, which is the default label for this path. The worst situation, what you can do, you can use this ugly parameter, which is here and basically, you disable a Selenux separation for that container. So this is the worst what you can do. So solutions. You can write completely new Selenux policy for your custom container, but this is the best solution, but it's really too difficult for system administrator because you need to have, let's be honest, quite good to Selenux expertise. The second thing what you can do is to add additional Selenux rule for the container underscore T type. This is totally ideal because you need to know how to write a Selenux policy and also these additional rules will apply for all containers on your system, not just your custom container. So to solve all these issues, there is a Uditsa project. Actually, Uditsa mean a phishing route in Slovak and Uditsa is a tool for generating a Selenux security profiles or policies for containers. So the best way how to describe Uditsa, let's use for example. Let's start the container, mounting slash home with permissions read and write. Let's mount slash var slash spool with read only permissions and let's expose port 20, 21. So as I told you, generic Selenux policy cannot read and write slash home. Generic Selenux policy cannot read slash var slash spool for containers, but on the other hand, can expose all ports. So I'm brave enough to show you live demo but let's finish the slides and then I show you the demo but what will be there is we start the container with the by mounts and ports as I described. Then we only run inspection of the latest started container which is here. And Uditsa will generate a Selenux policy based on the inspection of container with name my underscore container. Uditsa suggests or recommends to load following a Selenux policies which is this load command. We just restart the container with the following parameter where we are basically saying to, not to use generic Selenux policy container underscore T but our which is my underscore container dot process and that's it. And then just to prove it when we run PS, we will see that our container is running with our new Selenux label. So how it looks like what's under the hood. So the whole concept here is based on the block inheritance feature in a Sel language where Uditsa creates or combining the rules from predefined Sel blocks or we can say templates. So Uditsa inspects container and looking for mounts, ports and capabilities. And combines these predefined blocks with the base one, which is base underscore container underscore sorry dot seal. So this is the way we will use the base block which is needed for reading and executing files in slash user and reading configuration files in slash at C. Then we will use net block which we need for binding on TCP port 21 which is labeled as FTP underscore port. And next block is home, which we need for reading and writing to user home directories. Uditsa basically inherits allow rules inside these blocks and generate new Selenux policy with the name my container. We also mount slash var slash pool right inside the container, but for this there is no predefined block. So what Uditsa can do is to look to directory slash var slash pool, find all possible Selenux labels which could be inside this directory, generate block in this case, let's call it spool. And again, inherits all these allow rules to our Selenux policy and that's it. This kind of generated Selenux policy you can use with for following runtimes, which is Podman, Docker, Buildac and also Kubernetes. So these are the slides, so let's try it with demo. Okay, so I installed, can you see it? Good, perfect. So I installed all necessary packages which is Uditsa, Podman and Settles. I tell you about these Settles in a moment. So let's check what's the Selenux status on this system. Hopefully it's enforcing, okay, cool. So we have Selenux in enforcing. Let's start the container from the example. So we are mounting slash home slash var spool and exposing for 21. Okay, container is started and we see that Selenux label of the container is container underscore T, sorry, it's here and here are the randomly generated containers. Now you see command assist search. Basically I'm querying the allow rules inside the Selenux policy loaded into kernel if there is a present allow rule. The first one is related to slash home. Slash home has Selenux label home underscore root underscore T and if there is no, there is no output which means there is no allow rule. The same situation with the var spool, that's pretty clear. The pod has Selenux labels var underscore spool underscore T. T there is no allow rule. But on the other hand, for the port we can see that container which is part of these attributes can basically bind on NETCP ports. Another, let's continue. Now we run Udica. So we need the inspection file of the latest container. Put it to standard input of Udica and the parameter is name of the new Selenux policy. So let's start it, policy is created and Udica recommends to load following templates and final Selenux policy for our container using command AC module. So let's do that and we stop the latest container. Perfect and we start container again but with following parameter which is saying to start container with the following Selenux label. And as we can see, it's not running as container underscore T but it's our custom Selenux type and we are running again to AC search and as we can see the allow rules are now there. So this container can read and write a director is under slash home can access to slash var slash spoo and again on the other hand can bind only on port label ftp underscore port underscore T which is Selenux label for TCP port 21 and basically that's it. So now you are able to generate Selenux policy really quickly. So here are the links for Udica for Podman. If you are interested how it looks like the inheritance in Udica, you can look on Udica POC and you have the links. So that's it. Thank you and if you have any questions, I'm ready. Thanks for your talk. Now as a new Selenux expert, I was wondering why this Udica part, could this be in principle also part of Podman? I mean, you're basically already passing all the information to Podman to read write and read only stuff. Yes, it's good point actually. I discussed it with Podman maintainers to include it there but somehow unfortunately for me, we agree that Udica will be separate too. But we discussed that. I have a question about labeling. So you said that Podman is really labeling all the files in the container on container start? Not really. Selenux is not enabled inside the container, right? So the Podman is enabling this stuff on the host level. So we do it in the host, right? So there is no Selenux. There is no Selenux inside the container. There's a lot of file operation. Yes. Okay, so for if container is large, has a lot of files, it would be kind of slow. Yes, yes, we know about this issue. We discussed it recently inside the Selenux team and most probably we will work on this for future to kind of to make the relaping faster. Maybe there are ways how to do that. Some other questions. Let's see, there was someone in the back. Do you have some plans to integrate it into Kubernetes? Because right now you're writing Podman, then write Udissa and then run Podman again. If I'm writing a Kubernetes or any other orchestration manifest? Yes. I'm doing it once. So how well it works with Kubernetes, Measles and whatever else there is on the market? Very good question. Thank you for that. In near future there will be on GitHub there should be container with Udissa inside there and basically you can almost fully automatically generate Selenux policy inside the Kubernetes. We put annotation generate Selenux policy to Docker file, right? So there is operator, you recognize that some container was started with that annotation, the Selenux policy is generated. However, there are still technical issues. For example, how to maintain Selenux policy between the cluster and stuff like that. So it's not easy, but we are working on that. And some proof of concept you can already find on Udissa webpage. So maybe a follow up to the other question. So is there, I mean it takes a while for Udissa to generate the policy, but if the policy is really large is there also performance set on every container start due to Selenux having to do that? Or is that really fast? And or can you maybe show what the CIL was what that Udissa generated earlier? Yeah, yeah, I can do that. Actually, this is pretty fast. This is the Selenux policy. What you can see the first one is that the blog is called my underscore container. You see that you can see the second and third line saying that inherit all allow rules from these blocks, which is container, restricted underscore net container. The next line is related to capabilities. So this is quite quickly. And all rest of the allow rules are related only to slash var slash spoo because there is no predefined template. Actually, this is not problem. We can generate the Selenux policy really quickly because Udissa just parsing the inspection file which is in the JSON format and looking if there is predefined, if there is a predefined block, then just set the list of capabilities from inspection file. And if there is a need, generate also allow rules. But the main issue is related to relabeling, yeah. Okay, thank you for your talk. We've got last half a minute so we will probably end this talk. Thank you, Rukai, for the presentation. Thank you. Thank you.