 Hello. Welcome back to the Security Dev Room. We are here to hear a talk by Jorge about container security. Please welcome him. Hello, everyone. Thanks for coming. I'm going to use the next 20-20-something minutes to... Sorry. No, you're showing me. Yeah, okay. That's good. I'm going to use the next 20-20-something minutes to tell you something about container security. How many people here are using any kind of technique or strategy to keep an eye on what are your containers doing from a security point of view? Okay. So, hopefully you can bring something home. Who knows SysDec here? We are not in the cloud on monitoring track. So, SysDec is an open-source system troubleshooting tool with container support. So, you can think it's like a mix of multiple tools, HSTOP, VMSTAT, like all the tools you normally would use for troubleshooting, but with container support. We also have a commercial tool, but I'm not going to talk about that. Microservices. We love microservices because they are good for security. Least privileges, least surprise, less access. But there are, in the same way that for bare metal servers or traditional servers, there are many ways of implementing security measures for containers. We got the same thing. But I'm not going to talk about this today. I want to focus on one specific technique. There are multiple of them that you should be using, multiple of them. But we are going to talk about scanning. The containers, they are like black boxes. They are great to deploy. They keep the application there inside, less dependencies, just one process, hopefully. But from the outside point of view, we cannot see at least very comfortable what they are doing. We have two techniques to scan what those containers are doing. We can do static analysis, which basically means scanning the Docker container image before having anything running. And we can do dynamic scanning. And this is, again, where I'm going to focus. Static scanning, why we should be doing this? Well, probably you have seen Docker files like this, where we download crap from the Internet without any kind of HTTPS, without any kind of signature check-in. And then we compile things. So we leave for hackers all the tools to compile their own stuff there. And then developers, they never update those images. This is very dangerous. That's why we have static scanning. We look at what the containers are, how the images they are built. We look for non-vulnerabilities, and we trigger on those. But is that enough? Well, explain here how it works, but probably you know, or you can find it on the Internet. Okay. We scan the image. No known vulnerabilities. Is that enough? Are our containers secure? Well, even there is nothing known or no vulnerability known. These are usually coming from CVE or any other vulnerability databases. Still, when we shoot our own applications or things they could be misbehaving, we need to look at how those containers they are executing. And again, we have two options, so two strategies here. We can enforce. So it's more or less we will see this in a minute. We can enforce on prevent things from happening, similar to up armor or SA linux. Or we can just audit and keep an eye on how things are doing. And if there is something not expected happening, we will let the admins know. Pretty much like security webcoms. This is, these are the different tools we got for dynamic scanning sitcom. We can do with sitcom BPF, we can do more advanced profiles so we can define capabilities and define wireless syscalls. We allow the containers processes to execute. We can use systems that we know already like SA linux or up armor. And then we can also audit with AuditD, which is the login diamond from SA linux. But we have come up with something new using the SysTech technology, Falco. I'm going to compare a little bit with the other systems, so I'll give you a little bit of context. Second, basically, it's sandboxing the container, the process. Basically, we do one transition, one-way transition into a process, into a state where that process, they cannot execute or can't execute only some system calls. There is a strict mode that basically only allows to read, write, exit and return. In practice, in real life, this is pretty much useless because you cannot open new connections or spawn new threads. And when the process does any other system call, it's killed. So with BPF, we can create something more useful, which is creating different profiles of wireless system calls that that container or the processes running inside the container, they can execute. And then we can define different actions. If we want to kill the process, if we want to trigger and alert, if we want to skip and ignore the SysCall, so it's more advanced. This is actually available in Docker already. You can drop capabilities, things like that. There are some system calls that they are obviously wrong if you're running those inside a container. Imagine that your container could review the whole host. So it's obvious using this if we want to do some basic security. But still, Seccom doesn't give you all the flexibility and all the features. So that's why we got SELinux and AppArmor. I'm not going to compare them here. I'm not an expert on this. But these systems, they come with more features. And at the same time, they are more complex to deploy. They have concepts that they are above SysCools like processes, files. So you can define those higher level objects when you are defining your policies. But still, it's complex. It's complex. A difference also compared with Seccom is that these systems are mandatory. If you define that they are running in the kernel, processes, you cannot avoid them. Seccom is something voluntary that your process decides to adopt. Another option for just watching is AuditD, which is the login service from SELinux. Defining real is more simple than SELinux. So we can put here a couple of examples. So there is kind of a syntax. But still, it's missing higher level concepts that we think they are quite convenient. So that's why we created SysDec Falco. SysDec Falco is an anomaly detection system built on top of SysDec Engine. SysDec Engine, basically what we do is we load a kernel module, a very small one that allows us to capture all the system calls and we move those into a user space process where we decide what to do with them. We have an event in the stream where we get all the system calls, but I'm not going to get here into the details. The benefit of this is that for every system call, for every event, we have all the context, the users, the IP addresses that we are related to, that system call and everything. So we came up with a piece of software that allows to define a very easy way, different rules. We cannot block things, we can just alert. This is like a security welcome sort of thing. And then because we are running in user space, we can do things that you shouldn't do at the kernel space. So I said before, we work very well with containers. So given that we are in the user space, we are able to talk with the Docker API, with Kubernetes API, with different container orchestration API to apply those metadata, those concepts in the security and in the rules that you define. So I'm going to give you a few examples. So if someone runs a shell inside of a container, defining a rule similar to that container ID, it's not the host, so we are not in the host, and then the process name is bash. If something like that happens, trigger alert. Or if someone is writing a binary, so we do FD directory and then on our right of different protective directories and executes a right to school, we trigger alert. Or changing a container name space, things like that. So we use those conditions or those rules in, yeah, conditions to create rules like this. So this is an example of a complete rule. So we have a name, a description, the condition. If we are in a directory that we know, and there is an event that opens one file for writing, and it's not one of the processes that we have on a list, that we know that they are package management software, it's probably something writing in a place that shouldn't be happening. So let's create like a message and trigger it somewhere. We can trigger alerts into different places. Syslog files, execute shell scripts, send it into your login system. This is completely up to you. So now, because I have a few minutes left, I'm going to show you a demo of how this works. And I need both hands, so, okay. So you can hear me like this. Yes? Okay, so... No, that's not going to work. Yes, please. It's going to be just five minutes. So basically, I have here, I have Node.js service, which I know it's vulnerable to some cold injection because I created, not because I'm a Node.js security expert. So I'm going to create or spam this together with Falcon. So I have here some nodes. So if I... Let me remove this. So when I access this web service, it returns like just one number, but I know that if I insert some code like this, it's going to execute it on the server. Okay, you can see here that when I insert into this endpoint any JavaScript code, it executed. So I created a reverse shell. Let me find it here in my notes. Dad, it's going to connect back to me. So I open NetCut and I do... This is demo effect, probably. Yes, I'm not in it. I'm in the wrong directory. So hopefully, if not the IP address is wrong. So hopefully we'll get here a connection. Now we are not going into connection. So let me find my IP address that probably has changed. Probably it. Not very efficient, but... Oh, shit. So now quickly connect to that IP. It was... 17... Oh, I forgot. So I'm showing you how to find out your IP address. And hopefully now this works. No. But I can show you... I can show you Falco output already. Okay, so I got here the output. Okay, well, I'm not going to lose more time with this. Hopefully you will have to believe me because what I want to show you is that this is how my Falco Diamond... Not very convenient having all these terminals here. So what I want to show you is that... No, I can't hold this. I'm not going to do that anymore. So this is one complete example of a rule. Run shell in a container. I created, okay, there is a new spawn process and it's a shell process and the progname exists. And it's not one of those binaries, trust the binaries that I believe can do things. Some changes I wanted to show, but we don't have time, then trigger and alert. And if we go back to my other console where I have Falco, we can see that Falco is triggering alerts like this. It's probably at the bottom, but hopefully you can see it. So these are the kind of outputs we can trigger. Now you can send this to your ELK or any other login system or alerting system and just keep an eye on what your container is doing. This is everything I have for today. Well, I have way more, but I don't have more time. So it's probably a good thing if you have a look at these links I left here. More deep comparison of Cessic Falco with other Docker container solutions. Another example or an example of Falco for intercepting or preventing someone else intercepting when I download software from the internet and shell scripts and I run them as route. So preventing someone injecting things there and Falco website and GitHub. And probably we have now like four, three minutes for questions. Anyone? Thank you. We have actually seven minutes for questions. Raise your hand. Yeah, there's one. Raise your hand so I can see it and I will come with a mic to you later. Thanks. So I'm not sure whether I understood the rule system correctly. Is this about allowing containers to do certain things or disallowing? Is this black listing, white listing? Can you do both? You're going to be running around. So the question was if this system prevents containers from executing things or not, we don't block them at all. We just watch and alert. If you want to block containers from executing things they shouldn't be doing, it's more complex. So it's more difficult to bring it into production and you need to use SecComp, SELinux or AppArmor. There is no other way integrated with open source products so far. Our approach has been, okay, that's complicated. Probably you should be doing it as well, but the first step is to keep an eye of misbehaving processes and alert on that. Any more questions? Yes. Do you have a standard rule set? Yes. Yes. What I show you, is Falker rules, which I'm going to open on read mode. This is messy. It's the default rule set that we keep improving and improving. We have started, this is a very recent project and it's still in heavy development, POC more or less. We have started to create default rule sets for some of the most popular services. So, Subkeeper, Kafka, Memcache, a few others. So, MongoDB. So when you are running those, you can come here and enable those rules directly. I have a question about the output of Falker. I know that you can enable the login output or sending the output to a program, but it's possible and I don't know. It's possible to enable the shell. When you trigger a rule, trigger a shell that does something in a specific rule and not in all the rules, because sometimes you can, of course, send the output to a program and then you need to filter the events. Like, okay, when this happens, do this. I would like something like, when this trigger, this rule is triggered, please inform the output assistant, but also allow me to run a shell. I don't think that's possible at the moment, but rings me the bell, it's in the roadmap. In worst case scenario, you could do a second filtering phase in your script. But yeah, it makes sense to have something like that. Yeah, yeah, it makes sense totally. Over there. How can you start this monitoring agent? So, in Docker Compost, do you have additional container? Yes. So, for convenience, we run everything on a Docker container and I expose my configuration file inside. You can run it directly with Docker, run, you can schedule it with your Kubernetes or whatever. The idea is we need to have one file instance in each of your physical hosts. And from there, we get visibility of all the containers running on that machine. Anyone else? Yeah. No? Okay, now let's thank Jorge for the talk. Thank you. Thank you. Sometimes, not all the time.