 Well, hello everybody and welcome again to another OpenShift Commons briefing. Today we have a new member to the OpenShift Commons, a new vector. And one of the ways I like to introduce new members is to get them to talk about what they're passionate about, what they're doing in relationship to OpenShift and containers and Kubernetes and new vectors here. They're going to talk about continuous security for containers. And I know security is one of the highest priorities for all of us. So I'm really pleased that Gary is here and one of his colleagues is here as well. So I'm going to let Gary introduce himself. The way that we do these sessions as always is we let them do their presentation and their demo and talking and you can ask questions in the chat and we'll try and answer them. And then we'll open it up for Q&A at the end. So if you're watching this recording, the first bit is going to be the presentation. Everything in the last bit is going to be the really interesting stuff where we ask all the questions. So without further ado, Gary, please take it away. Okay, thank you. Thank you for the opportunity to give us this opportunity to discuss new challenges in the container security and also introduce new vector, our solutions. So a brief introduction. My name is Gary Duan. I'm the co-founder and the CTO of new vector. In new vector, we provide security solutions for the container environment. And our focus is to protect your applications and the services at runtime. So we think that's the most critical time and also the most vulnerable time in the container life cycle. So today I'm going to talk about, first talk about the new challenges and talk about several use cases we saw in our customers' environment. Also, I will discuss our solutions and I will do a short demo. After that, we can go to the Q&A session. So we know that the container guys are popular partly because the rise of microservice architecture. In this architecture, the traditional melodistic applications, they are broken into multiple pieces. They're running as a container. So it's very scalable, very easy to deploy, and there is a consistent way to deliver the applications. However, also introduce new challenges in the sense of how to protect this environment. So because the communication used to be contained within the server, now they become the network connections between containers. So the attack surface is bigger. So this kind of traffic is not the same as those traffic coming into the data center or going out to the external network. So these are the network traffic between your workloads. So we call them east-wise traffic. So those are the something it's very interesting and we want to protect that. Another point is the containers, they use a lot of open-source code. I'm now seeing the open-source code is less secure. It's just that the attacker, they can reveal the code, they can discover the defects in the code and they use that to attack your application. So today, when people, they think how to protect the workload in the data center. They normally, they put some security device. For example, the firewall and the APIs device at the entrance. They think this is probably good enough. But soon they will realize that that's really not a good solution because this device, they are at the age. So they don't see the east-wise traffic and they don't know the life cycle of the containers and they don't know how these containers are talking to each other. Another thing is very sensitive in application security is the insider attacks. So these insiders normally they have a way to bypass the security check at the edge. And intentionally, unintentionally, they could bring in malicious, you know, the programs into the data center. So that's another factor you need to consider. Another one actually is all this traditional network device, a security device. They, because they don't have the visibility to the container life cycle and the container itself, they are relatively short, short lived, right? So they can scale up and down pretty quickly. So here we capture some screenshots of traditional security device, the firewalls. As you can see that in order to keep up the change of this very small application, all those devices, they have to create hundreds of, you know, policies. But think of if you have hundreds of container hosts or thousands of them, I don't think this is a good solution that can scale. So then this makes us to think about what is really the good way to protect the container environment. So in fact, the container has some very good features that can help us to do the security more effectively. First of all, containers, they are software defined. They are very declarative. So the developers and the application owners, they use names, use labels, all these metadata to define their containers and they can define the dependency between, you know, the different containers. For example, we have a tied integration with OpenShift and Kubernetes platforms. So we know not only a lot of runtime information of a container, we also know how containers are created, how they are built and how they are deployed. Another very important principle in microservice architecture is that when you design a container, you want to make it do simple things and to do them well. So each individual container, they has relatively simpler behavior. So because of that, if we can inspect what the container is doing and use machine learning techniques to build a baseline of this trusted behavior is more effective. So actually, I have worked for this one of traditional security wonders 49 for quite a long time. This very long time ago, we already realized this so-called blacklist model is not working in many situations. But now we see in the container environment, because we have this knowledge, the knowledge of the metadata and the behavior of the containers, we've actually built the wireless model. We believe that it's more efficient and also more accurate thing about all the false positives and false negatives, traditional device that creates. So we believe the wireless model is more efficient. So we all know when we're doing security, we should do the protection in-depth. But we also think that to consider security in all states in the container life cycle is also very important. So we call them continuous container security. So in this slide, we listed several steps in the container life cycle from build to shape to runtime. So there are different solutions that covers the security concerns in all these steps. So for example, you want to make sure your system that container is running on, make sure those systems are secured. So you should consider run the auditing checks, run Docker Kubernetes, benchmark at least to harden your system. So OpenShift also provides some very good features, security features. For example, the access control, secret management, raw-based authentication. So those are good features you should use to secure the base for the applications. But what about those containers at runtime when they are running? So we believe that that's really a critical time to protect the applications. So it's also very challenging to think about if you want to monitor the network activities in real time and to inspect those. If there's an anomaly, you want to see that. I want to investigate. And eventually you want to stop those attacks before they reach the application. So those are the something that people are looking for. So talking about monitoring. So here is an example of our customer in the financial sector. So they are migrating their applications from the traditional forms into containers. So they have a database application running in the container. So after some application updates, all the network changes, these applications, they stopped working. So they use our tools. They found out there are some unexpected network connections. So they want to understand if there's really a misconfiguration or actually there are some security concerns here. So they look into all the network connections of the database. Eventually they figure out that some SSIO certificates was misconfigured. So at this time, the customer, they want to have more insight into what's really happening. They want to be able to drill down to network connections. They want to see the behavior at the transaction level, network transaction level, or the packet level, even at the batch level. The traditional tools, they don't work container very well because they don't know the container context. The container, their location, their IP keeps changing. So these tools are not very useful to capture the network activities. Also normally they don't have centralized the unified way to perform all these functions. Especially think about if you want to correlate this with the security events. So once the user, they have a good understanding of the behavior of their applications, their containers. They want to protect and enforce the security policies. So they want to detect the violations. If there is a deviation from the baseline or not, no matter this is a network behavior, the process behavior, or the system behavior. And they also want to know if there are anything suspicious happening, they want to see that. For example, if the attacks coming into your data center or they make a letter moves, they try to phone home or try to break out from the container. They want to be able to see that. But not only that, most of the customers, they want to stop the attack at the real time. So they want to block those malicious connections. Sometimes they also want to quarantine the containers so they can investigate. Also full context alerts and logs, that's very important to them. So here is another use case. So a couple of months ago, this is very high profile attacks run somewhere. What really happened was that some database applications, for example, the MongoDB and Elasticsearch, they were misconfigured. Their ports are open to the internet and their password protection was not enabled. So attackers, they just scan the internet, try to locate these kinds of applications, and they can get in and change the password. So the application owner now, they are locked out. So if we have the container behavior baseline already constructed, so this is pretty easy to detect those activities. So because these applications, they mean to serve internally. They shouldn't have any inbound connection from external network. So once we see that, this is a deviation from the baseline of the behavior, so we should restart right away. So this is pretty easy to cover. However, there are other applications. They mean to open to the internet, for example, your web applications. So actually we have some honeypots running in the public cloud. So every day we saw all kinds of attacks. Mostly they are, for example, port scanning or proxy scanning. If we only look at them at a network level, they look very similar. They all use HTTP, they all use port 80. But however, if we use deep packet inspection to parse those connections, the difference is very distinct. Because your normal application logic and those proxy scanning activities, they use very different protocols. So with that visibility into the application level, so we can identify more advanced attacks. This is another very popular use case. For almost all the enterprise customers we talked to, most of them they have a hybrid environment, which means they have some services already containerized. And they have other services running as in the virtual machines or even physical servers. So today there's no very good tools for the customer to define the so-called egress policy. So they want to define the policies to only allow a group of containers to access those legacy services running outside the container clusters. So there's no good way to define that policy in our policies. So this is one of the very popular requests we heard. So we talked about the inbound connections and the outbound connections. What is more interesting is what happens inside the data center. So we all know that no matter how many security devices, security measures you put at the edge, eventually the attacker will find a way to get in. So once they get in, what do they do? Typically they will do a couple of things. First of all, they try to escalate their privilege so they can have the root access. So then they could run some programs to understand where they are and their location and to find out if there is any network device or security device between themselves and their targets. They probably also want to download some software from the internet from the host in their control. So as a security device, security solutions, we also want to combine all these different techniques. For example, to detect the root escalation and to see if there is any suspicious process running and also to detect any unexpected network connections between different container services. Also identify attacks and those especially at the application level. So using all these techniques, we can break the kill chain in different steps, in different points. So that is more efficient and effective to contain and mitigate the attack, the malware, you know, those threats. So now I can introduce what New Vector offers and some of our feature highlights. So we believe that a security solution has to be easy to deploy. So we support both screen field deployment and the ground field deployment. So we are self-containers. You can deploy our containers in the same way as you deploy your applications. We have very tight integration supports with OpenShift and Kubernetes. So you can use the Password, you can use Demonside, you know, load balancers, all these constructs to deploy our containers. Once our containers are deployed, we automatically start learning the application behavior and try to establish the baseline. We also support several features in auditing. So we run Dockerbench. We are the first to open source Kubernetes as benchmark. Also the first to integrate that into our product. We also support vulnerability scanning for the container at runtime. So for protection and enforcement, we support our seven segmentation. So we can recognize those connection, network connections between the containers, also the between containers and external legacy services. So we can isolate them at the application level. Also at runtime, we detect the root escalation, we detect suspicious processes, and we can detect, you know, threats, exploits between those containers. Once those violations and threats are detected, we are able to block the malicious connections without affecting the good traffic. So your service is still serving the legitimate, you know, the clients. We can also quarantine the connection, quarantine the containers. So in this case, the container is running, but network is shut down. The administrator, they can log in and, you know, to investigate the incident. So we also support, we can also capture the network sessions, network packets for forensic purpose, if a customer wants. So with that, I will do a short demo to show some of our cool features. Okay, I already have some application deployed. So I'm going to run a dirty call attack. So dirty call is, is a kernel kernel bug, right? The attackers, they can use this to try to get a root access to the system. I also run another server. In my demo, I run this in my Mac, but this is to simulate the host that controlled by the attacker, maybe running outside of your cluster running on the Internet. So once I get a root access using dirty call attack, I try to establish a reverse shell connections to this server. So whoever control this server now has the root access to your container. So let's see what we have. So let me switch to our UI. So as you can see, we already have the application deployed. And this is the network connectivity graph, also the baseline of the, you know, the network activities. And here you can see we run several, you know, several containers. I'm going to run this attack from one of these containers. I just get into this container. Well, the reason I do that there are many ways actually in the Internet, the attacker they can inject, you know, this malicious code into your. Container. So I don't want to get detailed how I did that. But just for this demo, I just want to, you know, run this dirty call attack from this console. And as you can see here on the left hand side, I have already set up a server running. So I just run this. Now it's connected back to the server. So from here, I can do a list. This actually is the file. There are files inside the container. And if I do a one line, I'm the root. So I have the root access to the to the to the container. We go back to our UI and we take a look what happens. And we see a red line here. And you can see that this is abnormal behavior. This is a deviation from the baseline. So you can see the connection between this particular, you know, container connect to the external network. That's alert. And if we go to the incident logs, the log of the root root escalation event. And we also see what's a command that calls that. Yeah, actually use the password to attack. This is a special system program to to attack the container. Switch to the violation logs. So this is the violation actually from these clients from this container to the external. And if this is a real, you know, external IP, we can use the IP reverse look up to look at what they are. So yeah, let's conclude my demo. Awesome. Yeah. Yeah, thank you. So that's that's all for this presentation. Awesome. We're having a little bit of a discussion in here in the background. The Kubernetes sys benchmark stuff is really been a hot topic lately too, because they did a really good job at what does it stand for the Center for Internet Security, I think, getting the new benchmarks out like incredibly fast from for the 1.7 release. It's been really cool. And a couple of weeks ago, I did a tech talk with someone from Aqua sects who has started a little open source project around that. I was wondering if that was what you integrated to do that. But I think that's probably, you know, I probably not that ridiculously hard to do that. The project was just called Coup Bench in Aqua sector depth. And I think you're this is the first time I've seen someone integrate it in a production way and not just in an open source scripting kind of way. But this is it's really good to see that work integrated into our project. And I didn't if anyone who's listening in has any other questions that's please ask them. And I'll do that. You did an amazing job, Gary. I think that was a really great overview of the whole thing. And I really like the idea of application behavior, you know, monitoring the applications behavior stuff. And that's sort of a first for me on the approach that this was very, very well done. Thank you. But the other thing that I was thinking was that someday I'm going to have to do a briefing where I get together a bunch of colleagues and community people like yourself and do a briefing on how to hack a container. Just to get people riled up because I think, you know, it's not like I want to give people the tools to do the hack or whatever it is, but I think people just don't understand inherently like some of the issues around giving people root access inside of a container. And, and, you know, we have lots of tools for scanning containers and doing things, you know, nice things along that line. But the inherent insecurity in some containers that I've been playing with is kind of scary. And to see the networking side of it done so well is very nice. So I would like to do that. For example, the dirty call attacks, you know, it's discovered almost one year ago, right? So that is still existing in the wild because it's a kernel bug. It's very hard to replace your whole system to fix that. So I think there's, you know, folks like Dan Walsh and some of the other on the Red Hat team, I think, I think there would almost be like a, like a timer. I have this vision visual of like having a timer to see, you know, give people a registry of containers and see how fast someone can hack them or find it, you know, you just, not that I'm trying to get to stir up trouble. Just, I think it would be very educational to do that and then have someone explain, you know, you have to stand up and explain what the hack was or in your case, you know, what the network, you know, how they hack the network or something where it was running at it. It's, we keep talking about security, but we never quite say what it is that it did and the dirty cow and the other things that you were showing were really good examples. So thank you for that. And if folks want to get a hold of New Vector, there's an info at newvector.com email address and you can always check out their website or you could pop on to our Slack channel and ask a question there. Because I think the new vector folks are hanging out in there as well, as well as all the rest of the OpenShift Commons community. We have over 800 people in the Slack channel, not that they're all talking at the same time. Thank God. But it's a good way to reach out and touch and connect with people as well. So, Glenn, is there anything that you wanted to add? You could unmute yourself if you'd like. I'll unmute you. There you go. So, Glenn, if you're still with us, I think you are. Yeah, I'm still here. Thanks Diane. No, I mean, I think it's pretty comprehensive. So I agree with you that what the community needs is a lot more example than insights into some of the threat vectors and actual maybe a hackathon to raise awareness of the vulnerabilities in some of the containers. And I think I could get Docker to sponsor that too. So, you know, it could be interesting. But long story short, I think he did a very awesome job today in the briefing. I just wanted to thank you for allowing us to do the reschedule and apologize to anyone watching this afterwards for our VBN issues yesterday. And next time we'll see what we can do. If anyone else has any questions, raise your hands and chat. I'll give you a few minutes here. If not, I will be posting this video. And if you can send me the slide deck, I'll add the slide deck to it as a PDF on blog.openship.com probably in the next day. So look for that at the latest by Friday on evening. And thanks again. And when you have a new release or a new thing that you're passionate about, we'll be happy to have Glenn and Gary come back and do another briefing for us soon. Thanks again. Thank you.