 Excellent. Hello and welcome to Automated in Zero Trust for Cloud Native Applications. I'm Aaron Quill, I'm a TMM with SUSE out of San Diego, California, and I have with me... Well, Mike is from Spain, and the same TMM. I specialize on all of our Cloud Native products as well as Kubernetes. And I specialize in security. So, here's what we're going to be talking about today. We're going to do an overview of what Zero Trust really means. We're going to talk about how we actually implement Zero Trust. We're then going to do a couple of cool demos to show you exactly what you can do. We're going to then have some conclusions and we'll open it up for Q&A. So, I'm going to start with an overview. Actually, Raul is going to start with an overview. Thank you. So, Zero Trust, what is it about? Basically, it's about not trusting anything implicitly. So, everything is blocked by default unless we have explicitly defined it as trustworthy. This is not a new concept. It has been around for many years, and not only in IT. I mean, here, for example, we don't trust people to cross the door without the right registration and ID. But trust who, what, for doing what, when, and from where. These are questions we should ask to better understand how can we apply this concept to real-life scenarios. For example, are we trusting a network packet, a user, a system call, the applications we are using to implement Zero Trust must be able to identify what we want to focus our policy on. And we also need to define what are we trusting. Is it for the packet to reach an application? Is it to gain access to some data in our systems, or is it to execute another application? When we are trusting it, and I don't mean time, although it could be another variable, like, for example, when it contains a payload with MySQL protocol, when the credentials are correct, when the application, it wants to execute this part of an approved backup system. All this needs to be defined into security policies that will be enforced by a security platform. So, for example, if we want to only allow network packets that use the MySQL protocol to reach my backend application, when it's coming from my front-end application, we need software that can see the network packet coming from my front-end before it arrives. If the security platform can act before the packet arrives, it means it's just alerting, not protecting. And, of course, it needs to be able to see what's inside the network packet to see if it contains data formatted following the MySQL protocol. All this and more we need in order to let it go through, for the application to decide if it let it go through or not. We are starting to see this as a complex topic. We can go deeper into it. There are many variables to consider. Another challenge, for example, is to decide what is useful to look at or not. If we make it too complex, it's not very useful because we are humans at the end, so we will be overwhelmed. If we make it too simple, it may just give the false sense of protection, but it's actually not so secure. The real key to being able to have good security is to automate everything. Because if there's one thing that humans are not good at, it's repeating processes. When we rely on humans to repeat processes, they tend to mess things up. They tend to forget steps, or it's very prone to error. The other problem is when we rely on humans, it automatically inserts time into the process, and it slows things down. What we are trying to do is make our developers as efficient as possible. What we want to do is make it so that as soon as a developer checks in new code, we can automatically bring up that application, run through a test suite to profile how the application normally behaves, capture all those things in the profile that he was just talking about, and automatically apply that. As they do a new feature and might change the database they're using on the back end, we can automatically capture that and incorporate it into that security profile. Again, we don't want to slow down development. Of course, Kubernetes is a great place for us to go ahead and insert this automation into, because we can see containers spinning up, we can see pods spinning up, we can see the network traffic and everything. That's really where New Vector comes into play. New Vector gives us the ability to do behavior-based zero trust. That means, again, learn how the application works, learn all the protocols involved, all binaries that are going to be called, put those in a profile and use that as the security basis. Very easy to integrate into CI, CD integration. In fact, we're going to show that later on. We can do incredible things with data loss protection and a web application firewall, because we can see directly deep inside the packets and see exactly what protocols are being passed in packets and get actual, you know, if it's an SQL database, we can actually see the SQL call that's in the packet. We do real-time scanning for vulnerabilities. We do that in multiple different places that we'll cover in a couple minutes. We can again leverage all of this technology for compliance and for auditing. And we can also put this and just really use this to lock down what is running on your endpoint. It's also very easy to have multiple clusters all over the world and control these policies from one central location. So the first thing we want to talk about is really protecting against the known, right? Protecting against known exploits that are already out there that are defined in a CVE database. And just be proactive to make sure none of those known exploits are in any of our systems. So one other thing the vector has to protect from known problems is to scan in. So we can scan running pods as soon as they start and scan in again if we want as well. And against 12 known CVE databases. We can do the same scanning on the registry periodically and we can even scan the host. It's really flexible. So also we can... The vector has the ability to do deep packet inspection, which means it can see the actual data that is being sent on the packet. And it's not limited to protocols to layer 3 and 4 like TCP, UDP, etc. It can also understand about more than 30 application layer protocols like MySQL. So you can define policies that look into the actual application protocols. And that's the key here is we're actually seeing my SQL call to get all photos in our application. So it really allows us to see exactly what's going on inside the packets and leverage that for security and compliance. So we also use the packet inspection to scan for known attacks, for example. And this is by looking... We can do that, for example. We can detect and SQL injection exploit because we understand the protocol. So that's why it's so important to understand these application layer protocols. Yeah, cool. And we can leverage this for other things as well. Specifically things like compliance policy. So this is an application we built for a customer and it's an application that sits right on their point of sale devices, right? Right at the cash registers and what we built was an application that would automatically scan to see if they had an ID that proved that they were part of our customer program and we would automatically give them a discount. Now, this front end is on a device that's going to process credit card transactions. And what we're able to do with NuVector is say, we're going to go ahead and let credit card transactions happen within our environment. But we want to make sure credit card never goes out to the internet. So what we're able to do is we're able to leverage that deep packet inspection and not just look at traffic but digging and see what's in that actual packet and we'll actually look to see if we see credit card numbers and the normal credit card digits with expiration date and everything. If we see that, we're going to do a couple things. We're just going to kill that packet. We're going to drop it off the line. It's never going to get out to the internet. It's never going to get anywhere. We're going to go ahead and make you aware of that. We're going to put it in our alert and we're going to go ahead and write it to the log so that you're aware of what's going on. So now we've talked about protecting against the known, right? All the things that are already out there, the exploits that people already know. What about the unknown? What about exploits that we don't know about, that there's not a CVE or it's something specific for your application? That's really where NuVector shines. So NuVector can protect unknown by defining the zero-trust security policies. These policies are based on the behavior of the application. And to do that, we don't have to define how the application behaves manually. NuVector has a discovery mode, which means that when a process gets executed by the application, I mean by the container, it will detect it and then add it to the security policy. And the same for any other thing. And we can also see the network traffic. So how is the application accessing other pods or outside or how others are accessing. So NuVector will write all these behavior in the policy. And so you don't have to have a deep knowledge of the application. You just need to make sure that you run your use cases on the application because NuVector observes how the application works. So it needs to see how it's being used to be able to write what is the application doing. So once you have this policy defined, you can review or you can set it to monitor mode. Monitor mode will get this security policy and just observe what is happening and alert of what is not a behavior defined in the security policy. So for example, now the application, the container tries to run a curl command. NuVector sees that it's not in the policy and then generates an alert. It doesn't stop the chain of events. And this is important because if you are missing something on the policy or you want to see what happens, if it stops, you will not see what will happen further. The key is you put it in discovery mode, you create your profile and then because you don't want to just lock everything down, you run it in monitor just for a short time to make sure you really captured everything. You didn't miss one of those binaries that's being called or one of the packets that's being passed on the network. Yeah, if you are not sure you have captured all the use cases, you can set up the monitor mode for a while, let the users do their things and see if you have missed something. And then we have the protect mode. The protect mode basically does the same. It will generate an alert, but instead of just letting go through, it will just kill it. I mean, any behavior that is not in the policy, it will be stopped. Cool. So let's talk about the actual implementation. We typically, well, here's the key. We are a privileged pod that's running on all of your actual hosts. We are not a sidecar, right? We use all of the servers as intermediaries between the applications, the source, and the destination. That's one of the keys, because what we have the ability to do is to attack the network packets before they even get to the next host. So it's not the host waiting for that bad packet to get there. We've killed it before it's even gotten there. If we look at what we actually put on each machine, we have a scanner, and that's the technology that we use for scanning for the known. That's the one that's linked up with the CVE databases that's going to do all of the scanning as every single container is spun up, as well as, as he mentioned, periodically going through our registry. Then we have the enforcer that's also running on all the different nodes. That's what we're really using to lock down the profiles. And there's one key thing that I want to point out here. The enforcer is not running in Kubernetes as far as we're not just looking at Kubernetes pods being spun up in containers within pods. We're looking at all containers on the hosts that are running. And that's really key. If you think back a couple of years ago, there was an exploit that allowed people to kick up containers running on hosts that Kubernetes didn't see. The great thing is with New Vector, we would see that row container spinning up and we'd be able to shut it down. You can see all the traffic and network. I mean, all the network traffic. And this is so important because for performance, you don't need to have a sidecar on every pod. So it excels really well. That's one of the key things, too, is performance. Normally, when we talk to people about New Vector, they hear about the profile, they hear about looking at all the processes and all the network, and they immediately go, oh my gosh, did this slow everything down? No, we're talking about a couple percentage. You know, 3% slowdown maybe. It's very, very fast and very, very efficient. So now after we've gone and we've created our profiles and we've really defined how the application works, we need some way to get those updates out to all of the clusters in our node. So that's where we're going to use a piece of open-source technology called Fleet, and what Fleet allows us to do is to point Fleet to a GitHub repo, and that GitHub repo will store whatever it takes to deploy your applications. It might just be Kubernetes YAML files, it might be Helm charts, it might be customized that's modifying those Helm charts before it installs, but you point it to a repo, and then all you need to do is tell it what clusters you want to go ahead and put this application on. So you just select the clusters that you want to assign it to, and of course that's normal Kubernetes stuff, it can be based on tags or whatever, more labels, we apply it, and then it's going to go ahead and be installed on all the clusters. One of the key things is the way that Fleet is architected, the Fleet downstream cluster is the one that periodically connects to get repo to check to see if there's an update, that's how we get the killer scalability, it's not a centralized place putting all, or pushing all the changes down, each individual downstream cluster is regularly checking to see if there's updates. So now let's go ahead and talk about our actual implementation. So now we're going to run through the process of creating a security policy for your application with Nuvector. We will make sure Nuvector is in learning mode and then we'll run some use cases, basically do everything a normal user will do with your app, and then we'll turn the policy to monitor mode just to show you how it works, I mean it's actually not needed, but after that we'll show you how we can tweak it and harden, and then finally we will export it to automated deployment. So we will navigate to the policy and groups, this is where Nuvector stores the zero-trans policies. And then here we will filter by our application MSPACE and examine the development version of our application which is Web Photobook Dev. A policy with predefined rules exists already because Nuvector actually generates one every time a new pod is deployed. Note the policy type on the right, our production version says CRD instead of learning. This indicates that the policy has been auto-loaded during our CI CD process. Now, switching to the applications with UI, let's engage in various user activities. Let's do things with applications like upload a picture, edit an existing one. You really run through everything your application normally does so that we can capture all traffic, every binary called everything. Yeah, we're just using the application so that Nuvector knows how it behaves. And then if we go back to Nuvector and hit refresh, we can see it has created a few more rules based on what we were doing with the application. So the network rules, here we see it was not there before, some essays, I mean running bash. And then we are going to switch to monitor mode and we will go to the console and we will run a few commands which is not the normal behavior for the application. We are just doing this for you to see. So we log in into the container and just do run things. So if we go back to Nuvector, notifications, security events, we filter by our application. Well, we already see a bunch of alerts complaining about the processes, not including the security policy. Like DF, the command we run, and some other commands which are part of the... I mean, it's not what we execute but it's part of the process. So if we think one of these processes is legitimate, we can simply add it to the security policy by clicking review the rule and then deploy. And back to the policy, we can also tweak it via the UI. I mean, we can add it manually if we want, or we can just edit the YAML file. I mean, it's no secret. And you may wonder, why harden? I mean, isn't it all automatic? Well, the Nuvector approach is to make the policy robust without making it too complex. So for instance, we could add a rule for each external host connecting to the app, but that would be unmanageable. If we are certain about which specific IPs or hosts are gonna connect to the application, we can define it in the policy. The same applies to files. Why don't it add any files? Because no application normally touch a lot of files, access, a lot of files, et cetera. So it will be overwhelming. And besides that, Nuvector has this zero drift mode, which prevents file system drift in the container. And also it has a set of the fault file location it monitors by default. So to conclude this part, we will export the policy in protect mode. And as you can see, it's just a YAML file. It can be edited. You can import it later on on the UI via the UI, or you can just with Cube CTL apply it. Cool. So next we want to do, so we've created the profile, we've got our application, we've got it sitting in a Git repo. So what we want to do next is we want to show you how we can use Fleet to automatically deploy that on all of our downstream clusters. So we're gonna go in, I'm gonna show you how to configure Fleet. We're gonna prepare our resources. We're gonna show you what the resource launcher looks like, and then upload everything to the repo. All right. So the first thing we're gonna go do is we're gonna go to the application repo where we've stored all of the Fleet files as well as all of the dev apps that we need to actually create our environment. The first thing we've got is this Fleet Cattle YAML. And what that does is it gives us the instructions on how to monitor the Git repo, how often we're gonna check it, and what contents to execute. So this is what the YAML looks like. This is what we defined as a Git. There's the name of the application we've defined. The namespace here, you use Fleet local if it's a single cluster, you use Fleet default if it's gonna be a multi-cluster environment. Next thing is the actual repo where our code resides. So here you can see we've just got something stored on Git. And that's where we're normally gonna pull it. You define the branch. We're using main for warning. We default like a lot of things to the old default of master. So when I'm talking to people, this is normally the first thing that people get wrong is they don't put their branch in proper, but I'm sure we've all done that. And then you get to specify what path underneath that Git repo where all the manifests are gonna reside. You can have many applications in a single repo here, we've just got one. And then the target namespace is the namespace that the application's actually gonna be installed on the Git Hub. So it's really easy to define this. And we can have multiple clusters. We can do canary deploys and all sorts of things. All right, so now let's go ahead and take a look at the manifests themselves. So we're gonna go into that manifest folder in Git Hub. And we see two YAML files here. The first one is the setup of the application and the second one's that new vector security policy. So if we go ahead and we take a look at this setup YAML, we can see there really it's a normal YAML, right? It's what we're used to seeing every day. It defines our application. Just the standard Kubernetes YAML will have services and everything defined in there. And it's really everything we need for the environment. And then here's the new vector security policy that he automatically created, right? He just did the one that he created in the UI and exported it. And then here's just a Git Hub workflow that we defined whenever we see updates and we see an updated version number, we're gonna generate the new version and make an update and then Fleet will detect it and all downstream clusters will go ahead and grab it. We're gonna do demo. Yeah, we're gonna do a demo soon as he gets his mouse working where we're gonna do an upgrade to the application, right? We're gonna be a developer, we're gonna make a change to the application, then we're gonna go ahead and profile that change and update the security policy. So while we were talking our application developers have been working hard to make a major upgrade to my app. So my app used to work using a local file as database and now it's gonna use a remote database. And the development team of course wants to make sure the application will continue to run protected in production. So we're going to test the upgrade in development first. So we go to new vector policy. The external database, my SQL has been configured already and is being protected by new vector as you can see. So our application running on development also has been upgraded already and we have loaded the existing security policy but it's still using the local database. So we are gonna well this is the new version, we are gonna change the settings to use the remote database and see what happens. We're logging with the admin credentials to change the settings and here using dbtype we changed from SQL little to my SQL. At the same time we are running some automated tests in the background covering all these use cases we were doing before manually. So new vector logs should show if any of the new features require a change in the policy. And it's currently in monitor mode so nothing is gonna be blocked. So we're gonna see this is the test we are running, very simple shell script. So using curl to do what we were doing on the web UI. And it's basically we use it in we added it as a life less rough in the deployment. So here you can see test means something really simple. So now in new vector if we go to notifications, security events now we can see the security policy is warning us the application is not behaving as expected. You see these these two lines alert. So it's trying to establish a network connection to a mysql to the mysql container in the myapp dev name space using mysql application protocol on the port d3.0.6 dcp So we are going to update the policy to allow this new connection. We already spoke with the developers and all this is expected. So it's not now we see the policy and we can see the the rule has been added to the policy. Now we will export the updated policy edit to have it correct names because this is for development and add it to the repo of fleet is monitoring for production. So once we added fleet will detect the change and then it will deploy the new the new version. So for production application image is already in the registry. We set up the external database protect the value vector and all the data is made great. So basically the environment is ready for fleet. And here is the Rancher interface for fleet and we can see it has successfully deploy the new version together with the updated security policy. If we go to my app we can see it's the new interface and it's set up to use my SQL as external database. And we are going to go back to the main page. What is this? The app is not working. It's unable to connect to some database random.com This is not right though. Configuration seems to be changed. Somebody changed it. Did you change it? I didn't change anything. Let me check slap. Let's go to new vector and see what happened. We can see in the logs the application is trying to connect to this external database we never heard about. Luckily new vectors stop the application from connecting to this database so we are safe. Here's a message here. The developer found a bug in the application and they are going to try to fix it. They just can't get a hold of anyone. Somebody forgot to set up authentication on that API so anybody can go in and change the database. That's bad. How long it will take to fix it? The problem is all the developers are at KubeCon so we don't have access to them. We have one person who is not at KubeCon but they are at lunch and we have no idea how long it's going to be before they come back. We have no idea how long it's going to be before they come back. We haven't seen them. Maybe we can do something with new vector. That's what's cool is we can use that web application firewall to go ahead and define something to stop and change going to that API. We will create some rules to prevent clients from accessing the API. The part of the API has the bug. Here we are creating a new sensor. Now we are going to add a new rule which we can put a pattern and the new vector will inspect the packet looking for that pattern in the URL and if it finds it it will just block it. Here is API settings. This is the part that has the bug. We say it's a URL and we add it. So we create this sensor and now we are going to go back to the policy groups. Go to our MyApp security policy because we're doing in depth first and in WAF we will add the one that we just created. So now it should be blocking the request. But first we need to switch it to protect mode because now it's in monitor so we switch to protect and now this is the old API query. So we can see at the window refresh we can no longer access the API URL. I mean the one that has the bug. Actually all the API has the bug but it's So here in the events we can see it's been blocked so you see the action then I means block and we can see more information about the packet itself. We can inspect the packet itself and we can even download it if we want to. So let's try to do because this is a HTTP get that we try to do with HTTP post to make sure it's working everything so it cannot go through. So I think it's good to apply in production until the developers find a fix. So the key to that is we found a bug in the application. We couldn't get a hold of the developers to plug the security hole. We were able to do it purely through our new vector profile and using the web application firewall. So it's great we were able to stop a security hole without hitting our developers block it until they come back. And this applied to any application. Sometimes you use applications you don't have control, you don't have a source code. Doesn't matter. And another key thing is we are using all open source nothing, no paywalls, nothing some RCNCF projects and here in these pages you can well you can see the source code and some documentation for new vector fleet and also in the last repository we will place all the materials we use for this presentation so you can play with it and investigate. Awesome. And we also have a demo running down in the Rancher booth. So you can come down and see us in the Rancher booth. We'll talk to you about it and show it to you and everything. And with that oh my god, we're 25 seconds early. I just want to point out I've never ever ended a talk anywhere close to on time. So we have 20 seconds for questions. Questions? Alright, awesome. Cool. Thank you everyone and have a good QCon.