 Hello, everyone. Welcome back. We are going to start our next session. And it's a hands-on workshop, right? So get your laptops ready. Get your machines ready. And Enes and Rory, they are going to teach us first tip to full lifecycle security with open source tools. So over to you guys. Thank you. Is it on? Does it work? Relive. Hello. Awesome. Yeah. So yeah, we're going to talk about full lifecycle security. And we're going to show you some tools on how you can get started. Now if you're sitting in the front and you're taking table space and you're not participating, it would be great if you can make space for other people who want to participate and sit on the tables, just FYI. So if you're just checking emails, then please make space for others. Otherwise, yeah, if you want to join and participate in the workshop, then here are spaces in the front as well. Yeah. Cool. Awesome. Let's do the obligatory about us first. OK, so my name is Andres Orles. I'm the open source developer advocate at Aqua Security. I'm also a CNC ambassador. I got started in the DevOps space at the end of 2020. I worked as developer advocate and then as site reliability engineer and now joined Aqua. I also have a YouTube channel where I talk about Kubernetes and cloud-native tools. If you're curious about that, check it out. And I'm Rory. I'm a cloud-native security advocate at Aqua. My background is perhaps slightly different. I'm from the Infosec-slash-pen-test world. I was a pen tester before I did this. I've been doing security for a while now. And you can generally find me on Twitter, GitHub, and anywhere else as Racine. And I will give a sticker to anyone who knows why Racine. And you need to be in a kind of role-playing game person for the mid-2000s. You're going to really answer that question without Googling. So I'm a little work to project this. We've kind of got three things we want to cover across. We want to talk a little bit about how you can use tools at various different parts of the development lifecycle. So the idea of this workshop is we're trying to help people who maybe haven't done a lot of this before, who want to work out where they can get started with some ideas of what you can do, the commands you can use. And we're going to give you the examples that we've got, there are others. Three stages in the development process. Things that can be done in development. So what can you do in development during the development process? Then things you might want to do in CICD. And we'll talk a bit about that. We won't do his hands on for that, because it's kind of difficult to get everyone to start setting up their own CICD in the middle of a workshop. We don't have that long. And then we're going to talk about production as well. So we're going to talk about things you can do once the container is alive, once you're running maybe in a Kubernetes cluster. What sort of stuff can you do there as well? Of course, pre-rex, in terms of if you want to follow along, things you will need during the workshop. Some form of local Kubernetes cluster, kind, mini-cube, mic, anything which does Kubernetes should be fine. If it's based on QBADM, which all of these are, it will probably work best. And the ability to download and run binaries in one of those operating systems. So if you've got Windows, WSL will work fine, or VM will work fine. Linux, Mac OS, FreeBSD apparently as well. So if you're very unlikely that you have a FreeBSD laptop, but if you do, then this will also work. So yeah, any of those setups. Obviously, don't use production clusters for this, because we will be installing tools and running them. And people will be very annoyed with you if you're doing production. So test clusters are advised. And right, this is the most important slide. This is one if you're going to take pictures of slides. I would thoroughly recommend taking a picture of this slide. Ground rules. And this is where I mentioned, yeah, people in front if you're going to do workshop stuff, please do use the desks at the front. Also phones and cell phones. I think everyone's been good for that already at the conference. Materials. The slides for this talk are at slides.pundland.uk. The commands for this, and I'll show in a second what that looks like, are at commands.pundland.uk. And the setup notes, just notes, if you're going to do stuff, are at setup.pundland.uk. And this is because I hoard domain names. And that's a domain name I wasn't using for anything, which is why pundland.uk. But I'll show you the, oh yeah, if any questions, please feel free to ask. Shout out. We'll try and answer as we go along. It's kind of difficult. If you're out in the back, you might need to come forward. But we're happy to do questions as we go. We will also work around. So if you have a question, just wave at us. And in the first part, I'm just going to come to you with my mask on, or worry in the second part. So and then what we'll do is we'll have a quick look at this. Am I not working on command? OK, so this is what the commands file look like. And basically what you've got here is at this top, this is the title of the slide we'll be on. So if you're looking for what's the right command, I'm going to copy paste. This is in case you want to copy paste the commands. If you don't want to type in them, some of them get pretty long. Copy pasting them is pretty easy. And typing is going to be error-prone. So if you go to that URL, you'll get this. You can just copy the text into, like a text editor. I'm going to mention this one at the top, because we might want to do that now. We're going to all install Trivi. Conference Wi-Fi, as I'm guessing everyone has noticed, can be a little bit hit or miss. So what we might do now is we'll kind of change things around. We'll do a little bit of installing Trivi now so that people want to do it, but you have the time to get it down and working, because it might take a little while to download. So for things to do, if you go to commands.punluns. UK, bring that back up for everyone who's not got it, go to that URL and get the installation URL. This is the installation for Trivi, which we're going to use quite heavily in the first part of this. So I'm going to put that into... So there are various ways of installing Trivi, which we're going to use through this. And there should be something which works for whatever operating system you're working on. If you're doing Debbie Annaboun to just get in the dead, we'll work fine. This stuff for Arch, Homebrew, it will work fine if you got Homebrew. If you've got Nix, you can do an install script if you're feeling brave. I mean, obviously, generally for security, don't go curling stuff off the internet and running it. But if you're comfortable doing that, then that's one option for you. You can also get binaries, archives just from the GitHub repo. And you can, if you're feeling super fancy, do it from source. You can even do it from Docker Hub. So there's a whole load of different ways. I wouldn't use Docker Hub for the demos. It might be all right, actually. You can probably do that. Yeah, it'll probably work. You'll change the commands a little bit because you're going to have to put in Docker at the front of things. But it should work. So what to do now, if you're planning to go along with it rather than just watch, go onto that page, find the installation methods that work for you, and just kind of do that in the background. And we'll do some slides before we get to the first command. So hopefully, that'll give everyone enough time to have got to the point of running trivia. Also, there's still a completely free table in the front. So if you're in the very back and you would like to participate, there are still free tables. So we can do that, too. OK, right. So hopefully, everyone who wants those URLs has got those now. So I'll leave those up for a couple of seconds to make sure everyone who doesn't got them yet. The command's one is probably the main one. And the slides, all the information is in the slides as well. So if you want to just use the slides URL, you'll get all the commands there. Just kind of easier to get them out of that command spot. So security and development. When developers are writing code, what are the kind of things they could do to try and improve security? In every talk you'll get about kind of supply chain or about developer security, you'll hear the phrase shift left a lot. And it is a buzzword, but it is true. And the reason why people use it is because it is something that I thoroughly recommend, is doing security checks as early as possible in the development lifecycle. And if you're a developer and you're thinking, why should I do this? The advantage is it means security people are less likely to shout at you when you get to production. Because if you've done your stuff in development and you've fixed the things that you want to fix, you're going to pass things like admission control in production clusters much more easily. You're not going to get a nasty surprise when you try and deploy, and someone says, hey, you have left some flag that you're not allowed on your Kubernetes manifest, and we're not allowing that to go into production. If you've done this up front, and we'll show you, it's super simple, super easy to do, you can actually get that kind of like ease of confidence that things are going to deploy okay. You're not going to get told, hey, you can't deploy this. So we can do vulnerability scanning, and we'll talk about how to do that. And also IEC scanning, right? Everything's infrastructure is code. Everyone's using it, and it's great for scanning because it means it's easy to scan. So we're going to talk about IEC scanning as well. Yeah, so when are we actually using security scanners? So first of all, before we're using any third party resources, any other Git repository, any container image, you want to make sure that you know what's going inside those resources before you actually start using them. So before development, then during your development process, and also when you're then ending up to build for example the container image, deploying a test cluster, deploying a staging cluster. So before deployment in your CI CD pipeline, as well as after and during deployment, also in production. And we're going to walk you through all those different aspects and steps during your development life cycle of where you can use security scanning. So vulnerability scanning, so we're going to start off with vulnerability scanning, and we're going to use container image vulnerability scanning tools. It's a useful way of assessing base images, right? So if you're thinking I want to write an application and I've found a really cool base image on Docker Hub and I want to use that, one of the very first things you do is run a vulnerability scanner over that image before you make it the basis for your application. Because if it's, for example, being non-maintained and there are millions of Docker Hub images, a lot of them are unmaintained. A lot of them haven't been patched in like five years in some cases. So if you get the wrong base image, you're going to have a really bad time because you're getting a lot of very old software, a lot of very vulnerable software. So running a movie scanning tool is a great way of checking that before you get started. We're going to demonstrate with Trivi because that's Aqua's open source scanner, so that's when we know best. There are other scanners available, which we'll talk about as well. Just FYI, we don't take any data by you using Trivi or anything related. There's nothing that we track or collect from you. It's literally just open source. You download it. You do anything you want with it. Download the code for it, you know, all right. So how do vulnerability scanners work? This applies to generally how vulnerability scanners work because they can be a bit black box, right? You run this tool. You spit out all the results. You're like, how did that work? And the answer is it's actually conceptually fairly simple. Vulnerability scanning tools basically look for two classes of things. One, they look for operating system packages. So most container images will be based on a distribution. Debian, Alpine, Rel, CentOS, something. All of those things have got a security database. The security database says this version of this package has got these vulnerabilities. So all the vulnerability scanner does is it downloads the database. It runs the package management tool inside your container image and compares them, right? And it says this version has got this vulnerabilities. This is the one you've got installed. Therefore, you have all of these vulnerabilities. Conceptually pretty simple. There's a lot of complexity and nuance, but the basic idea is pretty simple. And then the other thing that most of the vulnerability scanning tools in open source will do now is they'll do programming language package scanning. So this will be things like MPM, RubyGems. They can also look for vulnerabilities in your libraries. And again, well worth doing. A third actually worth mentioning is that things like Golang, specifically for Golang, you can actually scan inside the binary as well. So Golang binaries will tell you what versions of libraries they have, so you can get that information out of a binary as well. So again, a lot of the scanners will do that. And it will assess whether there's known vulnerabilities in installed versions. There's been various talks about how do I know if this vulnerability actually affects my image. And the answer is that's a really tricky, in my opinion, really tricky question to answer. But the easiest thing to do is to try and get as few vulnerabilities as possible into your images before you start to put in production. And a scanner is a great place to start. So open source vulnerability scanners, there are lots. We're going to call it Trivy, because they won't mean it well. That does not mean it's the only one available. You can get Trivy, gripe, clear, sneak. There are quite a lot of different scanners. A lot of them work similarly, the basic processes. Those are all open source of various different scanners. So those are all ones you can choose. But the fundamental idea is pretty similar. OK, so installing Trivy, I'm hoping that anyone who's planning to type along has had a chance. If not, we'll pause for a little second. I don't know. Is anyone getting on OK with the conference Wi-Fi? Is everyone managed to get this on yourself? We have Trivy already installed. Yeah, we've got a couple of ha- OK, awesome. Right, fantastic. We have some Trivys. If you need help, hands up. If anyone needs help, Anais is happy to help. If it's just a conference Wi-Fi being slow, though, we probably can't help. 4G might be your friend. If you've got tethering and you don't have data limits, they're going to cost a lot of money. So yeah, there's lots of different ways you can install this. You can use homebrew, apt, yum. Yeah, hopefully now everyone's kind of got to that point. We've got stuff installed. So we're going to start off. Let's start off with scanning. This is the most basic form of vulnerability scanning. What I've got here is there's two commands up on the screen. The first one is Trivii, which is just a shorthand for image. So we want to do an image scan. And then the image you want to scan. Notably, you don't actually have to have this image pulled down locally for this to work. It doesn't stuff anyway. I've got two up there. First one is in Docker Hub. The second one is in the ECR mirror of Docker Hub, because I was trying to work out whether we would hit Docker Hub limits. So I'm thinking if we all scan above 2204, Docker Hub has limits, rate limits. And we might all get kind of like told, no, you're not doing it. So if you do get like a rate limited message and you are doing a conference Wi-Fi, you can try that second one, because that one should not be rate limited, I'm told. So let's actually do that and see what it looks like. Do, do, do, do. OK. So basically, it's just as simple as that. And it will. What you'll get back is, and this looks kind of messy. So I'm going to give you the neat, clean version, which means I have to make it a bit smaller. But let me just show you, because the table is super nice looking. How small do I have to make this? That's going to work. Oh, one smaller. Wow, you have to make it really small. Hey, see, the table looks really nice if you make your screens small enough. So basically, what you get back, if you run this vulnerability scanner, is you will get back a set of vulnerabilities. It will tell you severity. So we've got 13 lows, six mediums, no highs and no criticals, which is good. It will tell you what CVE is not, it thinks is a problem. It will tell you severity. It will tell you the install package. So what that's telling us is core util's version 8.3, 30-3 Ubuntu 2, is installed. And I think the CVE is present. And that's just literally based on checking the version installed, what CVEs, the security database for Ubuntu, thinks is present. And it'll give you a title as well. And also, we've got some links out. If you want to read more about it, you can also read more about the CVE. And that gives us a nice table. So if you just want to look at it and get an idea of how the image is, that's not a bad option. So, all right, hopefully, is there a way to run that one OK? Is it running over a conference Wi-Fi or 4G? Who tried to run it? Who tried to run it? A couple? And it working? Awesome. Right. Fantastic. This is going better than expected, if I'm honest. One thing I found out for any other presenters, there's Ethernet on the stand. So if you're up here, you're OK, because we've got Ethernet. So, fantastic. That's a basic scan. Now, the first thing we're going to get to next is something which is important for Ubuntu, specifically, and Debian images, which is there's a flag which you want to have on your vulnerability scanner called ignore unfixed. And what this does is both Debian and Ubuntu have a list of vulnerabilities that they have not issued a patch for. So even if you get apt update to the latest version, the vulnerability will still be present. And that is generally because they don't think it's that serious, mostly. But they're not that concerned about specific vulnerability, so they won't fix it. So what you can do with Trivi is you can just say minus, minus ignore unfixed. And if we do the same thing, so last time we had about 19. And this time, we get four. So a lot of our vulnerabilities just vanished. So the first question you're going to have for yourself is, do I want to run ignore unfixed or not? Because people might say, well, I want to know about all my vulnerabilities. But let's be honest. If you actually used any old school vulnerability assessment tools like Rapid7, Qualys, or Nessus, all of them effectively do ignore unfixed. They won't tell you. And there's not even as far as I can tell an option to turn it on. So if you're doing like for like comparison with old school vulnerability scanners, I personally would always turn on ignore unfixed. I mean, you can turn it off, like if you want to know. Like I really want to know what's present. But for practical purposes, if you're in an enterprise and you're trying to compare it with other tools, I personally would just put ignore unfixed on. You get a much more sensible list. And basically, there is one today. There is one vulnerability, one package they've not updated, which has got some low CVs. You're probably not too concerned about that. You could say, hey, that's a great image. I'm happy to use that now. So that's ignore unfixed. It's an important one to recognize. Because a lot of people get quite concerned when they first start using container scanning. And they run it on like a Debian image or an Ubuntu image. And they get tons of vulnerabilities, like what's going on here. That's the answer. So next thing, you might only care about high or critical vulnerabilities. You might not always care that actually this vulnerability is like a medium or low, depending on your organization, depending on your threat model and your risk posture. You might say, I only want to know about stuff that's super serious. And we can do that as well. So what we're going to do here is we're going to do trivia image. So you can put I or image, either will work. And you can just tell it severity and say, hey, I only want to know about highs or criticals. And then I only want to know about vulnerability type OS. So you remember when I said there's two types of vulnerability scanning, there's operating system and there's program language package. This is only going to look for stuff in the operating system. So if there's an NPM there, it won't look. If there's RubyGems there, it won't look. So you can fine-tune your scan depending on what your role is. Maybe you're thinking, well, I'm responsible for base images. I'm not responsible for the app depths. Cool, I only want to know about the OS images. Oh, it's one of those. So let's do that one. So if you're going along, again, I would probably copy paste these out of the commands. It's going to be easiest. You can type if you're feeling super confident. I personally, as you can tell, I'm not confident enough in my typing on stage, so I'm just going to copy paste. And if we run that, it will, yeah. So this particular image has a lot of vulnerabilities in it. And this is what I'm saying. You'll see this is older images. You can end up, this has got, what even is that? A large number. I don't know if I'm necessarily going to scroll all the way top. But this is an image that you would never, should never run into production, right? If you're running this image, don't put that in production because look at all the highs and criticals. Another thing I actually mention, and shout out, I don't know if I can get anyone from the app what open source team is actually in here at the moment, but shout out to them for rapid development because this secret scanning thing wasn't in present when I did my, when I got these slides ready last week. We now have secret scanning, apparently. This one, that's a good example of how secret scanning can be a bit tricky because that's obviously a dummy key. Because you can tell, because it says search snake oil. That's just a dummy key. But it's a good example of that. You can also use this for secret scanning. We're not going to talk too much about that in this, but because it's just literally turned up, but useful to know as well. So that's a good example of you can say, okay, I want to just get highs and criticals. I'm looking for OS volums. So. Should we take some questions? Oh yeah, do you have any questions so far? Or is that all super clear? Anything unclear up to that point? Clear? Okay, I'm hearing clear. Yep, awesome. Everything's clear, either that or everyone's stunned into silence or struggling with conference Wi-Fi. So just to give you an example of you can also do, you can also do library. So we can flick that round, right? So maybe I am someone who does not care about the base images. I only care about the stuff that's in the package libraries. I'm an app developer. I'm not in charge of maintaining images. So you can just say, volunteer library. And that'll run. This again is on a slightly older node image. And again, we get back tons and tons of volums. Actually not too many, but enough highs and criticals that I would not recommend running that image in production. So that's the other option. So we got our two different types of scanning. Critical flag are also really useful. And we're gonna dive into more detail on that when we are at the CI CD part. But if you want to have checks before deploying any container image to production or to your staging environment or so on, then you can basically make sure that the pipeline fails if there's certain types of vulnerabilities. Yeah, actually another thing just to mention is where the high and critical come from. This is the CVSS, based on the CVSS score. It's a bit of a tricky topic because what is high in CVSS or critical in CVSS, your particular organization may or may not care. But it's unfortunately the only, like if you're just getting a, like you're doing something generally like a tool, you can't pick out like what an individual organization's version of severity is. So CVSS is at the moment anyway, the most generally accepted way of scoring vulnerabilities. And the high critical thing just relates to exactly what the CVSS score is. So another thing you can do, another way of using Trivi, another way of potentially doing vulnerability scanning in development is say you think I would like to download and install a package from GitHub, right? I want to go and get this repository. I think this is a cool tool and I want to have a look at it. Before you do that, you might think, well, let's just check and see if this package is well maintained. And now, okay, you could go and read the files yourself, look at the changed log and work it out or you can just run a repo scan. So Trivi will happily scan GitHub repositories. And we can just get, we can tell what vulnerability type we're looking for, in this case, library. And then we just give it a URL. So a GitHub URL, any GitHub will work. So this will just go and go to GitHub and pull down the repository, scan it, and what it's doing, and I'll actually show you what it's doing when it's done it. Donk. And you get back a ton of vulnerabilities. The reason being this is a relatively old Rails app that I built last year and vulnerabilities stack up super fast in that area. But let me just show you what it's doing. So what it's done here is it's gone and found a yarn lock file. So what is a scanner doing? So sometimes when vulnerability scanners don't work, a useful thing to troubleshoot them is to understand what they're looking for, right? So to find JavaScript vulnerabilities, one of the things that Trivi will do is it will look for anything which is called yarn.lock because it knows that's where the package versions are. So it'll get that and that's when it does its analysis. So it's found a yarn lock file and it's also found a gem file. So it's found a gem.lock, gemfile.lock because this is a Rails application. It's got gemfile.lock as well. So it's got two different sets of vulnerabilities. It's got those ones which are obviously Ruby and Rails vulnerabilities and then it's got this little lock which are NPM vulnerabilities. We can also do it another way around. So if you've got files down locally, you've downloaded files onto your local machine, you've already cloned the repository or if you want to scan stuff that's your own code, right? You might not have this on a GitHub, you might not have this on a repository. So you might say, I just wanna scan the file system, right? And again, once you know about how vulnerability scanners work, all they're doing is looking for specific files, analyzing those files and comparing against security databases. It makes sense that you can scan a file system. So we should be able to do this. Where am I? So if I just do that, get cloned. It's a super small repository. So if you are doing things, the get cloned should just work. It's pretty fast. And then I can just do, just let me make this bigger again so people can probably see. It basically does the exact same process, right? It will go in, it will say, okay, I've found a yarn lock, I have found a gem file, I'm gonna scan those and I'm gonna give you back the vulnerabilities we've got. So it's super fast. You can also scan private Git repositories by providing the GitHub token. And one other thing you can do is you can say, you know what? I don't care about the Rails vulnerabilities. I only actually care about the JavaScript vulnerabilities. So the other thing I want you to tell me about is tell me what JavaScript vulnerabilities do I have in this package? And just give it a specific file name. So if you know the file name you want to scan, the other thing is if your scanner is not picking it up for whatever reason, you know, it's having problems detecting it. Maybe you've got a different file name from usual. Then you can say, hey, you know what? Could you please go and send that exact thing? And it'll go and say, okay, cool, I'm just gonna tell you what's in that. So that gives you back those vulnerabilities. And again, we can see we've got quite a few in there because I have not passed this application in a couple of months. So obviously one of the things we've seen so far is all the output that we've had is table format. And table format is nice if you just want to look at it on screen. But it's not super useful if you want to analyze it because obviously, you know, if you've got like 500 vulnerabilities, they will come out on the table. It's not really easy to share that information. It's not really easy to say, hey, I'm gonna save this somewhere and let someone else analyze it or I want to analyze it. So we might want to do JSON format, right? And the JSON format that Trivia's got is actually pretty useful. So I'm just gonna scan another repository I've got. So again, we're just gonna tell it, the only thing we're doing differently here is we're gonna do JSON format. But apart from that, this command's pretty much the same. We're just picking a repository on Docker Hub. Trivia, by the way, like Docker will default to Docker Hub as a registry. So just taking the same format. I know some, not every container runtime will do that these days, but it's just defaults to Docker Hub. So if you don't tell it differently, it assumes you're talking about Docker Hub. And I'm gonna go and get, you might tell what I'm looking for here because I'm gonna get an image called Spring for Shell demo. And yeah, you get a very large amount of JSON back. What's interesting here, if you are looking to get more information about the specific vulnerabilities, is the JSON format has far more information than the table format. You're not just getting version number and severity. What you actually do get is you get all sorts of fun stuff, like installed versions, fixed versions. You get severity source. So where did that severity source come from? Because one of the fun things about vulnerabilities is different bodies will score the same vulnerability differently. So if you look on a GitHub security advisory, it might give you one severity. And then you go to the national vulnerability database and it'll give you a different severity. Which of those you want to choose is kind of up to you, but if you're trying to work out why Trivia has told you it's a specific severity, get JSON output and then it will happily tell you the source. So that's GitHub. So this particular one, the information came from GitHub security advisory and it also tells you where the advisory is. So if you want to go and find the advisory and say, hey, how did I get the severity or what is this thing? Also it gives you this, which is this is the CVSS score. This formula looks very fancy, but basically that's how you score these things. It's got a number of different parameters you put into it and how it comes out with a number. I have a whole different talk about why that's not always very accurate, but that's not really for today. And it tells you what scores there are. CVSS has two versions, v2 and v3. So that's in there as well. And then we have references. So you want to find more, that's there. So there's a ton of information in there if you want to go in with JSON parsing tools. That's definitely an option. Any questions about that part? Yeah, because that's quite a lot of information. Anything unclear? You can raise your hand. You can raise your hand. We have time. How are we doing for time? We're doing all right for time. Yeah, we're fine. Okay, we're all awesome. Right, okay, we're doing fantastically. All going brilliant. So next thing we want to do, and I am going to have to cut very much if you do, if you are typing along, do copy, paste this one, do not try and type it. I would challenge anyone to type this command without making at least one typo. So one thing you can do once you've got JSON is you might say, well, you know what? I don't want to know about all these vulnerabilities. I'm interested in a specific vulnerability. So I, for example, I'm looking for Spring for Shell. Spring for Shell, if you didn't come across it, was a Java ecosystem issue from earlier this year that was fairly nasty. And so a lot of companies were very interested in what is their exposure to Spring for Shell. So you might say, well, I know the CVE number I'm looking for and I want to actually just find that. How do you do that? And I want to walk through this one because there's a little bit more to it. There's a couple of tricks here that are important if you want to do this. The first one, and this is one that tripped me up quite a few times before I got it right, if you're going to be parsing the output of JSON from Trivi, you need to give it the minus Q switch because otherwise it gives you some text before it goes into the JSON and any JSON parser will complain horribly and say, I found something that's not JSON in here, I'm not going to do it. So minus Q is a very important switch. Then we give it format JSON. We give it that Spring for Shell again. And then what we're doing here is just JQ. I don't know about anyone else in the room, but personally whenever I do JQ I end up Googling. So this is the result of my Googling for how you get a specific vulnerability. What we're going to do is we look in the results array and we look in the vulnerability. So we're going to look at the vulnerability ID for every vulnerability returned. And then we're just going to use a select. I want to go select the vulnerability ID where it's CVE 2022 22965. So this will just tell me about that vulnerability. It won't give me all of the other stuff that we got back before. And what you get back is, yep, in this case, this particular image does have Spring for Shell. It's actually got it in a couple of different places. And JQ very handily makes things look pretty as well. So one other thing I'm going to do, actually not in the slides, but I thought about it after I'd written the slides, is another tool you can use for this. If you've not used it, it's JLS. And JLS is really handy because it does this. And then what you can do is you can interactively go through the results. So for example, all the metadata you can have a look in there. You can look at the diff IDs, the repo tags, image config, which is quite handy. And then you get the results. So if you want to actually interactively look through, if anyone's not used JLS, although the installer got a bit funny when I installed it today, it did need some Rust libraries to make it work, it is actually a really nice way of interactively looking through this data. Because you do get a lot of data out of these tools. And if you're trying to analyze it, it can be a bit painful because even JSON, it's easier if you're good at your handy with JSON. But if you're not, I would thoroughly recommend looking at JLS. So configuration scanning. So far, what we've talked about is vulnerability scanning. We've talked about the idea that we're going to look for vulnerabilities. It's relatively mechanistic. I mean, essentially, what you're doing is which things haven't been patched. With the exception of ignore unfixed, the goal is basically to try and get as few vulnerabilities as possible. If your organization is large, you'll probably have a policy about how many of different severities you can have before you go to production. Ideally, I'd say zero. However, let's all be realistic here. If you've ever tried to get to zero and maintain that in a whole lot of container images, that's pretty difficult to do. But the other thing we can do, and is a great idea when as early as possible in the development lifecycle, is configuration scanning, IAC scanning. So we can do this either during development or when using third-party projects. If you are downloading people's Kubernetes manifests and applying them to your cluster, run an IAC scanner over them first to make sure they have been done with decent practices. This is important. I have seen lots of third-party manifests do some rather, I mean, you might be OK with it, but I would recommend running the scanner and making sure you're OK with it. So for example, they might do something like say, I want a privileged container. Or I want to create a manifest that gives this particular project cluster admin. And you might say, well, actually, I'm not too comfortable with this third-party project getting cluster admin. You could read the manifests, obviously. You can manually read all the manifests, read the Helm charts. Or you can run a scanner over it and get a quick view for what are the likely problems. Personally, I think it's not a bad way of checking that before you use any third-party projects. What I'd say here is this is a bit more variable. Each tool that does this kind of scanning will have its own rule set. The rule sets tend to have some common points. All of them will do things like, for example, checking Dockerfiles to make sure you're not running as root. Because that's a pretty standard check. But each tool will not only have different rule sets, but they will apply different severities. I'm kind of hoping this space will get better as it matures. And we can all agree on what the severity of things are. Because right now, I can see people running one IEC scanner. I'm saying, I've got these vulnerabilities. Then your auditor or pentester runs a different IEC scanner, gets a whole lot of different severities out. And you have this kind of like, what's going on? That's something to be aware of when you're using these tools. It's not like CVSS score. There's no CVSS score for IEC. It's generally down to the tool author to say how severe they think something is. And people do vary on that. But we might want to look at things like Kubernetes PSS. So this is Kubernetes Pod Security Standards. Kubernetes PSS, essentially, is a document which guidelines on how locked down you want your Kubernetes manifests. So there's three levels. There's unrestricted, there's standard and restricted. And of the two, you kind of want to see can you actually comply with those. There's also CIS benchmarks. So there are CIS benchmarks for Kubernetes and Docker. If you've not come across them for CIS benchmarks, they're essentially our vendor-neutral configuration guide, which we will be talking about on Friday. So if you are speaking to Kubernetes one, so if you're looking for more detail on those, we have a talk at 2 PM. At 2 PM on Friday, we're going to talk about different benchmarks, compare them, show you different tools that you can use from across the Cloud Native ecosystem. Yeah. So we're not going too much to those now because we want to spoil the talk for Friday. So what we're going to do is we're going to show you how that works, how does the configuration scanning works, and some reasonably simple examples of what you can do to do configuration scanning. So the first thing to do if you are following along and doing the demos is you want to do just get cloned. So just get cloned that. Again, this should be in the commands file. So if you're just picking up and you want to say, where is this? You want to go down to, there we go, configuration scanning docker. So you just find that in the text file in the commands list and you can get cloned that repository down. Shouldn't take very long. It's not super huge. So I will do that. We are also going to use that Trivedema repository in the second part of the talk. So it's generally useful if you can clone it now. Yeah, it's probably worth cloning now. OK, so we've got that down. And I'm just going to go into the Trivedema directory. That bit bigger. And then we're just going to go to the config. So what this is doing is this is just doing a configuration scan. And how this works, essentially, is it looks for files that are of the formats it recognizes. So it recognizes docker files. It recognizes Kubernetes manifest. It recognizes Terraform. And I think some others as well find out on the website. But definitely those three, because those are the ones we're going to talk about. When it finds one of those files, it analyzes it and attempts to say, what is wrong in here based on the rule set I've got? And we'll talk about it later on. You can't actually write your own rules. So if you're not happy with the rules that come with this, if your organization has got specific standards, there's nothing to stop you writing your own rule and adding it in and saying, OK, I'm particularly concerned about this. Or indeed saying, you know what? I'm not concerned about this. This particular thing is not a concern to me, therefore I don't want to be told about it. But we'll do a very basic scan first. So what this says, another delight of open source is that this output completely changed from yesterday. Thanks to the open source team being very fast on development. It actually looks a lot prettier than it used to. So what did we get back? It basically says, I found a docker file. And it found a docker file because it was called docker file, like most docker files are. And it found certain things in it. So the first one, it has rated. It says, medium, specify a tag in the from statement. So standard docker good practice is you don't pull from latest. Because if you pull from latest, you don't know when the underlying image changes. I'm sure everyone who's done a lot of docker work has been burned by pulling from latest. And then the next version of the image comes out and it removes a binary that used to be in there. I know the Ubuntu images, they spent a long time slimming them down. And a whole lot of my stuff broke because I was using Ubuntu latest. And the new version of the image came out. They pulled a whole lot of binaries out of it. And suddenly all my stuff didn't work. So don't do that. Don't run docker latest. Run 22.04, 20.04, 180.04, whatever tag that you want to use, but don't use latest. Next one, it will say, hi. And again, this is subjective severity. You might not think of this as hi, but this is the rating from the tool. You must have at least one user command in the docker file. So any docker container will run as root if you do not specify a user. Running containers as root is a really bad idea. If you could get one thing fixed in your container ecosystem, I would partially recommend make sure all your containers don't run as root. The reason for that is it makes it much easier to break out of a container. If an attacker gets access to your container and they are the root user, there's lots of CVEs that only work if you're root. There's lots of CVEs that will fail horribly if you are an unprivileged user who don't have any rights. You will make attackers lives much more difficult if you don't let people run as root. So if you take one thing away from this talk, if nothing else, go back, run this. Anywhere you see that, try and fix it. It's not super simple because you do have to make sure the application still works, but well worth doing. So we've got that one. We've also got some other ones, which were for weird stuff in this. And you can see we've got port 22 in a docker file. We're running SSH in this docker image. Obviously, you should not run SSH in docker images in the vast majority of cases. There may be some edge where that makes sense. Most of the times, it doesn't. And it tells you, for example, exactly what lines it was going to say, docker file line 5. There is a statement, expose 22. And it's taking that to mean we're running SSH inside this container image. Yeah. And it also gives us some other stuff as well around package managers, and around minus y. So this is where severity. So this is where, personally, if it was me, I wouldn't rate this as a high. But this is where I'm saying severity is a subjective. It's complaining here that we are running without minus y. And if you don't put that, sometimes it'll pause for user input. And so your docker build will break. So to my mind, that's like from a security standpoint. I'm probably not that concerned about that. But from a maintainability and stability standpoint, that's actually quite nasty, because your builds might break. Something asks for input. And then the last one, maintainer should not be used. Maintainer should not be used because it's deprecated. Maintainer was the old way. Many, many, many docker versions. Anyone who's around when docker 1 and 13 was a thing, you used maintainer statements. Now you don't. So if you're seeing things like that, it probably means the docker image has not been maintained in a long time. It's a very old docker bar. It's maybe worth also pointing out that you don't have to be a security professional to use Trivi and check your manifests, your deployments from its configurations. When I got started with Kubernetes configuration checks helped me to create better deployments. So this is really something that anybody can use to improve their deployments? That's a brilliant point. And I would say that's probably the best point about Trivi. Not just Trivi, but any configuration and vulnerability scanner is, as you can see from this, you don't need to be a security expert to run this tool. There's not a lot of specialist knowledge to read this. It has advice on what you can do and how you do it. So you have the opportunity to give this to people who are not security experts and say, look, run this. Try and work out how to get to some other highs. Try and get the vulnerabilities down before it goes further on to the development process. And people get used to it. I think once people got the habit of saying, for example, I don't want my docker images to run as root, it can become part of general practice. And that, to my mind, is the only way we're going to improve container security as a concept is by people gradually applying these practices, getting better. So let's actually try and do that, right? Let's try and fix something. We're going to try and fix this, and then we're going to rerun the scanner so that we can actually see it working. Who's going to visit this point? Hands up. Who's still with us? Who's still? OK, awesome. Awesome. Great. Brilliant. So what we're going to do is I am going to, will I brave Vi? Why not? Let's brave Vi. So what you're going to do is you're going to edit the file. Pick an editor of your choice. You can pick any editor that's working in the machine you are on. And what I'm going to do is this particular bad docker file is helpfully annotated so we know what's wrong. Down the bottom here, you will see we've got a thing that says, not specifying a root user means the container will run as root. So all you need to do to fix this, in this case, is just to delete that and activate the user line. So I now have a user. When this image is rebuilt, it will run as user UID 1001. Is it good? Yeah, it's a rough one. OK, awesome. So then what we can do is, once we've done that, so if you've done that, just go in, uncomment that line, and rerun the scanner. And what we should see, he said, hopefully. And it's kind of hard to prove, but where was it? I think it was there. It was the second one. So it's no longer there, right? So the scanner has rerun. And kind of predictably, it now says, hey, I found a user line. Therefore, you're not running this image as root. Therefore, I won't no longer flag this. I actually make one point about this, because there is an area where this is a great example of where scanners won't save you. There are some container images that what they do is they put the switch to a non-root user in the entry point script at the end, so in entry point or in command at the end. And so if you have a, because we had this problem with the Docker benchmark when we were writing it. And people said, hey, your remediation says you need to put user, but we do it a different way. And we're getting false positives. Ultimately, though, that's quite hard to detect automatically, because you would need to parse everyone's entry point scripts and work out where they're switching to a non-root user. So that's a limitation of automation, right? Automation can't pick every edge case up. If you get a scanner result and you're like, hang on, I've run this image, and it doesn't run as root, it is a false positive. But if you think about how the scanner works, it would be super difficult, but potentially possible, to go parsing every single person's entry point script looking for where it switches to a non-root user, because there's different ways of doing that as well. So that's just one to watch for. When you're running tools like any automated tool, don't take it as gospel. Don't assume that it's right about everything. But you can try to understand how it does it, because then you can go, oh, it does it like this. Therefore, I can understand why it's not finding this particular. And that's just what I know about, because it kicked up on the benchmark. So we can fix a Docker issue. So we can also run this on Kubernetes, right? We can do Kubernetes YAML. There's more, probably, to Kubernetes YAML, but we can actually run it on that as well. So let's do that. And what it's done, it has found some many, many things wrong. So this particular manifest that it's running over has got a lot. It's a really bad manifest. This manifest should never see production, apart from if you're a pentester. That's where this comes from. This is my manifest that gives me root on the host, if you don't have pod security policies and stuff. So it's basically all the bad things. So if we scroll, where did we go? Ah, there we go. We can see things it basically is going to complain about. So for example, it's going to say, allow privilege escalation should be set to false. This is actually a really easy one to set. It's very risk-free, because I've very rarely seen a container image that wants to go from an ordinary user when it starts to a privilege user once it's going. Most of the time, it goes the other way around. So if you set allow privilege escalation, it actually stops that happening. You can't actually do that. So what it's doing is it'll then give you a range of where it looked. So it looked inside the definition of this container. It couldn't find the statement, allow privilege escalation equals false. So it's decided you are vulnerable to this issue. And it has to give you a range, right? So the reason there's range of lines is that's the definition of that container. And it just says, somewhere in there, I would have expected to see allow privilege escalation equals false. I haven't seen that, therefore, you've got this problem. And we've got a whole load of problems. We haven't specified an app armor profile. We aren't doing cap drop all. So by default, Docker will give you access to certain parts of Roots rights, even if you don't need them. If you're running an application that doesn't need any part of Roots rights, if you've got an application that maybe you had it running on a VM, it was running as an ordinary user, it doesn't need capabilities. Capabilities are only needed if you're using part of Roots rights. So you can drop cap all. And that's another good hardening step. And so it's setting that up for you. It's saying, hey, you should do drop cap all, if you can. In fact, what we are doing instead is we're doing cap add all, because this is a very bad image and should never be deployed in production. Because we're saying, give me all the capabilities. So we've got all of these. We're doing all the bad things. And it's going to tell us all the bad things we're doing. And as you can see, there's quite a lot of them. But that's because this is a very bad image. So what we'll do is actually, hold on. Let me just let me run that again. And I'm going to find the one I'm going to try and fix. We're going to fix one of these as well. Let's find one to fix. Where is the one that's really bad? There we go, that one. One of the things that's flagged up is we should be setting privileged to false. Privileged to false is the default. But I actually recommend setting it explicitly in your manifest. If you've got a template manifest, privilege to false, because it makes it super easy for scanners to find, and it won't do anything. It's not going to break anything, unless you actually need to be a privileged container. In fact, what we're doing is we're setting privilege to true. Privileged containers, as hopefully everyone knows, are super, super dangerous. If you ever need to have a privileged container, make sure that whoever wrote that manifest really needs privileged. Like, they don't just need rights to do modify a fastest on path. They don't just need rights to bind a port. They don't just need rights to do raw traffic. All of those can be done more fine-grained with less privileges. Privileges basically turn off all the security. Give me, and any attacker who gets access to a privileged container will break out to the underlying node, guaranteed, because it's super trivial. There is nothing, you know, it's going to happen, unless you're doing something funny like sandboxing. But basically, privileged containers are super dangerous. Don't use them. So let's fix that. So here we go. On line 16 of this manifest, we know that the statement privileged true is present. So what we can do is we can. I'm going to let's brave Vi again. Has it worked last time? So I'm going to edit that file, and I'm going to go down to the line in question. I'm going to find privileged privilege true, and then I am going to change that privileged false. Great, fix the issue. Exit out, and then rerun the scanner. So all you have to do is literally, I mean, if you want to do that one, pick one of the other ones. There's lots of bad things in there. Host network is bad. Host PID is bad. Host IPC is bad. You can take any of those in terms of false. You could do cap drop all instead of, so just change that to be drop rather than add. That would work as well. But any of those things is bad, and you can change them. You can spend lots of time on that later. Yeah, this manifest takes a lot to fix. So this is literally the worst manifest. I deliberately designed it as the most horrible manifest. If anyone, if you run that on a cluster that you care about, and it doesn't get blocked, I would be concerned, because that should get blocked. That should never be allowed onto any cluster, because dangerous. But what we can do now is we can re-scan. And we should no longer see, he said, crawling up hopefully. No mention of privilege, no mention of privileged. No mention of privileged. Yep, okay. So there is nothing there that mentions privilege. So we fixed that issue. So if you are writing manifest or you're using third-party products, you can see how you could use this scanner to fix your issues one at a time. Look at the issue. Some of them are easy fixes. Like allow privilege escalation will never break anything in my experience. Doing things like dropping privilege. You want to understand why they wanted privilege first. And doing things like dropping capabilities does need testing, because sometimes you might break something. You do have to watch with some of these. Don't just blindly apply everything this says. This is like the whole, probably hopefully the whole underlying message of this part of this presentation, is scanners are useful. Do not blindly accept what they tell you. Because they don't know your environment. They don't know how your clusters are set up. They don't know how your organization is set up. They can give you advice, but you still need to interpret it. So, but that is Kubernetes core. And last one, so we fixed our Kubernetes issue, is Terraform. So if anyone's using Terraform, as I see, you can also use trivia to scan Terraform. And it will do the same job as it did for both Kubernetes and Docker. So we can just again, we've got a bad Terraform file in this directory. And it will happily go in tell us lots of stuff is bad here. Because again, this is designed. So for example, we've got something like a hard coded password in our Terraform. Obviously you should never hard code passwords inside your Terraform scripts. We have got one, this has identified it and said, hey, you have got a hard coded password. You don't want to do that. And you can see there's lots of bad things inside this. So this again, we've got a database. And this is where I'm saying every tool will be different. Every tool you use for IC scanning will have its own database for every format. So some of them will have more or fewer vulnerabilities. I kind of personally have this hope that someday we'll have like a kind of unified database of these things. But for now it's pretty much tool dependent. So do watch for that. You will get different results from different tools. And there are lots of different IC scanning tools. The same was lots of vulnerability scanning tools. But again, lots of things in there. It's worth mentioning here that Trivi uses TFsec under the hood, which is another tool from ACRA within the ACRA GitHub repositories. So TFsec is specific for Terraform scans, but Trivi is integrating it in its ecosystem. So if you will focus on Terraform, you can also use TFsec directly. And there you can write your own rules with JSON and YAML. Yeah, so, and we won't fix a Terraform one because I don't know how to fix a Terraform. One more command. Do you wanna talk about this one? I'll run it. Yeah, so Trivi has since a few weeks also Trivi Sbomb. So you can, we heard a lot about Sbombs. Basically repository like an inventory list of your, for example, your container image. So you can run Trivi Sbomb to generate an Sbomb of your container image. Now, we also have a Docker desktop extension and you can generate Sbombs with the Docker desktop extension as well as scan container images for vulnerabilities also directly through Docker desktop. So as you can see, this is not supposed to be human readable. You can make it nicer. Yeah, I can probably make it, I was thinking about it, I can probably make that human readable by just piping it through jail. That's right. You can also at the beginning. Yeah, MSQ. Yeah. Thanks, I was definitely going to forget that. So, so we can also like have a, you wanna have a look at it and that's what an Sbomb looks like. So this is a Cyclone DX Sbomb. We also support since yesterday since the newest update from Trivi, SPDX as a format. You can tell how easily open source team make it for us doing presentations. Thanks guys. Everything changed. There's like five new things. So hopefully we will catch them all. But yeah, so if you think about how container scanning and how Sbomb creation works, this makes sense, right? Because one of the things a vulnerability scanning tool has to be able to do is it has to know every package that's installed. That's literally part of what it does. So there's nothing to stop you putting that into an Sbomb. I'm actually gonna mention one thing just in passing where Sbombs or some Sbomb tooling might not find everything. If you have a container image and the main program is not installed as a package, right? So say I've compiled the binary and that's the main thing that my container image runs. An Sbomb tool might not find that version because how would it? Because what it's doing is using a package database. So something to watch out for just not just with Trivi but any Sbomb tooling is it's really tricky for Sbombs to pick up anything that was manually compiled into the image. So another one of those things of tools are cool but be aware of what they might not do for you. And that's one of the things that Sbomb tools might not do for you. And I know that there's some people in the ecosystem who are looking at how to fix that problem which generally revolves around building the Sbomb during CICD but for the time being most Sbomb tools do have blind spots potentially. So just one to note. Cool, and that's Sbomb. So the next thing to talk about, actually before we go on, any questions about any of the kind of in-dev stuff? Bond scanning, company scanning, yes. Over there. Do we have a mic or something? Do we have a mic? We have a mic. In the back? The man over there. The question? Questions. We like questions. Hopefully. Let's jump over the CICD part so I have enough time for Starbucks. Yeah, yeah, I'll go quickly. Yeah, we should move over there. Thank you. When you're scanning for the vulnerabilities against known CVEs, that's all well and good but suppose the open source package has been, malware has been updated by some malicious actor. How do we check for malware and what do you advise? So you can do malware scanning of images. I mean malware scanning fundamentally works the same way. It's gonna basically look for known bad things. So it has a pile of signatures and you can integrate that. The question is finding a tool that will understand container formats. So not all of the traditional VA scanners out of the malware scanners know about how containers work. And this is the tricky part. I mean there's technically nothing difficult about it but you would need to find a malware scanner that understands container images. I'm not gonna get commercial products because that's something that we do in the first slide but we're talking about open source today. I'm not actually aware and people from the audience feel free to correct me of a container scanning. So does malware right now? I'm sure there is one but if there is I don't know it. Basically you would need a malware scanner that understands container format or you can get repositories download them and use any file based malware scanner. The other thing I would say about that is this is a part about trusted images. So if you've got a bad actor who has put malware into an image this is where container image signing and provenance become so important because you want to have confidence that the image you're using is coming from a trusted source. In general one of the other if like people are just thinking one or two things I want to take from this talk don't download random stuff from Docker Hub and use it in your production environments. Please don't do that. I was a pen tester and pretty much every pen test I did of a containerized environment I found at least one image that someone had downloaded from a random Docker Hub repo and run it in their cluster. Don't do that because just as the question says that might have malware in it so we might have compromised that. So yeah basically it's finding a scanning tool that will support it would be the answer. Yeah the open source package analysis tool should be coming up but I think if somebody signs something and certifies it that has malware in it they could do that unintentionally so people need to be aware. It is a tricky one. It is a tricky one for sure though. Okay I'm going to blast through this when we just have enough time for starboard. Go through this really really quickly. Security scanning, oh no more question. Hello, do you have a good scanner for batch scripts? Oh I don't, there are linting tools. I'm not sure I've seen many security scanners. I know there are linters for bash and linters if you think about a linter it's a very similar concept to a config scanner. I mean what's it doing right? It's looking for bad practice based on signatures. Fundamentally the same thing. It's just a question of focus where these are security focused primarily and linters tend to be good practice focused but the concept's identical. In fact some linters do security, some security scanners do linting. I'll top my head now but yeah, similar idea. Same like for a Terraform. I mean you can style weird stuff and nowadays most of the open source tools throw you some bash script and then simply execute it. Yeah. Do a W get in bash. I mean it doesn't feel right. Yeah that is a problem. Ultimately a lot of people will end up when it comes to flexibility will end up using bash because it's super flexible and everyone likes a bit of bash to tie everything together but you do need to watch because everything you're running is part of your pipeline or part of your development process and if you just do Terraform and miss the bash you've potentially got a hole. Yeah that's a good point. So skew.tv and CI CD. We're gonna go across this reasonably quickly because we can't really demo this. I think people try this up GitHub actions in the middle of a talk and it would go very badly for us. But a couple of points to make. You can use Trivi as part of your CI CD pipeline and we're gonna give you a quick example of a couple of things to note if you're doing it in GitHub actions. Personally I love GitHub actions once I've got them working. If anyone looks at one of my repos you'll find like 10 failed runs before the first past run because the syntax can be a bit tricky so I just wanna give you a couple of pointers and stuff that I went wrong with. This is an example. You can pull that one out the slides later on. What this does, it's a GitHub's basic Docker plus cosine action. So this will build a container image. It will cosine it and I'll add one to the number of talks that mention cosine. It will cosine it and it'll also vulnerability scan it all in a runner so you can, every time you build you can say I'm gonna create a container image I'm gonna assign my container image and I'm gonna vulnerability scan it in a GitHub action which is cool because it's free and really easy. Couple of things you need to know when you're doing vulnerability scanning in GitHub actions. First one is in GitHub actions you have a permissions block and that's what can this GitHub action do to my account. You need to give it security events right. So in order to upload things to GitHub security you need to have given the action the right to do that. The right you need is security events right. This is all stuff I didn't know when I started doing this. Second one is how do you know to tell Trivi what image to scan? So I've just built my image I've cosigned my image and those were in previous steps in my action I come to do Trivi scan how do I know what the image is called? And the answer is basically this is the interesting part. First you tell it there's a variable for registry so it knows what registry to get it from then you tell it there's an environment variable called image name so you know what the image name is and then there's the one that took me quite a lot of Googling to find which is that's the tag. So it says steps.meta.outputs.version as a variable that's how you find out what tag to scan and if you've got those three things together it will essentially after it's signed it it will go okay the thing you have just signed is the thing I'm now going to do a Trivi scan on that's the one that tripped me up for a while and what that does and you can see the bottom is it's going to output Trivi results.sarif so this is Sarif format which we're going to use to upload which we can use to upload to get it and so you just tell it what format to run and this is our get up action so there's an Aqua Security Trivi action which you can use for this and then last one uploading it to get up security this is you then uses get hubs action so get hub I've got an action for uploading Sarif and you just give it the file name it's pretty simple so those three things will allow you to get your get hub repositories and have it scan every single time and very quickly because I don't want to run out of time I'm just very quickly going to show you what that looks like when it gets into the repository it looks like this and because I don't maintain this one very well I have 1281 code scanning alerts hopefully you will have fewer but this is a useful way of keeping track and it's also a useful way if people are doing this you can look at the repository and get an idea of fuel for security like how are they getting on are they maintaining this are they updating stuff or this one is something I haven't touched in six months so you've got 1281 lots and lots of vulnerabilities yeah cool so production now we've done everything right we've built it we've put it through CI CD now we want to do security once it's live yeah so Trivi, well that's for pretty much everybody who's interacting with any development resources it's not particularly useful if you are an SRE if you're a cluster admin and you want to check your in cluster resources past deployment so once your workloads are deployed inside of your Kubernetes cluster you still want to continuously check those workloads what's happening inside of your cluster how are your container images that are already running performing over time are there any container images that you might forgot about things like that so you want to have regular scans for compliance and assurance and runtime security to attack to detect any attacks or anything else that's going on in your cluster that's fine yeah that's fine okay so who has a Kubernetes cluster locally running any form of Kubernetes cluster right now okay a few people maybe if you don't have one check if you part like if your seat neighbor has one and then you could pair up that would be an option because for the next step we will need a Kubernetes cluster and we're going to install the starboard operator who's here familiar with the operator framework in Kubernetes who has used an operator before in Kubernetes you have a few hands awesome quite a few awesome so here's the white paper that I helped review it's a really cool white paper that details everything that you have to know kind of about operators how they work what's the theory behind them they are basically used to automate human behavior at my previous role as SRE we used operators pretty much for everything to deploy all kinds of tools across thousands of tenant clusters which is pretty useful so operators we're going to install starboard now starboard is running inside of your cluster this is from a previous demo I did where I used all of those tools that you can see I used starboard to starboard exporter to have the metrics from starboard and then visualize them through Grafana and Prometheus now we're going to not use all of those tools I just want to show you how starboard lives inside of your cluster it's just another set of Kubernetes resources pretty much that you can install so you can install starboard for different ways either through a helm chart if you use helm if you prefer helm or directly through kubectl apply the resources so that command is also in the command sheet and you can go ahead and install it inside of your Kubernetes cluster that will create a separate namespace called starboard system with the operator living inside let's do that now as you can see there are lots of different specific resources let's say that starboard is installing those are custom resources in Kubernetes that basically extend the Kubernetes API and allow you to have the configuration scans from starboard stored as another Kubernetes resource inside of your cluster and if you have basically everything as a Kubernetes resource living inside of your cluster that means you can integrate starboard for example with all of your existing Kubernetes related tools so it's not a separate tool that's living outside of your cluster it builds upon existing Kubernetes resources here again you can see a set of resources that is installing a set of CRDs so for example the vulnerability reports they are ultimately used to create to do trippy vulnerability scans from within your cluster whenever you have a new deployment or whenever one of your running container images change then starboard is gonna detect that change and is gonna create a new vulnerability report for that container image yeah so here we can go ahead and check whether your starboard installation is already up and running is properly deployed now the thing is which I kind of have to mention that trippy now supports everything that starboard does because we are going through some changes so you can still use the starboard operator like we're showing you here but with the new release from trippy yesterday you can do the exact same thing that we're doing here for starboard through trippy so it's this exact same code just that trippy is implementing it now as well so now we to create a vulnerability scan like to basically trigger starboard to create a scan we need to have a deployment now if you have any other deployment that you want to spin up in your cluster or you already have deployments running inside of your cluster you can go ahead and you can check already for vulnerability reports inside of your cluster if the deployment is within a specific namespace vulnerability scans are generally namespace scoped where that deployment is running in your side of your cluster so yeah, let's deploy that simple app that's a simple react app it's just creating a deployment and a service and now a few seconds later we should get a vulnerability of a part this is where we find out how well this VM is holding up let's see how fast it happened awesome so here you can see how many high, medium, low vulnerabilities there for that deployment and whenever you update that container image there will be, well, you will see the updated list but you will see also all of the previous vulnerability reports they are all gonna be stored within your cluster so they're not gonna be deleted if there's a new vulnerability report now we also have integration for, for example, lens so you can view the reports with more detail through lens, for example Javi has lots of different extensions and add-ons so have a look see which ones are useful for you maybe now there are also cluster scoped vulnerability reports and for most of the audits that Starboard does you will have a specific namespace scoped and a cluster scoped scan however the cluster scoped scan only runs every three hours so you won't get the vulnerability report on cluster scoped and like until the next three hours at least that's what I experienced you can check later in the afternoon yeah, come back later awesome so we also have configuration audit scans so while you want to use for example Trivi for your Kubernetes configuration scans misconfiguration scans or for your infrastructure's code configuration scans Starboard also performs configuration audits from within your cluster and again they are specific resource scoped but also cluster scoped it's a nothing yeah so we have here our config audit report and then you could go ahead and you could describe that config audit report you wanna do that so you will basically get a Kubernetes custom resource definition just, yeah, provide it I was, oh, might as well not you're good and you have to, yeah, namespace, okay yeah, there you go time completion, good work yeah, so here's the Kubernetes CRD now like mentioned, Trivi is really for any engineer anybody who's dealing with development resources Starboard is really focused on class statement security professionals who want to have a custom setup so you wouldn't just use Starboard as itself you would integrate it with your other tooling for example you could visualize the vulnerability reports ingefan and ingefan at dashboard if there are new vulnerabilities pop up then you can get alerted, things like that so infrastructure scans we also do CAS Benchmark scans for Kubernetes with a tool called QtBench QtBench is also part of Accra of the open source tools and Starboard is integrating QtBench so it will create a CAS Benchmark report for every note that you have within your cluster so for every single note it will create a scan you can also, with the Starboard CLI you can run QtHunter no, we're not gonna use the CLI right now it's also gonna evolve in the future towards being integrated with Intrivi so here's how you would run the CAS QtBench reports so check how many notes you have if you have a kind micro-kates single note cluster you will have one report if you have multiple notes you will have multiple reports see if it's wrong right yeah, yeah, yeah, it's wrong is that right? yeah, because we're doing it wrong no, no, that's good and then we can just do it's all good maybe it's the next one yeah, yeah, it also work it's NSA report I was wrong it's NSA report that does per note this one does it as well okay, this one does it as well okay and tab completion for the win I thought so okay awesome so again, you can see this is a Kubernetes CRD so you would use the information from within the CRD in other tools to visualize it, to go through it one thing I just very quickly mentioned on it is you'll see that you get a fail count, a pass count and a warn count the fail count is things that tested could test and were failures the pass obviously is things that tested and were passed and the warn is things it couldn't test and we'll talk about more this on Friday so if you're interested in why you can't necessarily automate CIS benchmark completely due to the fact that it's not possible to automate with a scanner but that's what warn means warn means I'm not able to test that it's something you need to manually go look at so dear there's one other tool that we could find that also does CIS benchmark scans and it's by a private project by one of my co-workers from the open source team which is called QPecan now we're gonna go into the differences of different tools on Friday so if you want to learn more about the difference between other similar scanning tools and Starboard for instance then join us on Friday and then here the cluster compliance reports so for that to run that you have to put an add-on to Starboard through QPecan apply and then you can run NSA scans so again obviously if you're doing in production do download this manifest and read it before doing what we're doing but this is a test machine which is going away at the end of this demo so it's fine so let's see if that's run oh it's still thinking about it yeah so we are gonna talk a little bit more about this on Friday but the way it works is the NSA has a hardening guide which is not a compliance benchmark but we've picked out things out of that which we think are checkable and we think can be automatically assessed and from that we've come up with a set of checks which makes sense again I'm gonna mention the fact that any NSA scanner will probably have different checks in it so if you say I'm gonna do an NSA scan and two people run two different scanners on the same cluster it's very likely you will get different results so another one to watch for tooling is do watch for the fact that just as someone says they do something you have to go and read exactly what they mean by that so what we've got an idea and he said see if it's gonna be finished oh it's gonna be slow hey if we get all the way to the end of this and only one demo fails I wanna count that as a win chances of standing up for 90 minutes having a single failed demo I'm like I'm gonna but it may just be being slow we may wanna like we may go on in the slides and come back it's being slow there are also two types of CRDs the cluster I haven't run yet but it tells you what it's gonna do it just hasn't actually finished yet the cluster compliance reports and the cluster compliance detail report which provide more information so yeah you can actually you can see what it's going to do it just hasn't done it yet so it is thinking about it at least some time and it's yeah so we'll we'll let it do it still okay awesome so here's an alternative for NSA reports which is by Cubescape so you could use Cubescape instead to produce NSA reports now this is like the breathe overview you can get a more detailed output from Cubescape if you put like an additional flag at the end of your command so you could do that now they also have a UI by AMO that you could use in addition to the CI which we're not going to show you because we're focusing on this stuff but yeah as I said there are different tools that do this once you understand how these tools work there's always going to be and it's great to see like a decent variety of different people attacking this process because they'll have different focuses so custom policies yeah so you can also I mean we mentioned earlier so we might not have to go into too much detail on that you can also write custom policies for you to scan for misconfigurations within your Kubernetes manifest so with Trivi itself you can use Rego with TFseq like I mentioned you can use Jamil or Jason depending what you prefer if you use Terraform yeah and then it's gone in so you can yeah that's chucked out the results for custom policies so again the sort of thing is your organization might you might look at the Trivi policies and say you know what these don't work for us but they're all you know editable and you can say you know what I don't want these policies I want my own policies so cool it's not super difficult write your own policies and then you can adapt and again just adapt the basic ones if you say well this is kind of like I want this one but I want it a bit changed these are things like Jamil and Jason they're not super hard to edit you can go in and make changes so yeah so what can you do with the tools now or what can you do next after this demo well I love using observability tools from the cognitive ecosystem so I like to integrate Starboard inside of my existing observability stack to get additional details on how my resources are performing security-wise you can also well try out Trivi play around with it it's completely open sirs if you like the tool give us a star on GitHub give us feedback maybe contribute yeah and let us know what kind of use cases would be important to you which you would like to see yeah the open source team have obviously got their ideas and lots of things are coming in like secret scanning which turned up yesterday um but that said we're almost very interested in hearing what people would like to see next like you know if it's something you're thinking hey it'd be great if it did this but it doesn't and we have a question we do questions we have yeah we've got a lot of minutes we're doing all right for time almost exactly on time fantastic yeah you know there's a chat there yeah in our company we use Flux to deploy everything I just now try Trivi with one of our repositories and it doesn't follow the custom resource from Flux the customization doesn't follow the URL so do you know if there is any plan to to add it so it can find specifics so for I used Flux and one of my demos that you can find on my YouTube channel with starboard and I could see like the comparison of like the vulnerabilities that I've found within Flux versus within Algo CD for example so I know it can find the vulnerabilities from the container images so if it can find the container images then it can run the vulnerability scans on them it can't I'm not sure if it can find for custom resource definitions the misconfiguration scans but yeah I believe it can yet but again that's a great one to add right and this is the thing that the ecosystem there's huge numbers of different platforms I know that at the moment the team are working on adding Helm so Helm chart scanning but if you absolutely want to file an issue we would be great to see that because then we give some idea of what the demand is right you know if everyone says I would love to see Flux and more scanning for that then that's I'm sure that's kind of you know skip up the priority list so yeah please I don't think it does at the moment but please do add it on the repo okay thank you any other questions now here the links again to the demo to aqua security and the opens as tools you can find us on Twitter yeah if you have any questions as well if you have any more yeah that's yeah that's pretty much where we are so hopefully that provided kind of a useful idea and you know give them how you just get started hopefully I'd get I'd say what you take from that is that things like trivia are pretty easy to get started with it's not super difficult you don't have to be a security expert and don't get put off if you do if you use trivia and you know what something goes wrong or you're not understanding it there is a slack for the open source projects pop on to slack do ask us and people will be happy to try and point you in the right direction and also and this has got great stuff on our YouTube so go read go watch that because you might well find the answer in there because there's loads of good videos there and if you want to look for this we're on those places and if you want to learn more about security scans and different tools we're going to talk about on Friday at 2 p.m. yeah security standards security benchmarks so more about NSA CIS that kind of stuff we're going to get a bit more depth there we've got time for on that also we're going to go into more detail on the differences between tools like starboard and cubescape when to use which yeah join us there thank you hope that was useful