 Okay, well, we are, we are live once again, ladies and gentlemen with our every Wednesday OpenShift Commons Briefings Operator Hours and today we are lucky enough to be talking about security series runtime analysis and it's a partner chat with Aqua Security and we have a wonderful man here from across the pond, Rory McEwn, a cloud native security advocate and we have our very own Dave Muir, a Solutions Architect and he's joining us today from the home of the Patriots of the South, aka Tampa Bay Buccaneer Town. Rory and Dave, how are you? Oh, I'm fantastic, thank you. You know, when we were doing our green room getting ready, we ended up spending most of the time talking about Tom Brady and the Tampa Bay Buccaneers and then I was like, hey, where's Rory, you know, you're not an American so he probably football doesn't, you know, grab you, but like, where are you dialing in here from? And he was like, Scotland. So then we ended up talking about Scotland for quite a while and and all the great Scotch whiskeys that they have and apparently you're right down the street from the Obon distilleries, right? Yeah, indeed. Yeah, not too far from there, so I can pop over there anytime I like. I'm hoping that we can come over and visit some time. Do they do tours there? Yeah, absolutely. Yeah, like always do it open does to have done the one open. It's good. Okay, and you're from Scotland. You're not just you're not just an expat living there. No, no, I've been in Scotland my whole life actually. So in various parts of it, different parts, but now I'm out in in the West in Argyle, which is a very nice part of the world. Oh, cool. All right. So what do you do there? How long have you been at Aqua security? I'm guessing. Actually, I don't even I can't I couldn't even guess. I used to know a lot of people at Aqua security, but I haven't really been close to them for a number of years. So I've been aqua for just over six months now. You know, it's gonna kind of time flies. It's hard to stay how long times have been these years, last couple of years, but it's been six months. And yeah, I'm really enjoying it. What I do here is cloud native security advocate. So my job is basically trying to help do education work primarily, you know, education and outreach around cloud native security. And I do some some industry work as well. So things like Kubernetes security. And I help with the CIS benchmarks, you know, for Docker and Kubernetes as well. You know, just basically trying to help out and keep things moving along. Right on. So you're not going to KubeCon. We already talked about that that's coming up in LA starting on October 11th. But Aqua security is putting on a they call it a day zero event. It's called the KubeSec Summit, right? Yeah. And I actually started that with Aqua. You guys used to have your VP of Biz Dev. His name was Upesh Patel. And it was probably five or six years ago. And he was like, Hey, let's let's create, you know, a day zero event. And I got to tell you, it was like, it was a really fun working with the people at Aqua security. But but that event is actually become a really important way for people to get, you know, updated information around how to secure your workloads in a multi cloud. So hopefully anyone who's going to be coming or watching KubeCon here up in LA should check that out. It should be really good. Yeah, we've got a great lineup this year. You know, we've had a lot of I've been helping with the CFP and I got talk at it as well. So yeah, we're looking forward to KubeSec and and KubeCon. You know, yeah, I'm really sad that I'm not going to be there. I had planned on it. I've been waiting for, you know, the US government to finally say it's okay, you can come from the UK, but it's just I had to accept eventually it wasn't going to happen. I think I think you know something you know, recording is on. It is what it is. Yeah, Dave, mirror over you down down south. What's what's cooking? What are you nerding out on today, Dave? What am I nerding out on besides great sports teams down here? No, we're going to be nerding out in this session about some reports that came out both from Red Hat and Aqua and talk about some of the what was said, how some of the Aqua reports matched what Red Hat says. So it's going to be a pretty cool show we have. It's all around, you know, security. Before we get into that though, let me just share this screen if I can. Yeah, here we go. And this, as you know, Mike and as Rory, you know, this is part of our monthly security series. Just wanted to let everybody know that we're doing a lot of content on a monthly basis called DevSecOps is the way, but it's basically three podcasts per month to Red Hat live streaming shows. This is one of them. We have another one in a couple of days as well. And then we put out a blog and as much content as possible on the specific subject or category that we have planned for each month. And man, it's already September. So we're winding down here on our framework that you see here. We've actually created a framework around these different categories, vulnerability and configuration management, compliance, identity. And this time is all about runtime analysis. So we've got content and shows and podcasts, all about runtime analysis this month that you can you can look out for that should be dropping here in the next couple of weeks. You can see some, you know, sites there on the left aquasec.com. Obviously red.hat DevSecOps where you can get more information as well. You know, the first time I met you, Dave, was you were working it for Black Duck software. And we were at DockerCon back when DockerCon was the thing. And I think it was Austin, Texas, maybe DockerCon used to be pretty big and Black Ducks was all was was always there. But security has been in your blood. You know, you were at Black Duck and you guys were bought by Synopsys and now you're over here at Red Hat. And you've been doing, you know, talking about security the whole time I've known you. Has it changed much? And I can leave this open for Rory and or Dave, like, you know, back then it was about how do you secure your containers, right? How do you know that the code that's in your containers is, you know, not going to introduce, you know, vulnerabilities inside your containers. And now, you know, fast forward to, you know, here we are today, it's all about securing your workloads in in a multi cloud world. Is it ever going to, is it ever going to change? I'll leave that open to either of you. I think it always changes, which is the constant changes always constant, especially in security. You know, back then, the Kubernetes wasn't really, didn't really have a large hold in production systems. But as it's grown, and we see more companies now, with critical applications in production, using Kubernetes, using OpenShift and containers, then you really have to think about not only just securing that container, but the entire dev and ops lifecycle. And it's absolutely different from like, you know, legacy, I call it legacy classical type of web applications that sit on a server and you have endpoint security, things like that. With containers and Kubernetes, there's a lot of different things to think about. That's one of the reasons why we at Red Hat created this framework that you see on the right, it's actually much more than these, you know, 10 categories. It's a way to help our customers understand what they need to start thinking about when implementing security, and the different integration points. So like, for example, obviously CICD in a DevOps world is very important. So what security methods or controls, did you think about when you're integrating and creating your pipeline? And while these are the main categories, there's like 34 odd different functions that we've identified underneath this, to help our customers understand DevOps and DevSecOps, so they can start, you know, implementing security and what we call sort of a layered approach or defense and depth strategy. Cool, Rory, what are your thoughts on it? I was going to say, yeah, yeah, so well, it's fascinating for me because I got into container security in 2015 around that time. My previous life, I was a pen tester, security consultant, and I started up seeing Docker and people putting Docker in. And I got the idea this is going to be popular, but then it was simple and it was early adopters. So you have these kind of early adopters and they had quite simple setups. You know, they had a couple of containers running on a server and that was fine. You know, you had to worry about what was in the container, but there you go. And then it's just getting more and more complex over time. You know, we started getting early adopters and Kubernetes, you know, some companies getting it in like 2017, quite early doors. But then you see the cloud native landscape starting to build up and you've seen that map with like a million different products on it. And that's just been developing and developing. And so for me, it's just the story of, you know, increasing layers. You know, you've got your container, then you've got your orchestrator, then you've got your service mesh, then you've got your CI system off the side feeding into this whole thing. And then now people are, you know, we've seen surveys saying that multi-cloud is becoming a reality for a lot of large companies. So it's not just now in one cloud. Now you're saying, how do I make sure that my container is running in one cloud are going to be as secure as my container is running in another cloud when the controls are different, right? The kind of things made available to you in each cloud are not the same. And for me, it's just, it's just the story of increasing complexity. And it's been very interesting. But, you know, I think it must be really daunting for people coming into this world now. You know, I've come into, I think you both did as well, you've been able to gradually build up those levels of experience as things get added. But if you get dropped into this now, I see companies coming in and they're like, you know, where do I start? Where do I go here? And that's, that's, you know, a big challenge, I think. OK. So Aqua, you guys have been around probably quite some time. As I said, you know, we were working with you folks for several years. You guys have been a pretty big name in the security space and it seems to be, you know, continuing to be that way. Dave, you mentioned that there were some reports. Is this something that, you know, is it like an IDC study that you just decided to throw some money at? Or what kind of reports? I mean, when I think reports, I think, oh, my gosh, I think I'm going to go make some lunch. Well, yeah, and I think to answer that, we're not going to, we're not going to go through these reports line by line. I think what we're going to show or talk about is pretty interesting. There's three reports that we're going to reference today and one of them was one of them was done by Red Hat. You can see on the left hand there, it's a state of Kubernetes security report a couple of months ago, we published this and Red Hat does this I think semi-annually or annually, but it's really focused on how companies and our customers are adopting Kubernetes. And then we're going to take some of the aspects of that report and talk about how they relate or don't relate to the two aqua reports you see on the right hand side there. So the one in the middle is the runtime understanding cloud native runtime protection security gap. It's a good report. It's focused on actual real challenges that folks are facing with cloud and runtime and Kubernetes. And then the report on the right is all about threats and it's pretty cool because it collects data from these honey pots. Honey pots is a new word actually. Just discovered a couple of weeks ago, but it's really neat how the data has been collected to understand all the threats that are occurring with Kubernetes. I think I got that right, Rory, is that... Yeah, absolutely. I know what you mean about reports where you go off and you get launched, but I actually find things like this really interesting because there's a tendency in cloud native and in product companies to assume that everyone's super advanced and everyone's off building these amazing things, but you really have to go out and ask real practitioners and say, when you do that and when you say to people, what are you actually feeling? You find a lot of interesting things like people aren't necessarily off in the super advanced, they're still worrying about the basics. And it's helpful because it feeds that narrative of well, whilst advanced features are great and they're useful to have, it's also good to think about where the basics are and where are people really finding challenges. And I think this kind of survey-based reporting for that side of things is really useful. And then you've got these threat reports and again, this is the other side of the coin, which is we hear a lot about, you know, if you go to security conferences, you'll find these amazing attacks that do you have marvelous stuff, but when it comes down to it, what is actually being attacked? What are the people who are trying to compromise systems really doing? Because that's where you should be focusing your effort. You need to focus on the stuff that's actually going to attack you. The other stuff's kind of cool to go and watch a conference talk, but if I'm in the security department, I've got limited resources to spend, I want to focus on the stuff that's really going to hit me. Speaking of security conferences, a couple of years ago, I went to Black Hat out there in Las Vegas. I'd never seen a conference like that. I mean, they were basically telling everyone to not bring your wallets with you because there were people setting up RFID scanners and basically, so we left our credit cards and everything in our hotel room and just walked around with cash for four days. I've done the Black Hat and Defcon thing, I think like two or three times now, and Black Hat's not too bad, honestly. Defcon, yeah, I would be, I am careful when I go to Defcon. I wouldn't connect to random Wi-Fi, and when I did Defcon, I did make sure to harden my laptop, and I actually remember I was going out there the first time I'd ever been to Defcon, and I'm in the airport and a vulnerability comes up in DHCP, the one thing you can't harden, because it's going to happen before your firewall goes up, and I was actually in the airport changing my configuration, so that when I got to, because I knew people would be trying to compromise this stuff, and I knew that's exactly where that happens. But isn't Defcon, isn't that part of Black Hat, aren't they like co-located at the same time? They happen at the same time, but they're not the same, so I also think of Black Hat as like corporate, so Black Hat's a corporate conference, you know, the people go there, they're paying thousands of dollars a ticket. Defcon is very much the wide, most of the kind of traditional hacker thing. Even this year, you could turn up with cash, and they don't need to know who you are, you don't need a credit card, you turn up, and people do, they turn up with cash on the door, hand their cash in and provide no, like, identification whatsoever, so that's more your traditional, you know, hacker event, whereas Black Hat is cool, and they have a lot of interesting research, but it's more your kind of corporate style. I did some training at Black Hat, and that's, you know, it's more that kind of world. Oh, okay, right on. Anyways, I didn't mean to digress, I just, it was, it was an interesting four and a half days when we were out there. Yeah. It can be either one of these conferences. I had a question for you. You know, you guys were talking about these reports that you're done and that it's important to bubble up this important, you know, disinformation so customers can, you know, learn from, you know, others, others' mistakes or, you know, learn, pick up best practices on how to secure workloads in your, you know, DevSecOps, environment stuff. How mature are we in the security space? Like, are people still learning? Is it, are we in its infancy? When do you think everyone will just get it? And we won't need to be trying to educate people on how to secure environments? Ah, I've been in security for 20 years, just over 20 years now, and I'm still waiting for that to happen. Because technology moves so quickly and security is, to an extent, always playing catch-up. It's really interesting to me because I submit a lot of CFPs to conferences. And I find that it's much easier to talk, to go to a developer conference. They're far more open to someone coming to them and talking about security. If I try and go to a security conference and talk about, like, containers or DevSecOps, it's often not like, you know, you don't get take up there. So I think there is, and it's actually something which comes out in the reports. There's a bit of a gap there where maybe security departments aren't, you know, they're having difficulty in, like, keeping up with what the state of the technology is and what to do in these modern environments. So I'm not sure we'll ever get there, just because Kubernetes and the CNCF move so quickly, you know, we're still, when they slowed it down, so it's one Kubernetes every four months instead of every three months. But in the enterprise world, you know, that's where they deal in years. You know, I said I'm installing the system and it'll be, you know, make a year to install it, and by which time there's been three Kubernetes releases. So I'm feeling that security will be playing catch up for a while yet. I hope so. Okay, good job. Yeah, whenever I see things, I'm almost like really glad I'm like, well, that's my job security for the next period of time, because this is going to be horrible, which is a bit of a cynical view. But, you know, that's just the way it is, I've found time and times. Right. And I think the other thing that she has, oh, sorry. No, no, finish your thought. I just, I wanted to ask you something else. Yeah, I think the other thing is that there can be, the other thing that plays there can be a bit of a bubble is in cloud native world. What I find is that sometimes, you know, you'll have these conversations with people in the actual projects, and they assume that the customers all know the details of exactly what is in their stack, like they assume the customers know what RunC is. The RunC is a low level container component, and I guarantee you that most people running Kubernetes don't even know they have installed. So there was a vulnerability in it a couple of months ago and I'm not sure most companies knew that they had to patch, because once you're sitting down there, write the depths of the world and they never install back. If they installed cryo or they installed Docker, they didn't install RunC. So there's still a knowledge gap there as well between the projects and I think the customers. Let me ask you a question about the sort of the initiation of vulnerabilities if you remember four or five years ago, there was spectre and meltdown spectre. And heartthrob, what the heck was that? Heartbleed, yeah. Heartbleed, yeah. Spectre and heartbleed. And that was actually a vulnerability introduced way back when computers, if I get this correctly, when main frames were first made, because it was a speculative execution and it helped speed up processing. And then that just kind of came along with how chips were designed, made it into x86 and x86, 64 and was basically just dumping information into the equivalent of an unsecured trash can and there would be all kinds of stuff in there that people could. So that was a pretty amazing security problem and it was fixed by Red Hat and Google and a bunch of other companies that got together and fixed it really quickly. But what about the vulnerabilities now? Are there security problems that are native to what are containers or Kubernetes that have kind of been around a long, long time and are only now becoming a vulnerability because of the way computing is changing? Yeah. So I think it's actually straight up that this ties directly into the report. There's an ongoing debate about do containers create a security barrier? So is a container a security barrier or is it not? And if you talk to a lot of security people, as time goes by, what we're finding more and more about how difficult it is to create this sandbox with standard Linux features is more and more security will say no, it's not. You know what? We've seen enough vulnerabilities now. This is hard to create a proper security boundary. We wouldn't put our money on it. But and that's something which I think is coming out as we get more research. That runs the vulnerability I mentioned a while ago. That was a race condition. And it was a race condition when you were mounting volumes into a container. But that turned out to be exploitable in Kubernetes quite easily. So, you know, it's that kind of thing that as more research is done and as more people as more security researchers are drawn to the field of containers and of cloud native, you're going to find more and more of these exploits coming out. Anytime there's a field that actually security will turn up and they'll start poking away and digging in and they'll find interesting things that have been like buried into the layers that is cloud native. So if I can jump in here, Roy, that's great because that's that's, as you mentioned, one of the things we wanted to talk about with the reports. If you could talk a little bit more about because I found it very interesting about the whole containers being a security boundary and only 3% of folks said it was not. I always remember it was down wall. She's called containers don't contain in the early days. That was one of these statements, you know, he came out to get tears on contain and at the time when I first was in containers, well, they kind of do. And to an extent, they do right against some attacks. They're not too bad. But I think it's fair to say that time is proving him right. And as we spend more and more time poking at the way the abstractions work. And because you've got these layers of all these different programs, it's really difficult for them to align. So you've got the run C project, which is owned by one group. Then you've got cry one container owned by different groups, then you potentially got Docker, then you potentially got Kubernetes and trying to line up all of that to create a security boundary is hard. Where you've got like other projects, maybe use like virtual machines like AWS wirecracker or Cata containers, they're designed to be security boundaries. You know, those things are dedicated or G visor, which is a prop, which is a design to be a sandbox. And I think that is a tricky one because yeah, it was surprised to me when I saw that survey result and it said 93% people think it is a security vendor. I'm like, well, you know, I'm not sure if it was me, I would put my, I would put my highly secure workloads. It's fine for some things, but there are definitely a level. Everything in security is no absolute, right? You know, I can't say it's not insecure because it's not, but I also wouldn't say it's perfectly secure. And I would say that the attack surface there is quite large and hard to get right. Yeah, and you mentioned Daniel Walsh, that's why I see you. I was going to say you brought up Dan Walsh. I don't know if everyone knows who Dan Walsh is, but I'm actually I'm actually impressed by how many people do know who Dan Walsh is. Dan, Dan was one of the first employees in the Red Hat office when we started here in Westford Mass. There were probably 12 of us in the office. And later on, you know, he became Mr. S.C. Lennox. And I used to travel around with Dan all the time. And he couldn't, it was a consistent theme that he couldn't stand it when people were like disabling. That's like the first thing they did when they stood up at Lennox's box. So I was like, that was Disable S.C. Lennox. And it would always drive him mad. But now he's Mr. Container, the S.C. Lennox throne has been handed off to somebody else. Those folks probably thought containers were a security boundary, but they didn't need S.C. Lennox, right? Yeah, and it's interesting to me because dockers, the way it does and the way it creates this little it's essentially the isolation it provides is all these different things. Like it's S.C. Lennox. So you can apply an S.C. Lennox profile and should provide an S.C. Lennox profile to containers. But when you put that into places like Kubernetes, for example, Kubernetes has disabled one of those layers, which is second. And people like, again, I'm not sure they always know that like Docker has the second filter, which what that does is it filters certain syscalls to the kernel and says these are dangerous, you know, don't allow them. But if you run Docker or anything else under Kubernetes, Kubernetes turns it off and only recently gave you the ability to turn it back on again at a cluster level. So there's all these different details that I'm not sure when people are like doing the surveys. I'm not sure we've done a great job of communicating, you know, exactly how this this thing is built and where the gaps might be. Hey, by the way, I thought I'd throw a plug out there. You know, we are streaming live on YouTube and Twitch and other places here today. If anyone is watching on any of those channels and you wanted to share a comment or ask a question for Dave or Rory, you can drop it in the chat and our bot will automatically pick it up and move it over here to our interface. And today we'd like to we'd like to call it Stump Rory Day. So we're throwing that the challenge out there and he's offering up a tour of the obon facility as soon as the virus is gone. Yeah, as soon as the virus gone, that's a safe one. We'll offer that. Yeah, probably so. So there's a ton of security vendors out there, right? I mean, there's, you know, from like end point security, like Dave mentioned from, you know, like a like a McAfee, which does end point security or others. And then there's secret security, you know, you get cyber arc who does secret security and a whole host of others. What part of the security story do you folks address and how do you do it? And and the second part of that question, or maybe I should save it, is like, how many different security vendors to customers need before they're secure? Oh, so I mean, let me give the easy part first, which is where Aqua comes in. So Aqua is focused on cloud native, right? So the idea is that, you know, we're looking at cloud native workloads and we're looking at all the different parts of that. We're looking at containers. So these are the fundamental blocks that you're building around. You need to have your security in there. We're looking at your clusters. So, you know, once you build these things in containers, you're putting them into clusters. You need to worry about Kubernetes security. You know, Kubernetes security that I've been dealing with for quite a long time is not out of the box. A complete secure picture, right? Which is one of the reasons why OpenShift is so great because it adds all those extra pieces you need. But we're looking at Kubernetes security, but then we're also looking into the cloud security as well. Because realistically speaking, when you're deploying your Kubernetes clusters and your containers, most of the time you're deploying it into a cloud and you need to worry up there as well. And you need that picture across your cloud, your clusters, your containers, up and down. But then you also need, like, through your lifecycle security because we're doing, like, DevOps, we're doing Agile, we're doing things quickly. You've got security and development, you know, analyzing my images early, making sure I've got the right hardening in place, but then you need it in runtime. And that's kind of where this report's coming in. You need to talk, worry, about when you're actually running those things. If someone or more, let's be real here, when someone breaks into one of your applications, will you spot it? Does your tooling understand containers? And knows what a container is so that it can say, hey, it's this application from this cluster instead of just giving you something which, like, means nothing to container land, like a host agent might do. You need something which understands your world and understands cloud native to give you the good information. And it's that kind of thing. So it's across the lifecycle, but it's up and down as well. But it's focused on cloud native, right? But that's where, like, a lot of companies are moving to these days, which makes sense. But how does it work? I mean, you know, like, what you just said sounds like you guys are miracle workers and that you can secure one of those. No, over here. Is it like, are you guys like a data dog where you have where you're a SAS and everything phones home to get instructions on what to do to the mothership? Or do you have agents that have to be installed everywhere? How does it work? Yeah. So there's a different couple of ways you can do it. I think that one of the things we're finding over time is the fact that that's evolving as customers of all. So we do on premises, you know, you can do it. You know, there are enterprises who do not want to be in the cloud, right? They want to be on premises still and that absolutely makes sense for some companies, but also obviously you can do in the cloud. So you can do your own installing in the cloud or you can do SAS and SAS is something we're moving more to. And that's because of customer demand, right? Customers want SAS services for better or worse, that's how they like doing business. We do use agents for a lot of our work and the reason is a really great reason for that, which is you can't block, you can't take active defence if you don't have an agent. You know, you can alert and you can say, hey, I've seen this thing, but if you're not running code on a VM, you're not blocking anything because how do you do that? You need to have so it's I think it's about giving that customers that flexibility. Some will want agents, some will not. But for my money anyway, you need agents to really get like protection rather than just detection. But doesn't it? So if if if part of your security solution is based on agents and the Kubernetes is revving, you know, multiple times of whatever, a period of quarter or week or whatever, how do you keep all of your agents current given the rate of change in Kubernetes and computing in the cloud? Yeah, you do have to have people kind of working flexibly to upgrade themselves. But what I'm saying about like Kubernetes does rev, but doesn't mean customers do. So one of the things that I've been doing like I've done personal research on is there's a great search engine called Census and it indexes all the publicly visible Kubernetes clusters on the internet. I'm sorry, it's called Census. Census, C-E-N-S-Y-S. Census Bureau. Yeah, but with a Y. But anyway, what they'll let you do is you can query their dataset. And so I've got a query that runs every night and it pulls every single publicly exposed version number. So a fun thing about Kubernetes that again, a lot of customers don't know is Kubernetes enables anonymous authentication so it lets you hit the API server without credentials and it exposes a couple of endpoints by default. One of them is slash version and that gives you the exact version and date it was made. So you can just scan the internet and say, tell me all the clusters. And what I found was there's an awful lot of people who are not keeping up today with Kubernetes. You know, we're talking like 20, 30% of clusters and given cloud will be running unsupported versions and there's still people running like super old versions. I mean, I actually saw on the OpenShift side, I think I saw a couple of clusters running like 3.2 which has been out of support since the arc. And there are people still running on the internet. As you said, customers don't like change, right? I mean, it's, we, I mean, there's still customers out there running systems, running OpenVMS on alpha server 4100s, you know? But they used to work there. I used to work for banks. So I worked in the banking sector some time ago. And yeah, I also remember that my very first IT security job was in a brand new shiny internet bank. Early 2000, we had a Solaris-E10K. We had all the kind of modern tech. But what was the big conversation every morning in the ops meeting was mainframe batch time and was how much will this shiny new thing impact mainframe batch? And it's probably that mainframe is still there today doing the same thing. So yeah, sometimes changes thought to happen. Okay, so second half, like, sorry, Dave. I was going to go ahead. I've got a stumped the Rory question once you're done. Oh, I just one more question because I'm just curious. Like, so how many security vendors do customers need before they're secure? Right? I mean, it can't be just Aqua, I would imagine that like, do you do, you know, do you do secrets management? Do you, you know, what about end point and so forth? Like, isn't there, shouldn't there be some kind of like a security committee or something like this where so customers say here's like the 10 or 12 security pieces that I need to worry about. And here's the vendors that have the best in class solutions for them. Yeah. Well, so yeah, you get back to the age old question. This is what it goes on forever with ideas. You know, you can take point solutions and you can say, you know, you can get a vendor that specializes and they go super deep into one area. The challenge with them is can you make them play friendly with all your other stuff? You know, with all your installed software and a lot of times that's a challenge, right? Because that vendor is so laser focused on what they're doing or you can take a vendor who has got a more holistic picture and is like giving you coverage of the areas you need. Now, you know, with my old school IT security hat on, I'm never going to say that one vendor can fix all your problems even though I work for one because I don't think that's realistic. Right. But I do think that, you know, for me it's about trying to find like solutions which match your environment as closely as possible. But yeah, sure. I mean, you know, there's areas we don't work in and there's areas we wouldn't try to but I think it's about trying to cover an area that makes sense. And to me, the kind of cloud native world feels like an area you could conceivably cover, right? Without trying to get into like, you know, laptop protection or anything, you know, strange like that. We're not trying to do laptop protection. We're not trying to do that world. But you can see cloud native is kind of like a coherent whole that it makes sense to address as a product set. Okay. Yeah. And I was going to add to that, Mike. I mean, this relates to discussion we had in the beginning is there's so many different types of security functions and a lot of times they're not really related. So if you're really knowledgeable, for example in third party dependency vulnerability management like Black Duck, like Rory said, they go very deep into the third party dependencies. Well, that's just one of 30 plus different types of security functions that you, you know, are thinking about when you're building a DevOps pipeline. So, and that's even on the application side, there's a bunch of ops type security like network visualization and, you know, behavioral analysis, things like that that are a little bit different. They don't have different concepts that require, you know, different skills and different technologies. But I don't think there's ever going to be, you know, the one ring, right? To rule the mall because you'd spread yourself in, I think, if you just had one tool. Well, I mean, so what about the gas pipeline hack or things like that? Like you have these big companies, it was, what was it six, seven months ago or something like this? And someone did a malware attack on some distribution pipeline of was it oil and gas, I think, which shut down, you know, half the Southeast for some period of time. When things like that happen, is it because someone is being negligent in that company from, you know, maintaining good security practices or are there just always so many new vulnerabilities coming out that the bad guy always has the one up on the customers? I think, you know, for me, and this is again, come back to my kind of like long-term security, this is what you probably found with that one. I think that a lot of these hacks is what actually went wrong was probably something quite basic and it could be bad credential management. It could be, people, a lot of people will like put like their remote management service on the internet because it's easy, right? And I can just like get to, I don't always worry about any fancy software like VPNs. And it's getting the basics right. You know, and if you can get the basics right, you're doing better than a lot of other companies as an organization. Am I getting the basics? I mean, for me, job zero in security is do I have an asset inventory that I can see where all my information and assets are and I know who owns them and what state of patching they're in. And if you went to most large enterprises and asked that question and said, can I have that to now or in the next hour? I think you'd be surprised how few, or maybe you wouldn't be surprised how few of them could answer that question. So I think there is a big mismatch between, again, this is like you go to a security conference that is all super advanced elite AI blockchain, whatever. But in reality, where most companies are, is they're really worrying about, I didn't know that someone in a branch office has installed a new internet connection so that they could support some third party software, right? And that's where they've ended up having a problem. And this mic relates to a lot of the data that we find in these reports that we've been talking about on the Red Hat side, there's a statistic in that report that said 94% of the folks that took that survey had a security incident already. And it seems to agree with some of the Aqua reports where Aqua was saying, well, the attack volume is up 26%. I think what Rory was saying as well is that there's new ways to attack. It's not just your traditional, I don't know, pen test type of attack where you're seeing what's worked. It's other things like CICD that they're trying to attack, go into GitHub or something like that. Is that right, Rory? Like there's all these new different lenses there. What you're seeing now is you're seeing like the, so there's a couple of things. One, you're seeing increasing complexity and increasing speed, right? Everybody's using super, super fast. If you want to do something super fast, it's hard to also maintain security while you're going faster, ever faster. I mean, if you look at the CodeCova attack, that was attack early on this year, and that was a supply chain attack on a company. They provided like a GitHub thing for checking your code coverage. So you added it to your GitHub project and it checked like, you know, how well your code was written, but they got hacked. And what happened was then when you went and called out to their software to say, hey, go and check if my software is okay, what you got was not what you wanted, what you got was malware. And it's that very complex environment where, you know, all you did was add like a tick box to a piece of code and suddenly someone's got access to your environment, right? That's super complex. But attacking things like supply chain CICD systems, which again gets mentioned in the reports, is an increasing problem. Because if you think about what CICD system is, it's an environment to run code, right? And if I can run code in an environment, I can execute things like a crypto coin mining software or I can execute something which will encrypt all your data like ransomware attacks. There's actually code execution environments and that's what Kubernetes is. That's why attackers love Docker and Kubernetes, because I always call Kubernetes remote command execution as a service, because that is literally what it is by design. It executes commands on systems. So it's RCEs a service and that's what attackers love because if they got RCE, that's the holy grail for an attacker, because then they can do whatever they want. You know, I was scanning over the report that was about attacks in the wild on the container supply chain and infrastructure on the black one. And you're right, Dave, they use this word honey pots in there. So I want to ask what that is, but one of the charts right at the beginning, which basically shows like attacks, attack trends between June of 2019 and December, 2020 have gone from 2,500 in 2019 to 13,000 to 17,000. I mean, it's like, it looks like the coronavirus curve back in two April's ago. I mean, is this, should we be, this is frightening. Yeah, so what this is is a couple of things. So first honey pots, essentially a honey pot is an intentionally vulnerable system that you place usually on the internet with the design of hoping to find out what an attack is. And there's a number of different ways you can do that. You can just put a virtual machine out there. In this case, what this is, is we put virtual machines on the internet running Docker by exposing the Docker socket, like on the internet, and then you see people attack it. But you can also do fancy things like you can, you can leave like honey, they call them honey tokens and you leave a string like an API key and a URL somewhere. And then when someone grabs it and tries to like use it, you get an alert saying, hey, someone has found this in GitHub or wherever else you put it. In terms of the volume of attacks, to me, this is like inevitable. Anytime a new technology comes along when companies are like in the early days and they're struggling to work out how to do it securely, attackers go, hey, here is a good source of compute power. Here's a good source of easy to compromise machines. And they are very quick at moving these days. I mean, if you read the tax on like, you read documents and reports on how like commercial malware gangs operate, it's a business. You know, they've got like developers. They have got like, you know, salespeople who sell the kit to other people. They have got a supply chain. They've got all their own stuff. It's commercial operation. And so it's not a long before, and Docker, as I said, like Docker is a command execution as a service. It's a great target, you know, because it lets them run their malware. It lets them run their, whatever it is they want to do, crypto coin mining, whatever it is they're looking to do that day, Docker will let them do it. And yeah, but in terms of complexity of attack, what that's tracking is someone making the canonical mistake of putting a Docker TCP connection on the internet. I'd love to say that no one made that mistake in real life, but I used to be a pen tester and I did see companies who did exactly that on their internal networks. So I'd scan the corporate network, I'd find the Docker socket, I'd run one command and I'm root on the machine at that level of complexity. But it's really, it does happen. I've seen that happen on multiple occasions. Okay. Well, we have 11 minutes left. And the last thing I want to do is have our video, our streaming producer, Chris Short, give me a poor rating. I need to get my blue star on the refrigerator. Where can we, where can, you know, a couple of things. If you're VP of marketing at Aqua was watching, and he calls you up after the show today and he or she and says, I can't believe you were on there for an hour. And you didn't talk about the following two things. What would those be? Oh, gee. I will get the Cubasec Enterprise Summit, which we've got coming up. So I must mention that, you registered it for very kindly. We have got that coming up. It's going to be a virtual event this year, but absolutely registered that. We've got some great talks. So that's, that's definitely worth looking at. And that's a, and Cubasec Enterprise Summit is the day zero event just before the QCON. So that's on the 12th of October. Yeah, so we're doing that live, but virtual. So that's going to be happening. And I think the other thing would be to get the reports. You know, we've touched on, you know, some of the good themes from the reports, but there's a lot more information in there. And I think to me it's a, this kind of stuff is useful because if I'm a company and I'm using Kubernetes, and I'm trying to convince my management that, you know, hey, you need to invest in security, then honestly, that's when I was a security person, like an offensive, this is the kind of information I needed because I'd say, look, here's real stats. Here's real practitioners in the real world telling us what their real problems are. It's not a marketing department. You know, these are surveys. We have the data. Same with yours, with your own as well, right? You know, this is, this is useful because it's like hard data. It's like, you know, this is real people telling you real things. Right on. You're sharing those reports, right? Yeah. Dave, you said you had a stomp the Rory question. I'm dying to know what it is. Well, yeah, I was going to ask, in one of the reports, it was the middle one here talking about the knowledge gap. I found it interesting that the more experienced folks were less confident about being able to secure, you know, their environment. So, I don't know, why do you think that is? Oh, yeah. So that's because the more you look into container security, the more you realize you need to look into container security more. So I said, I've been doing it for six, five, six years now. And I still find out stuff I didn't know about Kubernetes, you know. And as you look more into it, you realize quite how complicated it is. You're never going to be able to roll out a container cluster and like, get it just like, bang, that's it, secure, I'm done, I'm fine. And I think when you come into it, you may be an assumptionist, like, you know, if I'm getting a product from major vendor, can I not just install this and it'll be fine? And it's like, you know, no. No is the answer to that. And I think only once you get, like, start digging into it and you start running containers in production for a while, and you're like, oh, oh, these are the problems I'm going to have. Like, this is the challenge I'm going to have with vulnerability management, trying to keep all these hundreds of images up to date with the tidal wave of CVEs, you know, vulnerabilities that I get every time. And when you start trying to do things like admission control, so, you know, trying to stop people just misusing their rights to Kubernetes is not trivial. And so that's where I think, you know, I always think, you know, if you talk to people who've been in security a long time and you say, is something secure, they'll never ever say yes. Like, you know, no experience security practitioner says, yeah, that's secure, because that doesn't happen. Whereas some of these new security might say, yeah, you know, they might say, oh, do this and it's secure and it's like, no. So that's my take on where that came from. Yes, essentially saying the newbies really don't know what they're talking about. Yeah, and I think there can be an assumption, right? Because like, you know, you're getting products from major vendors, that that's going to be perfect. And, you know, again, with my pentester hat on, I know that's not always true. You know, I would do pentests on big organizations with big products that, you know, and I would find things that surprised me, right? You know, I get surprised about doing a pentest and go, I can do that? How does that work? And that's because, you know, people make assumptions about how things are going to be used, right? You know, you're making a product, you put it out there. You assume that customers are going to use it in a certain way. And then when they get it, they don't know that that's what you assume they're going to do and they do something wildly different. And you end up with like, you know, things that make pentesters have a good day. Cool. The other thing, I know we have a little bit more time. You know, mind Mike? Yeah, just one more question because I thought it was interesting. You know, when you think about application security, it's all about vulnerabilities and identity. Those are kind of the top two things you got to worry about. But in Kubernetes containers, what I've seen across all these reports is that bad configs and misconfigurations are kind of the primary reason. I mean, are you seeing the same thing? Yeah. Config is, so Kubernetes config is hard. One, because like there's 130 different ways to install Kubernetes. So, you know, it's hard to say what does good look like? Is it really tricky? And it's a problem we had with the CIS benchmarks. But also because things are complicated and kind of non-obvious. I mean, a really good example of this is there is a hard-coded group in the Kubernetes source code called System Masters. Any user who is a member of that group is cluster admin, regardless of what you do. If you remove every single right from that cluster, it doesn't matter. That user is still cluster admin. And that piece of knowledge is not widely known. So, kind of, it's really easy to make that mistake and make a misconfiguration error and give more rights than you needed to or expected to. And when I, again, if I kind of come back to my time as a pen tester, misconfigurations was the number one thing we'd pull people up on and say, you know, you have configured this person with far too many rights and that means that they could do these things to your environment that you probably didn't want to happen. And the more complex the thing you try and do with Kubernetes, the harder that can get. Cool. Hey, let me sneak one more in. There's lots of security vendors out there that have been around for a long, long time. There were some really big names. I think it was like, you know, Symantec and others that had, you know, enterprise, you know, security solutions, you know, 30 years ago. Where did all those companies go? Are the companies that are best suited for providing security in a multi-cloud? Are they all new? Are they all like built in Kubernetes for Kubernetes? Or are there any of the legacy security vendors still out there that are, you know, have mainstream offerings? I think it's a tricky one because I think the problem that the traditional kind of older vendors is moving to a new category and seeing that in advance and saying, hey, look, that's going to be successful. Whereas, and if I use the term, it's not in their DNA, you know, it's hard for a company. And obviously there'll be acquisitions and sometimes that works out, you know, as you know, and sometimes that doesn't work out. But I think that what you tend to find is that companies who are like built around the premise of this is the area where I'm trying to address will have the most success because it's their company vision. Right, it's not like some division or like some project that I'm doing that maybe I'm not, gets funded. It's like, this is where we're going, this is the thing we want to do. And that will be my view of it is, and also a lot of these other vendors, they might be happy with the area they're in, right? They might say, you know, I'm happy with doing whatever it is I'm doing. I don't feel the need to try and get to this new risk area that frankly might not take off. You know, if you go back even four years ago, people would have said, well, Kubernetes, will it take off, won't it? It'd be a gamble. And as it happens, obviously this time it did, right? If you bet on some other technology, maybe, you know, you spent a lot of money on a project that doesn't go anywhere because it didn't take off. And we're seeing of course over the last five or so years, we're seeing some of those bigger players acquire, you know, these newer type of security vendors that have a focus on Kubernetes or containers, just because obviously it's easier to buy than build these days, because it's going so fast. Yeah, I mean, you'll always see that. That's gonna be the way things will go. But I think with any market, right? There's always gonna be a period of time where it's like huge growth and then there's gonna be a period of consolidation and you know, exactly how that plays out will be one for looking at where they go along. Okay, well, I know when we were talking about doing this, having this fireside chat, if you will, you know, there was a conversation like, okay, initially we're like, well, how many slides do we have? And, you know, I was kind of joking. I was like, well, let's keep it under 100 slides, you know. I think it's really great and refreshing to be able to have people like yourselves from Aqua on here and not have it be a presentation of, now let me show you my slides. I think it's, and I think it's, I know for me it's a lot more fun. Yeah, it's sometimes nice to not just have to do like a fully structured, like I'm just gonna rattle through all this stuff. Let's actually have a chat and see what, you know, what's interesting at the moment. Yep, we got a minute and a half. Dave, you wanna plug your DevSecOps security series again? Yeah, thanks. And I did get it under 100 slides for you, Mike, we only have three, so. Well, that's actually good because you've got 90 seconds to run through 100 slides. So go. Right, but no, again, this is part of our monthly security series that we're doing every month around certain security categories under DevSecOps. You can see this month is all about runtime, runtime analysis. And we're delighted to have Rory and Aqua talk to us about all the threats and those reports we had, but stay tuned for other publications. I think I mentioned the next show is in a couple of days. It's actually tomorrow and we've got three podcasts that we're planning to drop this month as well as a blog that talks about this category, the security methods within it, the integrations. But again, for more information, you can go to those links or that email on the bottom left. And I think I did that in less than 90 seconds, but I want to thank Mike. I want to thank Rory as well. It's been a pleasure. Mike, you want to close this out? I think we're good. It's been a pleasure. Rory, I'm sorry, we're not gonna be able to get to meet you in person. I will be in Los Angeles. I will be there hosting an OpenShift Commons briefing day zero event. I'll try and stop in and visit as many of our partners while we're there. So if anyone's coming to the convention center, please look for me. I'm probably over two meters tall. I'm easy to find and looking forward to seeing you people. So that's it for another hour for the OpenShift Commons Briefings Operator Hours. Mike Waite, Dave Muir, Rory McCune. Thanks for coming today and enjoy the rest of your week. Here's all.