 The Cube presents KubeCon and CloudNativeCon Europe 2022, brought to you by Red Hat, the CloudNative Computing Foundation, and its ecosystem partners. Welcome to Valencia, Spain, in KubeCon CloudNativeCon 2022 Europe. I'm Keith Townsend along with my co-host Enrico Signoretti, senior IT analyst at GigAOM. We are talking to amazing people, creators, people contributing to all these open source projects. Speaking of open source, Enrico, talk to me about the flavor of this show versus a traditional vendor show with all these open source projects and open source-based companies. Well, first of all, I think that the real difference is that this is a real conference. So real people talking about projects, so the open source stuff, the experiences are on stage, and there are not really too many product pitches. It's about the people, it's about the projects, it's about the challenges they had, how they overcome some of them, and that's the main difference. I mean, it's very educative, informative, and the kind of people is different. I mean, developers, you know, SREs, you know, you find and some people, I mean, people that really do stuff, that's the real difference. I mean, quite challenging somehow discussing with them, but really, I mean, because they are really opinionated, but. So we're going to talk to a company that has boosts on the ground, doing open source since almost the start. Kirsten Newcomer, director of hybrid platform security at Red Hat, and Connor Gorman, senior principal software engineer at Red Hat. So, Kirsten, we're going to start with you. Security and Kubernetes, you know, it's Kubernetes, it's a race car. If I wanted security, I'd drive a minivan. That's a great frame. I think though, if we stick with your car analogy, we have seen cars and safety in cars evolve over the years to the point where you have airbags even in, you know, souped up cars that somebody's driving on the street. A race car, race cars have safety built into, right? They do their best to protect those drivers. So I think, well, Kubernetes, you know, started as something that was largely, you know, used by Google in their environment, you know, had some perimeter based security as Kubernetes has become adopted throughout enterprises as people, and especially, you know, we've seen the adoption accelerate during the pandemic. The move to both public cloud, but also private cloud is really accelerated. Security becomes even more important. You can't use Kubernetes in banking without security. You can't use it in automotive without security. Telco, and Kubernetes is, you know, Telco's adoption, Telco's deploying 5G on Kubernetes, on OpenShift. And this is just, so the security capabilities have evolved over time to meet the customers and the adopters. Really Red Hat, because of our enterprise customer base, we've been investing in security capabilities and we make those contributions upstream. We've been doing that really from the beginning of our adoption of Kubernetes, Kubernetes 1.0, and we continue to expand the security capabilities that we provide, and which is one of the reasons, you know, the acquisition of StackRocks was so important to us. And actually we are talking about security at different levels, I mean, so, and different locations. So you're securing an edge location differently than a data center or maybe, you know, the cloud. So there are application level security as well. So there are so many angles to take this. Yeah, and you're right. I mean, there are the layers of the stack which starts, you know, can start at the hardware level, right, and then the operating system, the Kubernetes orchestration, all the services you need to have a complete Kubernetes solution and application platform, and then the services themselves. And you're absolutely right that an edge deployment is different than a deployment on, you know, AWS or in a private data center. And yet, because there is this, if you're leveraging the heart of Kubernetes, the declarative nature of Kubernetes, you can do Kubernetes security in a way that can be consistent across these environments with the need to do some additions at the edge, right? You may, physical security's more important at the edge. Hardware-based encryption, for example, whereas in a cloud provider, your encryption might be at the cloud provider storage layer rather than hardware. So how do you orchestrate? Because we are talking about orchestration all day, and how do you orchestrate all this security? Yep, so one of the things, one of the evolutions that we've seen in our customer base in the last few years is we used to have a small number of large clusters that our customers deployed, and they used in a multi-tenant fashion, right? Multiple teams from within the organization. We're now starting to see a larger number of smaller clusters, and those clusters are in different locations. They might be, customers are both deploying in public cloud as well as private on-premises, edge deployments, as you mentioned. And so we've invested in multi-cluster management and sort of that orchestration for orchestrators, right? And because, again, of the declarative nature of Kubernetes, so we offer advanced cluster management, Red Hat Advanced Cluster Management, which we open-sourced as the multi-cluster engine, MCE. So that component is now also freely available, open-source. We do that with everything. So if you need a way to ensure that you have managed the configuration appropriately across all of these clusters in a declarative fashion, right? It's still YAML, it's written in YAML. Use ACM, use MCE, in combination with a GitOps approach, right? To manage that, to ensure that you've got that environment consistent. And then, but then you have to monitor, right? You have to- I mean, we're in all of these stack-rocks feet seen. I mean- Yeah, sure. Yeah, and so, you know, we took a Kubernetes native approach to securing all of this, right? And there's kind of, we have to say there's like three major life cycles. You have the build life cycle, right? You're building these immutable images to go deploy to production, right? That should never change, that are locked at a point in time. And so you can do vulnerability scanning. You can do compliance checks at that point, right? In the build phase. But then you put those in a registry, then those go and be deployed on top of Kubernetes. And you have the configuration of your application, you know, including any vulnerabilities that may exist in those images. You have the RBAC permissions, right? How much access does it have to the cluster? Is it exposed on the internet, right? What can you do there? And then finally, you have the runtime perspective of is my pod, is my container, actually doing what I think it's supposed to do. Is it accessing all the right things? Is it running all the right processes? And then even taking that runtime information and influencing the configuration through things like network policies, or we have a feature called process baselining, that you can say exactly what processes are supposed to run in this pod. And then influencing configuration in that way to kind of be like, yeah, this is what it's doing. And let's go stamp this declaratively so that when you deploy it the next time, you already have security built in at the Kubernetes level. So as we've talked about a couple of different topics, the abstraction layers, I have security around DevOps. So, you know, I have multi-tenancy, I have to deal with, think about how am I going to secure the Kubernetes infrastructure itself. Then I have what seems like you've been talking about here, Connor, which is DevSecOps, and the practice of securing the application through policy. Are customers really getting what's under the hood of DevSecOps? Do you want to start or, yeah? I mean, I think yes and no. I think some organizations are definitely getting it, right? And they have teams that are helping build things like network policies, which provide network segmentation. I think this is huge for compliance and multi-tenancy, right? Just like containers, you know, one of the main benefits of containers that provides this isolation between your applications, right? And then everyone's familiar with the network firewall, which is providing network segmentation. But now in between your applications, inside Kubernetes, you can create a network segmentation, right? And so we have some folks that are super far along that path and creating those. And we have some folks who have no network policies, except the ones that get installed with our products, right? And then we say, okay, how can we help you guys start leveraging these things and creating maybe just basic namespace isolation or things like that. And then trying to push that back into more declarative approach. So some of what I think we hear from what Conor just teed up is that real DevSecOps requires breaking down silos between developers, operations, and security, including network security teams. And so the Kubernetes paradigm requires involvement. Actually, in some ways it forces involvement of developers in things like network policy for the SDN layer, right? You need to, you know, the application developer knows what kinds of communication he or she, his app or her app needs to function. So they need to define, they need to figure out those network policies. Now some network security teams, they're not familiar with YAML, they're not necessarily familiar with software development, software defined networking. So there's this whole kind of, how do we do the network security in collaboration with the engineering team? And when people, one of the things I worry about, so DevSecOps, it's technology, but it's people in process too, right? And one of the things I think people are very comfortable adopting vulnerability scanning early on, but they haven't yet started to think about the network security angle. This is one area that not only do we have the ability in ACS StackRocks today to recommend a network policy based on a running deployment and then make it easy to deploy that, but we're also working to shift that left so that you can actually analyze app deployment data prior to it being deployed, generate a network policy, test it out in staging and kind of go from the beginning. But again, people do vulnerability analysis, shift left, but they kind of tend to stop there. And you need to add app config analysis, network communication analysis, and then we need appropriate security gates at deployment time. We need the right automation that helps inform the developers. Not all developers have security expertise, not all security people understand a CI CD pipeline, right? So we need the right set of information to the right people in the place they're used to working in order to really do that infinity loop. Do you see this as a natural progression for developers? Do they really hit a wall before finding out that they need to progress in this methodology or I don't know, what else? So I think initially there's a period of transition, right? Where there's sometimes this opinion, oh, I ship my application, that's what I get paid for, that's what I do, right? And, but since Kubernetes has basically increased the velocity of developers on top of the platform in order to just deploy their own code, and some people have commits going to production, every commit in the repo goes to production, right? And so security is even more at the forefront there. So I think initially you hit a little bit of a wall, security scans in CI, you could get some failures and some pushback, but as long as these are very informative and actionable, right? Then developers always want to do the right thing, right? I mean, we all want to ship secure code. And so if you can inform me, hey, this is why we do this or here's the information about this, I think it's really important because I'm like, right, okay, now when I'm sending my next commits, I'm like, okay, these are some constraints that I'm thinking about. And it's sort of like a mindset shift, but I think through the tooling that we know and love and we use on top of Kubernetes, that's the best way to kind of convey that information of, you know, honestly, significantly smaller security teams than the number of developers that are really pushing all of this code. So let's scale out. Talk to me about the larger landscape, projects like Promenas, Kubelitner, OPI, different areas of investment and security. Talk to me about where customers are making investments. You want to start with Kubelitner? Sure, so Kubelitner was an open source project when we were still a private company and it was really around taking some of our functionality in our product and just making it available to everyone to basically check configuration, both bridging DevOps and SecOps, right? There's some things around privilege containers, right? You usually don't want to deploy this into your environment unless you really need to, but there's other things around, okay, do I have anti-affinity rules, right? Am I running, you know, you can run 10 replicas of a pod on the same node and now your failure domain is a single node, now you want them on different nodes, right? And so you can do a bunch of checks just around the configuration, DevOps best practices. And so we've actually seen quite a bit of adoption. I think we have like almost 2,000 stars on GitHub and super happy to see people just really adopt that and integrate it into their pipelines. It's a single binary, so it's been super easy for people to take it into their CICD and just start running three things through it and get, you know, valuable insights into what configurations they should change. And then if you're asking about things like OPA, open policy agent and OPA gatekeeper, so one of the things happening in the community about OPA has been around for a while. They added, you know, the OPA gatekeeper as an admission controller for CUBE, there's also Caverno, another open source project that is doing admission as the Kubernetes community has kind of decided to deprecate pod security policies which had a level of complexity but is one of the key security capabilities and gates built into Kubernetes itself. OpenShift is going to continue to have security context constraints very similar but it prevents by default on an OpenShift cluster not a regular user cannot deploy a privileged pod or a pod that has access to the host network and there's SE Linux configuration on by default also protects against container escapes to the file system or mitigates them. So pod security policies were one way to ensure that kind of constraint on what the developer did. Developers might not have had awareness of what was important in terms of the level of security. And so again, the CUBE linter and tools like that can help to inform the developer in the tools they use and then a solution like OPA gatekeeper or SCCs, that's something that runs on the cluster. So if something got through the pipeline or somebody's not using one of these tools those gates can be leveraged to ensure that the security posture of the deployment is what the organization wants. And OPA gatekeeper, you can do very complex policies with that. And Leslie, talk to me about Falco and Clare. About what? Falco. Falco and yep, absolutely. So Falco, great runtime analysis and something that Stack Rocks leveraged early on. Yeah, so yeah, we leveraged some libraries from Falco. We use either an EBPF probe or a kernel module to detect runtime events, right? And we primarily focus on network and process activity as angles there. And then for Clare, it's now within Red Hat again through the acquisition of CoreOS, but we forked and added a bunch of things around language vulnerabilities and different aspects that we wanted. And we're really interested in, I think, the code bases have diverged a little bit. Clare's on V4, we were based off V2, but I think we've both added a ton of really great features. And so I'm really looking forward to actually combining all of those features and kind of building, we have two best of breed scanners right now. And I'm like, okay, what can we do when we put them together? And so that's something that I'm really excited about. So you somehow are inting at your roadmap here, putting everything together and again orchestrated, well integrated to get also a simplified experience because that could be the point. And as you mentioned, it's sort of that orchestration of orchestrators like leveraging the Kubernetes operator principle to deliver an opinionated Kubernetes platform has been one of the key things we've done and we're doing that as well for security. Out of the box security policies, principles based on best practices with stack rocks that can be leveraged in the community or with Red Hat Advanced Cluster Security, combining our two scanners into one Clare-based scanner, contributing back, contributing back to Falco, all of these things. Well, that speaks to the complexity of open source projects. There's a lot of overlap and reconciling that is a very difficult thing. Kirsten Conner, thank you for joining theCUBE. Conner, you're now a CUBE alone. Welcome to our main lead group. Great. From Valencia, Spain, I'm Keith Townsend, along with Enrico Signoretti. And you're watching theCUBE, the leader in high tech coverage.