 All right. Welcome everyone. I'm your track host for this session. My name is Pushkar. We have chip and brand. We're going to talk to us about path to production Sustainable compliance in strict environments Chip Brian Hello, everyone. Thanks for being here. This is our session path to production sustainable compliance in strict environments My name is Chip Zoller. I am a technical product manager at their moda and also a cover no maintainer Everyone my name is Frank killer I'm a software engineer at defense unicorns and kind of our talk today is to kind of walk through some design patterns around You know, how can we create? sustainable compliance in these Regulated environments and really regulated is called out here because it's more of an extreme case But as we look to commercial environments, I mean as we look to just general our dev environments We want to be thinking about okay. Who does this impact? Why does it impact should I be looking at specific things about my data, etc. So we're going to kind of walk through a couple things Stick with us to the end. We're going to have a demo to kind of walk through some automation That'll be a little bit more exciting But we definitely have to set the stage right a regulated environment. What is it? Typically, if you think about that your your health care financial Government, etc Some of these environments that we're building kubernetes on various cloud providers. They need to meet compliance with a certain set of standards ie your NIST a hundred fifty threes, etc and We've been doing it for a while now. It's you know, it's working for us But how can we start to push into you know, what's what's the best path forward? How can we automate compliance as a part of how we're delivering software to these environments? And so there's there's definitely a lot of things that go into that The like optimal scenario that we're talking about here with regulated environments is we want to know Absolutely to the most granular layer like what is being introduced into our environment? And I think that's very important. I think we're if we're thinking about kubernetes in particular We're definitely at the cross-section of GitOps with declarative configuration It's going to be pretty explicit about hey, this is what I'm changing I'm making the change in somewhere that requires approvals and reviews in order for it to then be introduced into the environment And on top of that or maybe as a layer below that we can think about Like what other systems come into play with compliance, so if we're doing the best case scenario for GitOps then below that right is it's introduced to the environment and Things are running as they should But what about at the you know access control to to that cluster? We want to really prevent this cluster from drifting from that configuration And so we prevent like manual modifications as much as possible right we're restricting access for who has the ability to go and Touch any of that infrastructure, but also having tooling in place. I like policy enforcement tools That can restrict certain actions from ever being conducted against the cluster And so that kind of sets the stage for like what a regulated environment is one of the the big key pieces I think of sustainability in compliance is Trying to get away from this Not invented here syndrome When we think about even commercial environments, we think about enterprise environments government, etc And in Kubernetes we're all leveraging similar stacks of products that are providing this common baseline We all need logging in our cluster we all need ability to monitor what is going on there we all need ability to enforce policy within that and These are very important, but what we don't need to do is we don't need to go and write all of our own orchestration and tooling Around that right we can leverage open source many of us. That's the very reason. We're here To go and really provide a lot of value We can say like who has the the most intimate knowledge of the logging stack will probably the maintainers of that logging stack They can tell us all about the decisions. They made and the capabilities it provides and we can we can leverage those ecosystems and like Probably for the greater portion of the last Decade we've been hearing a lot of things come out dev ops dev sec ops And we really want to They mean a lot of different things to a lot of different people and we really want to start to get into how can we collaborate on solving some of these problems Kind of an instance of this that we see kind of within the government space would be platform one and kind of that big bang kubernetes baseline for Compliance with the dev sec ops reference design from the DoD Department of Defense I call this out because again, it's it's a baseline It's open source and available for people to pull from that starts to think about how do I integrate my logging stacks with my service mesh And this is done elsewhere. It's not only done here But it's worth calling out that we're we need to collaborate on these things in order to provide like the best capability the most secure that it can be and You know, this is ultimately a common denominator across all of industry Healthcare, etc. These regulated environments And so as we delve into that we we reached the word reciprocity Bear with me if you're on the compliance side of the house is I can feel eyes rolling in the back of your head But really we kind of have to redefine this right? I originally want to say like reciprocity needs to die But I kind of started thinking about it some more and it's really, you know reciprocity this idea We're gonna we're gonna practice the exchanging of things for mutual benefit and how can this apply to us? How can this apply to dead-sec ops? How can this apply to? regulated or unregulated environments your death environments, etc and so kind of as a precursor to that like CNCF we're building applications. They're they're cloud native by design. They're they're meant to be scalable they're meant to be extensible and configurable But what happens when you have all of those things to meet the one node cluster of the 1,000 node cluster is that largely There's a lot of ability for those even if it's the same application There's a lot of ability for that to be different from the small to the large that You know the single to the scaled and so if we start to look at some of these things How can we think about you know what information is really pertinent for compliance? And that's where I think like reciprocity needs to be involved here to get away from the historical thinking of I've I've met my tests for compliance received my you know stamp for the next year. That says I'm good And no one's gonna check it in between then And here's why here is why I'm secure and get away from that and kind of move towards the like the how How am I secure? How can I reproducibly you know ingest any piece of software in the open source? or proprietary etc and When I have that how can I go and validate that it meets compliance with controls and This is really a space where we want to kind of like key in and target That's going to be largely important. And so I think this kind of falls into two two bins of different You know styles of work one is Compliances code if you've been in compliance work before if you had to run through NIST a hundred fifty-three Those standards are there for a reason and you know they're providing a lot of guidance and a lot of direction for how to secure systems But at the same time if you run a check and you get your stamp And then that ultimately sits in a document and doesn't go anywhere doesn't do anything doesn't evolve Then it's not it's not a living breathing document anymore, and you're gonna have to redo all that work again the next year and so There's this you know concepts that we're really pushing towards compliance as code We no longer want the Excel spreadsheets and the word documents etc where you're checking the boxes for yes We met a c3 But instead you're documenting in code in something that can be version controlled because we know the goodness of version control And and what that brings to us for the ability to audit and non-repudiation etc That we can we can really leverage the goodness in that and I'm going to show example here in a second of what that looks like But it it gets back into you know these are things that we can continually Evaluate we can make a change in the code base for a tool and we can then immediately say you know as a as an approval review process Is this changing my compliance controls? Is this changing what can be? Satisfied as part of that And so kind of as a General overview here. I'm gonna show a quick example. Don't read into it too heavily I get that it's a lot of text you have to parse through But the the general concept here right is for any given control if we talk about any given tool For instance, we're looking at service mesh from Istio And if we want to start classifying that hey if I use Istio as my service mesh What controls may be satisfied as part of that with this universe of controls? We can start to say okay. Well from a hundred fifty-three. We can this control AC three is satisfied under these conditions That's all great information. It's a lot of good direction that we want to push towards for what it what can be satisfied And we can also you know move scale that into Multiple sets of different standards that a tool might be compliant with whatever is most applicable to your environment And most important to you and start to classify this preferably with the upstream in such a way that it's controlled there Those who have the most in-depth knowledge of the tool can provide a lot of background for what is going on under the hood But one thing to note here is that this is the universe of controls These are things that can be satisfied by the tool And ultimately that's what we've been doing with our checkboxes. We've been doing some checks under the hood For yes, this meets this because of this awesome. I got that documented and available But it doesn't live it doesn't allow us to build stacks that can be you know Inherently aggregated if I want to deploy Istio and my logging tools and my monitoring tools each one of those should have these documents kind of Living in their code bases such that I can point at them. I can aggregate all of them and then within You know Seconds I can say here's the full list of universe of controls. Let's go and let's go and validate that somehow We're going to demo that here at the end that we're going to actually start to actionably automate that process And start to work out how can what does automation here look like who is involved? It's certainly not an end all be all right now. We're not done with it. It's not solved Compliances largely in ever-changing space, but Definitely worth consideration And so if you've got you know as branch showed you've got the definition of what Compliance standard that you're trying to meet and you've got that as code The next thing that you need is policy, which is kind of the glue that binds the controls to the actual configurations, so your compliance is going to be met by the tools that you're using and their Configuration well you have policy that can cover those things and perform validation on the tools And if those tools change then you have a set of policies that's for those tools So policy can really be the glue that binds the controls that you need to specify in the compliance to the compliance stocks through their Configurations, but the good thing is that we already have policy as code If we can have compliance as code we can marry that with the existing policy as code through one or multiple policy engines and store those things together or maybe separately tightly couple or Loosely couple those things together and be able to meet an entire compliance standard with code And so in this demo We're using caverno, which is an open-source policy engine built specifically for kubernetes. It's an incubation project Maybe you've heard of it But using caverno you can validate these configurations that satisfy the controls and also do things like ensuring software Supply chain security so whatever images that are being that are associated with those you can do things like verify that they have Signatures if they're attested and hopefully we get to the point where more of us are at least doing image signing verification Doing attestation verification. You can also ensure Through policy with the same engine that these images have been attested to and that even the contents of the attestations Are whatever we expect And with caverno, which is nice To be able to sort of get everyone on the same page with compliance. There's no programming language That's required. So not only is that easier for you But it's also easier for your auditors and maybe you want that maybe you don't want that but You your auditors are at least on the same page And everybody can read a simple YAML definition without having to parse maybe even an esoteric programming language to understand Alright, here's how we're meeting compliance. You can clearly see the definition. That's here. It's fairly straightforward and You know because we have policies code you can version control those policies as part of that compliance So you keep your compliance as code you have your policies that back to it as code everything's good and You're able to use your existing tools to then modify and manipulate those policies so things like namespace exclusions Like if you're deploying these tools in a kubernetes environment, you're often going to have some form of exclusion I need to exclude based on namespace maybe based on label. Obviously, you know you your organization Maybe your auditors need to get involved to try and understand that but if everything is as code It's a simple readable YAML definition and makes it super simple to not only parse that but also to programmatically manipulate that with your own tools and The other nice thing and indeed a necessary thing is we need to be able to check these things Throughout the delivery process. So not just ensuring compliance is running in the cluster We obviously all need that but we want to make sure that the start of the pipeline the entry all the way throughout that These things that are could be introduced remain compliant and with the use of like the Coverno CLI This is a perfect match in pipelines because now you can ensure that you're manifest that you're about to introduce through that PR That they remain compliant to your compliance standard mapped all the way down to the policies and you can have all of these things as code So some of the things in the future that you know if we're successful in doing this And we if we get enough traction that the world where we'd like to kind of get to is The ability for auditors to sign these policies So one of the neat features that's in Coverno one dot eight which was released earlier this week or at least the announcement was You can YAML sign your policies or you can YAML sign the definitions that are coming through your pipeline And if any tampering has occurred in that the signature will fail validation Not only will it not get pulled, but it won't get run and that's in addition to the image signing verification But also, you know, we've got this oscal definition that defines the compliance standard that we want We have a set of tools We should be able to decouple the compliance from the set of tools from the engine So these things are all loosely related but decoupled so that you can have different policy engines and Your set of tools that you want each tool maps to a set of policies and all of this is stored in version control as code everything from your compliance to your tools themselves to the policies to your engines and all of that's bound together and Kind of setting the stage for the demo So what we have what we've talked about right is we have these oscal documents. They're machine readable And so as such we can start to build automation that naturally organically facilitates reading that information and Taking some actionable steps for how do I how do I take the the state of I have the universe of controls that we've been talking about and Find out what is actually satisfied given the configuration of My environment of the way that I've deployed the tools of how I've configured them And this is really important. So if we take that oscal document and then we start to Run it through automation produce reports also in oscal so that all of our GRC governance risk compliance tools can read them with following the NIST standard for oscal in the schemas We can really get to a state where I've deployed a tool. I've checked that these controls are satisfied as part of my deployment of that tool and I've immediately made that available for the GRC tool to visualize and this can be a very continuous very event-driven process I'm going to show a CLI driven version of it, but it could be something that's continuous We can get to continuous compliance in the form of any change that's introduced into my environment I want to rerun checks against what's applicable. I want to rebuild my entire SSP for what is automatable and make that Visualized for whoever the persona is that needs to see that information if it's a car Authorizing official that needs to say yes, you're still in compliance. They should be able to go to That service that GRC tool that visualization any time and see the real-time compliance of that environment we're getting away from These checks where you know auditors came in they did the check and said it's good. I'll see you in a year and at any point in time not only can we visualize it but also the oscal compliance and the Documents that are being generated about what is satisfied is still the how we've translated the how from The tool makers the open source the upstream proprietary whoever We transferred that to those who are operating the environment and ultimately that gets has the ability to be transferred on to Anybody who audits so if we want to again have third parties Looking at the environment and validate it then that is an absolute option and it should be reinforced So what I'm going to do is I'm going to try a live demo, you know wish me luck but Really around the idea that on the bottom there very small text apologies You're going to see I've got a live Kubernetes cluster running on my machine. I'm just going to represent my environment And what I'm I'm going to be looking as I'm going to look to introduce a workload that I'm going to say does this the state of my Application this this workload doesn't meet compliance with a control We're going to we're going to run through validating that scene past fail scenarios and what that means And and then from there start to kind of run through That full that full end-to-end process So I'm going to start kicking off. You should see on the top here. I'm going to do it as we go I've got my You know air gap delivery tools are to kind of help me with this process because I'm Not going to rely on Wi-Fi. So I'm going to have everything I need locally and be able to Deploy applications to this cluster without the need of internet and Wi-Fi So the first thing that we're going to look at Is and this should be coming up here is That oscow component definition this oscow file right that we want to say we want to ingest And so there's again, you're going to see a lot of things here for what this looks like we're going to use Istio as an example and There's a lot of data in oscow bear with me. I'll point out like what's immediately The most important is really when we get to These implemented requirements. We're pulling from 853 as you kind of see there in the center And out of that we're going to look at okay for the given state of my environment Can I go and validate that some control is is being satisfied? today we're going to kind of look around the idea of AC4 and High-level, you know, how do you systems within my environment communicate with one another? For the service mesh People who are familiar with service mesh really we want to get to sidecar injected environments for the state We're not using ambient here at the moment But if we can go and validate certain things like is the sidecar injected we can start to really hone in on yes You know workloads are going to use MTLS to talk to each other And that way we can really you know dive into how do our systems communicate with one another one another and are they? Compliant so there's a lot of descriptive information that tetrate has done with the previous production of this Which is awesome. We want to know From those who are most familiar with the applications How to go and validate that a control it has been satisfied But what I'm going to point out here is kind of this rule section and this is something that we're working on with NIST for Oskow is Creating fields that we can go and automate we can evaluate with automation and then we can go and Prove out yes, this can be satisfied and let's go run the check. Is it satisfied is the control satisfied by the environment and so we've got Kind of a field here. Those are familiar with Kieferno. We're leveraging Kieferno engine under the hood To provide us with validation of kind of what that environment looks like So let's kick it off. We got our Kubernetes clusters on the bottom there What I'm going to do is I've got a ZARF package. I wish I could explain more about ZARF. Sorry for my ZARF friends But I'm going to deploy a package. It's all local to my machine in this package compressed tarball and I'm just gonna say we're gonna confirm it and then I'm going to set a label here really quick for True and this is gonna say yes, I want injection on this application For this package that I'm deploying and so I'm gonna go ahead. I'm gonna kick off the deployment of this workload and so you should see that introduced there and Yeah, as such You're what you're gonna notice about that is we can See that there there's some common things we can point out really quickly two of two I hope I can barely see this from my vantage point but The general idea is we're going to be checking to see if this thing is compliant with the control that we are looking to automatic that automatically validate with automation and so It's deployed and we have an oscal Definition that we can ingest with our tool Lula to go and then do An evaluation is this in the correct state? And so what we'll perform next is A Lula execute and this is going to ingest that document and then go and perform a Live check against our live cluster to see if it's in the right state so if we If we do that What you're gonna notice here immediately it's gonna go and apply kind of a policy-based check Against this environment you're gonna see things like hey resources passing one There was only one pod there to really go and validate against for this control to have been met. It's very high-level Check of this control not by any means completely fleshed out for what does Meeting a c4 across your full environment look like but we want to demonstrate this capability and so it's passing wonderful what this does in the interim is generate a Compliance report looks like I have a couple here And so 12 And so just kind of generic pass information What was it that passed so that we can then feed this into GRC tooling? We've worked with kind of partners in the past to Integrate different pieces together to really get to the continuous compliance framework And so there's information here that's pertinent to that But let's change it. Let's make it invalid So with our our ZARF deployment here, let's purposefully remove the injection of the sidecar from the service mesh so that it It couldn't communicate with anything else in the service mesh via MTLS and so We will run that and it's going to start deploying Any workload start terminating the old And now our updated version we've changed something. Are we still compliant? With the given control that we said we were And so it's it's simply a Let me remove these real quick and then It's simply a rerun of that loot that all-scow definition for what is meant to be compliant We're not telling Lula that anything has changed We're just going to be an auditor perspective and go and validate that that is the case So we'll execute and kind of as expected Super exciting it's now saying it's failing Right and so this is This is where we want to get to we're doing these checks against live environments We can do these checks with static manifests as well. And really we're we're looking at how do we? Help NIST evolve Oskow that we as an ecosystem open source Assist with that and then how do we start producing? Oskow Models that are really going to facilitate the end users of the operation of the environment everything from conducting assessments so if we take a look at That compliance report it's going to be telling you it's failing And that'll be that can be made immediately ingestible by GRC tools and Yeah, that kind of covers what we're looking at currently wanting to extend this outside of Kubernetes, but there's definitely a breadth of breadth and depth of you know discovery that still needs to be done in the compliance space How does Oskow need to evolve? How does Lula and other compliance engines need to evolve? to really get to a point where we can start to cut out You know tens hundreds thousands of hours worth of repeatable compliance Processes that are being done every day everywhere in every regulated environment And get to a better path forward so with that any questions And I come right to where you are. Okay. Have you guys seen any Resistance to moving away from the rubber stamp yearly audit I'm just going through an event with my CSO team who don't want to approve pull requests Any guidance for me Other than cell Yeah, that's a tough one There's going to be a lot of a lot of variables involved there. I think like if we Get to you know these frameworks where We're we're actively iterating on like how do how do we prove out the provenance of these queries, right? If if you looked at the high-level version of what we saw today It's it's not every single check that needs to be performed that we need to write that query We need to iterate on queries like those and in your instance of you know Resistance to change resistance to you know what's being introduced, etc. Like It's going to be very tough But I think that the the common denominators here are like We we ultimately want to to get things changed and updated and maintained because Vulnerabilities and kind of the risk that comes with not changing But how can we automate out some of these processes? Maybe compliance with certain controls is a path forward for You know proving out that something should be changed. It is your pipeline You know go and check like hey if I introduce this change is it going to Violate a control is or is it going to introduce something new is it going to introduce a new control that I am now compliant with because I've ingested that and I think there's a lot of possibility here to kind of leverage that to I Really wish I had a polite way to say this to help cyber with you know coming into the the dev ops The dev sec ops pipeline perspective not that they are not that there aren't people that are but These definitely have to be very cohesive teams Cheers So it it looked like from what you presented that That istio like upstream provided that list of controls that they could be satisfied by istio and then You've got the policy to check that can you just comment on What different people in order to make this a reality in terms of actually pushing all the way forward To like an authority to operate or something like that What would the different responsibilities have to be what would have to happen from different organizations? Yeah, I think that's fair Really it's It's leveraging kind of all of these open source communities to to start providing some some baseline and again Open source what's good about that is if you're very passionate about Compliance maybe you are contributing it to an upstream source But the idea is like have this live as close to the source as possible Again, so that those who are well versed in an application can Be able to evaluate those controls and look at them and say yes. No like here's our here's our list for this release But once you have that in place right then all the consumers of that can start to If it lives up in the upstream or if it's something you have to inject in your system I think there's a lot of work to be done here, but at the same time What's what's start to get it to a point to where it can be aggregated and really for thinking about The personas like the one aos are authorizing officials In in what information they need but also the operators Like they're the ones building these environments So the ones that are going to have to change things on the fly and or you know light largely React to something that is no longer compliant and fix it and so like I see multiple layers here of having the the document the universe controls available Starting to build out your environments or update and maintain your environments and ingesting the results of that those compliance checks And making them available and visualization Um Kind of some other things involved there with your question. I'm probably not hinting at right away, but Absolutely relevant All right, we have time for one more question Okay, we'll go here Hey, good afternoon. Um, I'm thinking about the Relationship with the compliance department. How did you guys streamline that relationship? So that they trusted that your policies satisfied their phone books full of rules It's a valid question. Um, I I would not make the claim that it is 100 validated by whoever sure is going to be doing this, right? And we're talking about tools such as lula. They're they're not tied to a hundred fifty three They're meant to be agnostic of the standards that we can you know do other things like socks and PCI etc and Given that There's still a large amount of work that still has to be done here and a large amount of validation that still has to occur And I think what's going to play hand in hand here with concepts We're well versed in or becoming well versed in is kind of like supply chain security and The idea of like provenance and attestation Absolutely applies here and getting things like having auditors signing policy having auditors sign You know compliance definitions for when something is met is Kind of a collaborative activity yet to take place. I'm hopeful that it can take place But we need automation tooling to kind of look at this compliance space And you know start to work with one another to build out something that Somebody would want to put their signature and stamp on. Yeah, I don't think we're there yet. Okay But I think we can get there. Yeah, cool. Thank you and something else that's going to help is reliability Like knowing that when you say or claim not that it's going to alleviate auditors concerns If your environment is compliant that when somebody else checks independently that the two line up because one of the historical problems is We are compliant. We tell you that we're compliant. Look look at our tool It says we're compliant and they come in it's like all right. There's 40% deviation. No, you're not so That's one thing. The other thing is putting control back in the people's hands who need to have control And that's a that's a question of trust and some of the things that can help with that are tools like signing and validation Of a lot of different things so that they can have the control you can run it The results are consistent and everybody's on the same page All right, that is time. Thank you very much for attending and thank you