 Let me go ahead and do something here real quick. I think we're down to 503. It gives us about 18-ish minutes or so. So, we're gonna hit this. I even added some extra time, but hi everybody, Bill Benzing with Red Hat. My only rule I have for these is my Beyonce rule. So if you like it, definitely tweet on it. If you don't like it, tweet on it. We can also, we can disagree in public. Be happy to do that. But as we go through this today, I wanna talk about this love letter. This is probably the only love letter you can put in your organization and not get in trouble by HR. This is a love letter. This is back about 2017-ish from the DevOps community to Security Compliance and Audit. Basically with the admission that, hey, the DevOps community has completely forgot Security Compliance and Audit. So today I wanna go through on how to bring Security Compliance and Audit in to the software development lifecycle in a way that's effective for the organization. So a bit about me. My history is shadow IT. I like to say those words because not many people cringed in here, but I firmly believe that shadow IT is where the business value comes from. It's as people that know the business domain that have some technical competencies, but everybody else doesn't wanna deal with them. And so my whole goal is, how can you make the right thing, the easy thing to do and make it a default way such that everybody in your organization can affect that top line value, which is the customer value. I am a recent author, new co-author to a book called Investments Unlimited. Not gonna spend too long on this, but if you've heard about the Phoenix Project, it is a Phoenix Project-esque book talking about Security Compliance and Audit in a Financial Institution, a Fictional Institution. Myself, my eight of the co-authors, one in the back here, Mike, we wrote this from our experiences. And so there's a lot of, even though it's a fictional novel, there's a lot of nuggets of truth in there, but the idea here is, what can we do to get everybody in the same page and in the same experience? This didn't come on the blue. This has been, since 2015, a lot of works of effort. So published by IT Revolution, came, its history was through a bunch of IT Revolution papers, as well as experiences. So as we go through this, everybody's been talking about how can we do compliance and security better, but we've really been thinking about it for some years. As I get started, the new thing I think about is your systems. Explainable, consistent, and efficient. As we're going into these new highly-regulated environments, Dora and such, explainability to internal and third-party auditors is key. Consistency, when we have development teams doing whatever they want, whenever they want, however they want, or inconsistent, makes it inexplainable, and it gets down to the efficiency argument. And this is my CFO argument. It's not about cutting heads. We have investment and internal resources. How do we optimize those investments? The impending potential recession that's coming on, this is something big in people's mind. So as we look at automated governance today, the question is, how can we help the organization be more efficient with its use of resources? Yes, we want to get to production more often, quicker, better, faster, all those big words. But there's a lot of human hands in the middle of that process. So what can we do to free up those hands and reallocate them to something that's more effective for the top line? On the flight here, I read a lot of Dora. I was very excited by Dora. There's not a lot of people that get excited by legislation. But I went through the directive, and what I loved about it was Dora just calls it out, hey, the financial system, this is no more if your bank or your institution fails, it's only affecting your people. You are affecting the global economy. We are so integrated, we are so together that what seems to be like a small failure somewhere else can have cascading effects. What I love about what they wrote is like, hey, at the end of the day, we've lumped IT risk into traditional operational risk, and it's not the same. We need to have a new risk pillar, focus on IT risk and separating it from operational risk. So we're not in the states, we're not in Europe, but at the end of the day, legislation like this tends to spread and the idea spread. And what I love about this idea is before it becomes something we have to march to here in the States, is how can we start making this a reality in our organization where IT risk management is a pillar of our risk management as opposed to lumping it under operational management. So IT risk management is not any more considered operational risk, and we need to make it primary management. So I'm gonna give you a bit of bottom line up front and go through this. That was my new stuff for this this week. I always like to add something new. So is that a curiosity? Has anybody seen me present before? No, oh, groupful, we have one person. Groupful of new people, this is great. So I'll give you a bit of the bottom line up front. People should not execute the governance process. We need machines, they need to execute the governance process. People though are key because they design, develop, and codify the governance process. Now when I say governance, I am referring to security, compliance, and audit in this term. And governance, I always say, is the current bottleneck for software delivery. I don't think I get many people that disagree with me on that one. But we need to modernize our governance capabilities to address this bottleneck, such as we're modernizing all of our other technology capabilities. And modernizing governance is automating the governance process. But there's always a big but here. It's more than automation, it's autonomous. And what do I mean by autonomous? It's not AIML or anything like that. I love the SRE book. I pulled the definition on autonomous from this. So by the way, chapter seven, the evolution of automation at Google. If you're using Kubernetes, it's a must read. The predecessor to Kubernetes, Borg, the influence, talks about the business problem that came and solved. But what you'll see here is they focus on judicious application of automation. There's an admission that manual processes are required in certain aspects. So you can't automate everything. But automation is not a panacea, just because you automate it doesn't mean it solves the problem. So what you're looking for is a judicious application of automation, which is building autonomous systems. So modern governance is a high level governance system design. Modern governance itself is autonomous governance. So as I wrap this in a day when talking about autonomous governance, this is more than just adopting technology. And here's what, I love this book by Ilya, it's not an audio book. It's like a series of lectures he gave in the late 90s. I mean, were there audio books back then? I don't remember. But there's a capture of that. And what he talks about is the adoption of technology. There's four questions you have to ask. One, what's the limitations or what the power of the technology? Two, what limitations does it diminish? But three and four are key. What are the old rules you operate it by? What are the new rules you need to operate by? So as we talk about a bit of technology today, I wanna talk about more of the new rules, the change from the old rules to the new rules. So the next minutes, what you're gonna find about, I pack a lot of stuff in here. I feel like it would be not fair to leave everybody wanting more. So I pack as much as I can into these 25 minutes because the beauty is they're all recorded so we can always come back and rewatch it. So there's gonna be a lot today. We're talking about the problem, solving the problem. We're gonna talk about a new concept, the governance as a service. Very buzzwordy, I'm trying to find something different for it, but we'll start there. And then we'll talk about engineering productivity and ultimately the governance engineering team, which is our call to action. So let's talk about the governance problem. And most organizations, governance itself is toil. So what is toil? Back to my favorite book right there. Toil is that type of work that does not allow you, that basically is say you add two more applications, you have to add two more people, it's at linear scale. That's the thing, that's toil in your organization. And what you wanna do is work to get rid of toil. What I love about this quote though, is Carla points out that if you have human hands in your normal operation of your system, you have a bug. So I look at a lot of our software delivery, a lot of organizations, we have people in the middle. And I look at those as bugs. And it's not like those people are bad, it's our system is designed, the implementation of our system is such that it's bugged by design. So the idea here is as we go through, we talk about our energy technology, but how do we refactor our systems to remove these bugs? So a bit of this toil, there's two types of toil, governance toil and delivery toil. Governance toil is the easy stuff. It's the humans turning the crank of the wheel. It's the manual, send me the paper, let me review it, that kind of stuff. The hidden toil is the delivery toil. This is the stuff that somebody gives me back an evaluation and I spend weeks trying to figure out what the analysis means just so I can take an action. That level of ambiguity, that is delivery toil. So what we wanna do is we wanna get rid of this toil. But because of this toil, what's meant to mitigate risks actually increases risks. So tell your risk management people that the risk management process is increasing the risk and I'm sure you get slapped out of the room. But at the same point in time, I have the numbers to prove it at the end of the day. So let's say it takes two weeks to get through our governance process. And let's say it takes 16 hours to get a new feature done, right? And all these changes we're gonna make these features they're independent of each other for this. So let's assume we're pretty good at what we do. And our probability of failure for any given change is 10%. So what's really means is every change, probability of success is 90%. And in most organizations what people do is they'll batch these changes together so they can go to, since they can't go straight to production. So of course they create the famous release train. Now if they're good they're gonna release, they're gonna paste the release train to the governance process which is the slowest part of our delivery process at this point in time. And remember this train departs about every two weeks which means 16 hours per change over two weeks. You got about a total of five changes you can get on that train. And remember the purpose of our governance process is to reduce the probability of failure. So we should probably expect that the batch changes is no less than 90% and possibly should be higher. Does anybody agree, disagree with that? Does everybody agree it should be at least 90%s? No, is he shaking heads, no? I see some people just afraid to shake a head. It's like it's a trap, right? They know the trap's coming. Well it won't be 90%. The aggregate probability of this release is actually 59%. So you're getting close to a one and two chance of failure or success with these changes. And now let's say we actually do catch an issue due to the governance process. And let's say that issue takes one of those changes from 90% to 99% probability of success. Well in that situation our aggregate probability actually would be 62%. So this is the idea of single peaceful and why you wanna put one change to production as quick as possible. The more you aggregate independent changes, the higher the probability of failure is. And so this explains why we all have that situation where every time we're pushing a production something seems to fail. It's because we're batching stuff. And so what you wanna do is you wanna reduce the probability in the risk management process because it's a bottleneck that does this. And by the way it's not just risk that does this. Any bottleneck in your system that requires you to batch increases the probability. Governance happens to be there. So feel free to walk your risk people through that math. Let me know how it goes. I'd love to hear the outcome of that. But let's talk about how we fix this now. So we're gonna solve this problem. Remember, we automate it. No, we wanna autonomize it. So how do we autonomize our governance? I'm gonna give you five principles here. First, collaboration is key, but you don't wanna over collaborate. Collaboration has a cost. It's the most expensive form of interaction in organization. But the biggest thing is you gotta get key people from development, operations, security, compliance, and audit in the same room. What we wanna do now is develop enabling constraints. So as I talked about before with consistency, enabling constraints, we wanna constrain the organization. By constraining the organization, you reduce the variability, variabilities where risk comes in. We'll talk about explicit evidence and item put processes, but no more of this giving the same controls and evidence to two different auditors and getting different results back. We wanna get rid of that. That's why we wanna make it explicit, not implicit, not implied. It's not based upon what auditor A or auditor B says. It's based upon a series of very explicit set of data and controls. And at the end of the day, zero trust. I will talk about the zero trust architecture here, but I'll argue our software delivery should be zero trust by nature, and this is more than just identity. And at the end of the day, ephemeral item potent and immutable. So this should be able to disappear. Should not be able to be mutated. It's independent of itself and disappear. So we'll talk about the here. But we need to think differently. So what we need to do is we need to go from our current subjective change management processes to verifiable continuous verification, but to do that, you can't skip the metal, which is the objective attestations controls. So to achieve the continuous verification and get to the attestation controls, we need to autonomize these human control gates. Here's a simple to control gates we're gonna go through and talk about autonomizing today. It's not limited to these, but I like this side to show you that, hey, that those blue areas, what happens in there, we need to be able to autonomously validate in those little diamonds that what happened in the blue areas was the right thing. So how do we autonomize these control gates? Well, we got two actual procedures to do this. One we'll call an evidence, evidence and attestation procedure. So when you run some type of scan in your CI pipeline, let's say it's so in our queue, you'd be doing some kind of static code analysis. You collect all that material. You're gonna normalize it into something we call a governance contract. We'll talk about that here in a second. But then what you wanna do is you wanna sign in and test that material. So it's like digitally signing our artifacts. We're also digitally signing all of our, all the material that's created by our process. Now what this does is help us create provenance and ultimately pedigree of where this came from to make sure things are explainable. So this gets down to the explainable aspect. Second, we have a policy enforcement point. Nope, policy enforcement point is the key word. I stole that directly from the NIST 800-207-0 trust. We wanna retrieve a policy, retrieve those attestations, and then we put them together and we validate it. So in this validation, we validate that we have the right artifact. We validate non-repudiation, which means any of the evidence and any things we have has not been tampered with. So it's tamper evident. And then we go forward and we automatically evaluate the audit. So again, to do this properly, we need a new concept. We need this thing called a governance contract. And what is this governance contract? So all the very techie geeky words, it's the semantics and syntax of our governance primitives. At the end of the day, for the non-geeky words, it's what we wanna do and how we wanna state it specifically in words. And so how do we, and this is also how we codify all of our governance expectations. So when I talk about the governance contract, again, I'll use the word serializing, but we're taking all the material that happens in that blue step and test just happens to be the one up here. And we're gonna take that and we're gonna normalize that into something that's machine readable so that at that diamond up there, the machine can determine whether we pass or not. So what does this look like? I'll give you these very example example of what it looks like today, but at a minimum, this is the ubiquitous language. Notice on this left-hand side, common vulnerability exploits. This is language we can use across the organization that is not geared towards the geek speak, as I like to call it. It's geared towards the business speak. It's technology agnostic. You don't see anything. You don't know this openscaps, this dig, sooner, you know, any things that come from like sonotype. You don't know anything like that because it doesn't matter. The tool itself doesn't matter. It's about the governance process. It's understandable by technical and non-technical and this ultimately is a data exchange format for the computers to assess. So three aspects of it. One, the governance procedure. This is just simply the control gate at the end of the day. Second, are your procedure elements? And so what are you wanting to evaluate? So we're looking here at high, low, and medium severity elements. And then account, what is the specific values that you want to evaluate? So let's talk about how we create this real quick. So we create this as part of the CI process. This is something here with an openscap. We did a container scan and we took this and we converted it into that in the pipeline. This is signed and this is attested to. So if anybody ever wants to go back and do some external audit at some point in time, they can see now it's all explainable. So then how do we evaluate this? Well, we evaluate it with something like this. If people are using Rego or people are using OPA around here, you can have a simple policy written in whatever policy engine you like because all you're doing is you're now evaluating that governance contract. You're not evaluating these N plus one, there's numerous multiple of evidence. And what you can see here is you're just looking to say hey, is my severity count less than or equal to one, less than or equal to from a medium or less than or equal to from a 25. This is where people start to collaborate and codify security compliance and policies code at the end of the day. And if we can evaluate it, we can codify it. So what does this look like when it's applied to software delivery itself? Well, CI becomes evidence collection now. Your continuous deployment becomes validatable but what you look at is before your deployment processes, you can start to audit what happened in lower levels of your environments and then use that deployment process to collect material to now provide evidence for the next layer of your deployments. And ultimately as you do this in a layered process, you get 100% commit to production and it's completely explainable. And so that's key here, you're pulling the human hands out the middle. From a business outcome, what you can achieve here is SOC two and SOC three, type one and type two auditing on the fly. Imagine if every commit to, every deployment to production, you can have an automatic SOC two, type one and type two audit to evaluate it automatically. Like that's a bit of a panacea. It's a bit in the future. But what you're getting down to is you can evaluate a lot of your five services, trust services criteria from this. Now I talked about from some of the gates is you can autonomize anything and these are include but are not limited to these gates. And so anything that a human has to manually check between commit to production should be autonomized. So now let's talk about a solution on how to do this. Talk about governance service but first we'll talk about golden paths. Golden paths are for everybody. These are the enabling constraints that allows an organization that are collaborated on and agreed by an organization. They are an internal product and they get you to production. And so we talked about constraining things, security compliance, audits and procurement come together to determine okay, what is, how do we, what the governance, what are our controls we need to have in place and how do we get, and how do we offer these multitude of paths to production. One note though, golden paths aren't golden cages. When I talk to some people like this is great but they're getting some of this one and only one way to go and that's not the case. What you don't wanna do is you don't wanna offer one and only one way. You wanna offer a paved path or a golden path one way to accept this way and the constraints behind this to get to production or you offer a manual off-road path. At the end of the day, they just determine which one they go. You always have to apply the same set of controls. So let's talk about building a golden path to production. So current state of software delivery looks a bit like this. Team A, the different teams have their own build test and sometimes they scan tools. They go through, they do everything independent sort of in their own unique environment and then security compliance and audit is sort of sort of a may I go to production. And sometimes it's just a, it's magical that things actually get to production. Well, let's talk about the future. So governance is a service. What we wanna do is what you'll see here is they have access to the tool, they own the tools as part of the team. What we wanna do is we wanna provide an internal service we'll call the governance as a service that does all your governance capabilities. What you notice here is there's another component called the gate. This gate, when somebody asked to deploy to production that gate, it's part of the zero trust architecture that validates from we'll call a governance control plane which is the governance of service can whatever they're asking to be put into my environment you put in my environments. And so that itself is an implementation of the NIST 800 and the 207 zero trust architecture. So for example, your enterprise resource is your production environment. Your policy enforcement point is that gate, your system and your subject that can be developer maybe their own CI tool. And then that control plan is your policy engine and policy administration and all the stuff on the edges is the context. So zero trust is more than identity. Identity is key to it but it's other context to evaluate to determine if you can do something to an enterprise resource. So what is this governance as a service looks like? Here's a simple example. It at the end of the day is simply another pipeline but what you're doing is what you will notice is it's not in the hands or in control. It can be logically and physically separated from your delivery teams but it's managed by its own team. SAS, DAS, any type of appliance there's so many scans you can do but what you wanna look at is you have an evidence repository. The automation is automatically collecting and signing the evidence. You're automatically creating attestations and those attestations are used by the gate and then your governance professionals this is where you're doing policy compliance and security as codes. They're changing the rules. We're going from the old rules of people in the middle going through things to the new rules of people codifying their expectations. Now I don't think you're gonna take a normal compliance spoke overnight and say hey go write some rego that's just that's not a reality but the idea is you start to work towards this in the future and how can you organizationally stand this up. So here's a detailed example of a governance and service but it has everything there from SAS, DAS to container scanning, composition analysis, software build material generation and the ultimate auditing attestation. This is the sample of it but again when you do something like this you make the right thing easy and the default thing to do. So governance engineering and productivity so I talked about a bit of the tech but let's talk about how organization to make this happen. I wanna point out one thing the state of DevOps reports basically shows everybody does DevOps. Nobody says they don't do this but everybody, a lot of people are stuck in the middle. This 18% of people on the side what do they do that are different than everybody else? Platforms, they have internal teams that work on building internal products we'll call them platforms for their engineers to be more productive. I love this as watching, being doing a lot of industry research on engineering productivity from the folks that actually do it as a lifestyle and I serve an example of things and here's something from a Netflix which you'll see these are teams dedicated to building tools and internal products for engineers to focus on different technical domains. Governance is just another technical domain. When you do this, you move from this traditional IT to this cloud native approach where everything is sort of like you have protective gatekeeping and per project infrastructure you class operational complexity to a standardized infrastructure and you impose these enabling constraints and these enabling constraints when they're imposed they become services to the organization and with this inverted pyramid and if you think about the resource utilization you focus most of your resources up there on actually building the applications not on all the stuff below. When you do this you are effectively focusing on changing the old rules to the new rules going from the old way of working, new way. So if anybody's heard about platform engineering and or internal development platforms this is the same aspect. A lot of this they focus on the developer I talk about the development experience or the developer experience. Yes, our developer staff creates a lot of value but there's a lot of other people that value chain that are just sort of left out. So how do you bring them in from an internal product perspective into a way that makes the software delivery lifecycle more effective for the organization? And at the end of the day engineering productivity it's there to solve top line outcomes. Makes the right thing easy way to do so what does this look like in your organization? Something like this. You have cross cutting concerns and monitoring, compliance, security and identity management. But as you notice the top those are all the people and others procurement they need to add procurement there integrate with this capability. You see the tools and the runtime infrastructure underneath that's whether you buy it whether you build it those are there but the canonical workflows that's key. And I want to focus on canonical workflows because these are the internal product capabilities that you deliver to your organization. And so like day one and day two onboarding and offboarding but then day two you're CI in your CD and then governance as a service. And so these are the things that you build and deliver to the organization that enhances the experience. And you do this with a modern governance engineering team. And to close us out with the modern governance engineering team what you want to do is adopt the mindset of engineering productivity. And I argue just repurpose your change approval board. Everybody wants to blow it up and I say like you know without the bad effects of blowing things up like get rid of it. But don't just make it disappear completely you want to replace it with an engineering productivity team. And I talk about engineering productivity and I think about site reliability. You can have four golden signals. These are like the golden signals you have that you do in an SRE. But think about things like human touch points, tack time, control ambiguity and control coverage. Control ambiguity being a good one. All the controls you have what are the ones that you know that actually apply or don't apply? That's so called the black box. If you think about red it's not passing. Green it's passing and black. Black is like a black hole. How many, what you want to do is reduce that black hole. And then this can get you down to the idea of governance level indicators, governance of objectives and a governance level agreement at the end of the day. On the airplane a couple of weeks ago I was sitting like okay here's a hierarchy for SRE. Could you have a similar one-to-one hierarchy for a government for governability? And all you're really doing is you're just replacing the areas of incident response with things that are focused around governance issues like a governance issue response, governance procedure design and analysis and then control planning instead of for example capacity planning. It can be very one-to-one but when you start thinking about governance from reliability standpoint a lot of these ideas and methodologies apply one-to-one. So go and pass our internal products. You go and pass they do automate the governance. Investment happens up front. They're canonical implementations. Now everything should be automated but it's an 80-20 but there are exception paths. So you do have manual evaluations that are still there. The costs are incurred at the CAD but these are appropriate for some situations. For example that mainframe app we change once every 36 months. Don't need to invest to automate the governance around that. Just go through the manual process. That seems to be better. So in conclusion, explainable, consistent and efficient. The idea is we want explainable capabilities. They have to be consistent to be explainable and efficiencies are spinning our resources as best as possible. So no matter road traveled you always want to apply the same governance by modernizing your governance with autonomous governance. You want to autonomize your governance with the team. So at the end of the day, thank you all very much. All right, time. If you have any questions, we'll be back. So thank you. That's it. If you have any questions, I'll pack up. I'll answer a question if you have any questions. What's up? There's no questions. Oh, just conversations. Well, thank you for calling me out on that. Yes, let's have a conversation. Yeah, that today was 110 or something. So you apply the infrastructure as code but if you think about in your infrastructure, I like that you pointed out because they have one on software development and all the other ones on the infrastructure. Everything is an artifact. And so how can you govern those artifacts by making them explicit but exactly in your infrastructure? You can inspect it and set a policy around that expectation or codified. But yes, it can be for the infrastructure and the application layer. And I like how you're talking about the infrastructure because it's key there as well because that's where you want to constrain certain things because as people talk about multi-cloud, then you start to get a lot of aspects there. But yeah, think about an infrastructure pipeline. Now we've recently did this for some critical national infrastructure where it is a CICD pipeline fundamentally but it's there to build and evaluate infrastructure concerns to ensure that all the infrastructure is homogenous. So yes, absolutely. So the attestations are just basically, the attestation is you can attest to the material or you can attest to passing a policy but that's all automated in the background. So what you want to do as you build the automation, you build the automation to create those automatic attestations. You don't want the humans creating it. So for example, say I'm going through an infrastructure, say I'm doing a, I'm doing a Stig. Say I'm gonna Stig a red hat in the Enterprise Linux virtual machine. What you saw with OpenSCAP there, I can run the Stig, I take, say I'm using OpenSCAP, I Stig it, I take the outputs and then I serialize in that governance contract and that governance contract is then evaluated against a policy. So exactly, yes. Does that make sense? Okay, perfect.