 Good morning, everybody, awesome. Welcome to the second day of DevOps days. Although, you know, Davis just did that, but I'm doing it. This is the second time I've spoken at, I almost called it Rockies, but we're not supposed to do that. Sorry, I spoke at the first one I missed last year. I'm really, really excited to be back. This is one of my favorite DevOps days to be a part of. Denver's community is one of my favorite communities to be a part of. This is sort of that thing where you're all supposed to go, woo! You know, like when a community's like, what up Chicago? We're here in Steeler Holly. Like, I know the name of that place. So anyway, right. So now that I'm all excited and doofy, we're gonna talk about shifting left securely. That sounds very exciting, doesn't it? But trust me, it's gonna be awesome. So real quick, who am I? Cause you gotta have that, right? So my name is Matt Stratt, and I finally got around to putting my Twitter handle on this slide. Although I realized the deck set now has footers, so I could be cool like everybody else was yesterday and like spew my Twitter handle at the bottom of every slide. But anyway, I work at Chef. I'm a customer architect, which means I design customers, I think. I'm not sure. I've been at Chef for a few years. I've been part of the Chef community for years. I founded and co-host a podcast called The Rest of DevOps. You should listen to it, cause it's rad. I am one of the founders and organizers of DevOps State Chicago. I'm one of the core organizers of DevOps State's globally, mostly which means I mess around with the website and ignore people when they want feature requests. So yay, it's like I'm a product manager. And yeah, that's my license plate. So, and that showed up in one of Patrick DuBois talks and that made me real excited. Background just a little bit. I've spent most of my career in system administration. I've worked for dot coms. I've worked for large financial institutions, insurance institutions, cause that's what happens when you live in Chicago. By the way, I'm from Chicago. So hey, what up, right? So the question I have for you is, have you engaged with Andrew's brand today? So make sure that you're engaging with Andrew. That's probably the most important thing you could do with DevOps. For the level of troll effort we just put in on this, that deserved a much bigger laugh. That's all I'm gonna say. Anyway, hey, there we go, cool. Okay, sorry about that. Just edit it out of the live stream later. We'll fix it in post. Did I say I do a podcast? So anyway, so what about stability, right? So stability is something that we attempt to ensure by reducing the vectors that can make changes to a system. So what is the thing that causes instability? Changes, right? If nothing ever changes, then shit's solid as a rock, right? The problem with that is what business like can never have changes, right? Changes are what drive our innovation. If we don't do that, and we can run into problems when we think about stability as being the most important thing. I had a colleague who worked another line of business connected to me, and she was the director of TechOps for that group, and she was incented upon uptime. So her bonus was a percentage of uptime to their site. How inclined do you think she was to allow any changes to their stuff? Like zero, right? It's like, hey, products like, hey, we totally need to ship this thing. And she's like, hey, and if you screw it up, that is cash out of my pocket. Guess what, big no. So just keep that in mind, right? What we're trying to do is we wanna reduce the vectors that can make changes to a system. Traditionally, that's what we think we're gonna do. We're like, well, if we minimize the blast radius, then we'll have more stability. That's usually what we do. And in practice, what does that mean? It means deployments are executed by trusted individuals with admin access. This is like the biggest line of bullshit I've ever heard. And I've been a sysadmin my whole career. But this is what we do. We're like, okay, well, the way we'll be safe is that trusted individuals who have admin powers will make the changes, and then therefore, everything will be delightful. Well, first of all, just to be clear, my background is in system administration, though this is not a bag on the sysadmin talk. I may bag on sysadmins, but that's my tribe, so it's cool. Yeah, I do have a skill endorsement for karaoke, and you can't see it on here, but I also have it for salad, I think. So some of my coworkers specialize in LinkedIn endorsement trolling. It's fun. Yeah, so here's the problem. So we have this separation of duties approach, right? Where we're like, cool, we have like our magic trusted sysadmins who have the pseudo access, and because of that, they never screw up, right? And those devs who don't know shit, they can't do stuff in prod because it's fine. The problem is that everybody makes mistakes. Let me say that again. Everybody makes mistakes. We are all human beings. We are all imperfect. We are all far from perfect, and we certainly, we do things like be overtired, have a bad day, be listening to a podcast when we should be working, maybe a rest of dev ops. We are distracted by our personal life, or we just mess up because we're people. A job title does not imply infallibility. Sysadmin does not mean pope of the server, right? Like you're gonna screw up, right? And this is the thing. Every sysadmin who thinks that their title means they won't make a mistake, if I take them out to the bar and we have a couple pints, they're gonna tell me about the 10 times they broke production by screwing something up, because we do that, right? It's okay, and what we have to understand is my title does not mean I'm infallible. Because I have been granted access, does not mean I'm not gonna make mistakes. I was talking last night to Greg, who gave an amazing talk about sidecars and microservices and a whole bunch of stuff I don't understand because I'm a big hand waver, but I was like, that shit looks rad, cool. But we were talking about stuff and I was expressing something I remembered. From I had a DBA who worked for me and we would get SQL scripts from the developers for releases. And it was great, that's awesome. I mean, better than what they were doing for the sysadmins, which was, hey, here's a big word doc of shit to do and you're like, awesome, right? Let's do that by hand. And then the problem was, I would say, well, DBA dude, what are these scripts do? He's like, I don't know. I just paste them into SQL Management Studio. And I'm like, so the value you add is that you have administrative access to the database. If that's the only value you add to my organization, I can fix that, right? So the thing is, it's around this thing where we, again, we're gonna make mistakes, but we have to understand it, but there's nothing, but again, there's a great example. I am administrator, right? Don't, that was your title dude. You did that for like 10 years and you made all kinds of mistakes because you're human. Hey, guess what? I've had great big titles too and I screw shit up all the time and tell you about the time like, I used to work, I worked for Allstate for a while and we used this back in total bare metal days and so when we had to rebuild dev servers, it was like the go, go into the data center, turn the thing off, drop in the smart start CD, blah, blah, blah. And like, of course, like every server at Allstate has a cryptic name that's like C58296852, right? Because again, naming, right? Sucks. So that fun thing where you go in there and I'm like, oh, I got a thing to say, oh, you gotta change this, you know, we're gonna rebuild this server, this dev server and it's like, you go in there and I go to turn it off and then I like hit the ILO and I'm like, the dev server's still turned on. What did I just turn off? Turn it back on, walk out of the data center, stand outside the door, see if anybody freaks out. Yep, nobody freaked out, cool, let me go back in and do that. And I am now a thought leader of DevOps so therefore everybody can mess shit up, right? Okay, so this also creates a huge burden on one group to be able to understand all the options and variants that steps in like a run book or whatever could create, right? So I'm like, again, it's like, oh, well, here's all the things you have to do. Well, I am the infallible pope of the system, right? So I have to understand every single shit. Okay, anybody ever do tar without Googling? If you raise your hand, you are a liar. Okay, so like it's fine, right? Like it's okay. So we don't expect, the problem is, but the implication is I can understand all the things, right, so that sucks, let's not do that, right? So how do we usually like do stuff? This is like more of just a slap on Agile, but this was the best graphic I could find that was slightly funny that had to do with sprints. It's not really what it's about. But anyway, the thing is like so traditionally when we think about security and compliance and compliance, I talk about compliance as like capital C and lower KC compliance, capital C compliance is like HIPAA and PCI and all these sort of regulatory things. And then you have lower case compliance, right? Just your organization standards. You need to be compliant with your organization standards as well as you need to be compliant with like the regulatory standards that are imposed upon you. So when we think about things like security and compliance, how do we usually do this? Well, let's say we have a project that's eight sprints long. First of all, like I don't think we're supposed to do projects like that anymore, but guess what we do? So just bear with me, right? So we have this eight sprint long project and we don't give a crap about security until that last sprint. We call that a hardening sprint. And then we go, oh, now we're gonna do the hardening sprint. Now we're gonna like pen test this shit and we're gonna security test and opposite and all that stuff. And it's all going to fail, right? Because we haven't been thinking about security at all for the last seven sprints. So the problem is, so now we've got like, now we're kind of stuck, right? We're like, well, so we're ready to go live. We're ready to push this thing and we totally failed compliance. So we have like a couple of choices. We can delay the release, which we can't do because the salespeople promised it to like the customer and right, that's a thing. So we got to ship because that's what we do. And we know we're not gonna do that. Or we go when we get an exception. We go to InfoSec, we're like, please, we have to ship this thing and I'm sorry that like we're still hard pleaded on this server, but we totally need to ship. And security's like, well, how much is the revenue impact? And it's like, oh, we could make a million dollars if we ship this insecure shit. So sure, here's your exception, right? And that's great. Except people that want to hack you could give a shit if you have a note from your mom that says it's okay that you have an unpatched server, right? So it's theater. The other thing is now what you do is you push all that stuff down into like some tech story that you're gonna fix sometime, which in some time means never. So now I want you to imagine that if you did the same thing with QA, if you said we are gonna postpone all functional testing of our application till the last sprint. First of all, if that may, you may have worked at other places I have, if that makes sense to you because sometimes people do that, but you shouldn't. That should sound ridiculous to you, right? Like it's nuts. You're like, how would we do that? We're gonna be test every sprint. We do all that stuff. So guess what? Security and compliance are just another aspect of quality. So what does shifting left mean anyway? So I talk about like we're gonna shift left securely and everyone's like shift left and shifting is cool because it's a word. So the reason we say shift left is just imagine a pipeline, right? You start, it's sort of like what John Willis would call commit to cash or you might call idea to feedback or whatever. So it starts over here and because we in Western culture read left to right, so it's like the beginning is here on the left and the end is on the right. So what we're doing is we're saying move stuff towards the left. What we've done traditionally is a lot of these things we think about them once they're out there. We do a lot of this testing towards the right. Like security testing, how do most people test security on their systems? Pen tests on shit that's in production. You don't get much farther to the right than that, right? So, and we've already been doing this with development. We're like, hey, let's do like early testing and local development and all this good stuff. And it's like, hey, guess what? Well, let's do this further stuff too, right? So the reason that we wanna do this is the closer to the introduction of a defect that we discover it, the cheaper and easier it is to fix, right? So I'm sitting there and I'm writing like a chef cookbook and it breaks and I run Test Kitchen and I see like two minutes after I write it that it broke, I can fix it right then. It's super duper easy, right? Or I've got some, you know, I wrote some crappy little go CLI that people are using, I didn't write any tests and I release and people are using and now I'm getting issues back that shit's broken. I'm like, God dang it, I gotta go back. But if I wrote tests, I can fix it then, right? So the problem too is if you gotta come back and fix something like two weeks or two months or six months later, you having the slightest idea how to go fix that stuff because your brain's moved on to something different. That's why we wanna know like as quickly as possible when we've broken something. How many people like totally get that when it comes to like code and functional testing and applications and everybody's like, yeah, that's totally what we know. Really? This should not be new. The idea is it's new for security. Okay, well, let me put it this way. Do this with your code too. Okay, so now we have some challenges here, right? Right, so I mean, if my tests don't pass, well I just changed my tests, man, right? So the light is green, the trap is clean. So one of the things, this is where separation of concerns, especially around security can really come into play, right? So whatever type of testing you're doing for compliance. And again, it depends upon the organization. You may be in an organization where you're like, the reason we care about security and compliance, it's, hey, we're like a 10 person startup and I'm the one person that does all the things. Well, it's not a problem that you write the security test and everything too because you just wanna get value. But sometimes you do have regulatory reasons, right? So you wanna make sure that the people who care who are expressing what compliance means are owning those tests in a way that are separate from what's happening. And I'm gonna talk a little bit about how that can get tricky because you're like, well, don't test, like, live with the thing. That's why we have pipelines. Okay, right? So now we talk about a lot of times, and I'll give you, just to give a little background, I know like, I'm kind of gonna be talking a little about treating your infrastructure, like, info code with this, but this does apply to applications as well. We just sort of have it solved a little better there in certain ways because we're already at least thinking about it. But one way or another, we talk about treating infrastructure as code a lot, right? And what does that mean? When you treat infrastructure as code, besides it being like, there's all sorts of buzzy things about it, but to me, info code, and treating infrastructure as code means that it's modular, but it mostly means that it's versioned and it's tested and it's in version control. When I say it's version, right? So why wouldn't you do the same thing with your workflow and with your pipeline, right? You're sitting there saying, you know what, we're gonna automate the crap out of stuff with Puppet or Shaft or Ansible, and we're gonna make sure that there's no way you can make a change to a system unless you like, check shit in to get, and it's gonna test it and everything like that, but we're gonna change the workflow, so I'm gonna go click in Jenkins and press a button and do whatever. It's like, no, man, that's all, it's workflow as code. So everything as code, let's just understand that, right? If it's a thing, make it be code. If you don't know how to make it be code, invent it, right? There we go, cool. Okay, so how does this actually help us with security? When we talk about all this stuff about versioning your pipeline and doing all this, what does it really mean, right? So I've already talked about this, right? I, my slide got a little out of order. But again, we wanna be continuously testing for security. I cannot say enough that compliance and security are just another aspect of quality. What you do not wanna do, this is another thing, this is what I call, you know, it says, you know, what you do not wanna do is have a test in your pipeline that cannot be run by a local developer. If you do this, you're basically a big jerk face, right? So imagine this, right? So I've got my little pipeline that says, okay, I'm gonna release my thing and I'm cool. I'm like, I'm hacking away on my laptop and I'm building my Go app or I'm building my Chef cookbook and stuff. All my tests are passed and I get it in there. It goes through a bunch of stages. It hits like the final stage and now it runs like a qualis test and it fails. And it's like, screw you. And I'm like, dude, I totally wanted to test. Now you don't assume people want to, but like maybe people wanna like do the right thing and test so you need to provide that ability. The problem is a lot of times we have these heavy things that you can't replicate. And that's super rough, right? And there were talks yesterday about how you can do that better, how you can sit down and you can actually, there should be no difference with what you can do. Again, as I said, we've got these powerful laptops. We should be able to replicate what we're doing for our testing locally so we can find these defects as early as possible. So for example, right? Let's say this is like the super simplest pipeline ever, right? It's like, oh, so we go and it does like a lint test and it's a functional test, it does a security test. And then it's like, hey, screw you because you failed the security test because you couldn't do that. And now I'm super pissed because I'm like, dude, I totally would have fixed that like two days ago if I could have run it locally on my machine. So, and the problem is again, these tests for security tend to be really heavy because a lot of the tooling around security is really heavy. It's expensive, it's like really central server based. It's all scanning, you know, you got to be like super duper infosec, you know, maven to like have access to the tool and all this stuff. And there's no way that you can run it against like some local VM on your laptop. So keep that in mind, we'll come back to that in a second. Here's the thing, right? So here's the thing, when you're trying to figure out how you're going to have some control of stuff, if you spend time, and again, this is when we talk about how we're introducing security and change. If I spend my time keeping people from doing X, Y, or Z, you know what they're going to do? Oh, that was weird. They're instead going to do A, B, or C to get the outcome that they want, right? We don't want to think about like, how do I keep people from doing the thing? I want to think about the outcome, right? And this is a little bit of a chef thing, but it's like, hey, you think you're going to keep people from mounting things with food critic? Let me introduce you to an execute resource, right? The thing is you don't want to block the how, you focus on the what, because otherwise you ended up in this weird arms race, right? You try to figure out all the things that someone could do. And the reality is I don't really care what you write in your code. I mean again, you know, there's static code analysis that looks to make sure no one's slipping in Trojan horses and all those other stuff. And that's important, right? But my thing, especially when I think about info code, I just care about what ends up happening. So I don't care how you go about doing it, because if I try to prevent you from doing a thing, because I think by stopping you from doing X, it's going to prevent outcome Y, you're going to go do Z, right? So let me just sort of, this is sort of the perceived idea that when people have problems with distributed configuration management, which I also call making infrastructure s'mores because I'm a dork, right? So first of all, the principle here is that you don't just have like the one person writes config management for your whole organization, right? Domain experts do the piece that they want. This freaks sysadmins the hell out, right? And this is exactly why, because this is what sysadmins think happens when you let developers write info code. Developer reads on Stack Overflow that disabling SE Linux will make his node app work better. Developer updates his cookbook to disable SE Linux. Sysadmins get fired because of evil hackers. This is what sysadmins think when you let developers write playbooks or modules or cookbooks. Now, this is not real, right? I mean, well, it totally could be. But it's, you can solve for this, right? So there's a better way to do it, of course, right? The better way is, again, we don't stop this, right? Cause Stack Overflow gonna Stack Overflow, right? So developer reads on Stack Overflow, the disabling SE Linux will make his node app work better. Bear in mind, we're not talking about a malicious actor at this point. We're talking about who just doesn't know, right? It's like, oh, I don't know what SE Linux is, but the thing says I should do it. Whatever, cause, again, this whole idea of full stack developer, by the way, there's like 12 of them, they all work at Netflix, so let's all stop pretending it's a thing. So now, again, the developer updates his cookbook to disable SE Linux. We're not stopping this. That's fine, do that all day long. Now, the developer runs local tests, which include the compliance checks, compliance checks, tests for the state of SE Linux. Test failed, developer says, well, I guess I can't do that. Now, developer probably goes, god damn it, I really wanna do that, and I'm gonna bitch and moan and everything like that, but the reality is, I got stopped and I know why. Cause the other thing that won't work is, we do these things where we're like, here is the tome of things you may do with your config management, and guess what, ain't nobody reads them. But when you get a test result that says, don't do this. The other thing is, even if they don't, you know, this is a quote I want you to bear in mind, right? So this is from Sasha Bates, was on an episode of the ship show called Keep Calm and Prod On. I don't remember the episode number, but you can find it, if you go to ship show, but she's like, if you treat developers like children, they will act like children, right? So with children, with our kids, we tell them what to do, right? We try to teach them how to be good and stuff, but at the end of the day, we're like, I'm your dad, that's why shut up, right? As an adult talking to another adult, that's not a real good way to interact with people, right? So we don't wanna treat the people that we're interacting with like children, unless they are children, you know, and if you've got like seven year olds contributing to, oh, your open source project, that's rad, and you might need to treat them a little bit like a kid, but you probably don't, because they're probably more adult than you, because they're seven, and contributing to your open source project. So now the thing is, so you remember back, I was like, oh, well, the developer runs that local test, and it fails, and they go, oh, that's rad, I just won't do that. And then the reality is, that's not gonna happen, because people are gonna run the tests or they might. I mean, hell, I don't always run my tests, because I'm a human, right? So it's trust, but verify. As you know, Ronald Reagan said, and lots of other people, so I always say, take a page from the folks at Etsy, right? We all know, if you're gonna dev ops, you have to know Etsy, because they are a monitoring company that also sells T-Cosies. So Etsy has a super-duper high-trust culture, but that doesn't mean they don't test stuff. It doesn't mean that it's like YOLO into prod, right? So, I mean, we enable local testing, but we don't count on it. So remember I said, you've got to be able to go and say, if I'm gonna run a test in my pipeline, you have to be able to do it locally, but I am never gonna assume that you did, right? That's your own problem, and what I tell my customers, is they're like, oh, well, then people are gonna get mad, like, after the third time that someone doesn't test something locally, and it fails in the pipeline, they're gonna get pissed and they're gonna learn how to start testing locally, because it's just wasting your time. If I don't run my local tests, and I wait till it fails in the pipeline, the only person's time who got wasted is mine, and the best way to get people to change is to make the right way the easiest way. So, the other thing too, is if you truly care enough about a thing, if you truly care about a thing, you care enough to write a test. So, again, let's go back to that SCLinux example, right? So on this assessment, I'm super duper freaked out that someone's gonna write some info code that's gonna disable SCLinux. Well, I need to make sure, so the best thing I do is I write the test, it says this SCLinux enabled, if not fail, and then put that into my thing. And the reason I bring this up, this may sound like, whatever, why are you saying this, is when I talk to people about this a lot, they're like, again, it's the, I'm too busy, I don't have time, whatever, and I'm like, well, you know what your alternative is? Is you review everybody's info code over and over again, or you take like 10 minutes and write a test, and now you're done, and now you feel super good about stuff going through the pipeline. The alternative of this would've been that Lego meme of the two dudes like pulling the cart with the square wheels, and the guys got the round wheel, hey, be better, and they're like, we're way too busy to improve, you know, so, whatever. So, let's go back to something Andrew talked about yesterday, about outcomes. Cannot say this enough, outcomes, outcomes, everything else doesn't matter. The how, who cares, right? It's like in the Phoenix project, it's the three-ring binder. Well, we have to do what's in the binder. Well, no, what matters is why are you doing that? So, we are testing for outcomes. Our pipeline is creating an artifact. So, in the case of InfraCode, that artifact is a converged node. The artifact is not like a cookbook or a module. Actually, the artifact you're creating is a machine that's in a certain state. You want to understand what that's like. So, what matters is the state of the outcome. How you got there is not the question. So, we're testing compliance and security against artifacts. And the outcome that we are testing is, is this thing the way it should be, or is it a scary nightmare that should never, ever see production, right? So, I'm not like testing to be like, is your puppet code look cool and awesome. I'm testing to make sure that your puppet code didn't create a node that has SE Linux disabled. Or has SE Linux enabled if that's against your thing. And by the way, like I keep using that example, you should be like testing for more than that, right? But people understand that and are also used to people like just going like, well, just stable SE Linux, that solves all my problems because dev. Here's the other thing. Please, democratize your compliance testing. So, you remember I talked about the hubris of CIS admins? InfoSec folks do this too, right? Tools are kind of the least important thing to think about with this. But make sure it's not a tool that can only run tests from your security folks, right? You need to be able to enable this stuff everywhere. If you care about compliance, move that stuff to the left. If your tools can't do that, it's time to find another tool rather than change your workflow because you're like, well, we just spent like $100,000 on, and by the way, I'm so not picking on QALIS. QALIS is pretty cool, but like everybody uses QALIS. But it's like, whatever, I already spent $100,000 on Supersec 2.0 and whatever. So let's just keep throwing good money after bad, right? If it doesn't do what it needs to do, that doesn't mean you have to throw out what you have. Because again, Supersec version 2.0 kind of thing might serve a purpose, but it doesn't serve this purpose. So you might need to add and think about it. You can't be like, well, this is our one thing we do it. The more that you can emulate whatever you care about testing and production into your pipeline, the happier you will be. Remember, monitoring is just testing with a time dimension. This applies to compliance as well. So whatever you're testing in production, when you've got your scans, you've got your thing, why would you not test that against stuff before it gets released? Because boy, you know what sucks the worst? Is I go to deploy an application or info code or something and I ship it in half an hour later, I get pinged from InfoSec to go, dude, you just totally failed a pen test. I'm like, god, dang it, if I only knew, right? So boy, here comes a vendor pitch, right? Don't worry about it. This is so not a vendor pitch. The vendor pitch is in like half an hour. But I'm just, I'm not here to pitch products, but I'm giving an example from something with Chef because I want to illustrate something. So this is something called InSpec and it's a way of treating your compliance as code. And the example I'm giving here, so I just want you to think about what makes sense in your world. But let's look at the two differences of how you might be able to evaluate the compliance state. So at the top, you've got, you know, good old shell script. So I'm going to grep for the SSHD config. I'm going to make sure that this file's in there, blah, blah, blah. What like InfoSec auditor is going to be able to write that? No offense, InfoSec people in the room. But like that requires deep knowledge of the system. And the reality is what I care about is, is this thing in the state versus being able to say, I have a control, here's the impact. This is how it connects back to my compliance stuff. And all I'm doing is saying the SSHD config because it knows how to do that. The thing is this is compliance as code. Now that being said, if the thing that works for you is having some kind of custom artisanally crafted minimum viable bash that does the equivalent of this, do that, do something. But don't just trust on whatever stuff. So the benefit of this is we're in a state of continuous audit. We don't have audit theater. This is audit theater. This is every audit I've ever been a part of. Okay, we're doing our regular work. We're like, oh crap quarterly audit coming. Well let's work really hard to get compliant. The auditors are there, we are so compliant. Now let's go back into our usual work activities and we totally drift. Sucks, right? Because it's just total fake reality. We want continuous compliance. And remember, sorry, you don't have to slay this whole dragon at once, right? Think about incremental versus iterative. And the idea of this is simply if we're thinking about building, like if we're gonna draw the, paint the Mona Lisa incrementally, that got weird. We would sit and we would do a chunk at a time and a chunk at a time. So what this means is you don't have to like write every single compliance test of your whole organization all at once. Like do one and see how it works and start adding it, right? Start leveraging things that might already exist like Sys Benchmarks and stuff like that just to get some visibility and understanding of like how bad your shit really is. Cause I got news for you, it's really bad. So to review, treat your pipeline as code. Treat everything as code. I mean like treat everything that isn't a person as code. Trust but verify your domain experts, right? So don't be afraid of letting your domain experts contribute but trust the shit out of it. I'm sorry but verify the shit out of it. Focus on the what, not the how. Outcomes, outcomes, outcomes. Use your production audit tests in your pipeline. And again, test. So I'm gonna, I don't, this is just like my little mic drop slide. But just a couple of things. So it's some resources, I highly recommend Sydney Decker's book, The Field Guide to Human Error because remember, we are humans, we screw up. You need to understand human error and we need to work around it. At my GitHub, I have a repo called speaking where I put all my talks so this will be out there. I'm on Twitter, my speaker deck and yeah, go to like listen to a rest of DevOps and stuff. So my timer is up so I don't know, we got a minute for questions or is that rad, is that okay? Okay, does anybody have any questions, comments, problems with your life? Okay, I guess either you're also like blown away by the ideas that you can't formulate a question or your board is shit. So I'm gonna pretend it's the first one. So thanks everybody.