 Who I am, just really quick, I'm a software engineer as chef over the last six months. I've been focused on some inspected related projects, compliance, kind of things. And what I'm going to talk about today is just a little bit about infrastructure testing, and then I'm going to show you some really awesome toolings for infrastructure testing. So what is infrastructure testing? Do all of you kind of familiar with infrastructure testing? Yeah, mostly. So it's traditionally done, it's been traditionally done with kind of one-off things manually. Then, of course, all the big management tools have made it easier to do it. But people are using test kitchen, server spec, other kinds of things to do infrastructure testing right now. And it's going well, but integrating that into your pipeline is a whole other story and just making those tests as solid as you can has been kind of hard. So at Chef, we've been working to make that a little bit easier. So I've got a question for you, this is kind of the audience participation a little bit. So what kind of tests do you guys usually do for your house? Data tests. Yeah, right. So you run those tests after you've deployed before? Before. Before, right? So what about your infrastructure tests? Are you running those before you deploy? Yes. Some of them? So we should integrate those into our continuous integration pipeline, right? A continuous integration system. So this is a screenshot of a CI pipeline that we have at Chef, kind of automated. So I just kind of dropped a screenshot of a CI pipeline for this. So we've got our unit test running. And they're running great, they're in the pipeline. But what we can do is we can create something so that we can easily run these within our pipeline and easily run the infrastructure tests within our pipeline and then get a report each time before it goes out so that we always know if our infrastructure is safe or not. Because nobody wants to hear their infrastructure is failing at campaign. It makes it totally very sad. And galley has to get up in the night to fix some galley infrastructure. You're right. We kind of know about infrastructure tests and we like infrastructure tests. What kind of tooling are you all using right now for infrastructure tests? Right. The service bag test kitchen is kind of the usual answer there. So there's been some new tooling in the works for the last couple years. And that's InSpec. So I'm going to tell you a little bit about it. So InSpec is an open source framework for testing your infrastructure. And it's a command line tool. You can do whatever you want with it. It's platform agnostic. We don't care. You can run it. And InSpec doesn't care if you're using Chef and co-public. You can test your infrastructure with SSH, WinRM, Docker. InSpec is standalone and doesn't care what you're doing. We just want you to be able to do the thing. You can test your AWS security groups within the system. So how many of you have had problems with OpenSSL in the last few years? I actually think I saw a post. Yeah. I saw a post, right? Yesterday that there were more vulnerabilities coming up. I think the people were working on doing InSpec for a while, right? He came out yesterday or a couple days ago. So we all kind of came up with our own solutions to figuring out those things. But this is what InSpec example is like. You guys are probably pretty familiar with it. It's pretty easy to read. You've seen this kind of thing with service bag, with RSVAC. It's the same stuff. I'm going to go into a little bit about how we use InSpec. And then I'll go back and talk about that. Because that's just the described block and that's cool. But there's so much more you can do with it. So InSpec is a gem. You just gem and install InSpec. We don't put anything on your production servers. Just run it. You have SSH, WinRM. So it's just a tiny gem you're installing. And you can run it in the command lines if you choose to. With InSpec and Sec. So you're going to get a little bit of an output like this. You're just running describe blocks. You'll see something like that. That just tells you what it was, did it pass, how many failed. It's a nice, easy, simple way to run it. And if you notice, in the last slide, there was a little card there that says target local. So that target local is going to tell you where you're running your tasks. So if you run it against other places, you'll see that the little target changes there. So you can run it against your module, against your set of tasks. Whatever. I'm going to show you a little bit about that. So I, well, if my computer wants to open me, there we go. I'm going to show you a little bit about this. So I set up a couple, provisioned a couple little machines before I got here. So you can see in our vagrant file, I have two machines here. One was set up with the WordPress simple role. The other one was set up with our WordPress. That's our secured and our Ubuntu local one. So if we go through and we look at those roles, you'll see the WordPress simple is just doing WordPress and connects. It's got nothing special. But if we take a look at our secured machine, we'll see that we think did up with a few more things. I'm going to give you guys the links for those in a minute. But just for now, just know that we beefed it up with some other chef recipes to just make it a hardened server. So if we go through now and we say we can use InSpec not only to run the test, but to also check what platform we're running on. So if we InSpec, no, that's not right. Hold on. So if we check on our Ubuntu machine, we're going to see that InSpec detects. You can ignore all those end results. And you're going to see that it tells us exactly what we're running. So if we do the same thing for our secured machine, we should get the same output because they're the same thing. They're just a little bit different with their roles. There we go. Sorry. Okay, so we see that it's the same thing. So now we have in here something called test OS hardening. That's what InSpec calls a profile. But what we're really interested in is just looking at what's in there. So right now if we go into the list out what's in test OS hardening, we'll see there's a controls part. And that's the part we're really concerned with right now. We'll figure out the rest of it later. So if we list out there, there's a few different files in there with some tests. We can check out this first one. It's kind of a big file. So we're just going to take a quick look just so that you know what I'm testing things with. So you can see in this file we've got some Etsy groups and password-y stuff. We have some login depths that need to be defined. We're making sure about, you know, execution rights, all that stuff. The articles. And now we can see how our Ubuntu machine kind of pops up there. Because so we're going to run that profile against that machine that we provisioned. And we see that we've got a few values. It's not doing so hot. And we can see these failures right here, right? So we know we've got some stuff going on. If we run it against the hardened server, we should see some different outputs because we took care to actually make sure that one was set up. So just come over here. And here we'll see, hopefully, that everything passes. Because that was the server that we provisioned with all the extra ones. And that's going to take an extra little second here. There we go. So that's kind of my example of how we run InSpec. So that you can see what the difference is and why it's important. Those profiles that our children are going to be looking for, they're there. That's just some different chef cookbook and then an InSpec profile that we have on. One thing about the described blocks. We've seen these described blocks before. So what's so special about InSpec? The thing about InSpec is that it makes it really easy to understand because InSpec has a concept of a control. So a control is basically like a wrapper around that described block that makes it easy for everybody to understand what's happening. You can give it an ID and then you can give it an impact. And the impact is what tells you how bad the test behavior. We're saying if you have an impact of 0.3 or less, it's a minor failure. 0.4 to 7 or 4.6 is going to be like a major. And then 7 to 1.0 is a critical thing. So that way you can decide each time in your test, in your controls, how important this one is. Because not all of our infrastructure tests are going to pass all the time in real world. So this way we can decide what actually stops our pipeline and what just kind of tells us, oh, we need to take a look at that spot. The other cool thing is that you're able to include a title and a description. So anybody who comes in can see easily what this test is for. Because as much as a lot of us can read these tests, a lot of people can't read these tests or don't know why. Or even if you can read the test, you might not know why you should do it. And so including that title and description allows everybody to communicate clearly across the organization because we can tell everybody, well, hey, look at the description if they need to know why we need to run these tests and why it's important. So INSPEC has a lot of different commands. And I'm just going to go over a few more kind of cool things with it. We saw the INSPEC detect earlier. We can use that to check out what's happening on all of our systems. Anything that we're running before we run it so that we know what operating system details we have. INSPEC also has a cool thing called an interactive debugging shell. It's the INSPEC shell. So what you can do is you can go into INSPEC and you can type out the INSPEC shell and it'll tell you anything you want to know. You can run it against a transport tube. So if you want to run it against that provision machine, you just do INSPEC shell dash G, same thing as before. And you're going to find out anything you need to know. So you can check on all the things on your machine without actually running the test if you're just kind of checking out things. It'll also help you write the tests because you can get some help on all the resources that it's going to have. When we've gotten these tests, you can actually decide if you've got, say, a test and you need to just check if one of them passes. You don't have to go through the whole thing of making sure that this one has this impact and this one has that impact. Sometimes we just want to check if one thing passes. One of the things we can do for that is doing a described dot one. And described dot one will pass if either condition passes. So it'll be marked as passing if the SSH can bake, in this case, the protocol is three or the protocol is two. So that allows us to have a little bit more flexibility in our test and it allows us to have some more, kind of, there's more. I mentioned this just kind of in passing earlier. Inspect has a concept of a profile. So what a profile is, is a collection of tests. So you have your file, your test dot rd, and you have a collection of tests in this file. Now, in your collection of tests, that's great. But then, what the inspect profile does is it takes that collection of tests and then you have an inspect dot yaml at the root and that's going to tell you the metadata about the profile. So that's going to tell you, you can give it the name, which platforms it supports so that you make sure it only runs on the platforms it actually supports so you don't get false failures. And you can do, and you can check that profile with inspect checks. You can also bring up a new profile with inspect in it, for example. If we were to inspect in it, you'll see we created a directory there. You'll see we've got an inspect yaml and we have our controls folder set up and inside of our controls folder then is where we would put our tests, for example. Now it's just set up with a nice little example so that you know what to do, basically. But this is where you would modify all the tests and then you could do whatever you wanted to the inspect yaml to tell everybody why this profile is important and all of that good information. So one of the other cool things about inspect having this concept of a profile is that it allows you within a larger organization. Sometimes you'll have certain profiles that need to be run in certain times and we can use the inheritance feature of inspect to describe which profiles need to be run when and actually go ahead and modify any tests that are in those. So, sorry, I keep saying it wrong. For example, we have in the group inspect we have a folder called examples inheritance that just kind of shows us what this does. So as we cat out the example in here we'll see that we're saying to include the controls from profile and what we do is in the inspect yaml we tell them where this profile is and we're going to skip the control temp 1.0 but we're going to set the control for board n.1.0 and impact of skew. So we can modify all of the profiles so that they follow exactly what we want out of it so that we're not stopping on silly failures or failures that didn't matter. So we can say only run it if it's on this, if it's on Ubuntu or if it's on Debian, anything we want. So for those of you that have been using Test Kitchen all you have to do is add inspect into your verifier as your verifier and then you can use it with all of your same Test Kitchen things that you can use it and you'll see that there's a little part here that says format JSON and where that's useful is if you're going to read in those tests and you don't want just that CLI output you can get a little bit better with it so if we do inspect exact inspect exact examples that example profile and then we do a format JSON we're going to get instead of that CLI output that we were getting before when we just run this we're actually going to get a full JSON format and as you can see by the way this has a bit of a different test output that's because we consider tests to be describe blocks and then anything that's from an actual profile that has controls that's a defined control is going to be called a profile sum so the test summary is going to count all the tests in your file but you're only going to see that if you include anonymous describe blocks that don't have that wrapping control but what's cool about this is that we do a format JSON then we can now put it to JSON and we can use it to kind of visualize all those results so if we pipe it to JQ it'll be a little bit nicer and we can see that we have all of our results here and it's a pretty simple structure it's just the version the version one a little beta one here that's just releasing instead 101 Monday and we've got our profiles it'll tell us everything we need to know about our profile what it supports so where this profile should run that way we don't run anywhere we don't need to and then it'll have our controls and each of those controls then because you might have a few different tests in your control where you're going to have all of your results so that's what makes it kind of easy to consume especially if you're working with a build system and UI and you want everybody to kind of clearly see what you need and what's been passing and what hasn't that's where you can kind of use that to just show them the whole deal without them having to run without them having to look at the see a lot of it all the time which is kind of what we did with the compliance server that cost dollars I know that that cost dollars is the most thing but that's kind of what we did with you for those of you that saw when I was running that profile earlier you might have seen some CIS things but those of you that are familiar with CIS so that's another thing that I'm sorry cost dollars that we have an algorithm that we worked on a couple quarters back to translate all of those CIS profiles that you've ever seen as CIS profile it usually has this wonderful XML with these Oval and XTCDF files that you have to track through the whole way not very pretty good but we translate those into InSpec with our InSpec ESCAP algorithm and so this is just an example of what you can do with that format JSON then you can show people like the profiles of these ones failed, these ones didn't like tips or go to InSpec InSpec.io we're releasing on Monday and I tried to get them to make it for today in another tutorial but they couldn't so on Monday go to InSpec.io because right now it's in the works and what questions