 Can everyone hear me at the back? Is that okay? Cool. I can see hands up at the back. So there we go. So who am I? As I said, I'm Peter Suda. I'm a senior Pro Service Engineer at Puppet Labs. I've been at Puppet for two years. I've been using Puppet for, I think close to six years now, it says five on there. But yeah, so I'm a senior Pro Service Engineer, so I work with customers. I do stuff like this, I do talks and things like that, but I also work in the community and I like opening pull requests on modules that I use. So I should give a quick warning before I start, is that I speak quite quickly, even now like the brain power to make sure that I don't speed up my normal speed was like this, is like taking up half my brain and remembering my slides in the other half. So if I go too fast, someone just wave or you know just say slow down and I'll see if I can calm down myself down a bit. And also I should probably put this disclaimer up as someone who is not a lawyer or an auditor or you know signed an NDA for your company or anything like that. Make sure that anything I advise in here you don't just, I mean hopefully you shouldn't need to be warned about something like this, but it does happen. Don't just take the stuff I'm talking about and just go and implement it in your company without talking to someone who knows what they're talking about first. So why are we here? Show of hands in the room, who has the deal with some sort of compliance at their company? Well, it's a lot of people, so it makes sense because it's the kind of talk. So what is compliance and what does it actually mean? So there's a pretty good definition here from the Computer Weekly website. Basically compliance the TLDR is that there is some sort of legal obligation or not always legal, sometimes it's just an agreement like a framework, but basically someone has told you that there are a set of rules that you need to follow. And this is a kind of a side thing here, but it's an important note that compliance is not security or more accurately compliance doesn't directly correlate security. It's a part of a secure kind of framework you have your company, but just because something is compliant doesn't mean it's going to be secure. It might help, but it's not just that. And a good explanation, and I'll put a link to this blog post later on, is basically compliance is the discipline of verification at scale. Because the general idea is that if you think about your system, if you had to document every single file that's on there and how they're working, it's going to be pretty difficult. So compliance is basically just a framework around making sure that at your company you've got an agreed upon kind of framework for how you're doing things. And as people who probably put their hand up, so people who put their hand up for compliance, put your hand up if the compliance or stuff you have to do your company just gets thrown over the wall at the last minute. There's a few hands there, okay. So the thing about compliance is it often kind of straddles different organizational silos, at least with some of the customers I've worked with. Compliance is sometimes literally there is a compliance team which is separate from the security team, which is separate from the operations team, which is bad generally. The whole point of DevOps is not just a buzzword, it's for everyone to kind of work together on the same team. But because of that, compliance often ends up being thrown over the wall at the last minute, someone just comes in and goes, oh, I ran a Nessa scan across your network, you need to fix all this stuff, and that kind of stuff needs to be brought left, brought into your kind of workflows. But regardless of what the definition of compliance is or if you even want to do it, a lot of times you don't get the choice and someone's told you that you need to follow the rules. So there's an alphabet soup of all the various acronyms and things that you need to follow there. These are quite American bias because a lot of the research online is American bias, but the ones you probably recognize from up there, if you're not American or you're working in Europe, probably gonna be PCI, DSS, maybe ITIL and some others, STIG maybe if you're doing work in the States. There's a bunch up there. And even if you're not following any sort of governmental compliance or legal kind of requirements, you might have compliance based on an agreed upon framework internally. So you might just have a set of what most people call hardening policies. So they're not directly correlated to some governmental standard, but they're just like this is what we need to do to get stuff working at our company. And in that, this is one that comes up quite a lot, the Center for Internet Security. It's basically a kind of non-governmental fork of the STIG standards. But this is fairly popular because a lot of banks and stuff, they get scared of using something like STIG because it's explicitly made by companies, you know, organizations like NSA and they're a bit worried about that kind of stuff. So CIS is like the CIS is the kind of more business friendly version of it. But essentially a lot of those, and we'll see later on, a lot of those CIS compliance standards are pretty similar to STIG, if not the same. And yeah, another reason the CIS is pretty popular is that it covers a lot of stuff. So it's a huge list there. You can pretty much see most of the things that you can think of, you know, free BSD, Microsoft, various databases, Amazon, Linux and things like that. You know, it covers a lot of different areas there. So the thing about compliance is that another good note on compliance is that sometimes it doesn't directly correlate to actual technical requirements. You have to dig through a lot of legalese to actually find something that you can work on directly with code. So an example is the HIPAA, which is basically just an American law around health insurance in the United States. So this is the actual text of the requirements around that. So if you look at something like that, and if you actually go to that website, you have the full, like, the whole compliance stuff around HIPAA is basically, it was written for senators and for law makers and stuff like that. So you have to take this document that's in legalese and drill down to what you actually need to know. So if we go through that, we need to look at 45 CFR Part 60 and Parts A and C of Part 164. So let's go and do that. So you have a bunch of headlines like this and, you know, if you're bored, maybe you go through each single one of them, looking for them. But the bit you probably care about is this here, the technical safeguards. So technical safeguards seems like something that we care about. So these are the technical safeguards and some of these are more kind of engineering problems and some of them can't really be covered by config management. So some of these will be directly config management tools and some of these will be maybe installed by config management tools. So the idea that you have to have, you know, specific logins and you have auditability, maybe you can't do that with your config management but an LDAP tool that then logs into some sort of, you know, syslog or something like that, which you then audit. So your config management tool helps with that but it can't work directly. And basically the level of compliance you can give and the difficulty you have with compliance is directly correlated with how friendly or how kind of adversarial you are with your auditors. Because at the end of the day the auditors are the ones that actually decide whether you're in compliance or not. So depending on how your auditing is being done, whether it's an internal team or an external team, you ultimately have to work with them and it can be an adversarial relationship but it's a bad idea because, as I said, they're the ones that hold the key to actually getting signed off. So it's good to work with them on this kind of stuff. The problem with this is that a lot of time compliance is a manual process, especially if you're in a company that hasn't gone through this kind of DevOps kind of transformation or any sort of agile transformation there. A lot of these processes are being kept in emails or in PDFs or in dead trees. You've got, you know, I've seen pictures of here is our compliance stack and it's just a stack of paper this big if someone's printed off a stick standard or something like that. Or the worst storage of information which is humans because, you know, humans are an unsolved problem in IT. It's very hard to get information from one human to another a lot of the time, especially if you're in the adversarial kind of relationship there. So, you know, put on my best infomercial voice, there's got to be a better way. So if you think about it, what is IT compliance? There's a series of rules for systems that need to be enforced and reported on. And what is configuration management? It's a series of rules for systems that need to be enforced and reported on. So we can use config management tools for this stuff and if you're here you probably have a little bit of experience with this. So what's the advantage of actually doing that? We've got reduced cost and time per release, you know, automating stuff makes things better. It means that you have a sort of a creed upon standard for things. You've got the idea of sharing and reuse and I'm going to talk about like the size new one said you need to stand on the shoulders of giants. A lot of people have already done the work for you. You don't have to style this stuff all over again. And for example, GDS, the government digital service in the UK, they open source all their work because technically taxpayer money is going towards all this code. So they try and open source as much as possible and then, you know, they've already done the work for you. There's no point in having to do it again. Yeah, and you have a single source of truth. If you have compliance with code then you have a single place you can go look for it instead of having to dig for a bunch of emails and talk to a bunch of people. And at one point I had no arguments about semantics but more accurately it's less arguments about semantics because if you've gotten a creed upon standard at DSL, the main civic language, you don't have to argue about like oh we should do this, we're said oh we do it with orc and that kind of stuff. A few arguments about semantics. Nice, a few arguments about semantics, yeah. And as I said a lot of the requirements for these compliance standards, some of them aren't actually technological things. So if we go back to the the HIPAA standards in there some of them are stuff like you need a security guard outside of your data center and that kind of stuff. Unfortunately we can't automate that stuff yet. So the more time you have to get rid of the technical stuff, the more you can sort of push the soft skill stuff away. So how does it actually look like in action? Let's take a really basic example on how we would actually do this with config management tools. So talking about the CIS standards, you can actually go and get the CIS benchmarks there. So we're going to talk about the central seven machine and here's central seven rule one two three. I just picked the most basic one just to show because I'm going to talk about trying to give as most basic example as I can. Obviously it's going to be more complex ones but the TLDR of this is that you should have GPG check. GPG check is one if you're at a red house system and you want the standards because hopefully your packages should be signed and GPG signed. But it's not the greatest color there but if you can see this is a really basic example of how to do it in puppet. You know there's a million ways you could do this, you could use August, you could use the actual you can manage the entire file but to make things simple I'm just going to use a file line and if you're not familiar with puppet this is basically going to regex look for GPG check one if it's there if it's there or something else it's going to change it. And again as I said I work for puppet but I'm going to try and be as bi-partisan as possible and give the other examples so there might be some chef people out in the audience who probably say there's a bow of doing this but this is just the first one I could find. So similar thing use the replace or add resource and then go through and look for GPG check and then similar in salt look through the yum.com look for GPG check and similar in Ansible and as I said no one put their hand up and say there's actually a better way of doing this because on most experience with puppet I just went for the most basic solution that kind of illustrates what you're doing. So one of the things that comes up a lot so I do pro services so one of the sort of basic requirements for puppet for pro services is someone's come along and told us that we need to be you know PCI compliant or something like that and they want some company managing to do it and there's a few different approaches to how you actually use config management to do this. So one would be you already have a bunch of config managing code and then you basically just make sure that that code you've already written gives you the compliance you need so for example I'm going to give a puppet example here for that yum check we had before maybe you already have a yum module that you've got and you just go in there and change the parameter to make sure that GPG check equals one so that's the and that's the method I recommend the most you should be using the existing tooling that you've got but I talked about silos earlier on sometimes someone's primary use case for compliance and config management is just for compliance so nothing else they just want to use this tool maybe they've got a different tool and they're using this just for this one area so in that case you might use a dedicated CIS module or stick module or something like that and then you actually use it for that way and a lot of people end up using some sort of if they're using their compliance tooling as a scanner as well so not just enforcing it or maybe they can't enforce it because again they're very siloed maybe they've got a change freeze but they need to scan through your system most config management tools have some sort of dry run mode in puppet it's no op in chef it's try run in ansible I think it's dry run anyone is it dry run now that's all check just check yeah that's it and then I think there's anyone salt in the room is there with dry run and salt testing is true okay makes sense so some people you know if they're in a siloed place or somewhere where they can't easily they're moving on that journey and they can't easily just run stuff on their systems they can run stuff in a dry run mode and go through that so there's a few approaches you can do here and when we talk about sharing reuse we don't want to have to write all this stuff all over again because lots of people have done it for us so one of the ones out there is devsec.io it's just a website where there's been a bunch of code written in chef puppet and ansible lots of pre-existing stuff of this and they're not perfect again you know as we said need to find something and don't just immediately implement it there's some comments on there saying this doesn't follow that in the right way but there's some good stuff out there and there's a lot of existing code that you can use so that's devsec.io so there's simp the system integrity management platform this is a kind of is an entire stack written by onyx point so this was something that they built for the nsa and for other and american government systems the idea is that the simp stack is puppet with a bunch of pre-rein modules that follow the simp NIST standards over in the states will be a nsa standard and then they have some cool stuff on top they've actually got some a really cool kind of higher tool that will go through and it will actually write compliance logs and write those kind of reports that were your compliance people like html reports that say this is what's changed from the puppet run you just had and a lot some of that stuff isn't public yet but if you talk to trevor von on the puppet slack in the security room he'll talk about that stuff one way and this ansible log and again this is just some quick perusing as I said I'm mainly puppet and I'm trying to be by to partisan as possible if there's other stuff come and talk to me afterwards that I can add in for next time there's ansible lockdown ansible partnered up with a security partner and they wrote some I think these are the stick standards so they've written some pre-written playbooks for the stick standards and for all this stuff you go on the community hubs so if you go to the forge you go to the cookbooks you go to the galaxy if you look up like a cis or stig lots of people have written this code before you so don't write all the stuff yourself if you have to try and use something that someone's written before and fork it if you need to however there's a two parts to comply to IT compliance there's the enforcement and then there's reporting so conflict management can be used for both but I tend to sort of say that you want to use the conflict management part for the enforcing and use a separate tool for reporting and there's a few reasons for that one is that as you said some of the stuff can't be caught with conflict managers much the other thing is that for certain standards that one yeah so for certain standards there are what are known as ASVs so for certain compliance standards only people that have been officially given the sort of kosher stamp like this is the thing you can do you can do that so sometimes you can't use puppet or chef or whatever because it's not been signed off yet and the sad part is we're a foster so the sad part is there's not as many open source versions of compliance tool because compliance you know it's a big business it's lots of enterprise companies they have lots of closed source software and I'm not saying that those tools aren't good I actually have some I've actually had some pretty interesting conversations with the Nessus team about some of this stuff but you know we're at the foster so let's talk about open source stuff so the most popular one and this is something there's actually I think I looked through the foster schedule and there's been I think there's two or three talks about open SCAP it's probably the most popular one out there there's lots of tooling around it Red Hat maintain a lot of stuff for it there's lots of preexisting standards things like FISMA and DSS and yeah it plugs into a lot of things so the most basic implementation of that is the actual command line tool so this is me running it on a vagrant box I'm just using it I'm just doing a young install to install it I'm running in a vowel with the most basic report and then it gives me a file on this like an actual HTML file that I can look through and see all those compliance standards there's also the SCAP workbench which is just a goodie for this and this works with SSH as well so you can do it on remote systems just means it's slightly easier than having to do it on the command line as I said one of the good things about OpenSCAP is because it's a standard people have written a lot of tools to use it so you don't have to do on the command line you don't have to do it exactly with the the benchmark there are other ways there's APIs and stuff like that so Foreman actually has a plugin for this and satellite is built on top of Foreman so satellite also has an OpenSCAP scanner and funnily enough I just added this slide in five minutes ago because I checked and there's actually a talk about doing Foreman with OpenSCAP so if you want to go learn more about that there's actually a talk tomorrow in the security track so go see that as well as OpenSCAP the other one I saw is Linus this is I think this is fairly new I think it's about two or three years old at this point and it's pretty cool it's open source it works on a lot of platforms it's fairly simple to install I think it's just a a binary that you can get the sad thing is that the actual compliance standard plugins like so where it works is it has like a sort of base system and then your plugins for the various compliance standards you got but the compliance standards are actually they're commercial and they're not open source so you can run it on a basic overall scan that looks something like this can't we see it but the idea is that you run it on your system and it just goes for and looks at the sort of standard stuff make sure ETC password has the right permissions make sure that your grub has passwords and that kind of stuff and the last one which people don't talk about so much is using packer for compliance so there's actually a really good talk by Werner Buck at the last hashi comp and he was talking about how they use packer with AWS and AMIs to do compliance because some of the compliance standards especially on things like file mounts and grub settings and things like that they're hard to do on running systems it's the kind of thing you probably want to bake into your golden image process sort of earlier on in the stack because it makes it a lot easier to do and yeah like you know hashi corp do all some products and packer did a lot of this cool stuff and the thing about packer is it has provisioners for like pretty much all the popular config management tools out there so it has a shell provisioner a public provisioner a chef provisioner so you can use that code that you've already written to use and bake it into your AMIs and your various kind of golden image settings so how do we actually report on this stuff other than using OpenSCAP and Linus and that kind of stuff there's a one known assistant testing DSL so the most popular one who everyone's heard of so who here has heard of service spec or is even using service spec all right cool there's like 20, 30 people all right so it's fairly popular and if you're not familiar with it it looks something like this it's basically if you're familiar with our spec it looks something like this basically sort of our spec helpers to do things like check like files are on system and they have the right permissions so here's a really basic CIS check it's for the one we talked about around the GPG and it basically goes through make sure that's all there so you run this on your system and it gives you a thing saying nope that file's not there or it has the wrong permissions so service spec is so popular that it's inspired a bunch of other kind of thoughts of it so you've got goth which is pretty much the same thing but we didn't go you've got infratester which is basically the same thing but we've reserved keywords for things like HTTP requests and headers and then MySQL and stuff like that you've got gauntlet which is more of a BDD version of this and similar we have BDD security so those are same they're written in Gherkin they're feature files that you run but it's basically wrappers around actual security tools so things like making sure that you're that's more of a kind of app sec kind of side of it but you can do some of that for your compliance stuff as well and even though it's competitive I probably should mention this because it's quite cool tool inspects by chef it's pretty much kind of like a a clean room kind of fork of server spec it works in a similar way it looks something like this again I probably should have chosen better colors for this but the idea is that it just extended keywords on top of server spec kind of language to say you know this is the compliance rule and it's this level of security that has these tags and then the code itself looks quite similar path config file etcum.com should contain GPG check is one and then there's a talk of config managing cal 2016 about using this and actually the beginning part of it is quite similar to talking about how compliance shouldn't be a bunch of papers and it should be actual code that you can give out stuff like that so what do we learn compliance is an enforcement of standards it's not security but it's part of a security flow and in the best case it should be moved left you shouldn't be throwing this stuff over the fence the last minute yeah as you said compliance responsibility ultimately is actually responsible for this stuff is it the ops team is the security team do you have a dedicated compliance team config management fits really well with the compliance kind of use case the idea that you have to enforce standards based on that it's pretty close and regardless of the tool you use there's a lot preexisting work so don't just start all over again and enforcement is just part of the puzzle reporting is out there as well and unfortunately there's not much open source software for this stuff but there is lots of existing tools that you can use but ultimately depending on the compliance standards you're following you might actually have to use a commercial product because they're the ones that have been signed off for it especially things like PCI so if you want to know more there's a bunch of awesome talks that are out there this one from last year's public comp it's actually Trevor talking about simp and how it all works and having open source compliance they talk about more of the complex use cases they actually have some pretty cool stuff where when they were testing all of their stick compliance stuff they were using beaker which is like a test kitchen like for automation tool and they actually caught a bunch of kernel bugs in various Linux distributions so using this automated software and using this they actually caught problems both in the compliance standards and in the actual upstream which is pretty cool and if you've not read the DevOps handbook there's a really good section by Gene Kim and others there's a really good section on compliance and so there's a lot of it's hard to find a lot of real life use cases of compliance stuff because people don't like talking about their compliance because legal stuff people get scared about it but they have a bunch of case studies in there that's really good so I've gone way faster than I thought I would so when I time myself doing this at home I did about this is about 35 minutes but I've done this in about 25 so we've got a lot of time for questions and people have questions no? okay then I guess I've got oh I've got one of here so for the let's say for all the different compliance standards then no support is it accepted by I think for summer is I know for PCI isn't I went through the ASVs for PCI and unfortunately open the Scap isn't but the general use case I've seen for it and talking to companies that use it is that it's a good way of catching that stuff before you have to hand over to the official scanner kind of team so it means that you can bring stuff on earlier on instead of having all that stuff out there and then having it back and going oh no everything's gone wrong the nest scan called the stuff you can bring it earlier on so I think I think the reason that it's not set for PCI PSS is that may not be specific PCI PSS obviously it's like nature okay yeah so the question was is open Scap signed off for compliance standards and the answer is it depends I know for PCI PSS it isn't but yeah it depends on the standards and a lot of the time people kind of use it internally for their ops team and then the compliance team handle the actual kind of official audit compliance there any other questions no okay well sorry about going too fast and come on and talk to me if you have any more questions thanks very much