 All right, cool. Well, thanks, everyone, for coming. So my talk is about securing OpenStack clouds and so much more with Ansible. So we'll talk a little bit about why the project came about, what we're doing with it, how you can get involved. And then also, at the end, we'll try and do a demo over SSH and the Wi-Fi. We'll see if this works. So my name is Major Hayden. I work at Rackspace. A lot of what I do is working on our private cloud products. So the orchestration, the upgrades, the deployment, the security, the maintenance, pretty much everything around that. So I've been working on OpenStack since the Diablo time frame. I do some stuff for Fedora Linux as well with the security team and the server working group. And one of the very few people, I think, that actually likes SELinux. So do I have any Set and Force One fans in here? Sweet, look at this. OK, well, you're wearing a red hat shirt. You pretty much have to, I think. Otherwise, Dan will come find you. And I own way too many domain names, so please don't give me any ideas. Someone tried two of my last talk and I almost had to go buy one. So to get started, security's hard. That's what people tell me. So I think this is hard because of so many reasons. There's so many challenges that we deal with. There's so many adversaries that we face. And for everyone in here that's at a certain organization, you face an adversary that somebody else probably does not face, and vice versa. But if we really start to think about it, what makes security so hard? So many people say, oh, well, security's hard. We'll never make it fully secure. Or you get the conversation with somebody where they say, oh, we'll make it secure. We'll just unplug it from the internet and turn it off. And it's super secure. Well, even then, it's not really. So what makes it so difficult is complexity. And so Bruce Schneier says, complexity is the enemy of security. And as systems get more complex, they get less secure. And so as we've heard, OpenStack is a little bit complex. You have a lot of actors working in the environment. I mean, Nova has multiple daemons that are doing things for you. You have something managing storage, depending on what kind of storage you're talking about, object storage, block storage. You've got networking going in all kinds of directions that you can configure. It's a very complex environment. But as we think about this, complexity is here to stay. It's not going anywhere. There's more OpenStack projects coming. There's new features coming from OpenStack projects. There's new demands on old projects to get old dogs to do new tricks. So is security a hopeless cause? It's a really good question. Some people say yes. I say no. And so Catherine Tate has a quote. And she says, nothing prompts creativity like poverty, a feeling of hopelessness, and a bit of panic. So how many people have been in the middle of a security incident before at their organization and had to figure something out? A few? OK, I've been in plenty. That panic really, really helps to get everyone energized and engaged. And so is pizza. But I think what comes from this is we have to find a way. So as the environment gets more complex and as we have more systems to deal with and more actors in those systems and more freedom for the users to make good choices and bad choices in the system, we need to handle that complexity. So if we think about it, if we kind of put our DevOps hat on, how do we handle this in the IT realm? We handle it with better designs. Or sometimes for some of our companies it means like actually making a design before you put servers in the rack. We sit down and we say, hey, how are we going to design this for right now? How do we design it for security? And how do we design it for tomorrow? What if it takes off? What do we do? We collaborate when we talk about this stuff. So you'll have a developer sitting in the room with someone that understands networking to some great extent. You'll have a security person in the room. We collaborate in trying to figure out what are all these problems that we're potentially going to run into and how do we design around them? And then we automate and test. I mean, so I think almost everyone here has probably had some level of experience with Jenkins or continuous integration, something like that. You can write those tests that say, hey, look, make sure the service is up. Make sure this system is locked down. Make sure this change has been made and you can test it. So my question is, why can't we approach security the same way? If we're already doing DevOps-type things to solve general IT complexity problems, why can't we solve this with security? Why can't we solve it with design, with collaboration, with automation and testing? And so what I'd like to suggest is that we imagine a world where we can harden servers without disrupting OpenStack. So we should be able to go and apply hardening carefully to any server without disrupting an OpenStack cloud. They'll give you gray hairs the first time you do it. And imagine a world where you have the freedom to tighten or loosen those restrictions at any time. So imagine maybe you say, hey, we're going to harden this system. And then later on you say, oh, wow. Like this system was in a non-PCI environment before. And now it is. We actually need to ratchet some of these controls up a little bit. How do you do that without starting over or causing more disruptions? And so also imagine a world where you can delight auditors with proof of compliance. And so I don't know if anyone's ever delighted an auditor before. I don't think I ever have. If you have, come see me afterwards because it's something we want to work on. Usually auditors are not delighted by the things they see. Usually they get mad at me for things. So I think we can get one step closer to that world with OpenStack Ansible Security. And we'll talk about what that is. There's a link here. It'll be available. I'll put the slides up after this so you'll have clickable things that you can go to. And so you might say, OK, well, what is OpenStack Ansible Security? So it's an Ansible role that applies industry standard security hardening through automation in a flexible way. And so what we've really aimed to do here is make something that's effective, improve security on a hosting, improve the baseline security, but do it in a very flexible way so that if at any point you want to make an adjustment later, take something out, add something in, make something tighter than it was before or loosen something, you have that ability to do it. How many people are familiar with Ansible? OK, good. All right. So let's break this down a little bit, because obviously that's kind of a long sentence there. And so what we'll go through is how we actually build OpenStack Ansible Security, how we start with it, and where it goes. So what we start with is DISA releases a STIG, a Security Technical Implementation Guide, for various operating systems, mobile devices, and hardware. There's a whole array of things that they release this for. And so what it is, is it's a way to configure your system. It's broken up into different pieces, different levels, depending on what you have on your system. But it'll tell you to do certain things like, hey, don't let root log in from the outside. There's other things around rotating passwords, which apparently Ness says we shouldn't be doing anymore. There's quite a few things around dealing with firewalls, what packages should be installed, what shouldn't be installed, what should be running, and so our Pyke release will contain the REL7 STIG final version, which came out just a couple of months ago, and we'll get into that in a second. And so then from there, once we have the STIG, which is a ginormous blob of XML, which I don't know, are there any people in here who love XML? Okay, great, no hands. All right, so we take that big blob of horrible XML and translate it into tasks, templates, and handlers in an Ansible role. So within that XML, there's a whole bunch of stuff, and I won't bring it up because everyone will run out of the room, and it tells you how to check for a particular issue, how to fix that particular issue, and why that particular issue is important, and then if you care about common criteria and things like that, it links it back to those criteria so you can check those off your list. So we turn that all into an Ansible role. And then from there, we go through and say, hey, which one of these is gonna disrupt OpenStack? Which one of these would disrupt a LAMP stack? Which one of these would disrupt, you know, a Python deployment with Nginx and Uwiski? And so we go through there and try to understand, is there a way we can adjust this so we could get that security from it but have it be less disruptive? Or we ask the question, hey, is this something we should just turn off by default, but leave the functionality in there so that someone can flip a bit from false to true and have that turned on if their environment is okay with that? And so I also have a note there at the end that this includes documentation and tons of functional testing. So every commit that we do goes through Jenkins, goes through CI to make sure everything is being applied properly. And then finally, the role gets some final tweaks so that it works well on multiple distributions. So every Linux distribution has little quirks around how they handle security. The Stig is actually produced for Red Hat Enterprise Linux 7. It's very easily translatable into SenseOS. I mean, it's almost completely translatable. And what we've done is actually said, we do a lot of deployments on Ubuntu as well. There's no Stig for Ubuntu at the moment. So we tried our absolute best to translate that into what works well for Ubuntu. And that's more than just saying, okay, we use YUM on one and app get on the other. It goes into, hey, there's a PAM module available for RHEL that's not available for Ubuntu. So how do we find something similar? Like how do we get an equivalent over there? And so if you're looking to deploy it, this is what we currently support. So we support Ubuntu 16.04 and 14.04, although 14.04 is deprecated. CentOS 7 and Red Hat Enterprise Linux 7. And so all of these are tested on every commit except for RHEL because it's not a completely free OS so we can't get that in the gate. But CentOS 7 is very close. We cover X86 and PowerPC architectures. So thank our friends at IBM for coming by and helping out with Power architecture. And you can deploy this with or without OpenStack. So as I said before, if you have a LAMP stack running on a system and you say, man, we got it hard in the system. Well, as long as it's running one of these OSes on one of those supported architectures, you'll be able to apply it. And this also works for new or existing systems. So if you're doing a green field system where everything is completely brand new, you can apply the hardening and then give the VM or give the system over to whoever's deploying their stuff on top of that. Or you can actually wrap it into like a golden master image. So do a deployment, apply the hardening standards, make your golden master image and then give that to other people. And then it also works for, I guess, with the, I don't know if it's acceptable term, but the brown field deployment where it's like you already have something that's deployed and then you wanna add this into it. So if you have a system that's been online for a year, running something, you can actually go pick this up and deploy it to it without disruption. And so what you get with it is that it's idempotent. It's highly configurable. And what I mean by those two things is that you can go and configure this, run the role, and if you run the role the second time, you should see that nothing has been changed. So you can actually come along later when the auditor shows up, re-run the role and say, look, there's no changes. Everything has already been applied on the system. And you can also get a report. As I said before, there's zero disruptions to an existing system. If you find a disruption, it's a bug. And so be sure to lodge a bug if you try it. You can also do a read-only audit of an existing deployment. So if your auditor's there and they say, we need to verify that you have applied your hardening standards, you can actually take that, run it in audit mode, then it will go through and print out. And at the very end, it'll say, change equals zero, show that to your auditor, copy the output, and provide that to them. And then as I said before, it's regularly tested with and without OpenStack. So our gate jobs actually apply it to just a bare Ubuntu and CentOS VM. But within OpenStack Ansible, which is a deployment framework for OpenStack, we apply it in every single gate job that we run. So we deploy the whole gamut. We deploy Swift, Nova, Cinder, Glance, Keystone, you name it, and we apply the security role before any of that gets started. And so maybe you have the question of, okay, well, this sounds pretty interesting. How the heck do I get started with this? So if you're an OpenStack Ansible user, does anybody use OpenStack Ansible in here? Maybe a few hands? Oh, nice, okay, great. If you're using that, it's been included since Mataka and it's been enabled by default since Newton. So you might actually not even know that it has been enabled. And if you did, that means we were successful. So you have that ability to turn it off and on if you do want to do a deployment without it. And if you're just a general Linux user, you can install the role via Ansible Galaxy, which we'll go through the demo of that here in a second. And you can use this as standalone. So you can just run this, have a playbook that all it does is apply the role and that's it. Or if you have an existing playbook that deploys your application, you can just add this role in. Have it run before, have it run after, have it run in the middle, it really doesn't matter. And so in developing this, I've gotten a lot of questions and comments. And one of them was, aren't Linux systems secure already? So they're consistently inconsistent about how they do security. You'll find on some OSes, there's a huge emphasis on preventing bad things from happening on the system or certain CIS calls or something like that. And then another system, it's a little more focused on user-level security and some focus on both. But you have that problem and then also configuration drift happens over time. So let's say you have a situation where you deploy a system, you disable root log in. And then maybe a CIS admin that's a little bit newer realizes that he or she can't log in as root and then go in and change that to enable it. Obviously that would be really bad when the auditor shows up. So you can actually have open-sack Ansible Security run in a cron job and have it run all the time and send you an alert if anything has changed. Or you can just have it reapply the configurations every night, every Sunday, whatever you want to do. And so some people say, okay, well we have this open-scap thing, it's fantastic. And it is great, like I love it. And if you haven't tried installing CentOS or REL in a while, give it a try because in the actual Anaconda installer there's an option to pick what open-scap profile you want to apply. So you can actually apply hardening directly that way which is great. The challenge is that it's a little bit hard to tighten and loosen the restrictions with open-scap and it can be a little bit challenging to integrate with a system if you already have it deployed and it's already running an application. And there's lots of XML. So XML makes me kind of scared and so there's quite a bit in there. It's doable and there's tools that help you manage it but you still have quite a bit of XML. And so what's next for us? So we've had folks show up and say, hey, we want SUSE support. And so we've said, come on over, let's work on that. We're also looking at getting it so that it works for Amazon Linux as well which is not too far of a stretch because there's a lot of similarities that sent to us. And then also we've had a couple of folks show up that want to do it on ARM. And so some of the things around audit D and auditing certain syscalls is a little bit different on ARM as it was for power. So we're looking at making those adjustments. And also we want to have it easier for auditors to actually get the output. So if your auditors say give it to me in CSV format or give it to me in a pretty PDF or whatever, we want to get a little bit closer to that. And there's, the reason I wrote ARA right there is there's an Ansible project. And I forgot what the name of it is. If somebody knows what the acronym means, shout it out. Ansible run time analysis. Ansible run time analysis, thanks Mark. And what it does is it gives you kind of this gorgeous output of exactly what happened with your Ansible playbook. So if anything failed or whether it was changed or not changed, it's all color coded in a graph. You can export it, you can put it in PDF, whatever you want to do. So now we can attempt to do a demonstration. And we'll hope that the wireless holds up. And I'm going to run this over to the other screen and then work off this TV down here. So this is going to be ultra awkward. All right, so I've got one CentOS 7 VM and one Zennial VM. And I can show you how we go ahead and install that in both of them. So obviously the easiest way to get the roll down is just use Galaxy, Ansible Galaxy. If you're not familiar with that or not comfortable with that, you can always just use get to clone the roll down to wherever you want it to go. So we'll do Ansible Galaxy. Man, this is really awkward. Let's see. And since it's a demo, we should really push the boundaries here and just install directly for master. So we'll just let's see. I am going to do it. All right, here we go. So all you got to do first is we'll just make a really simple playbook. A really, oh, did I already make one? Oh, fantastic. So what I've got here in the playbook, if you're not familiar with Ansible playbooks, it's all based on YAML. So what I've said is that I want this roll to be applied to all my hosts. And the roll is called Open Stack Ansible Security. That's pretty much it. So if you want to do this standalone, here's exactly what you would do. And so if we quit out of there, make sure I've got a host file in here as well. And so yeah, we're just doing it here on the local host. You can always deploy this remotely as well so I could go from the laptop to another host. But I don't trust the wireless that much. So we will do it locally. And so what we'll do first is we'll pretend like you're a sysadmin that's just been given this roll. And you say, look, I want to go see how far off my system is from where it ought to be. So we'll do it in check mode. So it'll run through and gather all the information that it needs from the system. And then all the numbers that you see here that start with v, these are the numbers that actually come from the Stig. So if one of these had an error or if one of these had something saying that, hey, you did you made a bad life choice in here somewhere when you deployed a system, you could actually go right back to the Stig and understand why that's a problem or what kind of adversaries might use that. So obviously, this runs all the way through. Any of the ones that you see shown is changed. So for example, you see down here at the end you see changed equals 19. That would be a sign that, OK, there's 19 things that the roll wants to do to my system. And you could go and review each one of those individually. I'll show you a little quick example from the documentation here in a minute as well. So now let's say this system right here would be a Greenfield system. There's nothing really installed on it. But let's say you want to actually go ahead and apply this roll. So what we would do is just take the check off. And so what the roll will do is the playbook will run through and apply all these changes for you top to bottom. And then we'll test the item potency there at the end. So there's a lot of the components that you'll see in here are adding and removing packages. There's some parts about installing a virus scanner, although that's disabled by default because I know you talk about virus scanners around Linux folks and they kind of roll their eyes. It does quite a bit around configuration of user authentication. Quite a bit around like password rotation and things like that. And so while that's running, it shouldn't take too much longer. Usually the first run, depending on how fast your network connection is, is usually like one to three minutes, depending on what you enable. If you start enabling a whole lot more of the features, if you want to initialize aid to do auditing on a regular basis, that takes a little bit longer, of course, because it has to audit through the system. And it's almost done. And so yeah, it also secures graphical components as well. So if you have graphical logins on your system, it'll go through and apply login warning banners. It'll go through and apply screensaver requirements. It'll go through and do all that. And I think it's almost done. There we go. OK, so you can see there's 101 things that it did not have to mess with. That includes a lot of things that were skipped as well. So for example, there's something skipped by default that will cause problems on your system. And those are all thoroughly documented as well. And so now if we actually went and run the roll again, we should see change to equals zero. If we don't, we have a bug. But that's one of the things that we check in our gate job. So if there's anything that would, when you rerun it the second time, if it makes another change again, we consider that a bug. So we want that to show change equals zero. There's some new RPMs on the system, so that'll take time. So while that's running, let's see. What I can do, let's see, do I have an install? So this is on Xenial. You know I always have that problem where when people are looking at you typing, you always type the wrong stuff. It's harder when you have this many people. So let's see if that finished. Still going, okay. So if you actually go into where the roll's installed, or you could just review this in GitHub as well, there's a default file that's provided that allows you to go and specify what you want turned on and off. So we have rail six content in here, and we also have rail seven content. And that's talking about the STIG content itself. That's not talking about applying to rail seven or rail six. But let me see, if I get down to where the rail seven content is, there's quite a few things in here. You can actually roll through here, and it's explained in a way that you don't really have to go run to the docs. Hopefully everyone can read that. But if you see about the third or fourth line there, it says initialize the aid database immediately, and that may take time. So if you're on a system with spinning disks, or a system has a lot of files on it, you may want to wait and do this to later. However, if you wanted to go ahead and do this right now, you flip no to yes, and just rerun the playbook, and it'll take care of it. So there's quite a few other things down here that are explained in more detail. So certain things that you want to audit, certain syscalls you want to audit or not audit, you can put those in here. The ones that are marked as no will cause problems on your system. They'll cause like a whole bunch of logging, some things that may or may not be valuable. But you may find on your system that every single time somebody runs Chimad, you want it audited. And that's fine. It's just we've found that most people have come back to us and said that creates way too many logs in the system. And so if we go back over to the other screen, it's done and changed equals zero. So that imposes where it's great. We know our gate jobs work now. And so what also comes along with that is documentation. If I can get this window to come over there, I'll make this a little larger. Ooh, maybe too large. Okay, so the documentation provides getting started guides around what's gonna be changed, what do you need to know about? Also, there's a great section on deviations. And that's probably worth looking at, especially if you're, you know, before the auditor shows up, or if the auditor says, hey, what do you do and what do you not do? There's actually documentation right here for the reasons for the deviations that you can copy and paste and give to the auditor. So for example, the aid initialization that I talked about earlier, it explains why that can be a little bit challenging on a system that has a lot of files on it, why you may not wanna do it in the middle of the day when you're applying the security hardening role. And then every control is provided with details on exactly what it is. And so you have that ability to actually go in here and say, hey, wait a minute, go tell me all the ones that are opt-in. So for example, we have 45 controls that are labeled as opt-in, which means all the functionality is there to enable and apply the control. All you have to do is flip a no to a yes. So there's, let's see, let me find a good one in here. So yeah, there's password requirement rules, like requiring all passwords to have at least one uppercase character. In some environments, this could actually be kind of bad if you don't communicate it to your users first. And so we also provide instructions there around, hey, it actually would be better to use Active Directory or LDAP or Kerberos or some other functionality rather than doing this. So there's recommendations on how to improve that as well. And then also you can actually go look at the controls for a particular tag. So we've tagged all of these controls based on what they apply to. So for example, if you wanted to go review those audit controls, you could actually, at the speed of the wireless, you could come in here and take a look at each one of them individually. And you'd be able to link back to a common criteria. You'd be able to get instructions on, hey, for example, this one right here is like, what do you do in the event of an audit processing failure? And it goes into detail, hey, if you set it to zero, one or two, this is what you get. And so the documentation is available for all the releases. So you can go and grab an older release. For example, we're gonna be removing the REL6 content that gets applied to Ubuntu 14.04 soon, but you can always grab an older release and apply that as well. Let's see, all right. So we'd love for you to come and join our community. So we're part of the OpenStack Ansible team overall, but as I said, the role works with OpenStack Ansible and without. So come and join us in pound OpenStack Ansible on FreeNode. You can email the OpenStack dev list. You can put just OpenStack Ansible in quotes or put security in quotes or my name in quotes or something broke, whatever you like best. And then the repo is right there as well. So it's OpenStack slash OpenStack Ansible security. And that's all I have. So what questions do you have? And if you yell at me from the chair, I'll say what you said again, but you're welcome to go to the mic. So what I'm wondering is, how does the contents of the Stig get translated into playbooks? Is that an automated process from the XML or was everything laboriously handcrafted? Laboriously handcrafted is more what it was. So if you actually go dig in the XML provided with the Stig, it has commands that you can use to fix the issue. So it says, like, hey, put this audit rule here or do this like this. But sometimes the things that it has on there are very heavy handed or they can be problematic in an existing deployment. So a lot of that stuff we've gone in there and said, hey, how do we apply this in a sensible way? So for example, there's, trying to think of some good examples, in the instructions or whatever, or if you ran OpenSCAP or something, it may restart a service immediately. Whereas we say, hey look, we'll change the config and then we'll alert you at the end that you need to restart the service or something like that. So and of course, when those get translated into Ubuntu as well, you can't take it like word for word from that. So if you run Set In Force One on Ubuntu, you don't get a lovely response like you do on a Red Hat based system. So a lot of those just had to be meticulously translated. The nice thing is, is that each one of the tasks, and I guess I can actually, let me see if I can bring that up. That's one thing I forgot to show. Let's see, where was I? So if I actually bring up a, I'll part of the, let's see, task, row seven, let's go into file perms. So these are, all the tasks are broken up into separate places too, so it's pretty easy to find them. If you actually go take a look at the tasks, every single one is tagged, not only with the actual tag that's associated with it, like what kind of work is being done. It also gets a high, medium, low, which corresponds to the STIG. So obviously a high would need to be applied to a high security system, and a low is something that would need to be pretty much applied to any system that's on the internet. And they're also tagged with the STIG ID as well. So that V-71849, you could take that and go to the OpenStack Ansible Security Documentation and read in more detail what that STIG control is about, or you could just go straight over to the STIG documentation and read it there, and you'd be able to compare notes. Okay, so the question was, is it integrated all with triple O slash director? No, but I have been talking with Keith Basil a little bit about it to see if there's something we could do. I do know at the Red Hat Summit there was a really good talk, and yeah, I see some familiar faces out there, about doing STIG hardening among other things for RHEL OSP. So I think it would be cool to collaborate on that kind of stuff, and yeah, we're still talking about it. Okay, so the question is, are we looking at how this would apply to containers? So our use case has mainly been VMs and physical hardware, but I would like to take another look at that use case, because I think it's something, if you had a Docker container, this could be one of the steps of creating a container. Yeah, yeah. So yeah, the comment was, if you ran out something like Atomic, well Atomic has OS tree as well, so you can't tinker with the system. And so yeah, we have some folks that have come by and said what they actually do is they deploy a node with this, and then they apply it, and then they capture the image, and then that's what they give to their people to run. But in the end, you could easily put it in a provisioning step, like in a kickstart, or the Ubuntu equivalent of kickstart, which has escaped my head there for a second. But yeah, you can wrap that into the provisioning process pretty easily. Oh, oh, so the example I was just doing there, Ansible actually just connects to itself without using SSH. However, like if I was doing the deployment from my laptop, you just use whatever is available over SSH. So if you wanted to do Kerberos, if you wanted to do LDAP off, you would just handle that on the server side, whatever you're connecting to would handle the SSH authentication. Oh, oh, oh, I see what you're saying. Okay, so I guess the question is around, how do you more safely handle the keys that you're using for Ansible to log into another host? So my number one recommendation would be to use Kerberos and just get rid of the keys. But if you still had to use keys, there's a product called Teleport. I think there's another one called ScaleFT that could do more of that key slash certificate management where you acquire a key and it's good for one session or five minutes or whatever, and then it gets thrown away. So there's some options out there for that. And I think that's one of the nice things about Ansible since it uses SSH. If you can make the server side authentication component work with SSH, you can pretty much do whatever you want. Well, Kerberos is awesome for that. All right, what else? So the question is, how does the stick compare to the CIS benchmarks? So when we originally went to go down this path, we went to CIS and said, hey look, we would love to work with you and do this. I think their business model did not work with doing that in an open source way. There's been some things that have changed the CIS recently, especially around how they've licensed their benchmarks. But still, it's like whenever I've run it by the OpenStack Foundation or our legal team at Rackspace, they're like, okay, maybe there's still some gray area. So the CIS stuff is great. In my experience, a lot of it looks like it's copied from the stick. Like someone just went to the stick and kind of extracted, there's a couple of extra things or whatever. But in my experience, the stick is pretty comprehensive. And CIS includes a lot of that with maybe just a few extras. But yeah, we wanted to do something totally open source that wouldn't use like the syscat tool or anything like that. So the question is the stick frequently updated. It depends. I think it depends like what type of community you're looking at within the stick community. I think Windows probably gets more updates than others just because that community is fairly large within the government. I think the rel one gets fairly frequent updates as well. But then I think that really is, I would, and I'm, I don't fully know this part of the process well, but I think that comes from a lot of people at Red Hat pushing that along and collaborating to get it done. So I know Shawn Wells at Red Hat collaborates a ton with people of the US government to just kind of keep those things going and improve them. But like version, the pre-release version 0.2 of the rel sevens dig, I think came out like last summertime. And then the final didn't come out until right now. And I don't think there are any, or like two months ago. And I don't think there are any releases in between. Well, so yeah, so the question is, is there a delay from when there's an update? Yes, there is. So we implemented the, the pre-release that came out last summer, we implemented that during the fall. And then the final release that came out in February, I think we implemented that in early April. So there is a little bit of a lag, but a lot of that comes from the, the one thing that really got us on this one was the pre-release had a completely different numbering scheme from the final. And then on top of that, they reuse some of the numbers and then throughout some controls and added in new ones. So it was basically like you had to go one by one and sort it out. So that was kind of troublesome. All right, anything else? Well, thanks everyone for coming. I appreciate it.