 All right, who here has heard of SaltStack before? Like before today? Okay, okay, good. Who here has played around with SaltStack? All right, we're down to 40%. Who here has SaltStack in a major installation? Excellent, we've got some. All right, that helps me know exactly what to do, what to talk about, how to proceed, et cetera, et cetera. Now, they're loud. I need to get a jazz band in here. This is, I'm fine. I was going to say this is New Orleans. I should have brought a jazz band. Okay, so let me start talking a little bit about Salt. So the objective of this talk is really to give kind of an overarching view of Salt as building blocks for infrastructure management, much more so than a bunch of demos. If you want a bunch of demos, we've got a birds of a feather in this room after we're done. What a great guy. Yeah, I do like him. Okay, so let me start with a quick history of what Salt is, where it came from, what some of those core ideas of Salt are. And it really helps to, I think at least, clear up or define a lot of how it works, why it is what it is. But so the original idea of mine for Salt was that I wanted to create a system, and I'd been working on this idea for a few years, to make it so that I could send a command out in parallel to large numbers of systems and get, and get information back from them in parallel at breakneck speed so that I could then use that information to make decisions. Okay, and so I'd made a couple of these remote execution platforms for different jobs. I made a couple for the US intelligence community. I used to be able to say that stuff I worked on wasn't in the news, but I can't talk about it. So don't ask. Now, but so I started on Salt after I left, well when I was in a job where I could start writing some open source software to try and get this idea, this design of a remote execution engine going. And that was the origin, really. It was in my basement, me writing code to just kind of fulfill this idea of, I bet if we could really make remote execution this fast, then we could do all sorts of crazy things. And so that, again, that's really the foundation of the project. Now, from there, everything blossomed into saying, well, we can do fantastic things. We can come in and put configuration management on top of this remote execution bus. We can come in and put cloud management tools on top of it and complex reactor systems on top of it, things of that nature. And then that last one, the goal of SaltStack Incorporated is to make money. It's a company. It's not a foundation. But I will talk about that strategy a little bit in conjunction with the fact that we stay completely open source. Okay. Now, the vision behind SaltStack is really this idea of state and flow. These concepts being that flow is remote execution that your ability to go out and grab data, modify your system on the fly, anything that you want to do. And make it happen. Have ridiculous speeds. And then state is the idea of saying that we can ensure that a system is in a certain state, set a system up in a certain state. And that those two concepts are, again, this foundation to say we can do that, so we can do anything that we want to do to the infrastructure. Now, I like this second bullet. High-speed communication means no need for a CMDB. We've had a number of users out there who have previously used the configuration management database and then junked it to just say, well, now we just ask our infrastructure what's going on. And we've got a number of users out there who use Salt to hook into a CMDB. And so that flexibility is something that's really, really important. The goal of Salt is that it can be dropped inside of any infrastructure and make it better. Less than saying here is a static prescription of what an infrastructure should look like and more of saying I'm just going to try to make your life easier because your life is different from everybody else's life that we've looked at. Okay. So this idea is what is Salt? We get compared to configuration management tools a lot. And that's primarily because that's where most of the interest for the project comes from. I shouldn't say most. Okay, most is working in the configuration management space. But one of the things that's really important to understand about Salt is that it's more of a concept of these building blocks. I think I saw this phone ring and he wasn't just leaving because I'm ugly. Okay. So it's this concept of building blocks for an infrastructure and the idea behind this is to say that Salt is a tool that can be used to solve whatever problems you're facing in your infrastructure, regardless what your infrastructure looks like. Whether your infrastructure is a couple of machines out on Rackspace Cloud, there's rackers in here and I don't think there's any Amazon guys in here, so I'll try to be nice. Or if your infrastructure is linked in or a large university or a supercomputer. And so these building blocks allow people to do specific things in their infrastructure to solve specific problems. Which is why we go back. We look at somebody like Wikipedia. And I love those guys. I mean not just because they're Wikipedia. But they deployed Salt for continuous code delivery. They didn't deploy Salt in production for configuration management. They manage another configuration management system using Salt. They use Salt to trigger those executions. And again, they hook Salt into their Jenkins server so that when a build is done, the build can communicate with Salt and say kick off the deployment. We've actually got a great representation. We just finished getting the official Salt Stack Jenkins server online. We were just using Travis until relatively recently. And it does some pretty cool stuff with respect to auto scaling and auto cloud management inside of that environment. But so there's a lot of these building blocks. And so I'm going to walk through and I'm going to talk about these building blocks and talk to what they can be used for, how they can be used together, and why putting these building blocks together into one unit gives you a lot more flexibility and power with respect to accomplishing your end goals of building a fantastic infrastructure. Wrong button. I'm fumbling all over the place. There we go. I found it. Now, this idea of remote execution, I've already been talking about this, but this is really central to what Salt is and how Salt works. So the remote execution, how it works is that we send out this very small packet of data with instructions down to all the clients, which we call minions, because it's more fun that way. Actually, I think it's extremely descriptive that there are minions, there are a little piece of software that does what they're told, very dutifully, and like a good minion, it's not very expensive. Now, it sends this tiny packet out and then asynchronously executes those routines. So the asynchronous aspect of this is really important because when you're executing a routine asynchronously, that master system that sent those little commands out hanging on to those connections anymore with respect to those commands. The information about the return is returned to the master asynchronously, but since that's happening, it is irrelevant where that information ends up going. And so we're able to redirect the data flow of all of these executions. This is another great example. We've got quite a few users out there who use this system, it's called returners, so that they're able to execute a routine environment and then have all those systems shoot that information back to, say, a Redis database or a MySQL database or any data. They're all supported. They're all in there now, even though I want more Cassandra support. Does anybody here a Cassandra guy? Nice. Okay. So you're able to redirect and send that data back. So this basic model of saying command run shove the data somewhere is, again, the foundation. Now, the nice thing about the remote execution in Salt is that it's tailored, you know I've made a mess of things again, it's tailored to be very, very fast. And so what happens is that, actually we got a great quote from Harvard University when they deployed us on one of their supercomputers. They came back and they said let's just take us about 15 minutes using an SSH-based tool that was doing parallel SSH executions on our supercomputer. It took about 15 minutes to run a command and get that data back. Now we use Salt, it takes less than five seconds. And so they find themselves and many Salt deployments executing these commands on a regular basis, sending information out to get real-time data about whatever they want. Now the next thing I want to talk about is the event system inside of Salt. Any questions or comments so far? I don't want any arguments. I'm tired. I got up early. Yeah. Okay, so you're asking we execute routines on minions asynchronously. What if they fail? How do we know that they failed? Okay. If an execution fails, then it reports failure data back. So it sends a message back that says that this routine failed and this is why. Does that answer your question? Okay. All right. So event driven. This, you know, I'm normally better at this with this mouse. It's the humidity. Okay. Now the events system inside of Salt. The idea here is that everything that's going on inside of Salt is happening on an event basis. Now the nice thing there is that everything can be run asynchronously. All we have to do is listen to events and then ingest them. And so on the master end, on the minions there's an event bus. Now a privileged application can listen on the event bus or can even fire on the event bus. We've got users who fire events on Salt's event bus expressly to trigger reactions inside of Salt so that they're able to, again, hook in arbitrary systems, give them rights to fire on that bus, and then stuff happens. Okay. But since it's event driven, then we are able to create some pretty slick things. I mean even the CLI, you run a Salt command and the CLI program that you started is listening on the event bus for the information about those minions returning executions. But so the fact that it's event driven, we can, again, do cool things. So the new SaltStack UI, I was really hoping that I would have cut the release candidate for this by the time this talk started. But it's delay after delay. It's done. But I just need to cut the thing for the not that 17 release. But that's going to come with integration with our new UI, which is open source. The new UI is, again, it's an entirely event driven system. And so you plug in and you just watch as Salt tells you everything that's going on on the fly. It's really, really slick. And it's one of the reasons why it took us a while to actually get it to work because slick can be difficult. Yes. So a UI for the minions. You know, I haven't thought about that. But it would actually be fairly plausible to put that together. So right now, yes, it's just for the master. But, Jeff, if you want to, if you've got an application there, then give us an issue and we'll look into it. That's what we do. We'd be nowhere if it wasn't for tons and tons of ideas. I mean, I can't come up with them all. Open source is awesome. Okay. So anyway, the new UI is available. It's out there now in the wild and it does work. But the event system allows us to do some cool things. Yes. How is it going to change? Well, it... What's it look like? I was going to say, I'll set one up during the boff and show everybody, or I'll try to, and it'll be very embarrassing, I'm sure, because it's new and I haven't demoed it yet. And we all know what happens with first demos. But yeah, as far as your interaction with Salt, it need not change. We are still designing Salt from the perspective that anything that can be done in the UI can be done in the command line. We're not doing any exclusive UI actions or activities. On the other hand, there are a number of more complex routines inside of Salt that I will talk about that are going to be much more cleanly abstracted inside of the UI. So for instance, inside of Salt, there's the Salt VRT system, which allows you to create a very, very lightweight cloud, private cloud, using Salt. And they're Salt Cloud. And a lot of those concepts can be difficult to ingest on the command line. So a lot of that data analysis and view will be available on the UI. Is that what you're asking? All right, now going back in and talking about the event system a little bit more, there's a system inside of Salt called the Reactor. Now the Reactor is really slick, and this is again one of those things where we're really taking a hard divergence away from the classical view of configuration management. But the Reactor, if you enable it, it's off by default, but if you enable it, it spins up a process on the master, and it listens to all of the events that are coming in. And then when those events come in, you configure it to say, I'm going to go use this logic to parse certain events, which makes it very easy to be able to say, if this event fires, then I can respond to it and go take care of that event. Now a really good example of this is, again, I'm going to use Jenkins. I think it's a good example. Who here uses Jenkins? Well, what do you use? What? Okay, okay, you win. Sorry, what was that? Cheap city. Cheap city? Okay. These examples could logically work with other such systems. But so, what was it? No, same scenario, but using a Jenkins server when a build is complete, Jenkins is sitting out here on a minion somewhere, a build finishes, it fires an event on the master, the reactor is configured to listen to that event, and says, ah, a build of a certain type is finished, tell the minions that they need to deploy a new app. Or you could configure it so far as to say a certain type of event has finished, go spin up a bunch of virtual machines using SaltCloud on Rackspace, or DigitalOcean, or OpenStack, or CloudStack, or those Amazon guys, et cetera, et cetera, okay? So again, the ability to plug into the event bus, react to real events and do it in real time. Bullets. So yeah, just a couple more examples in here, redirecting traffic, having events that can get fired when something is down, and then saying that we need to put traffic someplace else, having an event get fired when, say you've got a data center that's getting overloaded, and we need to expand the traffic pool to another group, things of that nature. Because the nice thing about the reactor is it's sitting there on the salt master. You can tell that thing to make any change to anywhere in your infrastructure as a reaction to any event that occurs. Does that make sense? Any questions there? Is the horse dead? Okay, now, execution modules. So now we're going to dive into a different part of Salt. So I mentioned earlier that the way it works, we send this little command out to the minions and then they run the routine, right? The question is, what is that routine? All these routines inside of Salt are in what we call execution modules. Now, what these guys provide is this really big system management API. There's something like 150, 160 modules in there. Honestly, I stopped counting around 110. But they contain, again, this vast library of routines that you can execute that covers everything from monitoring activity of specific applications, monitoring generic system activities, setting up users, installing packages, interacting directly with a MySQL database. It goes on. There's lots of them. Now, this is one of those, it's the foundation library for all of our interaction routines. As we move forward, we'll see the configuration management, for example, is built on top of execution modules. The reactor that I just talked about is made to fundamentally execute execution modules somewhere. We use an execution module to fire off the configuration management system so that it's encapsulated and can be called from many, many other directions, making it very, very flexible and layered. Okay. Let me actually open up a... See, I'm going to do this, and I'm going to make a mess of something now. Or I'm not... Well, I've got to be connected to the Internet. I'll open up GitHub on this tiny, tiny screen. And if we go into the Assault Source, under the Modules directory, we see all of these execution modules. Now, the nice thing about these execution modules, apart from the fact that there are many, you can see quite quickly, they scroll through a number of the things that they can interface with. But they're also made... Oh, yeah, we actually have really good support for Windows. I know that probably doesn't matter as much to this crowd, but it is there. Okay. But these are actually made to be also really, really easy to write. And so they're just Python. All right, I may not have picked the best... Oh, that's a good example. So this is the generic service code, which is executing the init system, the classic CSV init system, to start a service. All you have to do is define a function, and you're done. The function works. You don't need to inherit any classes. You don't need to bring anything in. You define a function. What it returns is what Salt returns when you run that remote command. Okay? So they're made to be as easy as I've been able to come up with. If you come up with something easier, please share. We'll implement it. Okay. Any questions there? Yes? Uh-huh. Sorry? We run on Python 2.6 and 2.7. So you do need Python 2.6 on your client system to get it going. We support back to REL5. So we primarily support upstream supported distributions. Okay. All right. So now we get to the part where people actually start caring, right? Everybody loves configuration management. Now, the idea behind Salt's configuration management was to make a configuration management system that doesn't skimp on the power. You can do anything and everything that you need to do in Salt's configuration management system, but to still make it as simplistic and straightforward and easy to learn as possible, which presents some interesting challenges. So we go in and we dive into what the configuration management system in Salt looks like, and it's all about data modeling and layers of data, and then you just interacting on a particular layer that makes sense to you that we try and make as easy as possible. And so this makes it look complicated and hairy, I'm sure, but you only need to worry about up here. The idea though being that, again, you don't have to face with the config management system on any level, which is great when you start diving down into how it actually works, or if you have a need to say, well, I just want to run these two state routines in my entire environment. I don't want to do a full run on everything. I just want to be executing this small set to make sure it's up. Or I just want to execute a single state routine in and of itself and then see what it's going to return or see where we're at there, so that we can have similar controls over what's going on. But so as we work up these different layers that exist in the data model, and then we get up to the top there, it's a system that we call the renderers, which is a word you don't need to remember. But renderers just represent the way in which you define salt formulas. And so since everything's data modeling, that means that how you define those is also technically arbitrary. Now by default, by default I get lost on GitHub all the time. There we go, that's what I wanted. Everything is done inside of YAML. So I'm going to pick a random one which I'm sure is going to be horrendous. Oh, that's not bad. So defining what it needs to do can be very, very straightforward. We're installing Libvert, and it's what's running. Now the magic that we're doing right here is that we're able to embed some Jinja. And so in this example, we've got another map file here that is mapping all these different platforms to what the package names are, okay? If you're only working on one platform, you don't need to worry about that, you just write Libvert there. But these are made to be very generic. I'll open that. Seth made it crazy. You can just use an if-else statement. He wanted to be fancy. Okay, so the end result here is that since we support templating inside of the YAML, it is Turing complete. So you can do whatever you want to there. And also it's very straightforward and easy to read. It's just YAML. So a basic understanding of YAML, which anyone in this room should be able to ascertain in five to ten minutes. Your grandmother might take a half hour. It's YAML. Well, mostly because you'd have to explain to her what a dictionary and a list are. Okay, so any questions there? Now these states that we're looking at here, there's this package installed and service running concept. So part of that question goes back into meeting lost on GitHub again. Oh, hey, it went to where I thought it was going to this time. You know, one nice thing about giving a talk and using GitHub is that if you're looking closely, you'll see that during the talk we've had something like six comments on issues in the last ten minutes. I'm very proud of our extremely active community and the only reason it's been an hour is because I haven't been merging a Polrex. I cleaned that out two hours ago. Very active community. All right, sorry, where is it going? That's right, execution or state modules. And so this is one thing that actually trips people up, so I like to bring it up. So we've got these modules here, which is where all this one-off execution concept's coming from. But states are defined in state modules. And so when we say package.installed, what's going on inside of that YAML file eventually is that it executes... This is so not the right... There we go. This function. Okay? That means that by its nature they're self-documenting inside the code. Now, with that said, they're very well-documented online. But the result is that these are the options that we can pass into package.installed when we define them in that YAML file, they just get passed on into that function when it's executed. Okay? And then if we have time, because I'm sure I'm just eating up gobs of... Oh, yes, gobs of time. So any questions about configuration management? I shouldn't have said that. Any questions about this stuff? Is there anything in particular that you think merits a little more discussion on at this rapid high-level routine? Yes. Somebody else brought XML up. Yeah, no one's ever asked for an XML one yet. But yes, the answer is yes. Um... I'm... I'm... But yes, that's the whole idea of this whole render system is, let's say you love YAML, but you hate Jinja. You can use Mako, or somebody actually wrote a templating language that looks like Ruby templating so that it would be easier for his... for his puppet guys to transition over to Salt, because they use this other language called Wempie. There's... we ship with the Pi DSL, which is a Python-based domain-specific language expressly for Salt. And we've got guys who use that. You can write them in pure Python, where what you do is you write a function that says run and it returns the raw data. And we've got guys that use that. So yes, it could be written in XML, it could be written in Ruby, it could be written in C, it could be written in COBOL. If somebody wants to contribute a PHP renderer, I will happily accept it. I won't recommend it. It will go on the docs. Actually, and let me show you what's involved in a renderer. I mean, while we're talking about it, so to support Genja, it's not a whole lot of code. You just chuck the stuff into Genja. The pure Python renderer, if I recall, has an impressive doc string and not a whole lot of code. So they're not... they're not hard to make, because all you gotta do is ingest a file, make sure it has a couple of values in it, and then spit the data back. It's not a big deal. We could have an XML renderer written by, you know, the end of the talk. Anybody up for that? We've even got a dictionary to XML library in here. Okay. Does that answer your question? Insufficient... and sufficiently obnoxious depth? Okay. Now, this is a topic that comes up, I think, a little more obviously than I originally intended. So when I wrote Salt, I made Salt declarative, meaning you say, this service requires this package. Only try to start this service once we verify that this package is installed. You know, that kind of stuff. And Salt was always made so that it will always execute in the same order. Now, I'm very happy I did that because we had people come back and they said, well, I want it to always execute in the order in which I write it in the YAML file. And my first reaction to that was, YAML's a hash map. It's unordered. But they came back, and so I made it not a hash map. Now it's an ordered dictionary. And so on the next release of Salt, which I'm talking about, they will be evaluated in the order in which they're defined. And then those require statements for declarative override it. Which means that it's really easy for you to say, okay, I'm going to list what state formulas are going to be executed. They're going to be executed in the order in which I include them. All the code in them is going to be executed in the order in which it's defined. Unless I do one of two things, which is to say, yeah, but this service requires this package. Or you can explicitly define the order. You can say order last, or order one, or order 48. And then it'll make sure that those are evaluated at those positions in the ordering scheme. Does that make sense? The idea there being, especially with this new, it's called state auto order. It's the flag. You can turn it on and off. But this new concept of being able to say that they execute in the order in which they're defined is, again, this goal to continually say, try to make it as incredibly easy to use and get going as possible. But not sacrifice any of that power. So this concept I'm talking about where we say this service requires this package and so we call them requisites. And I keep hitting this thing. And we've got a number of requisites in there that do some nifty things. Require says what I just said, don't try and start this service unless the package gets installed or unless this file has been successfully placed. Watch is what allows you to say the same thing is required, but restart this service if something gets changed. Okay? Use says I want to use the arguments from a different definition. And then pre-rack is kind of tricky. Pre-rack says only run this state if you're going to change this other thing in the future, which allows you to do something like say shut down Apache if I'm about to deploy code to this system or tell the load balancer to ignore me because I'm about to deploy code to this system and then bring it back in when you're done. And it does that by looking into the future by looking at that other state that it's observing and running it with testicles true in no op mode basically to say are you going to make any changes? Does that make sense? Does that make enough sense? It was really confusing to write. I don't want to do that over again. Oh, I am the guy who maintains that feature. Okay, so any questions about ordering states? Yes. Okay, do I have that on here? I talked about that. I talked about that. Okay, I don't think I have a slide for that so I'll just explain. Now how salt works is that it takes all of these, I'll just say YAML files, but however you get that data. It makes a big O dictionary. Out of said big O dictionary what happens is that that is called high data. Then we compile that down to a low state which is a list of individual dictionaries which have all the information that's going to get passed directly into that function I showed you earlier. Those get evaluated in order. And so the order in which they're sorted is defined by the order in which they're defined in the file or the order keyword can manually shuffle them around. When they're evaluated is when requisites are used. So requisites are called at run time which means that you set up the evaluation order and then all your requisites fire on top of that. Does that answer your question? Sorry? You think so? Okay, we'll bother me later if it didn't and you're just being nice. Okay. All right. Any other questions on the whole config management thing? This guy may have been waving a five minute sign at me yet and I haven't seen. Oh, thank you. I'm paranoid. Yes. Okay, the question is how do we store configurations? Those can get kind of out of hand and we use some crazy get stuff and it's kind of aggravating. Is that a good summary? Okay. Sorry, I have to repeat the question for the thing. Okay. So what's Salt does is yes, yes, they're by default flat files on the file system on the master but Salt ships with something with a plug-in interface. Salt has many plug-in interfaces on which we have not been able to talk about yet but it ships with one called the file server back-end plug-in interface. The file server back-end allows you to store that data wherever the devil you want and then retrieve it and one of the back-ends that we ship with is called Git. We also ship with a Mercurio one and we're not quite done with the SVN one and the CVS one isn't going to happen. Now, what the Git file server back-end does is it allows you to specify on the master that you're working with a number of different Git servers or Git remotes and you're able to prioritize said Git remotes and Salt translates the concept of a Git branch or tag into a Salt environment and it does it all on your behalf. So all you've got to do is go to your Salt master and say we store our formulas in these Git repositories and this is the order of evaluation. Now, this makes it very, very easy to be able to go and say all our stuff is in this Git repo or these Git repos. If you need to merge between environments you just merge from your Dev branch to your prod branch or however you do that and you can have that entire process managed in Git which is made to manage that sort of process and you never have to push anything back down to the file server or sorry to the Salt master. And then the benefit there being is that if you were crazy and you wanted to store all that data in Mongo then you could do that. Yes. So the question on giving certain groups access to certain systems or certain parts of the system Salt comes with a number of role access controls as well so that the remote execution systems can be managed so you can say these users have access to run these functions on these systems. So that's all built in. With respect to saying this group has access to create these formulas inside of Salt that construct isn't directly built in we would rely on an external system like Git to take care of that for us. Unless you have an idea on a good way to implement it in which case I am all ears I would love to have that built in. Yes. Yes. It's actually quite straightforward. Just define that your development servers are in the development environment and they'll look at that branch and get done. No hairy anything to take care of. Okay. Any other questions? So you know I have no idea what time it is because Chris left. I'm doomed. So I'll just keep talking. Okay. All right no seriously any more questions because I'm about to totally shift gears. All right I'm going to shift gears. Now the next thing you want to talk a little bit about is a concept inside of Salt called topologies. So the normal or classic Salt scenario is you've got a master and it's got some minions and you control them. But let's say you've got seven data centers or let's say you're spread across eight cloud interfaces or let's say you've got a big big big data center and you want to manage everything in individual groups. Let's say you're a university and you want to manage the individual departments. There's all sorts of scenarios around this. Okay. So that's where the concept of the syndic comes into play. You can think of it as being short for syndicate. What it does is it's able to say that we've got masters Salt masters in different disparate locations that are managing their own sets of minions. And then the syndic is a little hook that ties it into a higher level master. Now this enables you to do things like say I've got five data centers and I've got a single top level master that can control all of them but really all the work is being done on those edge masters which makes it really easy to be able to scale those things out. And basically how it works is that is that all of the events that are fired on the master which account for all of the data happening on your minions gets aggregated into packets and then fired back up the line and then replayed on the higher level master. And you can send commands down from that higher level master so that it becomes very, very smooth in that you can still be shooting commands down the pipe. Okay. Does that make sense? I always fear that what I say doesn't make any sense. Okay? Okay. So you can define the... so when you run something you define a target instead of salt and the target you can use a glob or you can use system information or you can use roles and all sorts of things. And so if you define that target up on the master then the target gets propagated down to the edge master and then utilized from there. Yeah, this higher level master sees all of these minions as if it was directly controlling them. And so it would make sense to assign a grain value in salt that's what they're called on all these minions to just say I'm in data center foo. And then you can say everybody in data center foo run this routine. Okay, I am just about out of time and I'm never out of slides. So let me cover really quickly a few concepts left and then ask you guys if you have any questions or anything like that. Salt vert is a concept of a super lightweight cloud. It supports live migration, runs on KVM, is used in smaller environments. It's really not intended to be an open stack style cloud. It's meant to be a... I just need to spin up some blasted VMs kind of cloud. Okay? But it has no external dependencies like the rest of salt. It's made to be extremely easy to set up and extremely easy to use and I assume it's going to get a little more attention when the GUI comes out. See, I'm out of time. See, this is why this happens. See, now I'm down to three. Chris is fantastic. Salt cloud is our utility that allows you to integrate salt directly with any external cloud. And so it supports these and a few more. Even that silly one down at the bottom that you haven't heard of. They're friends. They use salt. It was contributed. I like X mission. And then finally, one more thing that I want to mention is Salt SSH is a new system that's coming in the next release of salt. It allows you to run all of the salt configuration management and remote execution routines over SSH. Now the benefit here is that it makes it really easy for a really small environment where you don't need to worry about the fact that if you run parallel SSH across 5,000 servers, it takes forever. But it makes it very easy to just manage it from anywhere. It makes it easy for a developer to manage a cloud from his laptop, you know, like a test environment. And then all those states that he's building for that environment will be naturally translatable over into a full deployment of salt. Or just to continuously manage a smaller environment. Also, it can be used to bootstrap salt. So you can run Salt SSH with the scan routine and say scan this subnet and just tell anything that'll talk to me to install salt. Okay. But yeah, it's fully armed and operational. I've already had people come up to me and say, everybody's telling me it's only for bootstrapping. So no, no, it's fully armed and operational. Don't fly into the middle of it and shoot the triangular and then it blows up. Okay. Okay, somebody got that. Somebody got my Star Wars reference. Has it really been like 20 years or 30 years? Whatever. Finally, salt is open source. We're not open core. We're not holding stuff back. We're not tying off features. That's not how we roll. And we're the only guys out there that do this. Everybody else is open core. So I'm really proud of that. And I'm also very proud that we've been able to make money doing this as well. All right. Conclusion, salt's awesome. Any questions? Yes. How do we make money? Yeah? Okay. I would be happy to take it. We are out of time, I believe. So if you need to go and you're annoyed by the sound of my voice, which is probable, you're welcome. Don't give me the bird on the way out. It hurts my feelings. Thank you.