 OK, we're back on time, so I'll get started. Hopefully you can hear me. I'm losing my voice a bit. Mumble mumble, Piston Party mumble mumble. OK, can you hear me? All right, so I'm Steve Baker. I'm a software engineer at Red Hat. In the Icehouse cycle I was the project technology lead. And also during the Icehouse cycle I spent a significant chunk of time working on a significant chunk of what I'm about to show you today. So in this talk we're going to talk about application software configuration using heat. For a start I'm going to give an opinionated view on what software configuration versus orchestration is. And then I'm going to show you the new features that have been added to heat in the last cycle which enable software configuration. And then finally I'm going to give a quick run through of what it would take to integrate any given configuration tool with heat to use this feature. So we've come to the realisation in the heat project that software configuration and orchestration are actually two different things. At a high level view they do have a lot of similarities. Orchestration is creating cloud resources by invoking REST APIs. Software config is configuring software by running commands and writing out files. When you model them you can see some overlap. But the problem is when you put it together you don't always get the ideal scenario. So we've realised that separation of concerns is important. Otherwise you end up with a cat dog. So when you choose your abstraction for the problem you're trying to solve if you solve one problem first then the abstraction might not be ideal to solve the other problem. One of the problem domains is always going to miss out. And that also is not a good thing. Choosing an abstraction involves compromise. So admittedly when heat started it started out as the implementation of cloud formation and that was, to be fair, a bit of a cat dog. So I'll just show you a bit of a cloud formation snippet here. So we've got a resource definition. It's defining an instance, that's all fine. There's some metadata, it's a declarative metadata of software configuration that all looks perfectly reasonable. We're starting some services, great, fine, I can live with that. I can look at it and understand it. So we continue to scroll down on this template and we get this. So we have a user data which is a shell script embedded in JSON which doesn't support new lines. So we've got this, we had quote list join with no delimiter going on. We're inserting values by quoting out and inserting references to other heat parameters. It's not nice. And finally the most annoying thing is that it renders as red comic sans. So that was the situation we were trying to improve. Software configuration definitely needs to be a well solved problem. And heat needs to enable that. Since we're invoking the cloud resources which end up running the software, it's sort of up to heat to hand off the process to some other thing and enable that in a nice way. So the first feature I'm going to introduce is configuration resource. So a configuration resource is an API back store of configuration data. That data can include the script itself. It includes a schema for defining the inputs and outputs that that script will produce and consume. You can also specify some tool specific options which depends on the implementation. Config resources are immutable and can be passed by reference. So if I have a UUID to a config resource, I know I can use that indefinitely as long as it's not deleted. It will never be modified without creating a new UUID, so I can trust that. So configuration resources can be used on their own for boot config. So here's an example. Most cloud OS images these days are using cloud init for their configuration. So now we can expose the cloud init within the heat template. What we've got here is a special cloud config resource which is a config resource that lets you specify cloud config yaml. It's quite nice. You can specify the yaml inside heats yaml. So here we've got a generic software config resource that has a really simple shell script. Up here we have a multi-part MIME config resource which joins two config resources together into a multi-part MIME message which is exactly what cloud init consumes when it boots. And then finally in the user data what we do is have a reference to that one multi-part resource and this server will boot and then execute that cloud config. So boot config is useful for many situations if you just want to do something quick and then kill the server. But it's not enough. There's a couple of disadvantages. One is that I don't know if this configuration is complete unless I add some more resources to this template to specify a wait condition that gets signaled when everything's done. The other disadvantage is that if I want to make any change to this configuration and do a heat stack update, the only option is to kill that server and start up a new one with the new config which is not always what you want. So really there needs to be a way of having software configuration changes that can happen throughout the lifecycle of a server. And that's where the deployment resource comes in. So the job of a deployment resource is to map one config resource to one server resource. It also allows input values to be assigned on the deployment resource. Generally that'll be to assign values from other resources in the stack. A deployment resource remains in progress until it receives a signal from the server that the configuration is complete. This lets it participate in the heat stack lifecycle. So other resources can depend on this deployment resource. It actually, when it's signaled, it can also return output values. So other resources can depend on particular values coming out of that deployment resource. You can deploy on any heat action, not just create an update. So we have a suspend, resume, delete. The list keeps growing. So you might have different deployment workloads that you want to run in particular scenarios. Like an obvious one would be suspend and resume. You might want to mount and unmount your volumes before it goes to sleep. And then finally, you can always get back when the deployment resource is signaled and includes outputs which is the standard and standard out and status code from your server. Sorry, from the deployment application. So here's a really simple diagram of the dependency between these resources. So we have a deployment resource which depends on a config and a server. But for the purposes of what I'm describing here, it's easier to think of the deployment actually living inside the server and the config to find outside. So I forget, slightly more complex. Here we've got three deployments, but two of them depend on the same config. So this is a simple demonstration of why we've got two different resource types just to achieve configuration. We can reuse this config. These deployments can have different input values. So you're sort of instantiating a unique instance of this deployment in two different ways on the same server. So it gives you quite a little flexibility. So here's an admittedly contrived example of two servers with deployments that have dependencies between each other. This stack can't... Crashing can't be complete until every single resource has gone into a complete state. But because of the dependency, they have to go in a certain order. So I'll just demonstrate what that order would be in this case. So first, all the config resources would go into a complete state, essentially instantly. It's an atomic operation, it's quite quick. And they depend on nothing else in this stack. So they'll just go straight to complete. Then the servers will go to complete because they too don't have any direct dependencies on anything else. And finally, the deployments will start ping-pong until it's all done, at which point the whole stack is in a create-complete state. So here's a simplified example of how you could structure a scaling stack. So inside the yellow box in this case is the scaling unit. We're not limited to just scaling server resources anymore. We can scale any collection of resources. But that will generally be a server plus the peripheral resources that hang off it, like ports and pool members. And now deployments. So the deployments will be defined at the same level as the server. And when we scale out, the deployments scale up along with the server that it deploys to. So I've already mentioned you can specify inputs in your config and override those values in the deployment. There is actually some extra values that the deployment resource adds to the config that ends up on the server. And that's these sort of derived inputs. Now most of these inputs are really just to help with the authentication for signaling back to heat. So you can generally ignore them. The one which is probably most useful is deploy action which tells you what heat action this is running in. So you might want to do something different when you might want to have conditional logic in your config which says do one thing on creation, different thing on delete. So we also have the get file intrinsic function. This means that we no longer need to inline our scripts into our YAML which is a nice thing. So Python heat client will look for these get file calls and it will fetch the content, whether that be a local file or an absolute URL, and include that in the stack create operation. We also have initial support for including binary files in that. They'll be encoded and included. But there'll be some extra work in that area. So here's some deployment examples. So this one has a CFNInit config which uses a structured config that lets us declare inline YAML in the CFNInit format. The other feature of structured config is that I've got this sort of get input intrinsic function call here. Now because CFNInit is really quite dumb, one of the things that lacks is an understanding of what an input or a variable is. So we have to do that substitution on the heat side before it gets sent out to the server. So I'm calling getInput here and the special structured deployment interprets that and replaces it with this value before it gets sent out. And then we have a normal software config which is in the group script that specifies what hook should get used on the server to actually process it. I'll explain more about that later. And then we've got to get file call here. It's including checktempfoo.sh. A couple of other things here. I'm naming this deployment 10 and this deployment 30. That will specify the order they get run on the server which is related but not identical to the dependency relationship between those resources. And finally, this CFNInit deployment says signal transport, no signal. So in this case, I don't really care about waiting for that to finish. I don't want to be notified because CFNInit doesn't give any outputs anyway. So that can just go straight to create complete until about this one will stay in progress until they actually receive the signal. Here's the rest of this template. Server resource which makes no reference at all to the deployments or configs and then the script itself which is using those derived inputs I said before. Server ID, deploy action and it's getting the outputs back to the hook by writing out to a known file path. So the result was the name of the output. So really simple. So here's an equally trivial puppet example. We're including a puppet manifest and the rest is much the same. It's a plain old software config, plain old software deployment. Substituting some values. Server is exactly the same as last time and here's a really simple manifest. This one is also writing out a file to get the output back. There might be a better way of doing that in puppet but we can look at that later. And finally I'll show you an image based example. So triple O are using software config with image based deployments which means the vast majority of configuration has already happened at image building time. So all the software's been installed, all the config files exist and have templates. So all that's required is a little bit of data which has configuration specific to the stack gets defined in a block of YAML and deployed to a given server replacing input values with values from your stack. So here we've got a... we're building a URL, mySQL URL which includes the IP address of the server resource called controller zero. So the hooks end up on the server that you're deploying. They're actually quite simple. They consume JSON from standard in, write JSON to standard out. Their job is to actually invoke the configuration script and they map the heat software config concepts to whatever makes sense in that particular configuration tool. So for scripts that's environment variables, for Puppet, the inputs are facts. And the hook also has the job of discovering what the outputs are from that configuration run. So here's the software stack that needs to be installed on the server for this to work currently. So these first three tools came from the triple O project. OS collect config is an agent which pulls from metadata from a number of sources like the Nova Metadata service or the heat API itself. It then pushes this data down to OS refresh config which just runs a chosen selection of scripts in the specified order. And OS supply config is a templating tool which transforms configuration structure to configuration files written on disk. So going further down, OS refresh config will invoke heat config and heat config is a script which has the job of going through the deployment's data that it has and invoking the appropriate hook to perform that actual configuration. And the results get signaled back to heat. So it looks complex but every single box apart from Nova and heat is tiny really, they're very small focus tools. Or not even tools, they're scripts. So the hooks that are available I've already shown you examples of them all. Script, CFN init, puppet and golden image which isn't actually a hook, I'll just go back to this. With golden image this diagram stops here. Everything below this isn't required because there'll be other scripts and config templates already on the image ready to receive the new configuration data. Hook's yet to be written. Chef, there's lots of interest in that so I imagine that'll happen quite quickly. Salt and Ansible are interesting because they're both written in Python. They both use YAML as their native format so it'll be a really good match for configuring with heat. I'm sure most of you know of Ansible as an SSH push base tool but it doesn't have to be used that way. You can also invoke it on a local connection so it configures on whatever machine it's currently running on. And that's the way we'll be using it in heat. It's up to the hook to supply the configuration data and then it just invokes Ansible locally on that. And finally there is PowerShell for Windows configuration. So you can use Cloudbase in it for boot config but it would be really nice to be able to have deployment resources that trigger PowerShell configuration as well so hopefully the guys at Cloudbase will help us with that. So this is my best guess at how you would map the software config concepts with various configuration tools. It's a complete guess, I'm not an expert in any of those really. Probably Chef is the one with the greatest proportion of expertise in this room compared to my profound ignorance but you know, I'm hoping that domain experts with each tool will be helping us write the hook. So, writing a hook. Volunteers wanted. As I've already said, consumes JSON as standard in writes JSON to standard out. Typical hooks we have so far are only 100 lines of Python so they're not complex. If you wanted to you could contribute your hook to the collection that's already in the heat templates repository. But there's no obligation to use the hooks that are there. You could write your own hook which has the behavior that you particularly need. Since it's so small it's not a great burden. So when you're building your golden image it needs to have a few scripts installed. So Triple O have their own tool chain for building images called disc image builder. So we're leveraging that installing these OS config tools installing the heat config script and whatever hooks you actually need you don't have to install all of them. You only install the hooks for the tools that you're going to use. Then if you like you can install the actual configuration tool itself. And since you're building a golden image you might as well install the packages that you need or you could have a reasonably generic image and do the rest of the configuration at either boot time or deployment time. So it's quite flexible on the approach that you can take. So I won't dwell on this slide but I will point out that if you want to check out the current hooks that we have this heat templates path in the heat templates repo we'll get you there. So this creates an image custom image with all the agents we need and uploads it to whatever glance you're authenticated to. So what about the master configuration server? A lot of these tools have their own master server which acts as the single source of truth with heat configuration in the picture now there's two single sources of truth. So the choices are make heat the single source of truth you don't need a master anymore you're still using the configuration tool that you like but now you're getting the values from the service which actually created those cloud resources. By not having a master you don't have to have the complexity of syncing heat with a master config values unless you really want to or you could go the other way I only use heat config to point your server at the configuration master and then be hands off with any subsequent configuration. So how are we going for time? I think I talked quite fast. So the feature landed in in Icehouse a lot of the work we're doing will be in the periphery of heat so it can actually all land throughout the Juno cycle there will be changes to python heat client which which gets released whenever it needs to be and there will be a lot more changes to the hooks and the instance tools so we can continue to mature and you can make the most of the new features without having to wait for Juno which will be nice. But there's the odd thing that we will be doing in Juno on heat itself we'll be looking at other techniques for communication between heat and the server alternatives to the polling and the signaling that we do currently we'll be looking at using Swift as our as our store for the data so we can take the polling pressure for heat itself we can consider looking at using Marconi as the transport to get these messages out to the server and back again and we need to find a solution for how to get servers in isolated neutron tenants talking back to talking back to heat which is a problem that any server with an agent on it has so it would be nice to find a solution for everyone. I haven't mentioned Tosca yet so there'll be one more software configuration resource type which is action aware which means you'll be able to specify different configurations script for each heat action like create, update suspend and group it all into a single configuration resource and then the tooling that I mentioned before can only execute the required config so this is one of the main things we need to have a decent mapping between model and the Tosca model which will make two-way translation a lot easier more hooks already mentioned that Docker integration we can either invoke Docker by booting a nova server which has Docker installed and then using the Docker resource to deploy things inside it it'll be nice to get software config integrated with that similarly when the Docker drive is back in nova it'll be nice to be able to obviously configure things with deployments and Docker images it'd be interesting to think about that scenario window support which I've already mentioned be nice to add and one more thing we need it'd be nice to be able to specify quiescing workloads when a server is going to be rebooted or shut down forever be nice to say please remove yourself from the pool or do whatever cleanup you need to before we shut you down that's tricky in this model because in the scenario where you do a stack update and the server and the server resource goes I need to replace myself or I need to do a rebuild or I need to do a reboot there's two things that happens the old server gets shut down and the new server starts up this will work fine for the start up the update will happen and it'll be configured and it'll be all good but there's no way of specifying what's going to happen during the shutdown so we just need to have a solution for that that's it if you want to ask any questions please come to the microphones so that everyone can hear and it can go on tape the software config and the deployment resource is there a way of seeing them from a CLI to see what you have or what are currently in flight in the system the config resource should be exposed by the CLI you should be able to manage it it's only the heat client library has been implemented so far the deployment could expose it on the CLI it's really a sort of heat internal API thing but there's no reason why it can't be exposed on the CLI and you could in fact deploy things to servers that you can authenticate to outside the context of heat just by creating a deployment cool, any other questions? ok, thanks so much