 Hi again everyone. As Anna said, I'm going to have a quick introduction and deploy steps. I don't have many slides, but I do have a large blog post and a bit of code to show you. So I'll try to spend some time showing you a real example of how this can work. And you, of course, can read the blog post for details. The link will be in the end of the slides. My name is Dmitry, so if you don't know me, I work for Red Hat and Principal Software Engineer currently on the OpenShift team. You can subscribe to me on Twitter, and there is a link to these slides if you want to follow them later on. With that, let's get started. For those who remember, Ironic say, three, four years ago, the deployment looked roughly like this. You prepare a pixie boot, the RAM disk starts, magic, magic, magic, a lot of very complex and sometimes spaghetti code, and the node is rebooted, you are done. That's great, but over the time we've received numerous requests for extensibility or at least for a way to understand what exactly is going on, but mostly to be able to plug that in. So the deploy steps work was concluded to make this magical step transparent and to be able to insert your code or your options into this process. Yeah, so I mentioned the idea is to split one big deploy function for those who remember the deploy interface in Ironic, there's one deploy function that does everything, and into steps and then to be able to insert custom steps between them. When I'm talking about deploy steps, I'm talking about two kinds of them. Out-of-band deploy steps are steps that are executed via the BMC or anyway from the control plane, not from inside the machine, and in-band deploy steps on the opposite I executed from within the RAM disk. The audit, according to the priority, we're using the standing priority, so the biggest, the earliest. And we have split our deploy process into these five to six steps, actually, not all drivers implement all of them, but when you're doing direct deployment, for example, this is the most fundamental steps that will be executed. They are built into Ironic and will be executed by default. Deploy deploy, this unfortunate name is because of backward compatibility. So with this previous monolithic deploy step, it has priority of 100, and it's the only thing it does right now is prepare deploy and start the RAM disk. So contrary to its name, it doesn't actually deploy anything. What does deploy something is a step called write image executed to its priority AG. For iSky, the deployment which we are currently deprecating, it's actually out of band. For the direct deploy, it's an in-band step that runs from within the RAM disk. Then we have prepare instance boot, which sets up boot loader and boot device. It's partially, partly in-band, but the step itself is out of band, but it does go into the RAM disk. Then we tear down agent at priority 40, switch to tenant network authority and boot the final instance at 20. So when we'll be talking about custom deploy steps, you have to mind these standard priorities because your step has to be inserted somewhere between them. If it happens before deploy-deploy, so if the priority more than 100, your step will be run before the node starts the RAM disk. So that's a good place for doing some out of band action, I don't know, out of band trade maybe. If you need to modify the image that already exists, you probably want to put your step after write image. So for example, with 70 or 50. If you want to modify something that has anything to do with boot loader, it probably has to go before 60. And if you need the RAM disk to be running, it has to go before 40, but before I mean larger than. There is even an opportunity to run deploy steps after the final instance started booting. So if your priority is lower than 20, but don't use zero because zero means disabled. Custom steps that I mentioned, they can be coming from the RAM disk in-band steps or out of band. So from the node's driver or more precisely from either its hardware from one of these hardware interfaces. In-band steps are provided by an IPA hardware manager. I'm going to show you one today and usually use priority 99 to 41, because as I explained, you need RAM disk to be booted. So this is a priority range you can use. Next, I'll show you some code and some examples how this code was used. I didn't have time to record the video, but I do have quite a bit of text. The links are here. I'll start probably showing you how an in-band deploy step works. I hope when I switch the tab, you still see something. Yes. Not here. So here we see probably make a font larger. Is it visible? Yes, looks good. Looks good. Yeah. This is the hardware manager. Hardware manager are the ability beats of ironic Python agent. They can be used to change a lot of aspects. So one of them is this get deploy steps call, which is a standard call that returns the deploy steps implemented by this hardware manager. As always with hardware managers, you don't have to repeat whatever is already provided by other hardware managers. So just all this stuff we need. So that's how hardware manager, it has name, it has version. It has a very high hardware support, which makes it have a high priority. And we define one step. It's part of the deployment interface. It has name inject files, which corresponds to a method we define. Priority zero means it won't run automatically. If you put here, for example, 70, it will run after writing the image, but before setting up the bootloader. It can request to boots, it can be abortable, and it receives one argument, which is required. So this argument goes here in the signature. After node and ports, which are standard arguments. I won't really explain what is going on in this particular deploy step. It injects files, but you can read my blog post for more details. But this is a structure that you create to write a deploy step. And then in its metadata, you define an entry point in the main namespace, ironic Python agent, hardware managers. And that's it. You're install these packages on your ironic Python agent RAM disk, or hardware managers are loaded automatically. Your deploy step will be recognized. If its priority is more than zero, it will run automatically. If its priority is zero, or if you want to change that, you can use deploy templates. I'll just just snippets from my blog post, because it's easier. So a deploy template is an ability to modify the deployment process that is associated with a trade name. So here, for example, I'm creating a trade name custom inject files. And I'm associating a deploy template with it, with the following deploy steps. As you see, it's again interface deploy, the same name. Now I'm setting priority of 50. So this step will now be run as priority is written and provide some arguments, doesn't matter which. Now I have a deploy template. If we add a trade to a node with the same name, we will say, ironic, that this node is capable of executing this deploy template during deployment. So when during deployment we request this trade, and I'm using Metalsmith because it's much shorter, Metalsmith writes an instance and form, like this, instance, sort, trades, list. Ironic checks that, yes, this node is actually capable of running, of using this trade. I'm using allocations for that, but we can do it manually. And the matching deploy template will be applied. So at priority 50, this deploy template will now be run. Out of band deploy templates are writing in a similar fashion. I probably can show you something. Excuse me, I haven't prepared this part in advance. But for example, let's take Redfish, let's take Management. I think it has at least as a clean step. Okay, it has a clean step, but deploy step looks exactly the same. This is to update firmware. Maybe it actually have deploy steps too. It doesn't, Demetri. It doesn't. Is it even secure boot? Oh, that's different. Sorry. Okay, secure boot actually has a deploy step, priority zero, no arguments, nothing else. You can write that. Or there is a clean step, which we marked above. For update images, it has arguments. So when you request that, you have to provide these arguments. And the clean steps work in a similar fashion to deploy steps. So it would be just here, deploy step instead of clean step. What do I want to tell you? Yeah, my blog post contains information on how to build an ironic passing agent image with your custom hardware manager. I want to repeat that here because it's a bit long. And that's it. If you have any questions, let me know. I have one. Sure. So you mentioned you could do out of band deploy steps early on. So can you do it then before the RAM disk boots? Yeah, the RAM disk is booted on the deploy.deploy steps. So RAM disk is booted here. If you have priority higher than 100, it will be executed before the RAM disk is booted. Okay. And is that a fairly recent change or is that in the case for a long time? That's the case for deploy steps, but not for clean steps. I think you're referring to clean steps, which have this problem that is not fixed yet. Okay. Deploy steps have been designed like that from the beginning. I think there were some issues running before. I had a bug in Storyboard, but Storyboard is now down. But the issue was that because Sarah starts booting, there's a bit of race. What happens first from this already running or the out of band step has completed or not? But like, yeah, I have to wait for Storyboard to come back. So maybe it's not like reliable in all cases. Okay. Yeah, we need to check. But the idea is this. Thank you. You're welcome. Anything else? Thank you very much, Dimitri. Are there more questions for Dimitri? I have one then. Dimitri, you showed that the whole framework seems like to have like maximum flexibility and what you can do before the RAM disk starts during the RAM disk, when the RAM disk runs. I was wondering if you also plan to have or if there's already something where you have something like very simple where people can just, or operators or users can just inject something like a script that's executed. Something like without like doing a script, sorry, without doing their own step. You see what I mean? So similar to like a script that you can append to cloud in it as user data, just something where you upload a script or like pass on the script that's then executed while the IPA is running. I see what you mean. No, we don't have this plan. Actually, the in-band steps of work was finished on really Victoria. So that's pretty much, pretty recent. No, I was just wondering, because I think that most things probably will be or could be done within some kind of script. And I have another question about this, but this would like kind of the threshold for users to use this, like make it very low, very easy to use if they just can say, okay, this is my script that I want to like to execute when the node is running or when the IPA is running, rather than like hooking something, like the whole step and hooking this into the deploy step framework. Yeah, I understand. On the other hand, we don't currently have stable internal API in Aronic Python Agent. We have Aronic Leap, which is kind of stable. And we have Hardware Manager Interface, which is also kind of stable. But I think before we can get to really easy fire and forget scripts, we have to define some stable API, maybe some DSL instead of Python to do that, similar to introspection rules. Yeah, it's an interesting idea, but I think it will require a lot more thinking. Okay. The other question I had is, like what do you think is like the main use case for deploy steps? Because like apparently the potential is infinite, you can do everything now. But what do you think operators or users of Aronic will use this for mostly? So the example I have here in the blog post, and I want to upstream it actually in Aronic, is inject small files in the final image. For example, I have this request from OpenShift site, where they have chicken and egg problem with network configuration, and they don't use cloud identity. There's ignition, which kind of relies on network already present. So they need a way to inject the small network scripts into the boot partition because that's how chorus works. People also request, for example, grab configuration, the default grab configuration, the kernel parameters to be customized. I mentioned a few potential use cases in the RFE that I could not post because as I mentioned, the storyboard is down. But from two pool sites, I remember requests to configure tune D parameters because this requires a reboot. So if you configure them in your first boot script, you have to do another reboot after that. Right, one of the things that... Sorry, while you're talking, I was thinking like how we would use this as well. One of the things that we, for instance, do is configure huge pages, which is also what we do by a kernel parameter. And this requires us to reboot the node once more after it has been fully configured. So using a deploy step, we could actually do this earlier, and the first boot would actually allocate the huge pages already, and this would save us one reboot. Right, to pull up people also have this problem. You have as huge pages as well. Can it also be used, Dimitri, to configure the bare metal itself? Things like biosettings and rig configuration without having to put it into maintenance mode and do cleaning or basically do it based on the actual selection of the system by, I don't know, Metalsmith or Nova? Oh, yes, it's also another use case, is to do deploy time rate or deploy time bias. I even think we run that in the CI now. I'm not 100% sure I can check for you, but I think the standalone job in Aronic actually runs software rate in deploy time. The same can be done as hardware rate, just probably different priority. I see, thank you. So yeah, it's definitely a use case we also consider. Are there more questions for Dimitri? One final comment, maybe what I really like about the deploy step framework is that's very much an analogy to the clean step framework. That makes it very easy to like grasp the whole idea and like, I mean, if you have written a clean step, it's very easy, I think, to add a deploy step because it follows the very same schema. That's very nice, this symmetry is very nice. Sometimes it's even the same bison function. Right. Okay, more questions? No? Okay, thank you very much Dimitri.