 Good afternoon everyone, thanks for joining us. So afternoon sessions are special, right? Because you've had your food, you're settled in, you've got this spot now, and a perfect lullaby session is all you need. So you know what? I'm going to raise up. Right, so a quick show of hands. People related to IT here, OK? Seriously? Right. So people who have got half their wardrobe from free stuff, that's surprisingly few. I'm not going to take it back. How about people that are working day after 12 o'clock? Right. Ladies and gentlemen, I give you the sysadmins in the room. Right, so let's get started. A little bit about me. I'm an operations engineer at Endurance. So my day-to-day job involves working around a bunch of VPS solutions, bare metal servers, and dedicated servers. Sound working? When you boot this system into Pixie Boot, you're going to get a screen which gives you two particular options. One would be local, which is the default option, and the other one would be sent to a seven. So the default option basically means it's going to boot using the local disk of the system, of the server itself. And in case you decide to go with the sent to a seven option, it's going to start installing the sent to a seven ISO. So Razer does this thing slightly differently in the sense that it uses iPixie. Now iPixie extends Pixie in a few ways. The most important of which, I would say, is the ability to use HTTP directly instead of having to stick around with TFTP. Right, so finally jumping into Razer here. Before we jump into the depth of Razer and figuring out how things proceed over there, we'll just take a quick look of basic Razer installation. So basically, you need a DHCP server. You'll need a TFTP server. You need to set up your DHCP config according to the Pixie configurations over there. You'll need to set up something called the microkernel into your TFTP serving directory. And finally, you'll need a bunch of Razer server and client packages. And it's going to use Postgres as the backend store. So most of these things are fairly simple and straightforward steps. And at the same time, you always have puppet modules available. So that's going to automate the entire thing for you. Right, so at an age where we have so many existing tools available for network booting, what was the point of writing another one? And why should one start to even think about using Razer? Like what is the point of Razer? So now the whole philosophy of Razer is the need for wanting to be able to consume physical resources like it's not real. So you want to consume your hardware servers, your bare metal servers, like it is not real. You want to consume it like it's a virtual machine. The way you have VMs in AWS and OpenStack, where you specify, you know what, give me a four core machine with four gigs of RAM, and I don't care what hypervisor it's going to be on or what kind of hardware configuration is underneath. Similarly, you don't really have to know what hardware specifications are, you know, what is the latest density RAM which you can plug into this particular system. As long as that system has the set configuration, give me a system which has 40 cores and something which has more than 30 gigs of RAM, and I'm happy with it. I don't really need to know whether it's node one or node two. So you have another aspect over here, which is the whole cattle versus pets philosophy, where in a lot of tools treat their servers as pets. You're supposed to configure them up with names beforehand, like the way you do it in Cobbler. You're supposed to specify, you know what, this is the system I'm going to be setting up for this particular use case, and you specify the MAC address of it, and you specify the profile, and only after that your system starts to boot. As opposed to Razer, all you have to do is, given that you have five systems, three of which have four gigs of RAM, you need to say that give me a system which has four gigs of RAM, and it's going to pick one of those systems and it's going to set it to boot. So you have a bunch of components present in Razer, so which are quite necessary to understand what they are, to figure out where things are proceeding further. To start up with, you have something called the nodes. Basically any hardware node registered under the Razer server for management is called the node. Next up you have is something called tags. A tag is essentially a unique name coupled with a matching rule. So let's say if you have a tag of web servers and your matching rule is something called, you should have number of cores greater than five. So any of the nodes which have more than five cores, they're going to be attached with that tag of web servers. Next up you have is repositories. So repositories is nothing but an object representation of your IOSO, what your installation disk is. Next up is policies. So policies is the glue in the middle. This is what matches the tags to the repositories, and this is what is going to install the actual operating system onto the server. Next you have brokers and hooks. So strictly speaking, brokers and hooks are not essential components to Razer. So what broker is essentially is, since Razer has a very clear line between the time the OS installation is done and the time when your config management needs to start off. So you have the mediatory thing called brokers. So broker is going to hand off once the OS has been installed to your puppet agents or your chef agent, where it's going to start setting up the server according to your config management. Hooks is basically letting you allow to run arbitrary scripts at different points of time to give you an example of how hooks could be put into use. So we have a lot of management present on our switch, on our switch sites. So typically you want your deployment to be done on practically private subnets over private VLANs, right? And finally, when the system is going to be put into production, it's going to go over public VLANs over public subnets. So you can have a hook fire into your June OS or your Nexus switches at the end of the OS installation. And it's going to change the VLANs present over there into the public VLANs and your public subnets, given that you have specified the IP address over there. And on top of that, it can apply whatever shapers and filters you want. Right, so I might have mentioned about microkernel quite a bit back. So what this is is, it's essentially a CentOS 7 kernel loaded with puppet factor. And the point of microkernel is any system which has been pixie booted and is brought into Razer's management is going to be loaded in with the microkernel. So what the microkernel does now is, it's the idling state of any node present in the Razer system. So the microkernel collects all of the facts from the hardware and it's going to dump it into the Razer DB. Now when the facts are present in the Razer DB, the tags can have an async call run over there and match whatever nodes according to the matching rule. Right, so if you're looking at the flow of Razer here, so any system which has been pixie booted is going to be loaded in with the microkernel. The microkernel is going to go into the system. It's going to collect the facts from the system. It's going to collect your details about your physical hardware of the sort of what sort of disks do you have? Do you have SSDs or do you have hard disk drives? Whether you've got what kind of processors you've got and the counter processors you've got, whether you've got hyperthreading enabled on them, stuff like details about your network card, your memory options, so on and so forth. And finally, once all of this data has been collected, there's going to be an async operation going on in the background, which is going to check every single tag and its rule, correspondingly with its nodes. And if there is a match over there, the nodes are going to be attached to the tag and there's a sequence of steps which is going to follow on. As soon as the tag is attached, if there's a policy attached to that tag, the policy is going to be attached to that particular node and once the policy is attached, the system is going to reboot now and this time it's going to be loaded in with the particular operating system which needs to be installed. And after that, you have a bunch of kickstart files and pre-seed files, if you may, which are going to be created on the fly. And finally, once the OS installation is done, the brokers would be given control of the node and at the end of the complete installation, there's going to be a flag which is going to be set, which is basically telling the Razor node that this system has been installed and you do not need to reinstall the operating system on this particular node in case the device has been set to pixie boot and reboot it again. So I've got a fake live demo of this process here. So basically this is how all of your nodes would show up in the Razor database over here. So I'm basically firing a bunch of Razor CLI commands to the Razor server. So you can see that there's a bunch of nodes already present in this system here with their MAC addresses and a couple of them have tags already present over there and corresponding policies as well. So this is a brief description about one particular node. So you can see that the install flag has been set to false. If you look deeper into that particular node, you can take a look at the facts. So this is the entire factor dump for that particular node. So you get details about your processors, whether it's a physical machine or not, your architecture, if there is an existing operating system installed on it, what kind of operating system is installed. Further on you get details about your partitions as well, whether if it's present in a rate configuration or not. So moving on next, we have the definition of a Razor tag. So all objects in Razor can be put forward as a JSON object and directly imported. So here you can see that the actual check is the number of processors should be greater than 20 and the label has been put as demo. So if you list all the tags present on this system, you can see that we have one demo which is the number of processor count is greater than 20. It has one policy attached to it, but it doesn't have any nodes attached to it at this point because there's no systems present in this Razor DB which has more than 20 processors. If you look at the definition of a policy here, you can see that the repo and the tasks have been assigned over here as well as the broker. So we've assigned a no broker at this point, right? And you can have very generic stuff present over here as your root password and the max count would be the maximum number of nodes you wanna be installed into this particular policy. So let's say you have a caching service policy and you don't wanna have more than five servers present in that at any point of time. So you can apply that a max count of five is there. So finally you have a very important thing over here, the tags array which is basically all the tags to which this particular policy needs to be attached to. Next up is a very basic definition of a repo which is basically the endpoint from which it's gonna be collecting the ISO from for the first time and eventually it's gonna be cached in the system. Another interesting thing about Razor is it also manages to store all the IPMI credentials for each and every node. The fact of storing IPMI credentials for all the nodes over here is that now Razor has control of IPMI tools and it provides a direct wrapper of using IPMI tools and this is beneficial because now you can directly extend Razor's API and call IPMI tool commands directly from them. So basically if you have any power state informations present in the system, so basically you can specify in Razor that you do not want this particular system to be powered up. You can specify that this particular system needs to be on a desired power state of off. And finally you can see that this is the install flag over here which says that it is false. In case it was said to true, this system would not be pixie booted at any point of time. Right, so finally I'll be getting into what was the use case present here and why we decided to go ahead with Razor. So first up what we needed to do was we needed to install a whole bunch of operating systems and which wasn't specific to one particular flavor including Windows. Next was the fact that we needed to provision systems based on rules where you can specify that you know what this is the configuration I want and give me a system of this kind. And I don't need to figure out what kind of servers do I have and then figure out in that list and then start spinning up servers from there. Next was a discovery and inventory management system. So typically in a DC operations environment you end up having two systems. You have a DC system, something like rack monkey or device 42 where your DC ops team is generally gonna be filling in data and let's say after a hardware upgrade it's the onus is on them to update it manually as opposed to Razor which inherently has a micro kernel present. So the next time the system is rebooted the micro kernel is gonna be loaded and any hardware changes that have happened over the time are gonna be reflected directly. We also had an API which Razor has an API which is completely restful and quite extensive which was also quite useful for our purpose. And finally the complexity and ease of maintenance part of it. So this particular project was a lot of operational heavy rather than having a lot of dev work on top of these boxes. So typically we didn't need any config management system so we just needed to install a bare bones operating system on them and then give it off. So here's a couple of alternatives we evaluated over here. So we started off with cobbler because we've been using cobbler for a while around here. So cobbler doesn't have a direct rule based engine by default and it doesn't even have a management system, a discovery system of sorts. So which is why cobbler was not picked for this particular use case. Next up was OpenStack Ironic. So this is a typical example of the costs not being justified for the returns around here because this was a completely new DC and having figured out the intricacies and problems dealing with OpenStack clusters, we decided not to go ahead with OpenStack Ironic. And finally there was Foreman. So Foreman practically does everything which Razor does and a lot of other stuff as well. So this was actually a split hair decision. The fact that Razor can just do this particular part and it's based completely around the rule based engine was something which was, which tip the scale in the term of Razor. But if I think about it, if we had a typical backend architecture where you have your web servers, you have your application servers and the whole lot over there, in that case, Foreman would probably be a good option as opposed to Razor. So in the end, a bunch of these tools do a lot of things, but they do these things specifically well. So we needed something which was quite generic and basically throw, we can throw anything at it and it should be able to install it. Yeah, I guess that's about it. We have questions. Hi. So when you're using Razor, did you work with, I mean, did you try provisioning a UFI machine? I don't think so, no. Okay, so that, so you can only do it in legacy BIOS mode? Yeah. I miss you, right? Not that I've tried it off, but I'm not sure at this point. I know it doesn't work, so I just wanted to know whether you had tried it. That's where Foreman would actually work better. Right.