 Okay, how are you live? Okay Okay Good evening everyone looks like a full session. Thank you everyone for coming My name is Boris Renske. I am the co-founder and chief marketing officer at Marantis and Today we'll be talking about How to stand up an HA open stack cloud with open puppet manifests in under 20 minutes for goat the Three creatures that made this presentation possible is myself I'll talk the first maybe 20 25 minutes then Roman who will perform a demo for the remaining maybe 15 minutes and The goat which will come into play later So to start I wanted to spend a couple of minutes and set the context and Talk about The open stack implementation project. So Marantis is an open stack solutions company. What we do is we help organizations of various size build open stack clouds and building an open stack cloud is a very Loaded term can mean a lot of things and involves a lot of things You can be Racking up the physical hardware connecting the network and all the way up to are actually hacking the Python code of open stack itself There's many things involved, but there is one recurring theme with every single open stack project we've ever done and that is building the Automation infrastructure around open stack My automation infrastructure I mean things such as infrastructure for bare metal provisioning to bootstrap the physical nodes and the configuration management that ties into it to actually deploy open stack on it and At Marantis we've done probably between 40 and 50 open stack projects at this point Every single one of them involved building some sort of automation infrastructure using various tooling for the clients The problem is as we're doing more and more of these projects is that Actually to be able to successfully build automation infrastructure around open stack You need to know a lot of things and in fact that it requires kind of a new breed of talent that has a In-depth understanding of a lot of our technology areas that you can't just find on the market So you need to know Linux naturally you need to be pretty good with networking Virtualization if you're going to be hacking open stack you need to know Python Ruby if you're going to be writing some puppet manifests puppet itself Cobbler if you're using chef you need to know chef If you're doing some orchestration then there's some orchestration tools such as m-collective assault and Etc. Etc. The list goes on the bottom line is that you have to be really smart to actually be able to know all these things and Smart like this guy right there Now talking a little bit more about Mirantis We have a lot of our engineering in the city called Saratov and Saratov is right there It's very close to the border of Kazakhstan and The problem with Saratov is that there is a few very brilliant engineers But there is not so many smart people overall, but the good news is that as you may know from some of the Borat movies in that area there's a lot of goats so as we tapped out the pool of brilliant engineers in Saratov our board started thinking of an alternative strategy and figured a way to basically use our geographic location to our advantage and That way is to actually build an automation library that is easy enough for a goat to use so we can use the resources where our offices are and Today as we're building clouds for big guys like Webex for instance. We have on the front end Smart engineers that speak brilliant English and the very articulate What people don't know is on the back end We have a bunch of goats doing the actual work So today I'm gonna talk a little bit more about how we made this possible So the initiative started about a year ago And our board told Roman who's gonna share more to basically go ahead and build that Automation infrastructure library and Roman said yes and they told me since I'm a marketing guy that I Need to be able to pack it And offer it to the end user in a way Where the end user is goat? So I said no problem as well So this brings us to chapter one Talking about What did Roman build specifically? so there's a lot of things here, but Basically what Roman did is he went to our engineering teams and Tried to figure out what is some of the hardest things that they have to work on in the context of an open-stack implementation project He further looked at the different use cases and deployment configurations that we've been building for our customers when we deliver open-stack projects and What he spent his time on is basically scavenging out the open-source ecosystem and putting together a Bundle of all components required to actually be able to Deliver an automation infrastructure around open-stack and the popular components there is Naturally, you have to have some reference architecture first puppet manifests to Deploy open-stack in various configurations Cobbler and host OS precedes open-stack packages themselves And naturally you also need a detailed configuration guide so that people Or rather goats can follow step-by-step and perform the work I wanted to spend a little bit more time on deployment configurations And I'll talk about why this is important in a little bit as well. So unlike any kind of a Open-stack deployment and management automation suite today Fuel has been and by the way fuel is the name of that package that Roman is built for goats with fuel we Try to Address various different customer use cases in the form of embedding a variety of deployment configurations of open-stack And I wanted to talk about some of the ones that are available today And we'll have more to come and I'll also mention why this is important as well so this is the list of the available deployment configurations that you can use that you can basically Deploy out of the box with fuel The first one is probably everybody knows a single node. This is pretty much the same thing Dev stack does There is very limited use for it in a production context, but if you just want to Play with open-stack first time ever and this is something that you would use the second one is a Quite popular one and it's multi-node. So this is the one where you have a single non-HA non-replicated cloud controller And you have separated out your worker nodes and the final third Deployment configuration is what we refer to as multi-node HA and this is suited for production environments Where you actually replicate a cloud controller to make it highly available Inside both multi-node and multi-node HA with fuel there is a couple of configuration settings where you can choose how you want to deploy Swift and also quantum Swift compact is for the use case when you're gonna be using Swift just as a back-end storage for your virtual machine images standalone will actually Deploy Swift as a kind of standalone component when you're gonna be using Swift as a external object storage Quantum compact and standalone so the most common way to deploy quantum now is to deploy it on a separate physical node Lately we've added another feature which is a quantum compact which enables you to actually run quantum on a controller node and Get the highly available quantum to run so why is this important and why I spent all this time talking about was deployment configurations because the Unfortunately, maybe but the fact of the matter is that all open-stack clouds are quite different today so these are some of the open-stack clouds that we worked on and we specifically produced our automation library in the form of a library to Give you the maximum available flexibility you can use any of the deployment configurations that have already been built in and you can pretty much do a single button deployment supporting any of the deployment configurations that are available with few a lot of the box or In many use cases you actually have your peculiar environment and you want to adopt open-stack to your particular context Fuel library will actually get you maybe 80% of the way there and 20% you'll actually be able to tweak the scripts So I wanted to spend a little bit time talking about H.A. and the way we've done H.A. today in fuel this is via Multi-node H.A. deployment configuration. So there's multiple ways to do H.A. We've played with different ways to do it and we finally chose this one so the things to pay attention to here is that so most of the Open-stack services are stateless So it's not a big problem to actually do the replication. You can just put them behind the load balancer A load balancer which shows H.A. proxy which runs on every node So you don't really have to solve much for these guys But there is a couple of stateful components in open-stack namely the database and RabbitMQ which you have to Do some magic to do the replication So RabbitMQ you can do active-active Simply for configuration, but there's multiple options for the database and for the database was specifically chose to use Galera the Upside is we can do active-active replication of a controller node with Galera and The downside is that when you actually doing H.A. deployment Because we're using Galera and Galera is a quorum system You will need the minimum of a three nodes dedicated to controller and one node as a worker node So the minimum environment you'll be working with is for physical nodes And finally the additional component that is not part of Open-stack that comes into the Open-stack our H.A. architecture is KIPA-LIFE-D which is a virtual IP management software because Open-stack services running on worker nodes can only talk to a single API endpoint They expect to see a single IP and if you have a replicated Controller nodes you have to actually some we have to have some means to expose a single IP So we use KIPA-LIFE-D, but I think that KIPA-LIFE-D is used in the majority of the H.A. architecture for Open-stack today So this is a little bit about deployment configurations and the H.A. architecture specifically that we've Embedded into the puppet manifests that come with fuel To sum it all up how how does it work? How would you deploy an Open-stack environment with that library that the Roman is built will show you a little bit later But there's three steps first is you create What we refer to as a master node or fuel master node It doesn't have to be very powerful, but that's basically the physical machine that will do the work to build your open-stack environment There's cobbler running on it and puppet master running on the fuel master The second is you actually enter your hardware info such as the MAC addresses and IP addresses for the servers We're going to be deploying Open-stack and the ones that you're going to be bootstrapping you run cobbler and cobbler will install the host OS Of your choosing we support Santas Red Hat and you're going to operating systems now And the third step is you actually run puppet and After you have chosen one of the deployment configurations that I talked about before and the puppet does the rest of the work and deploys All of the open-stack agents across the rest of nodes so Now I'm gonna talk a little bit about teaching a goat So the first step was to find a goat and I found the one that looked smart so this one looks smart and Then I showed the git repo to the goat and goat did not understand git repo because git repo is really for hackers Goats cannot work with git repo So my job came in and I built what we refer to as a fuel portal That is a very simple way to Consume such a library of various open-source components for non-hackers so to speak it has everything that You know a goat will need to be able to do such as a big red buttons to download the ISO to create the fuel master node and detailed Instructions on how to actually perform the installation step-by-step you can go previous and next and Step-by-step by step it will take you all the way to a functional highly available open-stack environment And there is an embedded support system if something doesn't work you can file a ticket and our back-end team will help So I explained all the stuff to the goat And in fact you can even see I'm showing that same fuel portal that I talked about and the goat listened But the goat did not understand Despite the hard explanation that I did and then the goat thought a little bit and came back and Said that script is too hard for goat Must have UI so that brings us to chapter 3. I Had to go back to Roman told him that all that works wasted And now he needs to build UI because there's no way that we can have goes deploying open stack of script so Roman at first dispaired, but Then he came up with the idea that he's actually gonna use everything that is in fuel library And simply build a thin GUI over it that will enable a goat to actually perform all the three functions that I described before simply by Clicking a series of buttons. So this is how fuel web came about and this is something that we are starting the private beta for today and it's available for demos in our booth and It's available for anybody interested For a download through the fuel portal on the rents website. So like I said what it is It's a it's a thin but intuitive web GUI that allows you to work with the fuel library and deploy all those various configurations of open stack the drawback is that you lose some of the flexibility Because it's not script. It is a UI. So you can't actually hack at the Pap manifests anymore but you do gain a very kind of intuitive experience and you can click next next next and Basically deploy and operate and add nodes and things like that of your open stack cloud So this is basically a repeat of everything that I just said But the bottom line is that you get the web GUI the fuel library here so all the fuel library artifacts feed directly into the GUI and With GUI you can click the button and you can manage your open stack clusters and With that I'm gonna pass the word to Roman and he's gonna show you a quick demo of what it actually looks like And then we'll talk a little bit more so the Floors yours. All right Is the mic on all right perfect, so So guys my name is Roman Alexenko I run product engineering at Mirantis and Today I'm gonna show you the demo of the fuel web. So which is the GUI on the top of our fuel library so I am not gonna show any slides. It's just gonna be a live demo on this exact computer And also what's cool about this demo is actually any of you can repeat it yourself Within maybe 25 to 30 minutes. So and I'll show you how we can do it So first thing that I want to tell you is that Fuel web. Oh All right. It's a good catch Perfect. All right first thing I Fuel web is distributed as an ISO so If you guys sign up for the private beta you get access to the portal The first thing you need to do is you need to download the ISO as simple as that so Then what you do is that you use this ISO to create a master node, right? So the standard approach for Many of the products out there. So you need the master node in order to be able to provision other nodes On which open stack will be running Also, we made it very easy for people to evaluate fuel web And we provide the scripts for virtual box that you can actually use and it says easy as a like Single-click install. You just need to launch one script in order to do fuel web installation on the virtual box so Here let's see I Already have The master node installed from the ISO in order not to like waste 9 to 10 minutes in this room And it was done using this script. So it's basically if you download our Package for the script. So you're gonna see something like this All you need to do you need to put the ISO that you download in the ISO directory and the scripts will pick it up automatically The scripts that we have here so the launch that I say it's the main script And it does just three basic things Number one it sets up the network. It's called prepare environment. Then it creates master node And installs it. So it's basically I can do it manually in the virtual box right here. Just Here create a VM mount the ISO set the hard disk parameters memory and all the stuff put it to the right network Install from the ISO instead of that we just have a script which does all that so So right now This and this is already completed. So master node is installed So the next thing that you want to do is actually you want to get to the installation of open stack itself And here we use The approach of Booting the notes or additional notes or pixie from the master node. So that's how we discovered them So in your data center that would be an equivalent of just going to the hardware like physically or remotely using the IPM I and telling it to boot over pixie. So and as long as it's in the same L2 broadcast domain, it will boot from the master node. You've just installed and here again I have a script which is going to create Slave nodes for me will point them to the master node and they'll just boot over the network. So think let's Let's do that. So you can see that It was before it was only master node and right now Two slave nodes have been created, right? So now they're booting over network from the master node. So now Let's actually open the web UI. So the most interesting part So that's how it looks when you initially install it. It's completely empty, right? So you can see the notification right there that the master node Installation is done. So here is your master node So these guys are the slave nodes are still booting. So those are the notes on which we will actually install open stack So it will take maybe A minute or two. So while they it's in progress, we will go and explore the UI a little bit So on the UI, it's basically one simple screen Which allows you to create an open stack environment? So our tool supports multiple environments right now. So for example, you can go and say Like I want to create production environment production cloud running Folsom, right? So If you want to do another one You can say, okay, I want to have like QA open stack environment running for example Folsom as well so these boxes they represent Open stack clusters, right? So completely different clouds So then what you can do you can actually go inside and you will see the multi-tab interface for setting up the open stack cluster So the first thing here is You have to select your deployment mode Deployment mode right now with support to it's either multi node Which means one controller and the number of compute nodes. So like no HA and The second one is Multi-node with HA It maps into the HA mode supported by the fuel library It means like three or more controllers and the number of compute nodes and in this case will spin up Like for the infrastructure will spin up Galera, which is the Like not my sequel, but it's like an active active solution for HA with multi master Synchronous replication and we'll do clustered rabid and all other things which are not present in the normal Multi-node version. So I think for this installation will go with multi node because we have only a couple of nodes so let's select that and by the way here You can notice that These two guys which were previously booting. They are now discovered in the tool. So they booted over pixie Master node grabbed all the information you got the notifications. So Here are your notes you can for example kick here. So you can see that It's a new physical node. It detected that my manufacturer right now is virtual box Which is obviously correct and then if you want you can go and look Into the memory disks CPU stuff like this like explore the all the attributes of the note All the nodes that I have right now are virtual and they're very simple, right? It's one nick One disk and 768 max of RAM. So really simple configuration Alright, so the next thing would be to actually add them to the cluster that we've just created, right? So you can see that there is a place to add the one controller node Because it's a non-achained stall and you can add a number of compute nodes. So let's do that We can say that for example this node is going to be our controller node And the remaining node the only remaining node from the pool of an allocated node is going to be our compute node so We're done with allocation. So it's as simple as that. So the roles are Defined right now. Now we can go to the next step. So the next step is network configuration. So right now we The fuel web supports the subset of features of fuel right now. There is no quantum. So it's not a network You can choose between flat the HCP and VLAN manager and what we actually have is a useful feature to validate The network connectivity because what we discovered during the last like we've done more than 40 deployments of open stack, right? So what we discovered is that? The very typical error in deployment is the misconfiguration of a network gear for example network switches, right? so if you messed up your VLANs it means that some of the nodes will not be able to talk to each other in a certain VLAN and Typically you would know that only after you install open stack that like some of the stuff just like won't work You will not understand why so what we added is a way to actually validate that upfront Before you start the installation. So You can click this button and the tool will actually send the special crafted packets between like pairs of nodes and we'll catch them on the other side and we'll see Whether there was connectivity in the all the required VLANs or not So it's like a very handy thing that we found very useful So right now it's all succeeded There are no errors In case of errors, you will actually see the table like node a node B VLAN Like cannot communicate, right? So All right next step Settings of open stack itself. So it's like admin username password Some items related to scheduling hypervisor. So we'll obviously use chemo for this one as KVM will not work work inside virtual box Then the next Tab is locks. So for example, you can pull Remote locks from from the nodes, right? So for example, it's wire wire log messages. You can just go and Take a look at all the stuff here And the last tab with some administrative actions You can like rename environment or drop the environment in case you drop it will delete Messerboot records from all the nodes and we'll just reboot them and rediscover So now given that all we have all this configuration in place. So the last Remaining step is to we have to push the changes you have to apply them. So you click this Button which says deploy changes. It asks you for a confirmation And then you can deploy what happens is If you go to these guys the slave nodes, they are being rebooted right now And it's gonna be like a multi-step process step number one is First of all the operating system is going to be installed. So right now it sent us So it's gonna be in parallel on those two guys and then according to their roles like controller and compute The puppet will kick in and will install and configure the right set of packages according to to their roles And essentially it's a push button installation. So now we just have to wait It all it's all gonna happen automatically at in the end. I think we have that in in our presentation What you are gonna see is actually A URL to horizon right so Open stack is gonna be installed on this note. You'll be given the URL to horizon. You can click it There's gonna be a sample VM image and you can launch it So it's just within 15 minutes. You can get the working Open stack installed. So right you had just like 15 minutes ago You had just the bare metal notes without operation system and without open stack installed on them in 15 minutes You have open stack Also While these guys are being installed so you can see some like nice progress bars here showing the Overall progress for the cluster installation and you can see the progress bars here So right now they represent the status of installation of centos when the puppet is gonna kick in will show meaningful progress for For the puppet manifests And you can actually well while this is in progress you can go and actually take a look at the log files So for example anaconda, right? So In the real time you can see That the packages are being installed on the notes. So this is being updated automatically like every every few seconds You can even see it right now scrolling a little bit You can also Click for example this progress bar It's gonna take you to exactly the same screen of like where you can monitor our installation installation progress So I think While this is in progress I would like to spend a couple of minutes to talk about like what is coming and what's on the roadmap Would that be okay? It's like 505 All right, so What's on our immediate roadmap, so I think that's that that's very important. So a couple of items First of all here you didn't actually see the screens related to Let's say management of disks and nicks, right? So when you are doing install on the real hardware It's gonna be a multi-nick hardware, right? It was probably mixed on 1g and 10g interfaces and you will you will want to have tight control of what? Networks goes where right? So you might want to say that hey this opens tech networks goes to this Physical nick and it's gonna be tagged. It's gonna be in this vlan and this is gonna be intact It's gonna be tied to this physical nick. So right now the tool assumes that it's There is a like one nick if there is more than one it will all it will always use the first one, right? So adding the Management of all this and type configuration That's a step number one. So the same thing applied to discs Right now we do not support cinder in the web UI. So our immediate goal is to add cinder support and Add the like the handy disk utility tool where you will be able to actually take a look at all your disks on the nodes And you will be able to allocate some space like this goes for the operation system This goes for swift and that part goes for cinder. So you will be able to actually manage your volume groups through the user interface So that's two items right nicks and discs the next one would be Integrated monitoring and performance dashboard in this tool So it's basically if you you go into the cluster and you can see the health of your cloud whether or not all the components are Operational and pass all the health checks and also performance money metrics for your open stack services And the fourth major one I would say it's upgrades, right? I mean Every single company needs this functionality to just take a wandering button and do the upgrade of open stack to the next version So that's also what we're planning to build here And I'm not even talking about like things like adding additional features like support for quantum deployment and Along along this line. So I would say that's That's the summary Can you switch back to my presentation? Sure? all right so to conclude we specifically prepped the slide because It's gonna take probably another what like 10 minutes to Actually perform installation, but it works you can we encourage you to go to a field portal on our website download it Try it give us feedback Once everything is done like Roman said, this is the screen you'll see and you will actually get the URL To log into a horizon for your environment So to conclude After Roman went through this entire explanation the goat finally got it and it said this So the key takeaway here is that fuel and fuel web is a each a puppet recipes for goats You can get it here for three play around with it give us feedback Contribute here it's available on git and Again, we'd love your feedback Tell us what's not working. Tell us, you know, where would you like us to take the roadmap? And last but not least do come to our party tonight at 6 30 Miranda's shuttle is taking straight to party Thank you very much