 Okay, thank you. So my name is Chandler Wilkerson. I'm a senior software engineer here at Red Hat and My presentation is virtual machine infrastructure as code with Kubert and Argo CD So this presentation follows some work that I did on the upstream products first So I know we're on the downstream products first. We're probably supposed to do it the other way around but This is more of an experiment and Kind of Making sure that everything that we have in OpenShift works also when we're all Kubernetes, especially with the Kubert community project that we that my team does take part in And also with the Argo CD where we've done some integration work to help it work with Kubert and so I'm hoping that this can Have some value to everybody watching essentially the idea here is We can use virtual machines now within Kubernetes and that's what Kubert brings into the mix But then giving that a get ops flare with Argo CD controlling those virtual machines allows you to do interesting things with for instance multi-tier applications Suppose that you have a multi-tier application that runs Several virtual machines and you're looking at modernizing that containerizing pieces of it but it would be prohibitively difficult to Containerize the whole thing in one lockstep and try to get it all working with This integration work you can then bring along pieces of the you know different tiers as virtual machines into a Kubernetes cluster while Concentrating on modernizing one piece of it that you know gives you the most value first It also gives you some abilities to provide self-service virtual machines for developers in the DevOps model and There is a lot of potential going forward for using the orchestration and you know scalability of Kubernetes to drive virtual desktop infrastructures and Another interesting use case is to fit virtual machines as a component in a pipeline So if you spin up a virtual machine, it does its work and then spins down and have that it analogous to a container in a You know CICD kind of pipeline So I kind of wanted to start by telling a bit of a story about you know kind of my past as a systems administrator And you know like how I've seen the field kind of grow and and change so often the beginning We had this kind of concept where you'd have Systems administrators who would very much be territorial about the servers and services that they would run and They would kind of consider them kind of like pets or you know like a handcrafted kind of product So you would have configurations, you know, they would go in and modify the configuration It could be different on every machine, you know, there was a certain artsmanship, you know craftsmanship to it But of course it has some drawbacks If you made a mistake while crafting your configurations Sometimes you would have to come back from From take back up in order to restore the machine You know if you're lucky in some cases the Systems administrators if you're trying to look over another System in work you might find that they've locally managed to use change management and have RCS or that kind of Repository for for files on the server itself, but not necessarily centrally managed and That's where you get kind of personality conflicts with Things and of course if you wanted to avoid downtime you had to duplicate, you know, so still we were talking about physical servers The drawback of you know having all your services on physical services if you want high availability That means you buy another server and then set up a cluster around that So things got better we went to virtualization so that got us around having to buy multiple servers to arrange high availability and it got us around a Single servers hardware failure taking down an entire infrastructure piece We started using configuration management More and more piecemeal and so puppet and ansible allowed us to Homogenize servers and so you eventually got into this mode where You know the servers became more like cattle not pets I'm sure many have heard that analogy but essentially the way I think of it is the Place into which you apply your craftsmanship as a systems administrator or as a you know as a burgeoning DevOps administrator Would be more on the configuration management side and so you can put your craft there and then have the actual servers be Just units in the in the Orchestration and so then as you go on we get into the modern age where we have containers and Kubernetes and Everything is almost ephemeral you have this concept of eventually correct configuration with Kubernetes where you can define how you want it to be and Kubernetes will strive to make it so and That gives us a lot of advantage over where we were with even Managing virtual machines But of course there are still points where you want to manage virtual machines and So the two pieces that I want to talk about today are that get ops puzzle Piece and the Cooper puzzle piece that bring virtual machines into a kubernetes cluster So starting with Argo CD in the get ops piece we have this Kind of interesting idea where you have an application at the center of your your kind of plan for Services on a cluster and that application ties into a repository So you generally go to get a get repositories very often get hub And then you have this idea of a target configuration that you want to maintain onto the cluster from that get repository and Then you have your you know your target cluster where you're putting things and so that's these are the pieces that Argo CD sees and So then it goes in Synchronizes according to what's in the get repo and then puts it on to the cluster. So with Kuvert essentially everything is Operator-based structure and so you have an additional set of Custom resources you have an operator or several operators that handle API calls and Then there's a demon set that goes on to the nodes and handles virtual machines on the nodes And every virtual machine gets its own Bert launcher and so essentially the the basic idea there is for Kuvert you have a virtual machine running in a pod and then we have custom resources wrapped around those pods in order to allow them to Be managed in the Kubernetes way and the first of those actually is more around as a wrapper around Pbcs which are the data volumes and then you have the the virtual machine So I have an example of a virtual machine here and you can see it's pretty much a standard kind of Kubernetes YAML and the core of it is this template and with the the template you're essentially Designing what will eventually be a virtual machine instance So once you instantiate and set to running your virtual machine Then this template gets expressed and then that becomes a virtual machine and a lot of the fields in here Very closely match what you might see in the XML of a Libvert VM guest So you have the domain you have networks you have volumes going back to the the data volumes so what I'd like to show today and just a little bit of You know what I have set up with a previous Project that I did on OpenShift virtualization and OpenShift GitOps So I've kind of I've ported this into a Kubernetes cluster with Argo CD From upstream and then Kubevert from upstream my applications that I'm running with Argo CD is Working with an app of apps kind of model and so the the idea that You know if you have a long list of applications Why not make another application that will make sure that those applications are there? And so you have a tree and a kind of hierarchical kind of setup and so what I'll be showing is Just a little bit briefly to to demonstrate the The actual data volume side of things and then the virtual machines And so this is the repository that I'm working out of and So Argo CD just goes straight to that repository There's a set of directories on there that correspond to each of these applications and then I have a local One node Kubernetes cluster And so this is the Argo CD control panel and this is the last list of applications that are already on the cluster. I Figured I didn't want to tempt demo Troubles by Having it go completely from scratch. So I've already got this provisioned and everything is already running and healthy This is the app of apps So essentially you have your your one application that says I want to have all these other applications present and Any of those you can drill down So you can see on the main screen you can go into the app and see what it's actually running So these are all the pods and different resources created by Kuvert when it installs These are the data volumes that I'm experimenting with or demonstrating I have one for CentOS 8 stream and I have one for Fedora and I think that was 35 So pretty recent one and then I have two virtual machines Using the CentOS and the Fedora images And so as you create the the virtual machine Argo CD notices all the other associated Resources that get created and so it tracks them as well start out by Having the virtual machines seeing that we've got two running you can see where they are and You can get into the console of this one remembered the password that I've said so right now we're running a two gig Memory and thank you So we have one are actually two processors and so this is all defined in the get repository So I'm using customize to deploy everything because I want to make it work with either Kubernetes or OpenShift I haven't gone back and re-added the the OpenShift side of this, but this is the the upstream side of it and so we have a customization here that points at the base adds a Little bit of our back to ensure that we can pull images from a certain namespace Which is kind of the model that's behind a OpenShift virtualization We have a common namespace into which we load OS images and then when you clone Virtual machine you clone that image and then we have some extra patches to make sure that those images come from the right namespace so in the base we actually have the yaml for the VMs themselves and see our two core two gig of RAM server So just as an example of what you can do with GetOps driving your VMs You can go in and change this So we'll give it four and four Four cores, four gigs of RAM. Okay, then we go back to the Argo CD panel And we tell it to refresh its view of the repo And real quickly it'll notice that it's out of date, but because we have automatic sync on it'll just as quickly synchronize it Now here's where it's a little bit different from pods. So when you're dealing with a configuration manager or actually a GetOps kind of flow like Argo CD and You're using traditional container-based applications You can make a change to the repository and assumes it syncs It terminates the current pod brings or you know starts to bring up another one terminates the pod and make sure that Your configuration is correct with virtual machines. It doesn't quite take that initiative So as we can see Nothing really has changed here to reboot this within its own pod But because rebooting wouldn't necessarily change the the running configuration you would boot back up with the same resources however We'll see that we have a finished VMI and now we have a new VMI Up and running and give it a little bit of time to come back up now. We have four gigs of RAM and four processors So you see it's not quite as easy as DevOps, but with a little bit more You know more of a reasonable way of working with around virtual machines And suppose we had a problem with getting into the virtual machine in order to to change it If we're just using this in a pipeline You actually can go in here or even in into the virtual machine instance And you can use the Argo CD panel just to kill one and it likes to at least in the panel it likes to make sure that you've Oh It's not asking for the confirmation it asked for a confirmation You have to type in the name of the resource if it's a defined resource, but it this is a Subresource of a defined one so it didn't do that to me And you see that that shuts down BM just as if it were In a virtual machine infrastructure and you said shut down and again, it's brought it back up and running Okay, and there are other things that we can do We go back to applications there's one more piece that I kind of wanted to demonstrate on the data volume and I'm just I'm going to set this off and not really let it go because this may take longer than we have in the demo in order to Actually download because these are 1.4 or 1.3 gig images So if you see in the the data volume and kind of look at the manifest that's on the cluster right now We have this URL Dedicated and this one's a little bit old. It's from last June and I just checked today and we have a nice new CentOS stream image from this actually this January so I'm going to change to the data volumes directory Edit that yaml copy of it There's one from 2022 Wow, that's pretty recent Okay, okay back to our bocd we refresh and now it's trying to apply the the change and a little bit of an admission here I didn't test this before so I'm not quite sure what's going to happen Whether I need to delete the pvc first just like in the vm or if I need to Or if it'll handle it itself and look at all the dvs That seems to still be holding on that it has the old one So you can look at what's going on and see the diff of the manifest like sure enough There's a difference And so it's not taking that Which probably means We need to delete this And let it recreate it and now since this one is actually managed by good ops It wants me to type its name and now you can see The the icon may not be too clear to everybody but Now that it is completely missing from the system. There's a little ghost And so we say sink Oh, it's already sinking. So if that isn't working We can terminate this current sink assuming that it was set off with with the resource still present and then we can synchronize And it'll start loading And so as you see here is It's created the pvc already and it's assigned an importer pod to it And you can see in here just why This might take a long time So we can leave that running of course in the meantime the cloned version in the bm's shouldn't be affected at all So this bm should still be running but as soon as that pod finishes and as soon as that new Data volume is ready. We could actually delete this one and From here and then have it recreate on the the brand new image and we'd have a new virtual machine And I think I'm getting really close on time So I just wanted to conclude with some links I do want to mention the kuvert community is having a summit in three weeks And if you go to kuvert.io slash summit you can register for that. It's free. It's online only And This is a link to the blog post for the open shift virtualization open shift Get ops version of this particular work and thank you all