 Good afternoon, my name is Marek Libra and I will tell you how a virtual machine can be taken from VMWare to further live in Kubernetes. As you probably know, Kubernetes operates containers and for quite a long time already even virtual machines. And we have recently had the possibility to take over a legacy virtual machine from VMWare as well. Why me and this topic? I'm co-author of this new feature and I have been developing software for almost two decades now. Last four or five years I am with Red Hat and during that period of time I have been contributing multiple projects, mostly around virtualization or containerization. The virtualization projects were somehow related to much bigger project umbrella built by the Overt project. The containers are related for me with Cubird, which I will talk about in a few seconds, with Kubernetes, OKD and OpenShift and this kind of stuff. I am former contributor of the Cockpit project, which is focused on building Linux server management application with web user interface, very nice one. If you haven't tried it, please do. It's pretty easy to install it. There are packages for most of the major distributions. Most of my contributions while building Red Hat can be tracked down on the GitHub and I have a link in the profile as well if you are interested. So, very briefly, Kubernetes or OKD. What's the big deal here? Kubernetes became industry standard, how to orchestrate and distributively execute containers with Kubernetes. With Kubernetes you can build an infrastructure, a cluster capable of running containers. Since it's open sourcing by Google a couple of years ago, Kubernetes gained huge community on both sides, on the user base, same as on the developer side. Plenty of companies are contributing to either to vanilla Kubernetes or generally to the ecosystem created around Kubernetes. Kubernetes scales quite nicely. This horizontal scalability is achieved mostly by implementing a few basic principles, which include the declarativeness and the use of controllers there. If you communicate with Kubernetes, you declare the desired final state you would like to achieve. And there is something on the Kubernetes side, on the infrastructure side, which is taking care of it to make this desired state to happen. This something are pieces of software, processes, usually containers, which are managed by the cluster itself. And they are taking, they are usually single focused and they are taking pieces of the actions needed towards the desired state. As an example, for example, scheduler, one of the main controllers there, it observes changes in the database of Kubernetes, which is being proxied by an REST API service for changes on list of pods. A pod is a grouping of one or multiple containers, each of them sharing, which are sharing the same network namespace. And this is the unit for scheduling and placement on loads in the cluster. And if it detects there is a new pod being created by user or any kind of automation, which needs to be placed or executed somewhere. It is being actively notified from the API about this change and gets the list of nodes, means the pieces of hardware in the cluster with certain configuration. And based on its inner logic decides which node the containers should be running on. And writes back this kind of information and that's it. There are other controllers, for example, one detecting that these containers will require a storage for their execution. So based on the other parts of the configuration in the cluster in the Kubernetes, they allocates a piece of storage somewhere. And they provision that and writes back this kind of information and the progress of the provisioning back to persistent volume resource of this kind. Once this is done, there can be other controllers like for networking doing the very similar stuff. And once everything is ready, another controller is notified about the changes and it detects there's everything ready. So the containers can be started. And that's all the things to start the process of the container with all the infrastructure already provided by the other controllers. And regularly writes back the progress of the execution and the actual status. And there can be other controllers observing the similar subset of data in the database in the API server and reacting up on that further. If we are talking about Kubernetes, we can't miss OKD. With Kubernetes, you can build Kubernetes provides you infrastructure as a service. OKD shifts this paradigm one level up, provides pre-configuration and certain sets of services well integrated all together and goes towards the platform as a service. So theoretically speaking, whatever you can execute in Kubernetes, you can you can execute in OKD as well. I have mentioned that Kubernetes is focused on containers on execution of containers. There's a project called Kubert, which is an extension to Kubernetes and add on bringing the capability of running virtual machines. There brings virtualization to the Kubernetes. Why you should care about virtual machines in the containerized world of Kubernetes for plenty of reasons. For example, with virtual machines, you can achieve much better security because of the extra layer of the hypervisor there. Two decrease costs. If you have certain load which needs to be executed in virtual machines, it's nice to share the same cluster for both of kinds of loads because of sharing the same operations means people and the infrastructure, the hardware itself. Having virtual machines in Kubernetes provides you the option to decompose the existing legacy applications gradually and sees where it makes sense or continue to absolute the composition to containers. With virtualization, you have full freedom about selection of operating system, which is the most suitable for your applications and their deep configuration. On a Linux host, you can easily run Windows machine, which is hard to achieve, maybe even impossible with just the containers. Having both kinds of execution environments all together gives you much better options how you can orchestrate the services provided by the applications running somewhere, no matter of the kind of execution environment you have. You can be concentrated on just the services provided by them. How is the Kubert implemented? Just very briefly, there are other talks focused on that. Kubert is based on a common mechanism for extending Kubernetes, which is called custom resources or custom resource definitions. Kubert provides own small set of new custom resource definitions, so extending the inner database of Kubernetes for new types of objects which can be stored there and so managed or accessed by the API server. Having that, you can build your own set of controllers taking actions upon this new kind of objects. One of the most important CRDs that might be difficult, sorry, probably not at this moment. One of the most important custom resource created or introduced by Kubert is the virtual machine itself. With this custom resource, you can declare declaratively state how the virtual machine should look like. To give the information about the storage, what should be there, declaratively state networking, size of the virtual machine and similar stuff. There are controllers or containers created by the Kubert itself. One of them observes the database for new virtual machines and if there is any, which would be executed in the cluster, it transforms the declaration of the virtual machine to a new port, which is further taken and executed by common mechanisms of the Kubernetes somewhere. This spot contains multiple containers. One of them is the containerized LibVirt process. One per each virtual machine running in the cluster and the LibVirt process takes care of creating the virtual machine itself. Of the QEMU process, which has full access to underlying KVM on the host, which is it running on. There's a container, a virtual launcher, which imperatively is managing the LibVirt. Now start the VM, now stop it and similar stuff. There are other controllers involved. One of them observes the database for virtual machines and provides the translation between the Kubernetes-like language or description of the virtual machine based on the virtual machine resource kind to the language of LibVirt, which means Domain XML. Or provides proxy for consoles, in this case, VNC and serial console. By the way, there's support for RDP as well, but this is implemented somehow differently, but you can use RDP for Windows Guest as well. That's it, very briefly. Kubernetes and umbrella for other projects as well, which bring the benefits for even outside the virtualization itself. And like CDI, bringing yet another abstraction for storage and extra functionality for that. So let's consider we have a Kubernetes cluster capable of running virtual machines. How can you populate with your load with your virtual machines? You can start from scratch using vi editor and provide the YAML files or JSON files describing, declaring how the virtual machine should look like. You can reuse documentation, well-documented, you can base your work on examples or templates. This is the hard way, but fully supported and fully working. Or you can use RAPUZ interface, which is being developed under the OKD project umbrella and following the wizard there. A set of web pages, click, click, next, next, provide just the useful details like the names, something about the size, how many CPUs, how many memory will be used, what is the boot image and where it will be taken from, how many networks will the virtual machine contain storage, how many disks and where will be there placed on something about cloud in it. And the user interface will create for you the small set of related resources describing your desire. Or you can, you can take an existing, existing virtual machine living in YAML and pull it to Kubernetes. How can you do it? I'll try a small video. I was not so brave to do it, the video demo live. This is the example of the web user interface in this, now you can see one of the wizards there for the importing of the virtual machine. The user selects the provider, in our case it's VMware, provides the credentials, means username, password, the triplet username, password and the URL where to connect to the URL for vCenter. There's something in the backend, I will talk about it later, which is establishing the connection to the vCenter, downloading the list of virtual machines, the user selects one, another connection is established in the backend to download the details, metadata of the virtual machine, which are being prefilled and transformed to various fields here and to the needed required values for the Kubernetes. The user can tweak all the details based on the actual use case, provide the mapping for networks and storage based on the particular configuration of the target cluster, or provide the fixing of the names because the naming conventions used in vCenter are different from those used in Kubernetes. Click click, next next, review the details and you will end up with a bunch of resources being created for you all interconnected somehow. One of them is the virtual machine itself, others are for storage, persistent volume claims, persistent volumes and one of them is a so called conversion pot, which is a one time executed action performing the actual conversion. It means pulling the data from the vCenter and placing them after transformation to the persistent volume, so native storage for Kubernetes. The user can review all the details and watch the progress of the migration. Once this is done, you will end up with virtual machine in the Kubernetes capable of execution. That's it. If you are interested in more details, the process of the conversion can be split to two parts. One of them is the preparation. This is what you have seen in the user interface and the actual conversion pulling, transforming and placing the data in Kubernetes. The goal of the preparation phase is to create all the resources needed for the execution of the virtual machine and starting the actual conversion. For that purpose, there is a new custom resource created and a corresponding controller to establish the connection to the vCenter and download the data and perform the communication with the vCenter. The user interface provides the mapping of the metadata to whatever is needed. The user can fully tweak the process and what are the inputs and the user interface creates for you the resources, including the next conversion pot, which purpose is to pull the data, transform and convert the format of VMware to something what is consumable by QEMO process and place it on the target. The conversion is based on external projects v2v and libgstfs and is basically speaking three steps process. First of all, the source virtual machine disks needs to be inspected. What kind of operating system is there? What kind of drivers are there? Is vTO present? Then perform the modification. The source disks are never touched, but there's an overlay disk being created backed by the remote disk somewhere. The modifications include potentially uninstalling VMware tools because they will not be needed anymore. If needed, installing the vTO drivers, especially on Windows or checking the kernels which are potentially there for the versions of drivers and selecting the best combination, especially from the perspective of the vTO. Then there is the actual execution which can take tens of minutes, maybe hours depending on the size of the source disks and the speed of the network between the data are being placed on the persistent volumes. So, abstraction native abstraction by Kubernetes and we have tested it for both fast system and block storage or block volumes. What we would like to see next are definitely additional providers, not just the VMware and we would like to improve the bulk import which is recently possible. But just on the command line by creating the corresponding resources I was talking about from scratch, which is not very handy and we would like to provide adequate tooling to do it on command line to help you with this process. And we would like definitely to see some support in the user interface as well. Patch is welcome. If you are interested in more details in Kubernetes, in general, this is the home page of the project. The user interface I was showing you the video from lifts or lifts under the okd project umbrella, which homepage is on github slash open shift. It has been renamed, but not on github and console. This is a sub project of the okd and the scuba we have a mailing list and I will be more than happy to respond to direct emails as well. If you have any. So thank you. Any questions, please. The question is whether we have observed any kind of issues with orphaned VM of the same kind, which can be spawned somewhere else. This is something what is being handled by the Kubernetes itself and it will be considered as a bug to allow that. Pardon me. Yeah, definitely. It's based, the actual conversion is heavily based on the V2V project, which is being developed by LibVirt guys. No, thank you for whether live migration is possible, not at this moment, maybe in the future. Recently, the solution is based on the source VM needs to be shut down, then migrate it and then start it. Replace what? Open stack. Whether Kubbert can replace open stack. Hard question. I think the use cases are slightly different. They are not one to one. So I think there's still the reason to have open stack. But as from my perspective, much more lighter solution is Kubbert. And you have the containers all together inside one infrastructure. Creating, that was about, this presentation was about migrating the VMs from VMware here and what about creation of the VMs. This is possible from scratch, provide a YAML file, maybe 20 lines of code, maybe 100. Or you can follow the wizard which looks pretty similar, like for the import, click, click, next, next and it's there. Whether we support shared risks which are supported by VMware as well. To be honest, I'm not aware of this feature at this moment. Maybe it's there somewhere, not handled directly by the migration utility. You never touch the source, so that might be the point. And the migration itself does not handle that explicitly. Okay, we are out of time. Thank you. Thank you.