 It's my pleasure to deliver the presentation in KVM Forum this year. My presentation is about the QMQM upgrade test. I will introduce StabilizeAVI and the in-place upgrade to features since they are related to QMQM upgrade test. I'm Min Dong. I'm quality engineer from KVM QE team Red Hat. I'm responsible for StabilizeAVI. It's my pleasure to deliver the presentation in KVM Forum this year. My presentation is about the QMQM upgrade test. I will introduce StabilizeAVI and the in-place upgrade to features since they are related to QMQM upgrade test. I'm Min Dong. I'm quality engineer from KVM QE team Red Hat. I'm responsible for StabilizeAVI, in-place upgrade and some other features tests on both X86 and PPC. It's today's agenda. Let us talk about StabilizeAVI first and then in-place upgrade. What's StabilizeAVI? StabilizeAVI allows virtual machines to be presented with the same AVI across QMQM upgrades. From QE's perspective, regarding it has a subfeature of migration. When QMQM is upgraded to a new version, some aspects of this platform may change as new hardware capabilities are added. You need to make sure that the two machines are actually compatible with each other. If you migrate a virtual machine initially started on an older QMQM to a newer version of QMQM, you need to make sure that the virtual machine is running well in whole process. StabilizeAVI is that hardware profile of virtual machine should not change when QMQM is upgraded. Why we need it? This feature can help to avoid breaking down virtual machines and make sure every feature works well as expected after applying critical bug fix or security mitigation and inter-sector. Besides that, it can help to support new features and new capabilities of the existing features. How does motion type affect StabilizeAVI test? This slide shows you motion types on different architectures. First, motion type helps to emulate different chip sets and the related devices, such as PCN Q35, X86. Architecture, we will talk about it very soon. Second, it provides StabilizeAVI virtual hardware remains the same regardless of changes in host software or hardware. How to know which motion types are supported on QMQM? We have two ways. The first one, we are a single command. We easily know all supported motion types on some QMQM. Second, checking it from QMOS source code also helps. Let us have a look at the picture. We can see motion types on different architectures. PCN Q35, in this slide, let us take a close look at PCN Q35 motion types on X86 architecture. QMQVM supports two main variants of motion type for X86, PCN Q35. PC corresponds to Intel I440FX chip set released in 1996. Q35 corresponds to Intel A2 Q35 chip set released in 2007. The PCN motion type is considered a legacy and does not support many modern features into continuous to push the new chip set Q35, which supports PCIe, AHSI, and other modern features. Although at this time of writing, upstream QMQM has not reached an agreement to remove new variant of PCN motion type from long-term, probably stable Linux distribution are moving to support Q35 only. From QE side, we pay more attention to include Q35 motion type in a stable guest ABI test. As Q35 motion type is very important for virtual machines on X86, I'd like to give a short introduction about it. The Intel Q35 chip set consists of two primary components. The graphic memory controller hub and IO controller hub. The GMCH manages data flow between the CPU interface, the system memory interface, the external graphic interface, and the IO controllers through the direct meter interface. The ICH9 provides a multitude of IO related functions. There are new features are supported on Q35. The first PCIe, PCI X-Price, pro pro component interconnect X-Price. The second, but the ones to host controller interface, or AHSI, is a technical standard for an interface that enables software to communicate with serial other devices. The third, we are MMU emulation. The virtual machine we are MMU is a general device currently only in the Q35 platform suppose virtual machine we are MMU. The fourth, CQ boot. Spells and OVMF, they are included in our test metrics. So I want to introduce them here. Spells is an open source elegance in bells implementation and currently virtual machines use it as default PC bells. Q1 KVM testing will focus on whether it can bootstrap guest correctly, as well as handle as it is protected with orders passed from Q1 KVM command line. OVMF, UEFI for VMs on X86, is called OVMF open virtual machine from where it comes from EDK2, which is the UEFI reference implementation. The unified extension for mobile interface is a specification that defines a software interface between an operating system and platform firmware. Our test metrics of stable guest ABI on X86, PC and spells and Q35 and spells and Q35 and OVMF. Our player products and stable guest ABI test, we should know each version of Q1 KVM or real has been integrated into the upper layer product clearly before we start the test. We can benefit from this kind of information and make the stable guest ABI test more accurate. At least the red hat open stack, red hat open shift and the red hat virtualization with different nano versions. Each version has a cross-ponding real and Q1 KVM version. We put them into the test metrics with high priorities. We pay close attention to the version of Q1 KVM or real from upper layer products. Red hat open shift support for mixed applications running on virtual machine and containers previously known as the container native virtualization. Open shift is a feature of open shift to container platform and is delivered, integrated and managed by the open shift operating free park. Open shift virtualization is an add on to open shift the container platform that allows user to run and manage virtual machine workloads alongside container workloads. It supports Linux and the Windows guest application running on current and all the version of this operating system can be modernized now and deployed and managed as first class citizens using native Kubernetes tools built into the open shift platform. It adds new objects into the open shift the container platform cluster while Kubernetes custom resources to enable virtualization tasks. Now migration is one of those tasks. Basically the stable guest ABI test is aligned to our layer products and we are here. Generally speaking, we make the test strategy of the stable guest ABI from four aspects. First, product lines is about hosted metrics actually. It includes different real product lines as well as Kimokiram components. Second, we need to cover all supported features as well as new features and the new capabilities of the existing features. Third, as the test results is limited and it's not possible for us to test all variant of motion types or architectures. So we need to prioritize the test metrics. For example, the Q35 and the PC motion type on X86 we focus on Q35 much more. Finally, hardware on X86, we will cover stable guest ABI tests for both the internet and AMD host. Those hosts should be compatible with each other. On power, we need to cover stable guest ABI tests on different host too, such as from power eight to power nine, power nine to power nine and power eight to power eight. We have basic test principles for stable guest ABI as follows. Future migration post copy will be covered. Only interaction for motion types will be tested between two different hosts with different Kimokiram installed. And also need to consider spells or OVMFs priorities on different product lines. And it's a basic test workflow that demonstrates how to do a stable guest ABI test. The test workflow has been applied to all architectures. We have two different hosts and have Kimokiram and spells and OVMF installed on those hosts. We run ping pong migration with a motion type list supported one by one from south to destination hosts. The migration includes the migration and post copy. And those machines, virtual machines include both real guest and windows guest. At the same time, we will run heavy workloads on those guests during the test. In-place upgrade, what's the in-place upgrade? In-place upgrade is a way of upgrading a system to a new major release of Red Hat Enterprise Linux by replacing the existing operating system. The in-place upgrade tool is Libre Utility. Kimokiram related the test on X86 and the PPC-64LE and SV9-0X. In-place upgrade test will be performed on both VM and host. The reason why we need to do it on the host is because we need to make sure the Kimokiram component will be upgraded to the correct version and work normally later. Why we need this in-place upgrade feature? There are customers who don't like to do deployment again for their product environment for some reasons. Red Hat provides customers with a better way to solve it. I'd like to say that in-place upgrade is the final alternative before you have to reinstall your operating system. In-place upgrade is a recommended and supported way to migrate your system to next major version for real. After doing in-place upgrade, the existing old operating system is replaced by a new version. But users' configuration and user data will be reserved well. It's one of the advantages of this feature. We will talk more about these features and advantages in the next slide. Why we choose in-place upgrade? Compared with redeployment, in-place upgrade has the following advantages. First, you can preserve your old configuration on your machine. If you redeploy it, all configuration will be removed. Second, it can help you to bypass the subscription management. If you redeploy it, your machines have to be subscribed. Third, with the lift tool, it only takes less than an hour and then you can have your machine upgraded if everything is going well. But if you redeploy it, it takes much more time since you need to install the OS, install the applications and configure everything required on the machine. It's additional time and cost. Finally, it's not complicated for you to use it, so you don't need to be an expert. In this slide, let us talk about upgrade paths. For now, we have two upgrade paths. First, from real seven to real eight. Second, from real eight to real nine. The current strategies provide an in-place upgrade from the latest release of the South System. For example, real seven to nine to either of the latest to even number the manner releases of the next major version of real. If you are an early real manner release with the OS, you can update the system to the latest manner version using either the update or DNF update command and then perform the in-place upgrade to the next major release of real. I want to highlight this. It's not supported for you to perform an in-place upgrade directly from real seven to real nine. However, you can perform an in-place upgrade from real seven to real eight and then perform a second in-place upgrade to real nine. These slides show how to perform in-place upgrade. First of all, the real content should be ready on Red Hat content to deliver network or Red Hat satellite. And then you just did a system by Red Hat subscription management. Or without RHSM, but you need to prepare a repo file on your own if you choose this way, such as locally mirrored repos or mounted real installation ISO or HTTP hosted repos. And then using YAM and DNF update your system to the corresponding real version and then run repo, run pre-upgrade first and then run upgrade command. If everything goes perfectly, the new upgraded system will be available after less than an hour. About new system, it can be a virtual machine or a host as I mentioned in the upgrade path slide. From curious side, we need to do the post-upgrading test. It's another test story. This slide we will talk about in-place upgrade with or without RHSM. First of all, please allow me to introduce a bit of RHSM. Red Hat subscription management is a customer driven end-to-end solution that provides tools for subscription status and management and integrates with Red Hat system management tools. When customers purchase a subscription to a product, RHSM checks which system in customers inventory are registered to their subscription. Registered system are entitled to support services as there are irata patches and upgrades from the content delivery network. As a customer Red Hat, you have access to update and acknowledge for Red Hat products like Red Hat Enterprise Linux. The access to those products is provided by a subscription to purchase a subscription you procure at SKU, which ties the subscription to an entitled quantity and price. In-place upgrade with RHSM honestly is almost the same to customer scenario. We have three stages. The first, register your system for servers. Second, enable the real content. The third, install lib tool and then run it. Register your system by these commands. These commands help you set server information and the SKU should be attached to the user you're logging in with. So your system can fetch the real content from the server. As I said in the previous slide, which ties the subscription to an entitlement. We need to enable real content. According to your account information, you will have the opportunity to get a poor ID and then attach your system to that pool so you can enable the real content. After you enable the real content, so you can update your system to the latest supporting manner version and then perform the upgrade later. Install lib tool and run pre-upgrade first and then you can get a pre-upgrade report by which you can know if there's any issues on your system before you really perform in-place upgrade. Please fix all the issues on the system by following the instructions from the pre-upgrade report. Run the upgrade command and just need to wait for the lib tool to upgrade your system automatically less than the hour or even much earlier. Let us check the results. Finally, log into your system and you will get a new upgraded system that all configuration have been resolved. From QS perspective, it's not a single test because for us, we need to integrate multiple components and services together and then we can perform it. But on another level, as a customer of Red Hat, it's a very friendly solution for them. Reference QNN in time. Thanks for listening.