 Hello everyone, thanks for coming. Now let me introduce myself first. My name is Zhou Yibo and I work as a business. I'm a software engineer for folks on Kolo, KVM and the container. Now I'm going to talk about the aggressive PV API and parcel tamer technologies which have been applied in our private cloud environment. Now let me begin my topic. This is a agenda of the topic. Firstly, I will talk about the background, what problems are in our scenario. And then the solutions about these problems will be offered. And lastly, we'll talk about the future work for our solution, the background. Firstly, let's talk about the tamer axis. The tamer axis, which contributes to the overhead of virtualization significantly. Contrary to the tamer, the VM is emulated by VMVAM. Therefore, this army theory in tamers will cause the VM axis. In our product environment, most of the tamer axis are caused by the programming tamer as shown on the graph. There are many reprogramming tamer operations before tamer theory. Okay, let's say the problems to API axis. API axis are also a big portion of overhead in our scenario, since large VMs are readily used. The left table is the typical skills of VMs use the embeddance. The statics of the API VM axis is also present on graph down right. You can see the API VM axis of our scenario can cause up to 500 and 50 sound VM axis every few minutes. Also features of PVAPN, are implemented in upstream, but they don't resolve our issues very well. Okay, let's say solutions. I have talked about issues encountered in our scenario. Let's talk about how we correct these problems. To eliminate the overhead caused by the VM axis, tamer axis, exit next tamer need to be developed. There are existing solution developed by Tensing and Alibaba Cloud, but they don't meet our demands. Exit the next tamer proposed by me as the Tensing Cloud requires housekeeping CPUs and inject is paired tamer interrupt through post interrupt. Another design from John Alibaba Cloud introduce the modifications on guest cloud and the reserve of our dedicated CPU for handling tamer interrupts. Okay, let's see our solutions to overcome the problems in existing solutions. We proposed a new exit next tamer. We called it a pass through tamer. In pass through tamer, the VM can use a physical API tamer directly. So the host tamer will be uploaded to the pre-empting tamer when VM entry. When external interrupt exit happened, if the external interrupt vector is a local tamer vector, we should inject the tamer interrupt for VM. In order to VM use physical like a tamer, the napkin tamer of the VM should work in TSC data data line mode. And then in VMVM, the intercept of TSC data MSR should be disabled. Lastly, we must adjust the host TSC value when VM entry. For VM use the physical TSC successfully because the TSC value of the VM is less than host TSC value of this. You know, in my pass through tamer environment, the VM uses a physical tamer directly. So if we wanted to make a host tamer work normally, we must do some things. In VM entry, we must get the latest tamer, the host, which will be expired and upload it to pre-empting tamer. When our VM exists on the VCPO pre-block, we restore the host tamer to physical tamer again. And then the VM tamer to software tamer, which is emulated by VMVM. Our pre-empting tamer is paired, which indicates that the host clock even to will be called. This slider shows the difference between normal VM, network tamer, and the VM password tamer. You can say in our design, VM can program the network tamer without trigger VM exit any more. Okay, in this slider, the performance test result is presented. You can say throughput of sets and guess operations of member catch increased 35.5% after adopting our password tamer. Another test, technique test, which indicates the schedule latency also shows the improvement after use of our password tamer. Okay, let's talk about the IPN VM access. A possible implementation of exit Nest IPN has been developed by NIE from Tencent Cloud. This implementation marks all these CPUs in a bitmap, and then send the IPNs to all these CPUs together by one HEPCore, and then the VM scan the bitmap and get the dist CPUs from the bitmap, and then send the IPN to the CPU in bitmap one by one. In other words, this implementation multi-multi-IPN VMX into one VM exit. Okay, in our solutions, we adopt a low exit IPV IPN. We managed this low exit IPV IPN by pass through PID square to guests and don't do not intercept SCR, MSR. In VM startup, we offer dedicated PV SCR for guests to send the special interrupt, such as SMI and MI. And then in guest, it can send the IPN directly via post interrupt and without VM exit. Now, this diagram shows how low exit PV IPN works. For example, in guest, the VSPU 0 send the IPN to VSPU 1. Firstly, we must get the PID script of VSPU 1, and then we set PV IPN vector in post interrupt request the bitmap of PID script of VSPU 1. Then we set outstanding notification in PID script. The outstanding notification will tear the hardware. There is a pending RQ to resolve, to solve it. And then we get notification vector from the PID script. And the notification vector is always post interrupt vector. But if the VSPU in host data, the notification vector is a post interrupt wake up vector. And then we prepare SCR. And then lastly, we write SCR. So the API is triggered successfully and without VM exit. Okay, let's see the test interrupt. Okay, okay, sorry, sorry. Let's see the test result of the low exit PV IPN. After adopting low exit PV API, the cost of a single API operation in VM decrease from 11,486 to 412 cycles. Single PV API can perform on the level of bare metal from the perspective of the application level low exit PV API can also improve the remote significantly. In our scenario, lamp sets and gets operations per second of memory catcher raised 14.8% after intro to see low exit PV API. Okay, let's see future work. Also, there are many benefits by introducing password tamer and low exit PV API. But there are some potential problems need to be addressed. In terms of low exit PV API, the guest should be trustable because the guest can assess the ICR MSR and the PID script. This will allow guests send the PV, this will allow guests to send the IPS to an arbitrary physical CPU. In our private closed scenario, they can accept a trade-off between performance and the security because the guest is generally considered to be credible but the work of making low exit PV API secure still needs to be done when security is a concert. The security hard of PV API could be implemented. For example, their EPTP switch feature of when think. For password tamer, guests couldn't affect any other VMs. So the security of password is not the major concern. In future, works of supporting new migration and dynamic switch may be valuable. Okay, this is all contents of my topic. If you have any questions, you can click me via the email. Okay, thanks.