 Hello, I'm Wang Pengli from Tyson Cloud. I'm working on KVM for eight years. Today, I will introduce some synchronization problems in KVM. This is today's agenda. First one is boost fragmented WCPU in user mode. The second one is WCPU style issue. The last one is RCU reader permission problem. Synchronization operations in kernel is based on busy waiting. However, unnecessary CPU construction by busy waiting for the scheduled WCPU will significantly degrade performance. For example, TLB shutdown, remote TLB flash. That's a busy wait, which is fine in bare mental scenario. But within the guest, the WCPU might have been pre-empted or blocked. In this scenario, the initiator WCPU would end up busy waiting for a long amount of time. It also consumes CPU unnecessarily to wake up the target of the shutdown. There is a semantic gap here. OS makes liberal use of busy waiting, since they believe their WCPU are dedicated. There is another permission problem called log-holder permission. It arises when a log-holder running on WCPU is crimped by the high weather. Thus, forcing other log-waters to rise running on other WCPUs from the CNVM will perform useless spinning on the two-waste CPU cycles. Intel, since Westmere introduced false-loop exceeding, the processor compares the length of the two ports instructions interval against a pre-side parameter. It determines that the WCPU is spinning if the length doesn't exceed this parameter. Otherwise, the spinning stops. The spinning WCPU yields the processor to a WCPU that can make progress when the spinning interval exceeds a pre-side spinning threshold. The QVM searches the WCPU other than the WCPU that is just printed following the circle. Most SNP call function mainly calls are synchronous, mainly TLB flash and function call interrupts. Both the log-holder and IPI target WCPU are yield candidates now. Intel false-loop exceeding occurs when the spin-lock-watter is in kernel mode. However, IPI receiver can be in either kernel or user mode. IPI receiver candidate in user mode builds to be boosted. Workload like pbzip2 do the TLB shutdown in kernel mode, and most of the time they are running in user mode. In oversupply-rives scenario, it can lead to a large number of continuous PLE events. IPI sender calls false-loop exciting events repeatedly until the receiver is scheduled while the receiver is not candidate for boost. Let's boost the WCPU candidate in user mode, which is delivering interrupt. We can observe 10% performance improvement while running pbzip2 in 96 WCPU VM in another oversupply-rives environment. Next one is visibu-stat by wikafine. Wikafine is a feature inside the scheduler which we attempt to make processes running closely. It gains benefit mostly from catch-hit. One cumule or other visibu inject virtual interrupts to guest through waking up when sleeping visibu, it increases the probability to stack visibus or cumule by scheduler wikafine. A scheduler allows visibu to be scheduled on any physical CPU. This will cause the visibu stack problem that the logweaver is scheduled before the logholder on the same physical CPU. Let's take this figure as an example. The logholder on visibu 0 is limited when it enters the critical section at time 1. Unfortunately, the logweaver on visibu 1 is scheduled before visibu 0 on the same physical CPU, physical CPU 0, until time 2. Thus, visibu 1 has to wait time log wait to enter the critical section. Here the synchronization time spent by visibu 1 is extended by time task 1 plus time task 2. Let's see how often does scheduler stack visibus. We can run all visibu VMs on all physical CPU machines. Run the CPU bundle workload inside the VM. 100% utilization on each visibu. The visibus are easily to be stack in over the derived scenario. Let's disable wikafine visibu process to mitigate logholder permission. We test 3 80 visibu VMs running on 180 physical CPUs, and on the valid server, the ebz score can increase 17% after disabling wikafine for visibu process. RCU risk period in the guest cannot complete while visibu is permitted within a RCU reader side critical section. Thus, cores to synchronize RCU function cannot return. Although cores to call RCU function continue to return immediately, their callbacks cannot be invoked. In external code, can therefore continuously invoke for RCU function resulting in an unbounded quantity of memory that cannot be reused until the risk period completes when comparing between the RCU reader permission and logholder permission. While the usual symptom of logholder permission is to hung all parts of the system, RCU reader permission is that it blows memory footprints. Techniques to handle logholder permission says as into hardware, possible exiting feature can detect a visibu spinning on a log. However, these techniques cannot be applied directly to RCU because RCUs server build reset primitives do not make any state change detectable by provider or hardware. Thanks everybody.