 Hello everyone, welcome to Karen Forum, I'm Yongji Xie, a software engineer for the background. Today, I will give you a talk called video, VDP device in user space. Here is my agenda for this talk. I will start with some background information on VDP and video. Then, I will give you some details on video design and implementation. At last, I will show the status of this wall in our official plan. Okay, firstly, let me give you some background information. Here is a high-level overview of the VDP kernel framework, which has been merged into kernel after last year. VDP is short for watch out data path acceleration. It represents a kind of UI whose data is complied with watch out specification, but whose control path is when they are specific. To hide the complexity of the VDP devices and provide a unified interface for both kernel and user space driver, VDP kernel framework is introduced. It makes use of a software VDP path to abstract the common attributes and operation of VDP devices. Then, a VDP device driver can create and register a VDP device abstraction to the VDP path. And the VDP path driver can use the unified configuration operation to control those different types of VDP devices. Now, two types of VDP path driver are supported, including watch out VDP path driver and the rehearsal VDP path driver. For watch out VDP path driver, the VDP framework will present a watch out device. This allows the kernel watch out driver to control the VDP device as if they are standard watch out devices. Then various kernel subsystems could be connected to the watch out device for user space application to consume. For rehearsal VDP path driver, VDP framework will present a rehearsal chart device. This allows the user space of the watch out driver to control VDP device as if they are standard watch out devices. Various user space applications could be connected to watch out devices in this way. A typical use case is to connect a VDP device to QMU and use it as a watch out back end for the watch out driver surrounding the VM. Then, what is VDIUT? VDIUT is a framework to implement user space VDP devices. It allows user space team to emulate a VDP device which can be attached to a VDP subsystem. Through VDIUT, we can develop a unified user space approach for both VM and container workloads. For example, we can make use of VDIUT to emulate a VDP block device connecting to VDP storage such as safe. Then, both Cata container and Ground-Sea container can access or can use this emulated VDP block device to access distributed storage. Of course, we can also emulate a VDP device in kernel space, but a user space VDP device could be more attractive. Firstly, the user space development lifecycle is much faster and compared with kernel solution, user space solution is easier to maintain. For example, it is easier to do live upgrade for a user space team than a kernel module. What's more, user space development is more flexible, it has less limitations and can make use of also user space library. Even it can use some device emulation codes in QMU and Rust VMs. Okay, next, I will introduce VDIUT design and implementation. This is an overview of VDIUT architecture, including a VDIUT user space demon and a VDIUT kernel driver. As we can see, VDIUT driver on the one hand acts as a VDP device driver that will create a VDP address attraction and attach it to the VDP bus. Then, the container can accept the VDP device via the VDIUT bus driver, and the VM can accept it via the VDIUT bus driver. On the other hand, VDIUT driver explores char device interface to communicate with user space demon. User space demon will use it to implement device emulation and watch out data plan. The device emulation will be used in VDIUT device control path and watch out data plan is used in the data path. This slide shows the implementation of VDIUT device control path. The control path is mostly handed in the kernel. That means the user will not go through the VDIUT demon but return by VDIUT driver directly. That means when the VDIUT bus driver tries to access the device edge views of VDIUT devices. Those device edge views that are configuration space will be firstly initialized by VDIUT demon via the R control on the char device interface. And the VDIUT demon can also query and update those device edge views via the R control. The reason why we don't hand it in user space is that it is not so easy to prevent VDIUT attack from an encrypted user space demon on some control path. So we try to reduce the attack surface as much as possible. However, since the data path is handed in user space demon, there are also some data path related control message such as stating device data that needed to be processed by VDIUT demon directly. So machine negligence is introduced for what are those control methods to user space. User space demon can use read and write this call on the char device interface to receive and reply them. And in the data path, the call is how to access the data of DMBuffer in our user space demon. Which can be implemented in different ways depending on the type of VDIUT bus driver. In watch out VDIUT case, we make use of MMU based software LQB with fountain buffer magnetism to achieve that. That means the data in the DMBuffer or watch out driver will be copied into a bouncing buffer or review driver and back depending on the direction of the DMB transfer. The user's demon only need to use the MMAP with this call to map the bouncing buffer into its own address space to access the DMB data. This is used to prevent user space accessing the original DMB buffer directly, which may contain some other candidate. But in coherent DMB mapping, the DMB buffer is allocated by review driver directly. So it can be safely mapped into review demon directly. And that is the case for the watch out queue allocation. And in watch out VDIUT bus driver, the DMB buffer is reflected in user space memory region, which can be shared to the review the demon while transferring the fact buffer of the shared memory. This works like what we did for VDIUT. What's more, to implement a data path, review the demon also needs some ways to receive the watch out queue kick. This can be done while having the mechanism. And the view also needs to inject interrupt to the driver sometimes. This is shared by the controller. Next, let me give you some details on how to implement a review demon. At first, we need to create a review device. This can be done while the review create a DVL control on the control device interface. With this control, user space can specify some basic configuration such as device name, watch out feature, the number of watch out queue, watch out configuration space, and so on for disseminating the device. The device name is used to identify a review device uniquely. After the L control, a char device interface will be exported to user space for device imagination. Then we need to use the review VQ setup L control from the char device interface to add per watch queue configuration to the device. Now we only support one watch queue attribute, that is max queue size. After the initialization, we can begin processing review control message from the char device interface. The first message will arrive while attaching the review device to VDP bus via the linker message. The next linker message could be sent while the VDP management queue in IPv2 package. And now, there are three types of messages introduced by the review framework. Namely, we use get a VQ state, we use update LTRB, and we use set status. The review get a VQ state message is only used by review host VDP case to fetch the data of watch queue. The data of watch queue is different depending on the watch queue type. For example, first please watch queue is not the index of the available ring. And the review update LTRB message is used to notify user space to update the memory mapping for specified IOVA range. After receiving this message, user space should firstly remove all the mapping for specified IOVA range, then set up the new mapping IO control. The IO control will be used to map IOVA region to user space. We will introduce the IO control later. The last type of messaging we use set status, which is used to set the device status. User space should follow the watch IO space to process this message. For example, when the message asker use space to set the future's OK feature between the device status bit, the user space should fail the message if they cannot accept the associated feature. After the driver OK status is set by the message, the user space should start the data plan process. To start the data plan processing, we firstly need to use the video VQ Get Info IO control to get some information of watch queue, including the watch queue size, the IOVS or described table, available ring and user ring, the watch queue status and the review status. If the review status show the watch queue is ready for processing, we can pass the above IOVS to the review the IOTB Get FDI control to map the IOVA region to user space. This IO control will find the first IOVA region that overlaps with the specified IOVA range and return the corresponding file descriptor. The file descriptor can be passed to MLAP discord to map the IOVA region into user space. Besides the IOVA mapping, we can optionally set out the Kick Info FD for the specified watch queue with the video VQ setup Kick FD IO control. The Kick Info FD will be used by video kernel module to modify user space to consume the available ring. Of course, the user space can choose to pause the available ring without the notification state. And the drawing consumes the available ring. The buffer described by the descriptor should be also mapped into user space while the review IOTB Get FDI control before processing. At the end, once we complete a request and fill the user ring, we can inject an interrupt for specific watch queue while the review VQ injects IOQ IO control. Then, with those steps mentioned above, I think we can easily omit the OVDK device in user space. At last, let me show the status of this work and our future plan. This work started with October last year. Now, the version 11 of the patch state has been posted to upstream. Here I really suggest a step from Michael and other guides for their review and valuable suggestions. A supported color tree and a specific demo example are also over-sourced in GitHub. Someone who is interested in this work can use it to do some tests and further development. And for the future, there are lots of work in our man. Firstly, open-source user space library is needed. This can make use and reuse user space development more standardized and faster. Secondly, support for more device types is also appreciated in the future since the current version only support block device types. Lastly, improving performance is also important. For example, supporting entrapped banding and find some ways to get a radio on the bouncing buffer to enable zero-copy. That's all for today's talk. If you are interested in this work, welcome to join us. Thank you.