 Good morning. I am Sharon Santana. Today I'm here to present our work on building extremely fast and efficient NFVs with Unicraft. Before we start on this topic, let me introduce myself. I work as a software specialist at NEC Laboratories Europe and I'm part of a team of eight researchers who are working as a part of the systems group in our lab. One of the projects within our group is Unicraft and this work contributes towards the port of DPDK on top of Unicraft as such. So, for today's talk, we have structured it in such a way that we begin to discuss what is existing out there. Then we move on to introduce Unicraft, the motivation behind Unicraft and how everything is structured within Unicraft. Then we talk about how to integrate DPDK libraries on top of Unicraft. Then we speak about the functionalities which Unicraft should provide to support these DPDK libraries. And we make a performance evaluation between a Linux VM as well as a Unicraft image. And we discuss about the synergies of the two projects Unicraft and DPDK and how it fits into this ecosystem. So, when we began, what we had in our mind is something like the diagram in the slide where we have a guest operating system and we have KVM as a hypervisor and the host and the guest interact one another through a para-virtualized framework like Versailles. And then this framework was further optimized with something called VhostNet to further improve the performance of this communication channel. And DPDK came along with its own Vhost user which further enhances this communication model. But one part of this picture which has been left untouched is the guest operating system message. We believe we can do better with the guest operating system. We believe we can specialize the guest operating system. Where in, we take in only the required components needed by the application into the guest operating system. Thereby we optimize it for something like a boot time or image size or even memory requirements for an application. And if we are only running specific application within the guest operating system, we could also reconsider the necessity for having kernel and user space separation within the guest operating system. Thus, if you're focused on a particular use case in mind, then a unique kernel seems to be a good fit for that specific image as such. So if we were to make a comparison between a virtual machine image and that of a unique kernel image, it looks like something like something like this. In a virtual machine image, you have a number of applications and we have a large number of library pool and we have a single monolithic kernel. From an application, an application needs only some subset of these libraries from the library pool and some specific kernel components. Contrast this with that of a unique kernel image. A unique kernel image is built with one purpose in mind. Thereby, you only need an image which has the specific libraries and specific kernel components with the application as such. Just to summarize a unique kernel image, what a unique kernel image is a purpose built image with a thin layer of the kernel embedded into the application as such. Thereby, the application as such has to select which are all the libraries it needs and which are all the kernel components it needs. Since the kernel is embedded into the application image as such, the necessity for kernel and user space separation is no longer needed, and whatever was a sys call in a virtual machine image now becomes function called within the unique kernel image as such. And since we are picking and choosing components needed by the application, we can specialize the entire software stack right from the kernel to the libraries to the application as such. Where do we realize the potential of a unique kernel? We can realize it in terms of faster boot times because it needs only a few milliseconds to boot a unique kernel whereas it takes few seconds for a virtual machine image. Just lower memory footprints. For example, a unique kernel needs only a few megabytes in size whereas a typical virtual machine needs a few hundred megabytes or a few gigabytes in size. And also, it has a higher deployment density so it's easier to pack a lot of unique kernels and its performance is comparable to that of a regular virtual machine image. And also, since we are only making what is necessary for an application into the final image, it has a reduced attack surface because it contains what an application needs. And also, since it has only those necessary components, it is as a smaller trusted computing base as such. So a unique kernel looks quite promising, right? It has good performance, it has quite a nice isolation features in sensation time and image size. But why? The question is why hasn't it been adopted more often? There are a number of reasons for it. Building a unique kernel is quite tedious. It sinks a lot of development cycles to get it right. And since you are doing it for each of the specific applications, you keep repeating the same process again and again and it becomes a bit tedious as such. And the other part is specialization is quite hard to build as such. Because when you're specializing, you need a way to establish dependencies between the different components in the library pool and the different components in the kernel. And the kernel and these components need to have a well-defined interface as such. So bringing up this interface definition is also quite a difficult task as such. Also, since you want to run off-the-shelf application on top of these libraries, you need to make sure that you stick to a standard interface so these off-the-shelf applications can run on your library pool and on your kernel components as such. So with this, we decided to come up with a tool where it makes it easier to build unique kernel as such. So with that in mind, we had these common objectives in mind. We wanted to make sure most of these components were reusable so we don't have to throw away any effort taken to build a specific application as such. So we wanted to build as much minimal basic blocks needed to build off. We came up with a lot of basic blocks so that we can support a wide range of use cases as such. And also, we needed to make sure that we have necessary tools to make sure we can establish these links between the different libraries and the components. And also, we needed to make sure that the generic components or generic components are separated from specific platforms, specific architecture, specific components. So that you can use those generic components across the different platforms as such. And we wanted to support a wide number of hypervisors as well as architecture. So with this in mind, we came up with a tool called Unicraft where everything is a micro library or library in itself. And the two prominent components in the entire ecosystem are the library pools and the build tools. With the library pool, what you can do is you can enable or disable a specific library pool and the build system provides you the necessary convenience to do that. And also, you can establish a dependency between one library to another or one library and a kernel component approach. And this library pool also includes the OS components which are decomposed as such. So how does it affect and also Unicraft is open source, which is BSD licensed. So what does it take to build an application with Unicraft? Let's take an example and let's run through it. So if we take an application like a typical DPDK application in L2 forward, we needed to select which are all the libraries which it needs to select those libraries. These libraries also includes something specific like kernel components. For example, a net dev provides you a net device interface. So you need that in order to support an L2 forward application as such. And then Unicraft provides a different set of platform architectures which it runs on. So those specific libraries which are specific to a specific platforms needs to be selected as such. And also the architecture specific code. And once this is selected, the Unicraft build system as such bakes your final image together and you have your final executable image to run on a specific platform. So when we started our work on DPDK, this was the mental image we had to port a DPDK image to Unicraft as such. Where we have an DPDK application and we have something like a LibUK DPDK which provides the Unicraft functionality to a DPDK application and it interacts with the internal Unicraft libraries as such. But we also had some design considerations into account when we came up when we had this mental image also. One of the advantages of the DPDK library is its modularity and also the benefit of Unicraft is its modularity. So we wanted to retain as much modularity as possible. So to do that, we needed to make sure that the build system of Unicraft can understand the build system of DPDK, thereby it can use its modular libraries as such. And we wanted to find out where are all the possible cases where we can optimize the guest operating system where you can simplify a lot of operations for a DPDK application as such. And also the final constraint which is for quoting any libraries, we wanted to minimize the changes within the upstream code within DPDK as such. So to make sure these build system compatibilities are matched, we first compare and contrast the two build systems as such. Unicraft build system is K-config based whereas DPDK build system has some auto config generation happening. And also Unicraft build system has some specific library variable naming convention whereas DPDK also has a similar naming convention which needs to be translated from one to another. And also it had something called export sims in Unicraft which tries to hide unnecessary functionalities from a library to other libraries as such. Similarly DPDK had a version map equivalent. So in order to address these differences, we came up with a solution using a typical Unicraft approach which is to build a library around it. So we came up with this library called DPDK build library. What it does is it processes the make files of DPDK and translate it to the make files understandable to Unicraft as such. The advantage of doing it is we can add a DPDK library as such to Unicraft and Unicraft can then translate all its necessary dependencies to a Unicraft build system understandable make file as such. Thereby what advantage we gained is we were forward compatible. So as long as there was no build system change in DPDK as such, we can adapt all its libraries necessary to Unicraft without any further changes anywhere else. And also this library contained, if you have any specific configurations within DPDK, this library contained those configurations also. So with that in mind, we came up with this sort of an architecture diagram or a building block diagram where we retain as much as the DPDK libraries are concerned. For example, this RTE, ETH Dev or the memory pool M pools or M buffs were not changed one bit within Unicraft. Whereas for the abstraction layer, we needed to add some specific Unicraft specific code which we'll talk about later. And apart from that, there are two libraries which are of consequence here which are the UK Ring and the UK PMD libraries. We will talk about these libraries further down the top. So then the next part of our discussion was how to specialize the guest operating system message. We looked at three parts where we could optimize on where you could optimize the Unicraft guest operating system. One is the memory management system. The second is a device management system. And third is the scheduling part. In terms of memory management system, what Unicraft provide was a static memory image where we have huge pages of two gigabytes mapped into the first one gigabyte of the address space. So thereby we only support one gigabyte of memory and thereby we have a one-to-one translation between a PA and VA with physical address to a virtual address within that of a Unicernel lesson. And from an application perspective, an application can specify or reserve and memory range from the Unicraft Unicernel. Thereby you can make sure that this is specific to, you can reserve this memory region and you can take this memory region and use it for your memory buffers. And also it is possible to implement multiple allocators to understand which allocator is best suited for your specific application. In terms of the device management itself, what Unicraft provides you is Unicraft just probes the devices and lets the application take over the device controller altogether. So Unicraft identifies which are all the devices necessary for an application and just hands it over to the application and the application, it's up to the application how to handle these devices. You don't need any additional file system or anything else to handle these devices as such. And finally, for scheduling as such, Unicraft gives you the ability to make sure that you can run an application, an application can hog the CPU cycle until it can yield to another thread. So Unicraft uses a cooperative scheduler, thereby runs an application can run a task without minimal interference from the Unicernel as such. And also, if it is possible, if necessary, we can also completely do away with the scheduler and run the entire application within a polling mode where everything is event-driven, where everything is event-driven or interrupt-driven. So next, what is needed from Unicraft to support a DPDK library as such. So we compare and contrast how typical data flow works between a Unicraft library and a DPDK library. So on the left, you have the DPDK workflow and on the right, you have the Unicraft workflow where you have an application wanting to send calls the RTEH Dev burst in the DPDK and UK Net Dev burst. And then it calls the virtual drivers to send out the data out. So you need, and also here, DPDK libraries handle M buffs, whereas Unicraft libraries handle net buffs as such. So there is a difference in data structure between the two libraries. So we need a way to have to mangle and to mangle from one data structure to another. A similar diagram for the RX part of it. I'll skip this part. So what is the difference between a net buff and the M buff as such. So with a net buff, what you have is, you had a packet data with a net buff, you had this net buff structure and then you have the user private data within the, which is the applications usage of the net buff. And then you had the packet headers and the packet data. Whereas if you take the equivalent of an M buff, you had this M buff buffer, then you had the packet data and you had some private data for it. So we needed a way to mangle from one net buff to an M buff as such. So we came up with this UK Ring library, which does this mangling from one data structure to another. And also the pole mode driver translates the call, anything which is happening within the DPDK context to that of the Unicraft context within the net. So thereby we were able to translate from a net buff to an M buff and M buff to a net buff with just these manipulations, whereby we point the private data of an M buff to the net buff and the private data of a net buff to an M buff. So it's easier to translate from another. And finally, we talked about the performance evaluations. Here we used two systems. One was the packet generator receiver and the other system is the one which runs Linux VM or a Unicraft VM message. It is a Sandy Bridge server with six cores on it. And we were using DPDK in 1908, Kibu version four, and the Linux kernel, the host kernel and the guest kernel were 4.19 dp. And in terms of the next graph, where we show is the baseline performance. What we do here is we vary the packet sizes from 64 bytes until 1500 bytes and we measure how much throughput we get in terms of millions of packets per second. So what we gain is we, in terms of, in terms of, we measured it in four mechanisms. One is using the V host user and which is the DPDK accelerated version and the V host, then it's the kernel accelerated version. And in terms of Unicraft versus DPDK comparison, we are quite similar in transmission performance as such similar performance. We did for the RX throughput where the RX part of Unicraft with the V host user is slightly slower than it's about 11 million, 11.5 million packets. Whereas the RX of Linux VM was around 13 million packets for the minimal packet size. And for the rest of them was quite matching. So there are some bottlenecks within the RX which we need to figure out, which is part of the future work as such. So in the previous two experiments, what we did was a packet generator keeps sending packets and we drop the packet once we receive it or once we send it. Here we did another experiment, which is like key values to what we did was a packet generator sends you a stream of packets. We receive the packet, we read some specific based on the values in the packet, we read a key value store and we send out the packet out. So with that a single core is now processing the receive of packet, processing some data within the packet and sending it out. And we made a comparison of that with the Linux VM and that of a Unicraft and with DPDK and Unicraft with NetDev running. And all of them were quite similar in terms of six million packets per second. And in terms of resource usage itself, what we observed was in terms of memory usage, Unicraft needed to run the DPDK application. Unicraft is just one gigabit of memory whereas Linux VM needed a six gigabit of memory. So it's about six or the same savings. The boot times of Unicraft is about 87 milliseconds whereas the boot time of a Linux VM is unoptimized is about 12 seconds a second. And the image size comparison is the Unicraft image is about 1.4 megabytes in size whereas a Linux VM is about 2.5 gigabytes in size. So it's about a huge order of magnitude difference between in terms of resource usage. Finally, the next part of this journey is we needed to add support for SMP's drivers, SMP support so that we have multiple CPUs. And also we wanted to try out running DPDK drivers instead of Unicraft drivers which were run so that we can keep it up to date. So if you find this work interesting, you could also for further references you could I have put on some of the links, you can refer it. And that's to conclude my talk of Unicraft. We made us a comparison. Unicraft provides you multiple platform support, a specialized guest operator guest OS, whereby we optimize for image size, memory consumption, and also the boot times at the same performance. And we have a simpler manage device management, a simpler device management for a DPDK application. And we provide increased control for an application compared to that of a Linux guest VM. And in terms of DPDK, DPDK provides you the benefit of a highly optimized network stack, a specialized VNF functionality, and a lot of untapped potential from the DPDK drivers as such. So I think the two projects go hand in hand in the VNF domain. And it is a benefit that we use Unicraft as a DPDK image in the virtualized network functions. If there are further questions, I would like to leave the floor for further questions. If you have any questions, please ask. Thank you.