 Hi everyone. My topic is ADP with RDMA. That is ADP is implemented in RDMA, specifically sub-rocky RDMA. The agenda is, this is the agenda for today. The first is about ADP, including ADP introduction, ADP use case and ADP action. The second is RDMA, including RDMA introduction, Yifin Band and Rocky version 1, Rocky version 2, RDMA summary and sub-rocky. The third is high-level design about ADP with RDMA, including introduction, new flag, construct and output format. The fifth is ADP kernel implementation, including new driver, sub-rocky, ADP initialization, ADP send pass, and the reshoot pass. The final is a demo. ADP express data pass is IEBPF based high-performance data pass used to send and receive a network package at high rates of bypass most of the already set time network. Network stack ADP is a programmer and high-performance networking data pass. ADP defines a package processing process that runs very low in the system at close to the hardware as possible. BPF provides the probability for ADP allowing users to access the low-level networking data pass to implement word-to-switches, load balancer, firewalls, and for ADP programs are intended to possible access differently. ADP platforms, which includes not only linear kernel but also potentially NIC overload switches, DPTK, and other operating systems. ADP is IEBPF and early package processing. ADP is short for Expressed Data Pass. It is marked in kernel 4.8. ADP is runtime programmer package processing inside the kernel, not kernel bypass. ADP applications are compiled to platform-independent EPPF byte code. Object file can be loaded on multiple kernels and architectures without becoming compiling. ADP can be used in the following scenarios. DDOS, early drop-in malicious package, when DDOS is detected, the malicious packages are dropped with ADP drop-in or about silently. This decreases the CPU utilization load balancing with ADP redirecter. ADP will redirect packages to other NICs or user space applications to implement load balancer. The third is a sample monitoring the contents of packages. In my demo, I most used this sample to check the contents of ADM packages with software. The fourth is possible forwarding destination, networking device, or user space application with AF-ADP packages are redirected to user space applications or other NICs directly without network stack. ADP applications now ADP actions are shown on the right picture. All valid ADP programs must retain one of these defined values, unknown routine codes without resulting package drops on the awarding via BPR for valid ADP action. ADP drop-in about drop package immediately, about drop-in with trace-up point exception. ADP pass build-act KB and pass dot package to network stack. ADP transmit dot package back to the interface. The fifth is ADP redirecter, redirect packages to other NICs or user space applications. Now the ADP introduction is complete. Now I introduce ADM. ADM is short for remote direct memory access. It is a relatively new method for interconnecting platforms in high-speed networks that or comes many of the difficulties encountered with traditional networks such as TGP IP or Ethernet. ADM is a new standard, it is a new protocol. It is a new hardware interface called on the switches. ADM is a new software package. ADM implementations including IFIN band and ROKU version 1 and version 2. IFIN band implementation needs special devices. For example, HC hosts the channel adapter, IB switch and special keyboard ROKU, including ROKU version 1 and ROKU version 2. ROKU version 1 is based on TGP IP Ethernet layer. It is based on the layer 2. It only works in local network and it is not popular in data center. ROKU version 2 is based on UDP. It is popular in public and private cloud and data center. ROKU version 2 is based on UDP IPv4 or UDP IPv6. The target UDP port 4791 is reserved for ROKU version 2 because ROKU version 2 can be rootable. ROKU version 2 sometimes can be called rootable ROKU or ROKU. ROKU version 2 is supported in Malinux O5 2.3 or above. In Linux 4.5, ROKU version 2 is supported. E810 also supports ROKU version 2 and E810 also supports IWOP but we do not use IWOP, we just use ROKU version 2. ADM is a network performance optimization, a traditional networking stack, so-called TGP IP Ethernet, how inherent performance barriers are on the CPU or head. ADM is designed to reduce or limit these barriers. The following are the benefits from ADM technology, including CPU utilization and memory bandwidth, software implementation of the IBT ROKU version 2 specification, ADM transport service or Ethernet network. The following is the package of ADM. It is UDP payload. It includes base transfer header, IB payload and ICRC. The right is the cooperation between IBT and ROKU version 2. IBT stack includes IB network, IB link, and IB management while ROKU version 2 is UDP payload. It works in TGP IP stack. The use case, we use software ROKU developer new ADM features on software ROKU. It is easy to debug on developing application on software ROKU. We can use software ROKU test. It is used very popular in public or private data center. We can use software ROKU to connect to monologues firewall or Ethernet to check the package is correct or not. There are three parts of ADM software architecture. The first is user space. Software ROKU user driver is in ADM called software package. IBT ROKU is called the specific implementation of software ROKU user driver. The second is kernel. Software ROKU kernel driver is a touch on a specific NIC driver, ICE driver. This is the driver of the entire NIC driver. It receives the ADM package from more ADM stack build and put ADM data into UDP payload and send it to TGP IP stack. The third is hardware. We can use any NIC including E18. This is entire NIC hardware. In my demo, I use E18 without IDMA. IDMA is the IDMA implementation of E18, and I attach software ROKU on E18. This is the high-level design flowchart. The difference is that NIC driver transfer ADP program to software ROKU driver. The software ROKU driver uses this ADM program to map ADM data between kernel and user space. There are six parts. In this picture, these parts can be divided into two parts. The first is user space. Parts 1 and part 2 indicates ADM ADP user space C file. They can be compiled into a binary file. Parts 3 indicates this binary is checked by eBPF verifier. The second is kernel. In this kernel, there are three parts. Parts 5 indicates this binary file is attached to NIC E18. Parts 6 indicates that the binary is transferred to software ROKU driver. Parts 4 is mapping ADMA data between kernel and user space. The ADM application access not dealt with. Then ADP is transferred to ADMA driver. In my demo, I used software ROKU. Then ADP works in ADMA and receive both parts of ADMA. All the comets work in kernel 6.2. In this picture, there are three parts. Parts 1 attach ADP program to NIC by IP link commander. In the demo, NIC E18 is driver. Parts 2 ADP program is transferred to software ROKU receive parts when rounding dump ADM package to the user space. Parts 3 is similar to part 2. It works on software ROKU send parts. The following are the specific implementation. This is the user space application. In user space, 2000 are involved. The first is ADMA package cannot see ADMA package. In ADMA package cannot see the retained value is ADP pass. This means that the packages are resumed to travel in the kernel stack. In ADMA package, user see the following will do. ADP flag is ADMA is introduced. Parts 4 ADMA package. The third is output of information of packages. This will introduce the new flag, ADP flag, ADMA. ADMA flag is introduced. This new flag is showing in the following. This flag is set in to attach of ADMA package, user see ADP program in user space and checked in ISE ADP. ISE main seed in kernel part. This will introduce the core structure in user space structure like the year of code before. With this structure, ADMA package are positive. The core structure is at the below. There are three main variables. The first is the name. This indicates the operation name. The second is the length, the distance between the beginning of the beginning of this structure. The third is offset the structure in this ADMA package. The core structure is initialized in my source code. In this PPT, I just showed the first record. This is the example output of user space application. This output is one package from IP connections. This output is displayed as a structure. It is similar to CRUSH. The number following the structure member variables are the values in these IP connections. The output contains source, destination, IP address, UDP information, and ADMA package information. This is our IP package analysis, including IP, UDP, and ADMA headers and payloads. In kernel, two drivers are unidid. The first is NIC driver. In my demo, I used the driver of year 18. The second is ICT driver, software driver. In NIC driver, this function that in the old BPL of points, we all check the ADMA if ADMA is set. ADP program is transferred to software key. If not, this ADP program is used in the NIC driver. The third is in software key driver, three parts are involved. The first is initialized. The second is send parts. The third is receive parts. This is the NIC driver. NIC driver checks the ADP flag, the ADMA if the flag is set. ADP commands the ADP setup program. We all set that our right point arrow. ADP program is below. If not, ADP commands, we all work as euro. In ICT driver, that is the original IC ADP setup program is called to perform original tasks. In software key driver, the ADP initialization includes send and receive initializations. And the following scenarios should be considered. First, when software key driver has been attached to the NIC, the event that that will change is sent out. Software key driver receives this event and makes sending receive initializations. Second, before software key driver is attached to the NIC, the ADP user base application has already been attached to the NIC. During software key driver configuring NIC network, the software key send receive initialized down. This is the software key send pass. In software key driver, send pass, ADP fetch the ADP to a new ADP, ADP list, ADP user base application, fetch the ADP and analyze them. In the right diagram, the ADM data from ADM stack is built into a FKB package and stored in UDP payload. And at the same time, it is cloned to ADP FKB queue. Then the FGP program fetch our FKB package from this queue and analyze the ADM data. Finally, this FKB package is freed. The built FKB packages are still sent to network stack. In software key receive pass, the FKB packages are from NIC. If the ADP is enabled, the FKB packages are parsed by FGP program, user base application, face the other tools to ADM stack. If not, the FKB package goes to ADM stack directly. In the right diagram, FKB packages from NIC are sent to software key receive pass. Because of UDP port 4791, in receive pass, if ACT-DP is enabled, ACT-DP program will pass the FKB package. If not, the ACT-DP package are handled and sent to ADM stack. Now, ADM and the design for this topic are completed. This is the demo. There are four windows. I show you. The top window is to capture and display the ADM package from IP server. The bottom left window is IP client. The bottom right window is to capture and display the ADM package from IP client. The ACT-DP program on client and server are targeted. When IP client is sent to the server, then the packages are received and displayed on these two packages. Thank you. This is all. Thank you, Xu. I have a question. You said it's for the software raw key, right? So you mentioned XDP. Do you implement all these XDP TX, XDP pass, all this command? Or for your use cases, mostly for tracing, dumping some details of the SKB for performance and debugging purpose? Yes. This is, for example, the ADM package. If I get you corrected, you want to know the purpose of this application. I think this application is to sample the ADM package with software raw key. When the private and public data center, it is very important to check the content of the ADM package. So I use this to check the content of the ADM package. That is all. I think this is my answer to our question. If the purpose is for dumping a package, have you looked at tracing to do the same thing? Yes. We can use this to trace the ADM protocol implementation. When we make a firmware development, we can use this application to check the raw key version to a protocol parameter. This can help a lot of developers to check the raw key version to implementation, correct or not, if I can get you corrected. I think you want to know another purpose to trace the raw key version to implementation. When we develop an ADM application with year 18 or Monox file hardware, we can use this software raw key to connect to Monox and all year 18 and to check the raw key version to protocol parameter, correct or not. Simply, we can use this to check raw key version to protocol parameter, correct or not. That is all. One last question for me. Are you planning to post what you have in the kernel soon? You may change. Yes. This is already and I will send out very soon. This is used in several companies and several developers use them to make a sample and trace the raw key version to protocol parameter and we are very soon to send these out to the for code review. If some more body are interested in this, I'm very glad to discuss this. Thank you. Okay. Thank you. Sorry, I had this. It is my first to join this meeting. I'm very nervous and shy. If I do not well, please do not be stricken with me. Thanks for your presentation. The demo is very useful. Thank you. Okay. Bye-bye.