 Hello everyone, I am Santosh Fanande's Principal Engineer, Walmart Global Tech. I am passionate about systems engineering and distributor systems and building network solutions. Today I am talking about the product called LEAF, which stands for Lightweight EBPF Application Foundation, which provides complete life cycle management of EBPF programs. A challenge we faced when we decided to adopt EBPF in our environment. How do you manage and orchestrate multiple EBPF programs for a node in a large scale environment? How do we deploy or change the configuration on the fly for the thousands of nodes in our environment? We have thousands of nodes, different cloud vendors and multiple data centers to be managed. So there was no readily available platform to do similar functionality. We decided to build our own platform LEAF, which does the complete life cycle management of EBPF programs, which provides us an EBPF program as a service. It will give you simple APIs to add, remove, reorder the EBPF programs on the fly. It allows you to change the configurations of the EBPF programs on the fly. It also provides you the metrics generated by these EBPF programs to publish from this model. It also replaces proprietary applications and the hardware with the blazing EBPF code. So there is also a community-driven EBPF package repository where different capabilities EBPF programs are developed and uploaded. So that once it is built, we can use anywhere. It's a functionality of build once, deploy many times. Manage a configuration EBP programs per node basis. It's also a cloud and vendor agnostic. It has got an automation of configuration management. So much more capabilities in a LEAF platform. So going forward, we can go a little more deep dive on it. This slide depicts about the LEAF platform. On the right-hand side, we can see an EBPF package repository. We can see multiple EBPF programs have been stored. These programs have been developed by the LEAF community or a third-party vendor, which provides you to manage or upload in the repository. This runs in the build once and deploy anywhere philosophy. So that once you compile and upload in the repository, then it can be deployed anywhere as and when needed. On the left-hand side, we can see a LEAFD and we can see the APIs to manage EBPF programs. So any request comes to the LEAFD. It verifies for the EBPF program artifacts and the artifacts to be downloaded from the EBPF package repository. And then it will deploy on the node with the given configurations which are passed by the API. We can leave Damon orchestrate multiple EBPF programs on the node. It provides you XTP and TC chaining for the EBPF programs. This is a core functionality of the LEAF Damon. It also provides you health information and metrics generated by the managed EBPF programs. It enables with the secured web API to manage EBPF program. Leave Damon is a cross-platform. Today it runs on the Windows 2 port in LEAF. LEAF supports XTP and TC program chaining. It attaches the root program of XTP and TC to the interface. When it attaches to the interface, it also creates the root map, root program map. So that next program which comes into the in the chain will update its program FD into the root map. And there is a tail call which will actually does the context switching between the first program to the next program. How does we decide the location of the EBPF program in the chain? So the location will be provided by the user in the config API through config API. It has got a sequence ID, the number which decides the location in the chain. And this entire chain is built on the concept of linked list. And dynamically we can reorder the chain. We can stop the EBPF programs. We can attach the EBPF program in the chain like a linked list. All these things are achieved using Celium's Go EBPF library. LEAP provides to update the configs of the EBPF programs on the fly. So no restart is needed for any config changes for the managed EBPF programs. How does the sample config update looks like? It comes with a tag name map underscore arguments and it takes the list of key values. The key will be the map name and the value will be the value to be updated to that maps. The picture shows that LEAP de-updating the config BPM maps. LEAP provides the capability to publish metrics generated by managed EBPF programs. The LEAP will read those maps and convert those data into the promotional output format. How does it knows which map to read, which aggregator to use? So these parameters are passed to the LEAPD from the config API. So the sample config API looks like the list of monitor maps to that BPM program with the name, with the key and the aggregator will be defined. We support following aggregators. Max rate, maximum value for the given sample window size. Average, average of values for the given sample window size. Scalar, the counter value as is. The window size and the polling interval can be configured in the LEAPD.cfg. Below here we have depicts the sample promoquial output format, which contains different tags like host, map name and the network function name and the value of the metric. LEAP provides set of APIs to update the configurations on the node. Also to retrieve the EBP programs configurations on the node. We have enabled MTLS for our web APIs. MTLS stands for mutual TLS. MTLS runs on the zero trust security framework. No remote connections are trusted by default. We support secure download of EBP package over the TLS. This shows the sample config payload. You can see the host name, name of the host, the interface for which these EBP programs has to be attached. And the EBP programs are categorized as XTP ingress or DC ingress or DC egress programs. So we can see here sequence ID which says the position in the chain. Artifact, the name of the file which contains the bytecode. Which will be downloaded from the EBP package repository. Map name, the name of the map created by this EBP program to store the program FD for the next program. Then similarly we need to focus on the version of the EBP program. So admin status says that the program has to be started or stopped. Enabled means it will start the program. Disabled means if the program is running it stops the program. Let's look into the demo now. So we have a demo setup which where the leaf is running in the migrant VM. So we can see right hand side the migrant VM will be having leaf daemon, EBP package repository and web server. On the left hand side is a host which will be used to send request to the leaf daemon. So we are using the head tool to generate the load on the migrant interface. And we have a browser to monitor the Grafana dashboard. The left hand side of the terminal represents the host side of this. The commands which are typed are executed on the host. The right hand side terminals actually the console for the migrant VM. The commands which are executed are executed inside the migrant VM. Let's see the config.iml. Here we can see those host port representation, forwarding ports from the migrant to the host ports. So here we need to see the leafd code directory from where the leafd has been. Code has been picked up and deployed inside the migrant VM. Let's start the. So the leafd is running here with no configuration, no ebp program running. Let's check the swagger API. This gate API will retrieve all the ebp programs currently running. Let's check that. So there is no ebp programs are running. Let's go back to our terminal and send some load to the migrant interface and see how it responds. It took less than a second to run the 200 requests and 20 parallel connections. So less than a second it could complete the request. Now let's deploy the connection limiting and rate limiting ebp program which will actually limits the number of requests it can process by the backend service. So it will automatically slow request processing. Let's go to our migrant and use this. Try this update API. And I have created the payload in my notepad. Let's talk about that. So I've set the config map as a max connections per second is two for these two ports. So let's update it. So successful. Let's go and check on our view. So yeah, these are running. Now we can check our ebp map. You can see this so many maps are created by this ebp programs. And if you want to see the particular map which we had updated here should have value. You can see the two values set. Now we will run the hey tool again and we can see there is it took much longer than the last second. At the same time we can go and check the number of programs running on the node. You can see here the details of the programs running on the node. Check our Grafana dashboards. There are predefined. It's taking much longer time. It is dropping connections in the huge. It's finally completed. It took around 80 seconds. Pre-driven projects. If you are interested to work at the leave here are some links to start with. Thank you.