 to TCP V4 Connect. This is the corresponding user space code. By the way, this code is really from the original BCC code. As you can see, the user space code in this example, it handles command line arguments in this case, passing the configuration of the user, and it loads and clean up the EBPF programs, clean up the maps so it doesn't leak any memory. And it also loop to read the data of the map and then format the output to something the user can easily to read. So now on the next question, as you can see, the user program is actually pretty complicated. What if we could just write the kernel program? Because the user space program actually take way more lines of code than the kernel in this example. And also, we love the Docker experience. How many of you use Docker before? Do you like the Docker to be able to build, to be able to push, and then you can pull somebody else's registry? So we all love that Docker experience. So this is the open source Bumblebee project was born. So solo along Bumblebee, I believe in January this year. So if you scan that QR code, that would take you to bumblebee.io. We recently applied Bumblebee to be a CNCF sandbox project, so finger crossed. We will be accepted. So essentially, Bumblebee is enable you to easily build your ABPF programs and to be able to publish that program to OCI compliant registry and enable your audience, your users, to be able to take the program from the registry so you can easily run the program. On top of that, Bumblebee also allow you to focus on writing EBPF code while taking care of the user space component for you, automatically exposing your data as metrics and logs. So Bumblebee supports LibbPF compatible BPF code. It doesn't support the legacy BCC-based EBPF code. It provides a containerized building environment so you can easily build your EBPF program and then push it and run the programs with the OCI registry. So let's talk about building with Bumblebee. So as I mentioned, we provide pre-built containerized building environment. As part of this environment, which is this image you see here, it essentially includes the compiler, which is the LLVM and CLUNC, because we only support C at the moment. It also supports some, we also embed the BPF headers along with the operating system, dabbing here as part of the builder image. When you take your EBPF program and build with Bumblebee, you essentially run the B build command, and then you can, Bumblebee is going to take your EBPF program, build into the LFL, which is the bytecode we talk about early, and then Bumblebee also packages the LFL into the Bumblebee OCI image, which you can take that image to push it to any OCI registry compliant registry. When you run the Bumblebee OCI images, what you can do is using our runner. So you can run a local image, you can run a remote image, you can run somebody else's image as long as you trust it. So let's walk through how this works. So with the moment you specify B wrong command to run the images, what's this going to do is we're going to try to pull down the images to the local machine if it doesn't exist, and then we're going to load your EBPF program as part of the image. Remember the bytecode is part of the image, so we're going to load your EBPF program in the kernel for you. In the meanwhile, we're also going to create the map for you so your EBPF program can interact with the map. And as part of the running, Bumblebee also watch the desired map and output the data from the map and emit the metrics for you so you can config the premises to read that data. All right, so writing EBPF programs with Bumblebee. In this example, this is the same TCP Connect C. In this example, we walked through earlier where we focus on just the EBPF program portion. So this is the event which hash map, which contains the dimension structure, which has source destination address. And on the right side, we have the interaction of the map. So we are reading the source and destination from the event, and then we are updating all that relevant information to the map. So the hash map has all the information related to the TCP Connect event that we want to capture. So running with Bumblebee, in this example, you can see the source and destination address where you specify in your struct. We're going to display it as part of the Bumblebee UI. And the value, which is the map of the counter of the map, we're going to display how many times from the source and destination here. We'll also emit the metrics automatically for you without you needing to do anything. So you can see all the events related to TCP V4 Connect with source and destination all captured as the events in the metrics. This is another screenshot of the events from premises. With that, I would love to show you guys a demo. Yesterday, as I was showing demo, I had a lot of Wi-Fi issues. So hopefully, fingers crossed today, I would be doing better. But I did get the demo recorded. So this is a demo I developed to run out of memory on Kiosk application to detect out-of-memory Kiosk with the help with EBPF. In case you are not familiar with what does on Kiosk mean. So if you Google this, this blog was very helpful. So on Kiosk, it's all about Linux on Kiosk kernel process. So Linux kernel detects this process continuous monitors the node memory to determine memory exhaustion. So if such a condition was detected, it will attempt to kill the best process. So it kills the least number of process to minimize the damage. So this is not something related to Kubernetes. It's actually related to about Linux kernel process. When I first saw my pods as on Kiosk status in my Kubernetes, I actually thought it was from Kubernetes. So this blog really explained that well. So what I'm doing right now is we are provisioning a virtual machine in the cloud. And that virtual machine is going to be our playground for the demo to help us to observe on Kiosk event inside of a Kubernetes cluster. So we're hoping to, you know, with this to get a Kubernetes cluster with kind. And we're hoping to deploy some pods with memory leaks. And then we're hoping to leverage Bumblebee to detect the pod and the process within the pod that's causing the on Kiosk events. Does anyone have any questions where we stand up the environment? I know the room is pretty cold. I feel like my hands are so freezing. All right, so we do have an environment. And hopefully the Wi-Fi stayed with me. So now in my environment, it's a VM in the cloud. We're starting a Kubernetes cluster now. So you can see I probably have my Kubernetes cluster running. I don't have anything in the default namespace. And, you know, I just have a core DNS. And I have some nodes, hopefully. Yeah, I have three nodes, one master to work. All right, so the first thing we're going to do is we're going to look at the lead BPF tools, which has the on Kiosk. So we're going to look at this BPF.C program, which I have it open here. So this is very similar as the TCP connector I showed you earlier, which we have a map that's been created. And it's hooked up to the on Kiosk process in the Linux kernel. And it's trying to read out some of the data, like the from, where the, which process is from, which target process got on Kiosk. And, you know, it's trying to write that data to the map so the user program can consume. There is also a matching on Kiosk.C program, which I believe is the user program. So the user program is actually a lot longer. In this user program, we have the arguments related to the commands. We're passing the arguments. We handle the events. And, you know, we have a main method, which we need to pass the map and output the data so the user can consume that. So as you can see, it's like 189th of user code. So let's see how Bumblebee can help out, right? So you don't have to worry about user space code. So the first thing we're going to do is we're going to download this EBPF program, Omkio, that we just reviewed together on my machine. And then, also on my machine, I already have another version of the Omkio that works with Bumblebee. So what I'm going to do is I'm going to open up the two programs so we can take a look at the difference together. So the first difference is we didn't include a header here because it's a small program. So we didn't need to have multiple files. And the second difference is instead of using a perform event array, we're using a ring buffer. This is because a ring buffer allows you to reserve. And also, it's better performing. And Bumblebee also supports ring buffer. It does not perform event array at the moment. And we also change this to the map.counter so that Bumblebee can display the counter on the Bumblebee's UI. And then, as part of the K-Probe, it's relative the same except we deal with ring buffer with the event, the proof event array. As you can see, we can reserve a good size for the ring buffer. And then we essentially get the current PID ID. We're also getting the target process ID and the target communication. And then we're submitting right that data back to the ring buffer so the user space program, which is Bumblebee, can access that data. So with that, we're going to get out of the WIM differ program. And what we're going to do next, let me see how much time I have. So what we're going to do next is we're going to download Bumblebee. So the download Bumblebee is really simple. We just run the code command, specify which version we want to download. And now we should have the B command on the path. OK, so now we can review this program, which actually is very similar, which we just reviewed through the DIFF program. So we're not going to review in detail, but the key thing is we're using a ring buffer. We'll capture the from, which is the source of the PID, and also the target PID that got Armcare. And we'll also capture the pages of the memory when it happens, right, when the Armcare events happened. And then we're writing all that data into the map through the EBPF program as part of the K probe here. All right, so we have our Armcare program. And we're going to copy that program into the current working directory. And then we are using Bumblebee to build it. So what Bumblebee does right now, remember, the B builder has the compiler, right? So it's going to build the EBPF program, which is Armcare.c here. And it's going to try to build it as a byte code, right? So the L file we talked about earlier. So it's going to build that L file. And then it's going to package it as OCI images. So you are going to see the OCI images locally here on my machine. Actually, I shouldn't say on my machine, but somewhere in the cloud, which you are seeing locally, 2.vm. So now we build the image. I already have a registry running locally, which you can see. I'm running registry 2. So what we are going to do next is push that image. We just build together to my local registry. Well, that was pretty fast. I'm so thankful about the Wi-Fi today. And now we are going to take a minute to run that image. So this is the Bumblebee UI when you run that image. So you can see nothing happens, right? Because we don't have any interesting events. So let's try to generate some events, some omkio events. So the first thing we are going to do is we're going to create a demon set together. The reason we create a demon set is we want to have a bee runner to run the images we just created together, which is omkio version 1. And the demon set, one thing I want to highlight is have a label, which is app called Bumblebee. And also the demon set exposes a port named the HTTP monitoring with container port 1981. So that's for premises part monitoring later on, which we will show you shortly. So we just deploy the demon set to my Kubernetes cluster. You should see the demon set as parts. So it's running. Remember, at the beginning, we said we have three nodes for my Kubernetes cluster. So demon set deploy one part for each of the node. So that's just running. And the next thing we're going to do is we're going to install premises. So the first thing we're going to do is add through Helm. I did have Helm pre-installed. So we're going to add the premises community Helm charts and do a Helm repo update. And then we're going to install the Kube premises stack in the monitoring namespace. So now we just add it to the repo, and we got repo updated. And now it's installed the premises stack. So hopefully finger crossed. I think this is also using Wi-Fi space. So let's check if that's running. All right, looks like maybe we have one. It's not running, but the rest are thankfully running. OK, that sounds good. In addition to that, let's see if this reaches running. Yes, it did. All right, we're going to deploy a pod monitor to scrape our bumblebee paths. So if you remember, when we deployed the bumblebee demon set, we had app equals bumblebee. So that's how the pod monitoring is configured to select our bumblebee app. And if you remember the metrics endpoint, we had a port called HTTP monitoring. And the path is on slash metrics. So that's how we can fake the pod monitoring to scrape the bumblebee pod on the HTTP monitoring pod. All right, so we got that deployed. Now the next thing we are doing is to do a port forwarding. The reason we do a port forwarding to premises is just mainly making sure we can access the premises UI. So hopefully the UI is because of that port forwarding. Now the next thing we are going to do together is we're going to deploy a bad memory leak pod. So as part of this pod, it's going to trigger memory because it's keeping leak memory. So it's going to trigger on key events. So as you can see, it requested a 64 megabit of memory. But it has the limit of 128 megabit memory. So when it hits the limit, it's going to trigger on key events. So let's check it out. Get pods because you can see it already crushed once. Now if I go back to bumblebee, what do you think is going to happen? So bumblebee was through our EBPF program hooked to the on key event. And bumblebee was able to detect from which PID that's causing the on key event. And eventually this is the target PID that's got killed. As you can see, in this case, because our example was very simple. So the from to the to the target, it's the same PID number. So that's the PID number where you get into the out of memory leak pod. That's the PID number gets killed. The pages is interesting. So I actually did a calculation of the pages. So one page is 4K. So 32, 7, 6, 8 pages is about 131 megabyte. So if you remember when we deployed this pod, the limit is 128 megabyte. So whenever it hits this over the limit, that's when the on key events get triggered. And that's when the pod gets killed and restarted. So with that, I also want to show you one last thing. I know we are close to run of time. I want to show you that you can also curate the data in permissives. The way to curate the data is curate the events. If I can have my permissives back here. Sorry. Yeah, it's responding now. So this is a convention. It's the OMKEA, it's the EBPF program. So the events. So you can see every single event that you saw on the Bumblebee interface here is also captured in permissives. So this is really useful so you can set up with alert or you can generate your own UI to render this. So very nice way to get all the events on this. With that, I think that reaches the end of the demo. Now I want to pause one thing to mention. We are at the booth. If you are interested to ask more questions about EBPF, e-steal, Envoy, and we are also hiring. In fact, we run an EBPF workshop tomorrow, one of my co-worker, Adam. So if you want to do some hands-on with EBPF, I highly recommend that. That's all I have. I want to open up, see if you guys have any questions. I'm very thankful the Wi-Fi was good today. Yesterday, I had so much panic moments. And the best part is the audience actually helped me think of what's going on with my commands. And you guys were there, so it was very engaging. So any questions? I mean, if you guys wanted to learn about EBPF, I highly recommend Bumblebee. I also wrote a blog about Get Started with EBPF with Bumblebee. It shows the program, a Hollywood program, which is a little bit simpler than this program. So that's a great way to get started with the EBPF. It shows you all the steps you need to do. Like I said, the best way to learn is actually hands-on, because you could listen to a lot of talks. But the hands-on really gets me to understand the interaction between the EBPF program and the user space and why you need both, what are the scenarios, how you can potentially leverage Bumblebee, so you don't have to write the user space program. So we're really excited about Bumblebee. We're taking it to CNCF as a sandbox project. If any of you are interested to the project, we certainly welcome any type of contribution or collaboration help us shape the roadmap of Bumblebee. Thank you so much.