 Okay. Welcome everybody. I'm Kyle Mastry. This is time to go as well. Yeah, we're here to talk to you about EBPF and XDP and how it will revolutionize the telco 5G space. And so this is our thesis that we have, and we're not going to read through this. We'll let you read through this at your leisure here as well. But ultimately, we're going to try to answer this question at the bottom throughout this presentation. And Tom, we're pretty relevant for this, right? Wouldn't you say in terms of us being able to answer this? Yeah, we've both been sort of up to our eyeballs in and around this area for a while. I worked on the first nationwide open source 5G deployment over the last year and a half. And you've spent a lot of time on EBPF in the last couple of years too. That's right. Yeah. I mean, we've both worked in open source networking projects all the way back from open daylight. From way back when? We switch every all kinds of fun stuff, right? And that's one of the interesting angles here too is the open source meets traditionally proprietary tech, even though there are 3GPP standards. You know, it's kind of the analogy here is very much like SDN was, you know, 10 years ago, whatever that was, where disaggregation of the data and control planes had certain promises that went with it. That's right. 10 or so years later now, we realize what we've got. And so we're trying to overlay that model onto what's going on in 5G. Yeah. By the way, sorry. You can say that's a good segue to jump right into our stuff here. And so as you're going to explain this slide to us, but I, you know, when I look at this as someone, I think, well, can you explain how this 4G is different than 5G? So there's a lot of parts on here. The good news is that 4G and 5G are very similar. 5G is an add-on effectively to 4G with things like, you know, MIMO and more MIMO antenna and certain other things. But one of the more interesting things is in the disaggregation of the components. And we'll get into that in a minute. But like you can see the components here, you know, organizations like ORAN have been working hard to try to open up the interfaces between these different components to truly make them open so they could be implemented by different vendors or implemented as different software components outside of a single device box, for example. So I think if we look at it from this angle, this is kind of another way to interpret that previous slide, but maybe looking at it almost from where would you deploy those components, that perspective. Yeah. And I think this is also, you know, this is showing some of the different options. Again, a lot of you have probably heard about how a lot of these functions are moving to the cloud, even, you know, 5G. We've seen, you know, you've heard about a lot of the deployments that are out there today, which are really on-premise at a telco that's deployed 4G. What's happening today is, as you can see in the picture, some of those functions are actually being worked on to be moved into the cloud, which is interesting, but also brings different challenges. That's right. That's deploying a single box again. Yeah, for sure. For sure. And I think if we start to think about it in terms of, you know, how we got to those previous pictures where you mentioned the word disaggregated a lot. Right? So, you know, how did we get here? And where did we start to get to this disaggregated multi-vendor world? Yeah. And again, this is very much like SDN, right? I mean, you used to look at, you know, a box from one particular router vendor, some you and I worked on. That's right. So there's others. But the point is, the stuff that was stored in that proprietary box and integrated together, I mean, there's all of those functions you saw in the previous slides are all integrated in that single box on the left. What we're doing today in the disaggregated mode is, and in open source mode as well, is putting that stuff on commodity servers using commodity NICs, running essentially commodity top or X switches even in that deployment. But what you see the difference here is more, right? There's four things on the right. Well, there's more than four things on the right. There's four racks worth of things on the right versus one thing on the left. And the question comes down to one of complexity, managing that complexity, as well as the costs, you know, which is directly proportional to that. So that's something we'll get into why EBPF actually is going to help this equation. Bring the complexity down. I think so. That's that's our thesis. Remember, I think we can do it. I think we'll figure it out. I think we'll get there. And we'll get there. Right. But, you know, but when we start to look at this as well, right, here's another way of maybe breaking this down. In terms of the different layers and where things might land right and where they're going to exist when you get to that deployment. Yeah, this is this is actually trying to show how that picture a couple slides ago was is now been exploded and what the options are to implement those things. And you can see there are multiple vendor options at each one of these layers, depending on, you know, not just implementing the kind of the straight up ran architecture, the 5G ran architecture, the 5G core architectures that you'll see in the 3GPP docs and whatnot. But even the hybrid cloud deployments that were on the last slide and that's the stuff I worked on. That's really interesting. But what you'll find in those deployments is that the complexity is is is enormous. It's manageable, but it's, it's not free. You know, as somebody famously said, you know, there's no free lunch here. So there's a, you know, as you can see, there's a lot of vendor options, but that needs to be managed that used to be kind of managed by a single vendor or single integrator anyway with a few vendors. And now, you know, like the deployment I worked on had two dozen vendors at the party, you know, and that was that's part of managing the complexity. Oh yeah, for sure. And speaking of complexity, I think this slide as well, you know, we've talked, you know, on our podcast, the net dot lol a few times about open source, you know, both in terms of foundations and individual projects and the explosive growth of open source. And I think this slide, just in the context of 5G and open source networking kind of shows that explosion. I mean, look at all these projects, you know, projects are supposed to work together. Yeah, exactly right above right so you can see my point about additional complexity. So this is just the software complexity. There's actually no hardware here. Which is interesting and that to me is like, like you see the little O-Ran box at the top. That is maybe where we can do some software like the RF radio hardware. That's kind of one of the last bastions of the proprietary NIS in this picture. One thing though I wanted to point out too on this slide is that there are a lot of projects and you and I have worked on some pretty massive projects in open source at LF and other places and Yep, there's I don't know a dozen of those projects on here and those projects were hard enough to manage on their own. Now you've got to make them all party together and that that's definitely a challenge in here as well. Yeah, and like you said, we've even before this, we've taken just a few of these individual components and tried to get them to work and yet that proved challenging. You know, but I think, you know, if we're thinking about it from a positive perspective, maybe the various communities and you know folks like ourselves, having had that experience in the past, maybe it gives us a little bit of an edge for some of this, right? And it's about, you know, managing managing the whole thing and I mean if you look, for example, one of the things that I've learned through my experience with these different projects is that the project is I consider the project a super set, right, what you would want to deploy. And like if you look down at the cloud infrastructure box down there with the different networking options that are there. Nobody says you have to implement all of those things to make one of these deployments work. In fact, what we're going to show you is, is you almost don't need any of those with BPF. That's right. And in fact, that's what we're going to talk about if we zoom in using my fancy pointer here on this cloud infrastructure. And we start to look at some of those infrastructure projects specifically on the networking side. I mean, some of these have been around for a while. I mean Docker obviously has but even open v-switch. I mean, open v-switch is over 10 years old now as well. Yeah, OVN, OVS, yeah. OVN, FDIO, all of these things are there, right? And they've obviously served a purpose. They've been successful as well. But, you know, maybe, can we look at something like maybe BPF and XDP to slot in here and can it help address some of these problems? Yeah, let's take a look at that. Let's take a look, right? So I thought I'd give a quick overview and for anyone new on the talk to what EBPF and XDP is. Essentially, EBPF lets you run sandbox programs inside your operating systems kernel. It's kind of amazing when you think about it. It lets you extend that operating systems kernel and loading some code down there in this sandbox environment to be able to enhance it as well. And even extending the NIC itself too, the NICS functionality. Oh yeah, you can do it at all. And I'll show you in a future slide where you can load the XDP programs, but it's kind of up and down the stat for sure, right? Yep. And I think the obvious reasons why this was kind of conceived and done was because of what maybe some people perceived as a slower rate of innovation in the kernel. The lower, the closer to the hardware you get, the longer the lead time is to roll things out. I mean, you know, operating system updates. I mean, even just think about yourself. How many times has your laptop told you you need to reboot to update and everyone says, no, I'm going to wait. I'm not going to do it now. At least twice today. At least twice today. Exactly, right? One of the other promises of this too is that is either reducing the overall cost of those servers in that picture I showed. That's right. 5G deployment by offloading the CPUs and letting the NICS do things, which NICS have cheaper, typically cheaper processing components on ARM CPUs and we're not today GPUs, but or it frees the CPUs up to do more. And that's really important in like these 5G deployments where I got to remind people that not all of those edge nodes are in a big. That's right. Facility. Sometimes they're in a fiberglass hut on the side of a road. So power and cooling in space really matter. And you can do some really interesting things with XDP, for example, about very precise control over which CPU is executing which, right? Because, you know, you might have a CPU, your XDP programs running right off of the NIC. You might want to save that CPU for the next couple of packets, but you can forward. You can redirect to a different CPU and continue to process that packet if it's something you need to do that's going to take a little while longer, for example. So there's some interesting things you can do in terms of that power space and cooling. Yeah, definitely. So let's take a look at what that is. Yeah, we can just jump through really quickly, you know, EBPF really has 3 main use cases, right? So obviously networking we've been talking a lot about, whether it's high performance networking or load balancing. You know, security as well. How can you implement security at the operating system layer with EBPF and then observability and tracing, which I think as we'll talk about in the future is pretty critical for these 5G deployments, I think. And it's one of the areas that we've talked about where I think it's going to be pretty important. And where it really is insidiously important, I would say is if you go back to those slides I showed, you know, the software stack that we talked about, you know, used to be controlled by a single vendor who had insight and introspection built in. That's great. Those things are a bunch of components that either come from upstream open source projects or from a vendor and integration and observability is non-linear. Exactly. Exactly. And so, you know, when we, when we think about EBPF, and by the way, this is, I've reinterpreted this slide, but it's linked below from a great blog post by our friends at Cloudflare as well. You know, who was recently as 2019. I've already made the claim that EBPF is going to eat the world and they're a great resource for EBPF. So I encourage everyone to go to their blog and they've got some great stuff on there. But if you, if you look at this, one thing that I wanted to talk about was, you know, you really can load EBPF programs everywhere. And so what I'll do is I'll start on the bottom, right? And we're going to talk about XDP in the next slide as well, but you can attach XDP programs, which are EBPF programs to the network device driver itself. You can load XT BPF programs and at the IP tables layer. You know, at the TCP and UDP socket layer, I mean, you can do some really interesting things there with TCP BPF, some socket dispatch programs and the exporter. You can also do, you can load things at SO attached BPF, which is at the socket layer. One of the really interesting ones, which we're not going to talk about in this particular talk, but which I've done a lot of exploration with is the sock map stuff. And that's, that's super amazing because sock map, EBPF sock map allows you to take two sockets, for example, and glue them together in the kernel, and then the kernel will just transparently proxy data across them. So there's some amazing. That's super cool because that's again an operational efficiency that you don't really need to mess around with it just set up. You do, you do the set things up in user space in terms of creating the sockets and then you just glue them together and, and away you go. So that there's some really amazing things you can do there. Yeah. And when we talk about XDP though, in particular, you know, XDP stands for express data path right and it's really a way to do high performance to build a high performance and programmable data path inside of Linux right it doesn't require any special hardware. It does require some driver modifications, but the majority of the drivers that you're going to see will have those modifications at this point. Exactly. You know, it doesn't require kernel bypass either. In fact, it's almost the antithesis of that right because what you can do with XDP is you can process that packet right off the Nick and make a decision, modify the packet redirect it or. And even after modifying it, you can just let it pass up into the Linux stack so you can still take advantage of all the other things the Linux stack is doing, like for example, the TCP IP stack, or things like this. So there's, you know, it really plays in concert with Linux rather than being opposed to the Linux kernel. It's kind of step aside from it like the the other the other approaches were, you know, avoid the kernel. This is work with the kernel. Right. Exactly work with the kernel take advantage of what the kernel can do, but allow you to build an extend on top of it. So now if we take a look at maybe what the current state of EBP fxtp programs and telco networking are. I thought it'd be interesting to look at some use cases right so here's an obvious use case. You can do with with EBP fxtp programs which is DDoS mitigation. Again, if you look at cloud flares blog, they actually talk a lot about how they use xdp for DDoS mitigation. And Tom, you know, as a 5g expert, I think you can probably attest to that this is an important thing as well. Well, that's where, you know, a lot of the, you know, the sort of broadband access has been replaced, you know, with mobile access. And this is a thing that is done in those networks. And you know, here's the other thing is that the mobile backhaul in my example of the fiberglass hut on the side of the road, the mobile backhaul is not always, you know, a high speed fiber network. Sometimes it's a iffy microwave connection, you know, and exactly the network in those cases. You know, to your point about power and cooling as well, the earlier you can drop that packet without using as much CPU time in terms of this DDoS mitigation, the better, right. And so I think it fits in or other resources to, yeah. Yeah, for sure. You know, load balancing also important because from not mistaken, you know, Kubernetes is into the 5g world and Well, Kubernetes is intrinsically, you know, requires a load balancer. That's right. And and so it's a it's an intrinsic part of containerized ran deployments. Obviously not with, you know, open stack based deployments with the ends, but still, you know, even those deployments that I've seen have load balancers for the obvious reasons. Oh, yeah, for sure. And you can do some really interesting load balancing. Well, in fact, we'll towards the end of the presentation, we'll go into detail about one of our favorites that everyone should know about using XDP. You know, high performance forwarding as well. I mean, clearly that's important. You can do that with XDP and with eBPF programs as well. And that's going to help your 5g deployment too. You know, flow sampling and monitoring. I think this is pretty critical. I mean, you've got this 5g deployment. And like you said earlier, maybe previously you had a single vendor that was giving you some of that. But now with this disaggregation, you need to build some of this. And you could build, you could build some of it to augment what you buy or you, you know, and, and again back to power cooling. Yeah, ease of operation, you know, less is more. And, you know, loading BPF programs that can just do the sampling on the neck and export the telemetry is when, you know, without everything else needing to be there. Exactly. Exactly. So I thought, you know, we mentioned this a little bit, you know, OVN as well. So let's, let's take a look and compare, you know, what an OVN architecture looks like versus what you might do with an eBPF. Yeah, this is the typical architecture that you would deploy OVN in. And back to the 5g example. You know, you, you know, the, the, the OVN nodes themselves in a typical network, there are tens of thousands of these. So you got to remember, I mean, you know, we had, we had a hard time making this work even in a data center where everything was land attached with, you know, high speed, reliable, you know, links. And now we go to these 5g deployments where, you know, like I mentioned there, there are either congested backhaul links that are always congested by design. But losing, losing messages in this architecture is tricky. It's very tricky. Let's call it tricky. Exactly. And then scalability, you know, because the nodes have to continually talk to the controller and the, and so on. And, you know, that's another scalability issue that, that, you know, we'll see in the, in the next slide. Yeah, yeah, let's, let's talk about this. I mean, this is kind of what we envision, you know, you might want to build something similar using XDP and eBPF programs. And I think the main difference, some of the main differences are you can use an industry standard KV store and message bus for this kind of whatever you want to build your controller for this. The other point is you've got these agents, which are stateless on the host as well, and they're using these XDP maps to program the fast path as well. And so you've to some extent you've decoupled things now a bit more, right? You've got the fast path running off of these math maps and you've got these agents which are responsible for populating the maps as well. So it's a pretty clean architectural approach. Yeah. And there's two keys in there, right? I mean, what have we learned through our whole careers and networking, right, working on the internet? What, what are the keys to scale is aggregation and statelessness. That's right. That's right. And that's what we're getting here. That's right. It's pretty good. Yeah. So, so let's see. So let's, you know, we're going to wind things down here in a little bit here. So let's talk about some key components for successfully deploying these EBPF and XDP programs. I think number one, I mean, as of everything, hire the right team, but there's keys to remember here, right? I mean, XDP is, you know, it's, it's, it's going to be, you got to have C programmers, right? I mean, a lot of this is all written in kernel code. I mean, yes, eventually there might be rust, but for now at C, you need some SREs. That's a really good point. You've got to empower that team. Yeah. And that's a really good point, Kyle, too, because if you look at the modern crop of coders, they may not necessarily know C. It's challenging, right? For sure. So I think that's one step, right? I mean, you got to ensure you have monitoring and observability for the whole solution. Pretty critical on anything, but even more so for this, because the other thing to remember, you know, your packets with XDP, you're not going to be able to look at them with things like, you know, TCP dump and things like that, because they're not going to go through the Linux. And not go to the Chrome. So build that upfront, you know, understand how upgrades will work with your programs, what happens when you detach XDP programs and attach new ones. This is actually gets back to the complexity point I showed at the beginning. Oh, yeah. Upgrades are very challenging in a disaggregated model where you have components from 12 different vendors that all need to work in harmony. Together. And follow. That was the end the other picture with the different open source projects. There's also that, you know, harmonization of scheduling. So when releases come out, they're all synchronized so that, you know, we can do upgrades in a uniform way and so on. That used to be handled by a single vendor. And now it's, now it's not right now it's not and it leads into this right state is your enemy as well. The more you can keep things stateless. The easier all of that's going to be and and that and and I think there's ways to do that and we'll show an example here of another program right so so charting the future of this what does the future look as we kind of wrap things up here a little bit right so how do we how do we see this going right. Well, I think we, we see this as ebp fnxdp allowing further disaggregation of things right decoupling more network functions into ebp fnxdp programs right and disaggregating those network functions even more running them closer to the hardware right scaling things thinking differently. In terms of, you know, how things work right and we kind of wanted to leave to leave you with this right here's a great example so anyone who's looked at Facebook. Catron as well. This is a great example of what we're talking about so Facebook designed this load balancer using xdp and ebp f and as you can see. And again, this is taken from one of their blog posts the link is on the bottom of the slide. It's really great because what they've been able to do is they have a control playing inside user space and xdp program that works in concert with the local kernel as well. And because it works in contact, you know, in concert with the local networking stack, you can run the application on the same note as the back end which is pretty slick as well. You can do direct server return again taking advantage of the Linux networking stack. It's a pretty neat way to do it. But you know what's cool here too is that this is, you know, harkens back to the control and data plane separation from SDN. You don't have to run them separately everybody thinks about running these things separately on a nick. The beauty of this is that you can actually start running all this on the CPU and migrate it out as you need to, or want to, you know, if you've got a small site, small site deployment or whatever that that's right only can run on on a, you know, on a small, very small machine that doesn't doesn't have nicks that are capable of this or not you could still run this on the CPU and it works. It works. Exactly. And so, you know, we'll wrap up with this again. You know, this was our thesis. And I think that we've, I think that we've answered this. I think that XDP and EPPF can be a part of the future of 5G networks. And I think it can help to simplify things. Yeah, I think we've shown, you know, you can clearly see the reduction in the complexity or reduction in the concerns or reduction, you know, from state, a lack of state. And also operational flexibility, like I was just saying, you know, you have a choice as an operator to where to run this, how to run it. And you can evolve into it. It's not a, you know, let's flip over the whole network into, you know, using this or that. And that is actually, you know, gets into some of the other options that are out there today. You know, they require you to change from NVGRE to VXLAN and things like that. I mean, this just works, you know, which is nice. It's definitely great. And so, you know, with that, I would just like to say thank you and, you know, definitely feel free to reach out to myself and Tom on Twitter if you want to continue the discussion. Yeah, thank you for listening. And I just want to invite everybody to listen to our podcast. And I think we have time for questions after the talk. That's right. So stay tuned for those and come with your questions. Thanks for listening.