 All right, welcome back from lunch. Hope everybody had a great lunch. I know, like I said, this is the time where the food coma starts to set in, so I'll try to keep you awake. And that's why I was wondering if we can up the lights a little bit on this side of the room, because, you know, so I don't fall asleep later. So all right, so we're going to talk about the cross-platform future of EBPF. And I'm going to try to keep this part at a relatively high level kind of vision. And I work at Microsoft in the part of Microsoft that owns the core operating system. So we own the EBPF stack for both Linux and Windows. And across Microsoft, there's many different types of applications and things that are represented in other speakers and so on that kind of use EBPF under the covers. So I'm the group that owns the core platform between Linux and Windows. So I'm going to talk about the cross-platform future of EBPF. And since this is right after lunch, just to make sure we're all still awake, I'm going to start off with a question. One question quiz. Now they say if you're going to give like a keynote style presentation or whatever, you should start off with maybe a provocative statement or provocative question. So this is it, right? So here's my quiz to see if we're all awake now. Whether you know anything about EBPF or not, you can still vote or whatever. Here is the quiz. One question quiz. True or false? There's only two possibilities. It's multiple choice. EBPF is a Linux technology that can run send box programs to extend the Linux kernel. Today is this question, true or false? If you think it's true, raise your hand. If you think it's false, raise your hand. Well, you're all right because it depends on what you mean. And so we're going to talk about that. But I'm going to argue that this is already false. So we said why would this already be false? Because obviously the talk is about the future, right? But I'm going to back up and talk about the past briefly here. So Linux technology to extend the Linux kernel. I'm going to claim that that is already false. Let's look at some history here. 2014, that's when EBPF kind of came in to augment the place the classic BPF. So when we say EBPF as we know it today started in 2014, you could argue that the sentence was certainly true then. Now as we'll see, the extent to which you think it's true or false depends on what you're talking about, what's used mass in production versus what's available in projects and things. And that's why I said you're all right. So back in 2014, that sentence was certainly true. But even only a year later, the user mode BPF project came in. And that one said, well, I want to allow EBPF programs to be run in user space. Maybe I want to use them to instrument or to augment a user space demon. Any other type of use case for running EBPF programs themselves up in user space was where the EBPF came in. So even by 2014, the last phrase of that sentence is already starting to become false. 2017 was a big year. So a couple of things came in in 2017. We had the Rust BPF project that said, OK, we're going to run things as a Rust crate. And not only user mode for user space for Linux. By the way, I use user mode, user space, depending on which ecosystem you're in. Use the same parts. Use different terms for the same thing. If I mix and match, it's because I'm thinking cross-plat. If I'm thinking Linux, I call it one term. If I'm thinking Windows, I think of another term and so on. So with Rust, it already done work to make it work and say user mode for macOS. It's just a Rust crate, right? Later in 2017, you had to release the first release of Android, which is, of course, based on Linux. So that part wasn't hard. And then you had the generic EBPF project. The generic EBPF project was the first one that really started bringing stuff cross-platform back in 2017. So it tried to target many of the previous ones, as well as FreeBSD, both kernel and user. 2018, Rust crates. OK, now there's some initial work in saying I want those Rust crates to also be able to work on top of Windows. So now, user space in Windows. And in the beginning of this year, we at Microsoft announced the EBPF for Windows project that says, OK, we want to allow running EBPF programs in Windows kernel mode. The latest one in this long line of history here. And then finally, just a couple of weeks ago, co-located with the open source summit was the Embedded Linux conference. And that one, there was a presentation about bringing EBPF programs to an embedded Linux related system. And the OS there was called the TOC OS. And so the TOC project was the one there. And so that was, again, just this year. So all these different cases, EBPF, ever since 2015, has already been either changing whether it's in the kernel or whether it's Linux only. So it has this long track record. And of course, if you think about the spectrum that says, all the way from the science fair to in full blown production use, all these projects are kind of in different places on that spectrum. We're clearly the Linux kernel is in the production use case. Other ones are somewhere else along that line of the spectrum. And how do we get them to move closer to the production stuff over time? As EBPF expands and is in production use and more and more of these types of systems. The last piece has been talked about a couple times during the conference already. And that is hardware offloads. Once you can take a program and run it down inside the network interface card, as Liz and others talked about, well, it's not running in the kernel anymore. And so that's yet another platform. And that is in production use. And so between user space, kernel space, Linux, non-Linux, the main CPU, alternate CPUs, network cards, all these different places that can host EBPF programs. And that is why I claim that that statement was false even today, because you have production use, even in Nix, if you think about the production use case. So here's the revised version that I would say is my vision. And I hope a vision that we all share, which is EBPF is a cross-platform technology that can run sandbox programs to extend a privileged system component. Now, that privileged system component, it could be the kernel. It could be a network interface card. It could be a user space demon that's a privileged demon. Or it could be anything else, a separate processor, whatever it is. So this is the future vision that I hope we all share. So how do we make this become a tool that we can use in as many of these environments? Because it's a powerful tool. There's exciting things you can do with EBPF. The same exciting things you can do with a Linux kernel, people want that for the other cases, too. In the Windows world, we had people asking us, for Windows features, that EBPF already provides. Do we invent something different for Windows? Or can we just take EBPF and make that common technology be cross-platform? You can tell from the project that we announced what we think is the right direction. So if this is the cross-platform vision, there's some challenges. One of those being usability. So we said, what is the usability vision that's necessary to enable the cross-platform vision here? Two pieces. The first piece is to make sure that it's easy to build a cross-platform EBPF application. There's many people that build applications or solutions or products that are based on EBPF, typically built for Linux. And think about, what could you do if your product or your application could be run on other platforms, too? Suddenly a much broader user base, a much broader applicability for the same product. How do we make that be easy? Today, is it easy? It's probably not easy today. How do we make it become easy? That's the vision. The vision is that where we want to go is that it becomes easy to build things that are cross-platform. As easy it is to say, build a Kubernetes plot and deploy it on a bunch of different systems. To go along with that, how do I deploy those things? I want to be able to use the same tool sets, the same things that people are already familiar with, to deploy and manage those EBPF programs across platforms. The same time of promise that we often have in cloud native. These two things will take a lot of work to get there. There's already great starts on these that are already out there and projects that are there. There's just more ways to go to get to the grand vision where it's all easy. So we said, OK, if these are the things that we want to get to, what are some of the attributes that it takes to foster the success of getting here? The more attributes are present, the faster the transition will happen to be cross-platform and easy and same tool sets across platforms. And the fewer of the attributes that I'm going to show you, then the harder it is. It's not that you won't get there. It'll take longer. So let's look at some of the attributes and we'll talk about some of these. So here's my list of good building block attributes. And I'll talk in the next slide about what the building blocks are. Let me talk about what makes for a good building block, a good piece of technology that goes along with EBPF. Now, again, not all of these are necessary in all projects, but all of these are success contributors. They're all things that are synergistic with success. The first one is, of course, security. Because EBPF, you're extending a privileged system component. That's kind of table stakes, right? Now, you have the classic security things you might think of, but there's some other parts of security that kind of vary between projects and that can help. So for example, do you have the building block built in a type safe language? And that's where you saw things like the Rust BPF project come in. This is, OK, can I write all these things in Rust? Because then my core implementation of that building block becomes safer. That's one example. Do I do testing in CI-CD? Well, do I do fuzz testing for each of these different components? All these things that fit into the security attribute of a particular building block. Open source, well, we kind of all understand that when it's easier to use stuff across platform, when it's open source, easier for people to port and so on. But not just open source, but if we look back at that matrix that I showed you of all those different projects, there's at least three or four of them that came in that were brand new implementations of something because the preexisting one did not use a permissive license. So we found that permissive licenses are far more open to cross projects and to facilitate cross-platform work than restrictive license projects. So again, some of them vary by license. Documentation. There is some great documentation out there, but often people will list that as a challenge in learning eBPF. Well, it wasn't well documented. Or the documentation is you've got to go and read the Linux source code. Well, that's a barrier if you're trying to do development for one of those other operating systems, whether it's Mac OS or FreeBSD or Windows or what have you, to say, well, you just have to go and read the Linux source code as opposed to having well-defined documentation. So compiler agility. We've already seen this with can you use CLANG? Can you use CLANG? Can you use GCC? Some compilers are meant for specific OSes, especially when you get down into embedded and other variations that we'd like to see eBPF go to. Obviously, there's the platform agility. Is it written just for Linux? Or can it be agnostic like generic eBPF was my example there that was nicely platform agile. Execution context agility. What I mean by that is it meant to run a particular building block. Is it meant to run in the kernel? Is it meant to run in user space? Or is it a static library that you can compile and run either way? As we'll see, there are some projects that are easy to do, no matter whether it's kernel or a user, easy to move around. And that means it's easy at building blocks that be used in multiple different assemblies that I'll show you in a bit. And then finally, I'm having the source tree separate from the platform tree. And I'll give you an example or two there. Well, I shouldn't have to compile Linux to do the Windows version or compile FreeBSD to do the Mac OS version or whatever. Or even grab sources out of it if there's a better way to do it. Those are some of the key attributes. So this is a list of the typical building blocks that make up an EBPF runtime solution. So you have a compiler. You have something that can load EBPF programs. It's a library that's used by apps. You have a verifier. You have either an interpreter or a JIT compiler, and often both, meaning the choice of them. You have an EBPF program execution context. This is where the EBPF programs run and exposes general functionality, like the ability to have maps and so on. It's kind of what I put into the execution context bucket. And then you have things that are instrumenting other system components, right? All those hooks in Brendan's diagram. Well, I'm instrumenting the scheduler or the networking stack or whatever that are really platform-specific hooks and helpers often, because some of those might be cross-platforms. Some of those might be inherently specific to a single platform, right? And so today, when you look across the attributes, what you'll find is that I mentioned in that matrix there, all those different projects, for every single one of those building blocks, there exist multiple implementations already. You can see a list of them there. So that comes with both opportunities and challenges. There's benefits to that, right? Which is that, of course, it gets to be used across platform and all these different scenarios and contexts, and the challenges we'll talk about briefly. But the point is, this is kind of a set of building blocks that you can build assemblies out of to craft a solution. And we'll see different ways of doing that over time. Now, the reason that there are multiple of those is, again, because those different implementations of the same building block will vary by, say, what language were they written in? Were they written in C or Rust or something else? Were they written for a specific platform? Were they written to be, well, that one was meant to be kernel and this one was meant to be user. And do they vary by licensing? And so across all of those, all these different attributes vary, and that's why we have multiple implementations of everything. So this is the classic assembly in Linux, okay? Where your compiler tool chains, your loader libraries are all in user space, but the verifier, JIT compiler, assembler, and execution context are all in kernel, okay? That's the classic one that EVPF has had for years. But that doesn't mean that the other systems that I showed have exactly that same assembly, okay? Here's the one that we have in EVPF for Windows. So a couple of things to note here, right? The compilers and loader libraries are still there in user end, okay? And here you see things like the execution context and the interpreter are still in the kernel. But there's a couple of things that are differences here, and this is to illustrate that there's many ways to assemble the same building blocks, right? If I showed you the diagram for any of the other projects, it might look like a different assembly, you know, the talk version or the generic EVPF and so on. But this one illustrates just by doing the spot, the differences between these two slides, it illustrates a couple of key points. They get to what it is that kind of enables the cross-platform future. So I'll talk about a couple of things highlighted here. So first, probably the most prominently visible is in the top right, okay? Where the verifier and the JIT compiler are not in the kernel. Here they're in another protected address space that's not the normal user-land space, okay? It is, in fact, in user-land, but this could be in a secure VM isolated from the regular system. It could be inside of something like an SGX enclave. The point is it's a separate trusted address space, but it's capable of running user-land code, okay? But not in your normal user-land code, okay? So we've experimented with things like SGX enclave and with a VM, and it could even be on another machine. So if you think about the discussion and trends, you've heard about people talking about maybe signed EVPF programs, right? So one of the visions is that part of the secure environment could even be on a completely separate machine. Maybe it's in your back room or something like that, generating the signed programs, and you only have a checker in the kernel, okay? And so this notion that says, where is the stuff running? Sometimes in the future it may even be on another machine, right? So that's the promise of the signed programs, okay? So that's one concept here. Another concept in here is in the top middle, where you see the libbpf, and I'll come back to this on the next one, you see BPF tool and libbpf, right? Our goal is that those are exactly the same libraries and tools as you have on Linux, or on any other operating system. And as I'll mention in a bit, there's some challenges there that says work actively in progress. Another thing to keep in mind is the key that's in the bottom left, okay? The key that's in the bottom left says this project is called EVPF for Windows, not EVPF in Windows, because all the EVPF parts here are not in the Windows product. Now the reason for that is because we wanted to work with already shipped versions. We don't want you to have to wait for the next Windows kernel, okay? These are things that are already used, so if you see in the center middle, that EVPF shim there, okay? The left side of that uses a stable kernel ABI on the left side, such that it can work with many different versions of Windows on the left. And the EVPF shim and the execution context distinguish between the hooks and helpers that are inherently platform specific, that's that EVPF shim box, versus the things that are really cross-platform agnostic, like the ability to support maps, right? Which is in the execution context, right? So the point is there's different drivers in the Windows sense, kernel modules in the Linux sense, okay? That separate where the hooks and helpers are and the execution context allows for multiple of these things to be plugged in over time. So anybody could write another driver or kernel module if I were to use Linux speak to add other hooks and helpers and add them into an existing runtime system without having to rev the kernel, okay? And so that's kind of the, the point is to be able to assemble building blocks. And so we've tried to make this picture here be an illustration of decomposing things. And I could have used other pictures from other projects, but this is the one that we did. And so I get to talk about this one. And I have to answer questions here. And I think the last thing that I was gonna mention here, I think I'll get to another slide. I'll come back to it. All right, so some challenges and some opportunities. That this presents. When I say this, I mean all the previous slides. So there's a bunch of heterogeneity, okay? There's lots of implementations of a verifier. There's implementations of an interpreter and so on. How do we deal with this case when there's multiple implementations? And then you vary by features, right? Somebody adds a feature to one and then somebody adds the same feature to the other one. What is EBPF? Is this specific set of features? Well, if you add a feature to one, does that change the definition of EBPF? Those are some of the questions to wrestle with. What is the definition of EBPF? Is there a definition of EBPF? Is it just the instruction set that's the VM? That's one definition. So that's the challenge. There's some opportunities there for us. And this is the future where we get to work together to make this future happen. First, our opportunity is to drive that feature consistency across those implementations as much as we can, right? To get that coordination. If somebody adds a feature into one of the implementations, how do we make that feature be common across implementations, right? Number two, I think this one came up in the EBPF somewhat, if I remember right. So it's to create common test suites, right? Do we actually have stuff be tested in CICD? If you have multiple implementations, can you use the same test suite? Think about a verifier. If I got multiple verifiers, can I have a set of sample EBPF programs that the verifiers all use the same set to test verification with, is an example? Same things for the other ones, for an interpreter or whatever else. Number three, we just created the EBPF foundation, okay? The EBPF foundation, one of the goals of the EBPF foundation is to help evangelize what is EBPF? What is that common feature set? How do we get coordination across projects, okay? So that we don't have just silos or ships in the night happening and different people, different projects implementing different things without coordination. How do we get that coordination? Kind of the central focus of that coordination is the EBPF foundation, okay? And so we would love to have more people join the EBPF foundation and to help with the coordination, collaboration. If you have a project, want to share across other projects and stuff, this is a great place to host things, to host that discussion. And of course, not just to facilitate coordination on the feature set, but if you think about that set of attributes, right? Sometimes there may be works to say, how do I improve one of the attributes of that? So for example, I want to improve security or I want to improve documentation, right? As I mentioned before, improving documentation of things actually helps all the other systems, right? Whether they're using that building block to create an assembly or whether somebody else has an alternate implementation in Rust or in some other language or for another platform or in user space instead of kernel, okay? The better documentation, the easier it is to get that consistency. Second challenge is that a number of the projects that exist today, a number of the tools, and I said I was going to come back to libbpf and bpf tools and examples here. Several of those projects are still very tightly coupled to the Linux kernel. So for example, today, if you look at bpf tool or libbpf sources, you'll find intermixed Linux kernel specific code and generic code that is really not specific to Linux, all intermixed, okay? That is a problem for taking those to be cross-platform. Meaning it's a problem that's not insurmountable, it's just work, okay? When you take those things and easily factor them such that the code that is can be used across platform is cleanly separated from the code that's really specific to the Linux kernel. Why? So the code that's specific to the Linux kernel can easily be swapped out and replaced with something on a different platform. So for example, libbpf takes care of forming syscalls. So the bpf syscall on Linux is potentially different from the bpf syscall on other platforms, whether that's Windows or talkOS or macOS or whatever else. And so the parts of libbpf to take care of loading might be different from the parts that do the syscall. Okay, and you might wanna swap out the parts that do the syscall. All right, so there's a couple of opportunities there which is to separate as I was just talking about. Again, to improve the documentation to make that be easier. And the hooks and helpers I mentioned in the Windows picture that some of those were generic things like working with maps versus very platform-specific things like cooking the NTFS file system or something in Windows is to make those be cleanly separated. All right, so those are some of the major challenges and opportunities that we look forward to working on today. So let me wrap up with three key takeaways. Number one, Linux, I'm sorry, ebpf is not just for the Linux kernel anymore. And you can argue it hasn't been for a long time, especially since we started having SmartNix and so on. It's been since 2015, it's already been on the road to being not true. So it's not just for the Linux kernel anymore. We'd love it to be as many places as possible. Number two, the community is already enabling an ecosystem of cross-platform apps and products, right? If you're developing code, we want it to be easy for you to develop code that is not Linux only. You might be a Linux expert, right? But can you develop code that will work on these other platforms too, right? Can you develop code that will make it work attached to say a Linux user space demon, not just the kernel, okay? To make it be possibly easy to create products or applications or demos or whatever that are already cross-platform. And number three, there's lots of interest. There's people joining the ebpf foundation. So the takeaway to me is the future of ebpf is bright. This is an exciting time to be here working on ebpf because this is where it will really blow and go across the industry, not only to the Linux kernel where it's already huge, but also all to the rest of the ecosystem. And so with that, I'd like to thank you for being here, taking time after lunch, and I'm happy to take any questions. Yeah, so the question was specifically about, I don't know if the remote people can hear that, so I'll just repeat the question. Once you have hooks and helpers and things that it would like SK Bob's, which might be inherently Linux specific, how do you deal with that because you'd have to rewrite the application to use something else, okay? So there's a couple of different, you're right. I mean, there's a couple of different variations there that says, well, sockets themselves, right? The BSD sockets model exists on almost every platform that's out there, okay? Now, as you mentioned, the data structure itself may vary, okay? Just say, what is it on BSD versus Linux versus Windows versus something else? And so SK Buffs is a very Linux specific way of doing it, okay? So you have two choices. You could either say, I want to have if-defs and things like you're talking about where I have a different port for each of those is different things. I could create a new hooker, new helper that uses a generic version, or I could find a way to shim SK Buffs on another thing and say, I want to copy it into that structure. Those are really the three choices that you have, okay? And those are the same three choices that we have on Windows, the same ones you have on BSD or whatever. Which one do we collectively want to do? That's something that I think we want more input from the community to know what's the right way, but those are really the only three choices, right? Now, fortunately, most of the hooks and helpers that are out there can be done cross-platform, okay? Now, meaning all three of those approaches can work, like the shim approach wouldn't work if there's nothing to map to, right? So, but yeah, those are the types of things we actually struggle with and saying, how do we bring it to Windows or in the generic BPF case, how do we bring it to BSD and so on, is exactly those questions, so you're spot on.