 Well, hello everybody. Thank you for coming to KBE Insider and it seems like my camera has a little bit of a shake to it today but I'm Langdon White and I am formerly from Red Hat and I am currently now a faculty member at the at Boston University in the Computing and Data Sciences faculty where I teach data science stuff and I teach computing stuff Practicum and various other things We also have Josh Berkus who is co-hosting with me today and Are you there Josh? Howdy seems we have no video Yes, is video working? It tells me it's working, but I don't think we're turned on I think But we can do what I just keep running and we'll figure out how to get our pretty faces on the screen in a minute Anyway, I'm Josh Berkus. I work for Red Hat's open-source practice office and I am the Kubernetes person in our open-source practice office Making sure that Red Hat is doing a good job in contributing to Kubernetes and many related projects And As such I've actually talked to our guest today Liz Rice many times In the CNCF Awesome. Thanks Josh. And so yeah, we'd like to welcome Liz Rice Who is I've got a whole host of titles but maybe you could introduce yourself with a little bit of the teaser and Hopefully we'll get video soon kind of you know, otherwise, it's gonna turn into a podcast or what's what Twitter's new thing? Spaces is that what it's called? I Listen to one of the other day. It was actually pretty good on Developer evangelism or we don't call it that anymore. We call it developer advocacy. That's it. So Liz. Yeah, tell us a little about yourself Hi. Yeah, thanks for having me. Um, yeah, I don't know what's happening without the video But I guess we'll we'll work around it. I'll pretend that it's a podcast. Yeah Yes, so I'm speaking from North London Where I've been working at home for the last I can't remember how many years I'm now the chief open-source officer with ISO valent who are the company that originally created the psyllium project and I am Currently the chair of the technical oversight committee for the CNCF the cloud native computing foundation. So hi Awesome. Yeah, that's really cool Why don't we kind of start maybe with tell us a little bit about the psyllium project I think and maybe with a little bit of background and ebpf because I think that's Something that not a lot of people will have experience with especially Given that you have to have a very recent kernel relatively speaking To to get ebpf would that make a good place to start? Yeah, sure. So I think actually it's a very Timely question because I mean I've been Interested in ebpf for well, I think I first saw Thomas graph presenting on it at I don't know 2017 2018 and I thought well, that's really cool But clearly it needs a new kernel and none of us have those yet and you know, this is gonna take a while and Fast-forward to you know 2022 now apparently I can't remember to write 2022 or not and the Kernel support for ebpf has become much much more widespread the Kernels that almost everyone is using in production these days now support enough ebpf functionality that we can do useful things with it and like running psyllium and other ebpf projects so That's probably a Really really core reason why over the last let's say six months or so maybe a year You'll have seen ebpf just kind of explode in popularity because we can Just for a little bit of background so ebpf is kind of like a sandboxing tack for various functionality in the kernel And why do people care? Well because that way you can kind of host You know quote-unquote dangerous things right without having to worry about you know giving them kernel access except not giving them kernel access And so I think psyllium though is specifically in the kind of the networking space, right? Yeah, yeah, I'll expand a little bit about ebpf as a technology So what it lets you do is run custom programs in the kernel within the kernel So the sandboxing acts aspect ensures that those programs are safe to run for example That if you're going to access memory, it's memory that belongs to the process that's currently sort of involved or that you can't Crash you have to check that pointers aren't null before you de-reference them because you would be a really dangerous thing to Why if you crash the kernel you your machine is gone. It's died and So ebpf Sandboxing or the verification process makes sure that These programs are safe to run, but you can actually use it to customize the way the kernel behaves With you know restrictions on what you can and can't do with that. It's it's pretty complicated, but you can actually hook into for example any Entry or exit point to any function in the kernel or in user space You can hook into any trace points and for psyllium particularly You can hook into Various points in the networking stack So psyllium is it's best known as being a kubernetes CNI a kubernetes networking plug-in You can use it outside of the kubernetes context as well but what it allows us to do is create Networking endpoints for kubernetes pods and connect them together in a really really efficient way People quite often talk about how we can bypass IP tables. We can bypass a whole load of unnecessary networking functionality essentially in the kernel by Kind of hooking directly from pods to networking interface or from pod to pod Gotcha. Yeah, that's that's pretty cool I mean, I think one of the things that I was really interested to learn. I guess I was really surprised to learn Year or two ago is like how root has kind of been broken apart actually into like a bunch of different functions and So that's You know, it's kind of in the same vein You know, it's like, you know, it's not just a superpower user kind of anymore And you know and so you can now actually isolate, you know various processes from each other And that has a lot of ramifications for you know things like kubernetes You know and other even even virtual machines or whatever it allows you to Kind of give certain kinds of access but not carte blanche to Processes that you may or may not trust or kind of at least in my experience, right? I've written enough bugs in my life that you know, I don't even trust my own stuff, right? I'd like to be able to isolate it from each other So yeah, that's pretty cool. I think that that tech is is getting really interesting I'm saying in one of my roles I'm actually one of the reviewers for submissions to kubcon And kubcon Europe is coming up and I'll say we certainly have quite a few talk submitters who Think ebpf is what we should all be paying attention to The a lot of a lot of submissions with core, you know sort of either ebpf or I did this thing with ebpf Ideas behind them The ones that are I did this thing with ebpf tend to get rated higher Since I think at least within the kubcon community. We kind of know what it is Yeah, even even like, you know, it's a it's a much smaller conference, right? But I remember seeing stuff like this for devconf which is kind of the technical red hat conferences that we do that are Kind of very open source oriented, but even there we saw ebpf stuff kind of landing So I think you know, I think you're right on I think there's a lot going on With that tech and you know being this being the kind of insider show You know, it's it'd be a good idea if you're not already kind of tuned into what's going on in that community It'd probably a good idea to start paying attention to it Because it really does seem to be the wave of what we want to be doing You know for for lots of different kinds of functionality, you know, Celia miss is one exemplar but not necessarily alone or definitely not alone, I guess Well, since since we have the container security expert Here on on the broadcast with us the You know, I want to go straight into actually kind of the security questions, which is okay if we You know say project to the near future where we've kind of completed at least the current planned ebpf feature set The And people using ebpf plus Kubernetes plus, you know Container runtime, etc How close is that to the level of protection? We would expect out of a virtual machine platform in terms of isolation of processes So I Think all the time we're talking about Containers we have to be very aware that we're we're sharing a kernel all the containers running on a given host have one kernel and that kernel has visibility and access into Everything all of the processes running in all of those containers on that host now. This is where ebpf becomes really powerful because if you run an ebpf program in the kernel it has visibility and Potentially the ability to change behavior for every single process on that machine and that includes every single container in every single pod so, you know, there's a lot of power here and you know You do have to treat that power with respect. You have to treat ebpf programs the way you would treat anything else that you would run essentially as root But we can use those that We can use ebpf to create Stronger Let's say security profiles. I don't really want to say security boundary, but We can lock down more efficiently what any given application can do and this is something that I think people have been talking about for quite a long time and the idea that if you have a an application and it's only supposed to Contact certain network endpoints and it's only supposed to Read certain files, you know, maybe it isn't ever supposed to contact a payment gateway There are all sorts of things we can do with networking policy to limit what Any given containerized applicator well any given pod can Can do in terms of network activity But with ebpf we can take that even further because we've got this very very granular Visibility into what's happening So we're going to be able to say not just okay, you know, this pods has you know, it's trying to Contact this particular IP address we can say it's actually trying to contact this particular domain name and the process that Initiated this request was this given executable name. It was started at this time Here's the whole process and the tree of that Connect request for example and That's going to allow us to be very very specific about what What's permitted in our system? So for example, you know, how often do we really need to run curl in a production environment? Seems like it When when I'm doing things correctly So, you know, let's let's assume we're not ssh'ing into our pods and running curl it would be pretty cool if we were able to write some Network or some security profiles that say We don't expect you to run that curl executable or if we do we only expect it to be run to Access this particular endpoint from which this particular Application, you know has to download some information We can be very very specific That's the power and the promise of ebpf from a security point of view The fun bit is going to be creating all the correct profiles and making it easy to get those profiles, right? Yeah, exactly the you know, that's always it's always the challenge I I also like to bring up, you know, I read a lot of spy novels and You know in talking about security layers, right, you know, so in the tech world, you know I think most people will even have a You know kind of glancing understanding of security recognize that, you know, having layers of security is usually The right answer, right? But that's also true if you're you know a spy and you're trying to you know Break into someplace or whatever and so I think it's it's also really interesting, right because it's it's it's adding another kind of opportunity For a layer of security, but you know to your point, right? It does it does require us to really think about the security profiles and and not only Not only making them work correctly, but then also making them easy to develop and all that kind of thing, right? I mean, I think the se linux adoption went way up as soon as you know They started to have nice tools that could tell you how to how to fix what's broken because your Se linux profile is blocking something, right? But now you can get really good information about you know, why is being blocked? You know, is it intentional and here's an easy way to to change it if That is something you do want to be able to do for whatever reason, you know Sometimes you do want to be able to call curl for example So I think we would like to move on to maybe talking about some CNCF stuff However in an attempt to turn this from a podcast into a twitch stream, I think we're gonna try to restart the stream so Please bear with us for a few minutes while we you know, ask restream to restart and I'm also gonna throw this in the chat because of the delay on the audio but Does that make sense? Yep, we are we are restarting everybody in an attempt to get video exactly, okay