 Hey everyone, I think I'm live. Thanks for having me for this maintainer Q&A. My name is Matt Klein, a software engineer at Lyft. I'm still trying to figure out this meeting platform. I don't think I can actually interact with any of you directly, but there's a live Q&A tab on your screen. And if you would like to start asking questions, I can go ahead and answer them for you. Or I will also join the Slack room if we want to ask questions that way. So please start firing away and I will monitor the live Q&A. And also, if it isn't already obvious, this is a maintainer Q&A for Envoy Proxy. Looks like we don't have any questions yet. All right. Looks like we are starting to get some questions. Fantastic. First question is, can you explain the role of WebAssembly in Envoy? So for those that don't know, WebAssembly is an execution framework, you know, where you can take programs that are written in multiple different languages, such as Rust and C++ and Go and TypeScript and things like that. And you can compile them down to a common runtime. And then you can execute them in a safe sandbox. And, you know, it's a really exciting technology in that it allows us to extend things like Envoy in a set of programming languages that people are comfortable with. Whereas historically, the only way to extend Envoy has been with either writing C++ extensions, which has a fairly high learning curve and also obviously has inherent safety issues. And then the other extension option that we've offered has been Lua scripting. And Lua scripting has worked out very well, but again, not everyone likes to program in Lua. And, you know, we don't have extension points for all of the different ways that you can currently extend Envoy and C++. So we really view WebAssembly as the future of Envoy extensibility. And, you know, we view that for, again, a couple of different reasons. One of them is that will allow people to write extensions in different languages that they're comfortable in. So if people want to write in TinyGo, they can do that. If they want to write in Rust, they can do that. If they want to write in Safe C++, they can do that. And then over time, I would expect us to allow WebAssembly to offer extensions in all of the places today that C++ can be used to extend Envoy, which is, you know, again, at this point probably 10 or 15 or 20 different extension points. So I think it's going to take us two or three years before we get there. But I am really extremely excited for WebAssembly as a technology that will allow people to dynamically extend Envoy. And also when associated with other things that we've been working on around dynamic configuration of filters and extensions, I think there's also really interesting possibilities of allowing WebAssembly code to be dynamically distributed to proxies. So the use case that comes to mind is think of something like a web application firewall or WAF filter, where maybe we want to, you know, send rules dynamically down to the proxy, and maybe those rules are written in WebAssembly and they're compiled in a server off host, and then they are dynamically sent down to Envoy. So hopefully that answers your question. But, you know, I think it's a very, very exciting technology that's going to take us a couple of different years, sorry, a couple of years to actually see the complete fruit stuff. Looking, let's see. Next question is I'd be interested in an update on UDP support in Envoy. So UDP support is, I would say it's fairly robust at this point. We offer, obviously, UDP listeners. We offer UDP proxy. And within the UDP proxy, we offer various different hashing mechanisms. So I, you know, I would definitely encourage people to go through and look at that support. And if it doesn't meet your, what I would call raw UDP use cases, definitely let us know. Obviously UDP itself is also used for quick and HTTP3. So there's an increasing amount of effort going into UDP robustness. And I actually expect us to have a public alpha support of quick HTTP3 support in Envoy, probably in the next couple of days. So that is quite, quite exciting. But again, I think UDP support is ready to go. And I would encourage you to take a look and let us know if there are any issues with the current support. Okay, let's see. Next question. Do you think we will see OIDC support directly in Envoy at some point for using an external OIDC IDP and configure that directly in Envoy? To be perfectly honest with all of you, I am not an authentication and authorization expert. I do know, you know, that we have an increasing number of filters that are operating in this space around things like OAuth and then obviously we've had the external authorization filter for quite some time. I think the big challenge that we have within Envoy is obviously trying to keep it somewhat generic and not assume any particular backend server implementation. So, you know, I don't think I'm inherently opposed to additional support in this area, but it's definitely not my area of expertise. So I would definitely encourage you to open issues on this and we can discuss further. I'm trying to see how I can see the questions that haven't been answered. Okay, let's see. Next question. Hello, Matt, your deep dive on Envoy internals at Coupan EU Copenhagen is still to date my favorite tech doc. Thank you. To such talks help onboarding contributors to Envoy. I think when Envoy was starting out, I definitely viewed it as very important to, you know, help educate people on Envoy internals. And so I think doing talks and writing blog posts and obviously answering questions and talking to folks within the community has been a really, you know, important thing that has allowed Envoy to become what it is today. Sadly, I don't have as much time as I used to so between responsibilities for the project and having two young kids frankly. I just don't personally have as much time so I would obviously love to be giving more deep dive internal talks and I would encourage more folks to give those talks or write blog posts or do whatever makes the most sense for them. But I just don't have the time right now so. Yes, I think those types of talks and blog posts really help the project. And one of the pitches that I would say to folks is, you know, it's counterintuitive but I think most people, you know, think that it's tough to get people to work on a project like Envoy in terms of the low level code. And the honest reality is that it's actually not that hard to find people to write low level C++ code. What's really hard is to find people who care about documentation and education and examples and all these kinds of things. What's interesting to all of you is that from a contributor standpoint, we really don't actually need more people to write C++ code. We need more people to help with, you know, these types of deep dives and giving talks and writing blog posts and working on documentation. So I would certainly encourage anyone listening who is interested in contributing in those ways to reach out to us or myself. Thank you for your question. Could you give a quick status update on Windows support. I am not in the day to day of Windows and I actually tried to have one of the maintainers of Windows join this chat but unfortunately the platform was locked and we could not add him, but I believe we are pretty close to offering a Windows GA and, you know, that won't be 100% feature parity with Linux, but it will be close and it should be usable for Windows production workloads and I know that there's various organizations that are wanting to use it in this chat. So I think we're pretty close and I would definitely look for a blog post coming out in the next couple of weeks that will announce that and then you can give it a shot. Just make sure that I have gone through all of the questions that we have so far. I'm just going to delete the ones that I've answered. Please fire away. I have another question. With the rise of rust usage in low level stacks and the improvements to the networking libraries. Do you ever think I'm boy will have some rust in it. I think absolutely yes, I, I think we're already starting to see rust within WebAssembly. So I think my impression is that rust is the most common language that people are using to write WebAssembly extensions. So I think that's where we are going to see it start. And like all of the recent announcements of bringing rust to the Linux kernel. And then rust other projects such as such as Chromium. I fully expect that we will, you know, that we will see rust within Envoy. You know, I think it's a very exciting path that that language is taken. And, you know, I'm no, I'm no huge supporter of C++ myself so if there's a better alternative, I think that would be great. And like all of these other projects we have hundreds of thousands of complicated lines of code and it's not recently possible to rewrite everything. So I think we're going to be taking a pragmatic approach. And I think the bigger complexity for Envoy versus something like Linux is rust and C interop is a bit easier than rust and C++ interop. And I don't have a link handy, but I think that the folks over at Chromium have, you know, done some research and there's some really great projects within the rust community on C++ and rust interop. And I would expect that that's an area that's only going to get better over the next few years because there's obviously hundreds of millions, billions, I don't know, maybe even a trillion lines of C++ code out there. We're not going to rewrite it all at once. We're going to have to take a pragmatic approach. And yes, I would definitely expect to see rust within Envoy. Any other questions, don't be shy. Happy to also take questions in the main two KubeCon maintainer Slack room, if that's easier. Okay, let's see. Next question. What is the biggest thing you would want to improve in Envoy right now if you had all the time you wanted? That's a tough one. I think for me, it would probably still be around documentation and onboarding, you know, I think we still do a great job. You know, around the code and the features, the feature velocity continues to be huge. You know, there's always small improvements that we can do in code quality or, you know, in testing coverage or in performance or all of those things. But, you know, we have so many extremely talented contributors that work on that stuff that it's not a huge concern of mine. But I, you know, I think in terms of things that we can improve, it really is that onboarding experience. And what I do mention to people, which I find very interesting is that I'm very frequently told with Envoy, you know, two different things. I'm told on one hand that the project has amazing documentation and it's fantastic. And then I'm told at the same time that, you know, the project has the worst documentation ever and I don't understand how anyone can possibly use this thing. And, you know, the take away from that for me is that it's obviously we have a lot of different types of consumers and it's very difficult to have a one size fits all approach to project documentation. But one of the things that I think that we can do really well is to improve that onboarding experience and we have made a lot of strides here recently with improving documentation linkages. You know, making things less manual making sure that for extensions we can click through to the documentation for them and have examples. We have a lot more sandbox examples now. But there's other things that I think that we can do, for example, have a, you know, VS code Envoy configuration plugin so that where it has type ahead, and it has built documentation and things like that. So that's probably the one thing I think it's the onboarding and I think that's where we can bridge the gap to people that might be intimidated by all of Envoy's functionality because her better or worse Envoy is a very complicated piece of software that has lots of different features and it can be very intimidating for new users. So I think just really investing in that documentation writing examples and that entire ecosystem. I think that's the thing that I would like to improve if there is unlimited time. So let's see. Next question is, which benefits other than performance does XDS provide. So for those that don't know XDS is the name of our generic configuration API. We started a long time ago was just a few different XDS APIs, things like EDS or endpoint discovery service, CDS cluster discovery service, LDS the server discovery service and over time, you know, we have developed I don't even know now probably between, you know, 10 or so, 10 or so different APIs. And the main thing that XDS provides and I actually think that XDS has been the main contribution of Envoy versus the code is it has given us a well defined base where we care about backwards compatibility. So we take it super seriously in terms of the API design and now, you know, XDS in its current form is actually expanded beyond Envoy. We now see that GRPC uses XDS. I think there's internal systems and tools of various companies that speak XDS. So, you know, we've effectively developed a network configuration API, you know, for for these types of systems. So I think the real benefit, you know, it's not really want to performance I think the real benefit of XDS is it's giving us a common language that we can use, you know, that will allow us to actually configure these types of systems across the industry. And that has spawned huge vertical ecosystem of products and services based on Envoy and eventually GRPC and other systems. So, I think that's, that's the main benefit of XDS. Let me just go through some more questions now. Let me clean these out. Okay, next question. Based on your experience and growing and building a community around Envoy, what advice would you give to newly founded OSS projects, besides the onboarding experience. I've given several talks on this and it's something that I think about a lot. And I don't think there's, I don't think there's one one answer here. I do think that the onboarding experience is very important as has been noted. So, you know, things like a slick UI and making sure that there's examples ready to go and things like Docker and Docker compose, you know, and making sure that people can can get started with the project as well as with the contribution guidelines. You know, the, the main piece of advice that I give people, you know, around starting an open source project is that it's a huge time commitment to do well. Building community is hard work, and it really requires a lot of efforts and a lot of patience and a lot of time. And the biggest piece of advice that I actually give people which sounds really, really simple and probably obvious is just to be really nice, you know, when you when you start a new project, you don't, you don't have a community you don't have anyone to actually help, and so being new contributors and new maintainers and new people to join up is is the most important thing. So being very welcoming to everyone that comes, you know, making sure that communication is just welcoming and nice, and, you know, taking the time to answer people's questions and get them onboarded and facilitate helping them with the code and all of those things. I think, you know, that that is probably the most important thing. I think there's smaller items, like trying to have good issues for new contributors and again good documentation and all of the ancillary non code things that, you know, people tend to think about after that, but in my opinion in my experience they tend to be more important than the code itself. So I would, I would definitely, definitely encourage focusing on those things, and I don't have the link handy right now but I gave a talk at OS con a couple years ago. I think the tight it's on YouTube and I think the title is you know something around the, you know, amazing success of on boy. So I definitely search for that talk and I gave an entire talk on this topic. Let's see. I have a question. Do you think some part of the envoy magic could be rewritten using ebpf. Would it make sense on the performance side. I'm also not an ebpf experts. I, I definitely believe that there are probably large parts of on boy that could be written using using ebpf. Theoretically possible that all of on boy could be written using ebpf, you know, whether that's practical or not. I don't know. But I, I do think that over time, we will definitely see more functionality move into the kernel just from just from a performance standpoint. We already see projects such as psyllium that rely both on ebpf as well as on boy where they have a split of doing, you know, a bunch of functionality in the kernel in terms of things like TLS offload and more efficient routing of packets, you know, between processes. And then they also route up to envoy to do various policy aspects. So, you know, they would certainly be the best people to ask in terms of why they've chosen to split things where they have and whether over time they expect more functionality to move within ebpf. I would imagine the biggest issue on the ebpf side is that, and again, I'm not an expert here is I think there are restrictions on the type of code that can run within ebpf. For example, my understanding is that I don't I don't think you can have things like loops and various other things, you know, so I think it would be difficult to rewrite all of on boy as an ebpf program. But, you know, between some combination of ebpf and kernel offload and, you know, running in, you know, user space projects like on boy, I have no doubt that things will shift around over time. Okay, let's see next question. Since documentation is the main improvement part, what's the main thing and on boy lacking documentation for now in your opinion, besides generic newcomer stuff. I think the main thing that we're lacking right now and on boy around documentation actually comes back to extensions. I think. So, for those that don't know on boy uses protobuf as our configuration language on. I'm a huge proponent of protobuf I think it's an amazing project and I think in proto three they've done a really incredible job of allowing conversion between yamal and Jason and proto. And so it really gives you the best of both worlds and allows people to configure things in yamal but still have, you know, the real inherent structure of the protobuf ideal. And without getting into the weeds, the way that we handle extensive extension configuration with on boy from a configuration perspective is a little convoluted and can be very difficult to learn for it can be difficult to learn for new users. With the way that extension works that we basically use type URLs within protobuf. So there's a protobuf type called an any, and it allows you to specify a generic type and then have configuration within that. And it's done for a very good reason, which is that on boy allows extensions to be compiled into the binary that on boy doesn't know about ahead of time. So we need this generic mechanism where we can offer extensions on boy core doesn't know about, and then users can actually configure them. So, you know, we use this protobuf any and type URL configuration mechanism. The problem with that is that for new users, it can be very difficult for them to understand what is what are the extensions that are available. What is the magic type URL that I have to use. What is the configuration that can come under that type URL. And once you're, you know, an experienced user. That it's not a big deal. But for new users, it can be very daunting. And so that's what I was talking about before, where we're starting to put in a lot of effort into cross linking between various aspects of the documentation so that it's automated. So you go to an extension point and you can, you know, click through all of the different extensions that are available. We want to provide example snippets for every extension so that the user can just see what they have to put into their config. Again, I'd like to do a VS code plugin where you know it actually has auto completion. So I think there's some things that we can do there that will really help the 99% use case. So I think that's the main thing and I think the work is ongoing, but if you're passionate about it. Definitely please, please reach out and help us. Next question. Any plans for XDS v4. No, the XDS v2 to v3 transition was a very large learning experience for the project. And I'm going to be honest, I think knowing what we know now, we would not have done it. And I think that we are, we are retaining the opportunity to do a v4, but there are no plans to do it. And realistically, I think we are many years to never out from it. And instead of doing a v4, we are working now on a minor version proposal, which will try to actually have it'll be a compromise between doing a clean new version where we can actually delete fields, and allowing us to have clients that will remove deprecated functionality that may have been deprecated for several years. So, yeah, I would definitely watch this space. I think it's something that we will continue to improve and learn about. But at this point, I think the, the amount of effort around the industry that went into v3 was not worth the benefit. Part of that is actually just hindsight on on time and growth of the project. We started talking about v3. At this point, it was probably a year and a half ago, and just with the meteoric rise of the project, we didn't have as many users as we have now. So, you know, I think when we started talking about it, we really underestimated the, you know, the burden that it would cause people we didn't understand it. And I think we have a better understanding now coupled with just the large usage. So I think, you know, I think we've learned a lot, I think we're going to continue to improve in this area, and I would expect us to do better in the future and know I don't expect to see a v4 anytime soon. Well, it looks like we have time for one or two more questions. So please go ahead and type them in. Let's see. Since this is a maintainer Q&A, could you share with us how things went regarding your work as an envoy maintainer and as an engineer at Lyft at the same time, even if both are linked? Like what balance did you find between work on envoy that was related to Lyft needs and work on envoy that was unrelated? Was there any real discussion about this or did things just happen naturally? This is a complicated topic. And I've done a few podcasts on this and I would definitely encourage folks to, you know, to listen to some of the podcasts that I've done in this area. But very quickly, I would say it's complicated. I think as an industry, we don't do a very good job of being honest with people about the needs of open source. In terms of just what it means to do the things that I was talking about before about answering questions and helping people and fixing bugs and doing things that may not be directly related to the employer of the person who's doing that. And, you know, I will be honest in saying that in 2017, you know, that was a very rough period of time for me in which Lyft was supportive, for sure. But still, I was doing two jobs then I was, you know, basically working full time on, you know, really creating the community and recruiting maintainers and contributors and all of the wonderful people that we see today and that was a full time job. And at that time I was still working full time at Lyft basically operating Envoy and running the networking team. And 2017 was a very tough year because I there just weren't enough hours in the day and I had to choose between Envoy and choose between Lyft and, you know, there were some conflicts there. And obviously things could have gone better in certain cases but you know that's just life. And, you know, now, obviously with the success of the project, things are much improved. I, you know, I have a bit more leeway to, you know, spend time on open source and to make sure that I have a decent work life balance. But I'm going to be completely honest that it's a it's a really difficult problem and there's no easy answer here. And what I would say just wrapping up is that I really encourage a very open and honest communication around open source between employer and employee and be really honest about motivations and time. And I really encourage people just to have that constant conversation to hopefully avoid getting into that two job, two job situation. And I also encourage people like the poster mentioned in their question to try to find common ground of working on open source, you know, for things that the company actually cares about and I think that's that's the way that it tends tends to go the best. So I will stop there. I will be in slack if people want to continue this conversation. And anyway, thank you very much for having me and thank you for coming to this maintainer Q&A.