 Hello. Hello. While we're waiting for people to come, that bug, the one where there's that assert, Alyssa, on the message begin headers, I think it's a bug actually in HTTP parser. Yeah, do you know off the top of your head whether before the beginning of a response, so like before it says like HTTP 11, are you allowed to have CRLF in there? Because the code in HTTP parser in certain paths makes it seem like you can. But basically what's happening is that if there's a carriage return or line feed before the beginning of the response, it'll fire the on message begin callback multiple times. So it's definitely a bug in the parser. I'm just not sure if like it should be allowed or not. I think I'll file it down and see what they think. Yeah, I mean I think I'm just going to add a test and like because there's three different keys is where this happens and in two of the other cases, it just ignores carriage return line feed. In this one case, because of the bug, every time you get a carriage return or line feed, instead of just skipping it, it'll fire the on message begin callback. So that's why it's all going haywire. But like based on the spec, it was unclear to me if that's even legal, but I guess I'll just I'll make a test and file an issue. I have never seen that. Oh, but there's all sorts of things that shouldn't happen that are connected. I actually don't think this one is. Yeah, I don't think it's allowed either, which is surprising because they have strict mode and not strict mode. So I'm not sure what's going on there. But it's only message begin. It's not trading it as like, hey, you sent an email message on the display. No, no, no, yeah, it's definitely a bug. So yeah, we can work around it and then just have them fix upstream. Well, yeah, I mean, it's a pretty simple fix. So I'll just do the fix. But yeah, we can see how that goes. This is the one where we get the health check response before we actually sent the request. Well, yeah, I mean, so there's the fuzz test failure that Harvey found. So I debug that. I'm not sure if that's the same issue as the WebSocket one that was reported. I think it probably is, but I have to test it. But like, because I'm guessing that the WebSocket thing for some reason is like sending either a carriage return or line feed before the actual response. That's my guess, but like, I will go and debug it. All right, let's see. So we have a couple of things on the agenda. I guess, is Greg on the call? Oh, yeah. Yeah, I'm here. Okay, cool. So I don't know, I was thinking we could do the, like just the quick items first and then we could talk about the WebSocket stuff. Okay, I see the first quick one is it came up during code review that historically in release notes, we have not put bug fixes. I don't really have a strong opinion. Like I think some projects, they'll break out, they'll break out, you know, things into features and bug fixes and then with the bug fixes, they'll link to like the original issue. It would be a little more overhead for dealing with fixes to people have a preference on that. I don't really care. I think I'd be okay with kind of taking our case-by-case basis on the severity of the bug. Like if it's unlikely anyone has ever seen the bug, not bother. If it's a big thing, then definitely. Okay, all right. In between then just kind of. Yeah, that's fine. I mean, yeah, it kind of, it seems like unnecessary overhead to like list every individual bug fix. So that's fine. I mean, why don't, I will do a PR to the PR template that basically says roughly that, that like we can use our judgment as to if it's a pretty severe bug fix that we can add it. Any, anyone have any objections to that? All right, I'll do that. I guess, Dave, do you wanna give a quick network service extension update before we get into WebSockets? Sure, so I've continued to grind through the file descriptor, socket, call replacement. I've just about got the PR ready to do an initial push. The place where I'm gonna need some input and some help is how do we, how do you want us to take the network service factory and hook that into the config stuff? That was kind of a little confusing to me about where to put that. And so I think we can just discuss that. Yeah, I would probably not worry about that right now. Like I would get rid of the FD first like in all the different places. And like you don't even have to do that in one PR. Like if it makes you feel more comfortable, like you can keep a file descriptor like password is today and then just start passing this abstraction object that also has the FD and then like do incremental PRs where we like get rid of all the FD calls. And that might be the better way to do it than one giant PR. All right, well, I pretty much have the giant PR done which I think is certainly worth having as a roadmap. And then we can look at, so I'll push that the next day or so. And then we can roll out an implementation thing. So, I mean, some of the other minor questions are, there are some socket calls in the OSS calls class, Singleton class, and does it have value to leave those around? I initially ripped them out, but then today I was thinking, well, maybe for testing purposes, that's still a value, but now for the most part. We had put them in mainly for testing just so that we could override those calls. And I think essentially, once we have a registrable object, we could just as easily swap out an object that does a fake sys call. Yes, all right, yep. That would fulfill the need in a more clean way, but it may take a couple of steps before you get to that point again because you have to be able to configure it and then have that test mockdown one. Yeah. If you need to be able to test the actual object that does the actual sys calls, you may still want this. Right, well, that was really the thought I had last night when it's winding through some of my cleanup was when we start to test this stuff whether that still has value. It probably does. And so, and I'm still, I mean, there are still a bunch of places and well, I'll push something sooner rather than later at this point, because I think I've got enough volume to generate a good discussion, but there's still a lot of places, even with that where they're just native socket calls that exist around the code base that ideally we want to get rid of those, but we want to do that in a structured fashion. Yeah, I mean, this is a big project. It has a tremendous amount of value even outside of VPP, just because we're gonna end up eventually porting the code to Windows. And I mean, it's like there's just, there's a whole bunch of different reasons why this end up being super useful. So, I mean, I think all of the work that you're doing is gonna be great. It just, it doesn't have to be, I guess what we're saying is it doesn't have to be done in like one giant drop. Like if it makes sense to do it incrementally, that that's probably right. Well, I just kind of ran into the, pulling the thread and unraveling the sweater trick. And so I just kept going because it just, it made sense to me to keep exploring that. Yeah, I mean, let's just look at the giant PR. Like I actually think for the most part it's going to be fairly mechanical or it should be. So let's just see it. And then if it looks like there's some obvious places where we could split it up, you know, we can do that. Sounds like a plan. Great. Okay, WebSocket. Maybe either Alyssa or Greg, could you catch everyone up on the conversation because it's kind of complicated. Sure. So essentially right now for those of you not terribly familiar with WebSockets, essentially you send it to each TV header saying I'd like to upgrade to WebSockets and then everything following that upgrade the, and the response header saying, yes, your upgrade succeeded or no. And then from then on kind of what normally would have been HTTP payload is WebSocket data. So right now the way it's implemented in Envoy for HTTP11 is Envoy intercepts those headers. So it's great, this is WebSocket. And then basically detaches the rest of the request processing to the TC proxy session. Say look, this is no longer HTTP. We're just going to proxy the WebSocket payload upstream and be done with. So there was requesting, okay, that's all well and good but we'd like it if the WebSocket request headers went through the normal HTTP filter chain because the upgrade headers are headers. So that would be nice. Meanwhile, there's a second issue that about how to handle WebSocket for H2. And there's a spec in progress theoretically should be ratified in the next week or so if it hasn't already saying basically for HTTP2 where you can't take over the entire upstream connection you would use connect to effectively tunnel this WebSocket data on one H2 stream which makes a lot of sense. But the problem is at that point, the H2 WebSocket, the headers and all the data would go through your headers and normal HTTP data calls. Meanwhile, the HTTP path, we're doing it with this weird kind of hybrid of handling the HTTP headers eventually and then TCP proxy handoff for the data. Doing it differently seems like it's gonna cause a lot of confusion in the long run especially because if your upstream is auto is doing HTTP or HTTP2, like halfway through you're deciding what weird thing you're doing. So we're just trying to resolve what we wanna do and ideally unify these cases if we can find a way that makes sense. Yeah, I mean my idea which would be a bunch of work is to actually make new filters and then basically have parallel filter stack. And I think a lot of the filters that you have could implement both interfaces but that would basically allow you to specify even like on a per route basis potentially that you could fork off or maybe even not per route but like on a HTTP connection manager level you could have like a WebSocket filter stack and then like a non-WebSocket filter stack and that would require filters to opt in to basically being WebSocket aware instead of suddenly getting into a situation in which you have filters that might now be getting WebSocket data and then like some of the calls might be different. So for example, like you don't need 100 continue or like those types of things. So that was one idea that came to mind that seems like the most extreme solution but it might lead to the least amount of confusion. Yeah, the thing I don't love about that is again, I'm concerned that the API would be similar enough that you couldn't reuse the code. Like essentially you need the same headers and the same data functions. We just might mean them slightly differently. Well, it doesn't, I mean, I guess again, like this is mostly brainstorming. It doesn't even necessarily mean that we have to define like new interfaces in code. It would more just mean that you would have to as a user effectively opt in to say that like these are the filters that run if it's a WebSocket connection. Okay, so more of a config perspective. Because that's what you're saying. Right, so like it would be more of a config thing where then what the, so right now if you look at the connection manager code we end up instantiating the filter stack like right when the stream is created but there's no technical reason that we have to do that. We could actually wait until we get headers complete. So we would know if it's a WebSocket upgrade request. And then at that point, we could basically say like not WebSocket instantiate the not WebSocket filter stack. WebSocket instantiate the WebSocket filter stack. And like you could imagine that in the WebSocket case you actually have like a WebSocket router or like something like that that like knows how to do like all of this HTTP2 versus like TCP proxy stuff. So like instead of the TCP proxy being bolted into the HTTP connection manager you would allow it to flow through. And then at the end of the WebSocket chain it's like a WebSocket router. And again, like maybe that's mostly the same code as the current router, like maybe it's the same code. I don't really know. But that could take care of the is it HTTP1.1 like maybe do the TCP proxy thing. Like if not do something else that's kind of what comes to mind. Like I'm not sure how else you would do it without it being potentially super confusing if people are trying to support both non WebSocket and WebSocket routes. Yeah, I think having the config filters different is totally reasonable. I don't understand what value the TCP proxy session would have in that case. The current handoff we have. Because I thought about No, no, right. No, right. So I'm actually suggesting to get rid of like the way that we do it today which is to bolt it directly into the connection manager. I'm suggesting that we basically kill that. And then, but we still need something at the end of the line, right? That basically knows what to do with that data. So right, because like you're not, you're not proxying like a full connection at that point. You might need to make like a dedicated upstream connection and then kind of switch into TCP mode. So, so like I don't, I mean, again, I haven't thought about this enough but it doesn't seem like you could just rip the code out of HTTP connection manager and like use the existing router. Like it just, it wouldn't work. So that's kind of why I'm saying is that you could have the existing filters like an alternate WebSocket filter stack where you could do header manipulation and maybe even data manipulation or something like that. And then- I'm still actually a little bit confused why you think we need something different at the end. So just internally, we treat WebSockets as HTTP 11 data, it's just BiDi. That's the only difference. So we literally use, I mean, when you do the upstream connection you have to pass headers on and then we just pass headers on and then say, great, you're streaming data back and forth from that on. Yeah, maybe it would work. I'm just thinking through the cases where like if you went to a connection pool that was like trying to reuse the connection and like you had like previously sent like a non-WebSocket connection there and then you like reuse it on the pool. I just like, I feel like there's gonna be corner cases there. Like it's not clear to me that in the current code it will work with all edge cases out of the box. Like I feel like you could get into a situation where let's say that I have an upstream that supports both WebSocket and not WebSocket and I'm doing connection pooling. Maybe I've pipelined or sorry, I've like, you know, I've sent some requests, I've gotten some responses with Keep Alive. Can I then switch that connection over to WebSocket? Like is that okay, maybe? I don't know. Works for us? I don't know. No, no, no, right. So it's like, I mean, it might totally work. I just don't know. I think those are things that might need to kind of get sorted out. Greg, do you have thoughts on this? I think, I don't really have strong feelings either way. I agree that what we have now is a bit of a clutch. Does anyone actually have the time to do the work to sort this out? Well, if it's, I mean, large, it depends. If it's just allow a separate filter config for, you know, for WebSocket versus not and delay the filter creation, I could probably tackle that. I have WebSocket, sorting out WebSocket as one of my goals of this quarter. Okay. So that sounds like it's pretty straightforward. If we, if we, but this is where I was trying to understand like, do we need to move the TC Proxing code or do we just mix it or kind of what's the plan? Well, so I think if we've already established the upstream connection through some means, then the value we're getting from the TCP proxy is getting to be pretty minimal. I mean, you know, just sent proxying back and forth between two connections isn't that much code. No, right. So the existing router code should work fine for that. It's more, and again, this is just because I'm not super familiar with WebSocket. I feel like inevitably the, at least the router filter is going to have, if we use the router in both cases, the router filter will have to become WebSocket aware to some degree. Like I'm guessing it might have to do the connect or like it might, it might have to know various things. And like, and again, I'm also not sure this is just being not familiar with the WebSocket kind of spec is, is it okay that like we might be using a generic HTTP11 connection pool? Like, could we have used those connections for non WebSocket traffic and then can we upgrade it to a WebSocket connection? Like, is that okay? I don't know. Yeah. So I mean, it's like, if it's okay, then I think it's easy. If it's not okay, I still don't think it's super hard but it's just like, there might be some other cases here where like we might just have to deal with it basically. Okay. So I think, I think I'm willing to take on having a separate configurable filter chain. We're using the HTTP filters because we think they're API compatible and you know, buyer beware, all comment that you shouldn't configure non-WebSocket compatible filters on the WebSocket. I mean, that should be pretty straightforward. Yeah. I mean, right. And as people run into issues of like, hey, my upstream server doesn't support pipeline WebSocket. Can y'all have an option to not do that? You know, we can add it as needed. Right. And like, it might be good just to kind of list out like, what are the, like, what are the gotchas of the current situation? And like that, that would be one that comes to mind where like, I could just see that not, not working. Like if we- I'll comment that one. I'll comment the fact that we have to do the upgrades and downgrades somewhere. Oh, the other thing I'm actually really uncomfortable with this is the other one I wanted to bring up. And again, this may come under like, hey, you shouldn't use this filter for WebSocket is if we, we have to do the upgrade and the downgrade at the last possible point because we don't know if we're speaking HV11 or H2 until we hit the router filter. But the thing is that you're actually literally changing the method. So if you have something like the course filter, which is blocking via method, like what should the method be? Pretty much everywhere else in Envoy, we say everything that Envoy does is H2B2 and then we downgrade the H2B11 style headers on the way out. I feel like for WebSocket, it's, it almost should be the other way where we should treat everything as like a WebSocket upgrade all the way through. Like even on the H2 path, let me get it, say, hey, this is, this is logically a guess with an upgrade to WebSocket, treat it as WebSocket all the way through and then at the last, move it downgrade it to the next. And I wanted to get people's thoughts on that because it's doing twice the work for one of the past, but it ensures kind of consistency when you're going to the filter chain. Yeah, I mean, I don't, I don't have a strong opinion as to whether you like kind of make it one, one or like make it two, but I definitely agree that what has made that code and Envoy so much simpler is that we kind of normalize all of that logic. So I think- Yeah, you can normalize it. Yeah. And like every Rails and Envoy we're normalizing on each two, but normalizing on each two WebSockets is weird because they're like an afterthought. So I can go either way. I don't know if anyone else, Greg, if you have thoughts or opinions on that one. I don't feel strongly about it. The other thing I wanted to bring up is what are we going to do with the changes to the configuration that'll be needed? This will probably end up being not backwards but compatible configuration-wise. Yep, so we'll have to do the deprecation dance. So we say the old style of doing things is deprecated and wait a cycle and then get rid of it. So are we going to leave the old implementation there essentially until the deprecation is finished? I think we'd have to. Yeah, I don't know how you'd make the configuration compatible without that one. The configuration is based pretty heavily on the implementation. I think we'll add the new WebSocket path and then deprecate. The only thing that I could think of, which might be possible, but I'm not sure, is if someone had set the WebSocket true thing on any route, you could maybe make, you could implicitly make a WebSocket filter stack with only the router filter in it and that might work. But that requires more thinking. Yeah. I think I'm just gonna say the old way is deprecated. Once the new one's implemented and then we'll do the dance. That sounds good. Makes sense. Yeah. It's actually using it, by the way. Me? Yeah. We are about to. We're in kind of pre-release testing on some of it. So I'll try to get this done as fast as possible and then you guys can maybe test it out at your own pace and then let me know what else I need to do that does or doesn't work with your existing system. Does that sound good? Well, this is gonna be kind of weird for us because we have something that is, it's remarkably similar to WebSockets, but it's not compliant with WebSockets. And so we're trying to shoehorn it in. Oh, you're already doing some sort of weird dance. Yeah, yeah, we, yes. We have a, we made a network filter that modifies the request coming in to make it more WebSocket compliant and yeah. Oh, also on doing this, I think we should make it a general like, hey, this is an upgrade path, not WebSocket specific so that later on, if we wanna do like connect or whenever we just say this is how we do it, yeah. Yeah. I don't wanna have it be like the WebSocket filter chain and then we have to have it connect filter chain. Maybe people want different ones. I don't wanna support it yet. Yep. Okay, cool. Okay, I will document this, unbug, leave it open for a couple of days for people to discuss, see if anyone has strong opinions on, you know, which way we treat it and start working. Great, that sounds awesome. Cool, did anyone have anything else that they wanted to bring up? A question on issue management. Do we have some sort of a bot integration or else plans to have one for assigning an issue to you if you don't have right access to assigning yourself? You say I would like to assign to me and one of the maintainers assigns it to you. It would be nice to have one. I mean, there's an open issue on bots. If anyone wants to work on bots, that would be great. Would you, on this use case or would you prefer just like ping a maintainer and say, can you assign this issue to me? We would love to have the bot that let you auto assign. If you write it, we will approve it. Yeah, I mean, there's just search for bot in the existing issues and there's a wish list of bots. And it's like, you know, they've been doing a bunch of stuff with like ProBot and a bunch of other stuff. So it's possible these things will exist. It would probably be better to even do it as part of the ProBot project in GitHub and then everyone can get it. But that would be awesome. I think there's still the issue that you keep calling the unboil organization, but that can actually assign to somebody who isn't in the unboil organization. Oh, yeah. I think maybe we would just converging those folks who actually regularly would be assigned issues. There wouldn't be an issue. I mean, I don't actually think in GitHub you can assign to someone that is not in the org. So it's like, people have to get added to the org, which I mean, to be honest, like there's no, I just add people to the org who have like shown interest basically, like it doesn't really matter. So that's fine. Cool. Anyone have anything else? Hey, Michael Payne here. I've been trying to compile Envoy on Arch 64 or 64. And I did raise an issue, but just a super short update. It's still not working. So Bazel doesn't support Arch 64 properly. The rules go for Bazel, doesn't support it either. And Lyft's Protok Gen validate won't compile on anything as not Windows or Darwin from what I can see. So the struggle continues. So I'll keep you updated. Fight that Bazel fight. Yes. Very nice beard also. Thank you. I'm going to get straddler and this is what happened. I have to hop off, but if people want to keep talking, go for it. Bye. Thank you. Have a great week.