 Okay, so I'm going to talk about Proxing HTTP 3 with Connect UDP in Envoy. This is a joint work with Eric, Abhi, and David. So before I talk about Connect UDP, let's see the current landscape of Proxing protocols to understand why we needed a new protocol. So the simplest approach of Proxing would be using a regular TCP or UDP proxy, which simply forwards packets based on Vybituple. This approach may have some use cases, but it's limited to static forwarding rule based on Vybituple, so it doesn't allow a client to connect to an arbitrary target 3 proxy. So to overcome this limitation, the SOX protocol was developed a long time ago, and it defined the request commands to let clients open a TCP or UDP socket to an arbitrary target address port. However, it sends those information in plain text. So anyone who watches the network can see which servers a client access to. So the HTTP Connect method is an HTTP method that serves a similar purpose to SOX. But with HTTPS or TLS, it hides the target information from network eavesdropper naturally. However, it only supports creating TCP connections to targets, not UDP. So you cannot use Connect for UDP-based protocols for Proxing. Someone may argue that we can just use VPN protocols like OpenBPN or IPsec or WireGuard, but they require admin privileges, so they may not be always a viable option to choose. So to overcome those limitations, Connect UDP was recently standardized in IETF. It allows HTTP clients to send UDP packets to an arbitrary target 3 proxy. So a client sends payload data as HTTP messages, and a proxy sends those data as UDP data grams over to the target. It is defined in the RFC 9290s8, and it supports all existing versions of HTTP. So for HTTP 1 and for HTTP 2 and HTTP 3, the Connect UDP request is sent as extended Connect request, and for HTTP 1.1, it's sent as an upgrade request. So I will show you how this works with more details in the protocol level, actually. So let's see how this works. A client, to establish a new UDP tunnel, a client first needs to send a Connect UDP request in the following format to a proxy. So in HTTP 3, it's extended Connect request, and it's how it looks like. And let's focus on the protocol and path header fields now. To indicate it's a Connect UDP request, the protocol header field is set to Connect UDP. And the target address port information is specified in the path header field in the URI template. And the target address can be a domain name, and in that case, the proxy must perform DNS resolution before sending a response. Then proxy creates a UDP socket to the target address and port. And unlike Connect, it's a UDP socket, so no handshake to be done at this set of phase. And once the socket is created, the proxy sends a success response to the client like this. And as soon as the client receives a response, it assumes that tunnel is established and starts to send data using HTTP datagrams. So the HTTP datagrams was also recently standardized in RFC 90297, and I'll explain more of this and why we needed this one in the later slides. So with Connect UDP, we can overcome the limitations of existing proxing protocols I mentioned earlier. It supports an arbitrary target, and it provides useful security and privacy properties like HTTP Connect. And it can be implemented in the application level, and most importantly, it supports UDP protocols unlike Connect. However, I would like to emphasize that Connect UDP is not just for UDP tunneling. Combined with Quick or H3, it unlocks the power of application level proxing. So with H2 or H1, a loss in a stream can block other independent HTTP streams because it's TCP. But with HTTP3 downstreams, we can now multiplex multiple Quick connections or HTTP3 connections in a single Quick or HTTP connections without this head of line blocking problem. Also, the traffic to the proxy and also the target looks exactly like a normal HTTP traffic. So it provides excellent censorship resistance property. And also, obviously, the traffic is encrypted with TLS 1.3 inside Quick. So Connect UDP was standardized in ITF mask working group, and there are multiple efforts going on in the group to leverage those excellent properties of a Quick-based application level proxing in other use cases. So to explain how we implemented Connect UDP in Envoy, let's first look at how Envoy handles incoming HTTP requests in a very high level. So first, TCP packets or UDP data programs are received through a TCP or UDP listener in Envoy. And the Envoy HTTP codec inside HTTP Connection Manager decodes the decrypted byte streams as HTTP messages. And the router filter in HTCM is responsible for selecting a cluster based on the configured routing rule. And then the cluster manager chooses a host in the cluster and the corresponding upstream connection pool to that host is used to send a request to upstream through one of the connections in the upstream connection pool. So for UDP tunnel establishment, we've made the following changes. So I'll explain the details of those changes in the next slide. So let's first focus on the changes made inside HTTP Connection Manager. So Envoy internally normalizes HTTP2 and HTTP3 extended Connect request to HTTP 1.1 upgrade request. So this is not specific to Connect UDP. This is mainly to use the same HTTP upgrade logic regardless of the HTTP downstream protocol level. So this is an example of Connect UDP request headers. And it becomes the upgrade request in Envoy like this. So as you can see, the extended Connect method is transformed to get request with upgrade token in H1 inside Envoy. And the upgrade header field value is set to the same protocol value header field in the get request. And secondly, the second change we made inside HTTP Connection Manager was to make Envoy rewrite the authority or host header field. So existing Envoy codes and filters assume that the target address in the is located inside the authority or host header field. But as you can see, in the Connect UDP case, the target address information is inside a path header field. So we wanted to make the subsequent Envoy filters work just correctly with the Connect UDP request. So what we did was we made HTCM just rewrite the host header part with the target address information in the path header field value. So now the subsequent filters like router filter or any other HTTP filters would just work normally. And then we made some changes in the router filter. And as I mentioned, the router filter is responsible for routing a request to a correct upstream cluster. So we made existing Connect Matcher, which originally only matched HTTP Connect methods, match also the HTTP Connect UDP request. So now Connect Matcher can match Connect UDP request. And we also introduced a new upgraded type token, Connect UDP, so that a routing request can specifically match Connect UDP request. So in this example, Connect UDP config YAML file, HTTP Connect UDP request will be routed to a upstream cluster one here. So now let's take a look at the upstream connection pull part. So Envoy associates a connection pull to each upstream host or endpoint. And previously, Envoy had two types of upstream connection pulls. There was HTTP connection pull, which is used for HTTP upstreams. So Envoy creates multiple TCP or quick connections and reuses them for performance and efficiency reasons. And there's a TCP connection pull. And this is specifically for HTTP Connect method. So Envoy creates only one new connection for each target host in the HTTP Connect case. So it's not really a connection pull. And Envoy keeps the mapping between a downstream HTTP stream and a created TCP connection so that it can convert incoming HTTP data frames into TCP bi-streams. So we need to implement something similar for UDP upstream and UDP connection pull to support Connect UDP. So we added UDP upstream handling code that creates a UDP datagram socket inside UDP connection pull. So if the socket creation succeeds, the UDP upstream code, as the success status code to the response like this. Then it stores the mapping between the HTTP downstream and the corresponding UDP socket created like HTTP Connect case so that it can pass HTTP data to the right upstream socket. So now let's switch gear to the data path. I'll explain how we changed Envoy to enable data exchange through the established UDP tunnel. So as I mentioned earlier, HTTP datagrams and capsules are used to send data after UDP tunnel is established. And we've made the following changes in the HTTP codec and upstream connection pull to support HTTP datagrams and capsule. So before talking about HTTP datagrams, let me first explain the issue of HTTP 3 or quick proxying and to explain why we needed HTTP datagrams. So for H3 proxying or quick proxying in general, first client and proxy establish a H3 connection. Then a client sends a connected request to the proxy as an HTTP stream to establish a UDP tunnel to a target server. And through this stream, a client establishes a quick connection to the target. Then a client can send quick or H3 streams through this inner connection. And as you can see, quick connections are nested in H3 or quick proxying case. And this can cause the following issues. The most important one is double loss recovery. When there's a packet loss, both inner and outer quick connections react to that packet loss and thus redundant retransmission. And this can exacerbate the network congestion and degrades the network performance quite significantly. And there's another problem of nested congestion control. And this can make the client overreact to a packet loss. And it can lead to network underutilization. So to solve this problem, we needed a way to send the data unreliably in H3 stream. So to introduce this unreliability, a quick datagram is standardized in RFC 9221. And this allows an application to bypass the retransmission mechanism of quick as necessary. And RFC 9227 H3 datagram format was defined. And it uses a quick datagram to send HTTP data unreliably. So with those new two constructs, we can send the datagram unreliably in H3. So the previous picture is changed like this. The outer H3 stream can now become unreliable. So we no longer have the nested loss recovery problem or nested congestion control. What about in H1 and H2? So the unreliable delivery is impossible inherently because they run over TCP. But RFC 9227 still defined a datagram capsule to allow exchanging HTTP datagrams, even when there is a client or server or even proxy that do not support H3 or unreliable delivery mechanism. But it allows proxy can convert the datagram capsule to H3 datagrams when H3 connection is available in the path. So it lets client to indicate it wants to send the datagrams unreliably whenever possible to utilize the underlying transport protocols unreliable delivery mechanism as much as possible. So I will not delve into the details of it. If you're interested, please look at the RFC. Now let's briefly see how Envoy handles the H3 datagram and capsules with our changes. So first, we made H3 codec decode H3 datagrams as datagram capture for normalization. And this is mainly for our internal Envoy processing logic can just do everything on datagram capsule regardless of the HTTP downstream version. So for upstream connection pool, it can just de-capsulate datagram capsule and send the payload as UDP datagrams upstream. So it becomes UDP diagrams from the perspective of the target server. And in the upstream to downstream case, it's pretty much the exact opposite. The UDP datagram becomes encapsulated into datagram capsule. And for H3 downstream connection, the H3 codec encode the datagram capsule into H3 datagram using quick datagram. So now let's take a quick look at a demo of this feature in action. So there was no widely available Connect UDP client because Connect UDP was recently standardized. So our team created a task client for Connect UDP request. And I'll just use the same config file in the Envoy proxy GitHub repository. I think I should have duplicated the display. OK, let me see. All right, so I am running the Envoy server now. And this is running in my laptop. So as you can see, I disabled the logging and just enable the access log to easily see what's going on on the Envoy side. And yeah, this is a command for the client. As you can see, you can focus on this part first. This is the address of the proxy, which is running inside my laptop. And this is the port number. And this is the target address. So let's first connect to Google.com, which supports H3. So as you can see, on the Envoy proxy side, it received HTTP Connect UDP request. And it's transformed to upgrade request. So as you can see, it's actually internally normalize it into get request. And you can see the target information in the path field. And the target information is correctly parsed into Envoy. And on the right side, the client shows that it received. So the order is the opposite. But it first received the success code for the successful UDP tunnel establishment. And in the above, you can see the HTML content that Google.com returned. And we can go try YouTube.com as well. And it works fine as well. If you want me to try a non-Google server, Cloudflare, let me see, Cloudflare also provide a test server for H3. As you can see, yeah, this one also works fine. All right. So obviously, the major use case of Connect UDP is enabling H3 proxy. Well, with H3, a client can create H3 connection to a proxy. And then it can create multiple H3 connections through that proxy without the head of line blocking problem I mentioned earlier. So it really enables the high performance application of a proxy. And it also provides excellent security and privacy and essential resistance thanks to the quick-end TLS 1.3. And specifically with H3 proxy, the target servers can only see the address of the proxy, not the client, because the UDP tunnel is established between proxy and target. So it provides some degree of IP address privacy for a client. But the proxy can still see the target address and watch which servers a client is connecting to. So can you hop over this limitation? We can hide even the target address from the proxy by just introducing a second proxy and do two-hop proxy. And in this setup, a client opens a H3 connection to the second proxy through the first proxy using a Connect UDP like this. And it uses that stream to connect to talk to the second proxy. Then the client again uses Connect UDP to create multiple H3 connections through the target like this. So in this scenario, multiple H3 streams are nested inside outer H3 stream from the client's perspective. Because the outer stream is encrypted with quick between the client and proxy 2, the proxy 1 cannot see the Connect UDP request sent to proxy 2. So it hides the target address from the proxy 1's point of view. And proxy 2 sees the target address, but it sees all those streams coming from the proxy 1's address, not the clients, because the UDP tunnel is established between proxy 1 and proxy 2. So it hides a client's address from the proxy 2. So as a result, no proxies have complete information of which client accessing which target servers. And this idea of two-up proxies for IP address protection has already been realized and deployed. Safari provides iCloud a private relay that uses the exact same architecture I showed earlier. And Chrome has been also announced to provide the same service. And this two-up proxies with Connect UDP case, the second proxies are run by third party parties so that no parties can have the full picture of the user traffic. And this is actually quite similar to Tor network if you're familiar with that. And it has pros and cons compared to that. So in the case of privacy proxies services, clients are authenticated. So the services use blind signature scheme to hide the user identity when the proxies is actually done. So this can potentially provide better userability for rep browsing. Because if you have ever used Tor browser, the last IP address of the onion router are often blocked. Or the client should normally solve capture or their third party parties are often blocked because the traffic is often associated with normal or malicious activities. So authenticated traffic can provide some better normal rep browsing experience. That's one of the cons. But compared to Tor network, those services currently only provide two-up proxies. On the other hand, Tor network provides three-up proxies. But I think both Tor network and privacy proxies technologies are not orthogonal. So they can actually introduce the pros of the other approaches. For example, in privacy proxies, it can just introduce one more or several more proxies with Connect UDP easily. So to conclude, Connect UDP unlocks the power of HTTP proxies to a new level with quick NH3. And Envoy is the first open source proxy that supports Connect UDP. So we believe that Envoy supporting Connect UDP will accelerate the adoption of this proxies technology. And this is the end of my talk. And I'll be happy to take your question. Thank you. So you're specifically asking about the privacy proxies services. So this is mainly for hiding the IP address from the end servers. So for example, when a normal user just browse websites, those websites often have tracking service embedded. And the IP address and the third-party cookies are now being gone, although some browsers still support it. But IP address is actually a very powerful identifier to identify each individual user. Because IP address can barely change for normal users, actually. So this means that using the IP address, there is a lot of incentives to using the IP address as an identifier to link the identity of the user to their web browsing history. So by providing those privacy proxies services or multi-hop proxies services, we can unlink that linkability. So it could potentially protect the privacy of the browser users from those third-party trackers or things like that. Yeah. You're asking if it can be used for hiding the server IP address. So for this one, it's a little tricky to do that. Because here, clients establish those tunnels to the server. And the way the privacy is enforced is actually this kind of on your very similar to Tor. So your request is actually routed through the proxy. So the end server cannot see the IP address of the client. So I think just with this technique, I think it would be a little tricky to do that. But yeah, oh, I see. Well, I'm not completely sure about that. But yeah, maybe it's possible. I mean, we can just reverse the traffic direction actually from the server to the client. So maybe it might be possible. But currently, the main use case is for hiding the IP address of the clients. OK. So the first part, you were asking whether the captures are decapsulated in the proxies. OK. I think I'm not sure I fully understand your question. So here, just to show this again, maybe it'll be helpful to understand the part. So as I mentioned, there's two parts. One is UDP connection and UDP tunnel. And there's a separate part, which is a quick connection with H3 connections. So the UDP tunnel is established between client and proxy 1 and proxy 1 and proxy 2. So there are two separate UDP connections. But the quick connection, H3 connections is established between first client and proxy, and then client and proxy 2. Yes, that's correct. Yes, you're right. Mm-hmm. Yeah. Yeah, so you're right. So and for the second question, yeah, that's a very good question. So obviously, you might think that going through those routers, proxies, can increase the latency. But in often not the case that, I mean, even though the packets are going through a geographically longer distance, it may actually get better performance. For example, I think I saw that because Apple Private Relay has been already deployed and they have some data together. And they showed that actually they saw some performance gains in some cases because many of the target servers do not support H3. So they have to actually use H2 or even H1 to connect to an end target. But with this privacy proxies, from the client to the proxy 2, the traffic can be actually delivered through H3. And then only the last half will be delivered to H2. So actually in that case, this can actually improve the latency or even bandwidth. So there are some cases where actually this kind of proxies even provide better performance, even though it goes through the proxies. Oh, sorry, could I say that again? So yeah, it's a little hard to catch your question. So maybe I can talk to you offline, yeah? Yeah, we have the last part. Folks, it was working really well this morning. We have the question line, the question mic, if anybody wants to use it. You can take that up offline. Any more questions? We're actually out of time. So if anybody else wants to catch up, we can do that afterwards. Thank you very much. Great talk.