 Welcome to the gateway API project update. I'm Nick Young. I'm a senior software engineer at Isovalent And I'm Rob Scott and I'm a software engineer at Google and we're both gateway API maintainers. Yeah, so let's talk some highlights. So, actually before we do that, let's do who here does not know what gateway API is and wants to hear more about it. That's fine. Don't be shy. Yeah, cool. Okay, awesome. So, like, I'm actually surprised when I was cruising pretty low. Cool. So, gateway API is the Kubernetes network, Sieg network sub-projects that is aiming to build out a new... Let's see if I can get this all right. A portable, extensible, role-oriented replacement for Ingress and service-type load balancer amongst other network constructs. So, yeah, that's what we're aiming to do. Things are going all right, I guess. Yeah, you're all here. We must have done something reasonably all right. And as you can see from the highlights, yeah, we got like about... things have been going great. We got 140 plus contributors, more than 20 implementations of the API. Many implementations are fully passing the conformance test suite. And we've got at least these four cool integrations with the API. And then we've got heaps of implementations. Yeah, I see a number of familiar faces who are responsible for some of these logos. So, nice work, everybody. Thank you very much for making this whole thing work without implementations. There's no point in having an API. So, yeah, our most recent thing is that we did is released V06O. We brought reference grant to Beta and removed reference policy. Reference grant is a resource that's designed to allow you to more securely make cross-names-based references. We'll talk more about that later. We introduced the GRPC route object and we updated status. We added new conditions, accepted and programs. And we moved ready to extended conformance. And then, yeah, we're going to move that again. And we added... we deprecated gateway schedule, gateway listener to attach and a couple of other ones. Basically, we just did a big cleanup of the status conditions to make them more consistent across all the objects. And we did heaps of conformance test updates. Continuing on that theme, 070 is going to come out real soon now, TM. Yeah, hopefully just up to KubeCon. As you might imagine, KubeCon tends to make it hard to get releases out. But so, yeah, we're in the final stages of API review. We're targeting early May. It includes policy attachment update that I'm going to talk about in a bit. There's a couple of features graduating to standard that Rob will talk about. We've formally defined the interaction with the multi-cluster services proposal. Again, we've added heaps more conformance tests. And, yeah, the ready condition is now deprecated and reserved. Don't use it, use programmed instead. There is a whole discussion there. If you're interested in it, like I can lead you through it afterwards, but we've got a packed agenda today, I cannot afford to take any... any tangents, right, Rob? Okay, so before I go, does anyone have questions about anything so far? I'm going to do a couple of questions, stops along the way, just because... Okay, policy attachment. So policy attachment is a new thing we're building in Gateway API that allows you to use what we call meta-resources to affect the way that some Kubernetes objects will work. A meta-resource is a Kubernetes object that attaches to another Kubernetes object, and then the controller is responsible for reconciling them sort of are required to modify the function of the attached object. A policy attachment is a meta-resource that uses a specific struct, this one here, the target ref one, and has a defined behavior. And so they can be either direct or inherited. This is the thing that I just changed in a recent get. So a direct policy attachment attaches to a single object and modifies its function. Aside from that, you can do anything you like. And the intent here, sorry, I didn't make this clear, we expect implementations to make their own policy attachments. That will be implementation-specific support. But it seems likely at this point that there's probably going to be an example, at least one example policy included with the Gateway API CRDs, probably going to be time-out policy, we will see. But yeah, so the other example is, we're thinking about maybe doing a back-end TLS policy to describe the required TLS settings for a gateway to connect to a service that's doing its own TLS. So say, hey, you need to use this CA, you need to use this TLS version, or some other stuff like that. So the idea here is to have a way that you can attach that. In this case, back-end TLS policy and direct policy attachment are required because the logical place for all of that stuff to live is in service. And we all know that service is way too big and way too busy and objected to, and it's going to take a long time to make any changes to it. So the idea here is to have an extra object that you can attach onto the side and all the controllers will need to learn how to read it, but then that will enable this new functionality. Okay, so direct policy attachment is the simple version. The more complicated version is inherited policy attachment. This is the original policy attachment design for those of you who've been around for a while. It allows an object to attach to a point in an arbitrary hierarchy and have settings flow down the hierarchy. There are two different types of settings. You can ways you can have settings flow down the hierarchy, defaults or overrides. Defaults is pretty self-explanatory. You can set a default setting at a gateway level and then have that setting apply to all the HTTP routes that attach to that gateway, for example, or even a gateway class. And then you can also override. I'm hoping that people will use this sparingly, but you can have an override to say, this setting applies to all HTTP routes and you can't change it. As an end user who owns an application developer who owns a HTTP route, we get to set this setting and you can't do anything about it. So the idea here is to have a escape hatch for admins to be able to be platform admins to say, no, you definitely can't do that. For timeout policy, the most obvious one is, hey, I'm running a shared proxy. I don't want you to leave your client to have an infinite request timeout for every HTTP connection. That's the sort of thing that I want to see people using override for. There's a couple of example hierarchies here. The obvious one is gateway class to gateway to a route. But you can define it. It works on any hierarchy as long as the hierarchy makes sense and you can define it. So namespace service endpoint also works as a policy attachment hierarchy. Same space at a service or at an individual endpoint and have the settings flow down. Okay, but the biggest problem with this at the moment is discoverability. Right now we've got a mechanism to attach things to other things, but we've got no way for you, the owner of the thing that got attached to, to know what is attached to you. Obviously, it's a huge problem. So the design is not finished yet. We are working on what we're going to do about this. But because it can affect any object, we can't say, oh, you have to have this thing in your status for the object because a lot of the time we're not going to be able to do that for status object, for statuses. And so some of the API reviewers suggested a separate resource which we are going to look into. But yeah, we're still working on this problem. If you have ideas about this, I'd love to hear them. But yeah, with that, I think I'm going to switch over to Rob. All right, hopefully everyone can hear me. Great. So we've got a few new things coming in 0.7.0. And what I'm going to focus on is things that are graduating from what we call our experimental channel to our standard API channel. That standard channel means that it's going to be available in most implementations. It's generally considered a stable API at that point and something you can build on top of and use in production if you want. So first up, we have a GEP. And you're going to see a lot of GEP numbers throughout this presentation. So get ready for that. But GEP 7.26 is what we're starting with. That means gateway enhancement proposal. And then number 7.26. And this covered path redirects and all of rewrites. So what that means is first, we already have redirect capabilities in HP route. But we wanted to allow the idea of redirecting to a specific path. So I've got an example over here. And this example shows that right now we're doing a path prefix match on slash cayenne. We're going to replace that prefix match with paprika. And so in that case, you can do a redirect from cayenne to paprika. And that's going to be a 302 status code. Now rewrites, on the other hand, very, very similar syntax. But again, rewrites are slightly different than redirects in that they're not going to go back to the client. They're just going to go on to the back end. And in this case, we're saying we're going to rewrite the URL from example.com to example.net. And we're going to replace the prefix match again, but this time to fennel. So this is a gap that in 070 is graduating standard. Next up is response header modification. This is a gap that's pretty aptly named because it allows you to modify response headers. This is we already have modifiers for request headers. This was a very notable emission. Works the same way, but allows you to modify response headers. All right, so next up, we have this large gap, 1748, that talks about multi-cluster services and gateway API. I'll take a brief pause here. How many of you have used multi-cluster services before? All right, pretty small number. How many of you have used gateway API, just for a reference point? Okay, more. So this is talking about two of the newest Kubernetes networking APIs and saying how can we use these two things together? We call the combined concept multi-cluster gateways, the idea that you can do advanced routing across multiple clusters. There's a gap that talks about all the various details, that there's a lot of nuance here in exactly how implementations should handle multi-cluster services and the interaction with gateway API. So in this really simple example, we have an HTTP route, and that HTTP route is sending all right requests, so anything with a method of post, put, or delete, to our API primary multi-cluster service. And then any other request, regardless of method, is going to get sent to our API replicas. So you can imagine a read-write replica. This is one way you could do that. Then finally, conformance profiles. We keep on mentioning conformance. It's a really huge deal. I can't mention this gap without talking about the author, which is Shane right over there. Thank you. Shane is another gateway API maintainer. He's led the effort on conformance profiles. Conformance is a huge deal for this API. There's no implementation by default. We have 20 different implementations, and we're trying to make sure they're all providing a consistent experience. So conformance tests are a very big deal, and we're starting out with them from the start. So every conformance test now is gated by these flags. So in this example, HP route is one of our most commonly used resources, and there's a lot of features within it that not every implementation is going to be able to support. So what we're working towards is the state where every implementation can clearly state which feature they support and then have the test passing to back that up. So two things. One, we want to centralize the reporting of this so you can see in one place which implementation support the features you want. And then at the same time we want, you know, via gateway class or something like that for implementations in your cluster to be able to report the features that that specific gateway class supports. So this is a big effort. Huge thanks to Shane for leading this one. Another big change coming in 070. Okay, so yeah. You can see we're closing in on 1.0 as Shane always calls it. We are on the road to GA. And so what one wanted to talk about briefly is what we're thinking about here. So we really want to get this API to GA. We are targeting October. Yes, that means that ideally we will have GA before KubeCon Chicago. That is the plan. Well, maybe it's a dream at this point. I don't know if it's a plan. But anyway, regardless, that's what we'd like to have. So the scope is gateway class, gateway and HTTP route. Those are the current objects that are in beta. Yeah, and the other stuff we want to add is the centralized conformance reports, automated vulnerability scanning for the validating webhook that we provide. And we want to, and migrating to the production Kubernetes registry so that you don't need to use the, for that webhook as well, you don't need to use staging.kubernetes registry. You can use the actual production one and we have the same production readiness stuff that any other part of Kubernetes does. So yeah, this is a pretty big deal. We're working really hard to make sure that all of this stuff is all lined up and that all the details are sorted so that we can, with confidence, give you an actual 1.0 API that you can rely on the same as you would in Kubernetes 1.0 GA API. Okay, yeah, so let's talk a little bit more about what we're doing in conformance. Yeah, like, Gateway API has tons, tons of features, right? Like, at least five times as many features as Ingress already, you know, and we're not even close to being done. So we've got heaps of implementations and again, as I said at the start, our three primary goals for this APR are portability, extensibility, and role-oriented, right? Portability is the number one one for a reason. The only way that we are going to be able to have so many implementations and to be reasonably confident that you as a person who is using Gateway API can pick up your config and move it to another implementation and have it work is to have a really, really, really solid set of conformance tests that validate the behavior you are asking for is the behavior that you get with any implementation. So, yeah, and so that's what we're aiming for. We want you all to understand which implementation support which features so you can make a choice about which ones are important for you and what other trade-offs you want to do. And longer term, we want to be able to have a certification program similar to the Kubernetes certification program. So implementations get to have a badge that says, we are Gateway API compliant at this version. So that's what we really want to do in some glorious future where everything we've got today is GA and stable, you know, and Robert and I are sitting on a beach somewhere clinking a cocktail. You know, the, you know, we can have, you know, people can have badges that say, you know, Gateway API certified path of conformance. That's the sort of the dream. So, but how we do that? Well, let's talk about the conformance levels. So, you, thanks again, Rob, for this great diagram. The idea here is that, you know, we have three conformance levels, core, extended, custom or implementation specific. Core is every implementation that does that thing has to do it and has to do it in a consistent way. Extended is the thing is optional, but when you do it, you have to do it in a consistent way. Both of these things imply that there are conformance tests to ensure that implementations do it in a consistent way. And then custom is, you know, hey, implementations can do this however they want. We usually use this to mark things as we can't or won't make sort of common grounds here. And so it's kind of optimum implementations to do, to do what they do. It is definitely anticipated that over time implementations may do custom things, that then other implementations do something similar or the same and that thing will then move into extended. And for things that we find, extended things that we find everybody is doing, then it can move into core. Okay, like so this is not fixed in stone. This is intended that there's a path for things to move sort of closer into the core over time. So as of today, we've got over 140 conformance tests. Most core and extended features for gateway class, gateway and HDP route are pretty well covered. Conformance tests are required for all new features. And we also have an initial gamma conformance test. We'll talk more about gamma in a bit. So yeah, soon we want to have the centralized conformance reporting that we talked about before and conformance profiles. Now right now, the conformance tests are sort of defined as literally a list of the features that you support, which is essentially the test that you run. So what we want to do is to have that, to make that a little bit easier, we want to have profiles that say, okay, you support HDP. So that means that you support this set of features. And we can change the set of features over time as we have releases and stuff, but we want to have people have a short hand. So you don't need to say, my implementation supports blah, blah, blah, blah, blah, blah, and have like 20 things. We want HDP, TLS, Layer 4, Layer 7, something like that. I don't know. We haven't decided exactly what they're going to be, but we want to have it be simpler than a huge list of tests. And the other thing that we want is obviously full gamma conformance test coverage, the same as we have for North-South direction. Okay, that's speaking of gamma. So who has heard of gamma before? Hey, awesome. That's good. That's good. So gamma is the gateway API for mesh management and administration. The full name is actually the gamma initiative because we thought that sounded like a really cool name. Yeah, so I think, I certainly had been watching a few Marvel movies at the time. So it aims to describe the mesh use case like that is East-West traffic using new permutations of the existing gateway API resources. So those words have chosen very carefully. The idea here is to take the existing resources, not have to change them very much, and then put them together in new ways to be able to describe East-West traffic as opposed to the current implementations which are all about North-South traffic. So some implementations already have support for the initial sort of prototyping API. And there's an initial conformance test suite in the API repo. Yeah, but this is a very, very big topic that we just do not have time to go into today. So for more information, there's View Gamma Talks, View Gateway API Mesh Talks during KubeCon. Come to those, have a chat, or come to the gamma meetings. The QR code there takes you to the page that tells you more about the gamma meetings. Over to you, Rob. All right, Reference Grant. This is a big topic, a big focus going forward for Gateway API. Now, Reference Grant, for those of you who may not remember or be familiar with it, enables a really new concept in Kubernetes, and that's the idea of crossing namespace boundaries safely. So in Gateway API, we use this for two different purposes. The first purpose is you might want to reference a TLS certificate in a different namespace. For example, you might want all your TLS certificates to live in a single namespace. You can do that with this. The Reference Grant would sit in that namespace along with your certificate and say, I trust references from this gateway or this route to my secret. The same thing applies if you want to forward traffic from a route in one namespace to a service in another namespace. You have your Reference Grant sitting alongside your service and that's basically your way of completing a handshake that says this reference is safe. So Reference Grant has been a concept we're really excited about. We've started to open the doors to crossing namespace boundaries but doing it safely. So with that said, Reference Grant has actually made it to the standard channel. That means it's widely available, widely supported. We have good conformance test coverage. But one of the things we were hoping would happen actually happened. When we saw the concept of Reference Grant, we thought, well, this really makes sense beyond just Gateway API. So we took it to SIG Arch, SIG Auth, those kind of SIGs and we asked, hey, does this make sense outside of Gateway API? They said, starting Gateway API and come back when someone else is interested. Well, finally, someone else is interested. So we're there. Thankfully, SIG Storage has started to use the same API for cross namespace volume snapshots. Really excited about this feature. But that meant it was time to go back to SIG Auth and have that discussion all over again. So that's where we are and this has turned into a bit of a rabbit hole. But first, there's some things that we broadly agree on. This is going to go to SIG Auth. That's something that we know this is going to go to upstream. It's going to be included in Kubernetes. We have a provisional cap merged. There's some changes that most all of us agree on. So one idea that was hard for us to do just as CRDs was, hey, users should have read access to the things that they're granting access to. That's a nice thing. That's the kind of requirement that SIG Auth would make and it makes sense. Then we, you know, in reference granted Gateway API we kept on leaving room for the status. That means you have a spec and a status. And we could never find a use for it. We wanted to leave room for it and when pushed, we never found a good reason for it. So that's going away. Then also, you know, in Gateway API we used kind for references because kind is something that you can use when you know a very finite set of kinds that you're going to reference. And we have two that we support. So we could do that. But if we move this to SIG Auth this is going to be a much broader thing that can be used anywhere. And resource is a more specific term than kind. It's a really confusing API machinery thing because we have multiple kinds that are identical within a cluster. So resource is the way to go. Next, of course, Gateway API. This will move from Gateway API ownership to SIG Auth. And then finally, we want to have a shared library area that can simplify this implementation because there's a lot of work to make all this work so let's have some kind of shared library to make it all fit together. So those are the things we agree on. Now we're going to start to go off the rails. All right. So we have a lot of open questions here. Some people have suggested, well, you know, if we're going to make this a generic API what if we had verbs? So instead of just, you know, again in Gateway API we knew very clearly you're granting access to this secret because it's a certificate reference. But if we're opening this up to the rest of Kubernetes you may want to grant access for a specific purpose. That's the idea of verbs. Then there's a whole bunch of what if we had wildcard selectors, label selectors, resource aid. I don't know. There's a lot of questions here. So that's the state of the cap. But more recently, here at KubeCon there's been a real rabbit hole. So some of you may be familiar with the idea that many or most ingress controllers that you'd ever run are going to watch all secrets in your cluster by default. It's not a great state, but it is the state of API. It's not a gateway API as well. It's just kind of the default behavior. There's no other way to do it. You need to be able to access an arbitrary secret at an arbitrary time without anyone needing to take any other action other than referencing the secret. Right now there's literally no way to do it. So some optimize and they say well we're just going to watch secrets in this namespace and you can configure that. And that also works with Gateway API. But still it's not an ideal state. And then second, one thing that we've already alluded to reference grant is fairly complex to implement and although we have conformance tests it still requires every individual implementation all 20 of them to do all that logic themselves. And if we're saying we're going to bring this back into core that's going to increase the number of implementations. So there's a possible solution here but this is rabbit hole theoretical at this point. Very early days. We talked about it yesterday. Don't put too much weight in this. No guarantees are made. If you're interested we could definitely use some help getting this through. But one idea is can we tie this together with RBAC in some way so that a reference grant not only allows you to make these cross namespace references but also is the thing that grants access to your controller. So can we adjust this enough so that this solves both problems? Gateway controllers would then be granted access to impersonative resource like a gateway or an HP route. And then the API server would then need some significant updates so to allow that you only watch resources you have access to if not possible today that would be very cool. And then again the idea that you have a way to understand when access is denied. So a key concept with reference grant is that we revoke access and we expect that change to happen immediately. So if you allowed a cross namespace forwarding to happen we'd expect that to stop as soon as the reference grant went away and we have conformance test coverage for that. That doesn't work with the RBAC model today. So big rabbit hole. We've got to keep on moving but if you're interested there's lots of ways to get involved. I think this is mine still. So GRPC route is another thing that we're working on. We're just going to do rapid fire here because there's a lot going on in Gateway API and we only have so much time. But GRPC route is one of the routes that is closest to graduating from experimental to standard channel. It's pretty straightforward. We have multiple implementations already. The thing that's held us up so far is conformance tests for GRPC are different than any of our other conformance tests. So it's taking a bit more work to get those in. But working on it. Blix, again, Shane over there. This is Shane's project. I mean everyone's project but Shane is the one who started it so thank you. But this is a layer for reference and testing implementation of Gateway API. It uses all the cool technologies, Rust, EPVF, Go and control our runtime for the control plane. Lots of cool things happening here. And what we're hoping to do here is to drive the L4 side of our API because we have tons of implementations covering L7 right now but we're looking for something to kickstart the L4 side of this because this is not just ingress replacement. This is L4 load balancer replacement too. There's a lot of work. We're migrating to SIG network. I've got to keep on moving though. So go ahead Shane. So we've got heaps of GEPs still in early stages. Service mesh. Session persistence. Cluster local gateways. Mechanisms to merge multiple gateways. Timeouts. In-cluster gateway deployments. Per gateway infrastructure. TLS from the gateway to back-end for ingress. So there are a lot of features that are sort of these are all in sort of provisional or open PRs discussing them stage. There are obviously a lot of changes mooted as part of this. Part of the point of this slide is just to be like look how big that list is. There's lots of stuff I promise. So yeah. So I think that's sort of the message we want you to take away here. If any of those features interest you we desperately as always need feedback from people who will actually use these APIs. If you are going to use one of those features please feel free to find a GEP and comment on the PR or open an issue asking for clarification asking the SIG network gateway PI channel on Kubernetes Slack. We really need use cases and people's feedback on how we're planning on doing this. Otherwise we end up building an API in an over tower and no one is abusing it. Okay. So the rest. So TLS TCP route and UDP route are progressing like Rob said and Shane is very kindly working on we need more layer 4 conformance test to be able to do anything with those and more implementation to support it. So if there's any problem there for implementations it's hard to want to support it if there's no test to validate you're doing it correctly but at the same time someone's got to do it first. TLS route however is probably in a bit better state there are some conformance tests for TLS route we want to see more more you know more clarity there and more implementation supporting it. Ingress to gateway is our tool for automating auto converting ingress objects to gateway API objects for you it's nearing it's initial release there's lots of great PRs open to stabilize and restructure the code and the whole idea there is we want this is an open source tool we want people to be able to be like I really need these annotations because I've got 7000 ingresses that use this annotation and I want to be able to not have to manually type them all in. So yeah the whole idea is we want people to come to us and say I really need this annotation let's figure out how to address the gateway so then this tool can really help people move away. Okay Rob you want to? Yeah. Alright so first up there's a lot of gateway API content at KubeCon this here so if you're interested in any of the things you saw today are interesting check out one of these other talks there's a lot of people doing really cool stuff with gateway API. So I'm just excited to see so many interesting things happening definitely recommend checking out I think some of the speakers are in this room actually so yes I'll leave this up for just a minute the other thing I want to say is we have tons of opportunities to get involved in this API we've been working on a contributor ladder so that it's easy to get a role in the project a formal role and we want to recognize all our contributors this takes a lot of work it may seem like our updates are fairly incremental but it takes a lot of work to get to where we are so many and all help is very much appreciated we have weekly meetings for the overall project on Monday we're experimenting with Europe friendly times so if you're interested and you're based in Europe definitely reach out on Slack whatever we're trying to find times that work for as many different time zones as possible then Gamma is our mesh focused initiative that we meet weekly on Tuesdays then we have a fun co-gem on Fridays I think that's Europe friendly ish yeah and then contributors from all backgrounds are welcome we need heaps of docs updates a lot of this is writing a spec so any technical writers like I would love to talk to technical writers about making sure we made this stuff more readable and understandable for people who haven't spent the last four years developing it you can find us online on Slack on GitHub please reach out we'd love to talk thanks so much for your time so do we have time for any questions yes we do alright we got a question over here alright yeah anyone else who wants to ask questions bang those hands in the air can you say something about session persistence GEP 1619 like what the status is when do you think it will be finalized or things like that what does it look like we don't know yet basically please build session persistence we don't know yet session persistence is one of my favorite gaps for how it was structured first it's really a model on how we want gaps to work first it was just an overview of all the options that exist today then it was I think that's the state we're in right now of here's the common ground that exists here's the things that are configurable and most common implementations and then from there the next step is what API can we build that most implementations are going to be able to support and that's the stage we're at right now so hopefully soon but that's the latest update I have I think that's Grant Spence is leading that effort yes thanks any other questions there's one over there Shane ok hit that one first then we got thank you for your presentation I just have a question about IPv6 are there any issues related to IPv6 so there are none that I know of I think it's all going to come down to implementation stuff we don't have much conformance testing yet for IPv6 but there is nothing sort of restricting we've specifically tried to allow for IPv6 as far as possible the one thing where it's sort of directly relevant is there's an address that's struck inside the gateway that allows you to have either an IP address or a domain name and IP address can either be an IPv4 address or an IPv6 address so right now there's nothing in the IPVI that's stopping you from doing IPv6 but we don't have good conformance coverage that we can validate things cool hello thank you for this presentation in fact a lot of quite of insight about about in the first hand they are public FIIs they are generally REST based HTTP and in the other corner in the clusters there are more and more micro-services with GRPC so it's a question about GRPC routing, ingress to gateway so the REST throws coding to GRPC are very blue in the gateway implementations sometimes there are plugins very small plugins sometimes it's not clear so what your insight about this thank you yeah really quickly that we've had really great help from actual GRPC team members coming into gateway API and contributing and working towards GRPC route I know Richard Belleville specifically is someone who's been leading that effort and they've got some really exciting ideas to expand our existing GRPC route and get it more stabilized certainly if you have any specific things that are missing from the route if you'd like to see included definitely reach out Slack, GitHub, wherever but that is a key focus area for us and it's been great to have collaboration with actual GRPC team members. Hi thank you for the presentation it's the first time I hear about the multi-cluster setup and I was wondering about in that architecture which cluster receive the request is there any restrictions about like one cluster has to receive the request and then forward it to another cluster or if there's like a third cluster that receives the request and then forward it to other clusters or anything that's a really good question shameless plug I'm doing a talk with Lee Nguyen from AWS at 5 end of day today about that so far the only multi-cluster gateway implementations I'm aware of are cloud providers so they're providing they're basically programming a global network or something like that and so it's not like an individual cluster doing a little balancing it's some kind of cloud product on top that doesn't mean that has to be how it is that's just the current state of the world I'm familiar with yeah I mean I'd certainly love to see more work done in that area myself yeah anyone else cool yep last one though and then we're gonna have to call it sorry everyone thank you for the talk is there a plan to include regex groups or capture groups in the HDTP route redirects yeah so we've talked about this a couple times one of the difficulties here is that you say okay I want regex it's like which regex and then when you pick a regex flavor you might be rolling out specific unless you pick one of the really simplified ones you might be rolling out a lot of implementations so the reason we've left regex on the table for now is just it's really hard to sort of pick flavors and then make sure that and then make it portable that's not to say we'll never do it but like it's just you it's not very hard at the moment because we've got other stuff to do so yeah if you really want regex then I encourage please come and you know get started and make a gap and we can talk about you know how we can make this work but yeah the sort of the TLDR is it's really hard to do the costs the sort of portability that we need for this API we definitely have had future requests for that specific thing I would love to make it work it's hard to make it portable like Nick said but if we can find a way then we'll totally do it yeah we'll follow up we've got to leave this room but we'll follow up in the hallway if anyone has additional questions it looks like the hallway is pretty busy so we'll head like