 Hello, everyone, welcome. I'm so excited to see you all here at our presentation. Persistent pays off the path to station persistence in Gateway API. I know it's been a long day, and I'm probably standing between you and your dinner. So huge thank you for sticking with us at this presentation. For those of you who might be new to this concept, session persistence is a way to ensure that a user's requests are consistently routed to the send back end throughout their interactions with a website or application. And this can be crucial for applications that need to track the user preferences, the states, or even like shopping carts. And it was the top requested features for Gateway API at last year's KUCOM as Chicago, so demonstrating that's how important it is to this community. We are incredibly excited to share the latest updates on the station persistence and how this is going to be integrated into the Gateway API. My name is Gina Yee. I'm software engineer at Google, leading the design and the limitation efforts of session persistence in GRPC route and HTTP route. And today I'm presenting with Grant. Today I'm Grant Spence, I'm software engineer at Red Hat, and I've been leading and championing the session persistence gap with Gateway API. So let's take a few minutes to talk about Kubernetes Gateway API before diving into session persistence. The Gateway API is a significant evolution in how we manage network traffic in Kubernetes environments. It offers a powerful set of features to address the demands of the modern microservices architectures. And the Gateway API designs several distinct rows for servicing different needs for different users. It also provides a standardized way to ensure that your configurations will work seamlessly across different Kubernetes environments and network infrastructure providers. The API gives fine-tune traffic with capabilities like header-based routing, weighted traffic split and advanced load balancing. And the Gateway API, primarily known for its north-south traffic capabilities, is involving to support east-west use cases with the surface meshes. So that is, it can be used to route the communication between surfaces. And the route will attach directly to the surface, representing the configuration, mean to be applied to any traffic directly to the surface. The Gateway API is also seamlessly integrated with the custom solutions, enabling the use of the specialized load balancers and our surface meshes. Several key APIs of the Gateway API graduated to GA October last year, including Gateway, Gateway Clause, HDB route, and other significant resources are under active development and available in the experimental channel. And we are planning to raise a JRPC route to GA in the April release. And Richard from my team is going to have a JRPC route talk at 2 PM Friday. So if you are interested, please join us to learn more details about JRPC route. Gateway enhancement proposals gap defines a process for proposing changes to the Gateway API. And it ensures that changes will follow the center process and are discussed in the open source community. Can you help us to check? Thanks, everyone. That's very, that's very warm. So let me start it over. Gateway enhancement proposals defines a process for proposing changes to the Gateway API and ensures that the changes follow the standard process and discuss in the open source community. Gap 1619 is a proposal to enrich Gateway API with session persistence to ensure that the user's requests are consistently routed to the same back end throughout their interactions with a website or application. It is currently under provisional state and we are very close to move it into the implementable state, which Grand will cover in a few minutes. Discussions about this gap has been very, very intense in the past few weeks. As you can see from the screenshot, it has more than 100 comments on the PR. And there were lots of great discussions with a real sense of collaboration and excitement about session persistence. And it is so great to see many people actively participating and sharing their ideas. And the most exciting part is the PR was merged a few days ago. So in the next 15 minutes, thank you. So in the next 15 minutes, you will get the first hand updates about session persistence. And with that, I'm handing it over to Grant to talk about what session persistence is. Right, thanks, Gina. First, let's talk about what session persistence is. As Gina kind of introduced the topic already, let's define the official meaning of session persistence as it pertains to KWAPI. The first thing that we really want to do with a lot of the gaps in KWAPI is that we want to establish a common language, just so we're all talking about the same thing, just to make things clear in communication. So what is session persistence? It's about making sure that when you use a website or an app, your request is going to the same server for a certain duration or a specific amount of time. How can you achieve it? Clients can achieve this by providing identifying information like a cookie, which is used by a load balancer to route traffic to the same server. And you can also think of this as kind of an exception to load balancing. It bypasses a proxy's load balancing algorithm and goes directly to the back end that it's already previously established a session with. So unfortunately, session persistence is not a standardized name. You guys probably are aware of that. Some people call it sticky sessions. Some people call it session affinity. But here, we're not trying to really enforce some sort of global standard here, but we just really need to establish some sort of naming convention or a common language for the foundation of this gap. OK, let's define another term, which might be a little confusing, but hang on. Session affinity. So the best way to define session affinity, as it pertains to Gateway API and our APIs, is to compare it to session persistence, which we just defined. So as we're defining it here, session persistence provides guaranteed or mostly guarantees back end persistence to the same back end server. Whereas session affinity is work on a weaker form of session persistence. So strong versus weak. Next, we are defining session persistence as it's deliberately provided attributes on later seven. So you can think of a cookie or a header. Whereas session affinity is based on any existing connection attributes, such as like a client IP address. And the gap that we're working on in 16.19 is defining support for session persistence, but we are leaving out session affinity. So the idea here is that we kind of establish this common language so we could decouple these APIs. So we can tackle session persistence specifically in this gap and leave session affinity for another time. So as you might already know, Kubernetes already supports session affinity in the service objects. And that's spec.session affinity. But right now, we aren't supporting compatibility with that with GEP 16.19. Might revisit that later. But right now, it doesn't support enabling our session persistence and Kubernetes session affinity at the same time. So how can you achieve session persistence? Do it in multiple ways. The most common way is through a cookie. Like I mentioned earlier, the client uses a cookie HTTP header to response header from a gateway or a backend in its future requests. And the value of this cookie encodes some information about a backend that the low balancer will use to create a consistent session to a specific backend. Next, headers can also be used to achieve session persistence in maybe situations where cookies aren't supported. Here, the client uses a predefined HTTP header provided to the gateway in future HTTP requests. Like cookies, headers encode some sort of value which the backend uses to look up where the requests should be routed to. Lastly, URL encoding. They can also be used for session persistence. The server automatically rewrites the URL to encode the session information. You might see this with older job applications and specifically with that J session ID name. Next, I'll hand it to Gina to talk about how we initiate session persistence. Okay, thanks, Rand. The initiation of the persistence session is possible from various sources. From the global low balancer, regional low balancer, internal gateway, sidecars, or any component before reaching the backend. Here, I want to show, like start with a very, very simple example with a gateway and a few backends. And a typical flow of the requests and response looks like this. The client sends a request to a gateway which has the route resource attached to it. And based on the route rule, the request is forwarded to the backend one. And then the response is then returned to the gateway and then send it back to the client. And know that in this example, backend one does not have session persistence enabled. So the next request can be routed differently. And next, that's like a look, a take a look at how a gateway can initiate a persistent session with cookies. With a client send a request, the gateway routes the traffic based on the route rule. And this time, the request is sent to backend two, which has a persistent persistence enabled. And then the gateway includes the set cookie header, SID one, two, three, in the response to establish a persistent session. After that, the client can store the cookie and send the following request with the cookie, allowing the gateway to decode it and look up the SID and consistently routes the request to backend two to form a persistent session. This diagram outlines a very simple use case of a gateway initiating a persistent session by inserting a cookie in the response header and a client application applying the cookie in the headers of the following request. And one thing that I like to call out is as Grand mentioned earlier, cookie is just one of the mechanism to implement persistent session. And implementations can decide how you want to achieve it. Now let's take a look at another example of how a backend can initiate a persistent session with cookies. Similar to the previous slide, the gateway routes the traffic to backend two, which has a persistent persistence enabled. And this time, the backend includes a set cookie header in the response to establish a persistent session. And then the gateway may modify the cookie with a reference to a specific backend so that it can update the following request. It can route the following request to send backend. There are several approaches for the gateway to achieve it. And in this example, it adds an additional cookie BID to reference the backend ID. So when the client send the following request with a cookie, the BID in the cookies allows the gateway to consistently route the traffic to backend two. And all that in the gap 16.19 does not define details about the backend initiated sessions because it can be very, very complicated. So it is intended to explore this in a separate proposal. So now we have defined what the session persistence is, introduce how to achieve session persistence and explore a few examples of how it can be initiated. And now let's review how the API looks like. All right, so before we dive into the API that we're working on right now, let's review some gateway API definitions. So meta resources and policy attachments. While designing the gateway API, we've found that we often need to change an object's behavior without changing their spec. That's where meta resources and policy attachments become very useful. So meta resource is a class of objects that only augments the behavior of another object by attaching to it. And then furthermore, a policy attachment is a type of meta resources that uses a specific target ref in the example here and has a defined behavior. It can be a direct or indirect type of policy attachment. Here in this example, example TLS policy is a hypothetical direct policy attachment that augments the service to add a CA reference. Sorry, I meant to say inherited, not direct. It's direct or inherited. So a direct policy attachment is a policy that references a single object only and only modifies the fields for that specific configuration object, or that configuration for that specific object. While an inherited policy attachment is a policy that references a single object as well, but it can modify the configuration for any child objects of that object that it's modifying. And any configuration associated with those child objects. So if you want more information about direct or inherited policy attachments, I recommend reviewing KEP 713. There's also a PR that Nick has up that is making some substantial changes to that. So look for a PR related to that. Okay, so when trying to design the Session Persistence API, we found ourselves needing to augment the Kubernetes service object, or more specifically the traffic bound for the backends. A new direct policy attachment that we're calling back-end LB policy, and LB is for load balancer here, enables us to augment a service without changing the service API. And there's a delicate balance here on how generic or how specific these new policy attachments that we're designing should be. Having it too specific can lead to lots of CRDs and proliferation in that matter, but having it too generic can be kind of confusing and lead to monolithic API objects. For Session Persistence, we currently landed on back-end LB policy as an abstraction for configuring load balancing configuration for back-end traffic. So in other words, we anticipate that other load balancing related configuration will be added to this API. For example, load balancing algorithm section, which is not currently a feature in Gateway API, so that this structure kind of paves the way for that to be included. Attaching Session Persistence configuration with a back-end LB policy to a valid back-end enables it for any route that is directing traffic to this back-end. And so a back-end here can be a service, it can be a service import, in the case of multi-cluster scenarios, and it also can be any implementation-specific back-end object reference. Another big note here is that this API is provisional, so we are looking for feedback and we do expect some things to possibly change. And additionally, we also had a desire to add Session Persistence at a route level, because some implementations may not support it at the service level, and at the same time, it can be quite confusing to attach Session Persistence at a service level for certain scenarios, such as like traffic splitting, things get quite confusing in that scenario. So the API for this route level configuration exists within the route rule structure, so it's specific to each route rule. We also plenty support HP route and GRPC route. These are the layer seven routes that we currently have. Currently it uses the same API fields as back-end LB policy, so you can figure out the service level, you can also configure out the route level. And you might notice we brought up the idea of an inherited policy attachment earlier, but this API design currently isn't one. The idea here is that the inline route configuration in the back-end LB policy will work together, similar to the concept of an inherent policy attachment, but we're able to keep back-end LB policy focused on a service. And Gina will talk more about how these two APIs interact later. Okay, so let's talk about the specific API fields that we currently have. In this current state, we have about six fields that you can configure. Keep in mind that this API is provisional, which means it's probably likely to change. So first off, we have the session name variable. So this is a generic configuration for the session name. We've kept this field kind of loosely specified to leave some room for interpretation. Implementations may use it to indicate the cookie or header name, depending on what type of session or since they are using. So next, the type field, it specifies the type of session persistence. We currently have cookie or header at the moment. Next, we have a API Shrots cookie config, which is specific to cookie configuration. And inside that, we have a lifetime type at the moment. This specifies whether a cookie is a permanent or session cookie. And if you're not familiar with what a permanent or session cookie is, it's pretty simple. A permanent cookie uses the expires or max age attribute. While the session cookie expires when a client has chosen to expire that session. Move it on, absolute timeout is the, when the lifetime type is permanent, like I mentioned, it's gonna be the expires or max age attribute. But when it's a session, it's gonna specify the timeout, the absolute timeout as tracked by a different means, probably by the gateway. And then lastly, we have the idle timeout, which makes the session invalid after it's been idle for a specific amount of time. The other big note here is it's provisional and also all fields are optional at the moment. Thanks, Grant, for walking us through how session persistence is integrated into the gateway API. And in the next few slides, I want to go over a few scenarios of different configurations and what the expected API behavior looks like. As Grant mentioned earlier, session persistence can be configured with a route rule API. And in this example, we have a HTTP route and it defines two different route rules with different paths for the bar. And each route rule has its own session persistence configurations. And they are both direct traffic to the same baguette. So in this case, do the two route rules referencing the same baguette service share one persistence session? The answer is no. Each route rule should have its own persistence session and the persistence session shouldn't be shared between these two different route rules. In scenarios involving traffic split, session persistence impacts load balancing after routing. So when the persistence session is established and traffic split is configured across services, the persistence to one single baguette should be maintained across services. As a result, a persistence session takes precedence overweight when selecting a baguette. In this example, we have a traffic split configured to evenly distributed traffic to service one and service two. However, persistence sessions should be maintained over the traffic split's weight. So the request will be routed to the same baguette which handle the previous request regardless of the weight configuration. Another thing to consider when traffic split is involved, all baguette services should have session persistence enabled and it can be configured with either a route rule API or attach a baguette LB policy to the individual baguette services. Remember that session persistence can be applied with two different APIs. So you can attach a baguette LB policy to a service object or enable it with an inline route rule API. In this example, the HTTP route has a route rule session persistence configuration and it directs the traffic to a baguette which has a baguette LB policy attached with a different session persistence configuration. So which configuration should be used for the persistence session? The answer is the route rule one because the route rule operates at a higher level so it overrides the configuration applied to individual baguettes. And even if a baguette has its own policies for session persistence, the inline route rule API can override it and this is because the route rules has a more global view and responsibility for overall traffic routing. With these three examples that I just covered, I hope it gives you a better understanding of how the expected API behavior and it should cover most of the use cases in the gap. We also document a few age cases that I don't include it here for the interest of time. If you are interested in learning more details, please refer to the GAP1619 and feedback are welcome. So next I'm headed over to Grant to talk about the next step. All right, so let's talk about how we move forward with this gap. So right now we're in a provisional state and our next step is to get into implementable. To do this we need some general consensus on the gap which we've mostly achieved but we're still looking for feedback. We have one more to do item for clarifying behavior between the route rule and back in LB policy API. Optionally we could go to product typing before implementable and this would just simply give implementations a chance to try it out before even going to experimental. And after implementable is experimental which means we need to move the APIs into an experimental release channel and during this time it's expected that new edge cases will come up. As always and API changes are very much possible. And after experimental is standard we need lots of feedback from implementations while we're in an experimental and we also need performance testing for session persistence to be done as well. Additionally we wanna get some review from the GammaFolks which are leading the API and service mesh integration effort. So we have some open questions we still need to answer about the skip. So one of them is session draining as well as session draining timeout. This is kind of an area we still need to explore. Next currently we don't have an API that specifies how to handle existing cookies and by that I mean whether a gateway should modify whether it should insert or whether it should prefix a cookie. Currently we just leave it up the implementations to decide this for themselves. Next there might be a risky situation such as conflicting session persistence configurations whereas adding a status indicator could help could be helpful to end users to identify this. And the question is should we leverage a status in these situations? And lastly like I mentioned in the previous slide we still have some behavior to define with route level configuration and back end level configuration when they're both defined at the same time. Right, Gina. All right, so that concludes our today's discussion on our past implementations persistence in the gateway API. And we love to get some help and feedback on the current design. And you can find it at the short link at top and we have a weekly community meetings on Mondays, game on meetings on Tuesday, cold jams on Friday. All contributors are welcome from all backgrounds and you can join our community Zoom call and send messages on the stack channel. So we have a few minutes for questions. Oh, we have a question here and you will be the next one. Yeah, I think there's a mic. Should be... Okay, he's coming. And the first one, yeah, thanks for a great performance. My question is related to the genus part about the priority of the end the precedence like when we have several versions of our applications and then we have to decide if we apply the policy splitting first or the waiting policy first. From what I see, we have in our company, we use the same strategy that you suggest to use. But our users or developers say to us that sometimes they need to split the traffic according to the weight firstly and then apply these affinity or policy. But sometimes they need the other way. And we decided that we should give them the opportunity to configure that in manifests. Have you considered something like that? Yeah, thanks for the question. This is definitely a good question. As we mentioned, best case scenario, like we have session persistence enabled to close all the service backends, but it's not necessary to be a hard requirement for enabling session persistence, especially during migration, where some services may have session persistence enabled, some may not. So we actually had some discussions on this while we are putting together the design and we came up with three different ideas and we decided to let each implementation to decide what behavior makes sense for their users. We also have the three options documented in the GAP 1619. The first one is you can, if you're seeing like some of the services have this enabled, some are not, then you can decide if you want to reject the policy. That's one of the options. The other one, the second one is you can just apply session persistence to the service object that they configured. But as a result, all the traffic might be directing or routed to the same backend, or to the backends with the session persistence enabled. And that needs to be carefully handled. So on my team, on the Google side, JRPZ route, we actually prefer the third option, which is like we automatically enable session persistence across all the services involving the traffic split. We think that makes sense to make sure like your services are not overloaded and hopefully this may not have any side effects or an active impact to your applications. So like I said, this may only happen during the migration cases or migration time frame. So we hope this is an age case that we can let the user to decide. And we also think it might be reasonable to have each implementation decide what behavior they would prefer. Any other questions? We have one more question here. Yeah, thank you both. Just a quick question. If I want to, like you mentioned that HB route back and HB route take precedence on backend policy, right? So if I want to, let's say I have a backend policy but I want to exempt one route. So is there like, can I somehow set on HB route level? Like this is false. So they're like one route is exempted from cookies. So currently session persistence can be enabled at two different level. One is using the backend LB policy, applying for the service object. The other option that you have is doing that at the route rule level. That should be included in the HB route. And we're also thinking about the third option which is exactly what you mentioned at the route level. But that's sort of like one of the open question or pending item that we haven't explored much. But we think there is a possibility there. But my understanding is already available in the route level, right? HB route level. Actually we route up, but that's inside of the route rule. So in the HB route back, there are several. Yeah, so my question is like, if I have like, let's say I want to slash, I don't know, slash example to be exempted from persistence. So I have a backend policy on my service. My service exports a lot of routes but one of them is slash example. So I want slash example to only be exempted. So I was wondering if this is like something I could do like session persistence falls for this route. Yeah, I don't think we currently have a design for that. That's an interesting use case. I think the answer would be to use all routes. All routes and yeah, don't just make a policy. But then like, you know, if I have, I don't know, like two implementations in the cluster. So then I need to like, you know, do a lot of, it's tedious, is that what you're saying? Yeah, yeah, we should, yeah, we can talk after. I think that's an interesting use case and maybe document that and see if there's a solution there. No worries, yeah. Thanks for bringing this up. I think we are at a time, we can certainly discuss more offline and thank you all for your time and attention. We'd like to hear your feedback and about session persistence and you, we can always find our proposal at the short link on the screen and we appreciate if you have a minute, if you have a few minutes to leave us some feedback about this talk. Thank you, thank you everyone.