 Welcome everyone. It's great to have you all here. You may notice there are two speakers on the slides up front and there's only one of me. Unfortunately Lee-Win could not join us in person today. Don't worry, she was able to pre-record all her slides. She's got a great portion of the presentation today and we have some of her colleagues here who can help answer questions at the end. So thank you everyone for putting up with this hybrid presentation. But I'll get started first. I'm Rob Scott and I'll hand it right over to Lee-Win to introduce herself. Hello, my name is Lee-Win Wu. I am a software engineer at AWS focusing on AWS VPC networking for Kubernetes. My first major Kubernetes contribution was design and the development of AWS VPC CNI plugin for Kubernetes networking over AWS VPC. I am an active member of Gateway API community. Here I'm going to show you how we implemented Gateway API on top of AWS VPC lattice. I'm sorry I couldn't meet you in person. Hopefully I will see you in person in future. Cool. I'm so sad that Lee-Win couldn't be here but we'll have a great presentation. She's got some great content throughout this presentation today. My name is Rob Scott. I'm a software engineer at Google. You can find me all over the place. Definitely happy to connect and talk about any of these topics at any point. Let's get right into it. Let's talk about the APIs here. I may have skipped over it at the beginning but we're talking about two APIs here. We're talking about multi-cluster services and Gateway API. Before we go any further, how many people have used multi-cluster services? Okay, handful. That's impressive actually. How many people have used Gateway API? Okay, a few more. That's great. I'm very biased. I'm one of the maintainers of Gateway API along with Nick, Shane, and some great people over there. We're very interested in both Gateway API and how it could interact with multi-cluster service because these are both brand new Kubernetes APIs that have kind of evolved together. We're talking about how they can work in parallel. First, there's Gateway API. There's a lot of talks at KubeCon about Gateway API. I'm not going to go into lots of details here but understand that a Gateway in this case, when we're talking about cloud providers, which we are today, largely represents a cloud load balancer, then routing would be your routing configuration. We have HP route and a bunch of other protocols represented with different route types. Then the service is really, in this case, just the group of end points that you want to route to. In this case, we have a route for Acme Store and that's doing some traffic splitting between Store V1 and Store V2. This is a pretty straightforward Gateway API example. Now, multi-cluster services are maybe a little bit more complicated to explain. To understand multi-cluster services, you have to understand the concept of a cluster set. A cluster set is a group of connected services. In this case, we're going to say we have an East, West, and Central cluster. We're going to say that we have a service in one of those clusters and we want to share it, export it to the other cluster so they can also access this service. What we do is we create a service export and that service export is going to say to your multi-cluster service implementation, this service right here should be made available to all the other clusters in this cluster set. What happens is you get service imports in those other clusters automatically, all throughout the cluster set. That means you have a DNS name and IP address, just like a service in all those other clusters that you can use to access these end points, even if they're not in your cluster. Now, that's kind of the basic example of multi-cluster services, but kind of the next level is that you might have end points in more than one cluster that you want to kind of merge together into one large kind of global multi-cluster service. In that case, there's a concept of namespace, name sameness. In the multi-cluster world, that means that two services that are in the same namespace and have the same name are going to be considered the same service. So if you export both of them, they're going to get merged together and a service import is going to reference end points from either cluster. That's an important concept as we keep on going and start to layer Gateway API on top of this. So we made it this far, you may be wondering, well, how do you combine these two APIs? We've got multi-cluster service. We've got Gateway API. Well, we spent a lot of time talking about that. As you'd expect in Kubernetes, it takes a while to nail down these standards and there's a lot of nuance here. So if you're interested, we wrote a whole gap. That's a Gateway enhancement proposal about how these two APIs interact. And there's a lot of detail here. I'm going to try and give you the very high-level summary, which is that anywhere you can use a service in Gateway API, you can use a service import. So anywhere you can use a service, you can replace that with a multi-cluster service. And that's what we're going to be covering in today's talk. So let's go back to that example we showed at the beginning where Gateway API was doing some traffic splitting between Store V1 and Store V2. Let's say that our app developers got a little creative and said, well, actually, my Store V2 service, it's going to span multiple clusters now, or maybe it's going to be in a different cluster. What you would do is instead of just referencing a service, you just replace that one detail and it becomes a service import. You're referencing a multi-cluster service and a multi-cluster set of endpoints. And one other detail you might have noticed there is that's what we call a multi-cluster gateway. There's not much to it. A multi-cluster gateway is just a gateway that supports referencing multi-cluster services. So part of the reason you might be here is there's a lot of APIs here and you might be curious how cloud providers are translating these APIs into cloud resources. And that's what this next section is going to talk about. First, I'll hand it over to Lee-Wen and she's going to talk about how they're doing it for AWS VPC Lattice. Let's talk about EKS implementation of the Gateway API through AWS Gateway API controller. First, we introduce a new gateway class, Amazon VPC Lattice. And AWS Gateway API controller will reconcile gateways and HTTP routes on the Amazon VPC Lattice Gateway class. And create corresponding VPC Lattice resources accordingly. TRDR in Kubernetes Gateway APIs is a control plane, where VPC Lattice is a data plane. We have open sourced our Gateway API controller. Here's the GitHub link and the QR code. Here is a high level overview how we maps Kubernetes Gateway API objects to AWS VPC resources. Kubernetes Gateway API object is mapped to VPC Lattice service network. In another word, it is a gateway into a VPC Lattice service network. Kubernetes HTTP route object is mapped to VPC Lattice service object. Kubernetes service object is mapped to VPC Lattice target group object. And the Kubernetes paths or endpoints are mapped to VPC Lattice targets. So what is AWS VPC Lattice service? VPC Lattice service is a unit of application running on instances, serverless and the Kubernetes paths. It consists of listeners, rules, and target group. As you see here, Kubernetes HTTP route is mapped to Lattice service. And HTTP routes routing rules are mapped to VPC Lattice rules. And the HTTP route back in rough of Kubernetes services are mapped into VPC Lattice target group and the Kubernetes endpoints are mapped to VPC Lattice targets. So what is AWS VPC Lattice service network? A VPC Lattice service network is a logical boundary that is used to automatically implement service discovery and connectivity and apply common access and observability policies to a collection of services. As you see here, Kubernetes Gateway is mapped to a service network and HTTP routes which have same gateway are mapped to the Lattice services of that corresponding Lattice service network. So for multi-clusters and multiple VPCs, AWS VPC Lattice is designed to simplify and automate service connectivity and securely connect services across VPCs and their account. As you see here, there are four different EKS clusters are all connecting to a single AWS VPC Lattice service network called My Network. In another world, all of them will have a gateway object My Network pointing to Lattice service network called My Network. So as a service provider for s1.com VPC1 or EKS cluster one have a HTTP route defined for s1.com which has a parent RAV pointing to gateway My Network. Similarly, the service provider for s2.com in VPC2 or EKS cluster two, it will have a HTTP route called s2.com which has a parent RAV pointing to gateway My Network. Similarly logic applies to VPC3, EKS cluster three and the VPC four and the EKS cluster four. As a service consumers, as you see here, you have VPC one, EKS cluster one, all the way to VPC four cluster four can consume all Lattice services s1.com, s2.com, s3.com, s4.com. Thanks for gateway My Network. Cool. That's very cool to see VPC Lattice's implementation on EKS, how that works. And just to compare, we'll show how this works on GKE and our interpretation of multicluster gateways. So at a high level, there's an idea that gateway listeners in gateway API roughly map to cloud concepts, Google cloud concepts of a forwarding rule and a target proxy. Then again, roughly, HTTP routes and other kinds of routes roughly map to a concept of a URL map in GCP load balancing. Then finally, every Kubernetes service, each port maps to a back end service. Then we have this concept of a health check, custom health check configuration. So an extension mechanism of gateway API is that you have a health check policy, and you can use that to apply health check, custom health check configuration. And then finally, maybe one of the most important details is individual endpoints are mapped to network endpoint groups that similar to EKS allows us to load balance directly to pods instead of making that kind of second hop from a node and then to a pod. So all of this roughly maps together and you can already say, well, this looks a little complicated. Well, keep in mind that's just one layer of translation. You kind of have to map everything together. So both the routes, gateways, you have to connect everything on that side. And then on the other side on GCP side, everything has to be connected to you. So this is just a lot of stuff that you don't really need to worry about. But that's what that's what's happening behind the scenes in our GKE gateway controller. Now, an important concept to understand will be coming back to this over and over through the demos and later on in this talk is that of a config cluster and a config cluster in GKE means that within our cluster set or in GKE the term is a fleet but a cluster set in this case, we have one cluster that's designated as the config cluster. And that means all your gateways, HTTP routes and etc. are read from that single cluster. So the GKE gateway controller lives, oh, got out of sync with what I have here. Well, that's cool. The GKE gateway controller lives outside of your clusters. And that's well and good. And inside of your clusters, you have gateway configuration. And in this case, we have a single config cluster where your gateway controller is pointed. Now it caught up. We'll get there eventually. Hopefully we're in sync now. So we have some other clusters, but they don't matter to the GKE gateway controller. It's just looking at a single cluster. And we'll come back to that in just a little bit. Now our GKE gateway implementation supports two kinds of cloud load balancers, high level generalization at least. One is a regional ILB and the other is a global external load balancer. That's what we'll be focusing on today. That's, as you'd expect, resilient to a zonal or regional outage and can forward traffic to any backend network or region. An important concept is we're understanding that the demos further on is that all traffic by default is going to be routed to the closest region where there's capacity. So if you have capacity in a cluster that's close to you, you're going to be routed there. When that cluster fills up, you'll spill over to the next closest region or cluster. Now you may be asking, well, this is all well and good, but wow, this is really out of sync. I'm sorry. You may be asking what if a cluster or region fails? That's especially problematic, right? We'll be covering lots of different failure scenarios in the demo ahead. And I'll just leave it at that and I'll hand it back to Lee Nguyen to cover how this works on VPC Lattice. Now let me show you how a VPC Lattice Gateway API looks like. So here's the Gateway class, Amazon VPC Lattice. You see we have a new controller called Gateway API controller. And I have created a Gateway object called KubeCon demo. It is using Gateway Amazon VPC Lattice Gateway class. And I have also created a HTTP route called KubeCon-parking. In this KubeCon demo parking HTTP object, you see here, basically, I have for PASC, go to slash rate, we'll go to Kubernetes services rates. And for PASC slash review, we'll go to Kubernetes services review. And they're all using their parent graph all points to KubeCon demo. Now let me show you the traffic. So here are all the paths in this cluster. And this is HTTP route with the host name parking Lee Nguyen KubeCon demo.com. Now let me exacting to Kubernetes part. And as you see here, when I curl parking Lee Nguyen KubeCon demo.com slash reviews, I go to one of the review parts. So and these parts basically is a simple echo server just reply is who I am, the pod name, and give a name of the hamster. And so now when I curl slash rates, it goes to one of the rate part that you see it matches the name of the part. Now let me show you the blue green deployment across multiple cluster. In this case, in cluster one, I have a HTTP route inventory. The service owner of inventory wants to deploy a new version in class two. Here are a few reasons. One reason is the inventory service owner want to upgrade to use new version of Kubernetes. For example, I want to use new feature in Kubernetes 1.24. But other services in cluster one, say example, at one up to, they're not ready to upgrade to new Kubernetes version yet. That's one of the reason the inventory service owner I'm going to try some of my part workload in new Kubernetes service, new Kubernetes clusters. Another reason is the H the inventory service owner want to double the workload. And, and as you know, the cluster one is already at its full capacity. For example, it has reached the maximum number of parts in the cluster one. So instead of scale up and try out your scalability during the production, we want to scale out. So inventory service owner want to move the workload to double the workload to cluster two. So here I want to show you how it happens. So now I'm creating a new cluster, a yellow cluster. In the yellow cluster, as you see here, I have created inventory version two. And as you see here, the parts, they have the address of 192.168, the private address. I just want to show you the same how we solve the overlapping IP address in the first cluster. Same thing, I have, you know, inventory version one, I have all these parts, they'll use the same private address 192.168. Okay, now let's go back to the second cluster. Here I create a service export. And call inventory version two. What the service export does here is, as you see here, the key thing here, I have a multi cluster federation, Amazon VPC lattice. So our controller will look at the service export and create a target group accordingly. Now let's go back to the first cluster. I have an HTTP route, HTTP route inventory. As you see here, in the inventory here, just want to show you to right now as I'm start deploying awaited, so 90% of traffic will go to local version one. And 10% will go to service import version two in second cluster. Now let's me show you the traffic. Okay, it's back to, okay, let's go see the parts in this cluster. Let's go to one of the part. Let's go back, go into the review part. Let's just curl the inventory service. You'll see here, because 90% will go to version one. And now let's run a continuously traffic loop here. As you see here, 90% will go to version one. And let's wait until we see. Okay, now we see, you know, some, you know, 10% go to version two. All right. Now, I'm pretty inventory service owner, pretty confident about the second version. Let's just change the percentage. Okay, now let's change 10% and to version one 90% of version two to the second cluster. Let's just double check. We modify this correctly. Okay, now let's demo God with me. Let's apply the change. All right, we apply. Now let's just confirm it's yeah, it's 10% version one 90% version two. Now let's see the traffic shifting. Now let's see the new HTTP route gets populated VPC lattice data plane. Now you see here, it's get populated 90% will go to version two. Cool. Am I on I can't remember. Okay, I think I'm back. All right, that was a great demo. Let's move right back in because I know we have limited time left and I want to show how the same APIs can be used on GKE. So first off, we're going to start with two clusters in a cluster set or GKE fleet. One of those clusters is going to be based in the US. And the other cluster is going to be based in Europe just right over here in the Netherlands actually. And in both cases, we're going to have the same config mirrored in both clusters. So we're going to have a gateway, HP route and service import in all clusters. This may sound a little weird because remember our gateway controllers only looking at that one cluster at our config cluster, but soon this will become important. So we're starting here and let's work on actually building this out. So first up, we'll go end to end here. We'll create a couple clusters. You know, this is pretty straightforward at this point. So I'll skip over that. But then once we have the clusters up, we can go ahead and register them to our fleet, which is just our term for a cluster set. Now that they're registered, we can go ahead and get gateway class. And you can say, oh, look at that, we have some gateway classes that we can use for our gateways. And we've got a few and the ones that at end in dash MC mean they're multi cluster enabled. But you may remember, you know, we have more than one cluster here, there's two clusters. You probably want to see the config in both clusters. So for most of the rest of this talk, we'll be using a tool called kubectl MC. This makes it really easy to work with multiple clusters. It basically is just a loop that applies the same kubectl commands, whether it's get apply whatever to multiple clusters as you configure. So if you're interested, there's a link to it there. So I'm just going to run the same command again, but with kubectl MC. And now you can see our US cluster and our EU cluster both have the same gateway classes available. Alright, so we've got that set up, let's build out our store application. This is a really simple application. It's really just echo server. But we're calling it store to make it interesting, I guess. So first off, we're going to have a service in both names in both clusters, a deployment in both clusters, just with two pods. And then we're going to export that. So it joins into this kind of global multi cluster service that comprises endpoints from both clusters. So with that, first, we're going to apply that configuration. Again, in both clusters using kubectl MC, we can see it's applied first in us one. Oh, sorry. And second in EU one. Thank you. Sorry. Next up, we're going to get pods. And in that case, we're going to see that we have two pods in each cluster. These are just echo server pods, nothing fancy here, but it allows us to see when we're making a request, what pod we're actually getting into. So with that applied, we can also take a look at our service import. Remember, service imports give us an IP. They also give us a DNS name dot cluster set dot local instead of dot cluster dot local. And so that means even without gateway, you could make a request neither cluster and be routed to endpoints across clusters. And that would work. But gateway layers on a lot more, we're going to get there. Now, since we're talking about gateway layering on so much more, it might be helpful from a gateway perspective to be able to target some subset. So maybe you just want to target endpoints in us clusters or EU clusters, for example. So we're going to fan this out a little bit. It's going to look a little intimidating. But we're going to create a regional service. So just a service that's unique to our US cluster. That's just called store us. It targets the same deployment, the same pods, but it only exists in our US cluster or clusters and similar in the EU cluster. So what that means is we now have three different multi cluster services that we can reference anywhere. So we can reference our US pods, our EU pods or all pods. Alright, so we're going to apply this configuration. And first up, we're applying our US config, our US service and our US cluster and our EU service and our EU serve cluster. We've made it all this way. Let's get into our gateway configuration. We're going to use our L seven XLB multi cluster enabled gateway class. We're going to be listening on HBS. And of course, we just using a pre generated HBS cert, TLS cert. Next, we're going to go ahead and apply that configuration. And remember, we've applied this in both clusters for mirroring things in both clusters. But our gateway controller is only looking at one of those clusters. So with that said, let's take a quick look. Gateway controllers only looking at our EU cluster because that set up to be our config cluster. So when we use kubectl MC again to take a look at these gateways, you're going to see something like a little funny. One gateway is just completely ignored. There's no address. It's not programmed. It's just it's just sitting there. The other gateway has an address. It's programmed. It's ready to go. That's by design. It feels a little weird, but we'll get to why we might want to use that soon. All right, so next up, let's talk about our HP route. And in this case, we're going to send all traffic that starts with slash EU to our EU service, all traffic that starts with slash us to our US service. And then finally, anything else to our global store service. So pods regardless of where they are. So we'll go ahead and apply this configuration. And with that good to go, we can keep on moving and start making some curl requests. So I just took this up to personal domain. And in this case, you can see because my request were coming from the US, I got routed to US Central. That was the closest cluster to me. Now, if I make that same request and do slash EU, I'm going to get routed to a pod right here in the Netherlands. Now you may be asking, what happens if cluster fails? What happens when things break? That's that's why we're here. How do we recover from things breaking? So a really simple way to simulate that is let's say all the pods just disappear, they go unhealthy, any variation of that in one cluster. So in this case, I'm going to make them all disappear in my US cluster, and start making some requests. And regardless of what my request is to, I'm going to be routed to Europe, because that's the only place I have healthy pods left. So that's a nice kind of automatic failover for you. But what happens if we have an even more catastrophic failure? What if our config cluster goes away? What if our region goes away? What if something really catastrophic happens? Well, I've been setting this up for a while. But let's say that this config cluster, you can't reach it, something went awfully wrong with it. Well, we've been mirroring all our configuration along. So we just make one command and we say, actually, this this cluster is now our source of truth. Everything's the same. So we'll make one little switch. Actually, our config cluster is US one instead of EU one. That's done. And now that becomes our source of truth. So we'll keep on moving and I'm running out of time. So I'll run real quickly through this, but we're just going to say, regardless of where you're requesting, we're just going to send endpoint traffic to the pods that we know are healthy, which are in US one. So we'll go ahead, make that change. And we're making that change only in our US cluster because we're assuming that's the only one in this case that we have access to. So we make that change, go ahead and make some curl requests. And whatever we're requesting, including slash EU is going to get routed to the cluster that we know is healthy at this point in time. Now, I've got to keep running here, but there's so much more that Gateway API offers. I'm really excited. Gateway API is implemented by something like 20 different implementations. This is the set of features, a subset of features that are enabled on GKE, but there's so much more available out there. There's a link there that gives you a bit more of what's available. But this is really exciting. And we're just starting. There's so much more to come. So I encourage everyone to get involved. These this is all based on open source APIs. We showed how a couple of cloud providers are implementing these APIs. But there's tons of room for anyone here to help get involved and define the future of these APIs. So there's really three different things here. There's Gateway API. We're hoping to get that to GA this year in 2023. There's multi cluster service API. There's a lot of work going on to get that to beta this year. And then gamma is kind of the mesh initiative that bridges a lot of these concepts together. And we're hoping to formally define that get that to experimental. We've got I think every mesh vendor out there is involved in this. So really excited to see the convergence here. So there are all kinds of ways to get involved with Gateway API. We actually have two meetings every week. So we have weekly community meetings covering everything on Mondays. Gamma meetings. That's our mesh focus thing on Tuesdays. There's actually another meeting on Fridays. I forgot to add. We've got a lot of things going on. There's no shortage of ways to get involved if you're interested. And then finally SIG multi cluster they meet once every two weeks and also lots of opportunities to get involved and define the future of these APIs. Thanks so much for your time. We've got people here who can help answer questions both from the AWS side. And also if you want to reach out to Lee Nguyen, she's available on Kubernetes Slack. If you have questions, I know we're basically at time. But if you do have questions, if you can line up at the microphones, that'd be very helpful. Thanks so much.