 All right, so my name is Flynn. I'm here from DataWire to talk to you about Ingress. I have a couple of confessions to make before we start. The first one is that when I first started working with Istio, I kind of thought Ingress was a random little aside off away from the interesting stuff. Since then, I have come to realize that it is actually the thing that lives right at the intersection of the users and the problem and the technology. And really, if there's one thing that I'd like to leave everybody with, as I'm done with this, it is a sense of what changed in my mind on that. It actually is a pretty interesting problem and a pretty interesting one, it turns out. The other one is that Sharam was saying that I was actually here to rant about Ingress, and I've kind of given up on that. I think I'm just going to have to talk. Sorry, man. So you've heard some things before earlier today about diving down into the guts of Istio. Ingress is really about the view from the outside. Istio lives inside this cluster. It's not very useful if you can't talk to it from outside. And that right there, that is the thing that Ingress worries about. The question then becomes, who cares about this? Any opinion? Excellent. Nice to know that somebody does. Anybody else? Yes. Great. The first statement back there, I just realized nobody's going to be able to hear it. The first gentleman said he does, which is great. The second response was that anybody who's creating something that they need to expose to anyone outside the cluster, there's a word for those people. That is developer. Anybody who's seen me on the Istio Slack knows that I like to be specific. So let's get specific. We have here a developer named Jane. She's the microservice development lead for Evil Genius Cupcakes Incorporated. And she is working on a shiny new microservice. There are no users yet. There's nobody calling her service yet. She's working on it all by herself. Specifically, she has here a service whereby people can keep track of all of the evil things that they do in order to get the cupcake. So you'll have a user who will go through and hand off some sort of evil activity to the tracker. The tracker then has to go and save that activity in a database. And just for the fun of it, at some point they might like to read back what sorts of evilness they've done. A very important thing to realize here though is that tracker and the database live inside a cluster. I'm going to be talking about a Kubernetes cluster. If you look at what Jane will go through while she's working on this, she's going to start off basically just kind of hacking on this code on her own. If she has any users at all, they're people that you work with, or colleagues, some test users. And this leads us to a question of at this point, what does it mean to successfully deploy this? If you roll out a new version of this service, then what are you going to think of as good enough? Anybody? Something people can use? Anything else? Any other suggestions? Yes. Meets the spec. Assuming it doesn't fall over is, in fact, the one I was looking for. This early in the game, meeting the spec is nice, but you probably don't have a spec if you're just beating on the thing. If you're just beating on it in the beginning, then you might not have users who care about it or users that you need to make sure you don't disrupt. Really, this early, you roll it out, it doesn't crash. That's really good enough. At DataWire, we're taking to calling this stage one. Stage two, you roll it out to the world. There are real people using it now. What's good enough now for a deployment? Always be available. Very good. Anything else? Any other suggestions? Available, secure, those are good. Beed is also good. All of these, or many of them anyway, kind of boil down to, if you're going to roll out a new version of this, you don't want to disrupt your users. Ideally, you'll be rolling things out that make their lives better without ever making it worse. Scalable and secure, kind of get into that. Secure definitely, I think, does. Scalable in an ideal world, I think that I'd like to hope we can scale the service without having to deploy it. We can just go ahead and let it stretch out elastically. So definitely part of it then. Depending on what kind of a service you're doing, you may also land on a stage three, where there is coupling between you and other services in your application. Effectively, your service is getting used by other services internally. And at that point, what might make the successful deployment, what's your message? In the back of the room, again. Authentication delegation, rate limiting, things like that. Generally speaking, we tend to call this one that at this stage, you're working with a foundational service. You need to worry a lot about cascade failures. If something crashes, then you don't want it to bring down your entire application. Everything should be able to respond gracefully to that. Another point that's worth making is that at any given point in your application, not all of the services are going to be at the same stage in their life cycle. You may have many of them that are a stage two. You may have a few foundational services at stage three. And of course, Jane may be off hacking on some random new thing at stage one. So it's instructive to think about what sorts of things Jane will have to worry about along the way. She will probably first be worrying about, how do I debug this thing while I'm working with it? She can use fancy technologies like logging for that. And she will have to worry about that right from the beginning at stage one. Other worries. She needs to figure out how to safely upgrade this thing. Maybe she's going to use canaries for that. There are a lot of different ways she could do this. But all of them rely on, you have to be able to measure if your service is doing the right thing, or whatever the right thing means. You have to be able to route to one version or another version, depending on whether it's doing the right thing. At what stage is she going to have to worry about that stuff? I see some twos. I think I saw one, one. I'm going to go with two. Stage one, if nobody's using it, you can just kill it and restart it. No big deal. At some point, she's going to have to worry about grace under pressure. Maybe she could slash dotted. Or whatever the cool kids are doing these days. At some point, though, even if you allow her service to scale elastically, remember, there's a database in the back there. At some point, you have to worry about protecting the database. And so we can use rate limiting, for example, as a particular example of how we can protect against that. Where does that come into play? I also see in twos. Yeah, I think two is the right one there. Cascade failures, like we mentioned. OK. Obviously, this one's stage three, yes. That's where we start getting into things like circuit breakers. We start getting into the other side of rate limiting. It's interesting to note that these are actually things that Jane might worry about at stage three, or she might never worry about them. In that she might not get to stage three, but also in that if the service mesh is doing its job and it's doing it really well, then maybe the things that she's already put into place permit the mesh to handle some of these things for her without her having to think too much about it, which would be cool. Finally, at some point, actual users have to be able to talk to her service. And that is where Ingress comes in. And that's kind of interesting, because that's actually something that shows up right at the very, very beginning, possibly even before things like logging. So one of the most interesting things about the Ingress technology, the Ingress topic, is that it's going to be the first sort of meshy kind of thing that the Janes of the world are going to worry about. The other interesting thing is that Jane is not particularly interested in the service mesh as a thing to be thinking about all the time while she's at work. She's interested in solving a particular class of business problems. Ideally, we'd like to arrange for all this stuff to be something that she can just sort of flip into place once set up and then not have to worry about. Something that doesn't get in her way while it does allow her to get her job done. It may actually end up being, as I mentioned, the only really meshy thing that she starts thinking much about. It may be that the service mesh internalizes enough about logging and enough about rate limiting that she's able to get away with not having to worry too much about those things, which would be really cool. But at minimum, she's going to have to think about Ingress. API gateways also come into some of this. We're going to mostly talk about Ingress as opposed to the slightly higher level stuff that API gateways can bring into the future. All right. So with all that, we'll take a look under the hood and how this stuff plays out. A caveat, Istio has as a goal being able to support things that are not running on Kubernetes. I'm not going to talk about any of that. I'm going to focus exclusively on Kubernetes. Partly that's because Kubernetes is currently the only supported solution for Istio. Part of it is because as data wire goes out and talks to people, we keep hearing questions about Kubernetes as opposed to questions about other stuff. So we start with a cluster. And by the way, feel free to stick out if you have questions. In our cluster, we have some services. Services have multiple endpoints, potentially. Those of you who know much about Kubernetes will probably already be accusing me of playing fast and loose with terminology here. Again, we're kind of looking at this from Jane's point of view. Jane is writing a service that gets deployed. There are multiple instances of the thing that provide it. Yes, under the hood, there are Kubernetes pods and Kubernetes containers and Kubernetes nodes and Kubernetes services. But for our purposes here, we can think of the idea that a Kubernetes service gets implemented by some collection of endpoints. You'll also note that the Kubernetes API for those things is called endpoints. Let's just move on. There are multiple ways that we can arrange for access from the outside world into these services. The simplest one is just to use load balancers, as in a Kubernetes service of type load balancers. What that does is it pokes a hole off in the side of the cluster, arranges it so that when you speak to that, traffic gets forwarded through to the service. Kube proxy under here handles all the heavy lifting of actually round robinning around the endpoints. Works great. Pretty simple. As the downside that if you want to be able to talk to all three of these, then you're going to spend up three load balancers, which is arguably a little on the silly side. So there's another way. We can use one load balancer, and then we can stick a layer of 7 proxy behind it. This is not that uncommon. The layer of 7 proxy knows how to talk to all of your services. In order to do that, it has to both be able to figure out things about the services and the endpoints by talking to Kubernetes. And it needs some sort of configuration information to describe what to do with the traffic that's coming in. This URL goes here. This URL goes here. There could indeed be multiple replicas, multiple instances of the proxy. And you'd be relying on the Kubernetes load balancer to round robin amongst your proxies. I've done this. It works well. Any other questions so far? OK. Finally, we land on the Kubernetes Ingress resource concept controller mess, whatever. The point of this is that something magic appears. And using your something magic, then you can just reach through any of these services. And it just kind of magically works. Because magic is cool. We like magic. There is a thing in here called an Ingress controller. The Ingress controller is not the magic. What the Ingress controller is is a thing that looks at Ingress resources. Ingress resources define the way requests get mapped services. The controller knows how to look at the endpoints, it knows how to look at the resources, and it knows how to configure our magic thing off on the side. So, yeah, hold that thought. So, one kind of interesting note is that something magic actually usually ends up appearing as a Kubernetes deployment. And you can often go and ask Kubernetes for things in the Kube system namespace. And then it will go through and show you lots and lots of gory details, which are fascinating, but really not the sort of thing Jane would like to think about. All right, so something magic. Like I mentioned, the controller is not the magic. The controller has a job. It has to work with the magic. It has to arrange for the firewall to be opened if there are firewalls involved, which, of course, there are firewalls involved. It has to create something magic. And then it has to configure something magic. By definition, something magic has to be a layer seven reverse practice. That's kind of it. There's been some talk of the Kubernetes Ingress concept. I'm not in favor of that. I think that Kubernetes services do a fine job of talking about layer four. And we should leave Ingress resources and such for the layer seven use case that they were originally designed for. But yeah, there's a fair amount of discussion about that. Cloud providers like GKE and Amazon will typically give you default magic if you just start messing with Ingress control. The Google Cloud load balancer on GKE by default. It seems to be engine on Amazon. Yes. No, there only needs to be one Ingress controller in the cluster. One of the points of the Ingress concept in Kubernetes is that rather than having to spin up a separate load balancer for each of your services, you can just use one and let it handle multiple services. And much of the reason for that ends up being that it's relatively expensive for cloud providers to spin up load balancers. They have to allocate external IPs and do all sorts of things like that. But they'd really rather you not have one for every service. Does that answer the question? Cool. So yeah, if you just go through and you start asking for Ingress then in GKE or Amazon, then you'll get something that will possibly do what you want. It actually ends up being a little bit tricky to convince them not to give you this thing, which I find kind of fascinating and frustrating. Particularly because as you mentioned, Envoy is a kind of painfully obvious candidate for being a layer seven reverse proxy because it is a layer seven reverse proxy. And in fact, within Istio, there's two things that are already kind of starting down this path. There's Ambassador, which is the thing that I work on. There's Istio Ingress, which is the thing that shows up in Istio right now by fault. Ambassador is listed first here not so much because I wrote it, but because it's a slightly simpler deployment thing to talk about first. Really, trust me. Ambassador, it is an API gateway. It comes from DataWire. You can go to getambassador.io to read about it. You can go look at it on GitHub. You can read the proposal that's out there for having Ambassador takeover as the Istio Ingress controller. Presently, Ambassador is all about using REST calls to go through and configure custom resources to describe mapping. We're actually shifting that over to a more declarative model. That's actually the extent to which I'm going to go and dig into the implementation of these things right now. Because what we're really talking about here is Ingress as a concept rather than Ingress as the implementation. If you have questions, again, feel free. Call them out. This is what Ambassador looks like under the hood. We still have our services. Ambassador actually is deployed as a load balancer, so there's kind of no reason not to. So the load balancer is there. We have a Kubernetes pod for Ambassador. We have some configuration information. Inside the pod, Ambassador runs an Envoy. The Envoy is connected to the load balancer. So you can talk directly to the Envoy from outside. The Envoy, in theory, knows how to talk to your services underneath. At present, Ambassador relies on Kubernetes to do the heavy lifting of rotating between endpoints. So it just talks to services. There is also the also creatively named Ambassador.py person. In Ambassador, these are processes within the same container. Ambassador.py knows how to figure out what services are around. Services are around. It knows how to read the configuration info, figure out semantically what the configuration info means. And it knows how to generate an Envoy configuration from that, at which point it goes and updates the configuration of the Envoy so the Envoy does the right thing. Was that a question? Two. Hold that thought again. Actually, let me, yeah, let's come back to that after looking at the way this happens with this year. Yes. The API gateway functionality that Ambassador provides is mostly about trying to give a developer a relatively easy to use set of centralized functionality, like application level authentication, things like that, where if you are, if you're writing an application and you're not doing it with microservices, you're writing a monolith, then what generally happens is that all of the functions within your monolith end up drawing on the same set of functionality that's provided by libraries. And so you don't have situations where, just because you're writing a service, you need to go and re-implement authentication, for example. And by authentication here, I mean it might be, you might be using TLS client service for auth, but you might be doing something totally different, right? You might be using JWTs or whatever. In the microservices world, one of the things that can be tricky, especially in polyglot microservices, is that if you don't handle things correctly, then all of your Jains have to go and write the authentication checks over and over and over and over and over again. That almost always ends badly. So API gateways, one of the things that they're particularly good at is giving you a place to stand, to centralize some of that, and get closer to the development experience in the monolith, while still permitting all of the rapid deployment and velocity in microservices. Does that make sense? OK, hit me up after. We can talk about it more. Anything else? All right, I'm sorry, say again. Ambassador currently does support terminating TLS and also doing TLS client certificate authentication. And there's actually a bunch of other stuff that we've been looking at. One of the things we did was add an external authentication filter into Envoy so that you can hook Envoy up to whatever you want. So OAuth is probably next to our list, I think. Does that answer? Yeah, again, so perhaps I should have done some of the API gateway functionality in you. All right, anything else? OK, so that brings us over to Istio-angris, which is an actual Kubernetes-angris controller that is a part of Istio. Those of you who are fond of looking around in the Istio source, you can look in those two for some Go files that are basically where to start. Somebody asked how this interacted with Pilot. And you might notice the first component of those paths names. So yeah, in the Istio world, this ends up being strongly related to Pilot. First controller pays attention to Kubernetes-angris controller or ingress resources, excuse me, to define which URL goes to which service. But it also has, you know, thing in Istio, the Istio routing rule, to define the book info service and then which version of the book info service do you want. And so we'll be talking a little bit more about configuration later to talk a little bit about why those two things are separate. Yes, that's correct. Nope, it does not. There are a couple of reasons for that. We'll be talking a little bit more about that later. The first reason is that Ambassador for Simplicity is a service. It's not actually an ingress controller. The second reason is that if we decided to make Ambassador pay attention to ingress resources, the moment you add the ingress resource, then your cloud provider will try to spin up the default cloud backend for you. And getting at knots to do that is enough of a pain that we decided not to bother. Yeah, I know, it doesn't make any sense. But the main challenge is, yeah, I mean, if you are already doing something where it doesn't matter to you that if you're doing something on a large enough scale that this one extra thing coming into existence doesn't bother you, it doesn't really matter. And we'll actually take a look at that on the next slide. Oddly, yes. So Istio ingress under the hood. Istio ingress still has a load balancer out there. But we now have a proper Kubernetes ingress controller. It pays attention to the Istio routes, like I mentioned. It also pays attention to ingress resources. There is an envoy out there that knows how to do that. And then there's the proxy, which knows how to check all this stuff, pay attention to ingress, pay attention to Istio routes, and configure the envoy. Proxy's kind of an interesting thing in the sense of the way it can be executed. It's possible to execute proxy with a variety of different roles. And so in this particular chunk of Istio, it gets started explicitly told to be an ingress controller. And just for the fun of it, if you're doing this on, say, UKE, you will also find that there's this thing lingering around called the default HTTP back end. And under some situations, then Google will just decide that you didn't really mean to talk to Istio of ingress. You meant to just bounce around and get an error from the default back end instead. What you're going to do. So any questions so far? Oh, no, no, no. It's completely deterministic. I'm sorry. I didn't mean to give that impression. If you have defined an ingress resource for a particular path in the URL, then that will always, if you ask for something for which there is no ingress resource defined, then it'll bounce around and hit the default back end and you can get an error. It's 100% determined. Yeah, it's 100% deterministic. It's not that bad. All right. So then the question is, I said that Ambassador and Istio ingress are kind of a step down the path. Part of the question then becomes what's left? How do we get the rest of the way there? And the biggest one is to get rid of the load balancer that's stuck there up front. And as you were mentioning, this is something we probably are going to need help from Kubernetes folks on. But there's no reason to have that load balancer there. Envoy can manage that just fine. There's also some questions around handling configuration, which are a little bit interesting. And for that, we shall bring Jane back in. What's the minimum that Jane needs to be able to configure to properly give users access to her service? Yeah. OK. Anybody else have suggestions? OK. Say again? Yeah. We basically ended up with Ambassador. And I should point out that I am talking about levels layer seven here. For Ambassador, the stuff that we ended up settling on was some way to patch against the URL at minimum. Prefixes, regular expressions are the two obvious ones. Doing both is probably pretty reasonable. You need a way to describe the destination service and the version of that service. It's hard to do things like canaries, canary rollouts if you have no versioning information at all. You really need the ability to rewrite the URL in transit. And on the one hand, that kind of seems optional, but on the other hand, it turns out to be so incredibly useful when you're working with microservices, when you're trying to design an application that it's easier to just treat it as a necessity. In particular, it gives you the ability to do things like pick up a service and move it to a different chunk of the URL space without having to rewrite it. And it also permits for code sharing in some more useful fashions where you can more easily do things like have a common health check function that people may be able to draw on if they're using the same language. Finally, whatever we do for configuration it should be straightforward to work with. Again, Jane would like to be thinking about her business problem as opposed to thinking about all these little details all the time. Yes. So I'm actually going to go over some of that, I think. So give me a minute or two. And if it's still a question, let me know. There are obviously a bunch of other things we could throw into the mix. This is really a question of what's the minimum that we can get away with. This is an example of an ambassador configuration. The name is the name of the mapping, so you can delete it later. We're asking for anything with a prefix in the URL path of slash uotm slash. We'll get rewritten just as a bare slash and handed off to the service named qotmv1. The module stuff is some additional things that ambassador can do that we don't need for this example. This is how things end up looking under the hood if we try to actually go through and mess with that. This, I think, is addressing your question. The developer ends up asking ambassador.py to apply that route, apply that mapping. Ambassador.py turns around and writes that into the configuration info and calls it good. A couple of seconds later, it, or another one, notices that the configuration has changed. It goes out, checks to make sure that that service is actually there, generates an envoy configuration, writes that into the envoy, and then all were all good. So in the case of ambassador, ambassador is maintaining the storage. You speak to the ambassador process and ask it to make changes for you. Does that answer your question? If you have multiple ambassador instances running in the same cluster, they share a backing store. And so they'll all see the same set of configuration stuff. Yeah, right. None of the stuff that I've worked on is meant to be cross cluster yet. There's, that's also a topic of much discussion at the moment. Things get really complex when you start trying to do seamless networking across cluster boundaries. So I generally, at that point, just kind of look over and go, let's not do that. OK, this is an example of a Kubernetes Ingress resource. It has a name which refers to which Ingress controller you're messing with. That annotation for the Ingress class lets Kubernetes know that we're messing with an Istio Ingress as opposed to some other Ingress. And this is a rule that says, if the HTTP path matches slash qotm slash followed by anything, then go to the qotm service on port 9080. There is also an Istio routing rule that goes with this one. So we say that basically, if you're asking for this service, and we want to use the look for the version v1 tag. So at present, Ambassador likes the version number to be in the name, and Istio likes the version number to be in a tag. That's cool. You might have noticed that the Ambassador configuration is JSON, and the Istio configuration is YAML. No, I have to go through all the animation. That's through, but kind of neither here nor there. In Istio, things happen a little bit differently if you're trying to go through and do this. In Istio, what you'll end up doing is applying, using kube control to apply the Ingress resource in Kubernetes. So it will be stored in Kubernetes. The Ingress controller will notice that the resource has changed. It will then go and take a look at the routes that you had. It will take a look at what services and points are out there, and it will go through and update the envoy, which lives, again, behind the load balancer, which we would like to get rid of. You could also do this by using the Istio control command to apply a change to the Istio routes first, and then it would just look at the Kubernetes Ingress resource after noticing a change there. Yes. You don't actually have to create the Istio route. There's a default that just kind of round for ovens among everything it can find. Basically, yeah, you could. And that way, a lot of bunches of conversations about support. Yeah, it's interesting, actually. There's a bunch of changes coming along in the next version of Istio around. Yes. Ambassador speaks to the service and allows Kubernetes to go through and manage the layer down from that. Not an ambassador at present. Istio Ingress, I believe, is looking at the endpoints. And if I got that wrong, I'm sure Shirama will correct me. Yeah. That was my entry as well, yeah. No, the one that I was talking about being the one we'd like to get rid of is this one, right? Because this envoy is perfectly capable of doing everything that one does. This brings us up to one of the entertaining things about Ingress configuration right now, which is that the Kubernetes Ingress resource only kind of covers our minimum set. There's an annotation for URL rewriting. It's not part of the base spec. I think there are still things that don't quite support that annotation right now. You don't have any versioning information in it at all, which is the reason why the Istio route exists. To be able to do things like that. There are other reasons like you'd like to be able to do, if you have something like an envoy, you'd like to do really sophisticated matching for routing. But if you don't have something like an envoy, then you can't do that. And so the Ingress configuration resource, the Kubernetes Ingress resource ends up by necessity kind of having to be a least common denominator across whatever we could come up with. I personally think that Jane's life will be easier if we pick something that is a better match for what Envoy is actually capable of. And this gets us into lots and lots of very active conversations about how should we do this? What is the right way to do this? So in conclusion, Jane will be thinking about Ingress stuff very early on, which means that we should probably be careful as we're thinking about it, thinking about what's right and what's wrong. It would be handy to think about it from Jane's point of view rather than just the point of view of being an internalistic developer or data wire developer. And Ingress is this weird sort of place where part of it is straightforward and part of it is really not, especially as you get into Kubernetes and you start poking around and finding these ways in which the cloud providers mostly honor what you asked for and then randomly throw in other little bits that you didn't ask for. So yeah, there's some work we're going to have to be doing with the Kubernetes people to really make that work cleanly. Yes. How about let's take that offline, both because I'm running out of time and because honestly, I'm not the best person to talk to you about that. But I'll do my best and we'll find somebody who also is another good person to talk to. Yes. What are you volunteering to be talked to about, Lincardine? That'd be awesome. Excellent. Yeah. And honestly, what would make me really happy to be a well-supported, clean way to tell Kubernetes, look, cloud provider, I don't want you to do this other chunk of stuff. All I want you to do is the external IP part and then let me go from there. I think was your hand raised as well? The times that I've gone through and written applications that were supposed to be running either behind the ambassador solo or behind the ambassador with Istio, then the thing that I found most useful to think about from the perspective, from Jane's perspective, really, was what made the most sense from a business perspective in terms of resources that I would expose to the outside world? What does the API look like? How do I configure the Ingress to allow only the things that I want the users to be able to see in the outside world and protect everything I don't want to see? That's really where I was coming from. Yes. Yes. I think the way I phrased it was that it was important that you be able to specify some concept of version in your configuration. I personally don't really care if it's a convention in the name or if it is a tag. I think a tag is a little more flexible to a certain extent. That's about as strongly as I feel about it then. That makes sense. OK. Thank you very much. I believe I am out of time. I much appreciate it. If you have other questions, feel free to find me afterwards or hit me up at planetdatawire.io.