 Thank you. So yes, I am here to talk about the rise of layer seven in DevOps. And particularly, I wanted to talk about an interesting trend over the last few years where, on the one hand, we've been moving more towards independence in the way we write services, but at the same time, we've been moving towards standardizing the way the services actually talk to each other, which is a little counterintuitive in some ways and kind of interesting to look at why we did it and how it affects things. A long time ago, we tended to write applications that were centralized and all ran on a single local machine. We originally did things that way because we didn't actually have a choice. That was the tech that we had available. That was the way we did things. It did have a couple of nice things, which was that applications written that way are relatively simple to reason about and relatively simple to debug, because fundamentally, they can only do one thing at a time, both in the sense that you can only be executing one block of code at the time, but also in the sense that you're only doing one release at a time. You don't have to worry a lot about synchronization in different parts. So simple, but certain things become really hard when you do things that way. And the nature of those things that were difficult led us towards multi-processing and then into multi-threading and then distributed computing. The first time we tried this one, it was all custom stuff in the communications world. We tended to be doing very application-specific protocols, either at layer 4 or built directly on layer 4. And one of the things that we found was that distributed computing is actually really hard. And one of the things we found in particular is that if everything is custom protocols when you're doing your communications, then as you solve problems for one application, you have to go back and solve them again and again and again for every other application, which is problematic. The typical approach we've taken in the industry to problems like this is to try to push things down into a layer below so that all of your developers can go through and take advantage of them. And we did that for a while back in the Dark Ages. We ended up with some really crazy stuff in old IP switches and things like that. But overall, it's really hard. And that didn't work very well. If you track forward from there, there was a span where we went back toward centralization as various hardware resources became cheaper, and that introduced some other stuff was organizations got larger, software teams got larger. And all of that ends up landing us into the current microservices world, sort of our second push into decentralization and distributed computing, where we have independent services and we have each independent service being managed by an independent team with its own lifecycle. But one big difference this time around is that this time we actually have effective layer seven protocols that we can standardize on for the communication between these things. We have HTTP, we have GRPC, we have WebSockets, and we tend to be building applications on top of these rather than building applications on their very own custom protocols that we've built just for that application. The holdouts are kind of things like PubSub, Redis, various databases, but even those, they've been around for long enough that those protocols have effectively become standards in their own right. That makes life a lot easier, but underneath it's still a distributed system, and you still have to deal with it as a distributed system. You may have heard of the eight fallacies of distributed computing. People tend to design as if the network is reliable when it's not. People tend to design as if, or start out designing anyway, as if latency is always zero, even though it's not. Likewise, bandwidth is always finite. The network is always hostile. The topology will change underneath you. When it does, you're not gonna know which admin to go talk to. Shuffling things around costs you time and money. Various things will be different when you would really rather have them be the same. All of these are things where if you do not manage them, your application as a whole is not going to work, and that gets into the question of how do you manage them? So, for example, one thing you need to be doing is worrying about resilience. A simple example of that is if you send a request and don't get an answer, after a little while you should probably send it again. A whole retry concept. Observability is another important one. One of the things that's nice about the centralized world is that you actually have pretty good visibility into a centralized monolithic application. You can attach a debugger to it. You can go poke around at its state. This is not easy, but it's easier than doing it across 100 different systems simultaneously. And so one of the tricks with the distributed computing is managing that and giving yourself a way to look in across the application as a whole. Simple example of that one would be for one of your services, you would like to know the latency within which 99% of your requests complete. Finally, we can talk about routability. If a request comes into your service, that request needs to go to a service that can actually handle that request. Seems obvious. It should not go to an instance of your service which is crashed. Again, fairly obvious, but something that you have to deal with if you want everything to actually work. The stuff that I cited earlier is also kind of the bare minimum. Resilience doesn't just cover retries, but we also have timeouts and circuit breakers and outlier detection and all that good stuff. Observability may actually take you beyond metrics monitoring into fully distributed tracing so that you can actually watch a request come in from the edge and then flow all the way through your services and back out again to find out where a given problem actually is. Routability isn't just about finding an instance of a service that's up, but it's also about things like as you change versions, you probably want to start routing some small percentage of the traffic over to a new version of your service while letting the older version that you trust handle almost all of it, things like that. One of the interesting things about the Shift to L7 is that having standardized layer seven communications actually make it a lot easier to manage all of these fallacies. We'll pick on HTTP for a minute just because it's pretty well known. The URL and the headers in an HTTP request give you a fairly application-independent way of capturing information that you can use for routing. That job is a lot harder if everything is specific to a single application. HTTP formalizes the concept of the status to a request. And again, that lets you do things in an application-independent way about figuring out how many of your requests are succeeding or how many of your requests are failing. One of the more interesting ones is that HTTP actually formalizes the semantic of a thing called a request. So you have, again, an application-independent way of watching the latency for requests or watching the number of requests you can do per second. That actually turns out to be a relatively difficult thing to do in an application-independent way if you don't formalize your communications up at a much higher level than we used to 20 years ago. You get similar benefits of this with GRPC, similar, slightly less with WebSockets. But again, the fact that the protocols have moved up the stack to a place where we can formalize some of these things and rely on them across applications makes life a lot easier. Of course, it does introduce the need to manage all the layer seven stuff. And that gets us into the question of how exactly do we do that? The first cut of this happened 2010, 2012. Those sorts of time frames with various smart RPC libraries that people worked with. Historic came up around this time in Java and Netflix. Twitter working in Scala did their clinical products. Google did GRPC and Stubby and whatever crazy secret googly things they were working on back then. All of these things were used in the same way, which is that you'd take your application and then you would bake the library into your application and use the library to provide resilience, use the library to provide observability. This worked out a lot better than asking every developer to reinvent all these wheels simultaneously over and over and over again. It does have some problems though. The biggest one is that the library by definition can only help you if you're working in the runtime that that library was designed for. If you want to use Historic's and you're not working in Java, then you have two choices. One of them is you port your application to Java and the other one is you port Historic's to whatever you're working in. Both of those are expensive and likely to go wrong. That made it difficult to keep using these as the world started shifting to become more polyglot. Another somewhat more subtle problem to borrow Ben Christensen's phrase is that after a little while we realized that people were basically building distributed monoliths. They would use the libraries in ways where the library itself sort of created a tightly coupled protocol at a high level for the application with the end result that every time you wanted to change a piece of it, every time you wanted to upgrade one piece to a different version of the library, you actually had to do all the other ones in lockstep, which turned out to be problematic and created a lovely sort of crazy dependency hell and versioning hell and things like that. So that was better than not using these things but it still wasn't ideal. That brings us up to 2013 where Airbnb introduced SmartStack as a way of trying to deal with the problems that the smart RPC libraries had. SmartStack introduced the idea of a proxy running in a sidecar. They took HA proxy as a proxy, they took ZooKeeper to deal with service discovery, Crandall bus together and ran them in a sidecar, next to the services. This had the advantage that for one thing, it was application independent, it was language independent. It worked really, really nicely in the polyglot world because instead of your service code calling a library to do something, it would just do its normal thing making an HTTP request but it would hand that request into a proxy running in a sidecar right next to the application. The proxy would go and do all the smart stuff around finding an instance that was working, finding something that wasn't already crashed and then it would hand it off. There was another proxy over with the other services handling the other end of this sort of thing. Again, much, much better than what we had before. Still not perfect, nothing is perfect. One of the problems with SmartStack in the beginning was that HA proxies kind of old. It dates back to 2006. There are a number of things that it didn't have in 2013. Observability was there but it wasn't really designed to cope with a large number of services and things like zero downtime reloads or a runtime API so you could interact with it programmatically. Those things didn't actually come along until 2017 even though people were kind of clamoring for them. Another issue was that ZooKeeper, if you've ever actually tried to run ZooKeeper then keeping up with its care and feeding is challenging. Consulinet CD actually came along partially in response to that aspect of things. So that brings us up to 2015 with InginX coming along. InginX we got to use in pretty much the same way with the sidecar idea because the sidecar concept ends up working out again pretty well in a polyglot world. InginX also had zero downtime reloads which is great. Observability, in 2017 it got a runtime API as well but one of the interesting things about InginX is that a lot of the stuff that you would really like to have is actually walled off in InginX Plus which is commercial as opposed to open source. It had the observability, it had the zero downtime reloads. It didn't have something like ZooKeeper to go and beat on but the runtime API advanced load balancing algorithms, things like that were again locked up in this commercial algorithm, this commercial product rather. That leads us to 2016 with Envoy which had a bunch of stuff that we like. Data wire's been very fond of it. Zero downtime reloads, observability all the way down to distributed tracing. It has a runtime API, it can do shadowing. It's native HTTP2 and GRPC. It has a bunch of nice resilience stuff baked in. All of this is stuff that we liked, stuff that we thought was good. One of the other interesting things about Envoy is this year it became a CNCF product so you also have neutral governance going for you and you also have an extremely active community building up around Envoy. You can probably guess the next slide that's coming. No really, nothing is perfect, right? There's a bunch of stuff about Envoy we really like. Aside from all the technical stuff, we really like the fact that being a CNCF project tends to keep it a little bit more neutral. We really like the fact that the ecosystem is really active and is tending towards keeping it in sort of an open source vein. We really like the fact that it's very powerful and very flexible but the cost of all of that is that it's also very complex. Kind of like the difference between assembler and higher level languages. Assembler tends to be extremely powerful, extremely flexible, very performant and just a royal pain to work with. So this is actually exacerbated by the way where as you start to look more at Envoy's programmatic API, you realize that it's really designed for a computer to interact with, not for a human and that goes double when you start dealing with things like it's advanced discovery services and all that. This makes it hard to work with Envoy as a human and the end result there is that a response of the ecosystem has been to introduce this concept of a control plane which is sort of like a safety net if you will for interacting with Envoy. The idea with the control plane is that the control plane sits in between the human and the Envoy. The control plane gets to mediate all the configuration junk both in the sense that the control plane has to worry about taking a higher level description and generating the lower level configuration from it but also in the sense that if you get Envoy's configuration wrong it will crash. And so one of the jobs, one of the very important jobs of the control plane is preventing that sort of madness and making sure that you're doing things that will allow your Envoy to actually function. Quick sampling, here are three control planes that are currently active in the Envoy ecosystem. Istio is primarily focused on service to service communications. Ambassadors primarily focused on edge communications. Those two sort of complement each other. There's also the go control plane which is actually from the Envoy people and it's really more of a control plane kit written in go. So a lot of flexibility, a lot of cool things you can do just be prepared to write a bunch of code is all. To pick on Ambassador for a minute to get a sense of kind of how this stuff works. Ambassador has a configuration that lives in Kubernetes resources. Specifically it lives in annotations on Kubernetes service objects. Whenever you change one of those then Ambassador gets notified of those configuration changes. It regenerates the Envoy configuration, feeds it to the Envoy that's running in the same pod as the Ambassador. And the thing that's kind of interesting about this is that on the one hand, since Ambassador is writing the configuration for Envoy rather than the human, we both get to make Ambassador's configuration much higher level than the Envoy configuration and we get to make sure that the Envoy configuration is actually functional. But the other thing that's nice is that Ambassador does not have to do the heavy lifting of actually routing requests and responses around. All of that happens on the Envoy that's being configured. To give you a quick example of what those sorts of things look like this is a simple piece of Ambassador configuration. Most of that's a bog standard Kubernetes service. The annotation in the middle is the part that's Ambassador specific. That's saying that whenever a REST request for the resource HTTP bin comes in it should be handed off to the actual httpbin.org servers which are not running in the cluster in this particular case. This is the Envoy configuration, well no, this is a small piece of the Envoy configuration that represents the Ambassador configuration we were just showing. The thing on top is an Envoy route which describes taking a request to the HTTP bin resource and then handing it off to the thing named by cluster underscore HTTP bin underscore org underscore 80. And then the thing on the bottom is the cluster definition which tells Envoy where to actually, the actual upstream service that is going to respond to that request. The split between the two is because you can have multiple routes that are all going to the same cluster. And a lot of the logic in Ambassador deals with the fact that you would like the human who's interacting with all of this to not think very much about that. You'd like them to just say this goes here and then let something else worry about defining the clusters. One of the other things that's worth pointing out is that if you misspell the cluster name and the route, then things won't work. If you miss any of the syntax, then Envoy will crash. So it's a little tricky sometimes. So all of that kind of wraps up with a couple of things that are interesting I think to call some attention to. One of them is that although those smart RPC libraries like Phinical and Histrix over there on the left have not gone away, old software never dies. It just gradually loses its user base over time. Since SmartStack showed up in 2013, it's been a pretty dramatic shift over towards the proxy inside card idea. The world has been becoming more polyglot and having tools that can work with that seems to have really resonated with a bunch of people. Another thing that's interesting is remember that HAProxy's original release would be way off the left side of this graph in 2006. Ingenix's initial release is actually a little bit to the left of Envoy. What we're showing here with Ingenix and HAProxy in 2017 is that after Envoy showed up, both Ingenix and HAProxy have started to undergo a pretty significant amount of development seemingly getting pushed by some of the stuff that's shown up in Envoy, which is kind of interesting. And in fact, as an example of some of the changes that have shown up there, one of the last things on this graph at the far right, SmartStack has actually switched to Envoy at this point. It's still out there, but it's no longer HAProxy. So wrapping it all up, as the world is moving more towards microservices, this whole concept of your communication happens over layer seven protocols that are well understood and well standardized seems to be very important. And being able to manage those layer seven communications is definitely important in that world. We're seeing multiple different proxies showing up evolving to give you tools for managing these things. They are of course evolving in different ways and at different paces. And we at DataWire tends to like Envoy a lot given the neutral governments, given its ecosystem. But we think it's very important to point out that most people are not using Envoy directly. Most people are using control planes to make their lives easier, which we also think is a good thing. And I believe that that is it. Any questions? Thank you. Can you please say a few words about performance differences if there are any between the three top proxies you mentioned? Mostly what I can say is that, okay, so probably the simplest answer to that question is that all of the proxies take performance very seriously. If you have a proxy that constantly slows down everything you're doing, then it's not terribly useful. People tend not to use it very much. Envoy is the one that I know the most about the internals of. And Envoy has put a ridiculous amount of effort into performance to the extent that every time I look at it, then it just constantly trips my alarm bells for premature optimizations. And I have to remind myself that they're not premature. They actually tested it and profiled it and then made the optimization. So Envoy is extremely fast. I don't really want to talk much about HA proxy or Ingenix in that respect, other than to say that the stuff that I've heard about with those two mostly has to do with the HA proxy, not really liking lots and lots of services rather than either of them introducing a lot of performance delay on individual requests. Does that make sense? As a minor question, are the Ingenix new developments just going into the commercial version or some of those in the open source? I think some of it is going into the open source but I think it's mostly going into the commercial version. So would it be fair to say that Ambassador is basically an Ingress controller for Envoy or is it not completely compatible with the definition of Ingress controller? Are you talking about Ingress as a Kubernetes resource or are you talking about, okay. Could I slot it in basically as a replacement for the Ingenix Ingress? Yeah, so functionally, yes. It is not a Kubernetes Ingress controller because when we started working on Ambassador and full disclosure, Ambassador is a data wire project. I work on it a lot. As we started working on it, we realized that the Kubernetes Ingress definition simply was not expressive enough for the sorts of things we wanted Ambassador to do. So it is used instead of an Ingress controller to do similar things but it is not actually a Kubernetes Ingress controller. Got it, thanks. Hi. So you just hinted about some premature optimizations. Can you talk about some of those? Let me first explain my bias there. I've been doing software for a long time and when I first started off doing software, I was writing in C, Envoy's written in C++. Those of us who have worked for a long time in C, particularly those of us who have worked for a long time in C on underpowered systems a long time ago where optimization was a thing. Usually got our fingers burned a lot by people trying to optimize before they understood what was an effective optimization and the end result of that sort of code is that you end up with code that's much, much more complex than it needs to be for arguably not much gain. There is code in Envoy that is surprisingly complex and is surprisingly complex in ways where when I first saw it, I looked at it and went, oh my God, they optimized prematurely, this is awful. And then I did a little bit more research and found that no, actually, they had written it the simple way first and then they measured its performance and then they found out that that bit of code actually was slow enough to matter so they went through and optimized it at that point. But it's so rare to find a product where the optimization has actually been driven by metrics that my knee-jerk reaction is still that feeling of premature optimization and it makes me nervous. I was wondering if you could expand on Ambassador versus Istio, both in what they do today and whether their goals are different? So again, full disclosure, I work on Ambassador so I'm biased. Istio's design, and I have talked to the Istio people but I am not one of them, so again, biased, right? Istio is designed primarily to tackle the problem of services talking to other services, the so-called East-West direction. So Istio worries a lot about things like MTLS and policy enforcement around security and a lot of things like that, which Ambassador kind of doesn't worry about right now. Ambassador is, the way we've sometimes described it is that Ambassador is meant to be an envoy for generalists. It's a way to put an envoy at the edge of your product so that you can make your services accessible without having to go off and learn to be a domain expert in envoy. So at present, Ambassador is almost entirely focused on the North-South direction dealing with requests coming into your microservices application and responses going back. Istio has of course been sort of moving in the direction of handling North-South as well. To me, that's still pretty young in their product. Ambassador has not so far really begun moving in the direction of doing much in the East-West direction. It's very much on our radar, but we're still at the point of looking over and trying to decide, how do we wanna cope with this? One of the challenges that Ambassador has always had is that if you take something that's very complex and powerful and try to present a higher level simpler way of dealing with it, then there's a real challenge in figuring out how to allow your simpler configuration language to not constrain people who wanna do complex things. And so we have to think a lot about how to do new concepts in Ambassador in such a way that they work and they give you a lot of room to do what you need to do, but they still fit in with the rest of Ambassador's model. And I guess we are done. Thank you very much. I appreciate it. Thank you, Flynn.