 Okay, yeah, we got people online, so we should just go ahead and roll Right. Good morning everyone. Welcome to our session on cloud native building blocks We're gonna be walking through an interactive envoy proxy workshop this morning So welcome to you and to and to everyone who's online is joining us there We appreciate you taking some time out of your conference day to spend with us here we're we're hoping that this is going to be a Really useful session because it is something that's hands-on So if envoy proxy is something that you're considering perhaps in a in a gateway context in your organization or perhaps in a service mesh context then Then this session is something that I hope will allow you to get hands-on and to really kind of experience what what Configuring envoy proxy is like and some of the key concepts and some of the most common use cases that people see When they are when they are digging into it. So my name is Jim Barton. I am a field engineer with solo I've been here for a couple of years before that I spent the the prior decade architecting operating Developing enterprise systems at Red Hat and and Amazon. Let me also introduce my co-speaker here Adam Hi, all my name is Adam. Sayah. I'm a field engineer with Jim working in service mesh and API getways for a couple of years now and yeah, I've been with Kampai for a couple of years too So yeah, happy to be here and I'll help you guys with the questions. Yep, and Well, Adam has another session here tomorrow We'll throw that over to him at the end and let him let him plug that a little bit But for now, let's let's let's dive into this session So this is going to be an interactive session and I'll tell you that we have seen some network problems in this room And so it's possible you you could have some issues with the kind of the interactive part of the Experience if so we've we've provided you a a chat link there if you go to bit.ly OSS dash envoy chat you should see an interface that looks something like this and You'll see over here on the right You can you can enter chat messages and we have right here in the front row Brian Godfrey'd say hi Brian Brian is also a solo field engineer And so if you have issues with the exercise part of this rather than us trying to handle questions interactively in the session just put put a message in the in the chat there and Brian will respond to you and We'll try to handle it that way We do hope to have some time for interactive Q&A at the end possibly in the last five minutes or so Okay, so Alright, so I promise you there will be very few slides over the course of this But I do want to just level set for those of you in case you're not familiar with with envoy proxy I want to just talk to you a little bit about what it is So envoy lift actually had a problem back in the early 2010s And the problem was they were moving like like many of us have from a monolithic software Architecture right they had a you know some some number of monoliths were fairly static Didn't didn't change a whole lot in terms of the the the you know network deployment characteristics And they started to migrate to a fleet of microservices to to deliver these same features One of the issues they had was they wanted to have an API gateway technology that was built for this kind of environment But they they surveyed the landscape and they didn't see anything that really fit the bill And so what they did is from the ground up they built their own They built their own proxy to to serve as an API gateway that was actually designed from the ground up to run in cloud native environments And to be dynamically configurable to be extensible from day one and that is and that is envoy proxy, so if you You know if you take a look at the at the story there Envoy proxy is something that's extraordinarily fast and scalable. It's implemented in C++. It's been battle tested Really thoroughly at this point It allows you to work at both the L4 and the L7 layer in terms of processing requests that are that are coming Into your application network and it was really built from the ground up to be a Dynapt to work in a dynamic services environment, so it was donated to the CNC F and and graduated a couple of years ago and is now really the basis for for a number of cloud native application Networking products including including service meshes like Istio as well as multiple API Gateways that are deployed in some some very high-volume enterprise scale system So the bottom line is that it's something that's battle tested It's out there and there's a there's a very large diverse vibrant community Open-source community behind envoy proxy that that supports its its ongoing development okay, so I Think the key feature of this session from my from my standpoint is the fact that this is a hands-on workshop We're going to be using the instruct platform To to enable that so if you have a laptop and if you'd like to follow along I invite you to to get that to to get your laptop out now and go visit Visit this URL bit.ly slash OSS dash envoy dash workshop and What you should see is an interface that looks something like this and so you can add this Content to your to your study room then you should see an interface like this Go to your exclusives and you can see here's the intro to envoy proxy workshop We can take a look at the track and actually I'm going to stop this and start over Absolutely, I'm headed there headed there right now. So so there is the there's the link bit.ly slash OSS dash envoy dash workshop And that should get you on the path toward Toward opening up your envoy environment or to to your instruct your instruct environment. So What's going to happen is when you start this it's going to take it's going to take about a minute or two for that to spin up and So let me let me go through a couple of other things while we're all collectively waiting on our little provisions compute sandboxes to to spin up, okay I Set it up to be anonymous so you shouldn't have to provide an email address or anything like that That's right. Yeah. Yeah, you should you should not have to so the question was about Do you require any kind of email address or anything like that to log in? You should not have to have Instruct credentials to get into this you should just be able to go through the series of prompts and and it should take You straight to the workshop Okay So there are five exercises We're going to cover in the course of this workshop one is on on envoy basics second is talking about a specific kind of filter that is is particularly a Common use case when on boy, which is the hhttp connection manager We'll discuss some and work through some exercises around security around observability using things like access logs and Prometheus and Grafana and then finally you'll get to test all of the skills that you're going to acquire over the next hour in Working through a debugging exercise So that should be that should be fun and also At the end if you are if you are interested One of the things we do it solo that has been very popular is we produce We produce badges on a variety of open-source technology, so ebpf Istio cillium and we've also produced one for for envoy as well So if you'd like to earn your fundamentals for envoy badge that solo issues There'll be an option to do that You can do it either you know in this room at the end of the exercise or you can do it later on if you would if You'd prefer that so So the exam is available right now. It requires you to score Better than 80% on all the questions. They're typically multiple choice sorts of questions And you can retake the exam more than once if you do retake it You'll need to use a different email address for subsequent attempts at the exam And so assuming you pass that the badge will be issued within a few weeks You'll get an email about that and it's of course you can suitable for framing or your LinkedIn profile Okay So at this point Hopefully our We should be good to go So can I see for up for those of you who are following along with instruct Can I get a thumbs up for if you're at this point where you see the start button here? Okay? So go ahead and press the start button. That's going to launch the instruct environment All right, and I'm going to turn it over to Adam to go through the the envoy basics exercise All right You guys hear me first Good awesome. Okay So, yeah, thanks Jim for the intro here. We learned how important is envoy in the up in the open source world It's kind of a founding block for lots of technologies out there now Does we're gonna go through a couple laps? think for in total Where every time Basically the environment is pretty straightforward We're gonna go through some commands here. They're gonna just copy paste And see how we can configure and void to do Security and we're gonna configure for routing and all this this stuff, right? Okay, so first thing first, let's talk about the basics of envoy, okay? So and for itself As Three different con three main concepts that we need to understand, right? There are the foundation of the configuration, right? So we're gonna go through this ones as the first exercise just to understand the basics of of envoy. Okay, so Envoy allow us to Basically open it's a proxy we open a port on That component the traffic is taking there we do some processing like You know manipulation of that or that traffic and then we we're gonna That traffic will will go somewhere. Okay, so that's that's definitely definition of of a proxy now in term of Envoy itself it has three different concepts But the thing is the first step you're gonna learn here is like how to interact with with Envoy and the goal of this workshop in In total the goal of this workshop is basically to provide Some easy examples here to help you debug or understand the Envoy configuration, right? A lot of people are already using Envoy in some some sort either embedded in other technologies for example Istio or Glue edge another open source tool, but basically there's a lot of Software out there that's already using Envoy the thing is Envoy is a black box for a lot of this technologies today So this here will help you at least understand the concepts for tomorrow to kind of push the debugging a step further To go and look directly into Envoy to understand if your configuration is correct or not Alright, so first thing we're gonna do here is to just start our examples The way we're gonna do it is we're gonna use Docker compose here It just to start the stack that contain couple services gonna contain Envoy itself that we're gonna interact with and it's gonna contain a demo service and authorization server we're gonna use later and Prometheus plus Grafana stack for monitoring and we're gonna see that later, too all right, so first step here is to take a look at basically how Our doctrine course compose look like and here we'll see that we have Envoy and then that you know, we open couple ports here just for us to kind of route traffic to and Basically, we have an upstream service, which is just demo service. We're gonna have an alt service Which is an authorization server. We're gonna be using later a Prometheus plus Grafana stack for monitoring Now let's move Here and actually just run the Docker compose up. Okay, so that will start this demo stack here For that just click on this copy icon here and then paste into the terminal All right, I'm gonna take a couple seconds to get ready and then we're gonna be able to interact with Envoy now Envoy itself has different way of interacting with right so we can either Start Envoy with the static configuration like from a file or Envoy can connect to a server to call XDS to fetch the configuration from right So that's basically the control plane way of doing it and we're gonna mention that later, too But for this exercise and for this workshop, we're gonna just interact with Envoy using static files Now every time we're gonna do any modification You're gonna be able to see the Envoy configuration by clicking on the Envoy Configuration tab, okay, so here on the top if you click on Envoy configuration You will see which Envoy configuration we're using at a specific time during our workshop in this first Configuration here we have nothing but What we call the admin interface, okay? And that's a good segue to understand what an admin interface is and then in Envoy Envoy provide a way to just see what's going on in Envoy see the configuration interact with Envoy at certain level for example Deactivating the health check or activating the health check and so on This is really important if you want to try to understand what's going on in your Envoy instance, right? And if you want to look at the admin interface Just click on the admin interface tab on the top right and this is connecting to Envoy And this is what you see when you run the admin interface. You're gonna see a couple important Concepts here. We're gonna talk about later like listeners and clusters and and so on But for now just get familiar with this with this interface look at the different options We have here for example one of the kind of important one we have It's called the config dump right config dump will just give you kind of a Snapshot of what's going on in Envoy at that time you can look at the configuration that Envoy has either from a file bootstrapped or what Envoy got from a remote configuration server Right now and we're gonna go back to this admin interface a couple times right so Now I think our stack is ready and we see that all our pods Sorry our containers are running if you can run on this Docker PS here You're gonna be able to see that we have our five services running. Okay, and Envoy is there all right Now let's talk about The first again, I said there is three main concept that we really need to understand today So let's talk about the first one in Envoy The One of the most important concepts is a listener. Okay a listener is a Network address you can think about it as a port that you open in Envoy to take traffic, right? This one when you create a listener you open the port Allowing a client to interact with Envoy Okay, so for example I want to open a port for example 8080 to take all my HTTP traffic For example and route it to to my back end But the first thing to do is to open the door right there. You're gonna Create a listener to find this port allowing the client to connect to so this is the first step What's we're gonna do right now? We're gonna create a listener to do this Let's just look at the first basically here if we go back to the Envoy configuration We see that we have we have nothing right? We have just the admin configuration But now if you do this get checkouts of the specific lab and if we go back to our Configuration here, we'll see that we added the Envoy listener configuration here You see we are adding a listener. It's gonna listen on the port a thousand and then it's gonna execute a filter chain That is a good segue into the second most important concept here, which is filters now as we open the port on Envoy we need to do something with that traffic and The way Envoy deal with the traffic Is through filters, okay, so there's two types of filters There's a filters, which is just a TCP the raw TCP filters operating and at an L4 level That type of a filter will interact with TCP directly doing a TCP proxy For example, or just returning the traffic right there Someone and then you have the HTTP filters and this one we're gonna see later This one will manipulate your health l7 level, you know processing for example at the application level for example, if I have You know a certain path I want to match on or a certain host or you know If when I manipulate the traffic to add headers and so on. All right Okay, so let's Start our listener here. So now that we have the configuration the only thing we have to do is to restart Envoy For for Envoy to pick up our new config Now that we restarted Envoy, we can go back to the Envoy admin interface and if you click this time on listeners You see the Envoy listener 8000 so that's the one we just created right now. Okay now Envoy and The stands it has a port open on 8000 that can take traffic Okay, and let's take a look at what happened if basically I hit this port here from the Envoy configuration I mentioned there's multiple types of filters, right and in this case here I'm gonna make it bigger here. Maybe easier for you to see so And in this case the filter that we attached to this flow is an L4 Like a TCP level filter that is just doing a direct response what I mean by that is doesn't do anything with it is just intercept the traffic and Once the traffic is hitting the sport 8000 we return something in this case We're just returning hell word directly for any type of TCP Traffic hitting the port The port 8000, okay, so now now that we have this let's let's test it So we're gonna go back here, and let's do a curl going back to our terminal I run this and you guys see here The hell word coming back from Envoy. So at this stage win stood two concepts of Listener opening a port on Envoy and the filter dealing with that traffic and doing something with it in this case We're just returning a direct response responding. Hello word to any traffic hitting the port 8000 alright now This is not the common You know scenario what we usually do with a proxy we intercept the traffic and we route it somewhere Right, that's the core concept of any proxy the routing part. So that is an Introduction to the third concept here is really important and Envoy cluster a Cluster is a Representation of a destination right so it's a grouping of endpoint that represent your back-end service What I mean by that is if you aren't when I route for example, if you are using Envoy and Kubernetes a cluster would represent your community service, right? Or directly the endpoint to the pods So that Envoy can route traffic to that destination now You guys saw that we created like a demo service when we just start our stack now We're going to create an Envoy cluster that represent that demo service, okay, so To do this we're going to just check out the new configuration and let's go back to the Envoy configuration here and Let just scroll to see what's what's the new configuration we have here And this is the important part here We see that this time we created a cluster named upstream You can give it any name and this one will point has a one-end point that points to The host upstream which is our demo service on the port 8080 So now Envoy understand this destination understand that I have that address or that Service I can route to it Now that we did that we have a cluster we can use it in a routing So the second thing we did in this configuration Remove we removed the direct response We had in the previous example that returned the traffic directly But this time we are doing a TCP proxy to that specific cluster Okay So for that we are using this new filter here, which is a TCP proxy filter And the only thing we're doing we're routing to the cluster called upstream, which is basically our demo service, right? So here we're doing raw TCP routing, right? We are not dealing with any l7 Level we hitting the port a thousand on Envoy It directs all the traffic all the TCP traffic to our service upstream on the port 8080 All right now that we have this let's just go back to Our terminal and just restart the configuration here and see what happens so if I do Restart of Envoy and now I do a curl Run this we get hell back hell back is the response coming from our demo service our back end So in this case here in this example, we use the three main concepts of Envoy We create a listener to open a port on Envoy We create a filter to do TCP proxying to a cluster Which is a representation of our destination? our back-end service Quick tips here. So basically tomorrow You are doing, you know, you are looking you want to look into your Envoy configuration to understand what's happening or if something's not working You need to understand that you can use the Envoy admin interface to do so and for that I want to just highlight a couple things. So let's go back to our admin interface. Let's say tomorrow You see Envoy not taking any traffic. The first thing you would think about is Why is is there any port taking the traffic on that specific port? I'm trying to reach first thing you have to do go go to the listeners see if there is any listener created there Okay Now after that you probably say, okay, so now maybe the traffic is taken But I'm not able to route to my back-end service Is there any back-end representation of my specific service? You have to go to clusters And once you go to clusters look for Basically your cluster name that you're looking for here upstream is the is the name of our demo service So we see upstream here and then look if it has any IPs, right? And when you see an IP here it meaning like this cluster has addresses so We have the representation of the destination. We have the name the cluster called upstream But we also have the endpoint address to where I should connect, okay? And there is a lot of important information here for example in in the upstream you can see Request success you can see the request total The number of connection all this is important to kind of Investigate at an early stage what's going on with the specific cluster, right? If you see unhealthy on a specific endpoint definitely means that these endpoints isn't healthy and reaching it will result in an error, okay? And more more into we're gonna talk more about metrics and stats later, but this is important to understand you don't need Lot to understand the basics To see if everything is configured well check your listeners check your clusters At least that's the first the first phase there and we're gonna look into metrics later because envoy creates a lot of metrics They can use to kind of investigate deeper. What's going on here? All right, so Thumbs up if you guys ready for the next Lab that's cool. All right going back to Jim for The next steps all right Thank you very much Adam. So We're gonna be building on these we're gonna be building on these core concepts as we as we drive through the next Next few exercises that the next thing we're gonna talk about is the HTTP connection manager, which is Probably the most commonly used Network filter within within Envoy very very common You know Adam showed you a couple of examples with direct responses in with using Envoy as a TCP proxy Those are used as well But what we see at least in in in our user base is HTTP connection manager is very very Commonly used so so hopefully you've been able to click into the second lab if you're following along and you should see a Display that's something like this. So What you saw and you know one of the previous examples, you know, we used a direct response from Envoy right to to report back the Hello World A response we were looking for but you know much more commonly what happens is you're going to use an HTTP connection manager You're going to have that potentially do some do some processing right based on What kind of what kind of request am I am I looking for what host what prefix on the on the request path that sort of thing And then I'm going to take that request and route that to some External upstream service to actually do the processing that I want and then deliver Deliver a response back to me So so that's what we're going to do is HTTP connection manager knows how to handle L7 request and to serve as a proxy for those and so what we're going to do here is look through initially a very simple HCM filter that's going to match on it's going to match on Host names right so we're going to look for a specific one called hello dot Envoy proxy dot I o we're going to look for We're going to look for a specific path Prefix match and then we're going to route those that request to our to our upstream service okay, so to to Kind of show you what that looks like from a from a pictorial standpoint there you there you can see a diagram where You can see our HTTP listener on port 8,000 And it's using a single filter chain that filter chain contains the HTTP connection manager Which has a single HTTP filter inside it and we'll expand this out in future examples But for now what you see is within that HCM filter There is a single HTTP filter specified and that's the router filter And so the router filters always going to come at the end of the chain and its Responsibility is to take the request and route that off to the upstream service And so there you can see we've specified in green there There's our upstream service and it's connected to to the actual Service on the back end that that processes the the request Okay, so that's what we're going to that's what we're going to build out here so let's Let's walk through instruct here to go to the right directory. We're going to check out the branch that has the the configuration for this example and if you look at the There we go if you look at the envoy configuration For this you can see a couple of differences From what we had before so you'll recognize some of it's the same right for example Here is the here's the listener that's a watching for HTTP traffic on port 8,000. Okay, and Then we're going to add to that right this new filter here call HTTP connection manager and within that There's some there's some basic configuration, but what you'll notice here is that It's it's so it's watching for traffic on this domain. Hello dot envoy proxy dot I o and it's looking for routes That match the path prefix slash hello and when it sees a request that matches those Criteria it's going to route that to this cluster called upstream that you saw in the in the previous example Okay, and then finally you can see here's a list of HTTP filters that are going to be Handled by this connection manager and you can see there's not there's a single entry here Which is the is the router filter? Okay, so it's simply going to take the request that comes in that passes these These predicate checks and it's going to pass basically package that request up and and route it to this upstream service So you can see what you what you saw at the end of the previous lab the upstream service is still there specified Exactly as it was before okay, so this is a configuration. We're going to be we're going to be working with here All right, so let's go back. Let's go back here and So to do that again everything we're doing at this point We're using static configuration files to to program our envoy proxy. So every time we change something We need to restart on voice. So we're going to do that here Adam will talk in our final exercise is a bit about control planes Which is a really really important feature on boy was designed from day one to be Dynamically configurable and the way that works is is when you attach a control plane to your to your envoy proxy So that as you change Configuration that can be served up dynamically from the control plane to the running envoy proxy So you're changing configuration, but you don't have to bounce your envoy proxy. That's one of the really key design design capabilities from day one of Envoy, but again until we until we get to that point what we're going to be doing here is Is restarting our restarting our proxy is it a question I can answer for you Yeah, that's a great question. So the question is we're using Docker compose for this exercise Obviously Kubernetes is much more common in the you know in the enterprise these days And so that's actually where we see where we see envoy being used most commonly certainly within within the solo business Is definitely in a kubernetes context now? You can configure your envoy proxy to manage services that are deployed outside of a kube context But in terms of the proxy itself and the control plane that supports that 99 percent of the time We see all of that deployed in a kube cluster the reason we chose Docker compose for this exercise is just is just for simplicity Right, we didn't want to have we didn't want to spin up a k3d or something like that when it really wasn't necessary To show the basic concepts, but yeah, that's an excellent question. Thank you for that So we have we have restarted our Envoy proxy here. So our our our Change is taking effect. So let's go in and you can see our curl command here We're still we're pointing local host port 8,000 right we're using the hello path Right. So this should work and you can see there's our host header. Hello dot Envoy proxy dot IO So this should check all the boxes for the you know the criteria that that our filters looking for and there we go You can see we get back, you know that the hello back response, you know If we if we change this request in some way, you know, maybe we used you know, let's let's use a Let's use a different, you know, let's use a different path here Right. What do you expect to see Envoy is not going to match on that? And so you're going to get back a you're going to get back a 404 Hey, this resource is not found error because Envoy didn't know didn't know how to process that, right? Okay Very good. So one of the themes you're going to hear both Adam and I talk about over the course of this exercise Is metrics metrics metrics Because Envoy publishes a whole a whole raft of them and they're very useful and in terms of in terms of really deeply understanding when you have a complex Set of systems that are deployed you want to understand what's going on inside them These these metrics are extremely valuable for that and so We'll just show a quick we'll show a quick example here Based on this exercise, we'll take a look at We'll take a look at this stats endpoint again We could do we can look at the same thing from the admin interface, but here we'll just we'll curl the endpoint directly and just Filter out what we what we're looking for and so you can see there's a metric Envoy cluster upstream request completed It has a value it has a value of one right now. There's one that's been completed successfully But if we you know if we go back we reissue that same command again Actually reissue the correct command again All right, so that was successful We hit the we hit the stats endpoint again now you can see that counter has been bumped up has been bumped up to a value of two All right, so Very simple, but really valuable when you're trying to get some visibility into what's happening in your system Also, just included here a little a little graphical interface is kind of a teaser So this is a this is a Prometheus graph that actually shows the value of this particular metric as it You know as it changes over time and so Adam will talk a bit more about some of the more advanced Observability characteristics in lab four Okay now The first thing that happens after you get basic functionality set up You know so you can connect to some some back-end system that's that's performing some useful service is Well, what happens if the service fails? This service is you know is fairly unreliable I need I need to be able to have a way without offloading the responsibility to the client application, right? I want to be able to use my proxy perhaps to Configure some kind of a retry policy so I can deal with Unreliability in the upstream service and so envoy of course is designed to handle that sort of thing You can see from the diagram here What we've done we're basically making a change at the interface between the upstream Specification in envoy and the service It's connecting to where we're going to put in an automatic retry loop on certain error conditions to read to You know to retry any failed connections. Okay. All right. Hopefully that all makes sense. So let's move to Let's update to that wasn't what I wanted. All right So let's let's check out the lab 2-2 branch here and The way we're going to the way we're going to test this out Is we have we have built our upstream application in a way that it is it is unreliable So all right, I'm going to I'm gonna I'm gonna turn back the covers here Just a little bit and show you Show you what I'm talking about. This is this is actually pretty sophisticated code here We thought about closed sourcing this because it's such a sophisticated service But you know since this is open source summit we decided to go ahead and show you what's going on here So so again really fancy stuff. This is a python app and in this python app. You can see You know 80% of the time we're basically, you know, this service is going to fail with a 500 error 20% of the time it's going to succeed again, you know, feel free to share this with your friends back home Okay, so So that's what that's what we're going to do here So we're going to to activate the unreliability in this service by simply passing in a header that says hey we want you to you know behave in an unreliable way and It will it will do that for us, right? So if we just do if we if we curl this endpoint with the unreliable header What we're going to see, you know again 80% of the time is you're going to see a 500 Internal server error, okay, and again if in your world you're one of the lucky 20% Who had it succeed on the first try go ahead and just reissue the command and of course it will it will fail on subsequent tries All right, so that's our that's our reality And then that's the reality a lot of the times in the real world is you have an unreliable system that you know Nevertheless deliver some piece of mission critical functionality and you need to be able to accommodate You need to be able to accommodate its unreliability in your in your proxy implementation So here's what we're going to do we're going to make a we're going to make a small change to our configuration and The change is here, so you'll notice this is in our HTTP connection manager config and So whereas before we just had a straight We just had a straight routing config that would attempt to route the traffic And if it succeeded it succeeded if it failed it failed notice we've now added this Retry policy stanza to this configuration and so what we're doing is we're saying Anytime you see an error that that looks like a 500 error. We want you to retry that up to 10 times Okay There's actually a whole list of of predicates that you can provide here to do the retry out We're simply using you know a single one of those but you can chain those things together You can retry on 500 class errors or 400 class errors or a whole host of other things if you check out the envoy Documentation in this case we know what the failure mode looks like for this unreliable system And so we're specifically saying we only want to retry on these conditions So very simple change, but it adds a really nice capability that makes life for our client applications Hopefully quite a bit easier Okay so Let's there's our config All right, we've talked about that. So again, we've changed the configuration. So we will need to restart our envoy proxy, okay, and And what I want to do so you can actually see the impact of how this works We're going to switch over to another Terminal tab and we're actually going to tail the logs of this application and so that will show you Hopefully what's what's happening here with our system? So you can see here's what's been happening previously We've had a couple of 200 successful Service requests notice we also the most recent when we issued here where it failed You can see there's the you know, there's the 500 failure Request in the in the logs for the for the upstream service. So now let's see how this behaves when we Now that we have this change in our configuration to retry on 500 failures Okay, so let's try that again And you'll notice what we see on the client side is right. We just get hey it was successful You know, I got a 200. Okay, everything's good, but if you go back to the logs, you'll see what happened Under the cover. So so check this out. What happened right here. You can see Beginning about this point. We had a string of what it looks like about eight Failures right before we finally got a single success. Okay, so you can see the the the back-end service was indeed Unreliable, but our envoy configuration allowed us to to mask that from the from the client Okay You got an error and so it is possible. You'll you'll get an error, right? So, you know if we only configured this with 10 retry Yeah, depending on the number of retries you configure if it's you know If like if we had gotten 10 failures actually if we'd gotten like one or two more failures here It would have failed it would have failed us as well. So yeah, so I'm sorry It shows 10 in the right so so it's you know There is a you know, I guess we could do that we could do the calculations But there is a small probability that you would get 10 consecutive failures and and in fact the whole request would fail So, you know an exercise for the student if you'd like to try it later, you know increase the retry count And and you know and and see see how that would would impact things. Okay, but hopefully you you know Rather than building an ultra reliable service here. We're trying to to communicate the concepts So hopefully this this this did a decent job of that. All right, so All right now I want to show you So when you add this retry Configuration to envoy there are of course metrics that are Provided by envoy that capture what's going on there. So let's take a look at Some of the relevant metrics here Okay, so you can see those there on the left and so the you know The your numbers will likely be different from what from what I'm showing here, but you can see Right for this particular upstream You can see there are a total of 10 requests that have been that have been issued against it Okay, you can see that You know one of those one of those requests that were successful You can also see that you know, there were nine of those requests that have failed, right? They were specifically that failed with a 500 a 500 response but then you can also see that from the metrics that Those those failures, you know all nine of them failed all nine of them were retried and then finally You know one of them one of them succeeded here Okay, so a lot of good information you can glean from the metrics to understand what's going on Inside your inside your system, okay? So that is an introduction to HTTP connection manager and how you do some basic basic routing config what we're going to do now if you're if you're if you're where I am at the bottom of the exercise hit the check button everything should pass hopefully and Then we will move on to Lab three which is about securing traffic. So for those of you are following along can I get can I get a thumbs up? You're able to okay good. I see several thumbs out there. So so that's a good sign, okay? All right, let's talk about security using envoy and specifically we're going to We're going to talk about a couple of very common security scenarios that that you're going to encounter the first one is basic Security of the communication channel between the client and the proxy using using TLS, okay? So we'll start with that that's a pretty simple exercise But we'll get we'll get through that show you how that works and then then the second one is We're going to look we're actually going to make our flow a bit more complicated by taking a look at at a field at an envoy HTTP filter called X off Z and what that does it adds an authorizing authorization component to our Request flow so that before the request gets routed to the upstream service We're going to pass it through a filter that says hey Let's make sure this request is properly authorized before we route it, okay? So those are the two things we're going to we're going to work through in this exercise Alright, so let's just if you take a look at lab one. We're going to talk about TLS. Here's the here's the diagram We're working with this looks very much like what we did before with a couple of exceptions number one You'll notice we're going to change the the listener port here, right? So instead of port 8000 which is what we've been using before for straight HTTP We're going to switch to port 8443 for HTTPS now did we have to do that? No, we didn't why did we do it? We did it just because of you know conventions right so the convention is HTTPS port is going to look like like 443 So so we're making that change and then notice also we got the padlock on the connection between client and server there So we're actually going to encrypt the communication between the client and the and the envoy proxy All right, so that's what we'll do in the first part of this exercise So let's Let's check out The content for lab 3 and that okay, I got to go to the right place first first Let's switch to the proper directory, then we'll check out the lab 3 content All right, and so here is here is what's going to change In this exercise you'll notice a couple of what one small change one larger change So the small changes were simply we're modifying the listener to listen on a on a different port 8443, okay And then also we are modifying our our HTTP connection manager config to use To add this transport socket stanza, okay, and so what that's going to do it's going to point to a self-signed certificate that we've already pre-configured in the environment and We're going to require clients to present a certificate that will then be they'll then be verified, okay so Pretty simple stuff the rest the rest of it's the same the routing config is the same the upstream service is the same We're simply adding this This this certificate. Yes question That's right. Yes, so this is this is straight So the question is how does the transport socket identified with the listener and so you're right It's just it's just the way the yaml is structured So, you know yaml uses indentation to to indicate kind of containment and that's what that's what this is showing here And also the TLS level is you know is operating at an L4 level So it doesn't it's not operating at you know the HTTP level where we use the HTTP connection manager. This is why so you terminate TLS and then you deal with your l7 traffic Yep Very good Okay, so we have we have made that change and we are going to restart our Restart our envoy proxy. All right, and what we're going to do Let's go through let's go through a couple of a couple of modes that won't work And then we'll show the one that that hopefully will work So let's start off with Let's let's say we try to use the same call sequence that we used before where we're trying to connect to the HTTP listener on 8,000 well, that's not going to work anymore, right? We've taken that out. So that's no good Let's say we correct this to the right port 8443, but we don't change the protocol We're still looking we're still trying to use HTTP. All right that similarly that's that's going to fail and Even when we so if we correct both the protocol and the port number and we try to we try to do that We still get a warning from curl because this is a self-signed self-signed certificate and and so it's basically warning us Hey, you know this this may not be safe And so we will go ahead and add the minus case which just to say yeah We understand the risk go ahead and go ahead and process this and there then you can see The the HTTPS call works Exactly as we expect it to All right So pretty simple pretty simple stuff pretty easy way to add TLS support to To your applications without modifying the the underlying applications, but to handle it through the proxy. All right Now let's let's talk about authorization now in this case We're going to add something new to our environment. So this is going to be a little more sophisticated flow So we're going to add a filter that before we route off to the upstream server We're going to check and see hey is this request properly authorized so you can see Within the diagram here right the connection manager has two two HTTP filters one is the x-toff filter which is going to be processed first Right if it gets the thumbs up that this is good Then it's going to go ahead and use the router filter to send the request on to the upstream cluster okay So if that if that fails if the authorization filter fails Then by default what's going to happen is we'll get back a 403 Forbidden response, although you can configure that to behave differently if you'd like so The way it operates again We've written we spent a lot of time working on some really sophisticated Authorization code so again It's just it's an external Python service that looks for a header named authorization With a value of workshop if it sees that it gives a thumbs up if it doesn't it gives a thumbs down so pretty pretty sophisticated stuff here Okay, so let's let's apply this change alright, and Then we'll take a look at the we'll take a look at the modified config So there are two two significant changes here Notice what we've done is we have added a new cluster right before we had a single cluster Which was the upstream cluster. That's our actual application service now. We've added something new We're calling it the off cluster. You can see it's you know, there's a host out there called off listening on port 8080 and so when we Encount when we when we encounter a request right? We're going to route that first to this off upstream Right, and if that succeeds then we're going to send it to the to the to the upstream for processing So you'll notice that we have added to our HTTP connection manager We've also added a stanza so you can see there's our router stanza before notice That's at the end of the chain We've added to the beginning of the chain right this x-toff filter with with some configuration around You know where the request what upstream the connect the request is going to be sent to To handle the author quest and timeouts and that sort of thing so there's actually there's a whole library of Envoy HTTP filters here if you take a look at the Envoy proxy docs, I mean you can see There are all kinds of things that you can attach to your Envoy proxy We're obviously just scratching the service the surface in a you know in a 90-minute Exercise, but there you can see there's the documentation of the x-toff filter But you know this just gives you a sense of the kinds of things that that you can do and so there's a lot of flexibility And of course you can build your own filters, right? So if if Envoy doesn't come with one out of the box that that satisfies your needs you can build your own filter there even Filters like web assembly filters which should be in this list down toward the bottom Yep, there's a wasm filter where you can actually plug in your own You know rather than build a custom filter for these use cases You can use a wasm filter to as a as a you know a sandbox container for your own code That's written in save you know t plus c++ or type script or or go or whatever language you choose to implement your To implement your own your custom logic So there's a whole bunch of flexibility you have here definitely You know check out what comes out of the box because you as you can see there are a whole lot of a whole lot of use cases That are handled straight out of the box okay So those are the two changes we've made here We've added an x-toff filter to our to our filter chain and we have added an off cluster That it's going to that it's going to delegate to alright, so let's Go see this in practice. Alright, so we've already checked out this lab config That's what we did and so now we'll go and restart on void to pick up this change and So let's let's test a few options here. See what happens the first one we will test is You'll notice we're not We're not actually supplying any any off header at all And so what do you expect to happen you expect to see a 403 forbidden response and that's that's what we get all right Let's try something different. Let's actually supply the authorization header But we'll supply it with without the proper key and again What do you see you expect to see a 403 forbidden and that's what you see and then finally we will do it the Proper way with the authorization header and a value of workshop We'll try that out and then it passes the off check It's routed and we get back the the hello back response that we want with a 200 HTTP response code okay, so Working just as we expect now. I know you're not going to be surprised at this point to hear that there are metrics that are associated with these With these filters with the off filter specifically and so you know we can take a look at we can take a look at that and You can see some of the key Some of the key metrics that came out of this exercise that we just did so you'll notice Here in our upstream there is So this is this is a metric for the cluster specifically for the up upstream cluster You can see there's the name of the filter X to off Z and you can see hey There was one request that came in that was okay, right? That's a one and then right above that you can see there were There were a couple of requests that came into that cluster that were denied. There were two of those right You'll also notice there's a different set of metrics here that provide basically the same information But on the on the ingress point so this is obviously this is this is collecting all of the Collecting all of the upstreams that might be associated with this with this ingress point And so what you would see if we had say you know ten services instead or ten upstreams instead of one You would see the individual metrics for the individual upstreams, and then you would see it aggregated You'd see all those metrics aggregated under this under these these metrics here Okay, so a lot of value that that you can derive from these from these metrics all right So Let's do a check Make sure everything's good. Give me a thumbs up if you're tracking along with us here and We are going to move on the final two exercises Adam is going to talk to us about Observability, and then we're going to test all of your skills with a real live debugging exercise Evil laugh Yeah, I'm ready for it Okay, so Now we are You know you guys are solid and boy, but use a lot of metrics for different Things like you know any filter can have its own metrics. We saw the ACM metrics. We saw basic cluster Metrics and so on now it would make sense To kind of collect all these and create dashboard out of it, right? I think that's the most common scenario That's when once you have envoy there to deal with your traffic you want to have a mechanism to Aggregate all together in in one specific graph that specific graph will give you the How does you know your system behave which well you can create some Alerting around it if something is not working fine. You can just you know alert To operate and see what's going okay, so first thing here is to you know We already wired everything for you guys to take a look at the environment and what we did You'll create this graph fan at that tab here. That is already populated with a demo You know demo Dashboard for that what you have to do just log in with admin admin since it's like the default Credentials here Skip to the graph finish the dashboard and then here on top just Well first select the right time frame Let's say we're gonna see what's happening here in the last 30 minutes So and then let's click on the cluster upstream, which is our You know the demo the demo service itself And you're gonna see that there is certain metric you can collect for example We can see here how many odd denied to that endpoint we have that is result of the previous lab where We did some you know we made some requests without passing any Auth heathers and that's resulted in the service been denied access to right We also you know generated some some issues with like 4xx in this case isn't representing our Our 403 which is authorization still errors. We see our success rate for example If there is any you know like we saw some 500 generated by the upstream so we can collect this one's too There is a lot of metrics that we can use a cluster membership is really important to basically saying you know Like how many healthy upstreams I have The basic endpoint in that case now Let's take a look at in the quick example you can even create your own dashboard, right if you do add panel and You can just you know browse all the metric that you have available. So here once you you want to create your Query can do your own query dashboard. It's pretty small here Let's go back to something different We can do New panel be good, but maybe I should zoom out. I don't know The UI is not helping me to do what I was trying to do here, but the thing is Long story short the only thing you do here is to create your own you can create your own panels create your own dashboards to If you need to find your own dashboard, you can even template it for here For example, we have the upstream and it's actually listening to all our clusters So in this case I can switch to to the old server to see how it behaves and so on, okay Now this is really important because Once you once you when you you deal with envoy in production There's just a matter of time when you get in need to debug your environment Okay, so having access to metrics is a key thing trying to understand all the metrics you can use Try to create dashboards that represent enough data. For example, if you're doing off Try to collect the off metrics if you are doing, you know retries try to get the retries metric and so on So that's that's really important to understand. Okay, so again this dashboard here It just an aggregation of the metrics that are exposed from the stats here, right? So there's a ton of of metrics and you can all collect that and print that in a really meaningful way All right Now let's talk about one the most important thing One when you you want to debug and voicing what's going on, right? And that is the ax locks. Okay, we see the metrics metrics are great They'll tell you what's going on and so on but access logs are also really important ax logs will let you Let envoy print meaningful logs as you can collect and put in Any, you know log aggregator system you have a splunk or data dog or whatever you need and then you can always You know if something happens you want to just go back to your Log system and see see what's happening there. See if there's errors and so on so let's do this We're gonna configure our Access logs To do this all you have to do is to add this ax locks configuration to Envoy and if you go back to our configuration here, we can see That's we added The access logs operation in our HTTP connection manager filter. Okay, and this is using the default just kind of a default Log string, I'll show you how it looks like here if we click on the link So it's using From the from out the documentation here. We can see sorry if we can click here It's using the default format, right? So and again go back to Envoy documentation to It will help you understand all the options you have there and see that the default format We just have the request method Kind of the response code The bytes received and so on and then some meaningful information here though You can enhance that with a ton of of Other, you know meaningful configuration here. So for example, we can get You know response rail bytes or things that that's really matter But things I think if I have to pick one One thing here to really stress is the response flag. Okay Response flag by default are there in the default configuration But now if you increase your own config You need to define The response flags response flags will let you know What's going wrong with your system? Okay, so And I mean if you get 200s, okay all the time that's fine, but you will see That's a good deal, right? But if things go wrong, you want to know what's happening there? You want to see What you know why Envoy is not able to handle that specific communication? All right, uh, so let's go back to our Example here now that we have the x-logs configured Let's just restart Now we have now let's make some calls, right? Okay, let's run some examples here. We got some So many 200s, but let's take a look at the Envoy logs So if you do Docker compose logs Envoy You're gonna see this new There's five logs here, right? You're gonna see we we did a get on slash. Hello Five times and you see that it has different different request ID Basically every time it was a new request and you see that everything was fine So the response flag is not populated. It's just a dash when it's a dash to good news Right. I'll have to worry there If you see a code there Then that's Something is wrong there and you need to investigate it to investigate it Just take that code and compare it In so in the access logs you'll be able to to compare it with what it means But we're gonna see that in the next lab. So at this stage Are you guys ready for the next one? awesome all right okay, so It's taking some time here to mess up the environment Right and give you guys a really challenging exercise tough one Yeah, so basically what we're gonna do here What we learned today is the basics of Envoy we learn what's a listener. What's a cluster? What's a filter? We saw how to do authorization of an odd z in Envoy we saw also How you know how to configure metrics collection and access logs. So at this stage You guys are probably familiar with the basics now Let's put all this together into a small exercise and the goal here Is basically to put you know to investigate what's going wrong with one specific system so Now let's go back to the folder You know where we have all our configuration. Let's make some calls Oh It's not working. No healthy upstreams. That's bad Let's fix that. Okay. Now we see that, you know, we have no healthy upstreams Well You know at this stage we see that Envoy is returning something so The port is open. At least I'm sure that the listener is not the problem at this stage But I can't access any healthy upstreams. Something is wrong there So the next step to do is to investigate what's going on What we can do is to use the Access log that we just created previously in the previous lab to see what's happening. Oh So I run a couple tests here and I see that this time It's not just a dash Now it's u h So there's a code there meaning something is wrong Okay, so now that I have u h I need to invest investigate what what it means If you go to documentation here, I did, you know, we have a link here to To the list of all the option that you have with the x logs Envoy response flags values and you can map that to what it means in this case u h just means no healthy upstream hosts in upstream cluster In addition to a 503 response. Okay, so Basically our back end is having a problem. Okay, so this is the first thing try to grab the response flag That will give you the first indication where you guys need to dig more into in this case We saw that our upstream is the problem. So in this case, I'm going to investigate my upstream more All right, so what about the Actually, what about the The grafana right grafana is important. We talked about metrics. Let's take a look here and see what's happening So let's say last 15 minutes And I want to look into the upstream because basically that's what Envoy is saying my upstream is having troubles. Whoa Okay cluster membership Did a dip down here. So Something happened with my with my cluster. Okay, so First hint was a response flag second hint was basically our metrics to see that our cluster membership Is having a trouble here if you go back even to The Envoy configuration like the admin interface if you go to clusters Right Can anyone tell me here What's missing on the upstream cluster? the Endpoints Okay, so we have the representation of the cluster like I mentioned earlier. We see upstream is here Or you see no eyepiece for it. Okay, so basically Envoy doesn't know Where it needs to talk to and and that's uh, that's a quick indication that If I don't have any endpoint, obviously, I don't know how to talk to but if I don't have any endpoint, there's mainly Because it failed all the health checks and Envoy just decided to remove that specific endpoint because there is no need for it It's it's useless. All right So Well, that's giving me a good indication that something is definitely wrong with my upstream service Or at least I don't know even if it's available. So what I can do I guess my batch is just to check if my service is there. Oh My demo service is not here Really tough debugging, right? So at this stage I'm just gonna start my backend again in a now if I do some calls Awesome My back in back in is up again. So yeah, really tough, right? If you do refresh here kind of if I do a back On a fresh page if I do clusters again here upstreams now has its eyepiece Okay so Again, that's the end of the demo example Give you guys a really hard time there, right? But at least we kind of you know We were looking into using different things we learned today to put everything together to understand how to debug an easy example in You know in Envoy now I want to take just five quick minutes to talk about the control plane now We did the debug exercise We saw the Envoy configuration I don't know if you guys saw that Envoy itself to configure it For only a small example here. We did nothing crazy. We just routed traffic to an upstream We did some off there and you see all the Configuration we have here, right? And that can just keep growing if you want to add like some fine tuning like retries and And someone like many things can can like make this configuration really complicated If you think about it specifically if you are Dealing with a large system or if you're going to use Envoy with thousands of services This will not be manageable you can't just use one YAML file and If someone just mess up one tabulation it just brings down 10,000 virtual servers, right? You don't want that, right? So the solution to that is to use a control plane Envoy is a data plane Envoy is there to route traffic But you need a piece of configuration A control plane is a server is a mechanism that allows you to as a user push Meaningful configuration that can be validated Uh, you know and then push down to Envoy and that specific configuration can be easy Like human readable and so on everyone can can collaborate without impacting each other And that configuration is going to go down to uh to Envoy. There's many examples of of control planes out there These two is one, right? glue edge one of the one of the open source tools we have at Solotio is Used a lot as a control plane for Envoy. So what you do is basically if I want to expose the service I'm just going to create a meaningful small YAML in the cloud native way Especially if you deal with Kubernetes you can create like for example, what we call upstream It's just a representation of destination. It's a small YAML. It's nothing and it's allow you to To simplify your your configuration and we can probably take a look at at glue edge here Just from from a hello word standpoint just to get familiar to to understand how how how um A control plane operates. So if we do guides, sorry right guides Maybe a hello word here so basically I don't know if you have any YAML example So basically what you're gonna have to do if you want to find like an Like pointing to a service instead of going and creating a cluster and then creating the routing and all that stuff You definitely just create a YAML that you put with your application Your helm chart when you deploy your application it deploys this You know Custom resource to current is a control plane will understand this specific configuration Then compile it in a certain way push it to Envoy Envoy will route traffic to your upstream or to your demo application. Whatever you want to use, right? So really something you guys Probably should look into is using. Oh, sorry. Where is it? They got the wrong things. Yeah using Using control plane Istio Glue edges many that are operating on top of Envoy all right, so That is the end of this workshop. I hope you guys learned something today I'll invite my colleague Jim here on the same with me today. So we we we basically covered A lot of the basics today, right? So regarding Envoy how to deal with uh with listeners and upstreams and filters and so on we looked into We looked into metrics and so on but Envoy itself is really complicated, right? This is just This is just basics And we hope to see you Soon for more advanced one We're going to talk more into details to kind of fine-tune this configuration even even You know to deal with more complicated configuration there. All right. All right. Thank you, Adam So if you are passionate about application networking solo.io is hiring we'd love to to chat with you about that We have yeah, we're really busy right now. So we'd We'd love to talk to you. Um, also, uh, the we have a presence here at open source summit So in addition to talks, we have a booth s 25 back in the back corner Feel free to come and chat with us further if you'd like to hear more about about product offerings And also I want to invite you to another talk on a different subject that adam is going to be giving tomorrow So yeah, I hope to see you soon. Like it's tomorrow. We're going to talk about eppf So if you guys have any interest there, please join it's at uh, I think you have to noon tomorrow here All right Very good. And uh, also now the uh, the exam is open So if you're interested in getting a fundamentals for envoy badge We invite you to check out the link there bit.ly slash envoy dash exam Or you can scan that qr code and it will take you there And uh, yeah, so try your luck. I think we we've delivered this We've delivered this at uh, at kubcon and I think it was like 60 70 of the of the room who took the exam Past so I'm sure this room probably be up in the high 90s or 100. It's clearly very sophisticated All right, uh, we have uh, we have a couple of minutes left if you uh, if you have any Questions, we'd uh, we'd love to take those. Yes, sir Envoy itself has an integration with like remote routers. You can just configure it directly. I don't know if you have it there All right. Yeah No remote. I mean upstream. Um, no things downstream. Oh, yeah. Yeah, it's gonna be downstream Yeah Yeah, yeah, yeah, right. We have what what adams. Yeah, that's right downstream remote address So definitely that's just a part of you know, when when you can figure When you configure your, um your envoy, um You can definitely say okay use remote address and and that address this this specific field Can be printed directly in the x logs that I shared previously in the lab for and that's going to print your your Your downstream max, which is the client ip anything else Really, thank you all for your time today. I noticed a lot of people out there Clicking along with us on the laptops. So appreciate you taking advantage of that and uh, yeah, I hope you have a good conference We'll uh, we'll we'll see you around Thanks very much