 Now, I like a little bit of history before talk, so I'm going to do one for you guys today. Back in the 18th and 19th centuries, the coal mining industry transformed that world into the world that we recognize today. It was a huge agent of change and it fueled quite literally the industrial revolution. Now you and I probably don't think about coal mining too much these days, at least not compared to back in the 50s when they would send out movie reels to theaters and show a little mining preview before the movie started. But coal mining, the industry, the process, it's still pretty interesting. For example, digging out a mine, it turns out there's water in these mines and so you can't just go to Abby Kearns' favorite store and rent a water pump from Home Depot. So you actually have to pump out the water the old way using horses or in later steam engines. And with that water out, you finally keep digging deeper and deeper and finally you reach some depth of where the coal is, maybe around 50 to 100 stories deep. But this newfound coal is just one giant mass and you can't easily transport it out. And so they would bore seven-foot holes into the coal and place six dynamite in there. If you think that's cool, nod your heads. And then they would detonate it. Boom. Now you have shards of coal everywhere. So they load it on cars, they surface it to the top, refine it, clean it, clean it again, and then commercialize it. So pretty interesting indeed. But not only was it interesting, it was also quite dangerous, unfortunately. And not only were the miners and the carpenters and the engineers subject to physical harm, they were also exposed to a chemical one. And they were finding a lot of these workers were collapsing and passing out and dying. And they weren't sure what they were dying from. And it turns out that they were dying from carbon monoxide. They were being poisoned by gas that they couldn't see or smell or taste. And it was a gas being probably generated by the gas lamps that they were using and probably explosions like these ones. And it wasn't until the very late 1800s, early 1900s that a Scottish physiologist by the name of John Haldane came up with this idea of sending a canary into the coal mine. You guys heard this phrase before? All right. But why canary? Why this small yellow bird? Well, it turns out the avian respiratory system is quite unique in that when birds are breathing, they're constantly breathing in oxygen even as they're exhaling. And it's probably to power their cute little wings and their high metabolic rates. But when you take a burge, such as a canary, that is ingesting twice the rate of oxygen or air around it, and you place it inside an environment such as a coal mine that has potentially dangerous gases in there, it provided you with some early detection. Now, we too in software have our own environments. We have Dev, Test, and a whole slew of other lesser environments. But there's one important one, and that's production. And so we too could use something like a canary to provide us with this early detection from anomalies and misbehaving code. And with that, I'd like to invite you guys and welcome you to this talk about dynamic application availability, where we're going to talk about a couple deployment use cases or strategies, namely blue-green deployments, and that's more awesome cousin canary releases. We're going to leverage an application called Zool, and we're going to show you demos of how we can accomplish some of those use cases. Up here on stage with me is Grant Deshazer. Grant, say hi. Hello. He's a teammate of mine, brilliant developer, and he's going to be doing all the demos today talking about Zool. And my name is Jason Goh. Everybody calls me Jago, so feel free. I'm a tech lead, and both Grant and I work for Charles Schwab in our automated investing group. Now we probably have a mix of people here, so I'd like to revisit the HTTP protocol. It's something that we probably all use and love, I hope, to some extent. But it's a protocol based on a request response architecture. You send a request to some specific URL and you get some response back. But I'd like to talk about these responses just a little bit more and perhaps put them into a couple buckets or categories. And that first category are direct responses, where the data itself about this request is directly inside the response. Now in this case here we have a JSON object being returned back. This easily could have been a 404 with some kind of error object. And what do you think the other category is going to be? Direct responses or redirects, as a lot of people call them. And so instead of the data being inside the payload itself, you're getting a location header instead where it's telling you where potentially that data might live. But now I'd like to talk about reverse proxies. It's called this because of the direction the proxy is facing. It's facing away from the internet and towards your internal network, as opposed to like a forward proxy where it's pointing the other way. And generally this is how it works using this example that you see here. Sit somewhere in the middle and you're requesting some kind of document index.html in this case. And you make that initial request, that first request into the yellow box that you see here. But that first request is placed on pause while this initial service makes another HTTP request downstream to retrieve that document. And once that response comes back, it unpauses that initial request and then on it goes. Now we should probably give this yellow box a name, this proxy. And so enter the application gateway. Now this guy has many other names. You might hear it as just gateway or API gateway or edge services. But we'll stick with application gateway for this talk. And again, it sits somewhere in between the client and your downstream service. And in most cases, it's going to be doing reverse proxy. But in some other cases, a smaller amount there, it will be doing redirects for you as well. And this yellow box is going to be bundled with a bunch of other functionalities, as we'll see later today. Now since we're talking about gateways, let's talk about some examples of applications that exist now. And since we're going to be talking about Zool today, here's this guy here. But Zool actually iterated to a second version. And I think there's actually a third one as well. The difference between Zool and Zool 2 is that the first Zool uses a servlet architecture where every single request consumes a thread. Now as opposed to Zool 2, it's using an async model where it only really needs one thread to service all those requests. Another one is Spring Cloud Gateway, very similar functionality as Zool. But this one is also based on an async model. And then finally, I'll just kind of run through this kind of fast. Engine X is another popular one, big IP. And finally, we've been hearing a lot about Istio and LinkerD, not so much LinkerD, but Istio. These guys are gateway services, but they're a little bit more than that. They're what you would call a service mesh. So now you're probably asking the question, Jago, why gateway? Why add all this extra processing and additional latency and complexity to my system? I'd probably reason about it in a couple different ways. The first one are the runtime benefits that you get with having a gateway in place. You can log these requests. You can log the ingress in the egress request. And you can compute response times and trends out of that. This produces telemetry and insights about your applications. But also it can provide security, authentication, authorization. You might be getting some kind of opaque token. And you want to convert that to a token with claims behind it. It can provide caching or responding to just static content type of requests. It also provides dynamic routing, I'm sorry. Now the second way I would reason about it are deploy time benefits. Because essentially you have this gateway that would have a routing table inside of it to be able to do dynamic routing, you'd be able to light up deployment strategies like blue-green deployments and canary releases. And with that I'd like to turn it over to Grant. Thanks, Jacob. So very briefly our first demo is going to be about blue-green deployment. It's going to be pretty straightforward and not so complicated. Just to give you an idea of what a blue-green deployment sort of looks like is imagine we have a cloud situation where we've got some sort of gateway in the middle here between the incoming requests and our services. And we have our blue application, which is our current version staged and running with all traffic coming through the gateway to that application. And now we want to stage our next application, the next version, our green one, into the cloud environment. And with a software switch, what we'll do is we'll actually change the amount of the traffic that was originally going to blue and we're going to switch it to the green application. And this is essentially a blue-green deployment. Of course, there are several advantages to doing this is that there's no downtime because we're just changing a software switch. Both of the applications were already running. And then let's say the other advantage is that when we decide that green maybe isn't really what we wanted, we can switch back to the blue instance just as easily. So as far as the components for our demo go, there's not very many of them, but very briefly, first one and the main one is our Zool server. Zool is a Netflix open source software. Spring has adapted it into a package called Spring Cloud Netflix. Alongside of that is Eureka, which is going to be doing service discovery. It's another open source tool developed by Netflix and adapted by Spring. We have a config server, which is serving our configuration values to Zool. And we have a .NET Core web API, which is serving basic JSON. And we have an Angular application running on a .NET Core platform for our blue application. And then we have the same thing for a green application. So this is, we'll go to our configuration for our blue green deployment. Pretty much when it comes to how you configure Zool, there's basically this structure for pretty much every Zool application. We're going to have a Zool object in YAML, and then we'll have a Routes object beneath that. And then pretty much anything underneath that tells Zool which routes are you wanting to actually handle. This first key, the blue green demo web API, is a named application that has registered itself with Eureka. This key is actually how Zool looks up the registration in Eureka, and will get the actual location of Eureka. The way we actually talk to that is through this path here, this API slash v1. And your browser would be something like zool.com slash api slash v1. And then all of this stuff that comes after the v1 will get forwarded to your downstream service. As far as our blue green demo goes, we're going to have these two paths here. Normally in a production environment, you may only have one of these actually active at a time. And when you're ready to go ahead and switch it, you may actually just change this key value. But for the purpose of the demo, we're actually going to keep both of these applications on separate paths. And you can imagine that we're really switching one to the other. So if I go to my browser really quick, not that one, this one, this is just very briefly, this is our Eureka registration. I can scroll down here, and it'll tell me actually all of the different applications that are currently running and registered with Eureka. So the first one here is our config server and the gateway. Below that is our web API, our blue application, and our green application. So this page is our very basic sort of routing through Zool. This is our web API. There's no canary or blue-green deployment on this. It's just basic, straight-up routing. Very simple to do with Zool. And then here's our blue application. And we've decided to stage our green application. It's already running. I can actually change this route to our staged. But for this case, I'm just going to go ahead and switch the configuration value and do our blue-green deployment. So I'll change this from app to staged or retired, rather. And I'll change the staged application to the app. The nice thing about config server is that it all lives in a git repository. So you can actually see the history of what you've actually changed. So I'll go ahead and make a commit for this. Now the way to get Zool to refresh, rather than having to take down the application and redeploy it, we're actually just going to call a refresh endpoint on it with a basic post call. And what this actually does is it's going to cause Zool to dump all of its configuration values and request new ones from the config server. And this gibberish that it returns is actually all of the values that actually changed. So we can go back to our application and hit refresh. And now we've got a green application. And it would be just as simple to go back to blue, or if we really wanted to, we can add extra routes to Zool while it's running. And that's it for the blue-green demo. I will hand it back to Jago. All right. Thanks, Grant. What did you guys think? That was a nice and simple and elegant demo there. The nice thing about it is that you didn't really see a lot of code. Everything was out of the box for you from Zool to be able to do blue-green deployments. However, I sense a little problem there. It seemed like whenever we did a switch from blue to green or green to blue, it seemed like you're going all in with blue or green, whichever instance that was. And if you do that, you might have some problems with that other deployment. And now it's very easy to switch back to blue and green, but at that point you might have already lost that battle. And so would you agree, Grant, that's kind of problematic there? Yeah. All right. Now, it kind of reminds me of this meme that I saw a long time ago. I'm going to go and read it out to you. I don't always test my code, but when I do, I do it on production. Have you guys seen this before? Now, what if we could morph this meme into something that looks a little bit better and malleable? I don't always test my code, but when I do, I do it 5% at a time in production. This is essentially what a Canary release would look like. You're limiting that blast radius in case something blows up. All right? And so, Grant, you want to take it away and show us a Canary release? Sure, thanks. So Canary releases are pretty straightforward. We're going to be using kind of the same setup as last time. Very briefly, what we're doing here is we're going to have both of our applications staged, but instead of having 100% of traffic going to one, we're actually going to be able to allow 90% of traffic to go to one instance and 10% to the other. And of course, we can change these values because they're all just configuration values. As far as the components go, it's pretty much the same setup as last time. There's a Zool server, which will actually be handling the Canary release. And then we're going to have our Eureka instance doing service discovery again, another config server. And then we're going to have two web APIs running in .NET Core that are serving basic JSON. So I'll go ahead and go to that editor. I'll start with kind of going over the files in this guy. So again, we have our Zool server. It's a basic Java Maven application. You can find these pretty much everywhere on GitHub. Same with the web API. It's a .NET Core application. And then the other version of it, web API2, basically the same copy. And then our configuration values here as well. As far as the configuration goes, it's pretty much the same as the last demo. Although since we're doing a Canary deployment, there's some different stuff down at the bottom, but we'll get to that in a minute. These first two paths are mostly just for direct access to the applications. Again, for the purpose of the demo, we're including these, but in production, you may not actually need a direct path to your Canary instances. And then down here, lines 12 through 18, these are the actual lines that control our Canary deployment. These first two values are Zool values, and then everything from Canary and below are custom values that we've written into Zool to handle Canary deployments. This first line, Canary, is really just telling Zool that we actually want to Canary these values. And then this destination list lists out all of the different destinations that we want. What's important to note here is that these ID values actually correlate to the Eureka registration. Unlike before where we're using this root object to correlate, we're using these IDs in this case. The split value is determining how much percentage of traffic we're routing to that given ID. In this case, we're routing 10% to our web API green, and the remaining 90% is going to go to the blue application. And what's nice about the way we've done this is that you can actually have as many Canary instances on any given route that you'd prefer, whether or not it's practical to have 10 or 15, who's to say. As far as Zool goes, I'll kind of quickly run through how Zool handles more advanced logic. Zool uses filters to handle how we want to actually route traffic. And filters have four methods that you absolutely have to have. This first one, the filter order is going to control the order in which the filter is run in a set of filters, because Zool doesn't just have one filter, it has several. The smaller the number, the higher the precedence. Our filter type is a string value there. By default, our pre-filters, route filters, post-filters, and air filters. Each of those types are really just a way to help kind of organize how we think about the filters. And they really group a set of filters along the request handling that Zool will do. This should filter method is here to tell Zool whether or not to actually run this filter, which means we can turn off specific filters on a specific route for a certain set of circumstances if you really needed to. And then our run method here is really the brains of the filter. This is what the filter does that actually adjusts how the route will work. And in there, we're calling this get best destination, which is really the canary deployment itself. And without really going into detail about what it's trying to do, we have that list of destinations from our configuration. Each destination has some percentage registered to it. And what the method here is trying to do is it's going to calculate the current percentage that that destination has handled and figure out if that's less than the target percentage. And if it is, it'll select that destination. Otherwise, it'll go on to the next destination in the list and do the same thing over and over until it has selected some destination. And then as I mentioned, these are all running in .NET Core. So this is the startup for one of our web APIs. They are using SealTow to register themselves with our Discovery service. So this is kind of the setup for that. If I jump over to our browser here, I'll run one request. So this is our green instance. And if I go over, we have a stats page that'll help kind of keep track of where our values are. So we've got one request on the green instance, and our ratio is 100%. Now, if I go back to the green instance here, and I refresh again, it switched to the blue one, because the blue had the lowest percentage compared to its target value. And I can go ahead and hit the stats page again, and now it's 50-50. And if I refresh this 10 times, we're on 2, 3, 4, 5, 6, 7, 8, 9, 10, we switched over to the green. I think I had 11 there, actually. If I go to my stats page, we're at 11 requests and roughly 90% to 1 and 10% to the other. These values will actually get closer to their target values. The more requests that it actually handles, but at these low numbers, it's going to be kind of rough. And the nice part about the way we set this up is that we can go ahead and change this percentage value at runtime again. So we'll go ahead and do a get commit here. And again, we'll do another refresh on our Zool instance. And then when we go to this guy, we're just going to hit it a bunch of times. It'll see that it'll stick with green for a while until the values eventually hit 50%, and then it's just going to flip back and forth between the two. And we can go to the stats page and verify that it is actually at 50%. So that is essentially what a Canary deployment or Canary release would look like. I'll hand it back to Jago. Thanks, Grant. What you guys think? Yeah? I mean, it's still not a whole lot of code. We wrote one filter and some data structures to support what we were trying to do. But almost out of the box, you can support Canary releases with Zool. One thing to keep note is that every time Grant did visit one of those browser URLs there, that he stuck with the port number for Zool. So everything was routing through Zool in that case. Now, the next burning question in your mind might be, well, how do I get started? Well, first and foremost, you should go to the GitHub page for Netflix and go read about Zool. They have plenty of information there and probably some other links that lead to other pages, like medium posts. But if you want to get started right now, I would also recommend going to GitHub. But in this case, go to Spring Cloud Samples, go clone Eureka, config server, and Zool server. I promise you, you'll probably get started within 15 minutes with this. And with that, I'd like to thank you guys for sticking around. It looks like our Canary has killed over and died. I think it might be time to evacuate. But before that, any questions? Thank you.