 Thank you guys for coming. My name is Frank Greco. I am a cloud native engineer at Northwestern Mutual in Milwaukee, Wisconsin If you haven't done so you can check out our booth. We're not a vendor, but we are showing off how we do kubernetes at scale Talk about canali and maybe attract some talent along the way So Northwestern Mutual Built a nice tower opened earlier this year. All right. I do I do 27th floor 32 so it gets the top floors So I'm here to talk about canali, which is an open-source API gateway Kubernetes native cloud native as the plan is to Blow through a bunch of slides do a demo and then leave plenty of time for questions So I'm sure you guys will have plenty and So the first thing I want to talk about is API is why API is why an API gateway, etc Brush up on some of the terms if you haven't dealt with them a while or introduce them if you haven't So we'll start with a simple example. So let's say if a Google home and you want to talk to a nest What you'll do is you'll actually go through Google home will call the nest services and Then the nest services will talk to the nest for you But what's going to happen is it's going to go through a little piece of software called an API gateway And that's going to make sure that You're not maybe calling the API Make sure, you know, it rate limits it maybe does some transformation You get all these things that come with what's called API management So things like API API key authorization into JDBT authorization You know rate limiting transformation monitoring routing tracing, etc And it's this piece of software It's one piece of software it's a gateway and It's the entry point to all of your apis and then traffic goes from the gateway to your back end services So some common examples of those in the industry Might be apogee the azure api gateway mule soft kong the aws api gateway, so If pictures help you associate vendors to Solutions think of these when you think of an api gateway And this is what canali attempts to solve and we'll talk about it more in a little bit So that's what an api gateway is So now i'm going to take you through a little bit of history on the api management at northwestern mutual and how we've Evolved from where we were two years ago on kubernetes.9 to where we are today with canali in production So here we have two vpcs. This is the setup that we have for our Northwestern mutual website So here we'll have a vpc on the left and a vpc on the right and we'll pair them and we'll have our We'll have our micro apps or the apps that are running our ui In one vpc and then we'll have our kubernetes cluster in another And so here we'll have our cluster and the way we started out is we gave every kubernetes service an elb and we exposed it via a node port and then This blob of squares represents the vendor that we were using and all traffic from The left vpc would go through our api management solution And it is outside of the context kubernetes. It's a separate vendor And then traffic would get routed to the appropriate elb And then the elb would route traffic to the service The problem was it got pricey over time As we scaled the number of services each service needed its own elb And so we needed a better way And so we did what's lots of people in the community did is we had an ingress controller And then we threw one elb on top of it and then service would go from the api management vendor solution through the elb onto the ingress controller and then based on context route and some other Some other items traffic would get would land on whatever kubernetes service you wanted. And so then you got rid of the price now This may look all all well and good But what it's not showing you is the mess that may or may not be like the vendor solution So what happens is when you have a cloud native tech stack You want the tools that service your application to be as cloud native as possible And so if you're using a vendor as an api gateway, that's outside of the context of kubernetes You lose a lot of insight that you can get from having something running Inside of the context of kubernetes and then because it's maybe not a cloud native solution It's different and hard to automate. You may need like two or three different types of databases to store the information And it's hard to integrate into your Into your pipeline In your cicd model because now you may be forced to configure stuff through the ui when you want to do them as code And so there's a big mess Of stuff and it didn't really fit well with the cloud native approach that we wanted to take with our environments and so we thought that we could create a an open source solution for this and If we were going to create an open source solution to solve this problem We had some design goals some very high level design goals So when I had to be innovative so we needed to be solving a We need to be solving a problem not only in the context of northwestern mutual But also solve a problem in the context of larger open source community so that we can contribute it back and that others could benefit from it I needed to be lightweight so it needed to be just minimal resources Minimal you know amount of pods if it's in kubernetes running and it's just stupid simple to deploy Very easy barrier to entry Performance centric so this probably goes without saying it's a middle work component. So it needs to be very performant Robust api key management at the heart of any you know api Management tool. This is a big piece of it. So we needed to have a robust kubernetes native way to handle it You didn't have an extensible plugin framework. So Again, you'll also see this with api management vendor solutions It's maybe not a good idea a hard code Every single possible thing that you might want to do into an api gateway But create a way where other people can add their own Functionality, so if they want to do their own complex transformation for their business use case For example, they can do so I need to be kubernetes native so kubernetes is all about its metadata and When you have your gateway in the context of kubernetes you can do some pretty cool stuff with canary routing and also Things so we'll look at in a second, but you know custom resource definitions. So you're able to create kubernetes native specifications and you get things like storage for free diverse tooling So the gateway is just one piece of it, but you can't have a full api management solution without tools for tracing analytics and monitoring Automatic api key generation, etc So it needed to have this whole ecosystem of tooling around it And needed to be easy to automate and this is one thing that we did not get with our vendor solution that we were using before And by easy to automate it not just that it needed to be easy to automate It needed to fall in line with the same patterns that we were using for our existing cloud native automation so That spawned canali Logo is very new and there's Stickers at the sticker table in our booth. So feel free to pick some up. We have a lot of them Um So what is canali? Canali is a lightweight kubernetes native api gateway That together with robust network policies or together with network policies provide a robust open source solution To kubernetes ingress api management and api security So I'll talk a little bit break this down a little bit as we go through the next couple of slides But first I want to go back to the picture that we looked at before and look at how with canali this picture changed so basically Traffic would all go through the elb and now we'd be running our api management solution Which would also do our ingress for us and traffic would land on one of the pods and It would get routed to the service. So the complexity reduced tremendously Resulting in a lot less infrastructure needed and Then yeah, so So now now let's look at Some of the differences between a gateway and a service mesh. Um, so a lot of you have probably wanted to talks about service mesh talks. So istio envoy liquor d etc So we have canali at the top and then we have our three um application pods on the bottom And this is what we want to happen. We want everything to go through canali or api management layer and traffic to land on a pod the problem is is that this is also possible by default and This is not desirable because service service on the left could talk to service on the right and Not get managed. Um, not get verified, you know with the api key, etc But kubernetes provides a native construct for us to fix this net network policies So we can throw these lines in here and say hey only traffic is allowed to come from canali And so it'll make this um It'll it'll make this this traffic not possible. Um, and so with this we're able to take We're able to completely get around a service mesh and make our clusters a lot more lightweight and just have the gateway model all right, so I mentioned canali is lightweight and when you think of an api management solution Um rate limiting and quota come into there. Um, some people think as quota is a subset of rate limiting But just take that and or take that as a whoever you think about it But we needed a lightweight way to basically enforce this across multiple pods so If you're using a vendor as an api management solution or basically just like, you know, this monolith application It can You know check its database get an information back then proxy the request But one thing canali boasts is zero network overhead on any request So when you get an incoming request the only network traffic that happens is the actual proxying to the back end application and back And so basically canali implements a eventually consistent method to do base to do Rate limiting and so what'll happen is we'll have an incoming request here on the right It'll hit one of the canali pods that pod will talk to at cd over g rpc and then Via g rpc it will let the other canali pods know that has Had a traffic point and then it's able to update it sufficient in memory store and We have eventually consistent Rate limiting So this ecd is different than the actual kubernetes ecd or at least that's the recommended approach So canali is open source open source. It's it's a it's a very very new project. It's like two months Two months old. We've gotten some good traction on it so far And it will continue to grow But now let's dive into How canali works from like a user side of it? And this gets into more of the custom resource definition side Canali's a couple weeks away from a 2.0 release and so the specs will be 2.0 specs That we'll be looking at so Just to give a brief about Custom resource definitions Not going to spend a lot of time on this but as you probably know Keynote mentioned it this morning, but kubernetes is a very api driven Application and there's three parts to every api. There's a group name a version name and a resource name And so like if you look at the changelog, you know things might be moving from One version of the next A resource may be graduating to a different version Or they may be moved around to different groups. I think 1.8 did some of that And canali does the same thing So canali has a group name canali.io Versions and then resource names and it has four major resources which we'll look at This is basically how it extends the extends the api So the major spec is the api proxy spec And these three examples provide examples of a lot of functionality not all the functionality But the main thing to note is the different type of backends that you can do so Canali allows you to proxy to services Inside the cluster allows you to proxy to services outside the cluster and allows you to proxy to mock targets Which define mock responses that can be returned And this is key for Or the most interesting here is the middle one is the external endpoint. So it's a large organization We'll have a lot of our services running inside of kubernetes But what about all the services that are running maybe on legacy applications? You're still going to need some api gateway solution to do those, right? But with If you think about it, you can still proxy traffic to legacy applications But still have part of that as a cloud native So you can take your api management piece and make that cloud native And it has a lot of benefits because you get things like storage for free You get cloud native configuration or kubernetes native configuration You get a nice pipeline around permissions as code, which we'll look at in a second But that's something that That is cool and will be new in 2.0. But the other thing you'll see here is And we'll talk about plugins some more in a little bit But you'll notice a plugin section and this says what plugins will get executed in every response or on every request And one of the big things for us is we needed a way to have version controlled plugins and that was something that Was a foreign concept to different products that we looked at Um So here we're saying that we're going to use an api key plugin on every request We're going to use this version of this plugin. Um, and then we're going to pass in some arbitrary configuration To this plugin that the plugin will need to do its logic Um, and it's a list of plugins. You can create plugins. You can add them to your proxy And it will run in a synchronous manner So this is the api proxy spec We'll see a demo of this in a little bit. Uh, but next is the api key binding spec So canali follows of our back, um, like approach So, um, we have api keys which are abstract from, um, which are abstracted from proxies And then you have api proxies and then you need something that basically binds a set of keys to a proxiness So the api proxy spec does and it basically says here's a list of keys that have access To this binding In this example we're giving bob's api key access We're saying that whoever uses this binding bob's api key is able to call this proxy a hundred times a second And by default they have access to everything let them let them call everything Except if you come on the slash balance sub bath, then we're only going to give you, um, permission to use the Uh, to use the git hp method. Um, this is just a simple example of how you can configure Um, how you can configure permissions But notice that it's completely abstract from the From the api proxy Another thing that we're doing in v2 here on the right with the api key spec is we needed a way to enforce or not enforce but, um, To allow for non-breaking api key rotation And so we follow we bounce back and forth a few ideas, um, on github and ultimately we, um, landed on this one But um, say you have an active api key and you need to rotate that You'll basically create another version of or vision of that api key and make it active And then you'll keep the old one active until teams are able to update their projects at which point you'll make it inactive And then eventually remove it from the spec. Um, but this allows, um, non-breaking api key rotation In the api key spec Then there's a mock target spec which i'm not going to spend a lot of time on but if you do use the mock target back end option Just way to configure a mock response Um Canali's a go program. Um, so this just shows how you can configure go If any of you are go developers that uses cobra and viper So standard configuration you can do off of that So plugins so canali uses the go pack or the the go plugins package for plugins And the only requirements is that it implements the plugin interface and you can do whatever you want within that Within these functions that'll get called in the life cycle request and it will execute You can version these so for example the api key plugin is stored in the separate repository That is dynamically brought in when canali starts and And the logic executes and then we'll bring in the version that we want And so if someone else wanted to design their own plugin, they would create a project implement the plugin interface They would change the configuration And then dynamically load that plugin at runtime and then or yeah at runtime and then It would execute their logic. So decoupled extensible plugin framework Where they can do whatever they want There's a oman template which which does some of it. So tooling and we'll see more of this in the demo But the first tooling is you know, we need a way to automatically create api keys And so we do that with canali cuddle Which is a cli tool for canali And we use this extensively at Northwestern Mutual and developers seem to like it Seems to make their lives easier But it's just a binary that we distribute and allows them to create api keys And then they're able to add it to the cluster and Because api keys mean nothing until they're actually bound We're able to let developers create the api keys and then go to the service team and say hey Here's my api key add it to your add it to your service And that model's worked for us. But it's also permissions as code too And we'll see a demo of this next we use grafana and influx db for For analytics and monitoring And grafana is completely configurable the default dashboard comes with like response time total requests error rate stuff like that And then we use open tracing Yeager comes out of the box with it, but the main part is open tracing So it works right out of the box with open tracing. So it'll play nicely in that in that life cycle And then it's Yeagers the default client for that All right, so let's do a demo That's how am I doing on time doing all right All right, so I have I've went to the canali github page and I've executed The quick start steps here. So I've started minicube I have so I've flown the project. I've started minicube I've spun up canali as well as yeager and influx and grafana, etc And I basically have this Just just to make sure it's working here. I have So I have a deployment that just Is that big enough for everyone? So I have a deployment just a simple service just returns Just returns a simple message when you hit it And then I have an extremely simple version of an api proxy now. This is a v1 spec It's very similar some subtle differences, but basically i'm coming in on slash kubkan And that basically with this configuration the only thing it's doing is ingress for me So if I go here to my cluster And hopefully everything's still running awesome Fidu kubkan will apply i'm just going to apply this And then try to curl the service I'll get a response I'm just using Post will be used for a future example, but I have my ingress working So now let's go ahead and basically add some api key management to this So i'm going to go and create an api key So i'm going to use canali cuddle And i'm going to Create an api key. I already have this command saved somewhere All right, so i'm going to create an api key. I'm going to call it fringes api key I'm going to give it the name or i'm going to give it the location of the public key And i'm going to save it to This file All right, so here's my api key that i'm going to use the actual api key spec has been saved here So I can open this up in my editor And there's the api key that it created for me I can go ahead and add this to the cluster All right, so now I've went ahead and added this api key to my cluster now I can go back to my specification And I can basically say So now that I have my key I'm going to go ahead and enable the api key plugin on this proxy And i'm going to create a binding which basically is going to bind frank's api key To this proxy I'm by default. I'm going to let it call everything over the git method But if you come in on the foo sub path, um, you can only You can only call it via the post method so basically It works by finding the highest priority rule. Um, so if there's no That's if the incoming request isn't overwritten by a sub path Then it's going to fall back to whatever the default rule is so i'm going to go ahead and apply this All right, so that's applied I'm going to go ahead and try to Try to curl This service and it's going to say that api key is not found a request. This is enforced by the actual Um by the api key plugin, so i'm going to go back to where I configured the key which is right here Then i'm going to add a header And i'm going to call it api key and then i'm going to set it equal to the value And then it's going to give me my My response now if I change this And let's say so i was using the post method if I use this as the git method We'll see that api key isn't authorized and why so if you see my sub path It's slash coupons slash coup con slash foo bar now slash coup con Is the incoming path, but we're overriding the target to slash Now here we're saying anything that starts with slash foo We're only going to give post access to and the incoming request matched this sub path So it knows that it only has access to post and not git and so that's why That's why the traffic or the api key was not authorized So now we're going to we're going to We're going to curl this service a whole bunch of times And it doesn't matter if it's It doesn't matter for errors out. I just want to show Well, actually I want to show yeager. So Maybe it does api key equals this And then we'll set the method equal to posts So we'll have this goal a couple times now as this is going we'll go over to yeager And just by default out of the box. Um, you'll you'll you'll get your tracing with it. Um, so, um, if you've went to any Tracing talks, um specifically around yeager, they're great talks But here we see the overall span. We see the actual plugin on request Life cycle hook happening and we see that frank's api key is the api key being used Because the back end service also instruments open tracing. We can see the actual request hitting the back end Which is this right here Etc. So the talk this talk is not about open tracing, but it does come with open tracing out of the box And if I look at my metrics in grafana, um, I can see some metrics start to come across Total requests average proxy response time, etc response time percentiles, so um So yeah, it's a very brief demo So future work. Um, so v2.o is like One or two weeks away, but everything's tracked by the normal The normal open source flow everything's tracked by issues. Um, et cetera, so Give the demo So there is a tutorial at tutorial dot kanali.io This is where the documentation lives now It'll probably be ported Ported to something else in the future. But for right now, this is where it lives All right, I think I have five minutes left. Um, so I'm going to ask the first question. So what about istio? Um, so earlier, um, I hope I kind of justify this a little bit, but um, kanali is not a service mesh It is an api gateway And so because it's not a service mesh, um, it does separate itself apart. Um, but then also, um It's not an api management tool kanali's What are the only ones that I know of outside of kong that is actually An actual api management tool? Um Istio does have plans in the future to add api management to their product Or to the project And when that's done, um, we will see what comes about it But I know it's more for vendors to be able to hook in their specific api management to it To fall in line with that life cycle. Um, so kanali is an api gateway not a service mesh It's a very lightweight one and everything is fully included. So you get the api key management You get the api proxy you get the binding Everything is out of the box you get storage for free because it's kubernetes because the custom resource definition and it When when you start to think about istio is versus what kanali is The differences are very different so So yeah, so at that let me answer whatever questions you have and we'll start. Yes Um, correct. So it's so it's not a server smash and lightweight meaning It's it's just one pod. It is the kanali pod and that's it Um, so htp is the only protocol that it supports right now. Um, there's Future work to add other protocols But we just wanted to get it out the door and currently it only supports htp or htp um You might support htp2 as well because I believe I default the native go htp library supports htp2 I don't I don't believe so now We have it in the back there support what? Oh, so, um, there is no official jdbt plugin yet There will be that one that comes but that's the thing with the extensible framework Is the frameworks there and you can build whatever you want. So they're like The jdbt would be a plugin that would that would live in its own repo that would be included in So the beauty is the beauty is with open tracing is you don't It doesn't matter how many pieces are in the puzzle as long as you play nice and handle Handle the context as it comes in and then pass the context along when you're done with it Whatever downstream may If if someone else wants to do something with it, they can But canali plays nice and that flow will it it will accept and and use and then it will also You know start if not or and pass along so Yes So I say one pod, but it's one like deployment. So it's it's one so um in our deployment we run it as a daemon set But you could run it as a deployment as well But it's it's like one Like there's only one container Correct correct, but it's it's it's a it's a replicated pod In the back Yep, it's correct. So api so Canali at its best is an api management solution, but api management or the api gateway is also a proxy The thing that I didn't show is canali does do It does canary routing as well and dynamic back end service routing So you do get some of that as well Yep, so so we've been running canali in production for a few months now We I think there's been times during our batch job. We'll be running like 300 tps And it's I don't I don't know if we've hit it with a lot of load past 300 tps, but It's it's been running in production at times at 300 tps. So So it's kubernetes native so When you're inside of the context to kubernetes, you can do very powerful things But then also by using custom resource definitions You get storage for free the way you configure it is kubernetes native Everything about it is like it's like that that kubernetes native cloud native format And so like like with the aws api gateway, like that's abstracted from kubernetes And so you lose the context But then you also have to worry about how all that's managed behind the scenes And then if it's and then like in a cloud native context to be portable You'd you'd have to revisit your your solution So so we're running v1 in production right now v2 will introduce the scd grpc And and so we don't have v2 in production here But yes, it will be another one the reason why it's not recommended to use the same scd as kubernetes Is because if you have access to the scd that kubernetes uses you have root access to the cluster And so it's it's always recommend. I mean even with other solutions that use separate scds like calico they recommend separate scd So there's So kanali is greek for canal and so like all of these projects Canally open or kubernetes open source projects all fall like the nautical theme and they find a word and they turn into greek So kanali is greek for canal and it's kind of like that canal into the cluster but correct so there's probably Opportunity to also have them right in c and then port them to go and whatnot, but yeah And the way that'll work is they'll be in v2 They'll be like a requirements like diamo file or whatnot where you're saying like Here's the plug-in. Here's the version Kanali you use like in a nick container to load them in and start the kanali pod and you'll be good to make the wet Yeah, so uh, right now it uses influx out of the box But there's no reason why it couldn't use like prometheus or or something else right now. It's not pluggable But it yeah, it it should it should be in the future in terms of Yes, so cpu. So um, it outload. We'll see each instance of a kanali pod use like half a cpu and Probably like a gig of memory it's Where every every version it gets tweaked as we find different ways to improve on it But that's that's that's that's the v1 production numbers yes, so um So kanali does do ssl termination if I look I believe it was in the spec You were able to specify an ssl object And that basically you give it the secret name and it's a type kubernetes.io'd slash tls It enforces the cert in the key and if ca dot um cert is present. It'll be bidirectional ssl So believe we're out of time. Thank you guys very much. Um, I can answer questions after