 All right. Hello, everyone, we're going to be talking about creating a scalable foundation for Kubernetes multi-tenancy My name is Laconde Mouille or you can call me Luke and I'm a senior developer advocate for Kubernetes at AWS My name is Calvin Parotti. I'm a senior depth engineer at Heald Street Great. So we're gonna skip the formality of an agenda because this is a lightning talk But also where is the fun in knowing what's coming and as for that 404 error? That's my dad joke for the day. So I won't take offense if you don't laugh, but a smirk wouldn't hurt. Looks like I got one. Thanks All right, I want to start off by talking about why we care about this topic And so for starters we've got firsthand experience in seeing the challenges around Kubernetes multi-tenancy and We're aware that this may look different depending on the use case or the context Which means that some some of the solutions or strategies around how you solve the challenges around multi-tenancy may look different But we're of the mindset that there's some fundamental principles or universal concepts that can be applied Regardless of the use case and that will allow you to essentially build a scalable solution around your multi-tenancy challenges And that's what we're going to be elaborating on the second reason is that we find that multi-tenancy is a very common model Whether companies are using a single host cluster for several unrelated workloads or in some situations They'll go with a very strict or isolation approach having multiple Kubernetes clusters But whatever the case companies and teams are trying to find out the right way to apply security segregation of their workloads and essentially just structure things in such a way that they can seamlessly build and Easily onboard additional workloads or cluster personas as time goes on So this image over here just kind of depicts the idea of trying to find the best way to slice and dice your Kubernetes cluster Well, let's talk a bit more about some of the common best practices And these are fairly well known things like namespaces to have a general way of organizing of organizing your workloads In addition to that you got network policies Which gives you a stricter approach to managing the network traffic specifically ingress and egress traffic for your workloads Then you've got resource management to try and prevent any resource hogging for your workloads And of course there's role-based access control to manage the permissions not only for workloads But also for the different cluster personas And we're both of the mindset that each of these best practices essentially points to the main fundamentals that this whole talk is kind of aiming to drive home, which is Organization and isolation. These are the core concepts that we believe can help you scale when it comes to having an optimal multi-tenancy strategy and Something that we were actually discussing in preparation for this talk was the fact that we both believe that your multi-tenancy strategy is something that should actually start on a whiteboard and You can then have all the different parties of interests the different projects or teams that we'll be working on that will essentially have their workloads running on your Kubernetes cluster and based on what you strategize in that session you then Translate that to your Kubernetes cluster. We believe this is the right way to essentially create a blueprint for the future and This is something that we got to apply to some degree on a project that we worked on years ago Together and we were essentially both functioning as DevOps leads in a banking context on a particular project That was actually going to be the blueprint for several Kubernetes projects for that particular bank and so this is something that we had to give a lot of thought to and so there were multiple Eks clusters and one cluster was dedicated to running Argos CD, which would essentially manage the continuous delivery to different workloads and when it came to multi-tenancy Argos CD helped us in a big way because the concept of projects and applications essentially allowed us to Carefully organize the different environments that we had for our different applications and of course grouping that into a project And this allowed us to be able to then easily scale for the future workloads that were going to be on boarded to those respective Clusters that we were running in addition to that as time went on it wasn't just a case of onboarding additional micro services There were also More cluster personas different people with different functions whether it was solution architects or QA testers And we needed to find the best way of giving them access to the cluster And so Argos CD's concepts around projects applications and RBAC really complemented that But as most of you probably know if you've followed any kind of multi-tenancy strategy Multi-tenancy doesn't just go as far as your get-up strategy There's also the life cycle management of your Kubernetes clusters and so we want to also talk about how you can complement what Argo already has to offer and Going back to what I mentioned earlier about these two main concepts or two main approaches rather that companies typically take with multi-tenancy Whether it's a single host cluster or several separate Kubernetes clusters We want to touch on two solutions that could potentially help in this case One of them is rancher as you can see over there and the reason we're mentioning rancher is because rancher also has this concept of Projects, which is also meant to be a solution around multi-tenancy. It's a logical in the concept in the context of rancher It's logical grouping of your namespaces and you can also apply RBAC to those projects So you have the same idea of projects both in Argo and in rancher and it shows you how they complement each other and kind of point Back to organization and isolation But we are also aware of the fact that in other scenarios companies would rather opt to have a single host cluster And so for that you can make use of something like the clusters Which will still give you the concept of separate Kubernetes clusters But in a virtual context and you can run all of those on a single host cluster and still make use of Argo CD to essentially Deploy to these different downstream clusters and that's what Calvin is now going to demonstrate Sure. Thanks Luke. So yeah, when you think of multi-tenancy, there's kind of like two main ideas out there So multi-tenancy through namespaces or multi clusters the namespace Multi-tenancy has the problem of if your developers are using CRDs and they're building against CRDs those are cluster wide resources So you can't really use namespaces to divide then you get the multi cluster scenario Which is quite expensive quite big to maintain and to bring up a new cluster. It's quite difficult. So With the with V cluster and Argo CD We can use Argo CD to bootstrap virtual clusters and the developer gets the experience of he has his own cluster Argo CD can connect to that cluster and Run so this is the repo that has a little demo of what we can go through but essentially as the admin We have multiple applications which are essentially clusters. So we have if I just quickly scroll down to projects We have a project for classes These are actual virtual V clusters that are deployed via Argo CD and once they deployed They are essentially cumulus clusters on the main host cluster and they are configured if you had to go to Argo CD They are configured as clusters at the cluster a dot v cluster namespace So essentially now Argo CD can deploy to these clusters and now we can use this multi-tenants foundation to scale as many Classes we want and then each team has Pacific our back rules that allow them only to deploy to their Pacific clusters So in this example, yeah, we have this is project a they are back They can only see app a that's deployed to their cluster a service if we go to projects They can only see their project project a they go to the clusters. They can only see their cluster a so they are In the mindset of they have their own virtual cluster So for full isolation and the same would be for project B. They have access to their App in cluster B. Their destination is cluster B projects like this and in order to do this Yes, the is a quick. This is a real public repo that Deploy this entire solution. So if you could want to clone it you can go for it A quick thing with the V clusters every time and we're using GitOps to bring up these virtual clusters and every time We bring up a new cluster We obviously need to deploy initial manifest onto this cluster to give August CD access to this cluster So every time we created to use a new cluster We create a new service account and we just a bond that service account to August to a service account called August CD And August CD can use that token to access that cluster and then start deploying it to it as It was any other cluster In this example, I've used the make file to demonstrate the steps of how to go through each creating the projects deploy clusters But in a in a in a non-production environment you could use workflows to keep bootstrapping your clusters and scale infinitely and for in this example, yeah to get when we deploy classes, we need to get that token of the From the V cluster to tell all go yes the token to connect to and we just have a v-class to come on to connect to it But you can use all go workflows to do the same logic and in this way we can achieve a Multi-tent environment using August CD And the nice thing is if we have app sets using the cluster generator if we have a base foundation of Applications we just have an app set there and every time a new cluster gets deployed onto August CD it will automatically deploy all the necessary applications to this new cluster and that's easy way to scale Using app sets and August CD Yeah, that's it Thanks