 Okay Good afternoon everyone. Thanks for staying here so late on the final day of the conference My name is Bick Lee and today I'd like to tell you about Arlon an open-source tool that I hope I'll convince will make your lives significantly simpler especially if you work in DevOps or platform engineering and you manage many Kubernetes deployments and also the applications that run on them So the key part of this talk I hope is going to be an interactive demo. That's the I think the the fun part of the this talk and We're gonna Finalize this with some conclusions, and if there's time at the end I can hang out for any questions that you might have So before we go any further a brief word about myself I I Spent my whole career working on containers and virtualization management. I currently am a full-time maintainer on the Arlon project and prior to that I worked at platform nine as a co-founder and chief architect and then where We built open stack and Kubernetes management products and prior to that I spent many many years at VMware doing virtualization So let's talk about the problem today So as many of you know Kubernetes has become the de facto platform for running modern applications But as Kubernetes has become more pervasive You see it running in more and more diverse environments such as Edge devices right and this leads to a multiplication of the number of clusters and Applications that you need to manage and those can reach, you know the hundreds or even thousands and It's really difficult to manage all of those deployments using traditional methods and so what you really need is strong central management and one of the challenges that many people face is the Toolset and the apis for managing Kubernetes clusters and the content that runs on the clusters is Disjoint and inconsistent so for example many users Deploy or they use a tool like terraform or Eks cuddle to first provision a cluster But then they have to wait for the cluster to come up and that could take 10 15 20 minutes And then they need to grab a cube config for that cluster and stick it somewhere else For another tool to consume in order to finally start deploying applications Right, so there's a lot of sequential dependency here and weight loops and an inconsistent API across This problem Gets even worse when you talk about configuration sprawl when you have so many objects that you need to manage in settings and configurations you want to for example share a lot of settings and Distribute them across many deployments, but once in a while you'll have a Cluster here or an application there that needs a slightly different setting, right? And that's hard to manage and what many people will do is just copy and paste some yamls and make some Targeted changes, but this really breaks when you talk about updates, right? so change management is also a big problem where you want to be able to Make a small number of changes, but then propagate them to a large number of deployments while still maintaining those Individual specific customizations, this is a hard problem So that is why we built our lawn to try to alleviate some of those issues So let me walk you through some of the building principles of this tool So the idea here is Our lawn unifies the API and the object model for clusters and content running in clusters such as you know our back rules Covernal policies and applications, of course So in our lawn clusters and apps are first-class Objects and they're fully declarative style Resources that run within a Kubernetes API framework what this means is Using a single cube Cuddle apply command you can deploy many many clusters and the content that will run on those clusters All in one shot, right? So this is fire and forget. It's fully declarative You specify it once you specify the desired state and our lawn with its underlying tool chain will take care of the rest that's really a powerful idea and Our lawn embraces get-ups internally It uses a get-ups deployment engine to deploy manifests But since its API and its resources themselves are fully declarative Kubernetes objects You can put those objects in git and rely on the tool like flux and Argo CD to manage them In order to address the sprawl problem for configuration a Key concept in our lawn is profiles everything yet that you do involves profiles, which are really a way to group related sets of settings and Applications into reusable units. So for example, we have cluster profiles Which we also call cluster templates and you can clone many workload clusters from a single template And we have app profiles that let you group a bunch of related applications together And you can also make custom Changes for one or more deployments and those customizations will survive Updates so our lawn has a powerful mechanism that we call linked updates and that means you can make a set of changes to a Template or a profile and they will all Get sent to the deployments that use those profiles. Okay, and we're gonna take a look at some examples In order to implement all of this feature set Arlon Doesn't try to reinvent the wheel it rests on the shoulders of existing giants that are Well known in the community and are very good at what they do So Arlon is built on top of a stack of tools for the deployment of manifests Arlon uses Argo CD and for expressing and orchestrating cluster provisioning Arlon lets you choose between Either using cluster API manifests or cross-plane and Arlon also uses other tools that are well known such as customize and helm So how do you install Arlon? so the There's lots of instructions that you can find in the repo basically You need a Kubernetes cluster that's dedicated and serves as a management cluster And it's the same cluster that you typically use to host the other tools that Arlon depends on for example Argo CD and cluster API or cross-plane Now we offer Command line tools that allow you to Automate the setup of this management cluster and if you do that this is something that you'll end up with Based on this picture So on the left side is your management cluster and it's kind of busy But really what you need to understand here is It hosts all of your APIs and controllers and Arlon is going to add its own custom resource definitions and its own controllers You also need to get repo because we're going to use get ops operations heavily so all of your templates application sources profiles are going to be defined in Get and once all of this is set up Arlon will help you Deploy many many workload clusters which are shown on the right side So let's talk about some of the core principles in The Arlon object model first of all you have applications right since we're using get ops principles In general your application sources will live in get or a helm repository like this And to make those known to Arlon you will create Arlon apps which are just Pointers to those locations right so in this example we have guestbook and nginx which point to Directories and get and we also have Redis which is pointing to a location in a helm repository Once you do this you can organize your apps by grouping them into profiles so for example We can put guestbook in the blue profile and the other two in the red profile and the way you use profiles is That's how you're going to specify what applications to deploy to clusters. Okay, and I'm going to talk about that next so clusters in Arlon workload clusters are cloned from Something that is shared and that you store and get we call this the template or the cluster template and In this example We're going to deploy three workload clusters That are all linked to the same Cluster template and in this example. It's eks managed and pull which is an eks cluster With a managed node group Okay, so all three workload clusters will be exact identical copies of the template Except that they'll have unique names however, you can also specify on a per cluster basis some Surgical changes that you want to override So Arlon lets you specify customization patches For example Cluster two might use the same template that you might want to change the region in which it is deployed same thing for cluster three Now last but not least How do you specify what applications are going to be deployed in every cluster for that? we're going to use an annotation on every cluster and And so for the demo I'm going to use all Clusters will start out with the same set of profiles, which is the below and the red profile Okay, so initially they're all going to get the same set of apps So I think we're ready for the demo and this is a live demo. So I hope the demo gods are with me So I'm going to show you Provisioning from A to Z We're going to deploy clusters with apps and We're going to look at many update scenarios and show you how powerful the update mechanism is in Arlon And if we have time, I'll show you tear down as well Okay, so let's switch here. I hope you can see this this shows you the management cluster and it has many namespaces that are already there Argos CD is installed Arlon is installed and We also have several namespaces that are used by the cluster API system and if I look at the CRDs You can see the data types the custom resources that Arlon has installed so let's look at an example of The manifest that we're going to deploy So I have a folder here with a bunch of manifests that we're going to apply So we have the applications. There's three of them guest book Redis and and gen X Next we define two app profiles blue and red Okay, so blue gets guest book red gets the other two and then we have clusters So we're going to deploy cluster one cluster two Cluster three and they're all identical except that cluster two and three in addition to just consuming the template Which is here They're also specify a patch. This is what I was talking about earlier. This is the override so the template by default will Deploy the cluster into the Ohio region. So for cluster two, we're saying we want this to go into California and then the next one we want it to go into Oregon, I think okay Let me show you what the manifest looks like for the cluster template this is a cluster API manifest if you've never seen one and It's a Going to deploy a Kubernetes cluster of version 1.22 by default. Okay Now since The deployment will take 10 to 15 minutes What I did was capture a video of this first phase for you a few hours earlier So, let me see I had this video. So this is what happens when you Apply the folder. So as you can see We are creating two app profiles three apps and Three clusters, okay Now what I'm showing here is the Argo CD UI since Arlon is using Argo as its deployment engine Each of these boxes. It's called an Argo CD app But because Arlon also has a concept of app, let's use a different term. I call these deployment units, okay so you have a live view of the deployment units that we are creating or Arlon is creating and Since we're creating three workload clusters We end up with six of those boxes and the reason is there's two boxes per cluster So if you look at cluster one for example cluster one that deployment unit Contains the a copy of the template that describes the shape of cluster one There's a companion unit called cluster one dash Arlon Which contains resources that are needed to support the life cycle of cluster one So for example cluster one is a manifest, but it needs to live in a namespace so that it's isolated from other Cluster deployments and that namespace resource is provided by the cluster one dash Arlon Deployment unit. I hope that makes sense So what's happening in the background here is The cluster provisioning engine is Deploying the cluster to Amazon in this case and Arlon is Monitoring the progress of that and as soon as that cluster comes up and is available Arlon will automatically Register that cluster With Argo CD, which is the same tool that we're using here and then proceed to the next stage Which is to deploy the applications that we specified using the app profiles, okay? So after a while you're gonna see a bunch of new boxes Suddenly appearing here So we're going to fast forward. Let's see So in a moment you're gonna see a brand-new set of Deployment units being created automatically and there's gonna be one per app per cluster. There you go Okay, so clusters two and three just came up So now you see a bunch of applications coming up for those clusters and cluster one is gonna follow shortly All right So now we've completed the initial deployment phase And I'm going to Go to a live a live view of the Argo CD UI. Okay, so this is the real thing. Okay So now that we have all of this One thing I want to mention is the way you distinguish the applications From the clusters is any application gets this naming convention, which is the cluster name App and then the name of the app Okay, so what can we do now? I'm going to show you several live update scenarios and The first one I'm going to show you is remember that we specify the apps on a cluster through this annotation I'm going to show you this annotation so If we look at cluster one only this cluster one has three apps as expected if I look at the OEMO for cluster one This is the profile I was talking about. Okay, blue and red So what if we change the annotation to remove red Remember that red currently holds Redis and nginx I believe You can see that two apps got deleted automatically and this is happening to all clusters not just cluster one Okay, because the other clusters are also using No, I'm I'm sorry This is only happening to cluster one because we are modifying the annotation of cluster one only Now what happens if we completely remove the annotation? Then the final app goes away Okay, so let's restore the annotation And those apps come back Now let's get to what I was saying earlier You can also change the composition of a profile by editing it live And this will have an effect an immediate effect on all clusters that are using the profile So for example, if I were to edit The red app profile It's very simple. It's just a list of Of app names So let's filter this based on all the redis one So you can see that clusters one two and three have redis If I just delete This from the app profile They're all gone Okay, so live updates of profiles Have an immediate effect Let me show you Updating an application source So let's say we looked at engine x on cluster one and engine x manifest is currently living in git and let me show you the Manifest for engine x. So it's a stateful set. Okay Let's Maybe change replicas to three Let's do a diff Okay So we're going to commit this change and get bump replicas to three And then let's Push let's see what happens And immediately you'll see that the stateful set controller has detected the change through our on an argocity And has scaled the deployment to three pods. Okay and Last but not least The most powerful and impactful type of update that you can perform is actually cluster upgrades So imagine you have a hundred clusters deployed in many edge locations And they're all centrally Specified through a single template and you want to upgrade all of those Kubernetes Clusters to a newer version So how would that work? You would go to the manifest that we're using And let's see we want to go to version 23 123 Okay, so Oops Let's do a commit bump case version to 123 And before I do that Pay attention to the boxes labeled cluster one Cluster two and three. I think they're all on the left here What are you going to see is as soon as I commit it and push the change Those will refresh after a while and then they'll flicker from green to yellow and back to green It's happening pretty fast Yeah, so what's happening here is We've sent a signal to the cluster api subsystem That we do we want the new desired version to be 123 Even though it's showing as green What's actually happening is the upgrade is in progress right now. Okay and the way you can confirm that is to go to the Amazon console and you can see that it's showing The control plane of the cluster at version one to two But if you do a refresh, let's see It's saying updating right so we've triggered the potentially large-scale update And last but not least It's also very important That we be able to clean up All the resources that we've created whenever we want And since everything is fully declarative and everything is a first-class Kubernetes object You can just do a kubectl apply. Sorry a delete of the whole folder and it will Arlon will orchestrate The cleanup of all of those resources and that'll take a while. Okay So I hope that this kind of give you a taste of what this tool the kit is capable of and So we've shown A unification of The api and the tool set and methodology for managing clusters and applications We looked at profiles on how they Help you address the problem of configuration sprawl and we've also looked at how Arlon offers this very powerful linked update mechanism where You can make small changes to set of shared profiles and cluster templates and have those changes Propagate to all of your deployments While preserving any customizations that you may have specified For more details feel free to visit the repo There's a slack Channel if you have questions and of course you can contact myself and any of the maintainers I hope this was useful to you and i'm gonna Maybe take a few questions if we have time Thank you very much