 Good morning, everyone. Welcome to our presentation, A Minute More Clusters with Les Hassel with Argo CD application sets. Just to start by introducing ourselves, my name is Jonathan West. I am a senior software engineer at Red Hat, coming to you from Toronto, Ontario, Canada. Hello, everyone. My name is Sharma Jean. I'm a software engineer and a maintainer of Argo Projects. And we thought we'd start out by giving you a brief elevator pitch, say 90 seconds or so, on what application sets are, how they work, and some of the features that you might find interesting there. So application sets are a new Kubernetes custom resource and a controller that's responsible for managing that custom resource, which works alongside an existing Argo CD install. So you specify, with your application set, a template, a generator, which is used to customize that template. And then the application set goes and creates applications based on the template and generator combination you specify. So you can think of application sets as being kind of like a factory for Argo CD application. You specify the template, the generator, and then the application set controller automatically outputs a bunch of applications that meet your specific application specifications. And that lets you do some really neat things like, say, if you've got Git repositories containing 50 applications that you want to deploy, and you want to deploy them to 50 clusters, you can manage all of those combinations, all of those deployment combinations using Argo CD applications within an Argo CD application set as a single unit. So it's very easy to manage. Those cluster deployments can be customized using data from external data sources. So for example, as you scale up to add new clusters, as you add new Git repositories, as you add new applications to your existing Git repositories, those will automatically be detected by application sets and deployed to your cluster using the template. It does automatically react to new external changes on the fly. So as I said, infrastructure changes correspond to cluster changes, and that's both quick and customizable. Finally, application sets are based on Argo CD applications. So for folks that are already familiar with Argo CD and know and love it, you're going to get all the power that comes with Argo CD with Argo CD application sets. So speaking of harnessing the power of Argo CD, I will now hand it over to Shama to talk about Argo CD and GitOps. Thank you, Jonathan. So to begin with, I'll start by giving a brief introduction of GitOps. I'm sure many of you might already be aware of it. GitOps is a way of implementing continuous deployment of cloud native applications. It is a workflow that emphasis on Git being the single source of truth. There are two major components of that workflow. First is that the desired state of your software is stored in form of files on Git. And because we are talking about cloud native, your software environment will be stored in Kubernetes and you'll be storing Kubernetes manifest on Git. And the second part is that there is a GitOps agent, a piece of automation software, whose sole responsibility is to make sure that the desired software state keeps on running on the RENTAN environment. Hence, Argo CD comes into picture. So Argo CD is one such GitOps agent. Now what is Argo CD? So Argo CD is a GitOps-based continuous delivery tool for Kubernetes, which uses Git as source of truth. It is declarative and supports a variety of configuration management tools such as caseinit, caseinit, customize, and any custom config management tool configured as config management plugin. It is implemented as Kubernetes controller and CRD. And it provides a Kubernetes dashboard with powerful sets of source management features. It makes sure that application's current life state is same as the desired state. So the diagram here explains how Argo CD works. First, you will define your application's manifest in Git repository and then connect that repository with Argo CD. And then applications can be synced programmatically either via GRPC or REST API or they can be synced via Argo CD's UI or CLI. Now let's get to a quick demo. I'm sorry. So this is Argo CD's user interface. Here you can see various applications that this Argo CD instance is currently managing. Let's get started by creating a simple application. So for demo purposes, I'll be using this Git repository available as this Git repository of sample applications available as part of our group project. And today we'll be deploying customized application. So as you can see, Argo CD detected all potential applications in the given Git repository that can be deployed in the Kubernetes cluster. For destination cluster, I'll select the cluster where I want to deploy my application and the namespace. Here are some parameters that you can set for your application depending on the configuration management tool being used. Argo CD detected customized specification file in the repository path and hence it is providing some customized specific options. So our application is now created. Argo CD application is a logical grouping of related resources managed and deployed as a unit. In the middle you can see the resource tree and this is showing that there are four resources currently being managed in Git. Two service objects, one deployment and one stateful set. And each item in the resource tree is selectable and by clicking on them you can see further details. This is the diff of the actual and the desired state of this resource. And this diff that you are currently seeing means that this resource is yet to be deployed. So I'll go ahead and deploy our application by clicking on sync button. So now our application is being deployed. As you can see, Argo CD provides real time activity of the application. And it also shows relations between the objects. For example, this deployment created a replica set which in turn created three pods. And you can see further details such as pod summary, pod events and pod logs. Now Argo CD provides mainly two statuses. First is the health status which is the overall health summary of your application. And it is Argo CD gets started by aggregating the health of the application's resources. And the second is the sync status which is the overall synchronization status of the application. It lets you know if your application is in sync with Git or if there is any configuration drift. Now our application is deployed and is in sync with Git. So I will now go ahead and deploy another version of our application. And because this is GitOps, I'll make that change in Git. After making that change, I'll go ahead and refresh my application. As you can see, Argo CD detected the change and our deployment is now out of sync. And here you can see that the difference diff is in the version. It should be now at 0.2, but it's currently at 0.1. So I'll go ahead and sync it once again. So now our Argo CD application is deployed again and is in sync with Git. So that was a quick demonstration of Argo CD. I do want to mention some key features of Argo CD are as shown on the screen. Argo CD provides some many more powerful features that you can check out on its GitHub page. Now I'll hand over my screen to Johnson. Here's a demonstration of the Argo CD web UI. So this is the Kubernetes representation of applications as we were seeing in the Argo CD web UI. So like this is the central custom resource that Argo CD uses to do GitOps. So because this is GitOps, we have a source repository that contains the Kubernetes resources within a Git repository. In this case I'm deploying a desktop from the desktop path. So that looks like this. So just simple Git repository with a deployment resource and a service. So nice and simple. And I want to deploy that to a Kubernetes cluster. So I've got my source. These are the resources that I want to deploy. And this is the destination for those resources. So the guestbook namespace and the particular cluster, which is just going to be my local cluster. So you can think of Argo CD and Argo CD applications as being the glue between Git and Kubernetes. So Argo CD looks at your Git repository, looks at all the manifests that are there, whether it just be like in this example, a set of simple YAML files, or maybe something like Helm, Customize, and so on. And applies them to the destination, applies them to the cluster. Once they're applied, it's up to Argo CD to keep those in sync. So as we saw with Shama's demo, if the Git repository changes, it's up to Argo CD to make sure that what's in Kubernetes is always consistent with what's in Git. But notice that in the application CR, there's only one source and one destination. So you can't say, I want to specify, you know, I want to deploy this one Git repository to 10 clusters for like multi-cluster deployments. You can't say, oh, I want to deploy these 10 Git repositories, this one cluster for something like a monorepo. It's just a one-to-one mapping between a single cluster and a single Git repository path. Wouldn't it be great if we could say do many-to-many relationships between Git repositories and clusters? And I'm happy to say with application sets, you can do exactly that. You can deploy two multiple clusters. You can deploy from a bunch of Git repositories all from a single application set, managing them as a single unit. And I will show you exactly how to do that as a demo in a few moments. So the application set controller is open source, of course. It's an Argo CD sub-project hosted within the Argo Proge Labs organization. And it's built from contributions from many different folks, including individual contributions from folks from Red Hat, SNCC, Intuit, MLB, Alibaba Cloud, and more. It integrates alongside an existing Argo CD installation. It's very easy to install alongside that. And then they work together in tandem to do deployments. So how exactly does an application set allow you to deploy from many different Git repositories to many different clusters? Well, it starts out with the application set custom resource, which is the main custom resource of the application set project. In here, there are a couple of fields, which correspond to the two major concepts of application sets. We see generators, which are responsible for generating template parameters, and template, which is responsible for using those parameters. So let's start with template. You might recognize this from something I showed you a couple of slides ago. Notice that we've got the source here, which is a Git repository, and the destination, which is a Kubernetes cluster. And that sounds familiar because we saw that a few slides ago with the Argo CD application resource. And that's because this template field here is just an Argo CD application, a templated Argo CD application. So the exact same set of fields that you get with Argo CD applications, you get here. So you get all the power of Argo CD. But there is something new, and that's these curly braced parameter values here. So two for cluster, one for URL. And those allow you to customize the contents of the application of the deployment to a target cluster with template parameters. So if we wanted to deploy to a different Kubernetes server, or from a different path than Git, we can customize that within the template. So where do those parameters come from, you might ask? Well, they come from generators. Generators are responsible for producing the parameters that templates consume. In this case, I'm using the list generator. It's one of six generators that are currently within the application set controller, and the list generator is very simple. Within the YAML, you specify a set of values. In this case, we've got a value called cluster, value called URL, they're both strings, and those are being inserted into the corresponding values within the template. So this means take these values, put it in the template, and then that causes this Git repository to be deployed to this cluster. So far, so good. So that's all courtesy of the list generator. The list generator is definitely the simplest generator. It's literally just a set of simple keys and values. So everything that you want the template to do is going to be harder coded upfront into the applications and resource. It's the least reactive, but you can already start to see some of the power here. So in the last slide you saw we had a single cluster. Now we've got two. So just by adding these two extra lines of YAML, it now allows us to deploy our guest book application to a second cluster. So the first one was, let's say, an engineering team cluster for development. Now just by adding these couple of lines, we can deploy to the engineering teams production cluster. These two sets of values will be inserted into the template, one template for each set of values, and that will cause Argos CD to deploy to each of those different clusters. So you can already start to see the power of generators, even just the simple list generator example. So what are generators? Generators are responsible for generating parameters which are then rendered into the template, into the fields of the applications and resource. Generators are usually named for and based on the particular external data sources that they're generating template parameters for. So for example, the list generator is just a literal list of values you define within the applications of resource itself. The cluster generator is generating values based on the clusters that are defined in Argos CD, and I'll show you that in a moment. And the git generator is using files or directories from a git repository in order to generate those template parameters. So looking at the diagram, you can see the general pattern that generators follow. So the list generator produces a set of parameters. Those parameters. So in this case I've got a cluster field and a URL field and a couple of rows corresponding to each cluster I want to deploy to. So those sets of parameters flow into the template. These are replaced based on the keys from the parameters as we saw. And for each row, you get a corresponding application. So for the engineering dev cluster, that's going to be one Argos CD application called engineering dev guest book deploying to the dev cluster. One application called engineering prod guest book deploying to the prod cluster, again corresponding to each of these rows. It's up to Argos CD from there to actually look at these application resources and do the the deployments to Kubernetes from the git repositories that are included in those individual applications. So that's the applications that data flow in a nutshell generators produce parameters templates consume parameters. And then the application set controller turns those rendered templates into Argos CD applications. And from there it's the responsibility of Argos CD to actually deploy those to the cluster to look at the gate repository compare it with what's in Kubernetes and keep them in sync. So that's enough talking for the time being. Let me show you a demo of all that. I'm going to jump over to my Argos CD installation here, and you'll notice that I have no applications defined at the moment. And I'm going to bring up my application set resource. This is nearly exactly the same as the one I just showed you. So it's a list generator with a couple of values one for engineering dev one for engineering prod. So we have this book and we're doing it from the application set repo customizing the server field the namespace field and the name. So I can apply this example to our Argos CD namespace, and you're going to see that the application set controller is going to pick that up and automatically generate Argos CD applications for each of those one for each cluster that we want to deploy to. Next we had the prod cluster, the dev cluster. And here you see the application set has read the application set CR and generated the corresponding Argos CD applications. So, because I have am using manual sync, I can go in and click sync. And Argos CD is going to do the actual deployment. So this Argos CD is automated sync to automatically have it deploy applications as soon as they're created by the application set. So just as we saw with Shama's demo, we've got our service and deployment created the replica sets going it's creating pods, and our CD is doing its thing, and keeping the Kubernetes cluster namespace in sync with the repository that we have to find for it. As I said, you can customize applications based on the contents of the application set template. So I can go into our list. And, for example, just make a simple change. Let's say I want to rename our applications by adding demo in front of them, for example. Normally what I would have to do if I want to do this within the web UI is I would have to like go into app details, like edit, make the change that I want, save that, and so on. And I would have to do that for every single application that's defined here. I can just edit this single resource, the single application set, adjusting the template, saving that, applying it, and it will automatically take effect for all of the applications that we have to find. And you see it now says demo for each of them, so prod and guestbook. And this is our old one. It will be deleted as soon as it finishes syncing in order to stay in sync with the new template. So just like that. And there it goes. So just like that, you can see how I'm able to make changes to the applications that resource, and it will automatically affect all of the child applications. So I can put this back, and I can do something like say change the namespace that they're deploying to, so let's say three, four, apply that, and nice and fast, guestbook three, guestbook four. Finally, we can just delete the parent application set, and it will automatically delete all the child applications. So let's sum up everything that we just saw. So when you create an application set resource, the application set controller automatically detects that. If you have generators specified that are based on external sources like cluster resources, like our CD clusters like it, the application set controller automatically detects those inserts them into the template of the application set. And then the application set template is rendered in target CD applications. So as we saw, in that particular example, the list generator produce two applications, as soon as the application set was created. As you modify the application set resource, it automatically updates all the child applications. And when you delete the application set resource, it deletes all of the applications. So the application set and its child applications are always kept in sync. So that way, all you need to worry about is what's in the contents of the application set. You don't need to worry about the contents of the individual applications ever deviating from what you have defined within the template. The actual deployment of applications remains the responsibility of our go CD. So as we saw in the Shama's demo, our go CD will see your applications. And as soon as you click sync, we're using automated sync. Our go CD will automatically start syncing them, but it's still our go CDs responsibility to actually deploy those application sets. The only response that those applications. The only responsibility of the application set controller is to create these the actual connection to the cluster and say identifying deployments and services and doing all that comparison and all that good get off stuff. That still remains the domain of our go CD. So as I mentioned, the power of templates comes from the generators that feed into them. There are many different generators for many different use cases. We already saw the list generator where you had to hard code a set of values in order to insert them into the template, but they're much more powerful generators, starting with the cluster generator. You'll recall with the list generator, we actually had to hard code the particular clusters that we wanted to deploy to. But what if you could automatically detect the clusters that are available. So within our go CD, within the settings. There is this clusters section where you can see all of the clusters that are go CD has access to. In my case, it's very simple. I'm just running on my laptop. So there's the one, but if you're doing multi cluster deployments, you can have like, I don't buy 1050, you know, as many as you want with our go CD managing all of those. So what if rather than needing to hard code them as we did with the list generator, we could just pull all of those clusters that were already defined within our go CD. And that's exactly what the cluster generator does. It looks at the clusters defined within our go CD. And all you need to do is just specify this simple one line of YAML, and it will automatically detect those and start deploying your applications to them from get ops. So pulling the name and server values from our go CD and deploying your applications. Now let's say you've got prod clusters and dev clusters, maybe you only want to deploy to developments, you can target a subset of clusters very easily using cluster annotations and see the docs for that. And as new clusters are added or removed, the application set controller automatically detects that and updates the corresponding applications. Next we've got the get directory generator. Many folks follow the get ops workflow of having a set of Kubernetes resources for their applications defined within a directory. This is known as the mono repo approach. So let me show you what that looks like real quick. So within this cluster add ons directory. I've got a couple of applications I want to deploy one called Argo workflows, and one called Prometheus operator. Argo workflows is the Argo projects workflow controller for orchestration. Prometheus operator is for application monitoring workflows is customized and Prometheus operator is helm. Wouldn't it be great if we could just scan this get folder for all the applications contained within it and automatically deploy the applications that are found here. So, you know, not surprisingly, that is exactly what the get directory generator will do. It will scan through the path you have defined for applications contained within that get repository within the path and deploy them to the cluster. And again that's great for folks that have multiple workloads or components or microservices. And they have Kubernetes resources defined for them within a single get repository. This will find all of them and deploy all of them all from the single application set. Some folks prefer the opposite approach when they're doing get ops. So rather than having one single get repository containing the resources for multiple applications. Instead they have a one to one relationship between application and repository. So one application and it's Kubernetes resources go in one repository one to one. So for them we have the SCM provider generator, which will scan a GitHub or get lab organization. In this case I'm using GitHub, and it will look for all the repositories within the organization that match particular name. So in this case example apps. So if I had like example apps one example apps two example apps three, the SCM provider will scan through my organization automatically find all those applications that match that path, or that name and deploy them to the Kubernetes cluster, using the template values that are generated by the SCM provider. So this is great for folks that prefer to do get ops by having one get repository per application. And this allows them again to manage that all the deployment of those all from a single application set. The third get base generator is the get file generator. Real quick, this allows you to define Jason or YAML files within a get repository and have them automatically processed and parsed and inserted into the application template by the application set. This allows you to have more fine grain control over individual fields of the template versus other generators. For the last two generators. We really start getting into the kind of meta generators, especially with the matrix generator, the matrix generator combines the output of two generators, allowing you to gain the advantages of both. So what does that mean. So let's say that, you know, you like the idea of using the get directory generator to find every application within a path of your repository. And you like the idea of using the cluster generator to deploy to every cluster that's defined within our CD. But what if you want to do both. What if you want to find every application and deploy it to every cluster. So, before you couldn't before you could just do either one or the other, you either, you need to pick one of those two generators so either use the get generator to find all the applications, or use the cluster generator to deploy to all the clusters. Fortunately, with the matrix generator, you no longer have to choose, you can now have both of them together. So we've got the get generator here, the cluster generator here, the get generators going to run. It's going to find all of your applications under the path you specify cluster generators going to run find all your clusters. And then the parent matrix generator is going to combine the two. So let's say you have five applications, 10 clusters, it's going to generate all 50 so five times 1050 individual combinations, and then automatically deploy them. So with this matrix generator using these two child generators, you can deploy every application and get to every argue CD cluster. So you can really start to see the power of the matrix generator here. It'll work with any two generators. So just as another example, if you're following the organization with many repositories get ops approach, you could have it stand through, you can have the SM providers scan through your organization to find all the repositories. You could have the get file generator with Jason files to finding every environment that you want to deploy to and automatically combine them and then deploy them to both in the exact same way. So the possibilities really are endless. Finally, we've got the cluster decision resource generator, which is definitely the most complex generator, but probably also the most powerful. It allows you to outsource the logic of which clusters to place applications on to an external custom resource. So here you actually have to write your own custom controller, your own custom resource, define rules within that custom resource, and have your custom controller enforce them. But once you do that, it allows you to have those custom rules to find exactly where your applications are deployed to on which clusters and the applications that control are using this generator will automatically pick those up. So that is a lot of work. Fortunately, the open cluster management folks already have an example of this with their placement rule rule custom resource where they have their own set of fancy complex rules for where applications should go. And it's easy to hook up that placement rule custom resource into application sets using this generator, but this generator will work with any custom resource that meets the particular generic API contract that the generator is looking for. So hopefully all these generators have piqued your interest in how you can use them to handle your own individual Kubernetes deployment use cases. At Red Hat, application sets are built into our Red Hat OpenShift GitOps product, along with Argo CD, so the two are bundled together for use on OpenShift, and you can learn more about how to deploy using Argo CD application sets and OpenShift GitOps in these two blog posts. This is an introduction to application sets in OpenShift GitOps by myself and Dhawan Ahmed, and getting started with application sets by Christian Hernandez. And over to you. Yeah, so yeah, as Jonathan mentioned that with application sets you can automatically deploy a Kubernetes resource to multiple clusters from Git repository and manage all those relationships as a single application sets here. So leveraging that at Intuit, we adopted application sets to automate the add-ons installation customized across multiple Kubernetes clusters, one such add-on being Argo Rollouts. So Argo Rollouts is a rollout controller of Argo projects, and it is a progressive delivery controller that provides advanced update policies such as blue, green and canary for Kubernetes clusters. So at Intuit, each cluster is allocated an Argo Rollout instance, which in turn maps to an Argo CD application. And in order to automate the Rollouts installation, we selected the Git file generator of application sets, as Jonathan mentioned before. Hence, whenever an Argo Rollout is needed to be installed on a new cluster, a folder containing the respective configuration file is added to the Git repository as you can see on the screen. And the application set controller then detects the newly added configuration file and creates an Argo CD application which in turns installs the Argo Rollout instance on that new cluster. So yeah, that's all about application sets. To find out more about application sets, please visit our GitHub page or read our blog or connect with us on Slack. Thank you everyone for watching and have a great time. Thanks everyone.