 Hi there. Thank you so much for joining me here at GitHub's Con, where I'll be discussing applied GitOps with Argo CD Autopilot using multiple clusters and an application set. My name is Hannah Selickson and I'm a developer advocate at CodeFresh, which is a continuous deployment platform that supports both GitOps and Argo deployments. However, prior to being a DA, I spent several years as a .NET developer writing C-Sharp and primarily designing and building REST APIs for the insurance industry. Later on, I took on a role as such as a COE lead, which is essentially a center of excellence lead. I made sure that our developer organization had the right tools, resources, and trainings to execute their tasks efficiently. This also included vetting tools, monitoring workloads, and then reporting back to our VP of IT, management, and CTO with data that I either proved that we were working quickly and efficiently, or we wouldn't, and we would have to come up with solutions. So this is why I'm excited to be on the other side of the table nowadays, sharing with others these tools that can really help you. So today, we're going to talk about why organizations are implementing GitOps to their deployment process and essentially what does that mean. And then we're going to identify some of the challenges that these organizations face when they are applying GitOps to their workflow. Then I'll introduce you to Argo CD Autopilot. We'll explain how it works and how it can help those organizations who are utilizing GitOps and using Argo for their deployment process, and then we'll give a brief demonstration. So the infrastructure management growth. First, we start out with our programmatic automation, and this includes like your PowerShell, and then we evolve into configuration management, infrastructure code, and then GitOps. GitOps is becoming an essential part of the cloud operating model. It promotes DevOps best practices to the management infrastructure, to improving the speed and quality of your infrastructure service delivery, and maturing infrastructure operations. Making GitOps part of your cloud operating model enables you to standardize application and infrastructure management processes, driving both from definitions stored and version controlled repositories. So essentially your source of truth. The shortage and popularity of GitOps is due to the desire of speeder application deployment cycles. Organizations today want to be able to deploy their new features as quickly as possible to their customers. This improves business, profitability, and it makes everybody happy. So as a result, an increasing number of businesses are expressing interest in using this framework just this year in 2022. GitOps is attractive because there's a small learning curve. It's fairly easy to set up and it can be adopted pretty easily. It allows your teams to work faster, deploy their own code without the need of a heavy operations team, all in tandem with having boundaries put in place to ensure that those devs don't go breaking things and make unauthorized changes to your infrastructure. Plus it's important to mention that because just because you store your manifest and Git through your manifest file in your Git repository does not make your framework GitOps. Sorry. GitOps combines an orchestration system, Kubernetes, and your source control system with a monitoring software. This is to ensure that we create a close reconciliation loop in which an application running in production should always match its desired state. This is recorded in version control and your source control. Therefore any deviation between the two will be detected automatically. So in today's evolution, tech industry cannot afford to wait several months to release a new feature and get in the hands of your demanding customers like I mentioned before. GitOps provides business value to those who adopt it. And regardless of your size, big or small, based on the fact that cloud applications can be updated faster without compromising the security or the stability of their infrastructure, the benefits of doing this can sometimes be overlooked just due to ongoing debates of legacy ways that lots of us can relate to when it comes to bigger organizations with clunky legacy systems and an outdated culture. However, based on research such as the DevOps research and assessment group, aka your Dora metrics, there comes several organizations have found that when it comes to delivery for a Kubernetes or cloud app, there are four major variables that determine this organization's success. And those are described as your software delivery and operation performance measures. It's these four measures, frequency of deployment, lead time for changes and updates, mean time to recovery, and your change failure rate, which is otherwise known as the proportion of changes or updates that fail. With GitOps, it's a sure way to improve your organization's effectiveness against all those four measures, metrics that are providing critical and many business to market share and profitability. So now that we've really kind of proved that, hey, applying GitOps to your workflow can really improve your business, your profitability, and also probably the happiness of your organization and your teams and how they work because they're working much more efficiently. But when you apply GitOps and you're starting to explore best practices and making sure that you're executing this properly, there do come a lot of challenges. And those challenges can really be as simple as how do I handle multiple environments and apps and multiple clusters when I've containerized all of these services and now I'm using a CI CD tool and I don't know how to do this. Also, when I need to promote changes to those multiple applications and I need to also utilize different clusters, it can be a little tricky. So maybe what you need to consider when you are thinking about these challenges are essentially the time constrained engineers who are performing a lot of these tasks manually and just how expensive and inefficient that can truly be because the effort in transition to a GitOps workflow is super appealing. And you also have to consider like your testing and gatekeeping and your regular maintenance processes that can lead to really expensive and also preventable environmental issues and additives for your organization. So this brings up a question. How do we begin applying GitOps to our deployment process so we can deploy more frequently and reap those benefits? So it's no surprise I'm mentioning Argo is it? Argo is one of the most popular tools in the cloud native space and has continued to experience rapid adoption. Argo has built in a GitOps flow and so that's why it's so appealing to organizations when they begin this shift in their culture and in their processes that they utilize Argo. And I want to point out two major concepts when it comes to Argo. First it's their application which an Argo application is essentially what connects a namespace within your target Kubernetes cluster with the declarations of your desired state and your Git repo. And this can be done by defining a lot of your configurations within a raw manifest, a home chart or a customized overlay. And then the second major concept I want to mention is a project. A project in Argo CD allows you to group your applications which is useful when your organization has multiple teams. However, I also think of project as environments such as it could be regional, it could be your staging, your prod, and this is what's allowed you to restrict what can be deployed and restrict what apps may be deployed to such as your destination clusters and namespaces. I don't want to get too thick in the weeds about these two concepts. However, these two major components of Argo allow organizations to begin deploying their cloud native apps with GitOps quickly and efficiently. So let's take a second to identify some additional challenges when organizations are beginning to utilize Argo. So two questions organizations have once they're on the GitOps bandwagon and they're using Argo. How do we structure our directories? How can I enable Argo CD to manage itself? When organizations are first adopting GitOps and begin leveraging open source projects and tooling like Argo, these basic concepts make others feel blocked and it's because there's no one way to do any of these things. How do I structure my directories for multiple apps and environments? What structure will work best if I need to promote changes? And then also raises the question of how can we install Argo CD so it manages itself or how can we install Argo CD in a GitOps way? Because honestly trying to manually set up a way for Argo CD to manage itself is not easy because essentially what you want to do is you want to use Argo as your orchestration engine because it'll be nice to let it rule itself through your Git repo. Then it enables automation so that any change can show up on your cluster. So with these challenges there's a way that we can kind of just at least have a foundation to answer them or we have at least a roadmap that we can follow and that's when I bring up Autopilot. Autopilot is an open source project for the Argo community that provides us with a tool to bootstrap Argo CD. It also includes an opinionated GitOps setup for your directory structure. And one thing I'd like to clarify about Autopilot is that it is not a replacement for Argo. It's simply an installer you can use to install Argo in a GitOps way. So as I mentioned this tool offers the opinionated way for organizations to kick start their GitOps continuous deployment process and allows them to add applications and simplify the process of promoting changes across those environments. It also has a friendly command line tool for bootstrapping Argo CD on the clusters, setting up your Git repos, managing applications across environments, and even simplifying disaster recovery. You can easily set up one-off environments or staging environments and promote changes across those in a straightforward GitOps way. So how does Autopilot do this? Well, like I mentioned, there is a bootstrap command that you can execute that allows you to do this. And Autopilot saves operators times by installing and managing the Argo CD application using GitOps, providing a clear structure of how applications are added and updated, all centralized from Git, creating a simple pattern for making updates to application and promoting those changes across environments and enabling better disaster recovery by being able to bootstrap new clusters with all the applications previously installed. Autopilot is a tool which offers the opinionated way of installing Argo CD and managing GitOps repositories. So how does it do that? Autopilot bootstrap command will deploy Argo CD manifest to a target Kubernetes cluster and will commit Argo CD application manifest under specific directory in GitOps repository. This application will manage the Argo CD installation itself, so after running the command, you'll have an Argo CD deployment that manages itself through GitOps. From now, we're going to use a concrete project and applications that will launch it up, and Autopilot will commit the required manifest to your cluster. So once committed, Argo CD will do its magic and apply the applications to the cluster. An application can be added to a project from a public everybody or have the recommended local file system on your machine. Autopilot will also communicate to the cluster directly, only during the bootstrap bits, and this is when it deploys Argo CD. After that, most commands will apply access to your GitOps cluster, and when your object will launch an application, your remote cluster, the Autopilot will apply access to Argo CD server. So let's see that bootstrap command in action. So once committed, Argo CD will do its magic and apply the applications to the cluster. An application can be added to a project from a public Git repo or a path or directory in your local file system on your machine. Autopilot also communicates with the cluster directly, but only during the bootstrap phase, and this is when it deploys Argo CD. After that, most commands will require access to your GitOps repository, and when you're adding a project or an application to your remote cluster, the Autopilot will require access to Argo CD server. So let's see that bootstrap command in action. Just give me a moment. Hi. All right. So what we're going to do now is we're going to start by executing the Argo CD Autopilot bootstrap command. And so what this is going to do is it's essentially going to create our directory structure in our GitHub repo. And I've already assigned my GitHub repo, as you can see, and I also provided it with a token so that has access to this. So now I'm just taking a look inside my repo where you can see the directory structure here that I had mentioned before. We'll just take a look. This explains how to create an application. And within this bootstrap Argo CD customization file, this is where we're defining our resources. I'll just take a look in here now. These are our cluster resources. And this is going to be the Argo CD YAML file through this folder, your root. Okay. And this is where our projects are defined. And it gives you an example of how you can create a project. Now let's get back to the terminal. So when we complete the bootstrap command, it provides us a cube cuddle port forward command that we can use to then access the Argo CD UI. It also provides us a password that I can then use and sign in. And now you can see essentially everything that we just created and just saw within my repo. So as you can see, Argo CD is able to manage itself. Now with that command, I just created a project for prod. I'm going to go back into our terminal. Now I'm going to install an application and assign prod to it. Looks good. Now I'm using a different cluster and I've created a project for staging and assigned the application to staging. This is now completed. Now we're going to take a look at the UI again and you'll be able to see my other application for staging, still synchronizing. Then we'll take a look at the project so you can see the staging project. There we go. Okay. And so this is just a brief example of how you can use multiple applications, multiple apps, multiple environments. Fortunately, I don't have time to get too nitty gritty. But this is just a brief demonstration to show you how autopilot works. Now we'll take a look into our file structure. Okay. Yep. There's prod and staging. Awesome. Thank you everyone. So here's an example of a directory structure with Argo CD autopilot. I'm including a demo app and two projects and overlays for staging and production environments. However, whenever bootstrap command is executed, autopilot provides the following for you. A bootstrap folder which contains all bootstrapping resources as shown and root.yml file which contains your application definition for projects folder, a project folder which is essentially a logical grouping of applications. It's this project folder that can actually be used as a different environment like I mentioned for your staging, your dev, qa, prod. You also have a cluster resources.yml file and this is an application set controller managing your cluster resources folder by pointing the cluster resources in your cluster.jsm file. It uses Argo CD, git generator file and manages the Argo CD namespace in your Kubernetes cluster. An Argo CD.yml is an application definition to self-manage Argo CD itself. So autopilot also has tons of commands which you can use to add new projects and application, which is why I mentioned those earlier. It creates and commits any changes made to your GitHub repository or whatever repository you're using and ensuring that your desired state is applied by the Argo CD operator. So now as you can see within my structure, I have multiple environments and for this demo application I can also create additional applications and leverage different clusters. But how? Well nowadays Argo CD incorporates an application set with its installation. So what is an application set? Well, unlike an Argo CD application resource that deploys resources from a single git repository to a single cluster or namespace, the application set expands this functionality. An application set uses a template automation to create, modify and manage multiple Argo CD applications at once, while also targeting multiple clusters and namespaces. The application set controller is now installed alongside with Argo CD within the same namespace and it creates multiple Argo applications based on the application set custom resource. This is what ensures that your Argo CD app remains consistent with your declared resources. And it essentially takes the application set and outputs one or more Argo applications. Well, fun fact, something I didn't realize when I submitted my CFP for this talk was that Autopilot incorporates an application set and applying this pattern doesn't need to be a separate step when you need to create multiple apps. So before I learned about this tool, I thought wouldn't it be great if we could apply an app set after we install Argo CD to set up our directory? Well, the maintainers of Argo CD and the community agreed that hey, it would be really helpful if we could install app set when we install Argo CD because we all are working with multiple apps. Oops. So now that you've had your crash course of Argo CD Autopilot and seen a demonstration of how it works, let's review some of the benefits. Autopilot provides an opinionated way to build multi-project, multi-application system using GitOps principles. It also creates a new GitOps repository or you can still use an existing one that you have. It supports creating the entire directory structure under any path that you require. When adding applications from a public repository, it also allows committing as either a customization that references the public repo or as a flat manifest file containing all of your required resources. And you can use a different cluster from the one Argo CD is running on as a default cluster for a project or a target cluster for a specific application. So Argo CD Autopilot is still under active development and you can find it under the Argo project Argo Labs. And feel free to check it out, learn more about it. You can also check out some existing issues and find ways to contribute. I really hope that this helps you and your organization get started with your GitOps workflow and makes things a little bit easier or at least provides an opinionated way of how you can move forward. Thank you so much everyone. Have a great day.