 I can provide my desired state in a YAML file in case of Kubernetes. And then we can collaborate together and store those files in Git and somehow provide this desired state of my infrastructure to Kubernetes. The thing is Kubernetes does not provide out of a box a way for extracting this last state from Git. And that's exactly where Hargo CD comes into place, right? Hargo CD has exactly this responsibility to get the interface with your Git provider and it will present this latest state of your infrastructure to Kubernetes. And Kubernetes will kick off the reconciliation loop to bring up your desired state into the live state. Okay, and let's say I convinced you to go with Hargo CD. And what is the first decision you have to make? Which Git repo to use, right? This is the first question you have to answer. And we basically understand that there are two different approaches you can take here. One is dumping everything in one single repo, meaning you're gonna be using the same repo where you have all your search code to store your manifest files, your infrastructure configurations. Or you can use an independent repo which is totally disconnected from your search code and to store this configuration files, right? Maybe intuitively you might go with the first option. This is what I did when I first installed Hargo CD myself. And let me show you what are the pros and cons of this approach. So basically with a centralized repo, one of the pros is that things are placed in the same repo obviously, right? You can just open a folder and see what is the current configuration of your production environment. It's really in the same repo where you have all your search code, which is great. But what are the disadvantages with this approach? So we understand that the authorization model might be harder to define. Basically because most of the times, the role of the member of your team who has access to commit and merge code in your master branch, in your main branch is not the same role of the person who has the permission to deploy and change things in production, right? So this is something that is usually a requirement. And if you have everything in one single repo, it's gonna be harder for you to come up with some configuration, authorizing just specific type of users to change your production environment. Another disadvantage is that this is going to provide you some more complex CI environment. So if you're opting to get installed something like Hargo CD, it's probably because you want to have continuous deployment. And by that it means every merge to the main branch will trigger a job that will build your application and create a new version. But then you have to update the configuration files again and which will trigger the job. So it's a kind of a chicken and egg problem. And it's not impossible to address, but if it can be avoided, the better it is, in our opinion. And the last one is sometimes a requiring companies to have history of everything that gets changed in production. So if you just dump the files in the same repo as you have your source code, what is going to happen is that Commits is going to be mixed with production, configuration together with your source code. So if you want to extract a report to who changed, which resource in production and when, it's going to be a little bit harder for extract that if you have a mixed repo, a centralized repo. Okay, moving forward, let's see how it looks like if we go with an independent repo, right? So what are the advantages? In this case, it's a simpler CI or no CI at all. You understand that you don't necessarily need a CI so you can, if you have a totally independent repo like your application, there's core deployment and you dump your manifest in there, you can just configure RGLCD to watch this repo like the main branch, every single commit that gets pushed to the main branch will trigger RGLCD sync process which will update your cluster. So you don't necessarily need a CI in this case in simple places, right? So this is also going to be, provide you a much easier authorization model capabilities because now you're not tied to the same repo where you have your source code. Now you have the entire repository so you can leverage, let's say you use GitHub, you can leverage already available GitHub features to decide who can do what, who can merge what, branch protection, yeah, there are many features to help you with that and it will provide you a cleaner history. One disadvantage is that in this case, yeah, you're gonna have to deal with two different repositories, it's not gonna be centralized if that's an issue, we don't think it justifies all the advantages we have. Cool, let's say I convinced you to go with an independent repo. What is the next problem, right? You might run through, you need to decide what is the layout, what are the files that you're gonna be dumping in this deployment repo looking like, right? And one thing that we've been using for quite some time is customize. So customize provides you the ability to define this, this is the layout suggested by customize which allows you to provide the basic configuration for your infrastructure in this folder called base and then it provides this overlay concept that allows you to leverage this concept to implement things like per development, per environment specific setups. And we use this overlay concept to define different environments, right? So things, configurations in development environment that are going to be different from production and stage environment. So this is something that you can easily get from customize and then it's very, very easy to just configure Argo CD to watch those specific folders or maybe you have that in specific namespaces and deploy in, sorry, the specific branches and then you can configure Argo CD to deploy in specific namespaces and specific clusters, right? Okay, let's say you're good with this but now you want to protect your application, your team to interfere with other teams, right? You want to provide some sort of a multi-tenancy. How can Argo CD help you with that? So Argo CD provides this, it's a different CRD in Argo CD project called project where you can define things like who is allowed to use this project, which repos are allowed to be read from this project, define Orbex, define what are the target clusters that can be configured to deploy applications assigned to this project and then you can have your existing application and assign to those projects to kind of protect your environment in a good way, right? So there's still one missing piece here. I mentioned Orbex, I mentioned about users having permission but for that to be possible, I need to somehow have some authentication in place. I need to have users assigned to roles so I can properly configure this Orbex. And this is exactly how Alex is going to join this presentation providing this hands-on section with you, Alex. Cool, thank you. Thanks a lot for providing this great overview of how Argo can help. And now I want to talk about how can you get yourself an Argo CD that actually can make it possible? And before I go deeper into details, I want to describe my requirements for that Argo CD. And as Leo said, if we want to provide it for the whole organization, it has to be multi-tenant. And what that means is Argo CD has to be able to serve lots of teams and teams has to be protected from each other. Basically, I don't want anyone accidentally deleting someone else's production and things like that. And next, I want to provide a good experience to the teams. It's usually security is never convenient, but I do want to have very seamless onboarding process so every team can just onboard themselves to Argo CD. Plus, I want to take care of a platform team. I don't want them to work too hard to do it. And that's why I want to have this process as self-service as possible. Thank you. And is that, let's talk about things that we will have to manage. And then later on, we can talk about how we're going to manage those things. So the very first thing that we would have to configure is authentication. And if you ever installed Argo CD in your mini-cube cluster, you already know that it has a system accounts feature. By default, every Argo CD has admin accounts that has God level privileges and it can manage anything. And I just want to stop you from using this feature. So there is a reason why it's not called users. The system is called system accounts because it's meant for IPI integrations. And usually organizations uses it to automate Argo CD settings management. And we really do not want to use accounts to provide multi-tendency in production. So, and there are a ton of reasons to use SSO and identity provider. If you are a member of a platform team, you're going to be really glad you made the decision to use identity provider. The main reason is you just push, you save yourself a ton of time by pushing away this work of user management. Basically, you will not be responsible for storing users. You will not have to resolve tickets to add one member from, move member from one group to another group. And basically, that's why Argo CD takes a single sign-on so seriously. And few words about how to configure it. So the main integration point for Argo CD with any identity provider is OIDC protocol. And basically you have a ton of choices if already with just OIDC. There are providers like O0, Octa, Azure, IDP, and many more. So OIDC is very popular. And this little YAML snippet is just to convince you that it's not so difficult to configure. You pretty much just need to provide a URL and couple secrets. And that's it. Argo CD will be connected with your SSO provider. And even if you do not have OIDC, you still can connect Argo CD to your provider through an awesome project called DEX which is bounded into Argo CD. And I'm going to actually use DEX in the demo part. So we will see how easy it is to configure single sign-on with provider like GitHub. So once we've done this authentication, the next question is how do we authorize our users? How do we link the OIDC group that SSO provides to the user once login is complete? How do we map it to some RBOT configuration? And here Argo CD again decided not to reinvent the wheel. So we are not building any custom RBOT solution. Instead, Argo CD leverages an awesome project called CASBIN which is an engine that lets you manage your kind of access control configuration. So CASBIN allows to configure a bunch of access models such as RBOT, RBOT, ACL. And so at Argo CD we made a decision to use RBOT. This cannot be changed. However, you can use CASBIN configuration language to define RBOT rules. And this snippet here has to search the same purpose. It's really easy to use CASBIN. And easy use to configure CASBIN. It's not YAML this time. It's just a little CSS-like file that let you define groups and policies, assign policies to groups. And my point here is that it's super flexible. Basically you can assume that you will be able to achieve very complex use cases related to RBOT using CASBIN. And if you understand that project, you might leverage that knowledge in some other systems because it's used pretty widely in some other systems. And so now we have authentication and authorization. We get to make them work together. And this is maybe the last important concept that we have to learn before we jump into the demo. That concept is called Argo CD project. And it's pretty much the core of Argo CD multi-tenancy functionality. It let you take your configured SSO and your bug settings and kind of link them together, make them work together. And what you can do using project is you can group a bunch of applications. And what that means is every application belongs to one and only one project. Next, you can define boundaries on that project. And those boundaries will be enforced on every application that belongs to a project. And you can limit which source repositories can be used as a source of manifest for every application. You can decide which namespaces those manifest can be deployed to. And in many cases, and we suggest to use this approach, users use project to represent a team. Basically, you can create a project per team, define the set of infrastructure that team manages and let them use the source repository that they use to manage the Kubernetes configuration. And so once you have this boundaries setup, all you have to do is you just need to tell to the project which OEDC group manages the project. And I hope you can see it, it's the last line of this YAML file. So in this case, I have imaginary my work, my team OEDC group that probably represents one team and that team has access to a project. And you can use Caspin to provide fine grant access to the users of the team within a project. And so having those three things, we pretty much can solve the requirements that we described before. We can get the multi-tenant target CD instance. The question is, how are we going to manage it? And I think this is the whole purpose of that presentation. It took us some time to get here, but so my point is to convince you that you do not want to use tickets to manage all of that, like to manage our bark SSO and then projects and some other things that as you will see in the future shortly. So you kind of give a little bit more color to the problem. So of course you can set up a Jura project, you can tell to your users that every time they need to get access to our CD, they're supposed to create a ticket and then someone from the platform team will respond to the ticket and make changes. But in real life, maybe like six months later, you will realize that our CD is a critical piece of infrastructure that touches very sensitive, manages sensitive configurations. So that's why each and every change in our CD must be stored in some audit system. You want to have some approval process. You want to take care of the settings that you as a platform team apply to our CD and those changes also has to be stored in some form of auditing system. And if you take into account those requirements, then it's kind of too, it's already frustrating enough to try to do better. And luckily for us, there is a better process that we call GitOps. And so the biggest question is, can we use GitOps to manage our CD configuration? And the answer is we definitely can. And the reason is our CD has no database behind it. It persists everything in a Kubernetes cluster. And that's why all you have to do, you just need to manage Kubernetes manifest that represent projects, config maps, secrets, hopefully no secrets, just config maps and projects. And that's why we can use GitOps to do so. And with GitOps, developers are going to use tools that they already know, such as GitHub, for example. They're going to be familiar with pull requests concepts and administrators of a platform team are developers as well, engineers. So they would like this approach as well. And that makes everyone happy. So now it's time to see it in action finally. This bar code is for you. If you want to do the demo after the talk, if you want to play with it, just take a screenshot maybe, and it will lead you to this URL that I'm going to open right now. And the URL is a Git repository. The name of this repo is Control Plane. And let me describe why I chose this name. So the reason is this repository is going to have a bunch of Kubernetes manifests that describe everything that platform team manages. So if you look at the content of it, you will see folders like Argo CD. And by the name you can guess, it's going to manage Argo CD configuration. And next it has clusters. And so it will, the clusters folder is going to have everything that platform team manages for end users in the target managed clusters. And I will describe shortly why it is important. So next we can switch to the readme file. And readme is pretty much a step-by-step instructions of, it's a script of the demo. I already finished the preparation part. I installed the Kubernetes, I just use K3S cluster. I installed Argo CD into the cluster. And then I configured Argo CD to manage itself by running this QCTL apply command that simply inserted this YAML file. It applied this YAML file into the Argo CD namespace. And so let's take a look at this file. Let's see what it has. So here it's just Argo CD application that tells Argo CD to watch this same repository, look into the Argo CD folder and push everything from that folder into the Argo CD namespace. And that's pretty much everything we need to get this self-managed setup. So now Argo CD is configured to manage itself. And from the beginning, I kind of took advantage of it. And I created couple more YAML files here. One is Argo CD CM, which is Argo CD config map. And I did configure SSO using DEX and I connected it to GitHub. And so it's really, it's so easy. So there is nothing to learn about it. You need to create a GitHub of zero application and configure a couple of secrets here. And that's it. Argo CD is going to know how to talk to GitHub and DEX is already bundled in Argo CD so you don't even have to configure DEX itself. Next, I needed to create myself SSO powered administrator. And to do so, I created another config map that defines RBAC rules. And here I gave admin permissions to Argo itself because I have access to that account. I can't use it for demo. All right, sorry about that. Now it's time to see it in action. So this is Argo account. And I want to go ahead and prove you that the SSO works. So if I click login, yeah, SSO worked. So we were able to login. And because this user is admin, it can see everything. So in case you don't know, this is how Argo CD user interface looks like. It is showing me an application that manages Argo CD itself. If I click on it, you can see that it does manage few resources. It manages config map. It manages RBAC config map as well as the application resource itself. And here we get this infinite loop that Argo CD manages itself. And next, let's login as a user. As a user who has no permissions yet. So if I do the login here as a user, as using my personal account, I won't see any application, which is good. It's expected because I'm definitely, as a user, not supposed to see applications of a platform team. And next, let's go ahead and try to create an application. And if I do so, first thing that I have to provide is I need to choose which project my application is going to belong to. And there are no projects available for me. And so now I want to onboard my team, which consists of just me. And I want to, instead of creating a ticket, I want to use GitOps. And I'm just going to do it right now. So I'm going to make a change in that Git repository. And here I'm kind of providing you a convention that work for even a big company. So you can just have a project folder under your Argo CD directory. And you can have a convention to have a project per team. And so the onboarding process is to just copy like a sample file and provide team-specific configurations. I already created that file in the repo. It's not in the system yet because everything here is commented out. So I'm just going to uncomment it and create a pull request to make a mistake. So I uncommented it, created PR. I will show you the content of this file shortly. When I switch back to my admin tools and I'm trying to review this pull request. And so now I'm acting as a Argo CD administrator. And instead of reading the description and the ticket, I'm trying to understand what the user wants. I kind of, I have already a work done for me. I just need to validate that it's meaningful and maybe make sure that the person who created it follows the convention. And so here I have a project which name matches to the team name. It allows this team to manage whatever repo they want, use whatever repo they want to manage the manifests. And it allows the team to deploy applications from that repo into any name space that starts with a prefix that matches to a team name which is another powerful convention that might be useful for you. This way you don't have to manage each and every change you kind of give a little bit of flexibility here and it still let you separate teams from each other. So let's say I like this pull request. Oh, and by the way, here I'm also, I'm explaining to admin that, hey, the members of the team are going to be part of this group. And for the sake of them, I just choose email to be a group. So it will only wait at least me. All right, so if I like everything here, all I need to do, I just need to merge this pull request and approve and merge it. So now we have audit of this change. Next, if I switch to Argos CD, I have to click this refresh button just to save us like three minutes of time because Argos CD check it every three minutes. And so if I switch back to Argos CD as a user, you can see now that I have a project available for me for pretty much onboarding is done. Now I am as a developer ready to use Argos CD. I can create application. So I can have this pre-filled to save time but I do want to show you that I'm using the right repository and I'm going to deploy into namespace that starts with my team name and let's pretend it's my dev environment. So I'm going to give it a suffix called dev. So because RBAC is working, I can create this application and let's go ahead and deploy it. And that's intentional. It's not the demo fail. It's failed because we still have a lot of other things to do like it's not enough to just configure RBAC. We also need to get some cluster-level resources like namespace for example. So I chose this namespace team a dev but no one created it for me. And you might also, you might, we're short on time, we have five minutes left, okay. You might use tickets to create namespaces but it's going to be the same pain as a project plus this problem is like extremely, it's very similar to the problem of managing RBAC. So I want to convince you to use GitOps to manage cluster-level resources as well. And the reason is Argo has a great support for it, for that use case. The only problem, the only difference of managing cluster-level resources is that those resources are not part of Argo CD namespace. So you would have to create separate set of applications for each and every cluster and that's a lot of manual work for administrators and Argo CD has an answer that solve a little bit, reduce this pain. Answer is called application set. So application set is an admin tool that helps you to automate application creation. And this is the application set that I already had in this repository. I just need to push it and comment it and push. And application set is configured to watch clusters folder in my repository and create an application for every subfolder, assuming that subfolder is the name of a cluster registered in Argo CD. And so what it will do for us, it will automate propagation of every manifests in the cluster subfolder into appropriate cluster and I can let my users and users make changes in that folder. So this way administrators kind of, still all they have to do is to just review PRs and don't do much of manual work. So since I'm admin, I'm just going to push it into the dev branch, into main branch. I don't have to create PRs. Let's refresh application again. So Argo CD detected the application set. Application set created its application that manages cluster and managed resources in the cluster. Now if I switch back to my admin and user account, I can make a change and finish the onboarding. I'm currently in the cluster folder that represents cluster where Argo CD is running, which I'm using for demo. And I have a snippet of like name spaces for my team. So here I'm introducing three name spaces, dev, stage and probe environments. Let's go ahead and because we're short on time and I can, let's not do a pull request. Let's just merge it to a main branch. But it's bad idea to do it in production. Now if we refresh this application, we can see that Argo CD detected that trinium spaces should be created. Jumping back to the user UI and we can finally synchronize and deploy our application. And that's pretty much, this is the real life workflow that you can use to run the whole organization. So it scales really well. You don't have to manage a giant file and resolve more conflicts. And this is the link for you to tell us what you think about this approach. So provide us any feedback and we have like one minute left for questions. Thank you, we made it. I have a question there. Thank you for the demo. And I really like the application set feature, but my logical question is, is there plans for project sets? Like something that would be generating projects based on the directories. I think, I know that right now there is nothing like that for project. I know at least one company that uses API to generate the projects. I guess that's the best answer we have right now. So the company has another source of true for projects and there is literally a Jenkins pipeline that, so not Jenkins pipeline. There is a kind of controller that creates, watches the database and creates Argo CD project for every database entry. Hi, I'm Junaid. I'm working at a company called I-10HQ. Just wanted to ask, can we do resource ordering for deployment? Yeah, Argo CD. Yeah, there are two features. One is called sync waves. Have you heard about sync waves? I haven't. No, okay. I just started looking at Argo CD, so I'm not. I guess that's the right answer. So sync waves is a simple way to kind of orchestrate deployment process a little bit is if you apply a certain annotation on your resource, which is something like Argo project.io slash sync wave and the value of this annotation is a number. And so by default, Argo CD assume that all resources assigned to wave zero, so it synchronizes everything at the same time. If you want to sync something before everything else, you can apply the annotation and put minus one as a value and Argo CD will make sure to sync those things first. If let's say you're deploying, if you're syncing a deployment, it will also make sure it's healthier and then it precede to next wave. And this way you can, typical use cases, like, I don't know, you want to... For the namespace thing, we can do sync waves. Yes, yeah, namespaces work like this. Yeah, so it was the namespaces created and then the others. Yeah, thanks, let's... Okay, we have time for one more question. Hi, my name is San. So as a maintainer, do you think like maintaining Argo CD, Argo workflows, Argo events in a team? Do we need a separate team to maintain the entire Argo setup? Or would you recommend to have X teams that have their own Argos? I can share, I guess, my experience. I think it really depends on how many engineers you need to support. So in our case, we had a lot of engineers, so we had to have a dedicated team to manage Argo CD. And I think it really depends on use cases. It's just we learn it from experienced workflow users. Usually, they need so completely different set of problems. Usually it's like ML data processing. And that's why for that reason, mostly what happened with us is we had a separate team managing workflows and separate team managing CD. So I guess, yeah, if you have a lot of developers, then there is enough work to do on Argo CD site. And if you're most likely going to solve different use cases, so you will need two different teams with different skill sets to manage. I see, thank you.