 So, hello everybody, my name is Sergey, I work in the Operating System Continuous Integration team as a software engineer. In a nutshell, our team are helping subsystem teams to onboard packages integrating. On the other hand, we have like a bunch of tools inside the team that should be deployed. So, each tool is deployed in a different way. There was a problem. Some of them were deployed automatically, for example using GitLab CI. Some of them were deployed using Jenkins, and some of them were deployed manually. If you need to deploy something, and you are not an owner of the tool, so it's great if there are some documentation about it. It was really pain to upgrade or downgrade the tool, and it was really hard to say what exactly deployed on production. As you see here, like several tools and applications that we use for like in our team, and like most of all the people in the company use it. So, it was really difficult. And we needed this unified deployment process to know what exactly deployed on production and like deploy it in the same way to standardize things and etc. So, let's go for it. What do we want? We wanted to have like other code just deployed in production. And in case of any issues or new features, we want to be able to upgrade or downgrade our deployment. We want to deploy code to production, and we want to know what exactly deployed. So, it's worth to know like, for example, Git commensure of the source code, deployment configuration, and we should be able to roll out our application or tool any moment and create like the same deployment to another cluster, for example. As we update our tools maximum like once per week, usually it's like maybe once per month or once per several months. Deploy to production as fast as possible. It's not about the unified deployment process. The unified deployment process, it's about stability and creating like similar deployment. It's about debugging stuff and knowing what exactly deployed on the production. What tools we use for this? We use GitLab CI and Helm. GitLab CI, it is as you know, is a part of GitLab and you can use it out of the box. You don't need to install it or set up it. It's just in it. And it's very well documented. So, you can find any answer on probably any questions you have or issues you've got. And it's a really cool thing for newcomers. So, it's really easy for contribution. Compared, for example, with Jenkins, when you need to know like Groovy a bit, you need to know how it works. You need to know how to debug it. Like in our case, GitLab CI has the same functionalities as Jenkins, but as we keep things simple, so it's much better for newcomers to start from this and for everybody in our team because it's like just YAML files. Helm. We decided to use Helm. I just have to say like a few words what is Helm? So, Helm is a deployment tool for automation, creation and configuration, and deployment applications and tools. First of all, it is a package manager for YAM files. So, you can have a Helm chart package as well as you have any DevNF or, for example, YAM package version. With the Helm, you can have your deployment parameterized. Helm has a file with variables that could be set. Also, you can provide variables as a parameter in the command line, like, for example, this Ansible. Also, it has templates, tests, and those tests could be used, for example, for deployment. For example, if you can run Helm tests after the deployment to make sure that everything works as expected. Of course, if you, for example, use GitHub, you can use GitHub actions. Or if you, like, get used to Jenkins, you could use, for example, Jenkins instead of GitHub. And instead of Helm, you can use, I don't know, maybe Ansible or RGACD or Terraform, et cetera. So, it's our choice. And I will explain a bit more about why we chose it. So, continue with Helm. As you see here, we have, like, a simple output of Helm history. And it describes, like, how many times we deployed the application and when it was updated. Status of application, as you see, it's, like, the last revision is deployed on Friday, right? And here I will explain later we can, we have a chart version. As I said, that Helm creates a package and it has own version, like, for example, YAML file. And we can specify application version. And in description, we can see which image used for this and which configuration. So, and it's pretty easy to switch to any or roll out to any Helm revision you have. So, I will explain a bit in detail later about it. So, and we came up with the idea to have, like, to separate all the deployment process into three repositories. So, one repository have source code. The second one is deployment repository, is repository with Helm charts. And second and third one is repository with configuration. So, this repository is just, like, YAML file that takes parameters from code repository and deployment repository, and you can easily deploy the production. Let's take a look, like, inside each of repositories. So, source code repository, the idea is having docker image after you create a merge request. So, once a merge request is created, GitLab CI built an image, we just call it for it and sends it to the QA repository. A docker image has a tag equals to Git commensure. So, for example, for hello world application, we will have, like, we could have possibly this tag. So, and that's it. So, you create a merge request, you have image with specific tag. Here is an example where you can use, for example, QA repository for distribution tool that uses, like, different Git for each images. Deployment repository. It's similar to the source code repository. On a merge request, GitLab is built a Helm package and sends it to the Helm chart repository. Helm chart repository could be an FTP server, for example, where packages are located. And you need to remember that also each time deployment code is updated, Helm package version should be updated also. As you see in this example, it is like a normal FTP server and you have, like, each version of the Helm package here. So, Helm package, this here, it's just YAML files that will be deployed and then applied as, for example, Kubernetes or OpenShift objects. Deployment configuration repository. This is the last step in our chain. So, it has deployment config file and that contains information about the OpenShift and Helm configuration. So, let's take a look deeper. So, this, like, it's part of the file and it looks like this. So, here we have, like, OpenShift configuration. In other case, it could be, like, Kubernetes configuration or et cetera. And here we specify, like, image tag, we specify registry, can we get those image, we can specify, like, projects, namespace, et cetera. And we have Helm configuration. So, in this image, like, other code, like, source code is located, right? And for Helm configuration, we, like, can specify, like, chart name, config commit and chart version. And here, as I said before, we can specify, we can, like, pass values to the Helm through the file. So, for example, we have a production file, that Helm configuration production file, that we can specify, for example, Git repository where code is located, we can specify, I don't know, volumes, secrets, environment variables, et cetera. For example, if you want to deploy your application to test environment, to stage environment, and to production, you can have, like, three variants of files here. And you can, for example, specify one resources, like, to the for test environment, like, for example, Zap will use one gigabyte of RAM. And for the production environment, it could use, like, 10 gigabytes of RAM. So, it's nice to have several files for each configuration that you have. But also, you can use, like, deployment configuration file, like, also several files. But we use one file for configuration and several files for Helm files for deploying up to different, like, with different parameters. So, let's summarize. From the code source code repository, we get a docker image with specific tag. From the deployment repository, we have the Helm package with specific version. All of this, like, files could be specified, like, then it will pass to the config repository. Helm configuration is a file is about, like, configuration of Helm deployment. It could be, like, consist of environment variables, volumes, secrets, et cetera. And once we have, like, everything, we have image, we have Helm package, we have configuration for Helm package. We have, like, one file that, like, not all of the stuff there. And then we can, like, run a CI job, just deploy it to production or to any environment we have. So, as I said before, our deployment process is not about exactly, like, to deploy change, like, deploy changes fast, but it's about stability on the first place. So, if you want to deploy some changes, you should, like, make changes to the source code repository. Then you should, like, merge those changes and, like, get an image tag from it. Then you should go to the deployment repository with, like, and change deployment file with the new image tag, and then merge changes. So, like, from the first point of view, it's, like, a little bit over-engineering because you should, like, change code to times when you make some changes. But then, in a long-term perspective, you can just, you know, it's very easy to see what exactly was, what and when was exactly changed and what exactly is deployed to production. And if, in case of, like, any changes, you can easily, like, roll out or upgrade your deployment. And, like, for other needs, we just change, like, we divided these by layers, you know? Of course, it's possible to keep all of the stuff in one repository and you can, like, have, like, one repository and deploy for it. But if you have, like, a bunch of different applications and tools, it's pretty difficult, for example, to migrate from one cluster to another, like, having everything in one repository. It's very important to divide it by layers. And you can easily, like, for example, change, like, download one repository that consists of deployments, config, for all the tools. So, you can easily, like, deploy it and change it for all the applications. So, there was a point of it. And that's pretty much it. So, I will check for the questions. If we have some, looks like we don't. So, if you have any, please ask. Yes, you can write a chat, you can put it on the Q&A, or you can join with just whatever you like. Yeah. Also, I wanted to mention that, for example, maybe you, like, in your case, you shouldn't use, like, all of the tools or all of the applications that are here. For example, if you want to deploy, like, Jenkins, for example, you don't need code repository at all. You just could have, like, deployment repository and configuration repository for deployment. Yes, if you use some, like, applications, the third party, yeah. Or if you have, like, only the one application, you can, for example, create in Git, like, directory, and inside that directory, you can create subdirectories and keep it in one place, like, with one application. Like, for now, it's better to keep separate deployment, like, code repositories in one place, deployment repositories in one place, and, like, configuration in third place. So you can easily change for all of the tools that you have. For example, if you're going to migrate to another cluster, it's like, you can, it can be done in just, like, a few hours, all of the migration. And having it, like, in code repository, all of the deployment stuff and configuration stuff, it will take, like, much, much more time that, for example, in our case, when we separated it. Yeah, I have a question. I hope I didn't miss it. But so, how did you choose the tools you are going to be using? Why these specific ones? Yeah, so, so we had, like, several options here. And, like, as I said before, like, we use, like, GitLab CI, because it's like, it works just out of the box, and you don't need to tune it. You don't need to manage it. It just works. And, like, it's easy. It's easy for newcomers. And it's easy, like, because it's like documentation, a fit. And about Helm, we had, like, several choices. And we tried, like, different variants. For example, we tried Ansible for deployment. And, like, it counts, actually, that Helm is a little bit complicated. But in general, it's, like, very easy when you start using it. It's really easy to just adopt your files and YAML files to the deployment, to the Helm process. So, for example, if you use Ansible, it's much more, like, it's more difficult. It's more work for debugging, for, like, keeping it together. And what Helm is, it handles all of the resources you deploy. So, for example, if you specify, like, deployment config, like secrets, and, like, some other OpenShift object, or Kubernetes object, it, and you deploy it. And when you delete, for example, the application, it will delete everything that it created before for you. So, when you, for example, using, like, normal commands, like CLI commands for deploy something, it could be, like, created, like, a bunch of resources that you don't even know about it. When, when, then you check it, you should, like, delete it one by one. But using Helm and, like, create update resource with Helm, it manages all of this stuff. And so, if you delete, you should, like, you, you are sure that it's deleted completely from the OpenShift and no garbage at all. Yeah, so it's really cool. And I also want to mention that all the, like, deployment processes are automated. Like, we don't tune it manually. You know, sometimes you want to deploy something and then go to the OpenShift and manually create some stuff there. Don't do this, because you will forget later in few months that you created it and it could influence in on other applications that you have, for example. So, only in automatic way. You changed, for example, one should change something in deployment config. You test it, you change it, and then you just deploy it in automatic way and it works fine. Okay, thank you. Okay, three minutes to go. It doesn't look like we are going to have more questions. So, it's up to you whether you want to hang out here or in the work adventure or whatever. I don't know if people, if somebody has something to ask, please ask. If not, we just can finish here. Yeah. Or maybe we can continue, like you said, in the other room. Maybe people don't want to ask here. Maybe they don't want to be recorded. Who knows. Okay, thank you very much for your speech. And I'll see you around then. Yeah, thank you. Have a nice day. You too. Bye-bye. Yeah, bye.