 everybody. So the session is self-service OpenShift cluster creation. My name is Rastiv Agnar, I'm software engineer at Red Hat. And about the year ago, oh, I turned it off. And about the year ago, we started working on a new operator, which is called cluster as a service operator. It's already available on Operator Hub. Just some first version. In these days, we are trying to release a new one. There is this one line of description, which says that you can easily install fully configured clusters. So how easy did we actually make it? As any other or most of the Kubernetes operators, they bring in some CRDs. These are APIs that you can use to talk to the operator, tell the operator what to do, and then you can get some feedback. So one of such CRDs that our operator brings in is called cluster template instance. And what you need to do is just fill in the name and the namespace. And in the spec part, you need to reference some cluster template. So in this case, the cluster template is called cluster with Kafka, right? So the name can give you some hints that it will install some OpenShift cluster where you already will have a Kafka instance running, configured, and ready to be used. You will submit this cluster template instance, and some magic will happen. And after some time, in the status, you will get all the info that you need to be able to log into your cluster, such as API, URL, admin password, and the Kube config. And you can log into the cluster, do whatever you want. You may break the cluster. It doesn't really matter. And once you are done with it, you will just delete the cluster template instance, and cluster is gone. So that's it, really. That's how you can solve service your own clusters. Do you have any questions? Okay. No, not yet, right? So you are probably interested about the magic part, like what happens after you submit that cluster template instance. So cluster as a service operator is not reinventing the wheel, right? We are using the operators, controllers, and technologies that are already available. They are very, you know, a lot of, they have a lot of users in Kubernetes where they are popular, and they provide all the pieces that you need to solve service, but it's not very tight together, so we kind of try to take them and just give them some order to deliver the solve service experience. And the technologies and operators that we use are Argo CD, Hypershift, and Helm. So Argo CD, right? I guess anybody already knows that, but just to repeat, so Argo CD follows the GitOps pattern of using Git repositories as the source of truth for defining the desired application state, but it can not only be just Git repositories, it can also be a help chart, so you would be using Helm repository for that, so you have that source of truth. The Argo CD, you will tell the Argo CD that this is my repository, please deploy it on some cluster. Argo CD will deploy it, it will start monitoring it and making sure that the deployment is matching the source of truth. And Argo CD is really a center of cluster as a service, because we use it to deploy day one manifests on a hub cluster and day two manifests on the spoke cluster, so hub cluster is like the main cluster where the cluster as a service is running and all other operators at the spoke cluster is the cluster that you are trying to deploy. And what are these manifests? So day one manifests are custom resources of Hype or Hypershift. So these are operators or controllers that enable you to install Opershift, right? And the difference is that Hype installs standalone clusters, so these are clusters where you have three control planes, so you need three machines for that, and then you need some machines for workers to run your workload. And Hypershift also installs Opershift clusters, but the difference is that you don't need those three machines to run the control planes, but instead all those services are run as a port on the hub cluster. And these projects, they have different APIs, and day two manifests, that can be anything, that can be anything that you want to do after the installation. Because when the Hype or Hypershift, when they install the cluster, the cluster is empty, right? There is nothing really going on in that cluster. And as a day two, you would like to make that cluster actually useful, so you maybe want to install some database, maybe configure IDP, install some operators, run some instances, and so on, so really make it ready for maybe a developer to really be able to do what the developer needs to do. Okay, so how the flow looks like, let's try to visualize that. So we have those day one and day two manifests. They live either in some grid repository or in some Helm chart repository. Then we have a hub cluster, and on that hub cluster, the cluster as a service operator is running, of course, and there is also Argo CD running and Hype or Hypershift, or both. And what we do is that, we tell Argo CD, here are my day one manifests, please take them and deploy them on the hub cluster. So Argo CD will do that, Hype or Hypershift, we'll notice those new CRs that could deploy it, and it will create a new cluster, right, that's full cluster, the new one. And after the cluster is installed, we again tell Argo CD, here are my day two manifests, please, again deploy them, but on the new cluster, right, so because Argo CD can manage multiple clusters, not just the cluster on which the Argo CD instance is running on. And suddenly you have a new cluster which is running something and is useful. All right, so demo. So first let's try to create these manifests. So I will close my slides, see my screen. So for the day one, let's create the custom resource which will deploy a Hypershift cluster. I already prepared some skeleton, let's say. So here in the, I have the Dev Conf template, which is a typical Helm chart, we have a chart YAML, which is some metadata about the Helm chart, so we have some name and version. I will bump it right now, otherwise I would forget, we will be doing some changes here. We have values, so these provide default values for the Helm chart parameters, and of course we have a schema. And in the resources that we want to deploy, we have the hosted cluster CR, where everything, almost everything is hard-coded, right, but we would, let's say we would like to enable the user that is trying to solve the cluster, that not only for the user, but also for the user, for 10 version of the OCP will be deployed, but let's say that he can, he will be able to choose his own version. So let's do values, OCP version, so this is a new Helm chart parameter, and also for the node pool, where the version is defined also. All right, node pool specifies how many workers we want to have in the new cluster, and we will have just one worker, which is enough in our case. And the platform that I'll be using is the agent platform. It doesn't really matter what kind of platform you use, we don't have any restrictions, we support basically anything that HyperShift supports, so in this case I'm using agent, which is a platform where you just take some VMs or bare metals, you boot them from the ISO, and you can use those machines as workers. All right, so I made some changes. Let's say that in the values, I would say OCP version, I think before I had something like 4.10.19, so this will be default if the user doesn't provide a custom OCP version, then we will still deploy 4.10. And also we should probably edit to the schema, and we'll say the OCP version is string, even though we could do something more restrictive like enum, but yeah, this is fine for us. Okay, I'll make sure that all my files are saved, and let's package this home chart, so I will do home package, DevConf template, I will update the repository index, and I will commit all my changes to my key repository. All right, and now my repository is set up in a way that on every push it will run an action which will deploy a new home chart repository, right, so it's already running, it will take like a minute or so. In the meantime, we can take a look at the day to manifest. So for day two, let's say we would like to install StreamZ operator, and then deploy some Kafka instance. I just found some example on the web with some default values, so, and this is just, these are just plain old YAMLs, right, this is not a home chart, and this already is on my key repository. All right, so now we have day one and day two manifests. Now let's try to take them and just put them into the cluster template, and I will switch to UI right now because I also want to show this that we actually have a UI for the cluster as a service operator. I'm not sure if you ever saw the UI of the OpenShift, but the cool thing here is that OpenShift supports these dynamic plugins, so your operator can bundle the UI and simply run the HTTP server as a port, expose it as a service, tell the UI that here's my service and it can serve the UI, right, the UI plugin for the OpenShift, and then you can when you go to UI, it will dynamically load all those JavaScript assets, and it will render the UI here. So in this case, right, we are adding the cluster template navigation item, and when you go there, you will see the cluster template. So there is a few of these, like three that are default, but let's create a new one. So let's name our template, let's say the cluster. We can do some descriptions for the users that they will try to solve service and just type something here, right? This is like a markdown template, let's say, new cluster with Kafka. So the installation settings, these are the day one manifest. Let's add our HelmChart repository, and I will pick the DevCon template, and the new version 0010 is already in the repository. And we can say into which namespace we want to deploy all the files from the HelmChart. So let's pick clusters, and this is because I didn't show this before, but if you take a look at this hosted cluster, there's a couple of secrets here, right? So here you have a reference for the pool secret and some SH key, and these are, I don't want to expose this info on my publicly available HelmChart. So I already have these secrets on my app cluster in the cluster's namespace. So when the HelmChart is deployed, it can find them in the correct place. And now for the post installation, we can either choose a HelmChart or a Git, but in this case I will choose a Git repository. So again, let's add a new repository, which is this GitHub IDP, and we need to say from which commit branch or tech we want to deploy stuff, so let's choose the main branch, and directory path will be Kafka. Destination namespace doesn't need to be filled in, because that's already given by the resources here, so the subscription will be deployed in OpenShift operators namespace and Kafka namespace. All right, so these are the template is created, right? You can see that when I show you the YAML, in the spec part we have the cluster definition and cluster setup. And these are actually application sets, so this probably sounds familiar. These are custom resources of Argo, and the spec part contains all the info that we put into the wizard, right? So we have the HelmChart repository, which HelmChart should be deployed, and the version of the HelmChart. And also we have another application set for the day two, which is the Git repository. And it all seems okay to me. So the template is ready, and now let's try to use it as a user. So we don't really have a UI for the user, so let's do this from the command line. I'm already logged in as this user called Rav Agnar, and so I'm not a cluster admin anymore. And I can't really do anything on this cluster. It is a very restricting environment, so if I try to do Git projects, I have just two namespaces available to me. And let's switch to this Rav Agnar namespace, and if I try to do Git ports, I'm forbidden to do anything. I can't do really anything. But what I can do is I can solve service the cluster, which was defined. So what I can do is that I can get cluster templates. I will probably want to explore, let's say, the Defconn cluster, right? So we do Defconn cluster. And what is important here is that in the status, I can see the schema and the values of the Helm chart. And I'm seeing that, yeah, okay. So here you can see, right, the OCP version that we added for the schema. And so let's say I want to use this one. And I can so I need to create the cluster template instance, as I showed in the beginning. And I should already have the YAML almost prepared. So let's edit it a little bit. This is my cluster. And the cluster template that we want to deploy is called Defconn cluster, if I remember correctly. All right. So the CDI is ready. We can submit it. And I can see that I can just let's make sure that we are deploying it to the current workspace. Okay. All right. So the cluster template was submitted. We can export the content. Now we can see the face is installing. Let's take a look at the YAML here. So we can see that here in the status, there is actually a lot more than I showed in the example in the beginning, there are some conditions. And you can actually watch the progress of the cluster creation. So these are basically the phases which follow in the exact same order as are written in the array. So the first thing that is happening is that the application was created, right, the RDoC, the application. And the second condition is that the cluster is installing. And we can see that it's still being installed, right. This will take some time, like, I don't know, for Hypershift, it will take like 10 minutes to deploy the control plane nodes. And then maybe 15 minutes to deploy the agent. Then we have the... Then we are trying to create the managed clusters because we are trying to integrate with MCE. MCE is an operator which stands for multi-cluster engine. And it allows you to manage your clusters like in bulk. And then there is a clusterlet create. Clusterlet, I don't create it, right. So this is this one. This is also related to MCE. And after that's all done, we add the new cluster to the Argo. So Argo can manage the new cluster. And then we run the Day2 manifest. So in this case, it will deploy StreamZ operator and create a Kafka instance. And then we wait for it to succeed. And once all that is succeeded, we will see... We will get the API URL and all the credentials that we need. Not before, right. So the cluster needs to be completely ready. And just after that, you will get the credentials. And we don't want to wait here for that. But I already created another cluster from a different template. But it's basically the same. Yeah, I guess it's exactly the same. So it already deployed the cluster where Kafka is running. And what I can do is just take the credentials, right, I can log in. And I would do, see, get Kafka. And the Kafka is running there. And I can do anything I want to do on the cluster. And once I'm done, I would delete it. Yeah, okay. But that's not all. There is one important part missing. And that's when users get access to this cluster template instance, CRs, they can create an infinite amount of clusters, right, that's going to cost a lot of money. And you still want to somehow restrict them. So we have another custom resource, which is called cluster template quota. This is very similar to building Kubernetes quotas. It just, this one focuses on the cluster templates use case. And in this quota resource, you can restrict basically two things. And one is which templates the user can create, right? So which, which, which templates the user can reference in the cluster template instance. And also how many instances the user can create. And how many instances that can be done in two ways. So you either just use plain old like number, right? So we will say I have a template A and template B. And template A, I will say there can be two instances of template A and like three instances of template B. But that's not really very flexible at all times. So we have this other kind of more abstract concept where you can, where you can give every, every template can have some kind of associated cost. And you can think of it like, let's say, how much does it take to run such a cluster for a week, right? So let's say template A would be like $100 per month or per week, right? And template B would be like $500. And then in the quota you can put the budget. And the budget will be 500, right? So in that case, when user is trying to create clusters, it can either create five instances of template A or one instance of template B, right? Or some kind of a combination like that. Yeah. So maybe I can also show you how such a cluster template quota looks like. So in the UI, again, you can go to some cluster. And in the quotas, I didn't set up any yet. So I will create a new one, for example. And I will say that this quota is for the Ravagnar namespace. And I will allow creating only the DevCom cluster. And I will say that only one cluster from this template can exist. So we have a template. And if a user now would try to create another one. So let's do, I can also create. And it will say, oh, I'm logged into the Spoke cluster. So let's log in back. Okay. Let's switch to the project. And let's create the CTI again. And I'm denied because there is a webhook which checks whether I still have, I'm still within the limits that were set to me by the admin. Okay. Yeah. So that's all I have. There is a repository which you can go to, read the documents, write, try it out. Okay. Thank you. Do you have any questions? Yes? From Red Hat ACM. Okay. Well, this is actually, maybe at some point, will be part of the ACM. ACM doesn't really solve the self-service use case because ACM is really targeting for the admin users where they have like a fleet of clusters that they can manage. But you don't really give access to ACM to like regular users, right? Because they would take a lot of stuff that you don't want them to do. Yeah. This is like for admins to provide the self-service experience for the users. Yeah. But yeah. Like the users can be developers. Sure. Like it can be a team. It can be a single person or some kind of customer that just needs to deploy a cluster. Yes? Uh-huh. Okay. Oh. Sure. So you are asking if we can make a parameter not only the version, but like the whole URL. Yes. Of course. Yeah. You can do that. This is like a, you know, it's just a helm chart. It's nothing special about it. So you would, yeah, this is the DevConf template. And instead of the OCP version here, right, you would just make the whole thing a parameter like this, right? And yeah. That's really up to you. It depends on how much flexibility you want to give users to be able to modify the template. Any other question? And maybe one more thing that I forgot to do is that, so we, before, right, we made the OCP version a parameter. So in order to actually use that parameter, you can do, um, you would, in the spec, you would do parameters and then this is an array. So you would do OCP version like name is OCP version value, something like 41221, right? And then it would deploy the version that you want. Okay. So that's all. Thank you.