 Hi, everyone. Welcome to today's webinar. My name is Jia Chengxu, and I'm a software engineer at QuboMatic. Today I'll be talking about an open source project called Qub carrier and how you can manage thousands of QuboMatic's applications by using QuboMatic. Let me briefly introduce myself. I'm a software engineer at QuboMatic, mostly focusing on service and application management on QuboMatic Kubernetes platform, and I'm one of the core maintainers of Qub carrier project. Before we dive into Qub carrier, let's talk about Kubernetes operators. Operators are software extensions of Kubernetes, which make use of customer resources to manage application lifecycle, including deployment upgrades and backups. Operators are working nicely in single cluster scenario. However, in the multi cluster scenario, imagine you have hundreds of clusters or thousands of clusters. Managing application resources in every single cluster can be complicated, and the process needs to be automated. Relying on a single operator does not scale very well in those complicated scenarios. That's why we built Qub carrier, an open source platform for managing applications and services across multiple Kubernetes clusters, and it is built in a generic way to work with any Kubernetes operators. Also, it provides a centralized service hub to expose your services and applications to users. Here is a high level overview of Qub carrier. In Qub carrier, we have a management cluster, which has Qub carrier installed and running. A service provider can register service clusters to Qub carrier, and for each service cluster, they will deploy the application operator. Once the service user come to Qub carrier, the service user will only interact with the management cluster and provide instructions to select and manage services. And the real service instances and application workloads will be deployed on service cluster by the application operators. Then, let's talk about Qub carrier in more details. So as I mentioned in previous slides, there are two different types of clusters in Qub carrier. One is a management cluster, we call it a service hub. Another one type is service cluster, which will be registered by service providers and run application workloads and service instances. Also, as I mentioned, Qub carrier works with Kubernetes operators, which means the application operators will be running service clusters. And the Qub carrier has a discover mechanism to discover CRDs of those operators and make some available for user in the centralized service hub. Then Qub carrier also propagates the customer resources from the centralized service hub to service clusters to drive the operators to create application workloads and the service instances. Also, Qub carrier has multi-tenancy support, which means for Qub carrier installation, it supports multiple service providers and multiple service consumers. This picture shows an overview of multi-tenancy in Qub carrier. So in Qub carrier, there are three personas. One is a platform operator, which will operate the entire management cluster and manages Qub carrier installation and Qub carrier accounts. And another one is a service provider, which will manage service clusters, for example, register them to Qub carrier management cluster. And also, a service provider will manage operators and the service instances, which are running in the service cluster. Then we have a service consumer who will interact with Qub carrier management cluster and request and manage the services. And for each account in Qub carrier, it has a dedicated namespace to isolate the resources from each other. And the Qub carrier will create RBAC permissions automatically for each account to achieve multi-tenants. So those are basic concepts and the high level overview of Qub carrier architecture. Now let's move to the demo session. In the demo, I will show you how to manage radius database as a service across multiple clusters by using Qub carrier. So firstly, let me present today's demo setup. I'm using Kubernetes platform to create clusters in different providers. As I mentioned in previous slides, Qub carrier work with any Kubernetes operators, any Kubernetes distributions. So in today's demo, we have four clusters across four providers. And we have a management cluster, which will have Qub carrier up and running and we will have another three service clusters, which will be running the real radius database instances. Okay, let me share my terminal with you. So as we mentioned, we have four clusters and you can see this is the management cluster and the rest of the clusters are for the service cluster 1, 2, 3. And the first thing I would like to show you is about the accounts that we mentioned in Qub carrier. So here is an example of account. You can see this account is for the service provider. And if you look at the rules field. And this indicates the rule of this account in Qub carrier. For example, it can be provider, which means this is a service provider. And it can also be tenant, which means it's a service consumer. Also, it can be provider and the tenant at the same time, which means you can provide your services to other users. And at the same time, you can also consume services from other providers. And also there is a subject field is account definition. This is similar to the subject in the cluster row binding and row binding of Kubernetes object. And Qub carrier will create RBAC rules and row bindings for the users here to make sure that those users can have access to this account. So this is, as I mentioned, this is an example of the provider. And let me show you some examples of the tenant. So here are examples of tenants account. And you can see the rule is tenant. And currently in our system, we have already four accounts created. Let me go through them one by one with you. So the first one is a radius provider, which will be providing radius database services in our demo. And there's another three accounts are tenants. They will be consuming the radius services from the provider. Okay. And the next thing I would like to show you is about the service cluster. As you can see, we have currently we have three service clusters. And there are two of them, which are already registered to Qub carrier. You can see the service cluster one and two are already registered in Qub carrier. And we will use the service cluster three later in the demo. And you can also see the status are ready. And also the Kubernetes version of those service clusters. So, so now we have registered the service cluster to Qub carrier. And we have already deployed the radius operator on each of them. The next thing is to let Qub carrier to discover the CRDs of radius operator in the service cluster. How can we do that? We can do that by creating a catalog entry set. So here is an example of catalog entry set. And if you look into the discover configuration and there is a CRD field, which you can specify the CRD name that you would like to let Qub carrier to discover from the service cluster. In my case, it's a radius CRD. And also in the expose config, you can specify which fields of the CRD you would like to expose to user. So in this case, we specify some of the fields, and we want to expose them to the end user. And also we point to the CRD in the catalog entry set. So the catalog entry set will propagate the CRD with this name from service cluster to our management cluster. Okay, let's try to create this in our management cluster as a member of provider member. Okay, after this catalog entry set is created, next step would be create a catalog. So here is an example of the catalog. So for the catalog, there are two selectors in the spec. One is for selecting tenants. In this case, we selected the tenant by using label selectors. For example, the tenants with the label service with value radius will be selected. I have already pre configured our tenants A and tenant B by adding this label. So only tenant A and tenant B will see the services from this catalog. And also there is a selector for catalog entry. And in this case, for a simple demonstration, we will select all the catalog entries. Okay, so it's time to create the catalog. And after the catalog is created, we can wait until Kube Carrier to discover the CRDs from service clusters and make them available for the users. So for checking if the service are available for users, we can check the offerings objects from the tenants perspective. So you can see there are two offerings object that are available for the user. And those offerings are explained by the names. For example, for the first one, it's a radius service from service cluster one from a provider radius provider. Same thing for the second one. But the second one is from the service cluster two, but also provided by the radius provider. Okay, so as I already mentioned, we only expose the service to tenant A and the tenant B, but not tenancy. So we can verify that by checking the offerings object in every tenant namespace. So you can see for tenant A and tenant B, they have two available offerings. But for tenancy, there is no offerings in tenancy. This is because our catalog object that we created before doesn't select tenancy. So tenancy cannot see the radius services. Okay, so it's time to create a service for the tenant for the radius service. So here is an example of the radius service that we can create for tenant A and tenant B. So it's like a customer resource called radius. And here we specify some fields that are exposed to user. And then we can just create the radius customer resource as a member of tenant A. And then you can see the real workload is actually created in the service cluster. Not in the management cluster. And we can create another service for the tenant A, but in service cluster two. You can see there is another radius instance, which is created for tenant A in service cluster two. And another thing I would like to mention is that all the service instances in the service cluster will be isolated for every tenant. For example, if I create another instance for tenant B in service cluster two. And you can see there is another radius instance is created only for time B. And this is in different namespace with the one that created for tenant A. So the workload are completely isolated on service cluster. Okay, now let's make use of the service cluster three. So what I would like to show you is about how to register the service cluster to Kube Carrier. And Kube Carrier will automatically discover the CRDs and the services from the new joined service cluster. So let me show you an example of service cluster object. So in the service cluster object, there is a Kube config secret, which is pointing to the Kube config of this service cluster. And we can create this service cluster by applying this file. And after that, we can check the service clusters in the management cluster and wait until the service cluster is ready. Okay, so now the new service cluster is ready, which is service cluster three. And since for the service cluster three, we have already deployed the radius operator. So our catalog will automatically discover the radius service from services three and make it available for both tenant A and tenant B. Let's try to get the offering object for tenant A. And you can see there's a new created offer offering, which is a radius from service cluster three from a provider radius provider. And then you can just use this service and use this service offerings and create the radius instance in service cluster three. So you can see the radius instance is created in service cluster three for tenant A. So it is quite easy in Kube Carrier to register a service cluster and make the services available to the end user by the Kube Carrier service hub. Okay, this is all about today's demo. If you have any questions, you can send me an email or you can create issues on our project page on GitHub. Or if you are interested in more advanced topics about Kube Carrier, you can check out our documentation. Thanks for your joining and attention.