 Welcome you all to the session. In this session, we will see what is virtual Kubernetes cluster and what it offers. Myself Nathan and I work as principal software engineer at Ericsson predominantly into verification of machine learning operations and a test automation. The agenda of the session goes like this that starts with the background, then the overview of virtual Kubernetes cluster with a recorded demo to see how it works and finally it's various use cases where it could be used. Let's start with the origin story. The origin story is based on the challenge that we faced in our organization. It may be applicable for others too. As we all know in any cloud native product development team, right? Test environment that is Kubernetes cluster plays a major role in our day to day task like during study phase to do proof of concept, during development phase for individual developers to test the code changes, during testing phase to certify the release of the product especially in CACD pipeline, right? So now let's see these challenges one by one. First, let's see the challenge that a team faces for a development environment. Multitenancy is really tough with Kubernetes and there are pretty much two model which the people use to share a cluster across individual teams, right? The one model is that there is a shared cluster for everybody with a namespace isolation. So an individual team or sorry, so an individual or a team gets their own namespace and that's what they have access to, right? But the challenge here would be the teams might be developing a global objects like custom resource definitions that is CRDs and we all know that they cannot manage that in a namespace isolated cluster, right? So the only practical solution here is used to dedicated Kubernetes clusters to guarantee true separation which is known as cluster-based isolation where every individual or team gets a dedicated Kubernetes cluster and you know, even this is problematic for a lot of reason. It is an edit to the management due to the cost and the complexity associated with scaling the Kubernetes cluster as the team grows with respect to the complexity, right? Most important part is maintaining the clusters on the longer run by upgrading and performing other maintenance activities. Finally, it brings in environmental impact means an individual user may not use the full capacity of the resources in the Kubernetes cluster and most of the time, right, it will be underutilized. And as we could see in the comparison, the isolation is not that great in namespace whereas it is true in separate cluster but at the same time it increases the cost and brings in other complexities that we discussed, right? So what if instead of namespace and cluster isolation, an individual team member is given a dedicated cluster without all these challenges? Now, we'll answer this question in a while as we move forward in the session. The next challenge that we face is freedom of choice. Usually in a company, we'll have a central admin team who will take care or run the Kubernetes cluster and they do sort of an IT overload, you know, telling the team that no, no, we are running this version of service mesh and that's all we run in the Kubernetes cluster and this is how we set it up and everybody must obey by that and they don't want to run thousand different versions of Kubernetes cluster design, okay? So however, we can't blame them for it but practically their approaches are in the way of progress or flexibility for developers. Now, what if a developer is given a dedicated sandbox environment where they have more freedom of choice and still it is managed by central admin team? And we will, we will also answer this question in a while as we move forward in the session. The next challenge that we faced is less, less flexibility in CACD. In a conventional CACD pipeline system under test, we may need to have a Kubernetes cluster installed with the respective version every time manually or via automation, where the setup time of Kubernetes cluster will be like 10 to 20 minutes, right? Then the system test would start which gradually increases the overall feedback loop time of your CACD pipeline. Also, let's say we want to test a product in multiple Kubernetes versions. It is going to take a lot of time to test all these probabilities and combination because setup time of Kubernetes cluster going to take time. So what if there is a flexibility to have all these covered in the CACD pipeline with the quick feedback loop time? Then it is more advantages for any product development team, right? And more importantly, you know, it brings in more confidence before they ship the software package to the end users, right? Just like any organization, we also faced all these challenges and this triggered a study to explore technologies that could help us to use the development and test environment efficiently using cluster virtualization technique and reduce the cost and provide more freedom of choice to the developers. While exploring, we came across an interesting open source project called VCluster that looks promising to address the challenges that we discussed so far. Now the next question would be what is VCluster? On a funny note, going forward this session will be like the inception movie in a very simple term virtual cluster or in short VCluster or lightweight Kubernetes cluster that runs on top of another Kubernetes cluster. Now, VCluster is an open source project. Initially, VCluster is built in a commercial product by Loft Labs and going forward, I'll be using the term VCluster in place of virtual Kubernetes cluster. More specifically, VClusters are just a control plane that runs inside another Kubernetes cluster. What is a control plane? As we know, a regular Kubernetes cluster will have an AP server. Then we have a data store like HCD, standard backend for Kubernetes where the cluster configuration and state data such as number of parts and their state or store. Then we have a controller manager that controls the deployment, replica sets, et cetera. Also, we have a scheduler that actually takes the parts and assign them to the different nodes in the cluster where the container gets started. Then we have the workloads on top of this control plane. Typically, we'll create a namespace to essentially run parts in them. Typically, we'll have multiple namespaces, right? And that's how a regular Kubernetes cluster looks like. Then for anything, we always talk to this AP server, which is the gateway to the Kubernetes cluster. And in a regular Kubernetes cluster, we have the capability to spin up workloads inside the namespaces. And so why not a control plane inside a namespace? And there comes the v cluster, which is designed in such a way that a control plane runs as a pod inside a namespace of the underlying Kubernetes cluster. Going forward, we'll call this underlying Kubernetes cluster as host cluster. That means we have an AP server and couple of other components, which we'll talk about in a while, and which this AP server will be the gateway to the virtual cluster. And so we'll not talk to the host cluster's AP server anymore, and we'll talk to the v cluster's AP server directly. So in a nutshell, what they do in v cluster is that they give a way to share a host cluster, but at the same time, it gives the people an experience that feels to them like that they have their own dedicated Kubernetes cluster. I know this sounds like an overwhelming topic. And so let us experience the same with a quick start recorded demo. In the interest of time, right? So we have already prerecorded the demo. Now I'll walk through this prerecorded demo. Now I have already connected to the host cluster. And I have listed all the namespaces in the host cluster. Now, let's create a namespace called host namespace in the underlying host cluster, which is supposed to host our virtual cluster. Okay, now what about the permissions in the host cluster to run v cluster? Actually, we don't need any cluster wide permission, only the permissions to create a stateful set and a service in this namespace is required, right? When we have that permission, right, we can create a v cluster and inside the v cluster will will be the cluster admin now. Now let's go ahead and create a v cluster called VC one in the host namespace and there are multiple ways we can create a v cluster either through v cluster CLI or Helm or just a kubectl apply I have used v cluster CLI which is a very lightweight binary from v cluster project. And under the hood as we see here, it exhibits a Helm command that pulls the chart and deploy it to the cluster. Now let's wait for the command to complete. Now we list to the v cluster. There we go. Now we'll see what is inside this host namespace. We could see a stateful set and it's associated service and parts and the pod as we see here it has two containers running in it. Now let's wait for this host cluster to come up. Sorry, the v cluster to come up now as we can see the v cluster is up and running inside the host namespace. Now let's access the v cluster and again there are multiple ways we can access the v cluster. Here I'll be running a simple connect command using v cluster CLI. What it does is it does a kubectl port forward to the v clusters AP server. Now I'll split the terminal here. The v cluster CLI command creates an cube config in the current working directory. As you can see here, now let export it to the environmental variable called a cube config so that all my kubectl commands now points to the v clusters AP server and not to the host clusters AP server. So any kubectl that I'm going to execute now will run inside the v cluster. Now let's list all the namespaces inside the v cluster. As we can see here it is different from what it is there in the host cluster. Now let's get the pod in the cube system namespace of the v cluster. As we can see here there is one pod running called to core DNS. We'll talk about this core DNS in the later part of the session. Now we are admin in this v cluster. Now let's create a namespace inside the v cluster called NS1. Now let's get the namespace there we go it got created. Now let's try to schedule some workloads inside the v cluster by creating a deployment called a sample inside the namespace NS1 with the name engine X with replicas to let's wait on until this two pods are up and running. There we go. The pods are up and running now wait we saw in the previous slide v cluster had only control plane right remember well then where these parts are running that's the question we'll answer in a while. And finally, as the v cluster is set up and workloads are running, let also look how to wipe the v cluster. It can be done using a v cluster CLI or using helm or using cube CTL delete namespace. If you use v cluster CLI as used here it also does the helms delete in the back end. So that's it. The v cluster is deleted in a matter of seconds. Let's wait until this the host namespace also deleted as we want to the v cluster delete command to delete the namespace as well, right? And there we go. Yeah, so so the host namespace is also removed, right? So now let's go through a few slides to understand what happened in the demo from v clusters architectural perspective. Now I'm going to walk through command by command that we did in the demo to show the inside perspective of v cluster. Okay, now what did we do first? We took a regular Kubernetes cluster that is host cluster, right? Then what did we do? We created a namespace called host namespace on top of host cluster, right? Then using v cluster CLI, we created a v cluster called VC one inside the namespace host in in space, right? Then we also saw the components running inside the host namespace, right? So the v cluster has namely two components. It has a stateful set that we call as VC one is the v cluster that we created and it's as an associated service, right? And this stateful set essentially creates a pod with a single replica in this case and inside this pod is where our v cluster really leaves now. And we have two containers in v cluster pod. The first one is a control plane. This is where we can pick our own supported Kubernetes distributions like k3s, which is the default one and other distributions as listed here. It has an AP server. I mean the control plane as an AP server for any kind of kubectl interaction with v cluster and it has a separate controller manager. Then the controller manager is hooked up to a data store by default. It is SQLite and it supports more than like it's CD, etc. That means when we write something to this AP server, right inside the v cluster, it writes into separate data store and this data store is mounted as a volume to this v cluster pod. Then we have a second container which is sinker and we'll talk about the sinker in a while where our answer to the question where the pods are running resides. Then we connected to the v cluster using v cluster connect command, right? So what is done in v cluster is that we don't talk to the AP server of the underlying host cluster and run kubectl commands against it. Instead, we spun up a control plane itself that runs as a pod inside a namespace. So now we talk to this AP server, which is the core idea of v cluster. Now as we connected to the v cluster, let's get rid of the underlying host cluster and just focus on the v cluster alone. Inside the v cluster, we ran a command to create a namespace, right? Now namespace got created and it creates an entry in the v clusters data store. And as we already saw, v cluster uses SQLite by default, which is a lightweight data store. And by default v cluster uses k3s AP server, which supports SQLite as a storage backend. Instead of SQLite, we could also use our own SQL database or we could deploy a fully fledged hcd in a separate pods. And natural there are multiple ways to hook the data store. What did we do next? We created a deployment called sample with the image engine x with replicas to write it's pinned deployment. And now we have another entry in the data store about this deployment. Now this control manager inside the control plane, watch is the new entry in the data store, and it creates two pods. It means it writes two more objects into the data store. Now we have four objects in v clusters data store, right? We are namespace deployment and two parts owned by this deployment. Now these two parts are waiting to be scheduled. Now whose job is to schedule is a job of scheduler, right? To schedule these parts to the nodes. But as we can see here, there is no scheduler in virtual cluster, right? So the big difference between the regular Kubernetes clusters control plane and v clusters control plane is that there is no scheduler. Instead, there is something called a sinker. We cluster does not use scheduler from its distributions and it only uses AP server and controller manager and sinker is what really makes the v cluster really virtual at the sinker. What it does is it watches the v clusters data store and copies the parts down to the underlying host namespace that talks to the host clusters AP server. So now the host cluster scheduler, right? That will schedule these parts and the networking policies, admission controller, etc. in the host cluster will also apply to these parts. But these restrictions are only to the part level because all the higher level objects you know that the things that that are creating parts like deployment, stateful set, all these kind of things stays inside entirely virtual inside the v clusters data store and they will not be synchronized down to the host cluster. Only the parts are synchronized. And obviously we can create multiple namespaces right in v cluster and creates same deployments. But now how do we handle the naming conflicts? Since all these parts are scheduled in the underlying host cluster, what sinker does is when it sinks from multiple namespaces down to single namespace in the host cluster, it renames the pod and adds two suffix that is virtual namespace and virtual cluster name in which this workload is running. Without the namespace, there would be conflict, right? Let's say you have two parts of the same name into virtual namespaces as here, then we are mapping these parts into single namespace in the underlying host cluster and that wouldn't work right because this name would get conflict in the underlying namespace. So the namespace in which this workload is running is suffix. Also, we can have multiple v cluster inside the same namespace. So to avoid a conflict, v cluster name is also added. So that's what we saw in the demo. Now let's see how different Kubernetes resources are handled inside v cluster. All the higher level objects, you know, the things that are creating parts like deployment, stateful set, stays inside the control plane of the v cluster. Even the CRD states inside the v cluster and this is what makes the v cluster really virtual and gives the feel of running our own separate Kubernetes cluster. And these higher level Kubernetes objects, right, never reaches the AP server of the host cluster. So the basic design here is to reduce the request on the host cluster. Now coming to the lower level objects, sinker plays a key role in syncing them to the host cluster and sinker sings only few things, mainly the parts and other stuff that parts need to start. So the basic design here is to avoid a performance degradation like workloads running inside v cluster, right, will run or should run with the same performance as workloads which are running directly on the underlying host cluster. Okay, now let's shift our focus to the backbone of any system that is networking and understand how it is handled in v cluster. As we saw, sinker component in the v cluster sings the parts to the underlying host cluster, right? And so regular part to part communication works out of the box just like a regular networking and can communicate with each other via IP based networking. Sinker also synchronizes services, it also wire them to the right parts. What it rewires is that as we saw initially, there were only one part running inside the cube system namespace of v cluster, right? That is core DNS. So by default, v cluster has a core DNS deployment in its cube system namespace itself. This means when a sinker synchronizes the part down to the host cluster, it updates the part spec and point it to the v cluster DNS service instead of host clusters cluster by DNS service. And then the name resolution works as normal, right? So cluster internal DNS just works as expected inside the v cluster that that also means that you cannot directly access access the host services inside the v cluster, which is kind of a security that comes in built in as part of v cluster, okay? Now, what is the meaning of nodes from v cluster perspective? In case of nodes, we need to understand the sinker swings in both direction, right? So when a sinker sinks a part down and that part being scheduled now by the host cluster scheduler to some worker node, but there could be many things can go wrong, right? We could get an image pull error or image pullback of error. And so what v cluster does is it also synchronizes the status up from the host cluster to the v cluster. And part of the status is on which node this part is running so that the users in the v cluster can monitor the parts or see whether the parts is scheduled successfully or not. And the default option in v cluster is that it shows inside the v cluster only those parts that nodes that our parts are running. As you can see here, the host cluster has three worker node, but it shows only two worker nodes because our workloads are running only on these two nodes. And also we can see here, it doesn't show the real node details here. Actually, it shows all the fake node details, which means the v cluster when it copy the nodes from host cluster to v cluster, right? It renames them and obfuscates certain things. Say if a user doesn't want to have the fake nodes, they want to see the real nodes, right? Still they have option to install v cluster with certain configuration so that they can see these real node information in the v cluster itself. Now let's see the comparison between namespace cluster and v cluster. So giving each team a cluster is going to be expensive and giving them namespace might be too restrictive. How about something in between? Maybe creating v cluster is the way to go, right? Hope you'll agree based on what we saw so far with respect to v cluster. Now let's see the use cases where v cluster could be used. Lots of people uses v cluster for more ephemeral activities, one for development environment, where developers will have full freedom of choice. Example, if a developer wants to run service machine, a particular version, it is completely up to him because it is going to be only scoped to your namespace and it is not going to impact other people and give them sort of their own little sandbox, right? And next v cluster could be used for installing testing applications in a CI CD pipeline. As we witnessed using v cluster, a new Kubernetes cluster can be spun up in few seconds, right? And we can pick and choose the Kubernetes versions on the fly while we spin up v cluster and this gives us a lot of flexibility to test the product in multiple Kubernetes versions. And also it helps us to test the product that has different variants of installation like optional components or without optional components. And all these probabilities and combination can be done in parallel using v cluster in the CI CD pipeline itself that will save a lot of time and effort and brings in more confidence with the product development team when they ship the product to the end users, right? Now, as a user, if you are interested to explore more on v cluster, you can refer the v cluster website and their documentation that has more details on the architecture and the answer and part and say if you want to raise any issues or if you would like to contribute, we can always do that in their GitHub page. And if you need any clarification, we can always post that in their Slack channel. So this is with respect to the documentation support of v cluster. And that is the end of the presentation. Hope you have understood the basics of v cluster and how it works. I thank you all for attending this session. Thanks.