 All right. Hello, everyone. Welcome to SIG Cluster Life Cycle Introduction. Yeah, it's not like many people yet. It's also fun to be, like, first maintainer track of the day and then the beginning of the conference. But before we talk about our cluster life cycle, let's say a couple of words about, like, who we are and why we are here. Hi. This is the second time we talk about SIG Cluster Life Cycle in Kuomintka, China. And I'm Dixie from China. Of course, Chinese location, right? I work from, on financial, I am a top 50 code contributor in the whole community. And my GitHub handle is dcdx, so if you got any questions or problems, don't get high tech to contact me. We have our Slack. And I'm Alexander Kanievsky. I work for Intel as a cloud software architect. I'm based in Finland. My GitHub handle is CAD. So again, also, if you have questions, please contact. I'm one of the reviewers of QBADM, which is part of SIG Cluster Life Cycle. And I have also interest in several other areas like node and few hours. So SIG Cluster Life Cycle. So who we are and what actually we are doing. The SIG Cluster Life Cycle is one of the biggest, if not the biggest, SIG of the Kubernetes community. Like we have more than 600 people on the main list. We have more than 2,000 people on the main Slack channel, SIG Cluster Life Cycle. And if you look at all our Slack channels, like QBADM and ours, you will see a lot more people than that. We have incredible amount of companies. Like it's literally dozens of companies who contribute. We have contributors all around the world. Probably we don't have anyone from Antarctica yet, but who knows maybe. We have quite significant amount of commit counts or pull requests per cycle. So it's like in order of thousands for our projects. And SIG Cluster Life Cycle is not like just one project. We have a collection. It's an umbrella for like more than 15 projects in a world with smaller tools. So what we do, let me quote our mission directly. So the SIG Cluster Life Cycle objective is to simplify creation, configuration upgrade, downgrade and tear down of clusters. And it sounds simple, like creating the cluster or create, but actually every such of this operation is quite loaded term. Like what we do on each step, how we do it properly, how we do it user-friendly. And that's the reason why we exist, why we are making our tools. That was our concrete statement, which actually has many pieces into it. So we are doing this since because we don't want to end it up in a situation that the only way you don't go net is it's too complex. So you might spend too much time to install a whole cluster or spend $1,000 to some company to help you bootstrap a cluster. So we want to try to reduce the complexity. Actually we have made up several projects targeting different tasks like kubi-strap, cops, and also other, and kubi-nami, that or that. That's our reason. And actually the kubi-nami try to simplify the bootstrapping steps that you try to bootstrap a cluster. And try to intend it to be a built-in block that other tools can rely or reuse some modules that kubi-nami use to help you bootstrap your own cluster providers. Actually many companies are using kubi-nami to develop on kubi-nami to develop their own bootstrap tools. So to look in a bit graphical way, what we actually do. So with kubi-dm, one of our major and one of our first building blocks is a cluster lifecycle created. So it has a scope. Like as we are doing the unique style principle of defining the tools is like do one tool, make this tool to do one thing, one thing which is good, and be a built-in block to a bigger solution. So kubi-dm is used by other components of our projects. And then as soon as we start to evolve kubi-dm we found out our pieces what we consider to be like reusable blocks across different cluster deployment and configuration solutions. So that way we end up of creating the ATC, ATC-D, ADM like the tool which is specifically like special operator which helps to deploy the high-available ATC-D cluster for Kubernetes deployments. And then like cluster add-ons. We have inside kubi-dm two main add-ons what we need to have for have a conformant cluster like the DNS and kubi-proxy. But we might want to have a lot more additional cluster add-ons and we need to first define like what is actually cluster add-ons and what is the application and where is the border between those. And when we start to configure the cluster through kubi-dm we realize that what we need to have a common configuration mechanism. So that's how this component config working group organized and how we started to improve other components besides our scope. And when cluster API, again, it's a piece of building blocks that operate like what cloud providers or different vendor companies can create a specific implementation for way hardware, for way infrastructure. But in all and all, it's a common goal to have a unified API level how to manage the clusters towards like high-level end-to-end solutions. And if you look at this, this is where our seed cluster lifecycle is like. Several projects heading towards one goal. Yeah, actually mini-kubi is the first cluster tools that I use when I get to Kubernetes. And mini-kubi will try to bootstrap several virtual machines on your laptop or servers and try to install all the packages to the virtual machines. So you will have a standalone Kubernetes cluster in your laptop. Actually, mini-kubi is not working very well in China since it will pull all the registry from the GCRDIO. It's not working in China. And the Kubernetes RAID is kind of Ansible Playbox. It will contain all the rows to bootstrap all the cluster. So you can reuse all the rows of the Playbox in the Kubernetes RAID to customize your own organizations. All right, but let's talk about a few. So we're not going to talk about all the 17 projects we have. We don't have time for that unfortunately. And again, unfortunately, we don't have time for deep dives during this KubeCon. But let's talk about a few of our key sub-projects. Right. The first thing is kubi.dm. Everybody knows kubi.dm.1.2.g in since 1.13, right? It is the best practice cluster to help you end up on many more conformist Kubernetes cluster. Well, sorry. Well, the user experience is very simple, and the cluster is really the most secure. As everybody knows, security is a very tough knot. I cannot tell lie, right? So actually we are going to improve the security in the next few weeks. So if you are getting interested, please get involved. And we welcome all your contributions and environments. Actually, the kubi.dm. scope is very limited. It tends to be a built-in block. So if you are trying to... The kubi.dm. will only try to connect to the APS server once it's up. And in order to get to the APS server, we do write lots of certificates and talk to the local folder. Actually, kubi.dm. won't randomly access you into your machines and do something else. So it is very safe to use kubi.dm. in your production or in your cloud environment. Actually, kubi.dm. doesn't care about how kubi.dm. is running. Actually, you can run your kubi.dm. in content-wise or you can buy something. After the kubi.dm. is there, the next thing you do is to install the kubi.dm. by your RAM repo or your apt-get. And kubi.dm. will do all the rest of things for you. And the control plan is on your node and the APS server, etc. is out there. So the next thing is to draw kubi.dm. on the other node and to draw the node to the whole cluster. Actually, the thing is not in the scope of kubi.dm. because it is released with kubi. Actually, when you try to install kubi.dm. on your machine, the package manager will automatically install kubi.dm. in your system. Actually, all the components, all the facets in kubi.dm. is composable. That means you don't need to run kubi.dm. to do all the things for you. You can just use one or two facets like you can just use kubi.dm. to generate all your certificates for you and then you'll use other tools to help bootstrap your cluster. And all these things will split into smaller building blocks. So can you reuse in other shell scripts? Is it okay? Or your programming code is already preferred? Yeah, so again, to show it's a bit more in visual and graphical way. So if you look at the bottom part, kubi.dm. doesn't really care how you provide your hardware. It can be any of the public clouds. It can be a bunch of your servers in your basement. We don't really know and we don't really care how you provide us with hardware. What we have is as soon as your OS is installed, as soon as the kubi.dm. is running, you run the kubi.dm. and kubi.dm. will provision you control components and it will be able to join the worker node to your control plane. And then on the upper part, all kind of your addons, all kind of your additional preferences like dashboards and so on, it's out of the scope of kubi.dm. So one tool does a specific task, does a new specific area. And this is the key difference, for example, for the tools like the cops or ours. So those tools are covering the whole area. So we're taking care of, for example, creation with virtual machines on the Amazon cloud when running with kubi.dm to provision something, and when configurable things like load balancers and et cetera, et cetera, for your cluster. So here's the key differences between the tool concentrated on one task and end-to-end solution. We released 1.13, the G-release. And the general feedback with God is the usability is very good, pretty good. People like to use kubi.dm. to help you push up their clusters. And the common feedback we get is the high reliability. So this is the lack of the kubi.dm. in doing. So in the next two releases, we just improve the high reliability. So speaking about the last releases. So 1.15 got released just a couple of days ago. And if you're talking about kubi.dm, we have several key changes what came to this release. First of all is configuration API for kubi.dm get to a better tool level. Like we released a better one, I think, in 1.13, if I remember correctly. And after those couple of releases, where people already started to understand what our fields are missing, what our fields are not really useful. So based on that, we improved the API. Next thing, what also, again, was based on the feedback from the users was about the certificate management. So now the kubi.dm, during the upgrades, it tries to check if your certificates are going to expire and if we're very close to expire expiration date. We got renewed it for you. So you have a similar usage of your cluster without interruption. Testing is one of the key things was also for this release. So we have now an entirely new test suite to actually do a proper end-to-end testing of installation upgrades in our CI. And last but obviously not least is, as D mentioned, a high availability. It drastically improved in the last releases. Like Fabrizio working very hard and our contributors working very hard on that. We have a good documentation about it. Lukas recently did a screencast, so look at the screencast again. Sorry, we don't have time to do it today, but it takes about like 10 minutes. But it shows you all the steps how you can create the first node, first master node, and when, how you join the second, third master nodes into your cluster. And if you want to know details about how it's implemented, what challenges there are, I would highly recommend you to actually watch replay of KubeCon Europe session done by Lubamir and Fabrizio about KubeADM deep dive. So Fabrizio was exactly talking about technical details, what steps are challenging and why it took us quite some time to do high availability right. Yes, and one more thing you most probably already noticed in the previous slides is we have nowadays official logo of KubeADM utility. It was a poll and based on approximately like 20 different logo variants, this was getting a higher amount of votes and very thanks to Alex now it's KubeADM official logo. All right, but let's switch gears a bit. So KubeADM is fine. It helps you to create like a single cluster, multiple nodes, multiple worker, multiple master nodes. So most of the customers actually run not a single cluster. We usually have from like five to hundreds or even some customers are having like thousands of clusters running. So the question starts to have, okay, I created one cluster, how I do it the same way like for all of my clusters. How I can do the same over the different cloud providers. So in the end of the day, I don't really care how Kubernetes is deployed. What I expect, like I said, let's say application developer is what my application should run on Kubernetes regardless where this Kubernetes cluster is actually hosted. It would be like Amazon or some other provider or on my own prem installation. So the cluster API tries to solve that. So the cluster API is not a single program. It's a collection of vendor-specific operators. And with vendor-specific operators, we handle the specifics how to create the machine, how to create the node of your cluster based on the description. And then on top of that, you have a common layer of controllers which actually do the common logic. So like upgrades or tear down of the cluster or configuration. So this is common for all the providers. So again, to do it graphically. So we have bottom-level parts. We have cluster API where we have specific providers. But on top of it, you have a centralized API where all the clusters look the same to our upper tools. So for example, like Cops tries to already use the cluster API with Qubicorn already using the cluster API. And there are several commercial solutions from different companies which also trying to manage with multi-clusters in a unified way. So why this API is important and what it actually means to have it in declarative way. Here's the example of what you can see about like YAML of CRD, Custom Resource Definition for cluster API. So if you are familiar with Kubernetes, you can find a lot of similar concepts what cluster API have compared to your normal application developer. So you have port, right? If you forget about like vertical port after scaler, port is something immutable. In cluster API we have the same instance called machine. So machine once created, it's not changed anymore. So if you want to upgrade, you delete one machine and you create another machine with newer properties. Then one layer up. So for ports you have replica sets. We have machine sets. So it's exactly set of machine with exactly the same properties. And we have a machine controller which actually creates those machines based on common template spec. And when we have like in Kubernetes site you have application deployment which consists of machine sets. We have machine deployment. And deployment controller helps to do the proper operations like scaling it up and down and how to do the rolling upgrades and practically like similar approach how you manage your application. Now it's applicable to how you manage your infrastructure. Your infrastructure become described in a way like you have a YAML file, you have it in Git. It's completely repeatable. You have GitOps for your whole infrastructure running. So let's like in short what cluster API is about. And similar to PubeADM deep dive, if you want to listen about specifics or like details how cluster API is implemented what are the current challenges. I would highly recommend you to look at deep dive session from KubeCon Europe. Jason was doing a good job of describing it in more details. What Alex talked about is from a high cluster level but actually we have a small problem to solve. How many flags do you remember to start your API server? How many steps do you have to do to create your self-hosted webhook API server? So actually we are doing to make them to be component-based. What does it mean? It means all your components can you reuse the same webhook to build up your own services. And actually now the most distinguished feature of Kubernetes is its declarative APIs. So what about the component configs? All the component configs are configs with flags. And the flags will be deprecated in the next few releases. So you need to modify your flags. And it is incompatible for administrators to manage your clusters because it's not a version. It's not checked. All your flags are just sent over to the arguments of your shell scripts. It is not well documented. What actually component configs to do is try to make it more maintainable, more configurable, and the version. So we have just created a lot of internal configs such as control manager scheduler. And it could be proxy. If you notice, the components that start up by the Kubernetes have already used some of the component configs such as the query proxy. So only one or two flags are needed. No more arguments. This is just a relief for the administrators to keep a version of a stable ram file because it will be well compatible with older versions. So what do you like? Finally, so only one flag is needed. So if you just noticed, there is no kubic config in the flag. Where is it? It has been placed into the schema. So all the flags will be written to the single YAML file or JSON file and the scheduler component or the control manager component will just render the YAML file or JSON file to its own config API types. So it's very convenient for internal controllers and external codes to share the same settings. And we are still involving the component APIs. So if you get interested, please come to HubBus. And just to complement what you just said about the component config. The reason why we have it is it's not only the simplification of the flags. It's more about how to manage your cluster more dynamically. So it's often the scenario where you create a cluster. You create it with a specific set of options. And when, like, a few days later or maybe even a few hours later, you realize what, oh, I actually need to change that flag. And with current set of tools, it's really hard to change this. So what you usually do is, like, you tear down your cluster and then you start recreating it again with the right set of flags. So this component config we allow to do with changes of the configuration some more dynamic. You simplify a task of maintenance of your cluster. Yeah, it's always very easy to get a misconfig of your flags. You might just have a typo or have a misconfig the path. So if we have a schema to help you check all the flags, it's very convenient for you to diagnose your flags or your settings. And if we're speaking about contributions, so all the components, all the sub-projects, everything what we have is actually we need help. And some pieces of the projects are easy to contribute. Some pieces of the projects, like, we have more of, like, more boring and labor work. For example, like the component config, it's not a hard thing to do. It's not, like, rocket science. It's just, like, a big amount of work which needs to be done across all the components of Kubernetes. So speaking about the contributions, as already mentioned, we have hundreds of already existing contributors. And we see cluster lifecycle actually quite good of value of our contributors. So every cycle we, like CIG leads, we are talking about whom we want to promote. So if you want to get the Kubernetes membership, if you want to get, like, to the review or approval status, contribute and gain your pass to the Kubernetes community. And unlike many of our CICs, the CIC cluster lifecycle is not a U.S. centric. We have multiple contributors in the Europe time zone. So if you're talking about here, about China, we have a quite good time overall up where people from Europe can help, where people from Asia time zone, and then hand over it to U.S. time zones. Speaking of China, in last week I mentioned that we are trying to have a back-to-back meeting in China at the same time and finally it's come. So actually we just want to hold a bi-weekly CIG meeting. And it's China friendly time. So if you get interested, just tell me and give me some feedback. We are still collecting the appropriate time slot for Chinese users. So I will be the chair of the Chinese CIG bi-weekly meeting. And we are planning to start our first meeting from the next month, July or August. And how will you contribute? There are a lot of ways to get involved in the CIG. You can just search all the CIC cluster lifecycle level in the Kubernetes issues or pre-request. You can help review the code and help you get familiar with what we are doing right now. And we also have very documented docs on our GitHub. So you can try to read the whole document to help you get familiar with what we are doing, what we are trying to do next, and what you can get from our community. So again, coming back to graphical representation where we are as a CIC cluster lifecycle. We have multiple projects and all of those projects were in different shapes. So QBADM already in GA states of general availability. So it means stable component. But we have a lot of components which we are still in the definition state. So a good example is cluster add-ons. What is cluster add-ons? What we really need to have? What we can do it later? What can be considered as application? How we actually do cluster add-on management? Like when ETCD are there, we know what we want. We have some proof-of-concept code which was distilled from our previous experiences. But again, to make it a real tool, make it real GA level or stable level, it will require a lot of work. For cluster API, we reached finally like the first alpha stage where we understand what we are doing at the first thing. But to get it to GA level, to get it as a stable API, it will take a lot of work. And if you look at our small components, like component config, it's a lot of work to do. And for provisioners, again, something is already stable because many people contributed and some of them is in the very beginning. So you can find the project based on your interest. You can find the project based on your skills. There are like a set of easy tasks to help. There are a lot of complex problems where you want to get deep dive and all that. So if you are trying to know all what we have done before, you can just visit YouTube to see all our recordings there. And we also have a lot of Slack channels. You can just pin us or ask some questions in the Slack channel. And just check out the documents list here. It will just know better about what we are doing right now. So we don't have deep dives on many of our components, but at least two separate sessions for components which is belonging to C cluster lifecycle will be today. Unfortunately, both of them are in exactly the same time slot, but you might be interested. So Cube Spray is a single set of playbooks on how to deploy your cluster, like end-to-end solution. And Minicube, as D already mentioned, it was about local clustering with virtual machines for your laptop environment. And of course, check the recordings from Europe. In Europe, we had more sick sessions with deep dives with more interesting information and more technical details. Thank you for all your comments. And we are waiting for all your contributions to our SIG. So don't feel intimidated if you have any good ideas or comments on our SIG. Thanks for your coming. Yeah, and we have right now, I think, two minutes what we can do, a couple of questions. And if not right now, when we will be available after the session, we can continue discussion offline. Second microphone? Wait a second. Microphone for the recording. Okay. I just saw, I mean, I did a lot of recording, but I just saw the component config. I think this is... Now I think it's very good, but I just want to know what's going on in it, or what's the plan. Actually, we are doing the component since 1.15. Actually, I have done a lot of work in scatter component config. We are just trying to move all the flags of our component into a schema, the API types. So actually, there is a lot of testing works and compatible works for the next few releases. So it will be measured in one or two releases. So if you get interested, just come to our bus to improve more stable. I got a question here. So when you both describe a cluster, so customer want to ready to use cluster, not just a few nodes to master the nodes we want the storage backend to be ready to use. We should have the storage class defined for the backend. We should have the networking while defined, isolated from other clusters. So with current solution, the goal is to bootstrap ready to use cluster or just a scratch. Actually, it could be an easy product and ready to use to help you bootstrap a cluster. Yeah, but I understand the concern. So the QBDM responsible for just bringing the basic cluster. But what you are looking for is probably something like the higher level end-to-end tools are looking like MiniCube does for the local cluster where cops or Qube spray does it for like multi-machine clusters. So look at this high level tools which is built on top of APIs what C cluster lifecycle is maintaining and that's probably what you're looking for. I'm not sure if this, all these things can be plugged to a plugable or this is a plugable architecture, right? So we can hook into any steps to make sure this may be a total solution. I think you can trust the reuse of QBDM as your building block since we have split the function into smaller fancies we call it fancies. And you just try to reuse the code of the single fancies to help you to be hooked in your own customized tools or custom management. Yeah, and for example, I saw the different tools which was built on top of Qube spray. So Qube spray is a collection of playbooks so you have a role for most of the phases but nothing prevents you to reuse those rules into your higher level deployment tool with plug at your specifics. All right, we are out of time. Thank you very much for coming to our session. If you have more questions, let's continue in a hallway. Yeah, and see you in the next QubeCon. Thank you.