 Alrighty, hello, welcome everybody. My name is Michael McKeon, I'm a software engineer at Red Hat where I work on OpenShift, cloud infrastructure stuff, and I am joined today by Bridget Kramhout, I am a product manager at Microsoft, and we're here to talk to you about cloud controller managers. Wait, why are we going default? What is this cloud controller managers going default? Yeah, that is a great question Bridget. There's a big change coming to the Kubernetes community around cloud controller managers, but before we start talking about that, let's set a little bit of context here. Sounds good. And clicking, maybe error. Clicking, yes. Okay, so we have a lot of acronyms here, right? KCM, CPI, CCM, TLDR, like what is happening there? So there are three in specific that we want to talk about to begin with here, KCM, CPI, and CCM. And these are the ones that are really focused on what we're talking about today. So the KCM is the Kube controller manager, and that is a process that runs in every Kubernetes cloud that kind of runs to the core control loops in Kubernetes, such as namespaces, replication controllers, those kind of objects. But as it stands today, it also contains some loops known as cloud controllers. And these handle interactions with the underlying infrastructure, such as reading metadata services or managing resources like load balancers. We did mention that naming things was one of the hard problems. Totally. So next we have CPI, and this one's a little bit different. It stands for Cloud Provider Interface. And this is a code interface that was created to help abstract the functionality that the different cloud controllers perform so that it could be easier to kind of upgrade these things in a more orderly manner. And it also helps with the creation of the CCMs. And just as a point of clarification, CPI is not the same as CAPI, C-A-P-I, because why would it be the same? That's the cluster API project, and the cluster API provider. Something entirely different, wonderful on its own, not related to this. Yeah, and then that's a good clarification. So that brings us to the last definition here, the CCM, which stands for Cloud Controller Manager. And these are processes that run the cloud controllers on their own. So the KCM has cloud controller loops, the CCM has those as it kind of encapsulates them in their own binary process. Are you confused yet? All right, so let's talk about these cloud controller managers, like exactly how do they work? Yeah, so at a very basic level, the cloud controller managers run within a Kubernetes cluster, and what they do is they perceive and manage integration with the underlying infrastructure provider. There are four controllers that we're really interested when we talk about these. There's the node controller, the node lifecycle controller, the route controller, and the service controller. And each one has a very specific function. So the node and node lifecycle controllers functions are to interact with the infrastructure provider to get information about the node's physical location, zone and region, and also to have an awareness of whether it's present or absent, whether it's in the process of being shut down, whether it's been removed, those kind of things. And then the route controller helps to enable communications between containers across the cluster, right? So it uses advanced routing to make sure that every container can talk to the other nodes in the cluster so that you have continuous communication between the containers. And then lastly, there's the service controller, and the service controller helps to integrate with things like elastic load balancing services. So you can imagine in Kubernetes when you create a service object of type load balancer, if you have a cloud provider in place, what that's doing is interacting with the cloud provider and where possible creating elastic load balancing type services. So now it's worth noting here that, even though there's only four controllers here, these can be expressed in a variety of means by the infrastructure provider. So when you're examining your own clusters and you see these things running, you may not see a one-to-one parity between the containers running and the four controller managers that we're talking about today. Some cloud infrastructures use extra processes to help manage things like virtual private clouds and whatnot, those type of things. It all depends on the specific provider's needs. All right, so this brings us to possibly the most important question, since you're all listening to us talk about this and you're thinking, why should I care about CCMs? Like, this is actually important, probably to most of you, the default is about to change. Very scary words. Yeah, no, you're totally right. You're totally right. The default is about to change, what does that mean exactly? For the longest time, there's been code in the Kubernetes repository that contained provider-specific code. Like this is in KK. Right, right, in Kubernetes slash Kubernetes in GitHub. And there's code for Azure, there's code for AWS, there's code for GCP, and this is in the core repository. And this creates a condition that raises the bar of difficulty for merging changes as they gotta go into the core. And it's not just the difficulty, it's also the release cadence. Because as many of you might be aware, there are three Kubernetes releases a year. So if your provider would like to release something that is tied to the core Kubernetes release, well, they might have to wait to get that fixed in, or they might have to get it into the monthly patch cadence. So if you're a provider who missed the release window, you might be waiting, and then your customers might be waiting. Nobody wants that. Right, no, that's a great point. And like what happens when you need to backport a critical bug or something like that? You don't wanna be waiting. Or a bug fix. I mean, we could backport the bugs. Right, no, they can be backboarded, but it's easier to manage some of these things out of tree as it were. So when you hear people say things like in tree and out of tree, especially around the community, this is exactly the type of thing that they're talking about. In tree refers to the code that is still in the Kubernetes code tree. And the out of tree motion that we're making now is to take that code out and move it into these CCM so that it can be managed a little bit separately. And like it can be managed however the communities prefer. And but then that leads to the question. I don't know about all y'all, but as a old-timey apps person, I'm always worst case scenario gal. So I'm thinking, wait a minute, doesn't that mean they'll all end up really different from each other? Like how does that effect say someone like Red Hat? No, yeah, for sure. Like they do end up different from each other. But thankfully there's SIG Cloud provider in the upstream. And they've made great efforts to build a common framework that all the different infrastructure providers can use. So theoretically they would look the same and follow similar patterns at the interface layer. But when you look through the code and you look through the individual implementations, they do end up looking differently as different design choices are made and sometimes extra processes are needed to do things. Different exciting bugs. Always. Okay, so you did mention that there are a bunch of these providers Red Hat's paying attention to, caring about the interesting bugs for. So tell us about that. Yeah, so in about a month from now, I think when the, or maybe two months, when the 4.13 release comes out, we'll have general availability support for the six platforms that you see here in OpenShift. And you don't have to do anything extra. These will be automatically migrated for you as you upgrade your clusters or as you install new clusters on these platforms. We're planning to add more in the future, with AWS and Azure, like hopefully coming in for 4.14, that's our plan right now. You could try them today if you wanted to by enabling one of our feature gates called Tech Preview No Upgrade. Which is a little bit dire sounding. What's happening there? I mean, it is, yeah, that does sound a little dire, but that is a feature gate we have a lot of documentation on that allows you to turn on new features that are coming to OpenShift. But as the name implies, you will not be able to upgrade those clusters. So only do that on things that you're not using in production. And also, if I'm thinking, like looking at this thinking, hey, these three large cloud providers in Tech Preview State, what's going on there? I mean, no, that's a great observation. I work at one of those. Why are we at Preview? Yeah, I mean, totally. So these three, AWS, Azure, and GCP have had code in the core for some of the longest time, right? So they're very well tested. They're very solid. And in fact, the CCMs that were originally created out of those were actually wrapping the code that came from the entry. So we have good confidence in the migration of those, but we've been focusing on bringing some of these other ones in. And as you see, AWS, Azure, and GCP, they'll be coming. For someone like Nutanix, they never had an entry provider. So they had to start with an auditory provider because the core isn't letting new things in. So this is basically like how emerging markets get to jump directly to cell phones. And the countries with a lot of landlines are a little bit behind. No, exactly right, exactly right. And the way that the CPI is built, you can directly take things from the entry to the out of tree. So we have strong confidence that it's basically the same code running in both places. Hashtag probably fine. Hashtag probably fine, yeah, famous last words, right? All right, so if say I'm looking at an OpenShift cluster, how do I tell if I'm running these out of tree controllers that are the exciting future? Yeah, totally, there's a couple of different ways you can look at this. So if you're using the console, this is a really easy way. You can go into the workloads, deployments, and you can see here, I've done a search for cloud controller manager, and what it's come up with is two different deployments there. And so one of the deployments is the cloud controller manager, and the other one is the cloud controller manager operator. And how do you tell this is the out of tree manager? Right, so if this were using entry, you would not, yeah, you would not see this part here. And sorry I'm blocking all you folks now, but this top one, you wouldn't see the cloud controller manager. It says as your cloud controller manager, you would not see that at all. But you would still see the manager operator because we deploy that in everything from I think 4.12 forward. So the next way you could check too is by using the OC command. And there's two kind of namespaces you wanna look at here. One of them is the OpenShift cloud controller manager namespace, which is where you'll see that Azure cloud controller manager. The other one is the cloud controller manager operator namespace. Say that three times faster. So again, naming is difficult. Internally, we tend to call this the 3CMO, which I kinda like cause it sounds like 3CPO, but you know. So these are the shell commands you can use to verify that these deployments are the droids you're looking for. Absolutely, absolutely. And if a shell oriented approach fits your case better, this is kind of the direction you would go to confirm that these things are deployed in your cluster. So all right, let's talk a little bit about Microsoft Red Hat collaborating together. Yeah, well, so how are we collaborating together? I mean, we're giving talks. Yeah, I mean, that's something that's fun, right? We're also working on Azure support in OpenShift. I mentioned in the earlier slide that auditory cloud provider support is hopefully gonna be GA in OpenShift 4.14. And that's coming soon. Yeah, we don't have a firm date for 4.14 release, but if this follows our normal cadence, I would expect this fall you'll see 4.14 coming out, probably four months after 4.13 comes out. But as we mentioned, for those with platforms without general availability, you can try that scary sounding feature gate to turn these things on and give it a test on your non-production clusters. Because if you're, we talked a little bit about that, once you enable that feature gate, you can't upgrade that cluster. So not on a critical path. Yeah, no, it's really important to note that, and there is some documentation. If you go to the product docs and you search for feature gate, you'll find the documentation that talks about the tech preview, no upgrade feature gate, and it'll walk you through all the processes and it'll show you what features you're turning on to. And so when we have new versions of OpenShift coming out and there are new features coming, this is a great way to kind of get into it. And we've been talking about the OpenShift console, but you can also use Arrow in the Azure console. Absolutely, you can do all this from the Arrow console as well. Okay, so we all know to expect exciting failure states because computers, but happily with cloud providers, you've got a lot of help. Yeah, totally. I mean, there's a whole community collaborating together. Microsoft's been a great partner for us. You can see here, some of the examples of work we've done recently. We've been able to report issues really easily. We've been able to create code really easily. Yeah, it's been great. And this is all happening on GitHub in the community, in the Open. Come participate. Yeah, so in closing, you know, OpenShift operators and enthusiasts who are really curious about like this part of Kubernetes, check out SIG cloud provider, please. There's a ton of materials there. There's lots of docs and recordings and meetings and agendas that you could just really fill a Saturday with, you know, watching if you needed to. And it's not just Red Hat and Microsoft that are there. There's representatives from many cloud providers who come there and, you know, we generally have like a really great time. So thank you. Thank you.