 to what's new in Operator Framework. So today, we have myself, I'm Jesus Rodriguez, Principal Software Engineer at Red Hat. I'm the team lead for the Operator SDK team there. And with me is... Hey, my name is Jonathan Burkan. I'm an open source software contributor for IBM, maintainer on the SDK, which is what we're here to talk to you about today. So, show of hands. How many of you have heard of the Operator Framework? Wow, I'm impressed. What about Operator SDK or OLM? Okay. CubeBuilder Controller Runtime. Wow, that's great. You guys have, you surprised me. I honestly was expecting not very many hands. So this is really happy. So what is Operator Framework? It's an open source toolkit to help you create and manage your operators. There's two key projects. Operator SDK, which is the CLI. It's an extensible toolkit that can scaffold out your projects and help you create bundles to integrate with OLM. OLM, Operator Lifecycle Manager, will help you, it helps you get your install, manage and upgrade your operators. So basically, it takes away a lot of the hard work and just kind of lets it run and manage it for you. It also has a dependency model as well. SDK uses other upstream projects, Controller Tools, CubeBuilder, and Controller Runtime as well. So what's new? We just recently added Java's based operator support to the Operator SDK. So what are Java operators? They're effectively operators that you write in Java. Simple as that. So we currently have Go, and Ansible, and Helm support. So now we can actually help you create your operator, scaffold it out in native Java. We were able to do this. There is a new plugin, Java Operator's Plugin, which integrates with SDK. That does the scaffolding of it. And then there is a Java Operator SDK project that was started by another group that is effectively a Controller Runtime library like, except written in Java. So it allows us to not have to do a lot of the lower level Kubernetes pieces in Java, and it makes it easier to develop your operator. For the first version of the plugin, we're actually scaffolding out a corkis-based operator. So using corkis runtime, it'll make it faster, and it kind of reduces a lot of the code when we actually scaffold it. So why did we do the Java Operators? Well, one of the reasons is there were a bunch of Java developers. Everyone says it's the popular enterprise language, so we wanted to kind of target there. We have a mantra of meeting the developers of where they are. So instead of forcing folks to learn go and write the operator, let's get to them where they feel more comfortable. That's how we got Ansible. So a lot of folks were, hey, I need to get operators written, I need to deploy my stuff, and they already had playbooks and things in Ansible, so we went there. And then Helm charts, hey, I've already got a Helm chart, and I have to do an operator, kind of get started so we did a Helm operator. This is the same thing. There's been a lot of chatter of folks wanting to be able to write their operators in Java. So this was kind of a likely good candidate and good path to go down. What else? So phase two plugins. So phase two is in QBuilder. It'll soon be into SDK when we bring in the next release of QBuilder. So what is a plugin? Wikipedia says it's a software component that adds a specific feature to an existing computer program. That's exactly what the plugins have been doing with operator SDK. So we have three main methods in it, create API and create API, I mean, webhook. So these plugins allow us to extend those features. So when you do init and you give us the plugin of go, Ansible, Helm, and now Java, we can create the specific plugins that you need, the specific operator project that you need. We also added chaining. You'll see that in the literature slide. That allows you to combine plugins during one of these phases. So phase two, what about the other phases? So phase one is when we introduce the plugin architecture to operator SDK. So this again allowed us to go beyond go and add specific languages and extend the features to it. Right now, the phase one plugins are all go-based. So they have to be compiled into the SDK. They cannot be external. It's all one big thing. 1.5 is when we added the chaining ability. So you could potentially say, I want to initialize go and then I want to add, say, OpenShift or EKS or something that can maybe transform your project during initialization and do something to it. Those are still go-based only and compiled in. Today, the Java operators plugin, we wanted to get it out sooner, so it is still go, but the plan is to merge it and make it into a phase two. So what is phase two? Phase two allows for out-of-tree plugins. So they will no longer have to be compiled into the operator SDK. They can have their own release cycle. They can be written in any language. We basically have a simple input. We give you a, there's a universe that we pass in. Everything is done right now through standard in and standard out. And we can call your external plugins. We have discovery, so we know to find and where your plugin is and we can run that. In the current implementation, there is an example of a go and a Python to kind of illustrate that you can do this in the future. So again, any language that can handle standard in and standard out and can be baited into an executable but wait, there's even more. Jonathan will take over. Oop, hello, is this thing on? Okay, so let's do some more show of hands. Who here has worked on an application that was deployed to Kubernetes via a Helm chart? Okay, so unsurprisingly most of you are pretty familiar with Helm charts. Who here has tried to make a Helm operator based on those charts? Okay, quite a bit, I don't think I saw, one guy back there. So currently today the Helm operator offers pretty limited functionality. Its intent is that it's sort of a stop-gap measure. You have something that's deployed by a Helm chart. You'd like to make it an operator. You can quickly get up and running with a Helm operator but it offers very limited functionality. It pretty much is just sort of a one-size-fits-all. You don't get really any control over the reconciliation of the resource. Somebody creates an instance of the CR, it spits out the Helm chart. So Hybrid Helm is our attempt to sort of alleviate this problem, to add additional functionality to Helm operators. They're never really gonna be as advanced as the Golang operator where you have total control and can do arbitrary functionality in the reconciliation because you're implementing it yourself. But hopefully this will allow you to get some more life out of those Helm operators. So basically rather than right now today when you build a Helm operator, you get everything baked into an image that we control and it just sort of, you know, you get the image, you can't really control what's going on. So the Hybrid Helm operator, rather than having everything pre-baked into an image we make, it's more similar to the Golang where it's gonna spool out a bunch of scaffolding on your machine and you can go in and edit the various pieces of it. In addition, we have a couple extra little sort of helper features that allow you to manipulate the operator that's being provisioned in terms of like some Helm abstractions you might be familiar with, like an override's file or Helm pre and post hooks. But you'll be doing this in GoCode to manipulate the custom resource type from your operator. It's a separate repository from SDK for the moment that contains all these helpful helper functions and we're gonna go ahead and do a demo of it right now. Now I've been having this, I've been fighting with this all the yesterday because the conference internet is awful. But hopefully my cluster is still up and this should work. Oh man, that's running copy, pasta, some commands. Okay, so we're gonna go ahead and edit a hybrid Helm operator. We're gonna use the Bitnami WordPress chart. It's sort of the hello world of Helm. It's a non-trivial example, stands up a WordPress instance that's backed by, I think by the default configuration. It's a MariaDB instance. Of course, this would take forever to run. Okay, so if you're familiar with what the Go operator looks like, this should look pretty similar. We've scaffolded out the basic framework. We have a main.go, which is gonna run the controller manager. And it's gonna have a reconciler that's configured with the Helm chart and some pretty basic defaults. And then we're gonna go ahead and add the type based on the WordPress chart. See what's happening, that spools out. We've got an API directory now. It's taking forever. Well, I'll go over here and show you what it will look like, or maybe not. So normally I would be able to build and deploy this image, but I've already built the image and uploaded it, so I'm gonna go ahead and deploy it. Just to show you what the basic thing looks like. Hopefully that shouldn't take too long. Okay, so I'm spooling up the controller here. This is what was built out of that main.go. So if we wanted to, if you're familiar with how the controller manager and the control tools and control runtime libraries work, that's the same as the one that runs in the go operator and provided you're handy with a little bit of go, you can go in there and make it do whatever you want. Now it comes pre-configured with the same Helm operator library code that the regular Helm operator uses, but because we have the source code, we can go in there and make it do whatever we want. Go ahead and provision an instance. Okay, so we have an instance coming up. Like I said, it's got a pod that's basically the WordPress web server backed up by a pod that has a MarineDB instance in it. Hopefully the images should already be cached. So it shouldn't take too long to start. Well, while that's happening, we can go ahead and do the next step. So assuming that comes up, we'll be able to access it. It'll be a very basic WordPress that's got a little webpage running. Let's go ahead and make it do something fancy. So, like I said, this is the go code. If you're all familiar with how controller runtime or controller manager works, this is the same thing. We're gonna go hit in and use one of those helper functions I was talking about. So like I said, we have a helper function that lets you set basically the controller tools equivalent of a HALM overrides file. And we're gonna go ahead and stick that in here. So I'm just gonna create a map that's gonna set the WordPress blog name field to the overwrite blog instead of whatever it's set to. And now this is fairly simplistic. I'm just doing basic string to string. But because this is happening in go code, you could get as fancy as you want here and have overrides files based on what's going on in the cluster, even spin up a go controller tools reconciliation loop and do whatever you want in that. Hopefully this is up. Okay, a little networking magic on the backend so I can see it. I keep in mind this is the old one so without the override file. Okay, so we have this nice little blog that says the hybrid blog that was that demo I created. So now if we go back here, we undeploy the old version. And like I said previously, I've already got these images compiled and cached on the cluster. So I'm gonna go ahead and delete the old one, update my tag, go ahead and deploy the one with the override file fix. Hopefully that doesn't take too long to come up. There we go. I got a second demo file. Okay, so this has the same blog name, the hybrid blog is the previous one but because we have that override file that I added to the controller, once this comes up, we should load the webpage and it should have the override blog name instead. Well, that's running, we can take a look at. So the one I use, like I said, is an override function. We've got a couple other helpful functions in here somewhere. So there's the override values one. You can set things like max concurrent reconciles, how often it reconciles, how many releases you wanna have on the cluster at a time, annotations, bunch of other helm stuff. So even if you're not the handiest with writing your own custom reconciliation loop and go, assuming you've already got this thing packaged as a helm chart, you've probably used some of these abstractions and you can just go ahead and add them to your operator with these helper functions. Okay, that's up and running. Fingers crossed, has a different name even though that wasn't what I declared in the Kube YAML. So I used my override. So that's just a short demo for a hybrid helm and keep going. So that's sort of what we have, everything we just mentioned with I guess the exception of phase two but like the Java operator hybrid helm, that's all in the release, you can go download that and use it right now today. Now we're gonna briefly cover some of the stuff we're working on that's coming in the future. So for operator SDK right now, we're working on external bundle validation. So what is a bundle? A bundle, if you've ever used OLM, a bundle is the artifact you produce that encapsulates a specific version of your operator to then be consumed and deployed by OLM. Currently when you generate a bundle, it runs a suite of validations against it but those validations are entirely opaque because they're compiled into the SDK. They happen whether you want them to or not and you can't add additional validations to that. So we're breaking that out. We're gonna allow arbitrators to be called from arbitrary binaries. So if you have some custom validation of your operator that you want that's specific to whatever your operator is, you'll be able to add that in and also from an internal perspective, this is gonna be a lot better because now validators can be bumped independent of the SDK version. So previously which validators got ran was dependent purely on what version of the SDK was installed on your machine which maybe was good, maybe was bad depending on what version match you had between your SDK version and the version that was used to generate your operator. But this will do away with that. We're gonna be adding support for external language plugins so this is what phase two was talking about. So like Jesus mentioned currently today, even like the Java operator is written and go which is kind of weird. We're hoping that this will allow the community, like I'm aware of there's the Java operator, there's also a Python framework out there. But this will allow the community to say if they want a rushed operator framework or Python or whatever arbitrary language you want that they'll be able to go and write that and they won't be dependent on us as a bottleneck. OLM has some slightly spicier things in store. So right now today when you install your operator with OLM it creates this thing called a catalog that's part of how it keeps track of operator versions. And right now today that's stored in an SQL Lite database which is kind of hard for a regular human to use. Cause in order to talk to it you gotta do SQL queries and if you miswrite something you gotta extricate itself from the database. So this is being replaced with just sort of a file based catalog that's what it's called where this stuff is going to be stored in plain text YAML which hopefully should be a lot easier for a human to use especially because we're all familiar with crudding YAML up and down to use Kubernetes itself. And then also coming down the pike is Ruckpack. So that's the name they came up with it but basically what it is, it's OLM 2.0. So they're gonna be doing a major version bump of OLM so the API is going to be entirely different. It's going to be based more around the concept of the bundle. So right now today when you need your operator to be usable with OLM you make this thing called a bundle but it's a collection of artifacts and doesn't really have any one-to-one correspondence with actual Kubernetes resources that get created by OLM. So we're gonna be changing that so that the bundle is gonna be a Kubernetes type in OLM and you add bundles just by directly carting that resource. Eventually we're gonna have support for generic bundles so rather right now today we have our own one specific bundle format that we create and you have to use that. We're hoping to have support for generic bundles so a bundle can be backed by multiple different kinds of resources where you can stick with the bundle type that we have today or you could have a bundle that's backed by a Helm chart and that's how your operator is packaged. Hopefully this will be helpful. If you don't actually care about that you just wanna keep using OLM. When the upgrade happens all the commands you use with operator SDK to generate and use bundles today should still work. Everything should happen behind the scenes so individual users and authors won't have to worry about it. Okay, that's it for what we have prepared. Would anyone have any questions? And I think we got Martin here to run around with the microphone. One, two. Okay. Hello folks. So has anyone got a question? Just put your hand up and I'll come down with the microphone. So last week I wrote my first Java operator with the Quarkus SDK. It was really great fun. The part I was struggling a bit with was the status implementation. So there are really good examples, et cetera, but I was having a bit of a hard time on how to streamline that. So I think there's the other Go project cube builder where you have the more or less a defined standard saying status changes, like last transition message, et cetera. Are there any plans on standardizing this part as well or did you deliberately put that free? The Java operator SDK library does, they are attending the cube builder and controller runtime meetings as well. And so what they're trying to do is it's not going to be completely feature parity, but there are common things that happen on the Go and controller runtime side that they try to implement on their side that come up. So as far as a concrete plan of yes, it will be the same. I can't say that for sure, but they are aware, so I'm sure what's gonna happen is they'll look at it and then try to get it as close as possible to that one. The Java meeting is every Thursday at 11 a.m. Eastern, which I think is 5 p.m. Central European time. So if you ever have a concern, you're more than welcome to join that meeting and bring it up and we can always have a conversation about it as well. So we've no questions yet on the live stream. Anyone else? Hello everyone. Quick one, any difference in performance from the Go operator to the Java one? With Quarkus, we haven't done full performance numbers on it from what we've seen with Quarkus like startup time, it's just a hair slower than starting native Go. But Quarkus in native mode is way faster than starting a JVM operator. So that's one of the reasons we targeted Quarkus first is to allow that to get you there faster. Quarkus has surprised me, because not the first time I've seen it, because not to disparage Java, but at first I was like, well, that's gonna be a lot of startup time and everything and then when you put it in native mode, it's really fast. Okay, anyone else? Anyone else that would like to ask a question? I'm gonna ask a question actually. And it's of interest to me. So I was interested to see that you've no gone with this hybrid Helm operator. So I've used the original Helm operator where it just take the Helm charts and it generates code and you couldn't get at the code. So is this extension or this new hybrid code and the ability for if people have the Helm chart and then they wanna come on and add extra stuff to control afterwards, did you find that people were limited when they had the original one? Say the last part again. So did you find users were limited when they had the original Helm operator? Yes, one of the primary reasons for doing the hybrid Helm was a lot of folks would start with the Helm operator and then realize that they wanted to do more and they would often abandon it and go to create a Go operator and so that they can have more control. So one of the reasons we did the Helm, the hybrid Helm is to by default, it generates, it looks just like the old Helm one but because you have access to the main.go and the actual reconciliation loop and there's also a library in there, you can now start to extend it and add more features to it and not have to re-implement your entire operator. So that was the driving factor behind it. I think that's great, yeah. So anyone else folks before we finish up go on once, twice, three times. So please give a round of applause to our two speakers. Thank you all.