 Welcome to this state of the Operator Framework Update Talk. My name is Daniel Messer. I'm a product manager at Red Hat working on the Operator Framework Open Source Project. With me today is Jason Dobies. Hey, Jason. Hey, Daniel. My name is Jason Dobies. I'm a developer advocate at Red Hat working on OpenShift. Great. The two of us are going to walk you through the current state of the Operator and Operator Framework. Operator Framework really consists of three major parts, and one of which we'll see in a demo later in this webinar. The first part is the Operator SDK, which addresses operator developers, allowing them to building and packaging operators' ad-ease without writing a lot of the boilerplate code that is usually associated with writing operators against Kubernetes. And we have the Operator Life Sacker Manager, which is a central component on cluster, which you can use to centrally deploy and update operators. And then we have the third component, which is Operator Hub I.O., which is a community catalog to which you can publish your operator that you've written and make it installable and updateable from the web. Below is also the link to the upstream source code, which suggests you follow and start this repository. So let's take a look at what you can do today with the Operator Framework. First part is writing an operator, right? And we really have the ability to address virtually any kind of background in programming, if you will, when it comes to creating operators, right? So people that are usually packaging their software with Helm can use the Helm-based Operator SDK to convert the Helm chart to an operator. This would be incapable of installing and updating the managed application. People in administrative roles with ops background are usually very proficient in DSL called Ansible, an automation tool, also very popular upstream. You can use Ansible playbooks to write operators with the Operator SDK, so there's no further knowledge required other than how to express the automation and life-cycle management of your application in Ansible playbook tasks, playbooks, and roles. So this will give you install and update capability as well as cover some of the very basic day-to-day operations of an application. And then finally, the Operator SDK allows you to write fully-fledged operators that are capable of using not only the basic capabilities of Kubernetes to install and update applications and model day-to operations like backup and restore, but are also capable of looking inside the workload to do metrics and alert analysis and use this information to tune the workload so that it almost runs on autopilot. This is possible with the Godang SDK, where you write operators with the Go programming language and use the SDK to scaffold a lot of the boilerplate code that you would normally need in order to leverage client Go. The Operator SDK also comes with packaging, so you can readily publish and ship your operator to something like Operator Hub.io, and also comes with a test harness, which allows you to test and validate the operator that you just built. Here are a couple of things that we recently added to the Operator SDK if you are in the business of writing an operator. We recently transitioned to Helm V3 as the project made its debut into this new major release, so with the SDK, you can write operators using Helm V2 and V3 charts, and they're going to get applied using the Helm V3 libraries. We also have conversion support, so if you have previously deployed Helm-based operators with the SDK endeavor written with Helm V2, there's auto-conversioning happening in the background. The SDK also supports you now much better in generating the metadata for your operator as part of the scaffolding, and the Ansible part is going to allow you to reuse Ansible collections, so this is a big new step in the Ansible ecosystem to manage all those modules out of tree, and the SDK provides a collection for all the Kubernetes modules that you use as part of your playbook to create resources on cluster. Speaking of Kubernetes, the SDK is always keeping up with the latest upstream release of Kubernetes. It's typically in an N-1 fashion, and in the past releases of the SDK, we have been able to basically work in lockstep with the upstream Kubernetes releases so that you can always use the newest features that have been released in the Kubernetes API level. Recently, you are also able to create metric endpoints for your operators. The SDK is going to help you with that as well, so you'll really provide that insight into your operator and its ability to handle the application on cluster. And last but not least, there is a testing utility that we've shipped for some time in the SDK now called Scorecard. This utility helps you do some very basic validation and behavioral analysis of your operator and check if it conforms to known best practices, and this new version of Scorecard has a new output format that makes it very easy to reuse Scorecard in a release pipeline through which you can determine whether or not the operator actually has passed certain tests or not. When it comes to actually running operators, the operator lifecycle manager is really the central point on cluster to do that. It supports the operator developer in using the packaging that comes out of the SDK and put that into a catalog. A catalog can be instantiated on cluster, which is then used by an administrator who can basically pick and choose from this catalog to install an operator. And an installation is not just deploying the pod that has the operator as a container image inside, but it's also doing dependency resolution should your operator require other operators. OLM will automatically resolve those dependencies if available and will keep them in sync during the lifecycle of the operators, which includes updates, of course. Some things that can never happen on a cluster is that two operators are installed that own the same resource they are managing, right? So thanks to the built-in collision detection inside OLM, you and your cluster are basically prevented from that. So throughout updates, OLM will make sure that dependencies are always resolved and that collisions are avoided and updates can be automatic or manually triggered and they are basically available as soon as the catalog refreshes. The user component on the cluster is where tenants in your Kubernetes cluster are actually using the operator and its services. OLM helps them discover which operators are actually available to them and just for them without cluster-wide privileges like reading custom resource definitions, right? So at a very granular level, you can give certain tenants on your cluster, which is really useful in a multi-tenant environment, insight into which operators are available, which operators are installed. And when they use these operators, provide them with rich UI controls to actually interact with their services. So in the packaging metadata of operators through OLM, there are annotations that can be used by graphic consoles, as you will see later in the demo, that really make interacting with an operator feel like a cloud-native experience, but everywhere where your cluster rolls. Some noticeable features that we've added to OLM over the past couple of months was the ability to configure an operator and have this configuration persist across updates. We've also invested quite a lot in exposing more health data of operators and provide better error messages, as well as supporting operators that run in proxy or disconnected environments, which is very important for customers that run in a commercial setting, but don't want to expose their Kubernetes cluster to the Internet, right? The ability to install operators only for certain tenants and also make them only visible for certain tenants is a key component of OLM, so we do support operator tendency. Operators of operators now have the ability to define the update path through which OLM will update the installed operators, so this is usually done by pointing back to the most previous version of the operator that you want to update from, but you can also now extend this to a version range, for instance, so that you say that the V3 release is able to update the V2 and also all the V1 releases. Last but not least, we are also working on making operator catalogs something that you can store in a regular container registry. We have the ability to have these catalogs sit inside databases which are packaged inside a container image and fronted by a GRPC API, which is something that OLM can use as a catalog on cluster. And last but not least, we do have operator hub.io, which made great progress in the last year, now sponsoring over 100 operators that are frequently updated from a central source, easily discoverable via a web interface. So where are we going with all of this? Now a couple of things that are going to happen in the near future. First we are going to introduce a new way to actually bundle the operator metadata and ship this around. Today this is mostly happening in the form of tar balls that are sitting on disk and are getting pushed to backends like quay.io. In the future you will be able to take all of your operator metadata, which includes its custom resource definitions, its package format for OLM, the cluster service version, and put this into a folder from which we build a container image. This container image is not a runnable piece of software, but contains all the required metadata and some metadata on the image itself in order to ship it to any container registry of your choice and have it be something that can be installed directly using OLM on cluster. So you don't need to ship people a lot of YAML manifests that they need to apply. The only thing you need to do is publish a container in a container registry and on the cluster where OLM is installed, instantiate a new operator installation referencing that so-called bundle image. And that is really very similar to locally installing an RPM without making it part of a larger repository or a catalog. So something where you can quickly and easily install an operator and see if it works and still have the complete experience. This works with standard container and container image tooling and is available to OLM as long as it has access to that container registry. Operator hub.io in the future will start to accept this new format as well while maintaining the ability to submit in the older format that we have today. Moving forward, we also have the ability to form catalogs out of groups of these bundle images. So suppose you have multiple versions of these or actually multiple different operators packaged as these and you want to build a custom catalog that you want to expose to users on your cluster because it contains operators that you have tested and embedded and are known to be trustworthy. You can do so in the future quite easily with a new tool called OPM, the operator package manager. So if operator bundles are the RPMs for operators, OPM is essentially the YUM or DNF equivalent to that in the OLM world. It's a simple command line utility which allows you to reference existing bundle images that you've already pushed and put them into a catalog. A catalog, again, is nothing else than a container image, but instead of like in the past containing copies of the entire metadata, it will now just contain enough metadata of your operator to display it nicely in the UI like operator hub.io and for the rest, contain pointers to the actual bundle image in order to do the rest of the installation. So kind of very similar to how a YUM or DNF repository works, right? So here you see how you actually start a new index with just a single package inside and you can add to this index subsequently by just repeatedly calling the index add operation. That will build a new container image and as soon as you push that, you have eventually created and refreshed a catalog. And you can push this with any container tooling of your choice to a container registry of your choice and then reference that as a catalog for OLM which you can then use on your cluster to give people an option, a selection of installing operators. You see here this catalog is also able to refresh itself. So in this particular example, it has been set to refreshing at the 30 minutes. So this is how you achieve auto updates and always up to date catalogs on your cluster. Now what goes inside this package is essentially the same as of today. It's a CSV and more one or more custom resource definitions of your operator. Now in the future, we will also allow you to package your operators with very minimal metadata. So instead of inlining a lot of the basic Kubernetes objects like deployments, roles, customs, security context constraints and the like into a large file like the CSV, we will actually let you ship the plain Kubernetes objects that form your operator be the deployment, stateful set or demon set or whatever else, right? And in addition to that, we just have enough metadata to on the one hand make the operator display nicely on something like operator hub.io and make it something that OLM can install by allowing you to specify dependencies as well as ownership of certain APIs. That hopefully makes it much easier to actually package an operator for OLM because you just essentially copy the manifest that you have anyway into a folder location, add some minimal metadata to it and then build a container out of that. This is complemented by a new API that we will introduce soon, which should simplify the interaction with operators on an operator and an operator installation. And on the right hand side, you will see the new object that we simply call operator, which is the primary point of contact for the administrator to install an operator, be it from a catalog or from a standalone bundle, and all at the same time configure the way it's updated, which namespaces access to all this kind of stuff that we used to do in four different objects we are now able to do in one API. And that's also the point where you would start looking for information about how your operator is doing, how it is updating, if it has updates available, and the like. For users, this gets really natural on the command line by having the ability to essentially just cube CTL get operators and see what operators they have available in their namespace. So a very natural way to interact with the system and discover which operators have been installed by your admin. On the SDK side, we increase the test tooling capability significantly. We have realized that it was probably too much of an overhead and toil to create custom tests for your operator, which is why we are teaming up with the upstream community around a tool called Cuddle, which allows you to express tests for your operator just using gamma manifests. So in this example, you see basically two manifests that are working in a before or after fashion. So as part of a new custom test that Scorecard will support, you have the ability to define the desired state of your operator by using one of its services on the left-hand side. So in this case, we are creating a CockroachDB cluster using that operator with certain specs and properties. On the right-hand side, you see the expected state of things on cluster as that operator starts to work on your request. And these two things, you are basically able to package inside the operator bundle itself and call it a test. And what Scorecard will be able to do is take that test metadata out of your operator bundle and with your operator being deployed by OLM, start testing your operator in this fashion. First, expressing the desired state using the operator's custom resource and then using that very, very similar YAML syntax as you know it from Kubernetes and see if your operator actually did what it was supposed to be doing, which in this case is creating a deployment which is supposed to have three replicas in ready state after that custom resource called CockroachDB was reconciled by the operator. And that's all it takes to actually test an operator in the future. So you don't need to express tests or complex flows in Golang or Ansible. You can simply ship us these kind of manifests and in a before and after comparison, you can test your operator in a pipeline. Starting off the SDK, we will transition to using QtBuilder under the hood for scaffolding Golang operators. So today the operator SDK maintains its own Golang software development kit, which also uses controller runtime, a very popular upstream library for abstracting working with the Kubernetes API. And this exists next to the Ansible and Helm SDK, which is then something you can leverage to partially test your operator, as you have seen before, and use the packaging to create operator bundles. Now we have worked with the upstream community around QtBuilder to join that particular project in order to convene on a single implementation of how scaffolding and building scaffolds for Golang operators will work. So under the hood with the release of operator SDK 1.0, which will happen later this year, you will be able to use QtBuilder transparently under the hood of the operator SDK CLI. And that means you can virtually import any existing QtBuilder project and work with the same command line switches with the operator SDK to write an operator and add new APIs, add open API validations back and all this stuff. That basically makes one common project for scaffolding Golang operators a reality and the operator SDK project will join the QtBuilder upstream project in this regard. Some SDK and Azure SDK will remain available, of course, and will seamlessly integrate with the rest of the SDK's testing and packaging capabilities. To summarize all of that, for developers we make it easier to create operators and create operator bundles. We will make you use Kubernetes tooling, very familiar container tooling to actually build bundles and catalogs. We will have integrated custom functional testing, which is extremely easy to use and we allow you to write operators in the QtBuilder style while not missing any of the SDK's features and functional testing and packaging. On the cluster side for administrators, we make it easier to create operators using a new object called operator, which will be the single point of contact for those administrators in dealing with operators. We will also allow them in the future to actually pick the operator version that they want to install. So they are also able to install older versions of the operator, which is not something that we support directly today. And we will include tooling like the new operator package manager, not only to create yum dnf style repositories of operators, but also mirror those repositories to potentially disconnected registry should your cluster not be connected to the Internet. Last but not least, this will also be something very interesting for operator developers. OLM will be able to support webhooks on behalf of the operator. So as an operator developer, you no longer need to maintain your own logic and procedures to register your webhook certificates and have them rotated in order to stay current. That is something that in the future OLM will allow you to do. So with all those new fancy features now for some interesting demos brought to you by Jason. Take it away, Jason. Awesome. Thank you, Daniel. You guys should be seeing my screen. So a lot of what he talked about was the new improvements to the SDK, to writing, to testing. And as someone who's been doing this for close to 10, 11 months now, that's all really, really exciting. This demo, we're going to pivot a little bit and talk a little bit more about usage. So we did talk about, okay, so we did talk about OLM and how you can use it to catalog operators, how you can use it to install them. So what we're looking at right now is an OpenShift 4.3 cluster. I'm in a demo project you can see up at the top bar and we're in the operator hub. This is giving us a view of all the different operators we can install into our cluster cluster. If you look on the right side, I'm not completely sure if you can see my mouse pointer, but if you look on the right side, we're up at 203 items. I get super excited because every time I do this demo, that number increases every single time and it's really awesome to see we're up at this height at this point. So I'm going to filter out, just for AMQ, we're going to do this demo on the AMQ Streams operator provided by Red Hat. I can see some basic information here. A lot of this stuff is available on operatorhub.io that Daniel was talking about. We're going to like to install it into our demo project here. As some extra information, we didn't talk too much in this particular presentation about the update channels, but it is possible for OLM to manage multiple channels. So in this case, we have stable, but if we had a nightly or beta or however you want to subdivide your quality and your release cadence, you have that capability in OLM. We're going to say an automatic update approval strategy. In this case, for this demo, that's not going to matter because ultimately, we're just going to install this single operator and that's going to go from there. You'll also notice, Daniel mentioned, and this is the first time I heard about this. So this is really cool that in the future, we're going to have better support for saying, I want to install an older version. Right now, we're just saying, give me what's coming out of the stable, and it's going to go ahead and give us this 1.4.0. Now, while that's installing, I'm going to hop over to our developer console. So in the top left, if you didn't see my cursor, we have this drop down now in, as of 4.2, I believe, to switch between the administrative and the developer view. They carry the same information. Developer view, I'm just using in this case to show that we have a deployment, and at the end of the day, what OLM is doing right now is deploying this operator pot. So we see that it has come up, and if I hop back to the administrator view and navigate back to our installed operators, we see that, yes, our Streams operator got successfully installed. So let's take a look at what we can do with it. So we click into it, and we see a lot of things listed on here. Each of these items you see in these boxes under provided APIs are the custom resource definitions supported by this particular operator. For now, we're going to focus in on the Kafka one in particular, and it's going to give us the ability to create one. Now it's going to display YAML like we're used to seeing in Kubernetes in an OpenShift. I want to highlight a second line here. The kind is Kafka. One of the benefits of operators being built on the custom resource definition functionality is it lets us express our APIs in something the user is going to understand. We've modeled our objects after what they actually represent in our application. We can talk to OpenShift and say, I want you to create me a Kafka instance. So we're going to accept the defaults and hit create. And we'll see at the bottom here we have it listed that this was the new Kafka cluster we created. And if we click through, I'm going to come back to this overview tab, but for now I want to go to resources because it actually runs pretty quick. If you've never seen operators install a complex application, which is to say something beyond a fairly simple hello world demo, this might be a bit of a surprise. It's also a very cool surprise. Because I asked OpenShift, hey, deploy me a Kafka cluster. That's it. You all saw the single YAML definition. It has created over a dozen resources now. Everything ranging from secrets to services to pods. So it started by deploying the three zookeepers, which came up pretty much before I even clicked into this tab. And then the next step was to deploy the actual Kafka instances. Now the defaults were for three of each, which is fine, that's exactly what we've gotten. We're going to dork with that a little bit later. But for now, let's dig into one. And you should see this looks like your normal pod interface to OpenShift. But I want to be able to show you that, that I can't type when people are watching, but beyond that, I can query this server for the number of topics. And no surprise, there's not actually any topics here. That's, again, not a surprise. We just deployed this cluster. We'll leave that running in a tab and we'll come back to it. So if I navigate back to the installed operator, again, each of these represents a custom resource that I can interact with at the OpenShift API level. I can create them like I did with the Kafka instance. I can then edit them, patch them and delete them. But what's really cool is that it doesn't have to be something tangible on the server. I don't have to necessarily have it deploy a pod or create a service for me. In this case, this particular operator has defined a custom resource called a Kafka topic that represents, you go figure, an actual Kafka topic. So we rename it to demo for the purposes of the demo and hit create. And it's gone and created the custom resource for us. So if we go back into our running cluster and we list all the topics, we see the demo topic has been created. So again, we created a resource in OpenShift that didn't correspond to any of the existing resource definitions. There was no pods created, no services. But the operator understood that when you tell me to create a Kafka topic resource, I know what to do. And by that, it means go into the actual underlying guts of the server and deploy a new topic and create it and call it demo topic. So let's hop back over to the actual Kafka cluster itself. Again, I navigate through the top here. We're listing all of our custom resource types. And this is the one we deployed to the demo called My Cluster. Now on this overview page you see a graphic representation of its configuration. And we're able to see some basic things that it's configured to deploy three zookeepers, three Kafka brokers and so on. We could if we wanted to use this UI to edit the number of brokers and scale it up. The same time, it's still a resource in OpenShift. So we can always get to our YAML view to do our editing like we're used to. In this case, I'm gonna go down, I'm gonna set it to four replicas for the Kafka brokers itself. And I'll hit save and quickly bump over to resources. And you can see here, it was very quick to stand up. But if you look at the timestamp of this new one, this Kafka three was created just a few seconds ago. So the operator received the configuration or an update to the existing Kafka cluster that said, hey, now we expect to see four cluster instances. The operator took the necessary steps to scale things up. Now it didn't just scale up a new pod. We go inside of the instance itself. So I now have a terminal. If you look at the top here, we're looking at number three. And if we list the topics, we see our demo topic listed. And this is where we're adding functionality on top of what's provided by OpenShift. It's not just me simply saying, I wanna scale the pods up like you've seen in a number of OpenShift demos. So I'm not using the little rocker switches or doing a scale command from the command line. What it's doing here is the operator understands this is what it means when I get a new broker pod deployed. I need to bring it into the system and I need to have it accessible by the full blown cluster. So all of this extra knowledge that takes place beyond simply scaling up a new pod beyond deploying a simple container, you're not putting it on the user. The operator understands this is what it has to do to get connected to the underlying system in the underlying cluster. So there's a lot I can do beyond this. We're going to stop it there but the big takeaways I want you to remember are that inside of OpenShift we can get to this operator hub and this is going to list us all of our available operators that can be installed. Once they're installed, they are providing this very rich, very custom API that users can interact with. And it's not gonna be that difficult for the users to understand it because it's just like interacting with OpenShift now. If you can create a pod, if you can create a service, you understand the YAML manifest that's required. It's just setting the kind to one of these particular types. This is gonna let our users interact with it in a very custom manner. So I'm talking in terms of Kafka and in terms of topics. And again, that topic example, I absolutely love doing it in demos like this because it breaks everyone from the initial idea that, oh, my operator is just gonna create pods and create deployments. It's really showing you that these resources are modeled after the domain itself and the operator can understand when a resource is created in OpenShift, it's gonna take the necessary steps to make that stuff live. So with that, I'm gonna end it. Thank you, Jason. This was such an impressive demo and really highlighted the capabilities of operators that are specific to the workload it manages, right? And the interesting bit about the AIMQ Kafka operator is that it actually hasn't been written with the operator SDK. It has been written with a homebrew Java SDK, but still works just fine as a packaged operator for OLM, which shows you how pluggable this framework is where you pick and choose the tooling that you need in order to get your operator developed or on cluster or just published on operator hub.io. We'll wanna close this with a couple of links and follow-up information for you so you can reach us in the Kubernetes snag on the Kubernetes operators channel. The GitHub repository of the operator framework is where it all starts. If you want to contribute code or your operator to operator hub.io, the link to operator hub.io is also on the slide. And last but not least, should you have a question in either how to write your operator or regarding the packaging for OLM or testing it for Scorecard, there's a meeting list using the Google Groups organization called operator-framework. With that, I'm calling this a wrap. Thank you for listening. Goodbye and see you on the next one.