 Hi, my name is Keith Tenser, Principal Solution Architect at Red Hat. Today I'm going to be showing you to build Kubernetes operators using the Operator Framework and Ansible. Before we get started with that though, I would first like to talk about why operators are such a game changer. Of course, stateless is easy, stateful is hard, but it's really more than that, isn't it? It's about complexity and dependencies. You see applications are dependent on a lot of things. They're dependent on one another, other services, infrastructure such as storage, and as we all know, of course, they also sometimes maintain state. In the past we've dealt with complexity through a lot of domain knowledge, possessed by individuals who use that knowledge to create documentation, procedures, and hopefully a lot of automation. Now operators are really trying to change the paradigm and move domain knowledge from the individual's head into the code on the platform. This will allow us to scale in a much more robust, effective way. Operators also allow us to integrate applications with the platform in a much more meaningful way. Through API extensions known as custom resource definitions, we can package, operate, and deploy applications without, and I repeat, without application knowledge or knowledge of application semantics. This is huge. I think you would agree the world around us isn't more complex one. And if we think about the future, it's really about how do we scale that complexity? That is the challenge operators are in a nutshell trying to solve. This illustration depicts what we just talked about. Our all modern digital infrastructure or application landscape, whatever, being dependent on domain knowledge possessed by individuals. As we know individuals are individuals. Their interests change, their roles change, and of course their organizations change. And when that change happens, the knowledge goes with them or goes away. And we're left with basically our deck of cards that just simply falls over. That is not an effective way to scale into the future, not with the level of complexity that we're dealing with today. Let's now get into the operator framework and talk about some of the components. As you would expect, the operator SDK provides a tooling to build and package operators. More so, it provides scaffolding, so it's able to auto generate a lot of the Kubernetes objects that we need, including Ansible roles, and it abstracts away a lot of the low level Kubernetes information and knowledge that would be needed. So you really can just focus on your application and Ansible and be successful building an operator. That's the goal, and that's what we're going to see shortly. The second component is the operator lifecycle manager. This is getting into building more advanced or sophisticated operators that are getting into governance, lifecycle management, how do we do with backups, failure scenarios, environmental changes, those kind of things. The third component is metering, which is enabling historical information that can be gathered in order to report on or act on. If we think of auto scaling applications based on metering, that's where we're getting with that. The last part is not really component of the operator framework, but I wanted to mention it because it's important as the operator hub, which is providing a marketplace for our community operators so we can share our operators. And interestingly, Red Hat, of course, provides an enterprise Kubernetes distribution called OpenShift, and we're using operator hub default out of the box in OpenShift, and that's the mechanism that we're using to consume and provide operators, whether it be from Red Hat, third parties, vendors, or even the community. And the great thing is operator hub is, of course, driven by an operator, so you can use it and deploy it on any vanilla Kubernetes cluster. At this point, let's get a little bit more into the capabilities of operators themselves. So there's different levels, level one through five. Level one is a basic install, which means just being able to deploy the application. Level two operators are able to do seamless upgrades, they're able to do patching, minor version, major version, those kind of things. Level three is getting into what we just talked about with operator lifecycle management. How do we do with failure recovery backups? Level four is our deep insights, collecting metrics, analyzing workloads. And finally, level five is really autonomous operations, i.e. autopilot. Our operators are capable at this level of horizontally, vertically scaling our applications, auto-tuning configuration, anomaly detection, and more. Now this isn't to say that every application needs a level five operator. I think it's fair to say different applications have different requirements. But certainly most of the value is going to be in getting to those level five operators, especially for complex applications and workloads. Now there's different ways to build our operators, which is great. The first is Helm, which I cannot recommend because you're simply limiting yourself to level two operators. I know a lot of people are using Helm out there, but I don't think this is the way to go. Helm is not a programming language, it's not an automation language, it's a deployment language. And so all you can really do is deploy things. And so we're really limiting ourselves there, which for me leaves it to really Ansible and Go, which is okay because we have Ansible. Ansible is the automation language. We're all familiar, at least most of us are probably familiar with Ansible and used it in the traditional, non-containerized or cloud-native world to do everything from CI CD, application deployments, server builds, security patching, network management, you name it. And so we can leverage all of that knowledge here. Again, we don't have to learn something new and leverage it to build our operators. Even more powerful is we're not creating every application new because it's containerized or going to Kubernetes. We're transforming and modernizing applications. And so with Ansible, we're already using the language to do, manage and automate our applications. And now, in my opinion, this will greatly increase the speed at how we're able to transform applications and lift and shift them, modernize them from traditional infrastructure to a cloud-native architecture or pattern. Of course, GoLang is a language of Kubernetes. And so if you need to unlock some of those low-level Kubernetes primitives, Go is going to definitely be the way to, is going to provide advantages for doing so. However, I think Ansible by and large for most of us in most use cases is going to be the language of operators. Now as we get into the demo, we're going to see that and we're going to go through step-by-step and build an operator with Ansible. Now I talked before a lot about scaling complexity. And what we're going to be doing here is just a hello world operator. We're not obviously going to be building an operator to deal with a very complex application, but I wanted you to understand where things are going and really the ideas behind operators before we get into this so that we have a level playing field. Also, in order to build operators, you need to set up a development environment. You set up and install some packages and software. I'll provide at the end a link to where you can get more information on how to do that. So please look at that when we get there. With that said, let's start building our operator. The first part is the scaffolding. We're going to generate all of the Kubernetes objects that we need, CR, CRDs, role-based access, and of course also the scaffolding for our Ansible role. So I'm going to open up here a terminal in CLI. We're going to make a directory operator dash hello world. I'm going to go into the directory and I'm going to the interest of time copy, paste some of the commands so I don't make any mistakes. The first one here is to do an init. Again, this is going to create a lot of the Kubernetes objects as well as it's going to create a Dockerfile and make file. So we'll be using make to actually deploy our operator Dockerfile to build it. And under config, as I said, is where the Kubernetes objects are. And under roles is where our Ansible role is going to be, which is empty because the next step is to generate our role. Again, this is the scaffolding for the role, which we'll see. So we're going to create the role here. And now under roles, I'm going to have a role called hello. And under there, I'm going to have the hierarchy you would expect out of an Ansible role. And of course, where we're going to be mostly focuses on tasks, which is where we implement the logic for our operator. So I'm going to go back now to the next step. And we're going to add a task is going to be just printing a hello world. But we are, I am showing you here how to get information from the CR. Again, the CR is a custom resource. This is how a user interacts with an operator. It's how they interact with the CRD, which is the operator API endpoint. And so we're able to get parameterize, get information from the user in order to make decisions in our operator. And we're able to get things like what namespace is the CR created in. And that's exactly what that's doing. And we're only printing the message, as you can see here, when the Boolean toggle and score message is set to true. So we're allowing our users in this case to configure that if they want to see this message. So we're going to see, we're going to set up that task and then see how that looks. So I'm going to VI, roles, hello, roles, hello, and the main.yaml under tasks. And it's empty as you would expect. So I'm going to copy the task now and just going to grab it and paste it here. There it is. I'm going to save that. And now we're going to go on to our next step. So we've added it one task so we can actually do something with our operator. However, we added that toggle underscore message. So we need to add that to the CR. As you can see here, we'll see the SDK auto generated a sample CR. So we're going to use that and go back to our CLI here. We're going to just VI the config sample and CR. And as you can see here, they've added foobar, which is kind of just an example. And I'm going to put toggle underscore message is true and save it. Great. So now we're going to go on to the next step. And at this point, we can actually test our operator. So what we're going to do here is we're going to use make and we're going to make install. What that does is it deploys our CRD or custom resource definition, which again is our API endpoints for the operator. Then we're going to create a namespace and we're going to run the operator in local mode. So running operating local mode as you'll see basically runs it in the terminal. So we see everything in standard out. And the advantage there is we can quickly test. So we don't need to go through a full deployment, creating a Docker image, pushing it to repositories, registries, that kind of stuff. So let's do that. We're going to do our make install. And you'll see how to use cube CTL apply in order to do that. And then we're going to create our namespace. Got to name it right operator dash hello world. Great. And then at this point in another terminal window, we're going to start the operator in a local mode. Now, what's important here is before we do that, we want to set the context so that we're in the correct namespace. So we're in the operator hello world namespace. So I'm going to use cube CTL now in order to do that and set the context. And then what's important is we need to be in the directory operator hello world. That's where our, again, where all of our Ansible code is and everything is. That's what it's expecting. So I'm just going to run Ansible dash operator run local. And it's basically going to start, as you can see in the terminal, and it's going to start the controller, which is going to watch the CRD. And it's going to watch if a CR is generated against that CRD in the project operator hello world. If it is, it's going to execute our Ansible code. That's basically how this works. It's very simple. In order to do that, we're going to need to create a CR again. We've already, you know, have the YAML file for it with our parameter. So we're going to create the CR and make sure we specify the correct project. That's always good practice so that you know where things are being generated. So we're going to use Coup CTL create create minus F file config samples and our CR. And again, minus N for the namespace operator dash hello world. And it's going to generate it now. There it is. And as soon as that happens, you're going to see basically the Ansible code executing here. And we've just seen that. So it ran our task. Everything worked. The code is good. Hello world. I live in namespace called operator dash hello world. Again, the namespace we got dynamically as we showed you. And so this part was successful. So what we're going to do now is get more into how do we work with Kubernetes from an Ansible perspective. So there's a couple modules. The first is k8 underscore info. This is the module that you use to query Kubernetes objects and get information. In this case, we're interested in getting the application domain. This is the domain that basically application ingress routes are using. The FQDN for that for the application endpoints. And that is stored in the ingress object. So we're going to get that. And we're going to then set it as a fact. And then we're going to print that fact. So I'm going to go back to the other terminal window here. And we're going to vi the tasks file again. Just find it. Here it is. I'm going to open up the session here and I'm going to grab those tasks. Bear with me a second. I'm just going to paste them in here. Great. Got those. So those are our tasks as we just saw. So now we're going to go back to presentation in the next step. And as you know, noticed we're we're querying the ingress object. So by default, the scaffolding will create a role with some default permissions. However, you need to if you're going to be doing things with your operator, you need to understand what permissions you need. In this case, what we're demonstrating here is how to add permissions for ingress, because that wasn't one of the default ones. So there's a config RBAC role.yaml. And we simply need to update that. So we're going to go open that file up. Config RBAC role.yaml. And we can basically put it anywhere. I'm going to put it in here and I'm going to again copy paste this copy paste it in there. And looks good. So again, we're adding the API config.openshift.io, which is where the ingress is that I'm interested in here. Of course, I'm working against an OpenShift cluster. So there's different objects, different things. This is an OpenShift specific with OpenShift routes and the way we do things there. Okay, so we got the role permission set. Now, the next thing you can do is add some additional tasks. I'm not going to do this. I just wanted to demonstrate it. If you want to manipulate an object or create an object in Kubernetes, like for example, service as this example shows, you can use the k8s module to do that. So this is basically showing you how to do it. Again, for our little hello world operator here, we're not going to be manipulating any objects at this point, but I did want to include it just so you'd be aware of what that looks like. I have an example of it. So I'm going to move along. So we've added that task, the additional tasks to query the ingress object and print out the application domain. The next step is to test again and make sure that we actually see and get what we had. Now, I'm going to go back into the terminal here. And what we can do is just simply restart our operator in local mode. And it's going to run through, since we already have a CR, it's going to execute against it. And as such, execute our additional tasks that we've added. And so you can see here, it went through the first task again, worked, and the additional tasks we added also work. You can see here application domain is apps.ocp4.keysensor.com. And this really illustrates how you should develop with this, how easy it is to do so. And that's why I wanted to do this kind of in a step-by-step fashion so you see it and understand that piece. It's really, really awesome. Okay, so we've basically got the code the way we wanted to. We've tested it. And now the next thing to do is how do we deploy our operator? So we saw those Docker file and the make file. We can use make to build a Docker image. It's going to basically wrap our code in the container and publish that to registry. I'm using quay.io. Once we've published it, built our container, published it, then we're going to deploy it. What's going to happen here is it's going to do a Kubernetes deployment. Again, the SDK has already created a deployment for this. So we don't have to do anything. We just run this command. It's going to deploy it in a project called operator-hello-world-system. That's the default. You can change that. And then finally, what we're going to do is we're going to create a CR, as we did here previously, against that deployment and make sure it's working. So let's now run through those steps. So what I'm going to do here first is I'm going to control C out of my local operator since we're doing a deployment now. And I'm going to copy these next commands. They're a little bit long. I don't want to make any errors again. So I'm going to grab the first one, which is going to build our container image and push it to quay.io. As you can see, that's happening. And once that's done, the next thing to really do is just deploy it. So we're just going to do here, again, another command to deploy, to make deploy with our image. And off it goes. You can see what it's doing is also creating the roles. So that's why it's important also to update that role.yaml so that the operator has the right permissions. So all of that's basically wrapped in the deployment itself. Now, the next thing we can do is we can look and see what that deployment looks like. We can come over here and we can make sure it's deployed. So there we see we actually have a controller manager. It's ready. It's up. There's a pod running. We can actually as well get the pods, which we're going to also need to know the pod to be able to query the logs of that pod. So we got pods here. You can see we've got the running pods. And then the next thing we're going to do is we're going to create the CR against the project operator hello world system. So we're going to do our kube, CTL, create minus F, we do config, samples and our CR and then give the namespace operator dash hello world dash system. So that created that. At this point, the operators obviously executing everything. And what we can do now is we can use kube CTL again to verify that and look at the logs. So we can do basically kube CTL logs. I like to do a dash dash follow. It's basically like tail. And we're going to give it the name of the pod in this case. And this pod, as you can see, has two containers. One of them is the kube proxy, which is a sidecar container and our other is the manager. And the manager is where our code is running. So if we look at that, that's where basically things are going to execute. And then I can say minus N operator dash hello world dash system, which is namespace. And here you see the logs. Again, the output that we expect. And again, our operator is running in this project. Everything worked and we completed our deployment. That concludes the demo. So I hope you enjoyed that and got a real feel for how this works. Now, your learning is not done yet. I've provided a walkthrough guide with some exercises that will actually show you how to build a development environment and go through some more interesting scenarios that get into deploying an actual application. In this case, it's still not a complex application, but this is basically those exercises. And you can see here, we're actually creating tasks that are rolling out deployments, creating services, that kind of thing. So if you're interested in further knowledge, check that out. GitHub.com slash ktangular slash operator dash hello world. Also, I've linked the official docs. I've provided also a couple of blogs where I've written about some of the more background. If you're interested in more information about the operator framework or about building Ansible operators getting started with the SDK. And I've provided a link to the community operators. With that, thank you. I hope you enjoyed this presentation. I hope it was worth your while and I wish you the best of luck on your journey with operators, the operator framework and Ansible.