 the how and why for building cross plan providers. Hello everyone, my name is Krish, and today we're going to be going over how and why you might want to build your own cross plan provider. But first things first, a little bit about me. I'm a software engineer at Red Hat in the office of the CTO, and I'm a maintainer on the provider AWS and provider in cluster projects under crossplane. I'm also a member of the Kubernetes 6 storage where I actively contribute to a few projects. My Twitter is Krish Chow underscore, and my GitHub is Krish Chow. So in this talk, we're going to go over the what, where, when, why, and who for cross plan providers, as well as in the second part, the how for how you can create a provider. This will include talking about the main components, and then eventually live coding a new feature for the provider in cluster. So let's start with the what. So let's quickly go over what cross plan is. So the cross plan project allows you to provision cloud resources from within your Kubernetes cluster. Cross plan is responsible for managing the entire lifecycle of these resources, including update, delete, create and observe. On the other observing cross plan providers expose information about these resources on an ongoing basis through Kubernetes fields on the CRs as well as events. So from the cross plane docs, we have that providers are packages that enable cross plane to provision infrastructure on an external service. They bring in CRDs that map one to one with external infrastructure resources, as well as the controllers to manage these resources. So that's a good definition. But what are providers really, right? So essentially providers are similar to operators, but you utilize the cross plan operator for installation and management. And later on abstraction, providers use a lot of the same tooling as operators such as cube builder, control runtime, controller runtime, and controller tools. The main difference though is that providers are really designed to reference some external resource. It's responsible for mapping one to one with some other resource and representing that resource well. So for a developer, the really nice thing about providers is that you don't have to worry about things like RBAC deployment and bootstrapping since there's already solid tooling that exists to handle those. So the where, so where providers located all cross plan providers are open source and available on GitHub under usually either the cross plane or cross plane control organizations. Some examples are the provider sequel, which orchestrates SQL servers by creating users, grants and roles. The provider helm is similar, so it lets you manage and deploy helm charts using custom resources, namely a release resource. Lastly, the provider AWS is responsible for provisioning resources on AWS, such as databases on RDS, or even Kubernetes clusters using EKS. And it's really responsible for representing those infrastructure resources on AWS. So the when, so when does it make sense to create a provider? Well, as I mentioned, providers are designed to interface with some external API and manage the crowd life cycle for those resources. So the focus that is really on high fidelity resources, right? If you have some external resources that you can represent with high fidelity on a granular level, it makes sense to create a provider. If you're looking to create your own abstractions, then you should probably opt to use an operator as opposed to a provider. Another reason why you might want to use a provider is if you want to use compositions. So the cross plane has a powerful composition engine that allows you to combine several resources. And the best way right now to utilize this is to create a provider that manages your resources. So why? Why should you create or contribute to a provider? Cross plane is a CNCF sandbox project. Hopefully soon we'll be incubating. And many of the providers are open source and anybody can contribute. There's shared development and maintenance of common resources. Every organization shouldn't have to write their own version of the provider AWS. We should be able to all contribute as a community to one project that we can all benefit from. Cloud vendors can also expose their APIs in Kubernetes through a common interface of a cross plane provider. And lastly, you should contribute or create a provider because of the streamlined development and consumption process. As I mentioned before, cross plane helps a lot by handling a lot of the messy parts. So who? Who is really creating and maintaining these providers today? Well, there's three main groups from my perspective. There's the community, organizations, and vendors. So from the vendor side, there's engineers from organizations like Alibaba, IBM, AWS, and Equinix that contribute to the development of their respective providers. There's also contributors and engineers from organizations like Upbound, Red Hat, Squiz, and Accenture that are contributing to the active development and maintenance of providers. And lastly, with any open source project, there's a lot of developers in the open source community who contribute code back to providers and the main cross plane project. So let's get to the interesting part. How? How can we create a cross plane provider? Well, the best way right now is to use the provider template, which is a template that helps you bootstrap your development and it exposes some sample resources and the basic structure of a controller. So the best way right now is to hit the little green button that you see called use this template. And that'll take you to this screen, which allows you to define the repository name and owner for your provider. And just like that, you have your provider. And so now we're going to go through a little bit more in depth into the provider in cluster that we just created, followed by a live coding demo of us creating a Redis managed resource and provider in cluster. So this is our freshly created and cloned version of the provider in cluster based on the current structure of the provider template. So let's dig a little deeper into the various components of the provider in cluster and break down what they mean and how they affect our development. So from the top down, let's take a look at the API's folder first. The API folder contains first the generate.go file, which describes the workflow for how we generate CRDs along with necessary files for Kubernetes and cross plane. The template.go is similar. It describes all of the schemes which we want to add into our Kubernetes runtime. So in this example, we have two schemes here. First, there's template v1 alpha one and sample v1 alpha one. So let's take a look at sample v1 alpha one. So within the sample folder, there is another sub folder called v1 alpha one. And this contains a doc.go file, which specifies the group name and version name for this API group. Within this, we describe our sample resource, which is described in the types.go file. The name of this file doesn't matter, but more importantly, it's important that we have the necessary annotations as well as the structs with our type meta and object meta. This is the actual API type that this provider exposes. There's other generated files here, along with the register.go, which describes the type meta data as well as the initial scheme builder registration. Next up, let's take a look at the command directory. This is the main.go file that's actually run inside of our Docker image when the provider is created. This includes all of the command line argument parsing as well as the initial setup. So here we can see we initially run our APIs when we add them all to the scheme. And we also set up all of our controllers. Aside from that in here, we describe our manager as well as the signal handler. Within the example directory, we have examples of the different custom resources that our provider exposes. This is a good practice because new users can then see what the different resources are, as well as examples that they can run directly. Within the internal directory, we're able to see the different controllers for our provider. Under some projects, you may see that instead of internal, this folder is called PKG for package. But in the provider template, it's currently referred to as internal. Within this, we have our template.go file, which contains the main setup function for all of our controllers. For example, we iterate over our config setup function and our my type setup function. So this sets up both controllers, as I previously mentioned. Within, let's say the config one, for example, we'll see the setup function, which describes how we initialize the controller that reconciles provider configs. And so in this example, there isn't much. But if we take a look at my type, we'll see there is a setup function, as well as a connector, which describes how we produce an external client when we initially want to set up this controller. And additionally, we have our external struct, which has the create update and delete methods for our external resource. And the observe method is responsible for seeing the or examining the current state of our managed resource. The observe function returns a managed external observation, which contains references if the resource exists, if it's up to date, as well as connection details. This is vital because if at any point a resource no longer exists, or it isn't up to date, we'll run the create or update process as needed. So for example, for create, we'll see that it creates our resource and returns successfully without an error if the resource was created. The same goes for update and delete. When our resource is deleted, we'll call delete. And the reconciler will continue to repeat for the managed resource until the observe method returns resource exists equals false. There's also a my type underscore test file, which contains all of our tests for the observe, connect, create, update and delete functions. Next up, we have the package directory, which contains our CRDs that our provider exposes. So in this example, there's not many CRDs, but in another provider such as the AWS provider that has dozens of resources, you'll see quite a few CRDs here. So that's all the main files that you really need to look at as a developer. Everything else is handled opaquely by crossplane for you. And it's really that simple to get started. In the first part of this talk, we explored what crossplane providers are, as well as when it makes sense to create a provider and how to actually create a provider. In this part of the talk, we're going to go through the process of actually defining a new managed resource for the provider in cluster. At a high level, the goal of the provider in cluster is to define resources in cluster that follow the same interface as other crossplane providers, such as the provider AWS, GCP and Azure. So the first part of creating a new managed resource is defining the API. We do this inside of the API's directory. More specifically, we're going to create a new folder for cache v1 alpha one. Inside of this, we define a doc.go file, which contains the group name, which is cache.incluster.crossplane.io, as well as the version, which is v1 alpha one. We're also going to scaffold out an empty resource for Redis, which contains a spec and status struct. At this point, this is all empty, but we're going to come back to this in a minute. Next up, we're going to define our register.go function, which defines all of our type metadata, and it also has an init function, which registers this resource, our Redis resource, with the scheme builder. This allows us to run our code generation, which creates our deep copy, managed and managed list files. These are used internally by Kubernetes and crossplane. When we run our code generation, it also creates our CRD for the Redis resource. At this point, there's nothing exciting here, but as we build this out slowly, this will get filled in with all of our different fields. Now, let's actually build our controller out a little bit. At this point, our controller doesn't really have any logic yet, but we've set it up within this Redis.go file. We can see that we have a connector struct here, which describes how we connect to whatever API we're using. In our case, we're just connecting to the Kubernetes API server. We also define our reconciler, which has all of our CRUD operations, such as observe, create, update, and delete. At this point, we can also add the controller to the setup function. What this does is it sets up or creates all the controllers with the loggers and rate limiters when the pod is started for the Encluster provider. Next, we can start to scaffold out the controller. Before you actually write your logic, it's important to define the different functions that your client needs to have. In our case, our client will have a creator update function, as well as parse input secret and two functions for deleting the deployment and Redis service. At this point, these methods aren't defined yet, but as we build this out, we'll slowly see that things come together. We're also going to define four functions here for creating our Kubernetes resources for our Redis deployment containers, as well as the service itself. Now, we can head back to the API, and we can see that we've added a few new fields here. At this point, for simplicity's sake, we have three fields, namely password secret ref, config variables, and memory limit. Memory limit states the maximum amount of memory that we want to make available to the Redis pod in our deployment. Config variable allows the users to inject arbitrary environment variables into the Redis instance. And lastly, password secret ref references the secret which will contain the password for our Redis instance. If no reference is given here, then we will expose Redis without a password. Once we define these attributes, we're going to run our cogeneration again, and this will update deep copy as well as our CRD. Now, we can actually start the process of implementing our client. Here, we define parse input secret as a function which gets our secret and then extracts the password data from it. Delete Redis deployment just validates that the deployment exists, and then it deletes it. The same thing exists. The same thing happens for our delete Redis service function. Next up, inside of our make Redis deployment, we define our apps V1 deployment resource, which contains the object meta and spec for our deployment. The containers are defined in another function down below, which first sets up all of our environment variables as well as our memory limit. And then we define the pod as having container, which contains the Bitnami Redis image as well as which exposes the default Redis port 6379 with the maximum memory and environment variables. We also have a loudness probe to validate when the pod or container is actually started. With our client defined, we can actually start the process of now building out the controller itself. And so here, we can see that we've defined our connector function as extracting the provider config reference, getting the cube config from there, and creating a new client with that specific cube config. Our reconciler then has a reference to the Redis client, and it's responsible for observing the current state of our managed resource. So, namely, we check if the deployment exists, if the deployment is ready, if it's currently in the progress of scaling up, and we also validate if the service exists. And if any of these resources don't exist, the controller will fail, and it'll get put back in the retry loop for our reconciler. If everything exists and it looks good, we'll return successfully that the resource exists and it's up to date. We'll also expose the IP address of Redis at a specific key in our secret, namely the endpoint key. The create function actually runs the process of creating our Redis instance. So this means deploying the deployment and service resources. At the end, once everything is created, in our secret, we will expose the port and password. So in our case, the port is always 6379, and the password depends on the input secret if it exists. Update in the Redis resource at this point is a no-op, and delete simply just deletes the deployment and the service. So with that, our managed resource is implemented, and we can define some examples in the examples cache folder. And so this will have our sample Redis resource with the password reference as well as our provider config. So just like that, it was that easy to create a managed resource. All in all, this process took me about an hour, or less than an hour, and we have a V1 Alpha 1 release of Redis in cluster. So we can add some additional features. There's still quite a bit of work to do, such as a Redis cluster or additional testing for Redis, because at this point we haven't written any tests. But we have enough now where somebody can provision Redis in cluster using this provider. So that concludes this presentation. Thank you for watching, and if you have any questions, feel free to ping me on Slack or add me on any of the socials. My LinkedIn is Chris Chow, which is the same as my GitHub, and my Twitter is Chris Chow on your score. Thank you for your time, and have a good KubeCon.