 Welcome to this presentation where we will talk about cross-plane which is an infrastructure as code tool that enables us to use Kubernetes to manage and provision infrastructure. My name is Vijin Palazi and I am the head of technology and a trainer at CodeCloud which is a learn-by-doing platform specializing in DevOps courses. CodeCloud is a CNCF silver member and a certified Kubernetes training partner. So let's take a look at the agenda of this talk. We will start with an introduction to cross-plane and see how it differs from other popular infrastructure provisioning tools such as Terraform and Pulumi. We will go over the installation steps followed by an overview of the fundamental cross-plane concepts such as provider, packages, managed resources and composite resources. This will be followed by a quick hands-on demo where we will make use of cross-plane to deploy resources on the AWS cloud. And we'll wrap up the session with a round of Q&A. Before we begin, let's go over some of the prerequisites and assumptions. Since cross-plane installs on top of Kubernetes, a basic understanding of Kubernetes is expected. I'll be using some examples of core services in AWS cloud so a basic understanding of AWS or cloud computing in general would be helpful. In some of the examples, I'll also be making use of simple Terraform configuration which is written in HCL. So let's begin and get introduced to cross-plane. So when we start talking about infrastructure provisioning, one of the commonly used tools today is Terraform. And then there are tools such as Pulumi that have gained popularity in the last few years. Now these are some fantastic tools that allow us to provision and manage infrastructure on almost all platforms such as AWS, Azure, GCP or on-premise infrastructure such as VMware. And while these are some really great tools, they do have some certain limitations. Let's take Terraform as an example. It uses HCL or Hashacop configuration language. So here we have a sample configuration file to create an EC2 instance and an S3 bucket in AWS. And it is written in the HCL language that Terraform understands. And although it is a simple declarative language, that's yet another tool that the users need to learn. Let's see another example. This configuration, when applied, will create an EC2 instance in AWS with the tag of app version is equal to V1. Now Terraform will also create a state file that will hold all the details of the AWS instance, including the tag app version is equal to V1. Now, if this tag is manually updated to say app underscore version is equal to V2, then Terraform will not detect this change unless the Terraform workflow is invoked manually the next time. What this means is that these tools, such as Terraform and Pulumi, do not automatically detect drift. And as a result, they do not reconcile back to the desired state of the infrastructure with our manual intervention. And these are the types of issues that Crossplane aims to fix. Crossplane is an open source tool that is installed directly on top of Kubernetes as an add-on. And we can quite literally make use of any Kubernetes distribution, even the minimal distribution such as Minikube, MicroK8s, or clusters that are created using the kind utility or the QVADM tool. Once installed, Crossplane enables orchestrating infrastructure or managed services on multiple platforms such as GCP, AWS, Azure, Alibaba Cloud by making use of the Kubernetes API. And this is done by the means of providers. We'll discuss what providers are in detail in the upcoming slides. The platforms that are supported are not just limited to cloud providers alone. Crossplane supports several providers such as Helm, GitLab, Argo CD, etc. And you can also package your own provider for a custom platform as long as it has an API that supports Create, Read, Update, and Delete operations. Once Crossplane is installed and the provider configured, we can now define the configuration of an EC2 instance in AWS using a YAML file like this. And most significantly, we can make use of native Kubernetes commands such as kubectl create or kubectl apply to provision the instance. To check the status and check for the events of the resource that was created, we can run commands such as kubectl get or kubectl describe like this that we would normally do with native Kubernetes objects such as spots, deployments, persistent volumes, etc. And as you can see here from the output of the kubectl get command, we can now see the status of the AWS instance. And similar to native Kubernetes objects, controllers constantly check to see the status of these external managed resources and reconciles them in case of a drift. We will see more about these in the upcoming slides and then the demo towards the end. So let's now see how Crossplane works and talk about some of its fundamental concepts. So we already talked about how Crossplane can be installed on any Kubernetes cluster. Typically, it is installed using the official Helm chart in a new namespace called Crossplane System. And once it is installed, we can see the deployments and the associated pods running in the Crossplane System namespace. We can also optionally install the Crossplane CLI tool that extends the native kubectl tool, enabling it to do things such as installing, building and pushing Crossplane providers in packages. In the upcoming slides and demo towards the end, we will make use of this to install the AWS provider configuration in our Kubernetes cluster. Earlier, we touched upon the concept of providers a little bit. Providers in Crossplane are similar in function to providers in Terraform. They facilitate Crossplane to provision the respective infrastructure resource on a specific platform. However, in Crossplane, provider is a package that bundles two components together, a controller, which runs as a pod in the Crossplane System namespace and a set of managed resources that are installed as custom resource definitions or CRDs. The managed resources are the fundamental building blocks in Crossplane that has one-to-one mapping with a resource in the external system. For example, if we take the case of AWS provider, when it is installed, Crossplane also installs CRDs for instance, RDS instance, security group, subnet, VPC, bucket to name a few, each of them corresponding to the actual service within AWS. Now, depending upon the platform, the corresponding provider must be first installed on the Kubernetes cluster before we can start provisioning and managing resources on that platform. Now, this can be easily done by running a one-line kubectl command provided that the Crossplane CLI plugin is already installed. Now, in this case, we are installing the AWS provider with the version of 0.27.0 in our Kubernetes cluster. Once it is installed, we can see that the controller is installed as a deployment in the Crossplane System namespace. And we can also now see a bunch of CRDs installed on the cluster corresponding to managed resources in AWS. Now that the provider is installed, we need a way to authenticate to the platform. So in this example, Crossplane should be able to authenticate with the AWS account where we want to deploy the resources. For AWS, this is done by creating an account key file that contains the access key ID and the secret access key, and then creating a Kubernetes secret in the Crossplane System namespace using the key file. Now, this is different for every cloud provider, so the steps should vary for each provider and make sure that you refer to the provider documentation before setting up the provider config. Next, we can create a provider config type object to configure credentials for the AWS provider. Now, this object references the Kubernetes secret that we just created, and it facilitates the authentication with the AWS account. We are now all set to provision infrastructure in AWS using Crossplane. And to understand this better, let's make use of the same example of the EC2 instance that we saw in the previous slides. The object configuration for this managed resource in its simplest form would look something like this. The API version for this object is ec2.aws.crossplane.io with the version of v1, alpha1, and the kind is instance. Now, under the metadata, we have the name of the object which is called web server in this case. And just like any other Kubernetes object, we can also add additional fields such as labels and annotations under the metadata. Now, under the spec section, we have fields to define the desired state of the resources. The for provider field is a mandatory field, and it is used to define the desired state of the external resources, which in this case is the EC2 instance. The only required instance parameters are the region and the image ID, which have to be supplied in this definition file. There are several other parameters that can be supplied such as the instance type, security group, and subnet information, but these are optional. And if it is not supplied, a predefined default value are used while provisioning the resource. Another field that we have used under the spec section is the provider config reference field. Now, this is used to reference the provider config object that we created earlier, and it is used to authenticate crossplane to the AWS provider. And that's it. Once this configuration is applied, we can now successfully create EC2 instances or any managed resource in AWS. So how do we know the different required or optional fields under the spec section that can be used for a managed resource? For this, we can make use of provider-specific API documentation and look up the managed resource that we want to provision. In this example, we are searching for the CRD of kind instance within the API documentation page for the AWS provider. Here we can look for all the fields that are supported under the spec section for that specific CRD. And the documentation shows that the for provider is a required field. And the field called image ID is required for the instance type object, whereas fields such as block device mappings and CPU options and things like that are optional. We'll see how to make use of this in the upcoming demo. Before we head over to a demo, let's look at one other fundamental concept in cross-plane and that is composite resources. On the previous slide, we saw how to provision a single managed resource in AWS, which is an EC2 instance. However, while provisioning infrastructure in the real world, you would usually need to deploy several resources together with some resources depending on one another. For example, we may need to provision a VPC, a subnet, a security group that will then be used by the EC2 instance. Now, each of these is an individual managed resource that has one-to-one mapping with the actual services on the AWS cloud. So instead of provisioning them individually and writing YAML files for each of these resources separately, they can be grouped together into what is known as a composite resource with a shorthand of XR. Now, in the interest of time, I will not be going over composite resources in detail, but I wanted to mention it all the same as it is a concept that promotes effective collaboration between the platform or the SRE team that manages the cross-plane environment and the application team that requests for infrastructure. At a very high level, this is how it would work. The composite resources are configured by the platform or the SRE team that manages cross-plane. And while designing the composite resources, they define which managed resources should be created by the composite resource. And this is done by using another cross-plane-specific resource called Composite Resource Definition, which is XRD for short and Compositions. Now, the XRDs and Compositions are used to define things like which managed resources should be created by the composite resource and which parameter or parameters for the composite resource should be fixed and which one should be user configurable. For example, it can be enforced that the instance type of an EC2 instance resource be always be set to T2 micro, but the image ID can be something that the consumers of the composite resource can configure themselves to any AMI of their choice. So, who are the consumers of this composite resource? Now, these can be an application that typically does not have access to create the composite resource directly, and instead, they would use something known as a Composite Resource Claim. This is similar to the native Kubernetes object called Persist in Volume Claim, when application can request a Persist in Volume of specific class, size, or access mode. Similarly, when a composite resource claim is created, it would contain the reference to the composite resource along with the user configurable parameters that are configured for the composite resource. All right, now that we have gone over the basic concepts of crossplane, let's jump into a quick demo where we will use crossplane to provision a bunch of services in the AWS cloud. Now, for the ease of use, I'll be making use of code cloud crossplane in the Playground environment that has a single node Kubernetes cluster and crossplane installed already. And it also has an integrated AWS cloud playground that we can make use of to deploy our resources. So here I'm making use of a crossplane Sandbox environment which already has crossplane installed within a Kubernetes cluster. And it also has a short-lived AWS environment integrated that I'll be making use of to deploy resources. Now, I want to point out that you can pretty much use this on any Kubernetes cluster, even a local cluster running within your laptop. So let's first inspect this environment. So you can see here that this is a single node Kubernetes cluster that is running the version 1.24. Now, let's also inspect the namespaces which are available here. As you can see, the crossplane system namespace is already available within this cluster. Let's inspect all the objects which are created within this namespace. As you can see here, there are deployments for the crossplane as well as the crossplane provider for AWS. This means that the AWS provider has already been installed. And to validate that, we can run kubectl getProvider. I can see here that the crossplane provider AWS is installed and it is in a healthy state. Now, this environment is integrated with an AWS sandbox which makes use of a limited user with limited permissions so that we can deploy certain resources for this particular demo. Now, let's open up this console and see what is currently configured for this particular account. So we have opened the AWS console. Now, let's go back to the terminal and let's create the key account file that we can make use of to create our provider config reference. Now, this has already been created within the system and we can make use of this to create our covenanted secret that we will use to authenticate from the crossplane to our AWS account. And so to do this, we will use the kubectl createSecret command and pass in the key file. So now that the secret has been created, we can create the provider config reference. And to do that, we will make use of the provider config.yml file. And as you can see here, it's referencing the secret that we just created called AWS credits with the key of credits. So let's create this object. So now that this particular object has been created, crossplane can authenticate to our AWS account. So we can now go ahead and create resources in AWS. So I've created some sample configuration files in YAML for the managed services that we want to create inside the EC2 directory. And here you can see that we've got different YAML files and these are the resources that we are going to create. So we're going to create a VPC, a subnet, a security group, a route table, an internet gateway and a route table, and finally the EC2 instance. So let's go back to our AWS console and first take a look at the VPCs which are currently available in the Virginia region which is US each one. And here we can see that there is one single VPC right now in this particular region. So let's go ahead and create a new VPC. Now, before we create, let's just inspect this YAML file. So the API version for this is EC2.aws.crossplane.io which is the API group and the version is V1, beta one and the kind is VPC. The name of the VPC is going to be sample-vpc and under the spec section, we have four provider field which is mandatory and within that we have parameters which we have defined. The region is going to be US each one, the cider block is a custom one that we have selected which is 10.0.0.0 slash 16 and we have also opted for some additional parameters such as enable DNS support is equal to true, insistency is default. Now you'll also notice that we have supplied the provider config reference here which is the name of the object that we just created which is called default. Now let's go ahead and create this VPC using kubectl create command. So now the VPC has been created. Let's inspect the status by running kubectl vpc and you can see that the status is ready and it has synced. So let's go back and refresh. As you can see here, the new VPC has already been created. So similarly, let's go ahead and create the rest of the resources. So I'm going to first create the subnet. So using the same command kubectl create dash f subnet subnet and next let's create the internet gateway followed by the root table. And finally, let's also create a security group. But the security group is going to be called sample SG and it has a reference for the VPC that we created and a firewall rule of type ingress or to the port 22 from anywhere. So let's inspect the status of all the managed resources that we have created so far. And so here we can see that we have created the VPC the subnet, the internet gateway, the root table and the security group. Now all that is left is for us to create the instance which is the EC2 instance. And in here we can see that the name of the instance is going to be demo dash EC2 instance and under the for provider section we have the mandatory parameters which is region set to US East one and image ID set to Amazon Linux image MI. And we have also optionally selected for creating an EBS volume of size 30 GB. And here we are referencing the security group that we just created called sample SG and the subnet that we created earlier. So let's go ahead and create this instance. But before that, let's also go and check the image ID that the instance tab that is going to be created for this instance. So here we have not supplied instance type for this particular managed resource. So it will offer a default instance type which has been configured for this managed resource and a CRD. So to see that we can actually go to the cross-plane documentation and in here navigate to API documentation and go to provider AWS. And we are making use of the version 0.27.0 and in here let's look up the CRD that we are looking for here which is instance. And if you click on that under the spec section we can see that the for provider field is required and it only has two required parameters which is image ID and the region. Now the instance type is optional and if we do not supply it, it will by default create an instance type of M1 small. So let's go ahead and create this instance. Now let's inspect the state. I've just run a watch command here and run cube CTL get instance. So let's wait for it to be provisioned. So after a while you can see that the instance state is in pending state. Let's go back to our AWS console and navigate to the EC2 dashboard. And if I remove the filter here you can see that there is indeed an instance that has been created with the instance type of M1 small. Now once it's ready we can also try and delete it from the AWS console and once that is done you should be able to see that crossplane continuously monitors it and sees that it has been deleted and it'll try to bring it back up so that it reaches the desired state. So you can see here that the instance has been terminated and here the instance state is reflecting as shutting down and soon crossplane will bring up another instance to bring it back to the desired state. There you go, you can see that the instance state is now pending, a new ID has been created. I can see here that another instance has been spun up in place of the old one. Well, that's it for this session. I would like to thank each one of you for attending this talk. I hope that you found this session interesting and useful and a special thank you for KCD Chennai for hosting me. Thank you.