 Hello, everyone. Welcome to the Cluster API Provider AWS Project update. I am Sadaf Sawaj. I'm a software engineer at VMware Tanzu team. I am one of the maintainers of Cluster API Provider AWS Project. I've been working on this project over a year now. Hello, everyone. My name is Richard Case. I'm a tech lead at Weaveworks. I'm currently working on bare metal Cluster API based products and solutions for Weaveworks and also a Cluster API Provider AWS maintainer. This is our agenda for today. We will give you an overview of Cluster API Cappy and Cluster API Provider AWS Kappa. They are both C cluster lifecycle sub-projects. Then we are going to mention about some new features in Kappa and do a quick demo to give you an idea about how all this works together. Then we will talk about our roadmap and last we will mention how you can get involved as a contributor. So what's Cluster API and what does it do for us? Kubernetes lifecycle management is complicated with day one and day two operations and Cluster API brings Kubernetes style, declarative APIs to simplify this management. It manages Kubernetes clusters again by Kubernetes clusters and you see the Cluster API logo on the right. It is a reference to Tartus all the way down because we are using Kubernetes clusters to manage Kubernetes. In general, Kappa brings some automation to the management of the clusters in terms of creation, skating, repair, upgrades and deletion. You can create your Kubernetes clusters on either on-prem data centers or cloud providers and each infrastructure provider has its own way of provisioning resources like virtual machines, load balancers, etc. And Cluster API is infrastructure agnostic stuff to support these various infrastructure providers. It acts as a building block and provides interface for providers so that providers can implement these contracts and gets plugged into Cluster API so that users can provision clusters on them. Currently, we have provider implementations for all the logos you see on screen like AWS Azure vSphere. And if you want to see the whole list, you can check the link below. If you do not find your provider in the list, you can always build your own provider. There is a section in the Cluster API book that shows the contracts for adding a new provider. And I'm going to, no, next slide, please. Yeah. And Kappa has lots of cool features. If you want to know more in detail, there's a deep dive session on Friday. I suggest you see that session if you want to know more. And now I'm handing over to Rich to talk about Cluster API architecture details. This diagram represents the high-level architecture of how Cluster API works. There are three main portions to it. On the left-hand side, we have the cluster's definition as YAML. In the middle portion, we have the management cluster. And on the right-hand side, we have the resulting clusters that are created or the tenant clusters. So everything starts with a definition of a cluster. And this is created using YAML. This is essentially like you would declare the applications that run within Kubernetes, but for your cluster. The actual YAML was made up of a number of different kinds. And on the left-hand side, you can see a color-coded diagram that represents this. So the purple is the kinds that are provided to us by Cluster API. There's an upstream project. And then the pink and the orange are implemented by our providers. So essentially, you have to start with that. You have to create your cluster and its definition as YAML. This is applied then to a management cluster. So this is a core concept to Cluster API. You have to have a management cluster where you apply these definitions into. The role of the management cluster then is to run the controllers from the various providers and from Cluster API as well and provision the clusters, the tenant clusters on the right-hand side. So essentially, this is a two-step process. So the first step is it provisions the infrastructure in the target environment. So whether that is creating as your VMs or with an AWS, creating VPCs and EC2 instances. So the provider will need to provision that infrastructure first for Kubernetes to run on. Then a Bootstrap provider is used to bootstrap Kubernetes on that infrastructure and to create the cluster for you. So you can see within the management cluster, you can see the color coded representations of the various controllers for the different providers and also Cluster API itself. When a cluster is provisioned, essentially it will also store a kube config for that cluster within the management cluster and you can then use that to connect to the newly created cluster. So what is Kappa? First and foremost, Kappa is an infrastructure provider. What that means is it will provision the AWS infrastructure that is required to run a Kubernetes cluster within AWS. It provisions the infrastructure in an opinionated way. So you don't have to specify every aspect of the infrastructure. You can just let the provider take that opinionated view on what infrastructure it should create. Obviously that doesn't fit everyone's needs. So you can bring your own infrastructure. So you can provision your infrastructure via some other means like Terraform or Crossplane and plug in the details of that into the Kappa manifests. When creating clusters with Kappa, you can create two types of clusters essentially. So one we call unmanaged and that is basically an easy to instant space with a kube ADM bootstrap. And more recently, you can create what we call managed clusters which are essentially BKS clusters under the cover. So going back to the diagram we saw previously. So what I've taken here is essentially the representation again with the purple representing the cluster API resource kinds and overlay that with the Kappa resource kinds. Now this doesn't show every single resource kind that you can use within Kappa, but it shows you as representation. Main things to note here is on the left hand side, all clusters are provisioned with the set of credentials and this will be covered a bit later on within the talk. Then when we come to the cluster, we have an AWS cluster representation. One thing to note here is when you're using a managed cluster the AWS management control plane represents the infrastructure and the control plane which is slightly different to an unmanaged cluster. We look towards the bottom of the diagram, you see representations for the actual worker nodes. In this diagram we have machines and machine cores but they might also be machine deployments. And what you can see from this is within Kappa the resource kinds are AWS machine, AWS machine pool and AWS managed machine pool. And you'll see for managed clusters that sits across all of this is a bootstrap type and this is essentially for for managed clusters. This is called EKS config. Yeah and now I'm going to mention about a command line utility called cluster AWS ADM that we publish with our releases. It has some helper functions for Kappa. For example cluster AWS ADM bootstrap command manages identity and access management objects like roles policies and these are used by clusters and Kappa controllers. Also after every Kubernetes release we build and publish AWS images, AMIs using another Kubernetes subproject called image builder. And these AMIs comes with packages that's needed to bootstrap the node. By default when you create a Kubernetes cluster with a specific version Kappa picks the right image depending on the version from an AWS account. And to list all available AMIs for specific Kubernetes versions and OS distributions you can use cluster AWS ADM AMI list command. As a side note by the way we also support custom AMIs created by users. On the Kappa managed EKS site we have cluster AWS ADM EKS command to list available and install that once. Yeah so let's talk about some new features of Kappa. So in general we release minor releases every six months or so and a patch release almost every month. We are currently at version 0.6 release which was released last September. And I'm going to mention about some new features in 0.6 release which is common for bot managed and unmanaged clusters. The first one is multitenancy. Previously Kappa controller used to use same set of credentials. It is initialized with to reconcile all the workload clusters. And this was not suitable for especially large organizations where they would prefer to run their management cluster in a management AWS account. And each team has their own AWS accounts to run their workload clusters. So we introduce a multitenancy support with this release. Kappa can now do an IAM role assumption at runtime on a per cluster basis. And each workload cluster can pick and choose a different principle to be used during reconciliation of that cluster. And you see the CRDs we introduced to support multitenancy different kind of principles controller principle role principle and static principle. Also we have a new controller for backward compatibility for the existing clusters because we need to create a controller principle to make the previous clusters work, continue to work. And the next feature is AWS machine pools. So we added this feature with this release and it is responsible for creating and managing AWS auto scaling groups using a launch templates. So users don't have to manage individual meshes. So let's continue with our demo. All right. So before even creating our management cluster with Kappa and Kappa controllers, we need to create some IAM resources. And we use a cluster AWS ADM for this. We create as you see here a cloud formation state with all the necessary roles, instance profiles, and policies. And these resources can be customized by passing a content file to cluster AWS ADM. In this demo, I'm going to show a use case where management cluster is running on a management AWS app and a team has a different account that wants to bootstrap a workload cluster using a workload role. So on the left here, you see I created a management role. This role will be used to bootstrap our management cluster. So our Kappa controllers will use this role. You see this management role has the necessary permissions to manage AWS resources for our clusters like EC2, ELB, et cetera. On the right-hand side, do you see the workload role? This is the workload AWS cluster, sorry, the workload AWS account. I created the workload role here, which can be assumed by the management role on my management AWS account. It also has the necessary permissions to reconcile the resources necessary for our AWS cluster. Here you see. Let's see how we represent these roles in our workload cluster manifest. This is a simplified version of the workload cluster. You see a controller principle here, which represents the principle our management cluster is bootstrap. In our case, this is the management role in our management account. And then this role principle represents the workload role on our workload AWS account. We also provide details about how this role will be assumed with source principle. For our case, we want management role to assume this workload role. So we added this controller principle as source principle here. So for the sake of time, I already created our management cluster and a workload cluster. Let's see what controllers our management cluster has. So these are all the CAP and CAP controllers in their own namespaces you see here. Let's check out our cluster objects. We have a CAP cluster that's referred to the infrastructure provided cluster object, which is a AWS cluster for us. And here in the AWS cluster object, we specify infrastructure specific things such as networking. And also we provide a principle ref here. This principle ref will be used to create infrastructure resources for this AWS cluster. Let's take a quick look at the resources created by CAPA for a workload cluster. So here you see a VPC created for the demo cluster. And when we check our subnets, we CAPA controller created a public and private subnet, private mostly control planes and workload nodes are located in the private subnet. And load balancer is located in the public subnet. But these are all configurable. We create some route table rules, Netscape-based security groups, et cetera. And if we go to our EC2 dashboard, we see the API server that came up on the public subnet. With, let's check the instance, with our single control plane instance. When we check our running instances, this is our control plane, workload control plane instance. We also have security groups created for this workload cluster here. I'd like to show one final thing in our YAML file, which is machine pools. So you see this is the cluster API construct for machine pools, which refers to AWS machine pool. And here, AWS machine pool has details about how we want to create our launch templates for auto scaling groups if we go back to our console. So this is the machine pool created by CAPA controller with this launch template. With the specific instance site we provided and key payer name, et cetera. And when we check, again, our EC2 instances, this is the single instance, because we set the desired instance to one in our machine pool specifications. This finalizes our demo. And here are some more features that become available with 0.6 release. First one is we add a support for using spot instances in this cluster. Spot instances are not reliable EC2 instance types that are cheaper and more suitable for fault-tolerant workloads. The second one is we started watching for EC2 instance state changes using EventBridge and simple queue services. Previously, when there's an EC2 state change, CAPA was not detecting it until the next reconcile which could take up to 10 minutes. Now, when a change is detected, it triggers AWS machine reconcile. Also, in this release, we added support for dedicated instances which are basically EC2 instances that run on hardware that are occupied by a single tenant. Also, we added AWS request throttler logic because AWS throttlers API request for every AWS account on a per region basis. And we added a logic to stop our service request when we detect AWS starts throttling us. And now, I will ritual mention about new features in EKS. So, on the managed side of the provider, EKS support is new in the 0.6 release. So, this is still marked as experimental and you actually have to enable it explicitly via the EKS feature flag. So, there's two main CRDs that you need to create a cluster, EKS cluster. First one is AWS managed control plane. So, when you create a unmanaged cluster, you have a AWS cluster and you have a CUBE ADM control plane. Within the managed side of the provider, you only have the AWS managed control plane. It acts as both the cluster and the bootstrap provider combined into one. For the worker nodes within the EKS cluster, we support three types of worker nodes, essentially machines which are EC2-based instances and these are individual machines. So, you individually declare the machines you want to add to the cluster. And from a declaration point of view, you use the CAPI machine kind and the AWS machine kind when you're declaring these. So, there was already mentioned about machine pools and machine pools can be used within an EKS cluster and these again are essentially EC2 auto scaling groups. From a EKS point of view, you have to use the machine pool CAPI declaration and also the AWS machine pool. But in the EKS side, we also have another variant of the machine pools and these are called managed machine pools. If you're aware of EKS and use EKS normally, these get represented within as EKS node groups, essentially. To use these, you need to use again the CAPI machine pool, but then there's a different kind from a CAPI point of view and that is the AWS managed machine pool. If you want to use either variant of the machine pool, you must make sure that you enable the machine pool feature flag. And you can use all three of these within an EKS cluster. So, you can mix and match and have different types of working as within an EKS cluster. You just need to create the declarations and reference the managed control claim. So, we also support per cluster IAM role creation by default when you use cluster AWS ADM, it will create some default roles for use with the control plane and the worker nodes. That may not be suitable for your fuel type of scenario. So, you can enable via a feature flag that there is IAM role creation and then there's IAM roles per cluster. And within that cluster, you'll have an IAM role for the control plane and the worker nodes. So, that's something to be aware of. Also, if you use the EKS, you'll be aware of the way that you authenticate and authorize into the cluster. That's using IAM authenticator. So, the AWS managed control plane allows you to declaratively map users and roles that will result in the configuration, the required configuration changes within the cluster for AWS IAM authenticator. We also recently added support for EKS add-ons. There's only one add-on present, which is the VPC CNI. But this enables you to declare what add-ons you want within a cluster and then also upgrade the versions of those add-ons in a declarative manner. We've also added support for Fargate profiles. This is in 0.65. And this allows you, essentially, to attach a Fargate profile to your cluster and then effectively run specific pods within that Fargate profile. At the bottom of the screen, you'll see that they... How you install the GAPR provider into your management cluster. A few things to note. We've talked about feature flags quite a lot. There is a convention about how you can enable those feature flags via environment variables. And in the bottom of the screen, you'll see some examples of that. So, what does the YAML look like from a managed point of view from the EKS cluster point of view? So, here's two examples. On the left-hand side, in the box A, you see an example of the AWS managed control plane. So, this represents the cluster and the bootstrap, like I mentioned before. I want to point out a few things here. Firstly, we have that principle ref. So, a link to the credentials that are going to be used when provisioning this cluster. So, which account and which user, etc. We specify which version number we want to use within the cluster. And then, as I mentioned just previously, we have the section for add-ons. So, this is where you declare what add-ons you want within that cluster and specific version numbers. And this is where you would edit this area if you wanted to upgrade to a new version of the add-on. Finally, within this example, you can see that I'm mapping a role. I'm mapping the administrator access IAM role to the system master's role within Kubernetes or group within Kubernetes. So, this is where you do that declarative configuration of AWS IAM Authenticator. On the right-hand side, you'll see an example of an AWS managed machine pool. So, this is very similar to what Sedef showed in the demo. But this is just for the managed machine pool side of things. And you can see it's very simple. We are interested in two properties here, the min and the max size of that machine pool size. So, what does the red map look like for Kappa? So, we essentially have two versions of Kappa at present. So, we have the 0.6 release version. And this maps to the Kappa V1 Alpha 3 version. And you should consider the stable option. So, use this first, especially for any production-like scenarios. We're also currently working on 0.7. And this is compatible with Kappa V1 Alpha 4. Focus really with this is on stability and UX changes in the API that eventually going to lead to the beta release or the beta graduation of Kappa and Kappa. When talking about these two versions, you should note that we are only going to back port bug fixes and security vulnerabilities from 0.7 to 0.6. Yeah. So, we are still working on clarifying the features that will be in for V0 7 release. We will continue to focus on improving stability, that's for sure. But these are some of the features that are coming up. For the unmanaged site, we plan to extend our spot instance support by also supporting Amazon EC2 release and spot release. Also, we will verify using GPU EC2 instance type as workload nodes. So, for that, we will probably add conformance test that uses NVIDIA GP operator, which is basically responsible for installing necessary drivers for Kubernetes GPU support. And finally, we will add ignition support and start supporting ignition format for our bootstrap data instead of only using cloud in it to run our bootstrap script during instance initiation. That's all on the unmanaged site. So, on the managed side, we're focusing on three main areas. So, the first area is to implement the OIDC provider association. So, this feature was recently introduced into EKS. So, we're implementing that so that we get to the point of having feature parity within EKS. We're looking at a change in the way that we handle launch templates within the managed side. Currently, we generate a default launch template, but we don't actually allow you to configure all of the options within a launch template. So, this will become an alarm. From an API point of view, we are going to refactor the AWS managed control plane and essentially deprecates the AWS managed cluster. So, currently, the AWS managed control plane has grown organically a bit in all honesty, and we're going to change it and refactor it so there's a bit more structure to it. So, it's obvious when you use it, what options you should be using. So, how do I get started? So, first and foremost, try Kappie and Kappa. Kappie is a project that creates a quick start guide, and within that quick start guide, you can follow it through and choose which provider you want to use in order to get you from the point of creating a management cluster to creating your first tenant cluster. Once you've done that, you can then look at the Kappa-specific book that we publish. This has various topics that are specific to Kappa that will be useful, like different scenarios, how to contribute in our release cycle and stuff like that. Then, if you're still interested, come and look at the code, come and look at our repo. It sits within the Kubernetes SQL within GitHub, and so you can come and take a look at the issues, the releases, and the code. We also have our own channel within the Kubernetes Slack called cluster-api-aws, and we also have office hours that you can attend and listen to where we discuss issues and the roadmap. These are bi-weekly on a Monday at 10 a.m. Pacific time or 6 p.m. UTC. So, how can I contribute? So, there's a number of ways you can contribute. It's not all about coding. So, the first way, do you have right of skills? So, every project needs documentation, quick start guides, architecture diagrams, and so on. For the project to be successful, we need documentation. So, if you have those skills, we really welcome you to come and join the project. Do you have product skills? Can you translate actual real-world scenarios into requirements and issues that need to be implemented within Kappa? Can you help with prioritization with when we're planning a new release? These skills are really, really important to make sure that we deliver on what users want. Do you have coded skills? It's not all about actually cutting code. We need people to review pull requests. We need people to investigate issues and try and recreate them and create really useful tickets for us. If you want to code, that's great. We need more people to code, and you can look at the issues and see the ones that are tagged as a good first issue or help wanted. That's a great place to start. We also require people to improve our test coverage, whether that is the unit test or the end-to-end test. There's lots and lots of other ways that you can help with this project. Just come along and get involved. Thank you all for attending.