 So we have I think like 45 slides three demos and two people and 35 minutes So I think we need to just jump on into it and keep the pace going pretty quickly We're gonna try to make sure that there's time at the end for questions also Because typically with these sessions we've had lots and lots of questions We want to make sure that there's time for that too So yeah, so this is the cross-plane the projects introduction session and deep dive session So it's you know kind of a combined thing which is always interesting because we get to cover the full spectrum of cross-plane stuff But you know for people that are already already familiar with it There's kind of some introductory material you may have heard a few times by now And then folks that are new on their cross-plane journey might get a little lost later on But we'll try to keep everybody all together through this whole thing So my name is Jared one of the creators of the cross-plane project and I have my my friend here Chris Christopher Who's a prolific maintainer on the project as well? All right, so introductory material Let's talk about what cross-plane actually is and the best way to think about it is that it is a framework In order to help you build a cloud native control plane So, you know, we talked about control planes You know cloud providers have been managing their infrastructure with control planes for years and years, right? That's not a new concept itself but we wanted to be able to a You know create a framework so that all of you could build your own control planes to manage your Infrastructure and your resources etc and inject your own opinions into it as well You know and have a nice framework to do that with so kind of think about the top and the bottom of the cross-plane project So on the bottom here, there is you know, you can manage any type of infrastructure It's very extensible there so you can talk to any cloud providers on premises stuff It's very extensible and on the front end you can you know bring together There's resources and kind of declare your own platform API is your own infrastructure API's So highly extensible on the both the front and the bottom So just a quick history about the project. So this is the CNCF project, right? That's why we have this maintainer track talk and we're looking at it as a neutral place for companies and you know organizations Individuals etc to all come together and work on this mission of enabling cloud native control planes So we've been doing this for a little bit. We started this project. The first code was written back in 2018 So it's been a good You know for four years or so plus We donated it to the CNCF though in 2020. We got to a 1.0 milestone later that year We are now at the incubating stage of the of the CNCF We've got a 1.12 release coming out on Tuesday So just a few more days until that comes out there's some exciting stuff in that and then we are hoping to continue progressing towards a full graduated status with the CNCF Hopefully later this year So there's definitely some help that we could use from folks that are using the project successfully getting in production Ensuring your story with us in the adopters file So there's just some numbers project still continues to grow You know people are starring at people are contributing to it that my favorite stat of the whole thing though Is the top right one where you know over 1500 people have contributed to the project in some way now You know that's pill requests. That's issues. That's you know all sorts of things of Many people coming together with this mission of cloud native control planes, right? So I love that and that's my favorite stat by far So I'm gonna turn it over to Christopher here to get started with you've got an introduction so far We probably still don't really know what cross paint does from what I've said So let's start at the bottom layer and let's work our way up So let's talk first about the fundamental basics of cross plan We have here something called managed resources and this could be anything outside of a Kubernetes clusters if it has an API so Let's have a first look in one of the providers we have in the ecosystem It's for AWS and as you can see here, we can manage Hundreds of objects outside of the Kubernetes cluster. So to mention a few it's like networking databases Kubernetes clusters and We can managing them all in the Kubernetes way and that's also the idea from cross plane So have a look now on the right side. You see the AWS console for a three bucket And normally you have for a three bucket something like a region you get an Resource arm for the bucket you have a creation date and you also have something like attack on your S3 bucket And if you're looking now a little bit on the left side, this is the 100% Kubernetes resource View from the S3 bucket so you can see we have here the API version and the kind For this S3 bucket and we also using the versioning for our APIs We have also full metadata support for labels and annotations and we have here the spec and the spec is our Our thing how the object in the API looks like so our special stanza here for cross plan is the for provider section And everything what is under for provider we shipping to the API So if you're creating something we using all the The fields here and send it to the API This for provider thing here is high fidelity. That means if an resource in the cloud provider has hundreds of Configure things Then we have also hundred of configuration options here in the YAML file And it's a two-way communication when we're talking here from cross plan that means if you creating this bucket now in the API then we have also return values from the API and we Publishing these return values under the status at provider and in the case of an S3 bucket you can see here we get an arm and We updating this also in Reconciliation loops every time what we're getting from the API is we publishing here and the ads status provider Then we have something like events. So every time we are communicating with the API We saving the things in the events So in this case you can see it was successfully created But you can also thinking about if the authentication is not working in the API then you get in message here And the cool part is you can because it's Kubernetes you can have all your current Tools you're using for Kubernetes Adopted to the things here because events and so on. It's the same like normal resources in Kubernetes So look a little bit deeper when we are installing our providers in the Kubernetes clusters the first thing what happened is that cross plan installed the CRD's in the clusters and What happened then is for example, if we want to apply a three bucket then we have controllers for groups of resources and if you're applying on the left side now our S3 bucket in the Kubernetes cluster then the S3 controller watches on the API server and gets a feedback that there's a new resource available in the cluster for S3 bucket and directly starts talking with the S3 API from AWS and then it's thinking about is the object currently available So there's three bucket in the API if it's not available. It starts directly creating the resource in the API and if it's Currently available on the AWS side then we calculating a difference and Applying it and starting to manage it fully So if you're looking a little bit more in our providers in the internal stack then on the very bottom level what we're using here is Kubernetes runtime so that our our provider or controller Can be run in the cluster and it's also taking care of all your things what you're using like ingress services local answers in front of her services and If you're programming with Kubernetes then you have something called API machinery So what we're using here very very heavily is the custom resource definition and the open API spec What we're using here and the cool part here is that you can create your Custom types in Kubernetes and then it's acting like a normal resource in Kubernetes You're knowing like Qt city. I'll get update delete everything is working like with your normal resources Then we have the controller runtime. So it helps us for the reconciliation part so like create update delete events and the controller knows then what to do and What we have here from the cross-plane community part is the cross-plane runtime So because we creating not resources only in the cluster outside of the cluster and for this we have here to create update and delete thing and On top of the runtime we have custom logic. This could be everything. Yeah self-coded Create update and delete methods, but this could be also very very automatic what we're providing in the cross-plane ecosystem All right, so, you know, that's kind of showing the what I can kind of refer to as the bottom layer of cross-plane like You know you can create infrastructure and cloud resources, etc And cross-plane will go and make that happen out in the real world, right? But that's still that's a control plane But I think there's a lot more power that comes out of cross-plane when you can start injecting your own opinions in and you Can start building your own platform API essentially So the whole concept here is that you take a lot of those Managed resources that Christopher was referring to you can kind of assemble those together and Expose that as a higher level abstraction. You know a kind of a self-service Infrastructure API you can think about it. So a tangible Example here is going to be what does it take to expose a workload cluster an application cluster for your developers? Let's say and so you you could for example take together GKE a node pool the networks subnetworks all this sort of things You know some some roles some security policies blah blah blah You can compose that all together and then offer that as a very simple cluster abstraction to your developers So they could self-service, you know create a workload cluster if they needed one this idea extends You know across the entire ecosystem of cloud provider resources So it's you know not just about creating clusters and managing clusters, right? It's about databases caches Message queues buckets like all that sort of stuff there too, right? So this becomes a very powerful concept when you can you know essentially build your own opinionated cloud native platform You know it like we've seen before that that's it's declarative, right? It's mostly YAML configuration, right? You don't have to write any code But we're actually showing a new demo today in which you can inject code in to make it do more things that make it more powerful So here's a little picture to kind of put a little visualization to what we're talking about So if you work your way left to right in this diagram here The developer is on the left, you know, she's eating her popsicle and she's ready to kind of get ready to provision some infrastructure for herself So you know she can create a simple, you know the simple abstraction from your platform API that you've defined for her She can tune a couple of configuration knobs on it and then you know underneath the covers That's going to go to a composite resource and you know like what does that definition of that at runtime mean? That could be a whole bunch of managed resources put together So really we're kind of going left to right of a simple API and underneath the covers this environment complexity this policy these guardrails This configuration etc and a strong separation of concerns Between the developer getting their infrastructure and you as a platform engineer that's kind of design this platform for them You know another tangible Description here of you know developers the interface they get is they want a small postgres database That's what they get. That's what they get access to that's that's all that's exposed at that layer underneath the covers, right? We've got a composition for AWS. We could have one for GCP. We could have you know a golden a silver doesn't matter There's multiple different compositions that you can implement for one particular Infrastructure API and then we see here that that folds out into RDS Parameter groups the DB security groups all that sort of stuff So this is like this is kind of what the YAML looks like here We're not going to dive into every line of this but you can think about you know It's time to define your infrastructure API and so in crossplane you do that by creating a composite resource definition So you're essentially in here defining you're saying cool. I want to create a new postgres instance for sorry a new postgres API for my developers and then you're gonna go ahead and go ahead and define the scheme of it This is the shape of that API. This is what the developers are allowed to configure what they're allowed to set And this is the API for your opinionated control plane that you're designing right now Underneath the covers you've got to say at runtime. What does it mean when a developer creates an instance of this? Simple postgres API. I've given them. What does that actually mean? And then you can start specifying all the resources that they compose that API underneath the covers and you know get more specific about it And then of course you can provide a bunch of patches to it You can take you know configuration values from the user you can flow those down apply them to the resources that you're composing together and essentially You know have something that's configurable and Influensible by the developer who's whose self-service creating this infrastructure But it's still something that you as a platform engineer have designed and implemented and you're taking their their Configuration values and you're flowing it in but it's still you know, they're not you're not giving them direct access to the cloud creator API So you're not letting them do whatever they want. It's but the guardrails that you're setting down so You know we're gonna talk about you know, we've talked about cross plane as a framework, right? And so there's multiple different ways to extend cross planes. We can think about different extension mechanisms So we've you know providers are a way to teach cross plane new tricks of how to Configure provision manage various resources and various different environments cloud on premises, whatever So we call those providers and literally like like Christopher was saying anything that has an API You can write a cross-band provider for it. There's a pretty rich ecosystem of them right now But you know the community is out there and building more, you know every week we see more providers So that's that's one extension point and then when you are declaring and building your cloud native control plane Your opinionated platform you can package those up into a set of what we call a configuration in cross plane It's essentially a bundle a package of your infrastructure API's into a reusable, you know, publishable Deployable unit. So that's the that's a good way to think about extending cross plane So within the ecosystem, you know There's there's a number of logos on there for providers that we have and that continues to grow every day But I think a big takeaway here is that if there are scenarios that are important to you all or environments Or you know providers or whatever that you want to cross plane to be able to support then you know This this community does kind of listen to that when people can rally around an idea or rally around the scenario or use case for cross plane We see new providers pop up and spring up out of that so we have a nice neutral home to collaborate on all that together and Then there's also a place to you know visualize and see and discover and share all these So there's a marketplace around this as well for you know Essentially publishing your providers and extensions and cross plane and you know rally around those as well All right, cool. So that's a lot of stuff that we've covered before So now we're going to start diving into stuff that we have not seen before So this is all like new functionality that has not existed at a previous cube con before so I'm pretty excited about this stuff So this kind of guess is more or less the start of the deep dive session. All right So we just talked about compositions. It's a very very powerful concept in cross plane And you know, we saw what it looks like that for instance, you know You want to expose a simple database Objects or API to your developers and you'll let them configure How many gigabytes of storage is it going to have backing it and that's really all you're going to let them do right? You're not exposing the environment complexity underneath You're not allowing them to do anything more complicated than that. You're exposing to them a simple API Underneath the covers this you know, you can specify a set of resources You can specify how information and config value from the user flows down as patches But that's been somewhat limiting so far, right? We've had composition in cross plane for a while, but there are most certainly use cases that it does not cover So if we think about what are the limitations of composition today in cross plane? Well, you can't do complicated logic stuff, right? There's no flow control. There's no conditionals. It's it's you know, there's no advanced There's no advanced programming Features, let's say right like the list of resources that you're composing together is static. There's there's no real variability to it And also you can't get real runtime information that would influence the these these resources that you're composing together Basically patching and you know transforming and whatnot in composition in cross plane. It's not a programming language, right? We consciously did not want to grow a complicated DSL. That's that's you know expressed in YAML They would you would have to reinvent a lot of things around that and tooling and You know to make that work out Well, it would be half it would need to be very consciously designed with all this functionality in place And I don't think that's the right place in cross plane specifically So to solve this problem in cross plane we've introduced the concept of composition functions so this was as a v1.11 which is in January it was released as an alpha feature and So it's it's it's now available. You can start using it and the feedback we can get right now is really important as well as we Continue to mature it, but essentially what you can do instead of it's like statically defining these resources that you want to compose and You know how you want to apply configuration to them You can instead specify a pipeline of functions to run of actual code that you write So any logic that you want to include for your use case is now on the table You know, there's no limitations about what language to write it in that doesn't matter as long as you can receive You know a a function IO object and do some manipulations and write it back out. That's all you need So you do Python go JavaScript, whatever This opens up the possibilities to do a lot of new things in cross plane that you have not been able to do before And we're trying we're trying to find the sweet spot between declarative no code You can build your opinionated platform with cross plane without ever writing any code We're trying to find a sweet spot between that and you know building and rolling an entire controller and writing You know Kubernetes client code and reconciliation logic and all that stuff, right? So we're trying to find a sweet spot there So what this ends up looking like is that you know We've had compositions before and then that was a set of resources that we're composing together in the patching and Transforming logic, but now in addition to that you can still do that But now also you can specify what functions do you want to execute over these over these resources? What code do you want to call? So for instance, we have a function section here that's you know calls my cool function And then that'll go do something it'll manipulate the resources at runtime and give them back to cross plane to go make it happen Quick architecture diagram basically this whole thing runs as a sidecar Container to the main cross plane container inside the cross plane pod and then the main cross plane container speaks of a GRPC to the function runner container and inside that The default implementation of the cross plane function runner will invoke a set of rootless functions containers to Execute your code and in a series of pipeline pass them along to each other continuing to manipulate the data So that's what it's the architecture looks like But then let's see some running code now and see what this looks like here in an action So I need to make some things bigger on my screen. I recognize that so let's just start making text bigger and Then let's also look at here. Let's make that a little bit bigger too. Okay, so We're gonna look at an example here where we you know This is a classic example that comes up in cross plane all the time where somebody is like okay I have an abstraction that I want to give to my developers and underneath the covers I want to create a variable number of resources I do not know what how many resources that I'm gonna create until at runtime the user tells me to So we're basically looking at a simple claim here for a robot group. This is a fake thing. It's not real, right? There's no maybe AWS robot group service. I'm not sure, but it's not real It's an example of somebody, you know, your developer saying cool I want to make a robot group which is a group of robots and I want five robots Please give me five robots before and cross in composition across when you can't do that You you just can't have it's a static list. You can't have a variable number of these resources, right? So we hear about this all the time, you know for loops and conditionals like if this then create the resource We hear about that all the time So let's see that actually in action now Let's go back to our prompts and I think we still need to make this a little bit bigger probably Let's just see so I want to apply the composite resource definition I want to apply the composition as well and let's see what that composition actually is We look at the composition and it's similar to the example I showed before of when we see a robot group being created What we're going to do is we're not going to have any resources We're defining what we're actually going to do is we're going to invoke a function We're going to call this this Xfn mini function is what I've called it We're going to call this function here to create the number variable number of robots that the users asked for essentially So let me refer to my notes real quick. This is not for you all to see so don't worry about how small it is But yep, okay, let's go ahead and create an instance of that robot group That said what did it say it said five robots or something like that? I think it was right So what should happen right now is that the composition machinery should kick in It should look at the composition. It should see what function to run It should invoke that concept the function within that container That logic within the container should run It should create the these resources and a variable length And hopefully We have a number of five robots. I guess created now which we do which is great So our code executed and we also have some other logic in there that randomly assigns values to them as well Just to show like hey, you can do a lot of things in the programming there So we have each robot has a different color, which is very nice now I don't think folks would appreciate how happy I am that this worked because I don't know if any of you All have hit the docker hub rate limiting issues with everybody. Yes, you do feel that So this worked at home when I built it like at 1 a.m. Last night here here in the conference center It didn't work 30 minutes ago. So So all that to say this talk brought to you by Express VPN All right, so that is working. That's great. Let's look at one more thing Is what does that code look like right? We we did something cool So it's it's a little cluttered here. Sorry about that But essentially in our main function here I've written some go code where I read in the incoming cross planes telling me hey the user wants you to do this So I read the function IO. I go ahead and say, okay, there's some observed stuff There's some desired stuff right now. There's no observed robots. We're starting at zero So I make sure that any robots that exist or what we account for them And then for how many robots you've asked we go ahead and generate those robots We create those definitions for them and then we write that IO back to cross plane to make it happen So it's a fairly streamlined sort of thing read it in figure out what to do Write it back out to cross plane and let cross plane do all the all the heavy lifting there so I let me get back to the slides and I think that's it. I'll pass it back to Christopher on the other laptop here So Yeah, cool, let's talk about another feature For the cross-plane community it's observe only resources So in the past we had in the community a lot of questions regarding data source support like the guys and Terraform have or how they can adopt their legacy deployed objects from the API's to cross-plane and It was hard because you need to know the external name like the identifier and you needed to add all the Required names also from the resource from the resource you want to import. Otherwise, it was not possible to import the resources So we shipped a new feature as alpha for some of the providers and the cool part here now is that the provider Will observe existing resources without taking the ownership of it, which means You can import it and all the fields will be displayed under the status what I showed previously Because it's an alpha feature. It's currently only available if you're passing the enable management policy to the controller config and study provider with this and after that We're doing a few things under the hood for the CRDs we have in the providers. So we're using CRD validation rules that we can create managed resources without providing all the required fields We using their cell validation from Google in these CRDs and Regarding this we have a few API changes We want to introduce any providers if they are want to have support for observe only resources So we introduced a new spec manage policy and with this management policy We have now three new options or like adoption from the old ones, but we have full control So to this is the behavior we have today if you're running the Resources, we have often on the lead which is today the same like in the deletion policy the often and then we have observe only What we also want to change now is we have the full state Under status at provider, which means this is now a super set of spec for provider So we are displaying all the possible attributes under the status and then you Then you then you see how easy it is to observe the resources and Perhaps also changing after observe only mode to something like full control that you start to manage this resource also And in the future we also want to start deprecating the spec deletion policy. So for now We adding the management policy To the to the spec and have also the deletion policies So there is something like a table available at the moment in the design document What does it mean if you setting full control and management policy and setting a deletion policy to often orderly? There's a few things you need to check at the moment and in the future. We want to remove then the deletion policy on it So how it looks like now for VPC So thinking about your network team providing you the VPC and you don't you can't manage it But you want to observe it that you can use Fields from the VPC for other resources you have in your compositions. So you need to Set an external name now set under spec the management policy to observe only and the only thing you need under for provider as the region So in this case you is east one and then the provider starts to observe only the resource and at the end It looks like this you see at the status at provider you get now all the informations from the resource like cider block DNS host name support whatever and Yeah, that's quite awesome and helps a lot of people. I hope to adopting to cross-plan Let's see this very short hands-on so You can see here the controller config part what I was talking about enable management policy We configuring then and they provided a controller config and on the VPC side you see here I have a very easy VPC I have a label here an annotation to an existing VPC in the AWS account setting the management policy to observe only and I have here the region to use central one in this case. So then Create this VPC We can describe this VPC And if my internet is working then you'll see now there's all the fields now for this Observe only VPC. So there's nothing under spec for provider But all the fields we get them back from the AWS API is now available and you see it's a lot of things At the moment if you want to change now from the observe only mode to full control You need to copy pasting this status at provider things to for provider. There is an enhancement issue open that we want to Want to change it then if it comes from alpha to beta or general available feature that the provider will dust this Automatically for you if you're changing the management policy Cool this is everything now for the Observe only feature we shipped in one of the providers So and there's one other topic in the community. So like I need metrics to observe my resources What I have in the cluster so thinking about We have in crossplane like crossplane core. We have the providers. We have compositions We have claims like everything like this and normally guys asking in the community like so how you can Enable observability tools like Prometheus Grafana to see the status of all the objects and the systems so We had one idea in the community Building something similar to cubes that metrics, but the problem in cubes that metrics was that you need configure cubes that metrics for all the resources you want to have done there and Observed you need to configure it and it's very hard because you have the providers you have the compositions and so on So what we built up here is service called X metrics We will publish this right after the talk because I had no access to this but in general We have an endpoint for the metrics then available We're doing something like a state snapshot During the Kubernetes API and then we have a few options So like we're getting all the information from compositions from managed resources on the cluster level and on the claim level They are names-based and we're getting also the information like last transition time creation time Also the the if it's ready to sink through something like this and how does it looks like we have CRD support for this so like in this case for cluster resource We have a cluster metric here and under spec for match name. We have a string It has rec X support and in this case here We want to configure for our VPC. We import it now the metrics. So it's VPCs is to AWS up on dial and What happens afterwards is all the metrics are available then in your prometers for example, and you can also start building Dashboards out of it in Grafana. So let's have a look how it looks like as an demo So we have here as I said the cluster metrics and we want to apply this now in our cluster So like so you see now this resources created in the cluster we switching now to my Web browser and you can see automatically now a prometers endpoint is available for this metrics part And you can see that the VPC we created for the observe only resource We have no all the metrics available like is it doing is it available in the API one of the last transition time, etc, etc It's also the case. It's easy to change for example, you know here direct X group So say we want to see everything from easy to group in our in our X metrics, so we apply it again It's configured now and if you refresh it you can see it's no giving you all the Metrics you want to have from the easy to group. It's also possible to configure it like categories from CRDs So one of the popular Category is for example managed in cross-plan So you can say give me all the metrics from all the CRDs with matching for example the managed category And then you get all the metrics are created Yeah then Thanks, Christopher. So let's finish this off here. So we've got two minutes I think we might have let you down in terms of having a lot of time for questions I apologize about that, but we'll be here to keep that chat right after too But anyway, so you know we have built a community around cross-plane a lot of you here in this room We're participating in it you were what mate is making this project awesome, right? You're contributing code you're contributing feedback and issues and all that sort of stuff So please continue doing so there's lots of other ways to get involved as we continue to grow and scale the project And then I guess the final call-out here is You know we as we're making our graduation proposal We want to hear the stories from all of you that are adopting Cross-plane right now as you go to the main cross-plane cross-plane repo go to the adopters file and there's instructions there for how you can Share your story and tell us about how you're using cross-plane. So That's that I think that's everything and then we'll take questions. Maybe just one because I think we've got one minute All right awesome first question Thanks Coming from Terraform a you could detect dangerous operations using the plan Maybe some change in the cloud object will cause a recreation of the object. Maybe it's dangerous So is there anything like any kind of safeguard? Similar thing in cross-plane that will prevent me from doing dangerous operations Yeah, so so typically, you know, it depends on how the provider is is implemented But then a lot of the providers do have that same sort of safeguard Involved in them of you know, if this is something that would you know potentially destroy the resource or something like that then it won't it won't do it basically but essentially though You know, that's that's not Yeah, not a very very common thing and the cross-plane tries to take a philosophy of never deleting your resources You know without you explicitly asking to do so. That's a pretty strong philosophy Like sorry say that again. It's on a provider level. Yeah exactly it within the providers themselves that logic of like, you know The big specific mechanics of that cloud fighter and its API's and what's gonna do changes and what's not so alright So we'll be up here at the stage for a little bit too if anybody wants to come and chat, but thank you so much everybody