 Okay, thank you everyone and I will now start my talk so a Big thing about me. I'm from Ahmed and I work at a new company and Just quickly I have a cloud expert position, but I Was I'm a PG in HPC, so I'm very wide I have a very wide expertise so Let me tell you a story and Once upon a time long before the incident There was a man a man that has simple ambitions In your well terraform and wanted to use a brand new feature of clouding for his infrastructure At that time clouding already supported terraform But did not have implemented the new feature in that leg in yet the man said to himself Well, the features already present in the clouding CLI I Can ask something with a new resource and some local provisioner and It worked well enough at that time but soon the man wanted to modify the configuration of his resource and And he quickly found horrified that half of his infrastructure was gone Indeed it depended on that very resource and Terraform had chosen to recreate it Because it could not be updated in place How could it? It's just another resource after all once recovered from this disaster. He thought I Could write my home provider that would allow me to execute a command at creation and deletion, but more importantly at modification So he started to look at the documentation on how to create a terraform plugin With the hope to be a quiet and easy journey But he quickly found that the only way to write a terraform plugin is to use either the SDK or the plug-in framework both returning go But the man knew nothing about go and had no plan to learn it Little did he know that Terraform had also no plan to support anything but go and after looking everywhere I could on the internet for an alternative he realized that none existed So he resigned himself to learn the gopher ways and he followed tutorials after tutorials and Finally was able to write his plug-in. It was nothing shiny and it barely worked But it sure was something So during its exploration The man found in the documentation an interesting feature of Terraform in theory it is possible to have map blocks Which would enable some fancy resource definitions like this one with the read block and a key resolve example but reality crushed again the man soap as This was not supported neither in the SDK knowing the plug-in framework The man was very sad He could not achieve his vision and the level of quality he aspired to was just out of reach The story could have ended there But I would not be here today to tell you that story for I am the man of the story as you have probably guessed So what happened next? Basically, I went down the rabbit hole and I dug the documentation even further and started to look at tf code itself That's why that was quite a journey and I realized that the main part Use grpc and I already had some experience with it So for those who don't know grpc has official support for dozen languages and even more unofficially So I took the opportunity to use rust in an actual project and This is how I started my journey for oxidizing tf Oh and by the way for the rest of the presentation, I will talk about tf as Everything I explained is valid for both terraform and tofu So in theory the communication between tf and the plug-in is simple The bigger the plug-in is a grpc server that tf request But if we look into the details, we can see there are four grpc services Defined in the tf code The first one Called provider is clear enough. That's just the tf plug-in service Used by tf to tell the plug-in to create Update and destroy resources The next one grpc is tdio This one is used to let the plug-in log messages back to tf Weird way to do it, but fair enough Next one grpc controller is used to gracefully shut down the plug-in and appears to be redundant with the stop provider Call from the provider service But actually the stop provider call in in the provider service is never called and only the grpc controller shutdown is called and the last one grpc broker Appears to be unused by tf and I have absolutely no clue what it's for So if we dive into the detail of apply request We can see that tf sends the prior state the planned states and also the configuration of the resource as well as some other stuff Using dynamic value and Expect the new state back in a dynamic value also And the dynamic value is defined elsewhere in the proto So this is the definition of the dynamic value and as you can see It's either a message pack or Jason Encoded byte string Which means that for the provider to work with tf It has first to support protobuf for the grpc Layer as well as Jason and Message pack for encoding and decoding So that's basically three serialization scheme. However Drpc is only part of the equation in reality Tf has a kind of a meta protocol on top of grpc and This is a small uml diagram for those interested and Basically tf starts the provider process and pass a Certificate in environment variables Then the provider Is expected to generate its own certificate Start grpc server Using mtls with both certificates And it prints The info connection back to tf on std out With the certificate its own certificate as well as the Protocol version and after that tf Start perform grpc request first opening the grpc stdio stream and then perform some unary grpc calls First to get the provider schema then validate the provider config configure the provider Update upgrade the resource state read the resource validate resource config to finally Plan the resource change finally Then once it has done that Ask for a shutdown and wait for the provider process to end And this is repeated For every actions and every resources a bit complex if you ask me So now that we have a good understanding of how tf works This is some rust code to implement your own provider With rust And on the left This is the traits definition for resource So for those who are not familiar with rust Traits are a bit like interfaces And defines what a type must implement like methods and related types To be considered a resource and here We have many methods like a schema to get the information of the resource But also validate read plan create update destroy and stuff like that and also a state Which is the type used by the resource to to encode its own states So you might be familiar with the null resource from the standard The standard registry So here is a simple implementation of that resource Basically, you have the null state where you have a field ID, which is a string and a field triggers, which is a map of string and then I say to us that I implement the resource rate for my null resource where I say The state of my null resource is the null state And here's an example of the create method Where basically the framework We'll call With the plan state the config state the private state the provider meta state and Your provider is expected to return the new state and the new private state and here As it is a null provider is just generate a random ID and call it a day One important methods to implement is the schema method And the schema method is Used to give tf All the informations about the inputs and the outputs resource have So you basically define The main block And you specify which attributes within your the resource you have You can also define nested blocks More on that later And here in this example, I define two attributes The first attribute ID of type string Which is a computed attribute Which means it is an output as far as tf is concerned And the triggers attribute Which is a map of string And this one is a required attribute, which means it's an input attribute And I want to highlight the fact that those attributes correspond to the fields I define in my state only So now that I have a resource, I need to define the provider itself And the the way to define the provider itself is basically the same as the resource With a custom trait With a config type and some Some methods like a schema validate configure, but more importantly get resources And The get resources is called by the framework To know what resources is implemented by the provider And it's a map Where the key of the map is the is the name of the resource So in the case of the null resource the name of the resource is just resource And the value is The the null resource itself And by the way the provider can also have a config state But for sake of simplicity here, it's just a value empty so nothing So once we have defined Our resources and provider We can just call serve on our provider and specify what is the name Of our provider. So in our case, it's null So if you remember, I have a resource called resource a provider called null So the actual resource name is null resource If you remember a few slides back I talked about map blocks which Apparently is supported within tf But in no frameworks on sdk So is it possible with my rust framework? Of course it is and It's just As simple as defining a new block. So in my example read Which is a nested block of type map And then define within the block Everything needed as any other kind of blocks On the states implementation of things Now my state other Read the field which is a value map because it's a map block Of nested type And my nested type in is just another type With just a command Field which is just a string And this what allows to write Resources and data sources like this one Now I can say read resolve And specify attributes within and have another block Read something else And it will be encoded in my state with just Two items in my map So Everything is good at all But I've just The code in rust and I need to publish my provider And as a reminder From what I understand The way to publish providers with With terraform Is basically to create a release in a github And then the terraform registry Will fetch the release And call tfplugindocs to generate the documentation And then generate the signature and the release metadata To then upload it to the tf registry itself But if you have followed me until now You might remember that we do not have go we have rust So we cannot do that So what I did Is basically I created a pipeline within github actions And first thing first is to compile a static binary with rust And then I still need to generate the documentation So I still need to call tfplugindocs But I call it with a fake go And tfplugindocs Will think it will compile with go But instead it will just call a shell script That will copy my binary into the destination And then continue with generating the documentation Of course I also need to generate the signatures and the release metadata Into the release And once it is released I have another action that will aggregate all the releases To generate some json files and upload it to github pages And this is fairly easy to do because the provider registry protocol Is just a bunch of json files and links So this is an example of my release of my provider Where you have all the deeps with the binaries It's their signatures some manifest in json And a documentation zip And this is basically it So in conclusion With all this I was able to write my own provider With rust which uses features that are not available in other frameworks as far as I know And I'm pretty happy because I have published it In my github the github pages of my company And it works with both terraform and tofu with no need to changes So with my work I basically paved the way to create tfplugins from scratch in any language I have demonstrated it by implementing and publishing a tfpluginframwork in rust And I published my own provider for generic resources in rust And the last thing remaining is to write plugin frameworks in other languages And this is where I leave it to you Thank you very much Yeah, I think there's one Hi, would you be able to talk a little bit about what you're hosting in github pages and are you hosting The version documentation for lack of or the versioning information for the provider registry or are you hosting documentation as well? So I'm Publishing mainly the manifest jason's and binaries In theory, I Will not have much to change to publish the documentation but for now as you can see it's just a dogs.zip In the assets of the release But the interesting is this thing is Everything is in github pages So as you You worked on the registry you you might know that there is the dot well known terraform.json That should be at the root of the registry And this is also endowed with the github pages And basically I just redirected to the github pages of my Of my provider Also, I love that. Thank you. Any other questions? If there's no questions, I just wanted to say that this was extremely cool and the presentation was excellent This was really fun to watch. Thank you very much All right. Good job