 Hi, today we're going to be talking about how to get off your terraform. I am Priyanka Ravi. I also go by Pinky and I am a developer experience engineer at Weaveworks. If you have not heard of us, we are a startup with a globally distributed and remote workforce. A lot of what we do is based on open source. You might have heard of our projects Flux and Flagger, which are in the CNCF as incubating projects. Flux was also the project that really kicked off the term get-ups, and it's been really cool to see lots of adopters of the project and see the community grow over the last few years. So much so that large cloud vendors and other organizations like Microsoft, AWS, VMware and others have adopted it and are using it under the hood to offer get-ups to their customers. Cortex is another one of our projects that is in the CNCF that helps make Prometheus scalable. I mentioned that because Prometheus is a key part of the progressive delivery possibilities with Flagger. And of course, we have other projects like Weave Ignite, EKS Cuddle, and now Weave Get-Ups, which is also a free and open source tool that provides get-ups for your various needs and has a UI on top of Flux. And we have many more, so if you're interested, definitely check us out on GitHub under Weaveworks as well as in the CNCF where you can find our projects. So what is get-ups? Get-ups is an operating model for cloud native applications such as Kubernetes. I do want to highlight that it's not just for Kubernetes. If you are using doing a multi-cloud infrastructure, you can still use get-ups. And we'll be talking about that obviously more in detail today. Get-ups utilizes a version control system most commonly get as the single source of truth. It enables continuous delivery through automated deployment, monitoring, and management by a version controlled system. And with get-ups, you're managing your infrastructure and applications declaratively. So the get-ups principles are a set of best practices. They have been defined through discussions with many different vendors and users, experiences by the get-ups working group. And if you want to learn more about the get-ups working group, you can go to openget-ups.dev. And don't feel like you have to have all of them met in order to use get-ups. Everyone's journey looks different. And you can start using get-ups and add in, hardening, and tweak your setup to meet these guidelines as you go. So the first one is that a system managed by get-ups must have its desired state expressed declaratively. It's written in code, so it's more reusable. There's an audit trail and a lot of other benefits that come with that. The second is that a desired state is stored in a way that enforces immutability versioning and retains a complete version history. So there's no sneaking in a change, and it kind of goes back to the audit trail stuff, too. The third is that software agents automatically pull the desired state declarations from the source. And the fourth is that software agents continuously observe actual system state and attempt to apply the desired state. So you have an operator, such as Flux, that is automatically continuously pulling in the actual system state and making sure that it is what you have expressed declaratively. So why get-ups? There are so many benefits to get-ups, and individuals, teams, and organizations who implement get-ups experience many benefits. I, myself, at my previous company, was on a team that implemented get-ups using Flux, and we did see a lot of these benefits, including stronger security guarantees, increased developer and operational productivity, an enhanced developer experience, improved stability, higher reliability, and a consistency in standardization. Because of get-ups' unique ability to treat everything as code, it creates a direct impact on security. For example, if all configuration and security policy is treated as code, then everything can be held in version control. And so any and all changes can be made, reviewed, and inputted into an automated way. There's no manual processes, so you're less likely to be at work on a weekend. So what is Flux? Flux is a get-centric package manager for your applications, but get isn't the only system you can use. And it provides a set of continuous and progressive delivery solutions for Kubernetes. It is a natural extension of the benefits of Kubernetes. And at the core of it, it continuously monitors your version control system, and it applies the desired state that's been declaratively stated there. The nice part of this is that you don't have to worry about configuration drift because it reconciles on a schedule. So if things have gotten out of sync for some reason, it will set it back to the desired state. Flux really reduces developer burden because it removes the need for manual deployment processes. And the Flux command line tool is a convenient way to bootstrap the system in a cluster and to access the custom resources that make up the API. So these are some statements that we like to put out there about Flux and to really showcase what's really awesome about Flux. Flux provides get-ups for both apps and infrastructure. Using Flux and Flagger, you can deploy apps with canaries, feature flags, and AB rollouts. Flux can also manage any Kubernetes resource. And infrastructure and workload dependency management is built in. You just push to get and Flux does the rest. Flux manages deployments through automatic reconciliation. Flux also works with your existing tools. Flux works with your get providers, GitHub, GitLab, Bitbucket. You can even use S3-compatible buckets as a source. All major container registries and all CI workflow providers as well. Flux works with any Kubernetes and all common Kubernetes tooling. Customize, Helm, RBAC, and policy-driven validation, such as OPA, Kriverno, Admission Controllers, so it simply falls into place. Flux does multi-tenancy and, as we like to say, multi-everything. Flux uses true Kubernetes RBAC via impersonation and supports multiple Git repositories. Admit cluster infrastructure and apps work out of the box with cluster API. Flux can use one cluster, Kubernetes cluster, to manage apps in either the same or in other clusters. Spin up additional clusters themselves and manage clusters, including lifecycle and fleets. It also alerts and notifies. Flux provides health assessments, alerting to external systems, and external events handling. You just get pushed and get notified on Slack and other chat systems. I'm going to actually be showing that in our demo today. Users trust Flux. Take myself as a user. I definitely trust Flux, and hopefully you'll see throughout this talk why users do trust Flux. Flux has a lovely community that is very easy to work with. We welcome contributors of any kind. The components of Flux are on Kubernetes core controller runtime, so anyone can contribute, and its functionality can be extended very easily. What are the benefits of Flux? Flux reduces developer burden. It removes the KubeControl problem. You don't have to worry about KubeControl versions to be able to interact with the cluster. It's also extensible, versatile. It works with existing tools. It's flexible and modular, and it's a natural extension of Kubernetes. And it's also extendable. Because of the microservice architecture, you can basically pick and choose what you want to use to tailor your own experience. So like I mentioned, Flux has a microservice architecture. It's a set of Kubernetes controllers. And if you're not familiar with controllers, a controller handles the life cycle of objects in Kubernetes. What should be done when an object is created, updated, deleted, et cetera. And the Terraform controller is not officially a Flux controller. It is an add-on that was created by WeepWorks. But the controllers that make up Flux are the source controller, the customized controller, the helm controller, the notification controller, the image reflector controller, and the image automation controller. The source controller fetches resources and stores them as artifacts. The main role of it is to provide a common interface for artifacts acquisition. The customized controller is a Kubernetes operator specialized in running continuous delivery pipelines for infrastructure and workloads defined with Kubernetes manifest and assembled with customize. The helm controller is a Kubernetes operator allowing one to declaratively manage helm chart releases with Kubernetes manifest. The notification controller is a Kubernetes operator specialized in handling inbound and outbound events. And the image reflector controller and image automation controller work together to update a Git repository when new container images are available. So Flux works with tons of other tools. These are just a few, and then there's a lot more as well. So basically anything that you're really working with, Flux will work nicely with as well. So reasons why I and others love Flux. I myself, as a previous user of Flux, that I just come up with a little list of the reasons that I really adore Flux. And one is that it really just makes life easier. Having your application to production is the entire goal of the development life cycle. Flux is a tool that was created to make the process simpler and more efficient. It gives developers the ability to focus on what really matters most, innovation and the user experience. Also multi-tenancy is really awesome with Flux. In my experience, it's very easy to set up and convenient to keep working. So there's also a really neat feature called depends on where you can tell Flux to wait until something is up and ready to actually stand up the next piece. So if you had an API that required a database to be stood up, you could tell it that. Also Helm integration is really fantastic with Flux. And the notifications and alerting is really easy to set up. I myself did it earlier today and it was really, really easy to set up. Bootstrapping is a really cool way to get started with Flux, which is part of the Flux CLI. The Flux CLI is a super, super user friendly way to interact with Flux. And now the Terraform controller, another awesome reason to love Flux as well. So what is the Terraform controller? The Terraform controller is a Flux controller that was created by Weaverx that can manage Terraform resources. And these Terraform resources that can be managed by it are not limited to Kubernetes resources, which brings us back to that multi-cloud option I was mentioning earlier. And so you can, these are a few links where you can find the Terraform controller on GitHub. There are some really awesome Terraform controller docs out there as well. And in those docs, there is like an awesome list of use cases that we're going to go over in a little bit too. So what are the benefits of the Terraform controller? One benefit is that you can have full GitOps automation. GitOps, you can use now GitOps for existing Terraform resources. You can use GitOps to plan and manually apply Terraform. Also you can use the Terraform controller to do drift detection of Terraform resources. And I'm going to mention that in a little bit, but you can also use it to just notify you of any drift detection that happens, even if you're not using it to apply your Terraform. And it can be used as a glue for Terraform resources and Kubernetes workloads. There are lots of cool features of the Terraform controller to be excited about. And there are new ones coming out every day. So keep an eye on that guide that I mentioned a bit ago. One such feature is the ability to set manual or auto approvals. With auto approvals, if you make a change in Git, then the change will be automatically realized by the Terraform controller. If you set it up to do manual approvals, then it will create a plan, but it will not automatically apply the change. And so you can see the plan and decide whether you want to make the change. And there's a new feature now to actually output that plan into a config map. And I will show that in my demo. Another feature is drift detection. And with drift detection, you don't have to worry about configuration drift. It ensures that you have what you actually have declared in your Terraform is what is actually live. And you can also use drift detection only and tell the Terraform controller to skip the plan and apply. So if you're already deploying your Terraform in some other way and want to just get the benefits of being notified when something gets out of sync, then you can do that as well. And I am going to show in the demo an example of getting notifications for drift detection. And another feature is that the Terraform controller actually accepts a list of config maps and secrets as variables. So you can use those to input as variables into your Terraform. Also, a state file by default is stored in a secret for your Terraform, but you can actually set the back end to be whatever you want. And also, there are health checks that you can set. And so for some resources, it may be helpful to perform health checks on them to verify that they are ready to accept connection before the Terraform goes into a ready state. And you can also destroy resources on deletion. That is not the default action of the Terraform controller. So if a Terraform object is deleted from the cluster, the resources created by Terraform are not deleted, defaulted to be destroyed. To enable that, you can set a flag to true. And another feature is that you can actually write outputs to a secret. And you can specify a target secret. And the controller will write all outputs to the secret by default. And you can or you can also write outputs selectively to a secret. And there's also name mapping and other features as well. So the Terraform controller team is really excited to announce that the performance of the Terraform controller has been improved significantly. Now the controller is greatly scalable to reconcile and provision high volumes of Terraform modules concurrently. And the team has actually tested the controller with 1,500 Terraform modules. And so with that change, you can actually customize your runner pod as well. You can update the image if you want to. If you have any need to include certain things in your Terraform image, Terraform runner image. And also, you can config some pod specs as well. Recently, OCI support has been added to Flux. And the Terraform controller has been updated to reflect those changes as well. So the Terraform controller can also use OCI artifacts as a source. Another thing is the ability to force unlock Terraform state. And by default, this is not enabled. And so if you do want to use it, you can check out how to do that in the guide. And we're also very excited to say that there is Terraform Cloud and Terraform Enterprise integration as well. And that is also in our guide if you want to check out how to use the Terraform controller with those as well. All right, so now I'm going to switch over to a demo to show you guys the Terraform controller in action. OK, so we're going to be standing up an instance of Vault using Flux. And then we're going to be using Terraform to configure that Vault instance, such as adding a secret and adding policies. And so we're going to be using this repository that was forked from a project that Rust created in order to show a demo of Vault with Flux. And so in here in this make file, you can see that it's going to create a kind cluster. And then it's going to be running this Flux Bootstrap GitHub command. And so this Flux Bootstrap command is basically the easiest way to stand up Flux in your cluster. And in this case, we're actually standing it up in a repo that already exists. But if you did a Flux Bootstrap and the repo didn't exist, it would create you a brand new repo and then push up all the manifests that the controllers and everything for Flux needs into that new repository. OK, now that the Bootstrap command is done, we can run a kube-control-get-pods-end-flux system and see the pods that were stood up there. So you can see here that there are the controllers that were stood up. And by default, the Terraform controller does not come with Bootstrap, but I'll show you how that was stood up in just a second. So the GOTK Components YAML is what's created by Bootstrap. And it has the name space, the Flux system name space. It has custom resource definitions, all of the manifests for all the controllers, and things like that. GOTK Sync is where we have the Git repository source. So this is telling the source controller to listen to this Git repository Flux Vault demo. And every minute, it's going to go in and pull the manifests and make sure there's no changes. And then the customization is telling the customization controller to then apply the files that are found in the clusters kind folder every 10 minutes. And so it's going to apply whatever is pulled in by the source repo in that folder. And so if we go in there, if we go into that folder, we can see that there's also this apps.yaml file, which is going to be applied as well. And this is another customization that's then pointing to the space apps folder. If we go into there, we can see that this is where the Terraform controller and Vault are being stood up. So in here, this Helm repository is pointing to the TF controller Helm repository. And this is the Helm repository source that's created. For the Terraform controller, it's being pulled every five minutes. And then the Helm release is what actually defines what the chart is going to look like. So this is telling the chart specs. It's saying the version of the chart that you want to be stood up. And it also has the values that you set for the Helm chart. And so this is telling the Helm controller to go apply this chart. And then every one hour to go and then update it. So there's also the Vault instance that's being stood up in the same way. It's a Helm chart as well. And so that's just the basic instance of Vault that's being stood up. So now we're going to go into VS Code. And we're going to apply this secret.yaml. And this is basically just to apply the secret that points to the webhook address for Slack. And I learned this the hard way, but if you push a Slack webhook into a public GitHub, then it will actually delete it from Slack. So that's why we're applying it here. And then we're just going to delete this so we don't commit it to Git. And we're going to now go into actually add in the notifications for the Terraform controller. So the first thing we need to do is add in some patches to tell the notification controller to also listen to Terraform events as well. And not just the ones that come default with Flux. So this is something you'd have to do if you were adding in a third party controller support for the notifications. So we're going to push that. And then I'm going to show you what's in that actual Terraform notifications.yaml that we just added to the customization. So this is the provider that's created for the notification controller. It's pointing to that Slack webhook secret ref that we just told it. So it's going to listen to that Slack webhook. And then it's going to also be listening for Terraform events. So that's what the alert is saying here as well. So if there's any Terraform events, all Terraform events, then alert us in Slack. So now we're going to reconcile the Flux system Git source that's pointing to that repository. And it's going to realize that there's been a change that's been made. And it'll also trigger the customization to then run a apply as well because there's been a change. So now we're going to go in and actually add the Terraform object now. So this is what the Terraform controller will, this is what it's telling the Terraform controller to then go and actually apply those Terraform files that are going to be customizing the vault instance. So let's commit this change, add this spec in. And so what you can see here is it's telling it every minute to go and apply the Terraform files that are found in this base Terraform k8s vault config path. And it's pointing to the same source ref too. Oh, and here, because the approved plan is empty, it's going to require manual approvals. And the store readable plan in human is telling it to output the plan into a config map. And so in here we can see that this is the Terraform that we just told that Terraform object to point to. So this is the Terraform that the Terraform controller is going to be applying. And it has policies. It has a secret that's created and all of that. So we are going to now reconcile FluxSystem again, the FluxSystem source to pick up the change we just added. OK, now that we've done that, we can run a kubic control get Terraform in the FluxSystem namespace. And we will now see that there has been a k8s vault config Terraform object stood up. And so if we go into Slack, we can see. So if we go into Slack, we can see that there is a Terraform plan that was generated in this general thing. So if we take, it tells us that we can set a proof plan to this string in order to actually approve the plan. So we'll go in here. And we will add this in. We'll set that here. And then we will, oh, right. OK, so I'm going to do a kubic control get config maps first to show you the config map that was created with the Terraform plan. And that way we can actually check that the plan is what we want. And I can show you guys how to check for your config map. So in here, you can see that the plan is all listed out here. It has, if you're familiar with Terraform, you're very familiar with a Terraform plan. It shows you all the things that are going to be added. And so now we can go in and commit that change to approve the plan. So now the plan will be applied. And so if we go back and reconcile it, again, we can make sure that that apply is being taken care of. So if we go back and get Terraform, it takes a second. But if we give it a minute, we'll see that it actually, there you go. It's already standing up. So if we go back, we can see that now it says that it's applied successfully. And so now I'm going to go in and I am going to update the file to do auto approvals. And I'm going to remove the store readable plan as well. So now if a change is made, it will automatically apply those changes. So there's no more need for me to look at the plan and then manually apply it or manually approve it. It's still getting applied automatically. So now we're going to reconcile again. OK. So now I am going to log into the local vault instance that we created. And I'm just going to enter the token that was set up. It's just a root. And then we can see in here that the cred's password secret was created, the one that's in that Terraform code. And so I'm going to be a bad actor and I'm going to go delete it to show you the drift detection. And so in here now, you can see the notification in Slack that tells me that there was a drift detected and that the secret has been deleted. And so this is great to show you all what it would look like to get a notification if, for some reason, things got out of sync. And so now if we go back in here and refresh and retype it in, creds are back because the Terraform controller set it back up when it noticed that there was a configuration drift. It reapplied the Terraform. So that's it for the demo. And thank you so much for listening to me talk about the Terraform controller. We are very excited about this edition. And please give it a try and let us know your experience on the CNCF Flux Slack channel. And here are a few links for you to check out. We have a Weave online user group that also meets regularly and has different talks about the different tools that we are using or integrate with Flux. And please check out Flux on GitHub and give us a star on both the Terraform controller GitHub and the Flux GitHub as well. Thank you.