 I'm Priyanka Ravi. I also go by Pinky. I have a co-speaker, but he couldn't make it today. So I have a video recording of him that'll come up later. So I know a lot about Flux. I don't know a lot about Captain, I'll be honest, but he is, like, more of the expert, so we'll have him on to speak about that. So this is all about automated and observable deployments with Flux and Captain. So, yeah. So I'm a developer experience engineer at Weaveworks. We are a company that actually coined the term get-ops and we created the tool Flux and then donated it to the CNCF, so we're actually a graduated project in the CNCF. We're pretty excited about that. My background with Flux is that I was a software engineer at State Farm before I joined Weaveworks and I actually helped set up Flux on our Kubernetes deployment, our platforms. And then I started working with the Weaveworks folks and so now I'm at Weaveworks, but I also have, like, end user background with Flux. And then Forian is a senior software engineer at Dynatrace. He's also a Captain maintainer, so, yeah, he's pretty awesome. And then, so, I'm going to do a little background about get-ops and Flux first for anyone that maybe is new to the tool, and then we'll go into, like, what is Captain. And then I'll give, like, a brief demo, like, showing how observability works with both. All right, so, get-ops is an operating model for cloud-native applications, such as Kubernetes, but I do want to highlight that it's not just for Kubernetes if you're doing, like, a multi-cloud infrastructure. You can still use get-ops, but since we're focusing on Flux, we're going to be talking about Kubernetes. It also utilizes a version-controlled system, most commonly get, as the single source of truth. It enables continuous delivery through automated deployment, and it has monitoring and management by a version-controlled system, and it allows you to manage your infrastructure and applications declaratively. There are a set of get-ops principles that were put in place by a pretty awesome group called the get-ops working group, and if you're interested in them or really into get-ops, you can go to getopenget-ops.dev. We meet by monthly. I shouldn't know that. And we really love, like, you know, people to come and, like, chat with us and, like, talk get-ops. And we're always looking for new members. So the four principles are the first one is that a system managed by get-ops must have its desired state expressed declaratively. So things are written in code. You know, there's more reusability. There's an audit trail, et cetera. The second is that desired state is stored in a way that enforces immutability, versioning, and retains a complete version history. So there's really no sneaking in a change. And then software agents automatically pull the desired state declarations from the source. So that goes along with the fourth one, which is software agents continuously observe actual system state and attempt to apply the desired state. So what that means is you basically have something that's continuously looking at your code and actually applying it. So you don't have things like configuration, drift, and all that. These are just a set of best practices. You don't have to, like, have all these met in order to be using get-ops. Everyone's journey really looks different. And you can start using get-ops and then add in these later to, like, tweak and harden your setup. Yeah. So why get-ops? There's a lot of benefits that come with using get-ops such as stronger security guarantees, increased developer and operational productivity, a better developer experience. You know, you're not stuck at work hopefully on Saturday trying to figure out what happened on your Friday deployment when it was a manual deployment. There's improved stability, higher reliability, and then there's consistency and standardization. And because of the tools' unique ability to treat everything as code, it creates a direct impact on security. So for example, if all configuration and security policy is treated as code, then any changes can be made, reviewed, and input into an automated pipeline. So there's no manual processes, like I mentioned. All right, so now we're going to talk about Flux. And Flux is a get-centric package manager for your applications. It is a set of continuous and progressive delivery solutions for Kubernetes. And we, like, so I do want to mention there, though, that get isn't the only system you can use. It provides, like, there's also, you can have it listen to S3, OCI registries, there's a lot of other things. But yeah, so it was created with Kubernetes in mind, and it's like a really natural, like, extension of whatever you're already doing. And so at the core, it basically is continuously monitoring your version control system and applying the desired state that you have there. And the nice part of this is that you don't have to worry about things like configuration drift because if someone goes in and deletes a deployment for some reason and you need it there, it'll stand it right back up. It runs on a schedule, so most commonly I feel like I've seen 10 minutes. And so every 10 minutes, it's checking to make sure that everything's applied. It hard applies everything. And it really reduces developer burden because it removes the, like, need for manual deployment processes. And also the command line tool is really awesome. It's really convenient to just, like, set up on a cluster. You can just, what we call, bootstrap it. And it also allows you to access the custom resources that make up the API as well. And we have a lot of major clouds and enterprises that use Flux for their own customers. So you might be using us and not even realize, but if, so with Microsoft Art Kubernetes, AWS, EKS, Anywhere, GitLab also offers us within their tool as well. So these are some statements we like to say about Flux. We call it Flux in short. And the first one is that Flux provides GitOps for both apps and infrastructure, and you can use Flux and Flagger. So Flagger is another tool you can use with Flux that allows you to deploy apps with canaries, feature flags, and AB rollouts as well. You can also use it to manage any Kubernetes resource. And infrastructure and workload dependency management is built in as well. You just push to Git and Flux does the rest. So it manages deployments through what we call automatic reconciliation. That's that loop that I was mentioning earlier where it syncs and applies. It really just works with your existing tools. So it was created with everything that you're probably commonly using in mind. So it works with your Git providers such as GitHub, GitLab, GitBitBucket. It can even use S3-compatible buckets like I mentioned earlier. And all major container registries and all CI workflow providers, also OCI repositories as well. It works with most common Kubernetes tooling. So like Customize, Helm, RBAC, and policy-driven validation such as OPA, Kyverno, Admission Controllers, it really just falls into place with what you're doing. We also like to say that it does multi-tenancy and we like to say multi-everything. It uses true Kubernetes RBAC via impersonation and supports multiple Git repositories as well. And multi-cluster infrastructure and apps work out of a box with cluster API. You can use one Kubernetes cluster to manage apps in either the same or in other clusters. You can use it to spin up additional clusters as well and manage clusters including life cycles and fleets. And it also has alerts and notifications so it can provide health assessments alerting to external systems and external events handling. You can just get push and get notified to Slack or any other chat systems or if something gets out of sync for some reason there's configuration drift, you can be notified about that as well. And then I like to say this one, users trust Flux. And I mean, I feel like as an end user myself, I'm allowed to say that. I wouldn't have like joined this and been promoting this tool if I didn't love it this much. And the community is really awesome. We pride ourselves in like, you know, the fact that we welcome contributors, any kind beginner level contributors as well. And the components of Flux are on Kubernetes core controller runtime. So anyone can contribute and its functionality can be extended very easily. Also, if you ever want to chat with us, you can just catch us on the Flux channel within the CNC of Slack channel, Slack group. Workspace? I think there are workspaces in Slack. Anyway, so what are the benefits of Flux? Flux really reduces developer burden. It also removes the cube control problem and I'm sure at some point we've all had the issue of like our versions being different and not being able to interact with the cluster. So you don't have to do that hopefully with Flux. It's also extensible. It's really versatile. It works with existing tools and it's really modular. So I'm about to talk about like the architecture in a second, but it is basically a microservice architecture. So you can really pick and choose and tailor your experience with Flux. So Flux is a set of Kubernetes controllers. If you're not familiar with controllers and Kubernetes, they basically control the lifecycle of objects. So you can use them to create things, destroy things, update basically anything. And so ours is made up of these controllers. So the source controller actually fetches resources and stores them as artifacts. So this is the one that's pulling from whatever source most commonly get and like storing it and it looks for a new change or anything like that. And then the customized controller is the one that's applying those manifests. The reason it's called the customized controller because this kind of creates confusion is because it's actually using customized in the background. So if you have a customization.yaml, it's going to apply whatever is there. And if you don't, it's actually creating a customization.yaml by itself in the background. So it'll recursively search whatever folder you've given it and it'll look for any yamls and apply all of them. But if you don't want it to apply all of them, it's really recommended to have a customization.yaml to specify the files you want. And then the helm controller does deployment of helm charts. So it actually uses the life... It's using the actual helm API so it can manage the entire helm life cycle. And yeah, so you can create something called a helm release and then very easily deploy a helm chart. The notification controller handles inbound and outbound events. So you can use it to, like I mentioned a second ago, set up notifications to Slack or something to tell it, hey, a change has been made to notify you. Or you can also use it to handle inbound events. So you can set up, like, let's say you're doing a GitLab merge request and you can set up a webhook there to then notify the notification controller, hey, go pull this change. I don't want to wait a full 10 minutes, possibly, right? So that's an option as well. And then the image controllers work together to update a Git repository when a new container image is available. So it'll actually go update your YAML to the latest version if it sees a new version of an image. And quick thing to mention there, it doesn't come by default when you bootstrap. There's an additional flag you add into the bootstrap command to get the image controllers to be installed as well. So now we're going to talk about captain. And like I mentioned a second ago, Florian wasn't able to make it, but he was kind enough to record this part since he's the captain expert, not me. So I'm going to let him... I'm going to move these down, too. Hello there. My name is Florian, and I would like to talk about the captain's side of things for this presentation and why we think it's a good fit to go along with GitOps. So at this point, Priyanka already gave you a great introduction on what's cool about GitOps and why it's such an important concept. But we also saw some challenges that you still might encounter and that we aim to assist you with overcoming with the lifecycle toolkit. So first of all, when you deploy something, you want to actually check when is my infrastructure ready to deploy so you wouldn't necessarily want to deploy a new version of your application onto a cluster that currently has some infrastructure problems. So that's, of course, also an important aspect to check first. Then you may want to check if there are some other prerequisites in place before you deploy a new version. So for example, you might want to check if an external database is available that's required by your application. Then you want to really know what's happening during the deployment of your application. So an application, especially in a microservice-based architecture, might actually consist of several different services which are deployed as individual Kubernetes deployments and services and you might want to have a holistic view of every service that's part of your application and you want to figure out where exactly a problem lies if there is something wrong with the deployment of a new version. And finally, after everything is deployed, you might want to do additional checks whether your application you just deployed is really running correctly and is healthy and doesn't suffer from any performance issues. And those challenges, we would like to solve with the captain lifecycle toolkit and we, for that, we set ourselves some basic goals or principles that we would like to adhere to. And the first and foremost is that we would like to allow you to use the captain lifecycle toolkit with minimal configuration effort so we took it as a goal to not have to adapt too much in your existing application manifests in order to make use of our project. Then we want to be compatible with GitOps and we want to adhere to a pipeline-less delivery approach so basically that means that if you configure a project to use the captain lifecycle toolkit, we want to fully support that via Kubernetes manifests in order to also allow you to put all this configuration into your GitOps repository. Then another goal is we want to allow an easy integration of external tools and control planes. So that means, for example, if you would like to execute certain tasks before or after a deployment, we don't want to reinvent the wheel for everything but we would like to make it as easy as possible for you to maybe reuse your existing logic and existing Docker containers that you have implemented for checking certain things and also make use of that in lifecycle toolkit enabled application. Then we want to give you deep insights into your deployment process so we do that by supporting OpenTelemetry where we create OpenTelemetry traces that allow you to keep track of your complete deployment process, author application, possibly consisting of multiple microservices and provide you with a holistic view of the deployment process. And then finally, we want to be cloud-native so that's why we went the route of writing the captain lifecycle toolkit operators which are fully configured by Kubernetes custom resources and those will allow you to consolidate multiple microservices that are part of the same application and therefore giving you this holistic overview of your deployment. All right, I already teased a little bit about built-in observability and what does that mean. So basically, we support on one side metrics so the lifecycle toolkit will gather all sorts of metrics that fall into the Dora metrics category so what are those metrics? These are, for example, how many deployments you made, how many of them failed, what's the interval between deployments and maybe what's the deployment interval, right? And we do that by exposing all these different metrics via Prometheus endpoint that can then be consumed for example by Grafana where we also provide a dashboard overview that gives you an overview of these kinds of metrics as you can see here in the picture below. Then the other part of observability is traces. So in the traces, we want to provide you an overview of what's happening in your application deployment and we do that by supporting open telemetry traces and you see an example for that here in the picture below where we have the trace for the deployment of a complete application consisting of multiple microservices and here you will see individual spans for the pre-deployment tasks and evaluations and the spans for the individual workloads that are part of your application and then finally the post-tax, post-tax being post-deployment tasks and post-deployment evaluations, right? And we support that out of the box. So we do that by supporting human adversaries recommended labels like app, art of, and version and the only thing you will need to do is annotate your namespace to be supported by the Captain Lifecycle Toolkit and then what you will get out of the box are those observability features that will give you these insights into your deployment process. What we additionally support in the Lifecycle Toolkit are pre and post-deployment tasks and evaluations. These checks allow you to check for several things like for example, is your infrastructure healthy up and running so you can check for error budgets and you can also run any arbitrary tasks really that can fit into a container and we do support some various ways of executing those tasks so one of them would be to make use of our Dino task runtime where you can basically write JavaScript code and execute that but since a couple of versions we also support to run any container which you can refer to as part of your pre or post deployment checks and you can also do what we call Captain evaluations. We see an example here at the bottom where we give you the opportunity to provide for example, a Prometheus query which you can check against certain targets and this will allow you to do some additional checks after the deployment to see if your application is running correctly. Now let's talk a little bit about how you would configure an application to use the Lifecycle Toolkit so as I said before you want this to be as easy as possible for you and you can do that by adding annotations that refer to those tasks referring to those in your deployments is really simple as you can see here it's just about adding the required annotations for that. On the right side here you see an example for a Captain task and an example for a Captain evaluation you will see here, here we provide a query that gives us the available CPU capacity and in the evaluation target we want to evaluate that and check if that value is greater than four. Finally, if you would like to consolidate multiple services that are part of the same application you can define a Captain app and here you can refer to the individual workloads in order to find which version of which workload is part of this application and then also similar to above you can refer to pre and post tasks and evaluations in order to execute them before and after the deployment of your complete application. Here on this slide I would like to highlight the bigger picture of the deployment flow so everything starts with either a Qubectl apply or of course if you have any existing GitOps tool you can also of course use that to apply your manifests but as soon as the manifests are containing the Captain lifecycle tool configurations are applied. First of all the pre-deployment tasks and evaluations for the complete application are executed then individual workloads are started and once that step is done individual pods running the workloads will be scheduled once all the pods are up and running the post deployment phase of the lifecycle toolkit will start and the post deployment tasks and evaluations first of all for all the workloads and then finally for the complete application will be executed. Just to give you a real quick recap what have we talked about now we have talked about how we can consolidate multiple microservices into the same application in order to give you an application aware insight into your deployment process then we have seen how we can potentially include vendor neutral observability by using open standards like open telemetry then we have seen how we can integrate external tools to execute simple pre and post deployment tasks and we have seen how you could potentially configure your application to make use of the Captain lifecycle toolkit in order to make use of all those features and yeah with that I would like to hand back to Priyanka who will give you a demo. Thank you. Can you hear me? Is this good? Okay. So now I'm going to do a demo and I didn't wait I have to change this to mirroring. Hold please. It's not what I meant to do. I guess that's definitely not what I meant to do. Okay, well back here. Mirror. Okay. All right. So now that I'm done with that. Okay. So the, I wanted to just quickly do this, I wanted to make this live, this portion. I hate live demos. They never go well for me. So I don't usually do them. But in this case I'm just going to run this last step to, and then I'll explain what's going on while this is loading. I'll do this. And then I'll just make sure I'm doing this right. Okay. So while that is running, yeah. Okay. The, okay. So the way that like you get started with Flux is this bootstrap command and that is my GitLab token. I'll be deleting that after this. So this is the bootstrap command. And that I've used to run this. And set this up. So the way that this command works is it's using the Flux CLI, which you can find installation steps on our website. And it's saying that go look under my, you know, GitHub repos and find this repository called captain. And bootstrap into a path called clusters, my cluster. So the interesting thing about bootstrap, this command is that if this repo doesn't exist, it will, it will stand it up itself. They will create the repo and then bootstrap into the repo. And what that does is it creates the files that are necessary for Flux and then it sets Flux up to listen to that repo itself. So the nice thing about that is if in the future I want to upgrade Flux very easily, I can just upgrade the Flux CLI to the latest version and then rerun this bootstrap command and it will update all of those files and update Flux itself to the latest version of Flux. If the repo already exists, it's going to just go look at that path and bootstrap the Flux, the Flux folder, Flux system namespace folder itself. So what this did is it created this clusters folder, clusters, my cluster. And in here we have the DOTK components YAML, which has all of the necessary things for Flux to be running. So the CRDs, the controllers themselves, the Flux system namespace that it creates, anything like that. Then we have the DOTK sync file. And this is how you tell Flux to go listen to something, anything in particular. So in this case, we are telling the source controller, hey, go listen to this Git repository. And, you know, go listen to this URL. So in this case, the captain repo. And every one minute, go check if there's any new things, go pull in the manifests from there. And then this customization is telling the customized controller to then within that, sorry, within that Git repository that was mentioned above, go look at the path clusters, my cluster. And every 10 minutes, go apply whatever you find there. So it's actually looking for this customization dot YAML. And like I mentioned earlier, if this file doesn't exist, it's just going to look for any YAMLs that are within, recursively within that file path and apply them all directly. But in this case, we do have a customization dot YAML, and we can specify the two resources that we want to be applied. So if you're just, you know, working on something, but you don't want it applied, you can leave it off of this and it will not be applied. And then I then went through and followed the really simple steps of this getting started guide with captain. So like I said, I'm not a captain expert. This is actually my first experience using it. And it was very easy to get started. So this getting started guide will get you set up with observability. And it uses a Jaeger, Grafana, Prometheus, and it, I think that's everything it uses, yeah. It uses sort manager because Jaeger requires it as well. So I went through and installed captain using their helm upgrade. And I do wish I had made it all more getups friendly. I didn't have the time this time, but I do plan on after this going and trying, you know, to make everything pushed actually to get and then letting flux itself pick this up. But I did run these through the command line options. And so I installed captain with this and then followed all these steps. And it also, within this getting started guide, it has you create an application, a simple engine X application. And then we can, you know, it pulls Dora metrics. It, as he mentioned earlier with Jaeger, you also get the traces so you can see how long applications took to stand up. And by now, it should have, let me cancel that. And then we can, I think, yeah, here. We can just rerun this command. Yeah, so it's already completed. Now we can see there's a version two. So it, it also, here, let me go back real fast. They also within these steps, they have their own custom dashboards that it comes with, which is really nice. You don't even have to create, you know, dashboards. That's something for me personally. That's the kind of pain, but they have their own. So if we refresh this, it should, yeah. So now we can see that there's also a trace. So we can see how long, you know, the lifecycle operator took to stand up and things like that. And then also go back. I wanted to mention something that I think he mentioned. I do believe he mentioned, but I just want to like highlight when you're creating an application, the way that captain knows to look for it, because it's kind of like magic, is that it's just looking for these three labels. It's looking for a part of name and version. And if you have those, so most people already have those in their applications, then it's pretty simple. You just get this stood up and running. The only thing I want to highlight is that when you install captain, you're not just going to see things right off the bat. It, it looks for new deployments. So you're going to have to like reroll everything out. And then you'll start to see it pulling metrics for that, but it's not aware of already existing deployments, if that makes sense. So that's something to highlight. And yeah, that's, that's pretty much it. It's pretty simple. It was really easy to get started with. And you can also, like there's this last step, which I didn't do. You can also add a captain app custom resource, like he mentioned there. And then you can add a captain app to it. So that's pretty much it. It's not optional, but that way you can control more specifically what workloads look like within the dashboards. And that's the demo. Are there any questions? Yes. Jessica. Oh, wait here. Wait, wait, wait. I'll repeat it. I'll just repeat what you say. That's a very good point. So I really wish you were miked for that. That because you brought up a really good point at the end there. So she was asking basically what is the use case for not having a customization dot YAML. And instead just letting the customized controller traverse the path and just find all the YAMLs and apply them. Because as she pointed out, that's a very good point. So I really wish you were miked for that. Because you brought up a really good point at the end there. So she was asking basically, what is the use case for not having a customization dot YAML and apply them. Because as she pointed out, that could be pretty dangerous. You might not want it to behave that way. So that actually is mostly user preference. I've actually done it both ways myself. Like at State Farm. But I think customize can get very nuanced. Like you can do a lot of things with customization YAMLs. You can do overlays. You can do patches. So the way that we were using it at least is that we had a separate repo. We were in insurance industry. So we had approvals that had to be made for anything to go into production. And so we had a separate repository that was just our config repository with all of our YAMLs. And they had to be approved by a manager and all the people that were necessary to get those approvals. So in that case, I could see you not needing a customization.YAML because you already know that anything that's in those YAMLs is what you want applied. But you're right. It's a dangerous game and I really do recommend customization YAMLs. Because I've seen things where I'm like, oh, I just accidentally pushed a YAML. I didn't mean to do that. There's another cool thing too. If anyone in here uses VS Code, speaking of like that situation, if anyone uses VS Code, there's actually an extension in here called the GetOps extension. And it will, it's going to take a second, but it will pull up all of your sources and workloads. And then you can go in here and actually pause them or suspend them. So let's say if you're developing already within VS Code and you push a change and you're like, hold up. I didn't want that to actually be applied. You can come in here and it's really taking a second. Yeah. You can come in here and you can, so I have to drop this down. I can suspend it within here and say I don't want you to pick up any changes or the same thing with any of the customized controllers. There's also a UI called the Weave GetOps UI. It's also open source and free to use. And it also has similar features as well, but also shows you all of your workloads for Flux. Any other questions? Where did my slides go? Sorry. I don't know where my slides went. Oh, no. Oh, there they are. Yeah. All my other stuff. Okay. So if there are no further questions, I just wanted to leave this to you. So if y'all are already using Flux, one thing that I wanted to mention is we do, we've worked to provide services and we can help you all out. So if you are interested in that, you can email me. You can also email me for other, you know, like Flux questions or anything like that. If you have Captain questions on the other hand, please email Florian. I'm not an expert in Captain. So I will probably just like forward you on to them. They also have a separate Slack that you can reach them at at captain.slack.com. This is both of our websites and our repos. If you love our tool, please go start it on GitHub. And you can reach us like specifically if you have Flux questions at cloudnative.slack.com and then we have a separate Flux channel there. So just hashtag Flux. And we're very active in there. So if you have any questions, please reach out there.