 where we dive into the code behind cloud native. I'm Annie Talasto and I am a CNCF ambassador and I will be your host tonight. So every week we bring a new set of presenters to showcase how to work with cloud native technologies. And as always, they will build things, they will break things and they will answer all of your questions. So you can join us every Wednesday to watch live. So this week we have amazing session on automating Kubernetes deployments coming right up. And as always, this is an official live stream of the CNCF and as such it is subject to the CNCF code of conduct. So please do not add anything to the chat or there's a question that would be in violation of that code of conduct. Basically, please be respectful of all of your fellow participants as well as presenters. So with that done, I'll hand it over to our marvelous presenters to kick off today's presentation. Hello, I'm Kingdon Barrett. I'm an open source support engineer and a flux maintainer at WeaveWorks. And I'm Yoazos Gagalas. I'm a DX developer experience engineer at WeaveWorks. Welcome everyone and thanks for joining us. I'm just gonna share my screen real quick. Hopefully this works. Can everyone see my slides? Yeah, they should be seeing it live right now. Speaker note here is what I want there. Okay, so we're not gonna do slides for very long. I know this is not a slides venue, but we did wanna set some slides to sort of introduce some concepts before we dive right in. So this is the automating Kubernetes deployments presentation and hello and thanks for joining us today. And the main idea that I think we wanna get across is that there is really not just one set of best practices. So there are different ideas in different contexts. And we're gonna show how we're gonna introduce a concept of templates. And we're gonna show how you can use templates to automate Kubernetes so that you can show your team a different way if there's a different context. You can provide the best way for your team and you can help them. And then when you say you, do you mean me as an app developer or a platform engineer or some other role? That's a great question. I actually mean both of those people and sorry I have to, I wanna see you while you're talking to me. So I'm just gonna move you down here so that I can see what I'm doing a little bit better. Sorry about this. Yes, I mean both of those people. If you're a platform engineer, you need to be able to provide examples to your team. And if you are an app developer, you wanna be able to consume those examples without having to think too hard about them. Those are the main personas. So yeah, so this is about providing examples to customers. As a word I'm gonna use, but let's be clear we are talking about humans. So it's all about reducing cognitive load and making sure that you can access the essential complexity in your examples, whatever they are in your templates. Well, being able to gloss over the incidental complexity, that's what our aim is to help people with a template. So we'll be showing these three separate examples and keeping with the topic and the description that I've given so far. It's sort of an evolutionary model. As an app developer, I don't want to learn a lot about Kubernetes. So how do I start with the most simple example? Yeah, so the most simple example is a dev branch. We're going to have a dev branch and we're gonna automate that branch. And just to back up, before we talk about any individual example, what does it mean that we're automating a branch? Well, this is GitOps. So we have Kubernetes over there and we have Git over here. We want to deploy our changes from Git to Kubernetes in an automated way. And that's what we'll do in the first example, as minimal friction as we can and we'll have just one environment. And that's a hint of what are we gonna do next? We're going to show promotion. So in a promotion model, we will have more than one environment and we will need a mechanism for moving the change from one environment to the next. So I've used the word pipeline here and to explain a little bit what I mean by pipeline, it's not a physical thing, it's an abstraction. The pipeline helps us to test our changes before they go to production. That's really what it's for. So there's a little bit of friction that we're adding here to avoid problem in production down the line. So we can do our testing in an environment that's as similar as possible to production and then promote the change. And as a developer, I can set up a demo environment that is a production-like example to show the rest of my team using pipelines, right? Yeah, yeah, exactly. And then when we, so what we're doing in this case, we're talking about release versioning. Release versioning is an important thing to use in your release pipeline, but it's also an extra bit of friction that we might like to avoid. So that's one of the differences that we'll see between these last two examples is the way that we use versioning to achieve our pipeline. And in the last example, we are nailing things down, I guess. The idea is that we'll be doing formal releases and we need that so that we can keep track of what's in production at any given time so that we can roll back if there's an error, if something goes wrong, we can see what version was there before, roll back to that version and then continue on with development without an incident, hopefully. So there are all these different models and this one we're also going to start to talk about OCI at that point. We'll gloss over it a little bit and we'll see the examples do include OCI up to this point but we won't really talk about OCI until we get to the third example. And we're gonna talk about Helm also. So we haven't talked about Helm yet but Helm is the templating tool that we're gonna be using today. You don't have to use Helm, you can, there are many different options for templating tools but the concepts can be applied from here to other tools and Helm is a simple one to get started. Yeah, not only simple but also extremely popular. Vendors have all basically sort of settled on Helm as the tool if you're a Kubernetes vendor. So we'll show some advanced Helm features including OCI Helm charts. Hopefully we get that far and let's keep things moving here. So the short version is there are the three examples that we're gonna show. And here is the small version if you got it all lost in that if you want the screen shot of this so you can refer back to it. We're gonna show all three of these examples in succession. Yes, let's get started. I'm a developer and I want to work on my local system to start using Flux and to start using Git like process as far my development workflow that later I hope to promote to whole team and to enterprise environment. Okay, so, and this is our first example we're going to deploy a website and the website is deployed from this Git repository and we're not gonna look too deeply at what is this website. We're gonna click on this link just so you can see it is a website. It's called example cube config because I needed an example cube config to be served from somewhere that people could join my cluster and use. So that's what this is. It's just a website. And as a newbie, I probably want to start the deployment YAML for this website. Yeah, exactly. Okay, so let's look and see our deployment here. So we've got a couple of directories to look at. This one is the main one. This is where we keep our manifests and there is a deployment.yaml. If you're unfamiliar with Kubernetes, don't worry. If you are familiar, this should be relatively familiar already, of course, because this is one of the fundamental units in Kubernetes, a deployment. And what we're gonna see about this is that this is a deployment that creates a pod with two containers. They both, the purpose of this structure is we wanted to use the existing engine X image because we're serving a website. That's all we're doing. It's not a program that runs, it's just a website. And we do need to retrieve that content from somewhere. So we could build it into the image but if you really spend a lot of time with Docker build, you probably find that there are a lot of more efficient ways you can build things than by doing it in a Docker image. That's one of the things that we're trying to show today is how you can separate content from structure. So that's what we're doing here. So let's get a change pushing so we can see that we can actually push changes to this website and then we'll look at how does that work? And like I said, we have a dev branch. That's what my, can everyone see this? I probably need to increase the font size. Let's see, how does that look? It looks better, but I think in this case, the bigger the font size usually is the better. But I think we're getting there. And if the audience, you feel like you need a bit more zooming in, just let us know. Okay, all right, so here we have our dev branch on our example cube config repo. And we're gonna make a change. Okay, I'm getting feedback that we can actually read. That's good. Okay, so we're just gonna change some content in here. I think we have a hello GitOps. We're gonna change this to helloocncf. And what this is going to do because this is an MK docs site, it's gonna trigger a CI tab that builds the site. Let's make sure we can see where that update is going to come from. Ah, okay, so here's our first hurdle. We said that there were some problems in the earlier models that we'll explore. This is the problem number one with using image update automation. So this is the simplest model, is that you will occasionally have to rebase your changes over automation, which is not ideal, but this is a small example. Can you show how image update automation is defined in our manifest? Yes, absolutely. Okay, so we have, like I said, several directories of manifests. Here is our main directory, and then underneath here is our flux automation directory that's deployed with a secret. So this Git repository reference points at our dev branch, and it has a secret ref. This secret is able to write commits to the dev branch. We can see our build is finished, and let's see if we can catch the change here. So it says hello, GitOps in the previous deployment. We're gonna look at the image update automation, and we're gonna see the deployment go fully. Yeah, so here's our previous version running, and we should see momentarily a change as flux detects the new image. So this is what detects the new image. It is scanning the Git repository. Can you zoom in a little bit on the website? Thank you for the reminder. Okay, so this fellow is scanning, and it is updating inside of this path. Okay, so we've seen it's just done something here. We've got our new pod starting, and what's actually happened is we have pushed a new commit to the dev branch. So let's take a look at that commit. Oh, there's an audience member checking in as well. Would it be possible to get the GitHub link, repo link as well to the attendees, or if it's not, obviously we understand? Yes, absolutely. All this stuff is public. So this is the repo. I'm not sure where I can paste it because I don't have access to the chat from here, but if... I'll paste it right here. Okay. Yeah. And I can highlight it then afterwards if we get it to the comments or to the private chat. Great, thank you. Okay, so this is the commit that was just made. We can see what's happened here. We've got this comment that tells Flux's image update automation where to look for the image ref. So this is a tag ref. That's what this tag means. And that's to distinguish it from a fully qualified ref. If we left this colon tag off the end, then we would get the name of this image in full, including the repository where it comes from. But we're... We actually, we only just need this string to change. We don't even care about what content it is right now because of the way this example is built. And we can see that it's deployed here. We could do that again. I think we have enough time to still do that again. Yeah. And I think you can see the GitHub link right now highlighted in the stream as well, as well as in the comments section. I'm going to move this up a little bit so that bar is not in the way. Okay. So before I commit this, I'm going to pull again so I don't have to rebase. Okay. And we looked at one part of the automation. So this is the commit that actually changes our manifest. And here is the rest. So that image automation is the business end. Then there are other parts that are important for this process. There is the image policy and the image repository. So this is the actual repository that our dev branch is pushing to. And you note that it's a separate repository from the main. There is no, there is no prod environment here. This is a dev only example. So you could imagine that we would push another image here for the production environment, but we're pushing to the dev branch here. Okay. So I have this set up now, but I want to extend it. I want to have versions and I want to share it with my team and promote things to several different environments. How, how do I go forward? Okay. Outstanding. So what we want to do, let's recap what we did here. We used the Docker file. We haven't looked at the Docker file, but the Docker file builds an image and we push an image. And those examples, by the way, those are all GitHub actions. So you can follow them here. And we're going to switch to our next example where we will change a few things. So first we're going to change what type of app we're deploying. So it's no longer a simple website. It's now an app that runs. And we've also added a Helm chart, which has some interesting properties we're going to look at. And Git repository is connected to a Helm release. So let's look at that. All right. So here's our app. This is a CLI. It's written in Ruby and there are a couple of other components. This is a good example of fibers. Well, good example, maybe a stretch. My Ruby is a bit rusty these days, but okay, here this is the app that runs. There's a web server. And this is actually, you can see, we've put in a print statement here so that we can see, because this is not serving any content. The app is doing something else in the background. There's a job that we want to run, but for our purposes, that's not super important. So let's go to app directory. And all right. So what we want to do is create a branch and we're going to update that message. So first we want to see this running in context. So let's switch to our cluster where that is running. That is, I think it's this one. Yeah. Okay. Here's our running app and here's how you know it's the app. Okay. So we're going to change that message and get CLI going. So we can run this quickly. So one of the things I want to point out here is that we're creating a pull request. We're doing this because we have things pointed at the main branch. We have flux syncing from this main branch. This is a little bit aggressively large. I have to tone down. Sorry. Hope we can still read. So there are two directories to look at here now. One is the Helm chart itself and there is where flux deploys from. Since Helm charts are not deployed directly by flux, they're deployed through another set of automation that's part of the GitOps model customization. So here's our Helm chart. And I like to highlight that the template is very lightweight. There's actually very little in here. If you've ever written a Helm chart, you're probably going to wonder how I did this because this is probably the worst part of writing Helm charts is building out all the templates and filling out all the boilerplate. Wouldn't it be great if we could just skip all that stuff? Well, yes, you can. So here's our chart definition, chart.yml. And how do we skip all that stuff? We're going to use a library chart. So I had never heard of this company until I heard of the Helm company info. This kind of sounds like a made up company. I'm not sure how much I trust them, but Helm releases are versioned. So we know that this isn't going to change out from under us. It's immutable, hopefully. And let's do, see how that build went if it's still going. Okay, so our build is ongoing. And when we're done with this build, we are going to merge the change. So there are a few other things in here that are not ideal about our setup. And then there's an audience question. So Garov is asking, we need to make a commit in the Git repo for previews running code slash image and Flux will do the deployment again. Yeah, so there's something we're building up to that we'll see in the next example. And I kind of don't want to go over that point yet, but I, so the question that you're asking, I think is will there be a commit that we can roll back? And the answer is yes, there is a commit that you can roll back, but the first example that we looked at, we saw automation is pushing commits to our working branch and that causes extra friction. So we actually want to avoid that. We want to migrate from the latest commit determines what's deployed. We would like to migrate from that to the latest released version determines what's deployed. And we're on a road to get there at this point because we're creating a Helm chart and that gives us something versioned that we can publish. And we're not publishing it yet. So we're actually deploying this Helm chart from the master branch here. So this branch will show the other directory here. This is our deploy basis. And this is very similar to the last one, except there's no automation primitives. There's no flux automation resources required. And this is to help understand something else that flux will do that will make our lives easier. So we have a Helm release and it's a source ref get repository type. This is a special thing that you can do in flux that depending on this particular setting, what's going to happen is, yes, this new link here, let's post this as well. Perfect, I'll get it over. Yeah. So what's going to happen is we're going to check under the get repository for a directory named charts slash mark here. That's how I'm going to pronounce that. And it's got to be in the main branch which we've defined in that get repository as actually being named master here. And then someone was asking to zoom in a bit on the get side as well. Sure, a little bit further. Yeah. So there's not much to see in here. The only thing that I would highlight is there's a version that's implicit here. Since we haven't mentioned a version, it's just going to deploy the latest version, whatever metric it has for determining what the latest version is. So because we're using a get repository, that's whatever's in the latest branch. If you want to revert, then you revert to get commits, right? Yeah, thank you. That's the bottom line. So we, I would like to fix those things. Sorry about that. That's just a distraction. So we're going to create a pull request. Like you said, it's already been created and we've got to merge it. Now that CI has passed, let's merge it. If these things can be done automatically and depending on the environment, things can be approved or automatically merged. But what we should see is, as soon as that get repository reconciles, we'll see the chart kick off and start updating itself to the newest version. And what has changed in the chart, actually nothing. This is another thing we ought to look at because this is another thing we're going to improve in the next example. So how does helmet know if there is no template, what to deploy? Well, all of the important details go into this values file. So here you can see we've pointed out an image called Canary. And that is the image that we build on any branch. And there are some other details in here. How is the port exposed? These are details that are important if you're serving up a webpage, for example. Anyway, we can see that we have actually deployed a new pod now. And then there has been a audience question that we can take whenever is a good moment for that. Sure, before we move on to the next example, we're a little bit ahead of schedule now. Good, so the question is, in case there is a requirement to do a backup of newly deployed codes, last images, does flux automatically do it by observing the new deployment status or new commit is required? So flux will apply the latest manifest in the repository and the latest images using whatever specified policy you have. So it will not roll anything back unless that's what Git is the final source of truth. But you can set up things like Flagger or other system that will do the rollback using GitOps for you. Yeah, and there's actually like about three large exceptions to what you just said that are important to understand here. So one of the things to note is you can do testing in a Helm chart. You can do testing in a pod with a health check. And those tests will help determine whether the release is going successfully or not. So I haven't created any health checks for this. So when this pod rolls out, as long as it runs, then the release is okay as far as Kubernetes is concerned. So we have to kind of leverage these tools in order. We wanna use the health checks first because that will help Kubernetes know whether this pod is healthy or not. And that's the only signal that Helm can use to tell us if the release is okay. Then down the line, flux can look at that signal from Helm and say this release is a success or not. And you can configure remediation strategies in flux to say, what should we do when a release has failed? If you want it to be rolled back immediately, then you can configure that remediation strategy so that it's rolled back immediately. Going to look for the doc here because this is a good Helm thing to know about configuring failure remediation. So this is, here's that version line that I mentioned that we aren't using yet. We're gonna see it in the next example. But here are the parameters for remediation. And you can see that there's a different thing that you'd like to do on install and upgrade because actually if it's a failed install, we can't, there's nothing to roll back to. So we'd like to retry. And if the upgrade fails, actually this example says, no, don't retry, just let it fail. Don't remediate the failure. Why do we do this as a default actually because we'd like to be able to debug the failure. We'd like to be able to see what failed and we need to be able to inspect the logs possibly. And then for a second reason, because we've configured health checks on our deployment. So this upgrade that failed doesn't represent an incident in production, or maybe it represents an incident, but it's a different class of incident. We're not serving up a dead release to end users, right? It's just there in production. It needs to be remediated by someone, but we can configure that to happen automatically if you want here. And then there is no commit that represents that remediation. So that hopefully that's a good answer for your question. Yeah, the question I asked here already said, got it, thanks. All right. Good on that front. And then there was another question, it now is a good time for that as well. Yeah, let's see, what was the other question? I heard Dev at QA branch is the Dev branch for Dev environment, QA branch for QA environment. Good question. Okay, so what we're showing, we actually want to get away from the idea of using separate branches in this progression in this evolution. So we're starting with a Dev branch because that's the easiest thing to conceptualize and because we can configure automation to push to a Dev branch without breaking everybody's workflow, right? If you're not on that Dev branch, you won't even notice that those commits are in the way because you're on a different branch. So that's why we start with a Dev branch but in most of the flux docs, you will find that they do not advise you to use separate branches. And that's because inside of the branch is a ref that points to the branch. And what do you do when you merge? If you've got a Git repository here that points to a branch that we're deploying from and now we want to merge but this ref is still going to be pointed at that old branch. So this is the main reason why we want to get away from using branches as an instrument of deployment, right? We want to use the branch as a place to review to run CI. And then once it merges, then it gets deployed. That's the idea. So we're trying to move away from simple trunk-based workflows that don't version and we're trying to move towards versioning. So one thing we didn't see, let's look at that deployment or that pod, we didn't see what Helm is doing with that version. So we had this, we saw this version in the chart.yaml really briefly, this one came from Flux. When Flux found that there was a new commit and we had that revision strategy set, Flux said, well, Helm charts are immutable. So I need a new version if I'm going to have a new chart. And so it reconciled a new version and then our deployment went out. And here is the other thing I want to notice in here is I think we already saw this. We're just using colon canary. So this is again, this is not an ideal deployment strategy because this is like latest. This image doesn't, this tag should change over time. But we're going to show in the next example how we can do that as part of the release workflow because we've already seen how we can use automation to change that tag. So, and we've seen how we can run a file of that and it can be kind of irritating depending on the context. So we've got these two examples in opposition. And now we're going to switch, I think, to the next example. Okay, so we had an example with no Helm chart. We had an example with a Helm chart but it's the Git repository backend. And here now we're going to switch to a fully release oriented Helm chart. So I've used this link here. We can put this link in the chat as well. These are the workflows that I developed. One is for publishing an image and one is for publishing a Helm chart. And this is another website example. So we're going to go back to simple here for the purpose of this example. And there's another link I'd like to share. If you have a Helm chart and you're wondering how can I release it as OCI? Well, this is what I would suggest first. It's actually really easy to build the OCI release workflow into your release workflow. But we wanted to have an action. I think most people want to have an action that they can point at that'll handle the hard parts when we get to it. So this is an action that's not published by Helm official and I'm going to show the Helm official action for a moment to explain why. Chart release or action. Okay, so this is an open feature request. We would like to add support for pushing charts to OCI. We mentioned that this is a new feature in Flux. It's OCI Helm charts. It's a new feature in Helm as well. So there are places where the support is lacking. I think this is the most popular chart releaser. Probably a lot of people are using this to push their charts to GitHub pages. And this probably will work for almost everyone for a long time. But the people who this won't work for very well are the people who are trying to push a lot of releases all the time or who have a lot of microservices. So why doesn't it work very well? Because of the structure of a Helm repository, the legacy kind. There's this index.yaml file where all of the metadata for all of the releases for all of the different applications that are on this repository is collected in aggregate. And every time a new release comes out, whoever consumes the Helm repository has to download that entire index.yaml again, even if only a small part has changed. So that's really inefficient. And that was one of the things that OCI charts were designed to fix. So let's see our example. Let's pull up that blog. I think it's called Bart. Yeah, here we go. Anyone's familiar with Fermion in the audience? This is the Bartholomew Wazen blog. Nothing to see here. It's pretty cool. I like it. I'm gonna build a website on it. And as you can see, I've already started. Okay, so if we go to slash blog, we can see the title of the page. Here it says, hello, GitOps. Let's make something a little more visible since that's kind of small. Here, let's change one of the titles. This is the top article. I know it's not. Okay, try again. Okay, so as we did before, I'm going to change a branch and I'm gonna commit my change. By the way, if you're not familiar with this GH tool that I'm using to watch the workflow runs, it's really very similar to what you see on the GitHub Actions view, but maybe this is a little bit easier to follow if we go see. Here's our new action that we're running. And we've got a couple of things that run in parallel. We can see that that's happening over here too, but so while it's running, there's a question. How would you handle different environment settings using GitOps? Credential. That's a good question. So we have two environments that we'll deploy this to. One is called test and the other is called stage. Let's look at those environments. Where are those defined? Here is one more repo. This is where all of these examples are collected and deployed from Kingdom CI slash fleet intra. So this is our cluster repository. And this example comes from, let's see. There was also a question from the audience on the recording of this meeting or this event. And no worries, this is gonna be available essentially immediately after this in the CNCF YouTube channel under the lives or in the playlist for cognitive lives. So you can find it there immediately. So you can always check back on the recording, no worries. So what we wanna see in our test and stage environments is what's different between these environments and what's promoted exactly. Let's see if we're deploying things before we get any further. Okay, so we're pushed into a branch. So nothing should be deployed. And in this helm release, we have a version pen here. And ideally in my perspective, this is how we handle a difference between environments is by creating a parallel structure in both environments. So we know that this test environment here is called Bart.howard here. And if we go to look at the other environment, we can see that it has a different DNS name here. This one is called stage. So again, this is kind of backwards because I don't have a production environment. I have test and staging, it's kind of backwards, but you could imagine adding a third environment and we'll show how to make staging a little bit more production-like in a moment. So this is using a helm repository source and that's where we publish our release artifacts too. So what we wanna see is the test environment deploys as soon as we've merged our change to main, see it's a Git repository source. And then the production environment or the staging environment is going to deploy as soon as we've published our release to the helm repository. And these should behave the same except they have values that are not the same. So this one says they're both in the public Ingress class, they're both using TLS, but they have a different host name and you can configure them differently if you need to. But would it change? Yeah, go ahead. Yeah, there was a question in the audience as well. Do you think flux is better than Argo CD? I'm not gonna be baited. No, I work on flux. So obviously my opinion should be fairly obvious. I think that flux is great. I think that Argo is also great. They are separate projects for different use cases. Stacy, you're exactly right about that. Thank you. So flux from my perspective is better for infrastructure builders. It's better for platform teams. Gives you a little bit more control whereas Argo maybe gives you a bit more flexibility in a couple of ways. I'm not gonna go into detail about that because we're almost finished here. We can talk more about that at the end but I'd like to see this full release happen. So okay, so what we've got is we've gotta merge this pull request. Where did we create a pull request yet? No, okay, we have to create a pull request. And because this is gonna be a new release, there's a few extra things we have to do. I said there's some friction that we're gonna add. We're gonna solve the friction with automation too. So we've got this make file. It's a little bit complicated but the main thing to look at is here at the top we have these two versions that we can use a tag. That's the app version and somewhere here is supposed to be the chart version. And we've got these workflows we can use to release when we've tagged new versions. So let's tag new version. First, I wanna make sure I know what version we're at. 0.2.6 is our app version. And I think the chart is one behind that. So let's do version set. We're gonna be 2.7 now and let's see what that did. Okay, we updated app version and chart.yaml. Here we've updated values.yaml to point at a new tag. We have not built that tag yet. And we have updated spin.yaml to know what version it is. Let's commit that new app version. Are you committing this to a different branch? I am still committing it to this branch. We could commit it to either branch. This part actually it could be done before or after the merge. That's a good question. And we also have to update the chart version. So like I said here, our chart was at 0.2.5. We're going to push a new version, 0.2.6. And we see that is correct. There, new chart version. Okay, so we don't need to wait for CI here. I'm confident that CI is going to work as long as we, what's going on here? GitHub, are you not going to accept my change? We should see more commits here. We don't see any more commits. I'm not sure why. There's no fork. Well, I think the branch protection rules will let me merge this directly to main. Let's just try it. We had some issues with GitHub this morning. I don't know if anyone was watching the status dashboard, but I really was nervous about this. Everything went green about five minutes before we were supposed to go live. And this is the part where we really need all those things to be working. So make release app first. Here, we want to push this and merge that change. And then we'll do make release app. This is going to get tag and push the tag. So this is, you could do this manually or you could build this into your release workflow, but this is how we push an image and it should trigger our deployment workflow again. And then we'll do a different. We are starting to have actions freeze here. This is not great. If GitHub doesn't know if anything is wrong, but I'm pretty sure it is. Let's see. So hopefully these things only take about 30 seconds. That's how long they should take. And this is our tag. Okay, it's working. GitHub is fine. That is true. All right, now we push the chart release. And this is just doing the same thing, but tag chart. All right, and let's see. Yeah, I know, I know you literally just pushed and commit two seconds ago, but this morning I was watching it go up and down right before my eyes and they did have an outage for sure. But no excuses now because everything's green and it seems fine. So let's see. All right, our test environment deployed the change. It's already running and this one should be momentarily there. We even caught it in the act and let's see if we can see the updated website. There it is. Hello, CNCF livestream. Everything deployed from a helm release. It wasn't too much friction. I hope my goal is to help you understand how you can adopt helm, but also to help you understand how this is not the tool for every job. There are different tools for different jobs just like we've seen with Argo and Flux. There are different tools and helm may not be the template tool to solve every problem, but we've used it to solve this problem. If you're interested in this blog and how it works, you can certainly follow these links and check out this repo too. And hint, hint, there are some conference talks coming up at open source summit and get ops con that will be related to this. So we'll see more from that. Okay, so while we still have a couple of minutes, we have time for more questions. Josus, do you have any questions at this point? No, thank you for the demo King and I'm glad it all went smoothly. Yeah, me too. Maybe we wanna look at some of this example in a little bit more depth to see what we've deployed exactly. Yeah, I think that sounds good. And while that's happening, audience feel free to write your comments and questions in so that we have the time to go through them. Yeah, so this is a good question. We had only one branch that's configured for deployment and we pushed to multiple environments. Yes, that's correct. Let's look at the two environments. Now the environments are designed here in this separate repository. And this is something that's up for debate. Do you take the environment definitions from the app itself or do you keep them in a place that's more closely tied to the cluster? Those are options. So here we have a Helm repository. Well, actually it's a Git repository. And we have a Helm repository that's defined next to it of type OCI. So if we look at the other environment, we'll see the same thing there except we're not using the Git repository for anything. But yes, it's deploying from branch main. And this can cause problems as well. So the really nice thing about our OCI repository example is yes, it depends on the release cycle. If we wanna deploy things from versioned releases only, OCI is great. If we wanna change our version strategy so that we version a different way, like so that developers don't actually have to create semantic version numbers for every single release, then you can do the same thing. The differences, okay, so there was one thing I wanted to show here. Let's see, where is this? Maybe it related to this. Did we look at specifying versions in version constraints in Helm? Yeah, so here in our stage environment, we've got a Helm release and that Helm repo again. And we've specified a version constraint. So this says deploy any release so long as the major version hasn't changed. And it has to be at least 0.2.0. So this is gonna scan our Helm repository for versions that match that constraint and it's gonna deploy anything it finds. Because this is also this Helm repository type OCI is also an image repository structurally. OCI, Docker, they're the same standard under the hood or OCI is a superset, I guess, of the standard. What this means is that you can also use image update automation in this place. So if you have a requirement that actually I heard someone ask earlier, is there a commit that I can roll back? This is the place where you can insert that. So we saw in the very first example, how you can monitor an image repository for changes and then deploy those changes by updating a YAML with the new tag. Well, that's what we would do here if we really wanted to have a commit that we can roll back that represents the rollout. But in this environment, in this particular environment, we haven't got thousands of releases yet and that's unnecessary friction we don't want to add. We don't want to have to have create those extra image update automation resources. We'd like to just be able to say source, applyer and pair them together and let those do their job. So where this says version, there is another trick that we can use. If we are using a Git repository or a Helm repository, I think, actually, sorry, not only Git repository and OCI repository source, we'll do this. Where is it? Helm repo. Okay, let's make a change here to this. And we have this one audience question already in, so whenever we're ready for that, let's do that one. Okay, but what I can do here, oh, this is where VS Code is gonna help actually, I'm gonna pull up this example in VS Code because VS Code has a little bit of awareness of the YAML because it's connected to our cluster and it knows about these YAML types. So if we go into this environment and we open this up, all right, so what we wanna do, we'd like to use the Git repository again for some reason. Maybe we've decided that OCI is too fresh and we're not ready to adopt it yet. So what we're gonna do now, instead of saying ref branch main, this is our staging environment, I think, yeah. I'm gonna say ref, remember, same thing we said in the other guy. All right, so we're gonna put that there. And then in our Helm release, we're gonna change this to use the Git repository. And this no longer matters. This one, I think we can leave as chart version because we are actually tagging and releasing versions and we are actually changing chart.yaml where the version number is, but you could change it to revision here if you weren't doing that. And there's one other thing we need to change because chart name is usually just a chart name. And here now we're pointing at a Git repository so it has to be a path. So I think it's charts slash bar. That's where we'll find the chart. Well, let's give that a deploy and see if that works. Oh, there's more things changed than I wanted to change here, aren't there? But what I'm trying to show here is that OCI doesn't drastically change things in a way that's unreversible. If you haven't yet adopted OCI, most of the same options are still available to you for the Git repository. So you can do this with an OCI repository or you can do this with a Helm release. The version spec goes in one of those three places in the Git repository, in the OCI repository, which has the same structure as the Git repository or in the Helm release. How will we know? That it's switched. Kingman, before we run out of time, there's an interesting question about automation for SOPs and secret management. Yeah, okay. So what we see here is it actually has switched and because there isn't a new version, we haven't gotten a new release. So if we pushed a new version at this point, it would come from the Git repository, but it would be using the Sembrer ref. So there are many ways that we can do this automation. Okay, so the question is about how to use this with SOPs and I have a really good direct answer for that. It's in the docs. If we go to the docs, I will show you what I mean. Refer to values and config map and secret resources. This is not the example that I'm looking for. Things have moved around in our search again. Let's see, where did it come from? Are you looking for a Mozilla SOPs guide? That's Tacey shared. No, I think I'm looking for this FAQ or maybe it's here, this one. Refer to values and config maps generated with customize. All right, maybe this is the page we were on already. So there are three or four good examples over here that are in succeeding order of difficulty. The very first one is about generating a config map. I think here. And you can use this to refer to values from a config map. Okay, so but you said with SOPs, that is the hardest example. We don't have time to cover all three of these today, but we do have this documented. How do you use SOPs with a helm release and a secret generator? And spoiler, this solves one of the problems that we saw in the very beginning is how do we get our deployment to reload when things are upgraded? We actually need a secret to be changed. The content of the secret changes and that needs to filter down to the helm release, which should also change. So what we see here values, pod info values is a name of a config map or secret rather that we've encoded and encrypted in YAML. And then we're decrypting here and we're using customize config to locate this reference. Here is spec values from name, here is spec values from name. And it's going to take and dynamically update this reference so that every time you have a new version of this secret, the helm release will get a new version as well so that upgrades proceed immediately. So hopefully that's the answer that you were looking for. Yeah. And then there was the final question. I think we have time for is OCI repository question mark. Is it a version control like Git question mark or an image repo like Docker? Yeah, it's a little bit of both. So the OCI repository in flux that's been added, the best documentation right now it's probably under the resources but then there's also this more expansive cheat sheet version that shows how we expect OCI to work. Their short answer is it's supposed to be an extension of your Git repository. It doesn't replace the Git repository. It has traceability back to the Git repository. So here's an OCI repository definition and here's a customization that's pointed at that. And here's some more examples. And this is the workflow that we expect people to use. So the user pushes to the Git repository which triggers a CI job and then you build an image for a container registry and the only thing that changes at this point is we're not only building a runtime app image we're also building something like a chart. It's an OCI repository artifact. It's not a template in the sense that a helm chart is a template with holes punched out that the user is meant to fill in but it is in that same spot in the workflow. So what this really enables us to do is it enables us to avoid creating a right credential for that first example. We want the first example to be able to write back to a Git repository. We expect that this is the first thing that your security team will balk at. Say no, there's no way we're not gonna make CI able to write back to the source of truth. It's unacceptable. So depending on your security posture that may be okay or it may be a problem but we have the answer for that. So hopefully that helps. Perfect. That's the time that we have for today but amazing to see so many questions and comments from everyone that's always lovely to see. But as always, thank you everyone for joining the latest episode of Cloud Native Live. It was great to have a session about automated communities deployments here today. And we also really love the interaction and the questions from the audience. Always great to see those. And we always bring you the latest Cloud Native Code every Wednesday. So in the coming weeks, there's gonna be more great sessions coming up. So tune in then. Thank you for joining us today and see you all next week.