 Welcome everyone. I hope you're here for the flux project update We're gonna get started now. My name is Kington Barrett. I'm a We've works. I'm an open source support engineer, and I'm a flux maintainer and This is my co-worker and co-maintenor some touchy Hello, my name is some touchy on your career. I'm a developer experience engineer we've worked on a flux maintainer And we're just gonna talk today about This data flux we're it's gonna be fairly informal Questions will be welcome at any time and Let's get started So the latest flux update. I think was Monday flux 0.36 and This update and my understanding it rounds out our integration with cosine So now if you want to verify your sources before you apply them to the cluster you can use cosine to do that Whether they are helm charts or not whether they're customized overlays. So this is big big news. That's great their support for both keyed and keyless cosine integration and We have a list of other OCI related features I didn't mention OCI there, but the the bit that makes it work together with customized manifests is the OCI Repository source support that was added in a few releases prior So there's been a flurry of releases in the last few Months and I wanted to point. Yep. I think it was that one. Yep This blog post that was it's the top one on the flux CD.io slash blog right now has the progression from 0.31 to 0.36 and You can see at this top or hopefully you can see is this large enough So we started Adding support for helm repositories of type OCI which has been a long time coming People have been asking for that feature for a long time and it was unblocked by helm Who brought that feature out of experimental and put it into the SDK where we could access it? So that was added in June and in August we added support for Customized overlays and also terraform code In OCI artifacts. So what that means is you can take your Manifests out of the git repository Which may be a mono repo it may be expensive to clone and clone again and you can push them to an OCI repository Very similar to an image repository similar standard but capable of holding all different kinds of formats of media types so You load your manifests into an OCI repository and then they have the contextual semantics of like an image so you can use the features of flux that are related to images for Manifests which has been another Difficult area in the past so We've also updated the flux CLI to also help you push if you if you sort of have a folder containing this manifest, right? There's a flux push active OCI command that helps you to push or pull if you've already pushed an image You can easily pull it with the Fox L I so basically it's would probably be most useful in your CI Maybe you want to do a release so you could use the flux CLI to say flux push This directory into this OCI image and push it to your OCI repository So in the succeeding releases, we've added a few more features for configurability with OCI and OCI artifacts and repositories You can read more about that Not exactly sure what the details of those are do you know 33 and 34 here Okay, it just means at first We expected the OCI in like the first layer of the image, right? That was like our first iteration of OCI So it needed to be like the first image, but now we've added some you could specify Some metadata for it to be able to find which Layer that the manifest and basically just makes it more flexible, right? You don't have to package is as a very first one. So that's basically like we're extending the features Yeah, and then in 0.35 the support for verifying cosine Signatures on OCI artifacts those flux push OCI artifact artifacts was added and Then in the following release now as I've mentioned before you can also verify helm charts that way so That's our dress welcome Do you want to come up? We might need an extra mic if we have a moment for So So we cosine and there are two ways you could use like kingdom mentioned earlier keyed and keyless Basically, we cosine you could generate keys or they have supports for keyless Verification of like your container images where it uses something like a github token to Instead of you, you know provisioning like a private and public key You don't need to do like the key management for you basically that's like the key less feature we could sign So if you're using the key or keyless feature and flux allows you to verify your active fact it always So I want to highlight Another thing that we've been working on is growing our contributor pool We'd like to add more contributors of different standing in the community like and user contributors You can join the flux community in a variety of different ways But we have this contributor ladder now that we've added in the last year where instead of having such a wide gap between Person on the street and you know github star giver maybe that's a nice thing that you can do Actually, please give us a star on github code flux CD slash flux to and add one would be great But you know you'd like to do more than that maybe or maybe you've already done more than that And we want to recognize that if you're part of the flux community So we have this project member status now that you can apply for You need to sponsors and they should be from two different companies If you if you know a flux maintainer or another flux project member anyone can be a sponsor And we'd like to see as many people are using flux Join us in the community and give us feedback. That would be great. So Any kind of contribution really is welcome and We feel like you all here are pretty in a pretty good state to contribute to flux because you know What's good with flux and also what's wrong with flux or what flux fails to get right? So if you have any ideas about you know, oh flux should be doing this instead of this or you know We created a flux could do that as well Like just you know try to create issues even that is for form of contribution. So we do really appreciate that as well I think we also want to highlight that hopefully no one here is using version one of flux But we will be archiving the repo really soon And I think we're still running migration workshops for people who want to migrate from the first flux We want to be to Putting a lot of work to make sure there's future priority between the two and of course V2 has been getting all the good new stuff. So if you're still on V1, please migrate You could come around the flux boots if you want more details on like joining in on the migration workshop So here's another thing on the website. I want to highlight. This is the flux roadmap you can see right at the top the focus is on production readiness and We're moving in the direction of GA. We should hopefully be able to release GA soon for certain definitions of soon Hopefully be graduating soon. Also, we have an application for graduation in the CNC F We're hoping to open up the comment period As soon as cube con is over. Yeah, they decided that We they can't do it before cube con so we have to wait on to keep going that's over and then they'll open the comment period there's a lot of work that goes into cube con and a lot of the people who are Their comments are necessary to make the graduation process meaningful. You know, we want to give space for them to think and Give us meaningful feedback. So that was a decision that the CNC FTOC made and We're looking forward to going moving towards graduation very soon We also have a formalized our RFC process now You could check Slash RC folder in the flux V2 repo so you could see if you have a feature request you we've basically formalized the process of you Know submitting an RFC getting reviews getting it approved Tracking tracking the various pull requests related to that RFC So you could take a look to know or what the current features we're trying to add in or if you have something to add You couldn't also create an RFC You can see here on the roadmap There are a few of the RFCs that we've already added there a few here for OCI You can see whether they've been completed or not I Have a few more down here towards security and a few that haven't been finalized given a number yet They I think they get a number once they're accepted as an RFC by the flux community. So and also so one of the RFCs like two of the RFCs I think are about multi-density and they're right now blocked because I mean we can't reach a general consensus Or we if we even if we can we're finding difficulties So if you want to voice your opinion how we how you want to see multi-density shape up in flux You feel free to like comment in the RFC or just reach out in the slack Or you know join a community meeting and bring up your concerns about how you want to do multi-density in flux Yeah, sans-garde just use the word Consensus by the way for folks who are not familiar with the flux governance model It's a little bit different than some other projects in the CNCF Where maybe a vote is necessary for any decision in the flux model We actually try to obtain consensus from the entire group of maintainers who are involved in the decision So if there isn't consensus generally, we won't move forward until there is consensus we think that we should have consensus and You know broad consensus rather than a tenuous vote-driven consensus Which we talk about next are there any questions at this point We could talk about the Various good improvements we have made over the last few months So as you know Get can be a bit tricky to get right because you have SSH and SHDP and What we have worked on over the past few months is improving our support for good So we have Made it basically easier to contribute to any any of our good packages. So if you go to flux city slash package There is a good package now So earlier it used to be a bunch of spaghetti code For the last two three months. We have been trying to refactor all of this out into a much more good-looking shape and We have also if People are you know like reading up on flux after a long time DevOps Azure DevOps and get a AWS code commit support that has been improved Significantly because we have made some changes to how we clone repositories using lip get to so that that Majorly improves clone speeds and also memory and CPUs it so you should so if you are Upgrading from something like I don't know point 0.25 or something if you haven't upgrade your flux and solution a long time And if you upgraded it to like I think point three six is the current one oh point three six Yeah, so if you upgraded you'll choose to see a major difference in time taken to clone repositories and overall CPU and memory usage of source controller I I think if you use like Azure DevOps or AWS code commit there's a PR out to basically For those that use it, you know that we have a git implementation field where you need to specify lip get to Because that's what work is works with those repositories But there's been a PR out to make you work with go gets which is the usual default So if you'd like to test it out is is not in a release yet But we have a pull request open and there's an image posted there So if you you know like to try out our new stuff you could go on there and test it out on your Home clusters or ever and give us some feedback. So the RC is out. We can just pull it up So like if you if you ever want to like try some bleeding at stuff So we usually post our RCs on the flux contributors channel on slack So if you want to try out some bleeding at stuff and you know help us figure out whether we were moving in the right direction Please try to use those RCs at RC images as well. You could easily do that using a customization patch Do you know which one? Oh It's this go good. Yeah, okay So it's this PR. I don't think we publish an RC yet, but we will soon. So it's this is the PR which enables go get for Azure DevOps and code commit as well. So with that you should see much more stable clones because This is like very into the weeds kind of thing, but go get is written completely in Golang But there is another thing which you use called it get to which is written in C So because obviously all source control and all our controllers in Golang We are forced to use ego to have like bindings between Libgit 2 and our controllers which is Painful to say the least Yeah, so Yeah, so so we are looking forward to this as well So this if so if this gets in right now it's draft and we are like still discussing it over But if this gets in you should see a much more stable Controllers especially image automation controller. So I I don't know how many of you here are running image automation controllers. Oh Okay, not as many suspecting but okay, so If you've noticed sometimes it tends to crash every now and then, you know, it's it restarts That is because of the seago stuff. So hopefully that will go away as well Sorry, just a quick checking that we don't understand. Is there anyone here who's hearing about flocks for the first time? Okay, okay, we have a couple of new people welcome Welcome. So yeah, sorry. We just sort of got right into it But flocks is a tool for github's hopefully, you know what github's github's is basically declaratively Defining your yaml's in the version control system the most popular of which is giths and you know flocks sits in your cluster and Continuously reconciles the states you've defined in your git repository to what is defined on cluster. So Just a quick overview of the controllers that we've been talking about source controller is for pulling in the yaml's wherever you've defined it So so far we have you could keep them in git repositories Estuary buckets, of course, we also work with Helm So we also have supports for Helm repositories And the newest thing that we're talking about is the OCI repository, right? For those who haven't heard about OCI before is this new spec image Spaced by the open container initiative. So basically the part of it that we're using in flocks lets you package artifacts basically files in your images just the same way you store your images in git You can store them in your in your images in container registries So if this maybe fits your use case you could package up your yaml's that you want to deploy into your OCI images And of course flocks, this is a pretty new thing and it's not pretty new but not a lot of people are doing it So flocks provides, you know command line tooling for you to be able to do that in your CI CD And it's part of what we're talking about earlier For you to be able to package up this manifest and push it to your image repository So the source controller also supports pooling OCI repositories Customize and Helm What we call applyers basically they they apply what source controller pulls in on the cluster So if you are already using customize or Helm This will fit right into your workflow The last three are the notification controller, which is basically for giving you updates in slack discord It could also post back to git The commit status notification, the blue check you have beside your commits Flocks could also give you an update there just so you could look at it and know that oh this commit deployed successfully The last two are the image repository and image automation Controllers which is basically for posting back to git when let's say your CI deploys a new tag A new image tag basically flocks can help you instead of you going in there and trying to just update the image tag Flux could do that for you. So that's just a quick rundown. There's not a lot of new people bored. Yeah, we're always happy to see new faces So broadly about the image automation. I just wanted to say Since image update automation is fairly complicated to get set up I want to make it clear for everyone. There's at least three different types of automation that we're talking about when we say automation and flux The first one is the basic Continuous reconciling pull automation that is described in the git ops principles So there is a git repository We need to pull it periodically to see if there's a change and when the change comes We want to pull it into the cluster and notify anybody who's waiting to see if there's a change so that they can reconcile immediately and get our update. So that's the Entry level automation that you get with flux when you first install it and the second type of automation That I would talk about is the image update automation that we've already mentioned This is an automation that will help you commit a change to your repository when a new image comes and the OCI support for automation for artifacts means now that you can use this type of automation with Manifests where before you could only use it with application images. So The way that this works is you write a comment in your YAML file that marks the line. It's called a K YAML setter And there's probably a good guide that we can show the example of and when you do that and you connect the image API resources in the Fashion as described here in this automate image update guide You'll get a commit every time flux notice is a new image So this can be a bit Clunky depending on your workflow like maybe maybe this is exactly what you wanted because you have an audit requirement Or because you have a rollback requirement and you need to be able to see a commit that you can roll back Every time a change comes that is not good or so on but the third type of automation that maybe gets overlooked an awful lot is the source automation So this is actually a lot simpler to configure than the image update automation. It doesn't create a commit in the repo Or it doesn't spam your repo with commits. This may be another way to say it What it will do it will just monitor for those things according to a policy that you set So let's look at the source In the toolkit components source controller We look at the git repository source. There's a similar section in the OCI repository source You see the spec Describes what what is it that we're going to pull and this one points at a ref for a branch master so every time flux reconciles it's going to pull that branch see if there's something new there and Update the artifact so that the downstreams can get it, but you don't have to pull from a branch There are a lot of different ways to write the spec you can write it with a tag a tag and Also some orange. So if you want to make sure that you're always pulling from 1.0 point something or 1.0 point 3 whatever is the latest for you you can do that of like Flux can filter out tags based on a similar policy that you can define and here's an example of that You imagine this says 1.2 point X or an actual somewhere range you can use greater than less than symbols the reference here that's linked is the mastermind somewhere so you can see Exactly what types of things you can say to get a different range of images or maybe you just want any image You can put a star in there and you'll get the latest semver That's a release. Yeah, I wanted to ask some people here who you're is evaluating Sorry, who you're evaluating using something other than YAML for your configs So like I don't know Q-lang or some kind of a json net kind of thing So, okay, so there is a very good use case that we have Kind of you know published here with OCI support So instead of so right now if you look at the state of flux It's pretty simple. You have a repository which has to have a YAML manifest, right? Like some deployment or cluster IP services, right? Source controller pull it and customized controller will apply it but What happens is that if you want to Basically use something like Q-lang right for to define your Kubernetes state instead of YAML With OCI that's possible so how that works is that You have you define your Q-lang files right in in a good repository and what happens is Every time you push it Every time you every time you push a change to your good repository, you can have a CI process, right? That CI process can basically you in that CI process you can do like a Q-lang build So which will output the YAML that the Q-lang actually that the Q-lang files will output, right? And that that those YAML files can be packaged and using what some poetry described using the CLI Right, you can do a flux build which will package those YAML files as an OCI artifact And then you can do a flux push which will push it to a registry an image registry, right? And then you can have an OCI repository object in your cluster So source controller will look at that good repository and so say that oh, there's a push here So let me know a reconciled stuff because a change has happened and then you have this OCI repository as well So like that actually you know gets the YAML files, right? So now you have you have a good repository You're declaring a Kubernetes state not using YAML using Q-lang or you know, whatever JSON net, right? but flux can actually understand that because of this Compatibility with OCI because you can actually you know just like get the YAML out and give it to flux in CI Because flux doesn't care where it comes from as long as it it's available to it. It can reconcile it So, yeah, so that's a pretty good use case and if you want to discuss more we can talk about it more So in general OCI unblocks a lot of use cases regarding as can be mentioned even like mono repos so if you have like a giant mono repo which has like a I don't know a good history and it's like it takes It's like 200 mb in size so it can take so clones can take like you know more time It can take more CPU it can take more bandwidth, right? so you can also use OCI repository to Help mitigate those kind of issues as well. So, yeah, we're really excited about the OCI support that has landed in flux Yeah, any questions so far you are welcome to interrupt us at any point just raise up your hand This is like very informal. We are so okay question So I noticed a lot of your bootstrap documentation has Just flux being installed via the like I'm sorry the cluster being bootstrapped via the flux CLI Do you guys have plans or what are those plans to support like things like terraform installs and stuff like that? We have a flux terraform provider Yeah, so the flux terraform provider. It's actually terraform provider flux is the name of it here and it is Sort of sits between the git repository provider and the file provider and the way I understand it maybe that's accurate and This is one way that you can install flux and also just to Mention this is also getting like some refactoring So like so some of the existing issues that we know with it So like Philip who's a maintainer who's not here right now. We he works at I think Janet. Yeah, so he is Making some good changes to this terraform provider. So to be even better in the next portion Pardon. Oh, is there a flux provider for close pain? Oh No, we don't have that yet you could contribute if Yeah, we're always welcome to New contributions. Yeah for anyone who has not done bootstrap before or doesn't quite understand from end to end What bootstrap is all about? There's I'm trying to get all these correct, but correct me if I skip a step When you run bootstrap the first thing it does is it checks if the repo exists And if it does not exist it will create it for you using an API and the next thing that it does is it pushes a definition to that repo Well, it pushes a deploy key. I think and a definition for flux itself So that flux can be managed from the repo So the idea is that if you want to have a closed loop one of the rarely Quoted principles of GitOps because it's not in the official list then you can do that. So after that deploy key is in place The bootstrap process will install flux from that definition and then it will add flux system Git repository and customization and start them so that you have GitOps running on your cluster. Yeah Also, I just want to quickly ask how many of you Like what is your update process look like? How do you update flux? Do you are you listening for change log change logs or? hard how regularly do you update flux just like like Less than every two weeks or just do randomly like oh let's update flux. It's been a long time Okay, so we always recommend you to be on the latest because There is this like you know, we push out so many security fixes every now and then So this is a good option that you can put in a good repository which has the flux manifests which basically Enables flux to update itself. So flux cannot update Whenever there is a new change. So this runs this runs a cron job. I think every day. I Think every night, right? Every hour here. Yeah, so this one's a cron job Which basically checks if there's a new flux release and it will then tell your current Installation of flux in your cluster to get though get the new release and then auto update itself. So that's pretty good Yeah, so that's one of the reasons that the terraform provider is not always on the top of the list when we talk about ways to install Flux because we like for flux to be able to manage itself and we'd like for people to be able to manage flux with GitOps Which now it's also possible through terraform. Thanks to the terraform controller But that that's a whole other presentation. We're running out of time today. So Terraform controller is not officially part of the flux CD projects. It's Project by we've works, but it is built on the GitOps toolkit and it is open source So please check that out as well How many of you is anyone using flag or you as well? Okay, yeah, we've got some flag or users. Oh, nice. Hey, I just want to ask you about this specific workflow You update these flux manifest and then you create the pull request So that when this pull request is approved then another workflow will run that Performs the update. Oh, am I right? So no, so this creates so like fluxes Installation itself is stored in the GitHub repository Right that flux itself is reconciled. So it's it's sort of a loop if you think about it Like it can get like a bit inceptiony like it's a sort of a loop, right? So so fluxes state is also in the GitHub repository. So what this action will do is it will check, you know, whether there's a like a new release So that so the new changes Are pulled and then a pull request is open to the GitHub repository which has the flux manifests, right? So when you hit the merge button on that PR that change will be merged into your like Let's say main branch which is supposed to be and then the it's the current installation of your flux on your cluster We'll notice key that there is a change in my main branch So let me pull that change and reconcile that and then flux will auto auto itself because you know the changes are now in GitHub Okay, yeah, this is what I meant. Thank you. Yeah Yeah, that's why the PR right like it's not a it doesn't push it creates a PR so you can review the changes a Related question to the previous one and you mentioned is there a flux way of Install the flux bootstrapping process so So very before you upgrade worry about upgrade of flux itself The very initial install of flux agent in the clusters. Is there a flux way of doing that? Yeah, there is the most flux way there might be you can stall flux in a cluster the most flux way I would say is using the CLI so there is a flux CLI So you give it a cube config with which is pointed towards your Kubernetes cluster and then you run flux bootstrap So that will essentially do everything Kingdon talked about it will it will pull up the it will create a GitHub repository for you If you if one does not exist and then it will pull up the Manifest and then it will apply those manifests and install flux for you on the cluster That is I would say that is the most recommended way If anyone has more questions, you have any questions this we have the first before the floods booth And there's the CNCF flux channel where you can reach us And thanks everyone for coming today. I hope you have a great QCon Thank you