 All right, we're going to go ahead and get started. We're going to turn on the recording so the slides will be available for you as well. I would like to thank everyone who is joining us today. Welcome to the CNCF webinar, Making the Most of Helm Tree. I'm Archie, Hybrid Cloud Specialist at Google. I'm also CNCF Ambassador from Canada. I'll be moderating today's webinar. We would like to welcome our presenters today, Dan Garfield, Full Stack Engineer at CodeFresh, and Anna Baker, Software Engineer, Technical Writer, and DevOps Evangelist at CodeFresh. Welcome, guys. Thank you. We're excited to be here. Excited to have Anna with us. And excited that you all want to talk about Helm, get to celebrate the birthday with us. Just to introduce myself, I am Dan Garfield. Dan, if you just allowed me, I would like to quickly do the housekeeping. Oh yeah, go ahead, sorry. A few housekeeping items before we get started. During the webinar, you are not able to talk as an attendee. There isn't Q&A box at the bottom of your screen. Please feel free to drop your questions in there, and we'll get to as many as we can at the end of the webinar. This is an official CNCF webinar. And as such, it's subject to the CNCF Code of Contact. Please do not add anything to the chat or questions that would be in violation to that Code of Contact. Basically, please be respectful of all of your fellow participants and presenters. With that, I'll hand it over to Dan and Anna to kick off today's presentations. Have fun, guys. Good luck. All right, thanks. So we are going to talk about making the most of Helm 3, and we're going to go through a number of things. Just to introduce myself, my name is Dan Garfield. I am on Twitter at Today Was Awesome. You can follow me for fantastic tweets of all kinds. And I'm the chief technology evangelist for CodeFresh. Anna. Hi, everyone. I'm Anna Baker. I'm the DevOps evangelist at CodeFresh, and you can follow me at Anna underscore CodeFresh on Twitter. And Anna joined us from Red Hatch, a pretty badass engineer that's been on our team for a few months now. So excited to be presenting with her for the first time. Just to kind of set the stage for why we're doing this. We're from CodeFresh, which is a CI CD solution that's focused basically the default CI CD for people using Kubernetes. And we have had for a long time, people on staff who were Helm contributors. We've done a lot of work to contribute to Helm as a company and a ton of our users use Helm every day. So it's really critical for us to get the support right and have great integrations for it. So this is kind of the reason why we're so invested in Helm and why we're so engaged with Helm. And with that, we'll get right into the agenda. We're going to go through just an overview of Helm for those of, that might not be super familiar with it. And then we'll go into migration, what's new and get into some demos. And first off, this is actually perfect timing because Helm just graduated from the CNCF. It's now a graduated project as of last week. So notice that there was an intense amount of interest in this webinar today just because this is now an officially graduated project from the CNCF. For those of you that don't know the criteria, one of the really critical things, and I think the one that people are most interested in is that that means that Helm has passed an independent third party security audit. So it is considered a secure production ready solution. And it also requires most of the technical oversight committee to a super majority, not just the majority, the technical oversight committee at the CNCF all has to vote and agree. And basically the idea behind graduated is that everything's been done to make this project ready for the majority of people to use it. So if you're not familiar with crossing the chasm, basically the idea is that there's this ramp up in a product. And you have this early adopters group. And then you get the early majority of people and then graduated is meant to sort of symbolize, hey, it's now ready for everybody to use. So that's, that's wonderful. And that means I expect we're going to see a lot more adoption. I've noticed that, you know, Helm two, which has been out previously has had a pretty diverse group set of adoption. There were a lot of people opposed to adopting Helm two because of tiller, which we'll get into. And now with Helm three, the breaks are off. Everybody's welcome to jump into it. So if you're not familiar with Helm, I like to use the analogy that if Kubernetes is the operating system of the internet, Helm is the package manager for that operating system. And I really do treat it like that when I'm using it. So you can do things like Helm install and then whatever packages and there are an enormous number of packages out there, hundreds of thousands of packages that you can use. And they're, they're very, very useful. It's very useful in this way because you can actually use packages as dependencies. You can say, I've got my home chart and it relies on this public Helm chart and it will, it will pull it in for you and install those dependencies on the cluster. And you can configure values and reasonable modifications to that. So the reasons why people use Helm, there, there are really a couple of number one, you have the ability to actually keep track of all of the revisions of what's been deployed to your cluster and you could do rollbacks. So that's really useful. I know a lot of people are living in the GitOps world. And I think that that's the correct choice. People should be doing GitOps. However, sometimes you end up in situations where your, your Git might not be in a state where it's easy to rollback. You might have problems with trying to merge everything and get it all figured out. And if you're looking at stuff on fire and you're having downtime, having the ability to just run a Helm rollback and get back to the revision that you were previously on while you work out the Git situation, incredibly useful. Also integrates really well with CI CD because you can templatize your Kubernetes manifest and just add in and inject values as needed, which is really nice. One other thing, a few other things to mention, obviously installation is very simple upgrades are very simple. And, and of course dynamic values is really valuable, especially with CI CD where you might be building an image and you don't know what the tag on that image is going to be before you, before you start it. And so, instead of having like a separate pipeline, you can actually dynamically create an image and inject its value into your Helm chart that you deploy for something that is repeatable and reliable. Just the structure of a Helm chart, you essentially have Kubernetes manifest that are templatized and you can have values in there. And then we have a values YAML that will set default values. And oftentimes, so what I do personally, with, if I'm consuming charts from, from the stable repositories out there, all lots of times just create a values file and that's it. That's the only code that I maintain and then I can just consume the chart that exists out there. And that's very convenient. And then every time you actually install a chart, it actually creates a release on the cluster that keeps a record of everything that was deployed there. So using this is pretty easy. You can install a Helm chart from a repository. Like I just mentioned in the example a second ago, you can install one from a local archive. So you can actually package the whole Helm chart, put it on to like a thumb drive, for example, go that direction. You can also have an unpacked chart directory. So like you might clone a Git repo and install a chart from that repo or you can install it from a URL specifically. So this gives you a lot of options for how you consume those Helm charts and Helm really has two main verbs. I mentioned rollback earlier, but you have push, which will push a chart to a Helm chart repository so that you can reference it from other locations. Or you can just install that chart and create a release or update a release. So new in Helm three, the biggest piece of news is Tiller is gone. And for those that are unfamiliar with Tiller, and we'll look at a diagram here in a second to explain this. Tiller is basically this component that sort of had super user privileges usually on your cluster and could actually run installations. You can kind of think of it like a prototype for operators. So if you've ever used an operator, Tiller was sort of like the proto operator and it existed before there were things like CRDs or operators in Kubernetes. And so because of that, it had some architectural limitations and it is the number one reason that people did not adopt Helm two. Anybody that chose not to adopt Helm two, their chief complaint was almost always Tiller. Now there were ways to make Tiller secure, but it was fairly difficult. So Tiller being gone means there's no server side super user component to Helm anymore. And that means that all those security concerns are essentially gone. And there are a few other things that we'll talk about to resolve that. Another one is in Helm three, we also have three way merges. So the way that this works is in Helm two, if let's say that I had installed a chart and then I had made a modification to it. So I've installed a chart that's a release. And then I just go in using coop, you know, coop control and I changed the number of number of replicas and I up that. Well, if in Helm two, if I did a rollback to that version, it would see that there was no difference in the number of replicas between my new version and my old version. And it wouldn't update it, even though it had changed in live state. So now with Helm three, it actually looks at the live state and says, Oh, actually this thing has been changed. There's been drift essentially since the configuration has been applied. So we're going to actually change those things back. Previously in Helm two, you had to use a force command to essentially treat everything as like a recreate, which is a little bit more messy, less desirable. So that's, that's another great news for Helm three. A lot of people were interested in how Helm three was going to use Lua now that is still planned, but it has not happened. There is no Lua in Helm three today. There is a plan eventually to have some programmatic elements in Helm three, but it's probably far out. I think Anna, you were guessing that it would probably be like Helm four before we saw. Yeah. Yeah. Hopefully. So I wouldn't, I wouldn't look for that. I'm kind of bummed about it because there was some plugins. I was working some tooling I was working on for canary releases. And I was thinking, Oh, you know what, I'm not going to upgrade this because once we get Lua, I'm going to do this in a totally different way. And that was two years ago. So, so don't do what I did. Just, just get to work and work with what you got. Let's see. And then there are also some changes in name spaces. So previously there, there was some idea that you would automatically create a name space for Helm three. And then there was some idea that you would automatically create a name space for release. People didn't decide not to do that. Wasn't, didn't really make sense in the end. So that's that's not a thing. And then also we're using secrets for storage now instead of config maps. So if you think about the architecture of Helm two versus Helm three and Helm two, most often you had tiller installed in a specific name space, usually the coop system namespace. And that tiller instance usually had super user privileges. Now you didn't have to, you could work around it. You could do a tiller per name space. That it was a lot of work and it was hard to maintain. And that meant you had to, if you had like 10 namespaces that you would, you had 10 tillers and you had to upgrade all of those tillers every time there was a new version. And you had to maintain all the permissions and privileges. And it was fairly difficult to work with. So that was a bummer. And then you had your applications that were running in their own namespace as usually, or you know, even the default namespace, but all the information that tiller consumed was stored inside of config maps. Now config maps were used kind of because of the limitations of Kubernetes at the time, but config maps actually aren't a great place to store manifests for one thing. You don't have encrypted variables in there. So that means that if your Helm chart had secrets in it, it's now just sitting in plain text in a config map in Kube system. So that's a bummer. And the other issue is that config maps have a limitation in size that could actually cause problems. So you actually saw people who, hey, I can't, I can't deploy a new version of my Helm chart. And it's because they have like 700 versions stored in a config map and it won't let them push new, new elements to it. And it also caused some issues with potential instability and like you would go to do like a Helm list on a release and it would like slow your cluster down because it was just trying to pull like this super large config file, config map. And it was a big pain at Codefresh. We actually developed this whole caching system for the config maps so that we could handle those things. So that was, that was a bit messy. So in Helm 3, it's much simpler. Because we have CRDs, what Helm 3 does, it actually stores the release information in secrets in a custom secret CRD. So if you look at, if you did like a Kube CTL get secrets and you put a field separator, field selector type equals Helm.sh release. You can actually see this is, this is from my personal cluster. So you can see what kind of Helm charts I've been consuming lately, but you can actually see those secrets. And each time you release a new version, it creates a new secret. And then Helm actually does some work to by default that will actually trim how many it keeps so that you don't get overwhelmed on your cluster. And these secrets, this CRD element actually lives inside the same namespace of wherever the application is, which means it's subject to just the same permissions that you use to create that application. So the permissions management element of this is much, much simpler. The security element of it is much, much simpler. You don't have to worry about Tiller. You don't have to do all that. And I noticed that somebody just asked, what's a CRD? It's a custom resource definition. So it's actually the main way that Kubernetes in Kubernetes that you create extensible objects. So the release object for Helm 3 is essentially a custom resource definition within Kubernetes. And so that resource is essentially a secret with a few different components on it. So you can see that the type on the output here is actually a Helm type, Helm release type. So let's see, any other notes on the architecture here, Anna, before we start getting into the migration path? No, I think I'm ready. Cool. Just a, just another note, I mean, we didn't really get into this with Helm, but when we talked about the, why we use Helm. The templating thing was a big one. So the ability to inject variables. The other is distributing your application for others to use. Since it is the package manager for Kubernetes, if you want to distribute your application. So for example, Minecraft has a Helm chart. So people can actually just go and install that Helm chart and they'll have a Minecraft server installed on their cluster. Same thing for my SQL, same thing for these other elements, right? So that ability to distribute your software as a package for others to consume is really useful. Now a lot of people internally don't actually distribute the packages to other people within their group. Hey, I made, I made a home chart for my application. I don't distribute that application. I just deploy and maintain it. So they don't necessarily worry about the distribution aspects of Helm. Instead, they're really focused on the values aspect. So anyway, with that, let's talk about how we actually take these config maps, turn them into CRDs. Let's talk about how we actually migrate from Helm two to Helm three. And for that, Anna has actually prepared a demo. So I'll stop sharing and you can take over. Okay. Cool. Can everyone see my screen? Yep. Looking good. All right. So I have already installed Helm two and Helm three. I have an alias. So home two is alias to home. And home three is alias to home three. As you can see, there's no server component of home three. Because tiller was removed. So what I'm going to do first is install a chart using Helm two. And now that we have that, I just want to show the migration process. So in order to migrate, you need to use this plugin called Helm two to three. You can install that from GitHub. It's linked in the slide deck if you're curious. And if we take a look at different commands, we have available, we have a cleanup command that will release data and destroy the tiller component. Convert, which will migrate the Helm version two releases to home three. And we have a move command, which will move the home to configuration in place. One thing I wanted to note is that with Helm. Two. The stable repositories automatically added for you, but in home three, that's not the case. So when we move our configuration, we should see the repositories added to Helm three afterwards. So let's do three, two, two, three. Move. Do a dry run. Yes. So as you can see. The, the different operations that are being done on home two and home three, it creates a new config directory and data directory. So we can go ahead and run that without the dry run option. I think I have to run that as pseudo. Yeah. So here, here really what we're doing is just taking our Helm client. Preferences. And settings and migrating them to Helm three, like Anna mentioned, you don't have any Helm three by default. You don't have any, you don't have any like Helm repositories added, for example. And it'll move your plugins as well. If you, if you have any plugins. Okay. And we're mentioning, I think, I think you might have said it, Anna, but having the, having Helm three as the command is something that I see a lot of people do. If you're just installing the Helm command line, it'll just install the latest version, which will be Helm three, it'll just be Helm. So Anna has actually created an alias here for that specific binary just so that you can see the Helm two client and the Helm three client on the same machine. So in order, that's not going to be the default setup. If you're just going to install home. Take a look now. You can see now we have the stable repository listed because it migrated it over from our Helm two settings. Three, two to three. All right. So the next step we're going to do is convert that release. We install, or that chart, we installed the MySQL chart over to Helm three. So let's do that. Two to three. And you have to do these one by one or create a loop. We only have one installed. So it's not a big deal. So home three, two to three convert MySQL. Now, if you take a look at our version two, we still have the MySQL chart deployed. And if you look at version three, we also have it deployed. So the next step is to clean up. So this is the last step we'll do. This will delete the Helm two release and the tiller component. So if we take a look at our pods and COOP system, you'll see here we have tiller running at the moment. Oh, do a COOP CTL get a config. So you can see the config. Yeah. So we have the MySQL config map. And we have the MySQL secret. Three clean up. So it's deleting everything in Helm two as well as tiller. So if we do home list, it says could not find tiller. And as you can see, it's been removed. And then pull up the config maps too, because they shouldn't be there anymore. Yeah. And that's about it for migrating. It's pretty simple as long as you get the plug in. Now. Oh, go ahead. Oh, I was just going to say you could afterwards move. Yeah. Yeah. So the main reason like why you would even want to go through the migration process is really all about just keeping the release history on your Kubernetes cluster. Right. If you don't care about keeping that release history there, you can just uninstall the components and move on. Like you can just run the cleanup and just install using the help three command line. And I actually haven't seen really any issues. I've been using Helm three now for a few months. And I haven't run into any weird situations where like something didn't work. Like everything kind of has been working the way that I have expected it to. And I actually in some situations didn't even bother to like go and get rid of the config maps or anything on Helm three or for Helm two. So this, this plugin is handy and there is a link to it in the webinar slides. So, so when you get those, you'll have a copy of it. Let's see. And then we're going to go into the CI CD workflow. Just want to check lots of good questions in here. So we're definitely going to get to these. But from a, from a CI CD workflow perspective, what we most commonly see is people will have their get repo where they make a change that goes to CI CD, which builds their application that builds the, the Docker image. It pushes that Docker image up into a repository. And then we packet, we create a Helm package and we can either push that to a repo and deploy that, or we can deploy that Helm chart directly to the Kubernetes cluster. Now, why would you do one versus the other? Well, if you push it to a repository, then that means it's an object that you can reference from anywhere. You can just say, hey, go install this home chart. Done. Right. And it also allows other people to consume that Helm chart as a dependency for their software or to install themselves right in their own clusters. Now, if you don't need other people to use it and you don't care about having a, you know, a single distribution point for that, then you don't need to push it to a repository. It's a good idea to keep track of it, but you don't have to. You can actually just, Hey, I just checked out my, my repo. I ran my bill. I built my Docker image. I deployed the Helm chart that I rendered and deployed the Helm chart in a single step. And even if you do like a, a get revision, you know, if you, if you roll back, get to a specific revision, everything should just work. You know, so you can still drive. You can still be totally get ops oriented with Helm. And everything is just going to be derived from the, you know, the specific get revision. So that's a pretty common flow. And I think Anna, you've got a demo on that as well. Right. Yeah. Okay. Can you see my screen? Yep. All right. Okay. So I have a pipeline here that I'm using code fresh for to store and deploy a Helm chart to Kubernetes. So I'm going to run it. Well, it's running. We'll take a look at one that's already been run. But the pipelines and code fresh run from left to right. Each step is, composed of a Docker image. We have four steps here. One that clones the main repository. One that builds the Docker image. One that stores the chart in a repository. The built in repository that comes with code fresh. And one that deploys the, the chart to Kubernetes. So if we take a look at the, the, you can see here, I mentioned already, we have a clone step, a build step, a store step. But the cool thing here is I'm using two steps that implement the Helm. Step from our step marketplace. So we have this cool marketplace here. With all kinds of steps available. Different integrations. But we'll be taking a look at the Helm. Step. And if you look at the source code for this. For any of our steps. They are all just Docker images. So you can easily build upon these or create your own. But anyway, the. Let's see. For this pipeline, I'll be using the push action. You just define up an action. Push install or off for off. You can run your own custom Helm commands. And then installing. Actually deploys it to Kubernetes. And so one thing, two things to note. The only difference here to make this two to three ready is you just need to change the version number you want to use for Helm version. And for Kube context. This should be automatically integrated with your code fresh account. It's a global setting. It's just an object. And if you refer to it by its name in the step, you can automatically access your cluster. So let's see if it finished running. Did. So after this. It should end up in your. Helm repot or your default repository, which mine is here. And then you can run your own. I think you can see my terminal. Oh, no, we can't see your terminal. My terminal. So if I run. On three lists, you'll see it deployed here. But that was using code fresh. To deploy. It was pretty simple. Yeah. And then you can run your own. And then you can run your own. And then you can run your own. It was pretty simple. Yeah. And that, that deploy step is public. It's open source. You can find it at code fresh.io slash steps. And feel free to use it. And so just convenient way to work with both helm two and three, because you can specify which version of how we want to use. So with that. I'll let's see. Can I take the screen back over? Yeah. Yeah. All right. So related resources, obviously, there's a lot of things that you can do. You can do that. You can do that. You can do that. You can do the helm documentation. Code fresh has a lot of documentation around helm specifically within the use of CI CD pipelines. There's actually, oh, I should have put the link in here. Put it in the chat. Taryn will put it in the chat. There's a ebook on helm best practices. With CI CD. That I think is really valuable for people that are doing it. And with that, I think we'll move into questions. You're welcome, of course, to go try any of this out at code fresh.io. You can do that. You can do that. You can do that. You can do that. You can do that. You can do that. And there's also a view of all your helm releases that works for both helm two and three. So you can actually see what's moved into helm two and what's moving to help three. There's just a little switch there that you can use. So. With that, I think Archie, you're going to come be our moderator and, and tell us which questions we should answer. Yeah. Awesome. Thanks. Dan and Anna. For a great presentation. I hope it was really interesting for everybody. Thank you. Thank you. Thank you. I have some time for questions. If you have a question that you would like to ask, please drop it in the Q&A. Tab at the bottom of your screen. And we'll try to get, get back to you as soon as possible. So I have a few questions already dropped here. We save plenty of time for questions. We can have a whole discussion here. Great. Yeah. So one of the question is from Prem. I think you shared a few links already about the learning home. CACD practices. There are other resources that you can recommend. Anna, you, you basically just learned how I'm somewhat recently. Yeah. I've been learning it recently and I've just been going by the official documentation. It's pretty thorough and useful. So I would recommend starting there. Yeah. And I, I don't know that I would worry too much that you're going to learn something. That is for home to that's not applicable for home three. I mean, besides like Taylor and config maps and kind of the back end of, of how home is actually working. Most of that stuff still applies. I mean, the template is still works in the same way. Really? The way that you actually just install an upgrade charts. From a, from a command line perspective is basically the same. So I wouldn't worry too much that you're. That you're learning too much about home too. If you're spending a bunch of time on figuring out how to secure Taylor, then you know that you went around on the wrong path. You don't have to care about Taylor anymore. But other than that, you know, stuff is mostly pretty straightforward. Yeah. I just want to add then that, you know, just, there's a lot of materials everywhere. How to, you know, use helm. But if you want to build your own helm charts, you can probably also brand start with the helm create command. This will create like a boilerplate for you. So from there, you can, you know, customize to create your own chart. And then obviously there is a Kubernetes Slack channel where people will be happy to help you out as well. Yeah. Great idea. The V shell is asking about like config maps. I CRD is playing. Yeah. No. So if you actually look in those secrets, they will actually do some work to obfuscate and hide. Sensitive items in there. So, but, you know, by default, I'm trying to think here. By default, there is some work that's done in there to help you out as well. Yeah. Great idea. The V shell is asking about like config maps. I CRD is playing text too. No. So if you actually look in those secrets, there's some work that's done in there to help hide, hide values. However, there's potentially additional work that you might want to do for super sensitive stuff. Because I remember looking around in a secret scene, a bunch of encrypted values. I don't know, Anna, do you have more on that? Not really. No, I'm not. I just learned about CRDs as well. All right. Thanks. So fan is asking about what's the advantage of using operators of the helm tree. This is such a great question. I love this question. Okay. So operators are actually solving a different problem. Operators are a way that you define how a specific type of application installs upgrades, pulls down scales. So for example, if you're using a state of application like my sequel, you might have a very specific way that you want that to start up and stop, right? Especially because you have to deal with the storage and it's a little bit more complex. And so you might have a specific procedure for, for when different pods need to start and in which order things need to be done. And so operators are really designed around the life cycle of an application usually. And some people go hog wild with operators. I know a company that has over, frankly over a thousand custom resource definitions, each with their own operator associated with it, which is crazy. I don't know if I would recommend doing that. But helm is really about packaging and templating an application. So it's really about the definition of that application, less so about the life cycle of that application. Now you do install help charts and you can do rollbacks and you can just create temp. You can just output the manifest. But operators are really designed around the managing complex life cycles of applications rather than just creating packages for them. So you can actually use operators with helm charts, for example, you can have an operator that consumes a helm chart and installs your application that way. So they're not really competitive. They're more potentially complimentary technologies. I personally actually think that operators are probably a little overused right now. You know, it's a new feature that we just got like six months ago and people, well, I guess a year ago, and people kind of went hog wild and we're like, oh, I'll just create an operator for everything. And you can even see some people have thought, well, we'll actually distribute our software through an operator install our operator in order to consume our software. And there are some reasons for that, but it's not a replacement for a package manager. It's not meant to be. Yeah, totally. I see a lot of people using Prometheus operator and deploying it as a helm chart. As you said, you can combine those things because, you know, helm is packaging the application and then operator is really focusing on the life cycle of the application. All right. Going to the next question from Tiago Carvalho. He's asking about, is there a way to roll back database schema changes? I suppose if it's a, you know, you're deploying SQL and then you have some changes in the schema and you want to roll back. Yeah, this is actually where an operator does become valuable because helm is about applying Kubernetes manifests. If you think about what helm actually does, if you run the command helm template, and you're charting your values, it will just spit out Kubernetes manifests. All helm is really doing is templating and then applying those manifests. So it's actually not, it's not necessarily doing a lot of work to manage the life cycle of something like a database schema change. That's where an operator could be more useful. Or in this, you know, what I see a lot of people do is it's, it's less about rolling back a database schema change. And it's more about doing backups and having like custom and kind of scripting around changing stateful applications. I think that stateful applications and Kubernetes are still somewhat immature. I think Kelsey Hightower has a lot of opinions about the state of state in Kubernetes. But basically it means you're going to probably have to do a little more manual work is what I imagine. Is that what you, is that you guys concur? Yeah. Yeah. Yeah. Yeah, same thing. Okay. Yeah. So I'm not asking about, is it possible to use how I'm during the Docker build process? Oh, that's interesting. Is helm involved in the Docker build process? No. No. I think it's, then there's a step before, right? So we just, we can definitely put the build off Docker process in the CICD pipeline. But you know, the packaging of application is going to be in the next step of the pipeline. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. No, actually, actually one more thing on that because, because I think that maybe our CICD diagram from earlier might be a little bit misleading on this. And maybe this is why there's some confusion. A helm package. Is just manifests. It's just manifests with variables, essentially. It will include a reference to where that Docker image can be found in a helm repository or in a Docker registry. So if you want to do something like, if you put like a helm chart on a thumb drive and took it into an air-gapped environment and plugged it in and said, Oh, go install this chart. It would say, okay, here's all the manifests applied. And then Kubernetes would look at the Docker images and it wouldn't be able to get them. Right. Cause they're actually packaged in the Docker image. There's another standard for this called CNAB, which is cloud native application bundles, which is concerned with the idea of creating like an offline package potentially that you could just, it includes the Docker image and the helm chart and everything. So that's what you would need that for. So the helm chart is really just about manifests. The Docker image is about the Docker image. All right. Felipe Santos is asking about plug-in work. Is it available? So basically, will the plug-in work on charts that aren't yet in home three compatible? Do you have, do you know anything about the plug-in work? I actually haven't seen, I have heard that there might be some potential issues with charts that are not home three compatible. I haven't actually seen any, any chart that I've tried that has been, that I made with helm two just worked with home three. So my experience has been as well. Yeah. And I've heard somebody say that maybe they had an issue one time, but I just haven't seen anything like that. So I mean, when in doubt, you know, use a, use a staging cluster, use the dry run command to actually see what's going to be rendered. And, and I mean, like 90, 99% of the time, I think you're going to be fine. And if it works in staging, you know, you can be pretty sure that it's going to work when you've played it a production. So I wouldn't, I'm not terribly worried about it. Thanks Dan. Daniel Zilberman is asking about, I think we partially answered this question is what if you don't have a helm chart yet, but would like to define one is any DevOps tool to help put that task. So I mentioned that you can do helm create. And it's going to build you like a boilerplate chart that you can start to work with and, you know, start learning the goal templating, but maybe Dan or Anna have something as well to recommend. Isn't there draft in. Yes. Draft is what I was thinking of. Yeah. Thank you for, you remembered it. I was just Googling to figure out what the name was. Yeah. Draft is a tool that is designed to help you create. Manifests and helm charts, I believe. Yeah. Yeah. So you can actually use draft to do this. It's draft.sh is the website. It's a pretty cool project. I think it's backed by Microsoft. And you can basically just do like draft up and it will. Build it. It will build a Docker file and try to figure out how to do it. But also just browsing other people's helm charts. They're fairly easy to read. You know, usually people will organize it with like, Oh, this is a deployment YAML. This is the service YAML. And they'll have, you know, the specs in there. So they're, they're fairly easy to, to get into. Yeah. I would just probably recommend before going to create your own chart. You can go to the helm hub and, you know, search. If there's this chart is already available by somebody, you know, and if you want to use that one or and contribute to it. Or you can potentially, you know, fork it and, you know, create your own chart from that as well. Yeah. Okay. Moving forward. James Parenburg recommendation for learning to write manipulate helm for non, for non-go programmers. Recommendations for learning to write manipulate helm for non-go programmers. Well, I think a, I can answer partially this, you know, you don't need to learn goal because it's a goal templating is not exactly what you're looking for. It's a little bit of, you know, kind of part of the goal that you just need to understand how it works. So you don't need to, you know, learn all the goal language to use and to build helm charts. So just kind of look into the documentation and you understand the patterns and you're going to become pretty familiar quite soon with that. Yeah. You know, the whole point of, of helm is that you don't have to get into that kind of stuff. It's really just the templating and it does use go templating. You're right, but it, um, like the, the only time I've had to really know any ghost syntax was when I was creating, um, uh, scalable home charts that could create, um, multiple deployments with different options. And in that case it's just, it's just defining it in an array. And if, if you have a, if you have a, I have a repo skithub.com slash it was awesome slash color dash coded. I'll put it in the chat. And, uh, you can actually see under, I'll throw a link in here. You can actually see a, uh, a home chart that takes values and creates multiple deployments based on the values that are defined. Um, um, and you can see exactly how that works and how it sets context and stuff. So I'll throw that in here. Great. Thanks. Have I see another great question coming from Kenneth Riedler. So he's actually asking question that I was hoping to ask as well, but I wasn't sure if I can. Uh, so how do you describe the applicability of relationship between helm and customized? Included in Kube-Karol. How, if at all, would they be used harmoniously? Oh, that's a good question around using customize and helm together. Yes. Customize, it's not a CNCF project, but it's part of the Kubernetes chart repository. Right. So, um, customize allows you to, uh, without modifying your, uh, Kubernetes manifest deployed to death staging and production. Um, I guess, you know, people are asking why would use, uh, both or separately. Yeah. So I'm not a, I'm not familiar with any conflicts between, customize and helm three. Um, but I also don't have very much, very, very little experience with customized. So I wouldn't consider myself the expert on that by any means. Uh, I'm not aware of any conflicts that would happen and can't imagine why there would be a conflict between those two. Um, my, my understanding of customize is it's really about templating, not so much about packaging. Right. Um, but I, I could have that wrong again. I, I don't know customized that well. I think you're right. And, uh, in terms of like how to use harmoniously, I would, I would also actually, uh, I thought a lot about this thing. And I think one of the ways you can still use helm chart, but you can apply your customization on top of helm chart. Sometimes this is, you know, requirements. Uh, so you applying customization on top of customization, that could be potentially, that could be, you know, CACD situation and whatnot. So it is possible to use them together or one of another, but the, you know, the nice thing about helm is that it's packaging the application and, you know, you can distribute it to everywhere. Whereas with the customize, it's, it's really focusing on the solving the problem of, uh, you know, deploying in a different environments. Customizing them. Both are great tools and they're highly recommended. Uh, silver Kumar, uh, my applications consist of many services and deployed for few environment in some environments. I don't have to have all the services, uh, sort of like addition, any suggestions on structuring the helm charts. Would this be a one home chart with the variables to enable disabled? Or is this, or is there is a mechanism to set up dependencies? I'm not sure I understood that question. Whose question was it? Uh, Silver Kumar. My application consists of many services and deployed to few environments in some environments. I don't have all the services, sort of like addition, any suggestions on structuring the home chart. Let's be one chart. Uh, okay. So yeah, you can actually do a chart of charts. Um, so, uh, so for example, in code fresh, um, there's a cool webinar on this about how we do CI CD for code fresh because we actually have this, uh, conflict complex stack of microservices. Um, and you can create a chart of charts that is a single chart that references several chart dependencies and it will go and make sure that those are installed. Um, there are some challenges with that. So for example, if you have deployed a dependency to a Kubernetes cluster that wasn't deployed using helm, your application would work, but helm wouldn't be aware that it was there. So it would try to install the additional dependency. Um, and that could potentially overhaul that. Um, and that could potentially overwrite what you have already there, or it could conflict with it in some way. So, um, uh, yeah, it's a chart of charts is definitely an option. Um, there are some, uh, some limitations in how that will work because, um, just the nature of managing that many charts with a single installation and you want to be able to, um, usually people want to be able to, um, usually people want to be able to deploy a sub service, a sub dependency of a chart without having to upgrade the rest of the chart, um, because you have different people managing them and maybe different people have different access control, you know? Um, so I tend to not see the chart of charts model used very often. Uh, but it definitely is an option. If you're thinking, Hey, I'm going to, I'm going to have this whole stack that I control and I'm going to deploy it in lots of different clusters. That chart of charts seems like a reasonable option. Right. Yeah. I just, um, you know, recommend as well people to look into the helm file project that you can basically specify many different charts in the one large helm file, which is, you know, highly customizable and, you know, it's quite vibrant community as well. If you go into the GitHub home file, you can find out about this project. All right. Miguel asking any plan to include equivalent command to helm status of the two to show the status of all deployed elements. What is changed on the three for that? Um, on version three, you have to specify the release name. They still have the helm status command, but you have to specify the release name. I don't think there's a command that shows the status of all deployed elements. Can't you just do a home list list? That's what I was thinking. I'll show it all. Won't it? Yeah. Uh, oh, well, homeless doesn't show. Yeah. To your point, homeless doesn't show that it is running properly. Um, yeah, it's a good point. And I haven't seen any discussion about making helm status work for all releases rather than a specifically named release. In codefresh, there's a sweet dashboard that will show the status of all releases. That's, uh, that's even better. Great, great. Um, Christian is asking about a random question. Um, Is there any chance to validate the home charts before they are committed to an SCM? So I think in helmet two, we have dash dash dry run. Command for that. I don't know if home tree also has this one. Yeah. Well, actually this is, this seems like it's more of a workflow question. So like, um, if you're going to change a home chart, you should do it in a new branch. And then your CI CD should actually trigger a build off that branch that actually tries to render that chart, test it, tries to deploy it. And then once that's happened, then it's, it's accepted. And, uh, you can go ahead and merge it into your, your master branch or your production branch. Um, that would be basically treat your helm charts just like you would your code or your Docker images or anything else. It's code just like everything else. So you should treat it the same way. Um, and then, yeah, to your point, uh, doing a dry run, um, uh, will allow you to actually render it. Um, I think there's a lint command too as well. Isn't there Anna? Yeah, there it is. Yeah. Home lint is another option here that will help you see issues. Um, there is some work being done around helm test. Uh, where you can set up some tests for a chart. Um, but it's fairly limited in my experience. And I find that I usually just end up relying on my CICD to do this work rather than trying to, um, build it into just the help command. Yeah. I think we're on a tip over the hour. Uh, maybe we can take one or more questions. I don't know if you see anything that you want to take them. There's so many great ones. Uh, there was a question, somebody asked about scaling deployments and they were saying, Hey, if I had, if I have a home chart and I want to deploy it in a hundred different places, is there a great way to do that with CICD? And, um, for that one, I, uh, I typed an answer, which everybody should be able to see. Um, we did a webinar about how to do CICD for microservices. And basically the crux of it was around using a single pipeline with lots of different variables. So you can run one pipeline with a hundred different variables that will deploy to a hundred different locations. Um, and that becomes a much, much more scalable way to manage releasing and updating across lots and lots of places. So I would point at that as the, as the best answer is to, is to read that guy. Great. Great. Sounds great. Um, so thanks, Dana and Anna for a great presentation. I don't know if you want to put the screen with your Twitter handles and whatnot, uh, or how you want people to reach out to you. Um, and that's all I think for the questions we have today. Right. Um, thanks for doing it today. If, if you do have more questions on Helm three, um, Codefresh is doing, uh, we do these hangouts every week. Um, so if you, if you want to check it out at codefresh.io slash events, we're doing a, uh, it's like a coffee. It's Codefresh coffee hangout. And, um, I know that this week, I think it's, uh, Thursday we'll have a few people that are definitely super helm experts and use it even more than I do. So, um, if you didn't get a chance to have your question answered, go to codefresh.io slash events and go check out the codefresh with coffee. Or, uh, I'm sure, oh yeah, Karen's already posted in the chat. Okay. Yeah, I was trying to find the link. Fantastic. So I just want to let know everybody that the webinar was recorded and the slides will be online later today. Uh, and we are looking forward to seeing you at, uh, future CNCF webinars. And that, with that, I just want to say have a great day and thanks Dan and Anna again. Thanks everybody. Take care. Thank you.