 Well, hello everybody. Thank you for joining us for yet another episode of KB Insider. And this is the show where we try to get a sense of what's going to be happening next in the Kubernetes kind of ecosystem. As I've often said on this show before, and as I'm sure many of you have experienced, getting a sense of where open source projects are going can be a little challenging because the engineers or the people who kind of work on the project, not just engineers, docs, tests, you know, release engineers, et cetera, et cetera, have a lot to do with what actually gets shipped. Sometimes in alignment with groups like product management and sometimes not so much because it's open source project. They can just, you know, kind of deliver what they also are interested in, maybe on nights and weekends. There's no set code base. So as long as the community thinks it's important, it can be added to the system. And we, so for this show, what we like to do is interview people who are kind of actively contributing to the to the work in various ways so that we can try to get a sense of what's going on. So by way of introduction, so this is the KubeBuyExample Insider show. So if you go to KubeBuyExample.com, you can find us under community. And, you know, we do an episode every month in the last Tuesday of the month as a way to remember what it is at, obviously, this time, 10 a.m. Eastern, other times elsewhere, as you might imagine. And I'm Langdon White. I'm a clinical assistant faculty member at Boston University. I basically teach introduction to data science as well as support a bunch of practical classes. Currently this semester, I'm working on a class with a bunch of students called Data Science for Good, which is a bunch of social science students, as well as a bunch of tech engineering tech type people. And they are working to build a system to help do police accountability. So that's kind of what we've been working on this semester. And I would like to introduce my co-host, Daniel Oh, who had some technical challenges this morning, but hopefully are resolved in his lovely home, which, you know, sometimes it's a toss up whether Daniel's actually at home and then then we'll introduce Andrew. So, Daniel, who are you? Hi, everybody. Yeah, thanks. Yeah, I'm so sorry. I'm so sorry about that. I'm really, really joining because I got technical difficulties this morning. But yeah, everybody always having the technical difficulties. So yeah, my name is Daniel Oh, I'm based on Boston and I'm working for Red Hat as a developer advocate and pretty much like a CSF investor to help many people to build their own CS like a cloud native style. And Andrew, why don't you because, you know, I worked at Red Hat for a long time, I am well aware that it is a terrible idea to try to figure out exactly what your title is or what department you're with as it regularly changes. So if you want to introduce yourself and, you know, then we can maybe start talking about the Kubernetes. Absolutely. Thanks, Lenin. Thanks, Daniel. Hi, everyone. My name is Andy Blanc. I'm a distinguished architect at Red Hat's Services Global Office of Technology. What does that mean? It's kind of a mouthful. It means that I not only work with customers globally to implement Red Hat solutions, but I try to unify all of our solutions across the board. So what we're doing here in North America is the same in APAC, in EMEA and wherever we happen to have our delivery teams working with our customers and very much into working with open source projects and in the community. And we can talk about a number of the projects that I'm working in. And I guarantee, especially if you're in the Kubernetes ecosystem, you probably used at least one of them. Right. Right. And of course, as we were chatting a little bit before the show. So where are you currently? I am in Houston, Texas, in our Edge Solutions Center. So this is not fake. This actually is real. If I go ahead and he throws something, it may break something instead of being in the background. So, yeah. And so just a by way of a little bit of context, I don't think the term edge has actually pervaded terribly well in kind of our industry. So can you just say what you mean by edge computing? Edge computing really has it depends on the context. Holistically, it's basically anything outside of a traditional data center. So in the telco world, it could be a facility that it has a couple of 5G antennas. It could be a back store of a grocery store. It could be a device on a platform in the middle of the Atlantic. An oil rig. Gotcha. It can be anything. It's basically anything that can be outside the data center. And at least, you know, what I think is, you know, why it kind of gets its own category, not just for buzzword effect, right? But also the it changes how development works, right? Because you have, you know, it's actually like a lot of the problems we had, you know, because I'm going to date myself here, you know, in the 90s where, you know, things like throughput and RAM and, you know, network access and all that stuff can be much more limited. And so as a result, you kind of have to change some of the your development approaches. You know, you know, think about it, you know, we have our phones and we're tied to it constantly. Think about it when you go on an airplane, you have to either turn it off or you have really, really, you know, non-trusted connections. You make them up and make them down. You have to design your solutions so that there is no guarantee for network access as well as other considerations regarding the amount of footprint that you can actually put on these devices. Those of us who take these devices for granted that have gigs and gigs of memory, that's not necessarily the case when you're working in edge workloads and in edge environments. Right, right, exactly. Yeah, I actually just gave what we call a tech talk on about edge computing at BU the other day. And I was when I was preparing for the talk, I was I realized, oh, my goodness, a lot has changed in the last like three years since I really looked into this particular space. So I think it's particularly interesting. It might be something that we should do, probably delve into more in a future show. But maybe today, I think we wanted to talk to you specifically about Helm. And if you could maybe we could start with, like, can you describe a little bit what Helm is for people who don't use it? Yeah, so Helm itself is a package manager for Kubernetes. Think about when you're trying to deploy any resource to Kubernetes, you're going to have a set of manifests going to be a deployment to service. Maybe you can big map a secret couple of different objects. Traditionally, with Kubernetes, it's always been a challenge to deploy these resources. They're all independent. They're not necessarily packaged into a single atomic unit. You can install each one of these manifests independently. But what Helm brings is a way that you can package all those resources up together and manage dependencies accordingly. But in addition to that, it also gives you a templating engine as well. So you can add dynamic capabilities to these resources to no longer just flat and static. Gotcha. And so why or let's kind of change it all around. What are you expecting to see happening with Helm over the next, I don't know, six months or a year? Or what do you what would you like to be seeing with that? So I joined the project really focusing in one area specifically. And that is around OCI integration, the Open Container Initiative Integration. Helm integrated and introduced a concept where you can store charts inside OCI registries. OCI registry you can think of as a container registry these days. So Docker Hub, Quay, your Artifactory, Nexus, your local corporate repository can do more than just store container images. You can actually store other resources. So one of these resources are Helm charts. You can distribute Helm charts a couple of different ways. One of them is storing them in a traditional HTTP server. But a newer way that's the easier is to store them all within the same container same registry that you store your container images. And I came into the project to really focus on bringing it over the finish line from what they were calling experimental status to full GA. And then we were able to actually achieve that last year. So on the project today, I am working on hardening our support around OCI but also looking into how we can integrate more of our secure software supply chain technologies into Helm. So looking at technologies like Sigstore and other assigning capabilities into hardening the provenance of the life cycle of Helm charts. That's awesome. Yeah, go ahead, Daniel. Sorry. Yeah, thanks, Rangdon. Yeah, so you know also I got a lot of some questions from like a developer and a platform engineer. Sometimes they're really looking at Helm, but they're also looking at like Ansible or Operator stuff. And then I think you got a ton of the experience to use Ansible, Operator and Helm chart as well. Is there any some little bit tips? What is the differentiated between those technology? Well, okay, it's interesting that you mentioned it because every single one of those that you mentioned Ansible and Helm are supported by the Operator software development SDK, the software development kit. So you can use every one of those technologies and package them all into an Operator. But what I think about is if you're going to develop an Operator, have a reason for developing it. Is there a reason for having to manage the resources beyond day one or date, you know, some high level day twos? An Operator is very powerful. However, it's somewhat complex to develop and maintain an Operator. So especially if you're just deploying a simple application, do you need to develop a whole Operator to manage it? Probably not. Helm chart is really easy. There are tooling available. There are IDEs that some companies have created that really simplify and abstract a lot of the concepts for creating it. Monocle is one type of company and technology that I've worked with in the past that offer an option to make the process of managing Helm charts easier. And looking at OpenShift, OpenShift has an entire user interface that allows you to manage the life cycle of your Helm charts, as well as include a number of Helm charts that come out of the box. Those of you who have worked with OpenShift for a while, you may remember templates in OpenShift. So those are great, trust me. When they first came out, they were brilliant because Kubernetes had no templating mechanism whatsoever. But nowadays, developers and platform engineers are really looking for trying to make it easier for developers and platform personnel to create dynamic environments, dynamic capabilities that's what Helm provides over templates that are still somewhat structured because while there is some templating in OpenShift templates, you can't dynamically and programmatically determine, let's say, I want the secret in this environment or I want to do some more conditionals. There really is no conditional logic that's baked into OpenShift templates. That's cool. Yeah, thanks for the shouting to these sides. I would like to point out a project that was done a number of years ago at Red Hat by me as well as a bunch of other people called Atomic App, which does a lot of what you're describing, which was basically ship a container and then deliver containers or deliver a kind of a template for an application. I think we might have been a little before our time, but yeah, I am a strong believer in that kind of model. I mean, I look into OpenShift really being the foreground of a lot of Kubernetes technologies. Look at Ingress, they had the routes in OpenShift that existed before Kubernetes even thought about internal access from external resources. Deployments, when Kubernetes first came out, there was no deployment resource. There was no way to easily just deploy an application. It was all just basic pods and replicates. Right, yeah, I mean, it's gotten, it's really interesting where, I mean, this kind of goes back to why OpenSource is really cool, right, is that there's all kinds of innovation going on that sometimes kind of eventually lands or some variant of it lands, but because there's no block, right, to kind of try new stuff to when you're kind of working in an open source world, I think it makes a big difference and it can make the software a lot better. I was going to move in on a little bit as it seems to be coming up as a thread a lot in our episodes lately, but I guess have you done some work with Backstage? I have. I've actually worked with one of the projects that Red Hat's sponsoring called Janus. So Janus is helping the community around Backstage. So I've worked on a number of integrations around the security space and making the developer lifecycle a little easier for those not only getting into the IDP concept, but also looking at deploying on OpenShift. So if you go to the Janus IDP blog, we can post the link in a second. I had written a number of articles out there. Yeah, I've written a number of articles and gotten a lot of interest around the space right now. So it's great for those of you who just were tuning in, take a look at it and join the community. You know, we have a very healthy open source community, have a Slack, have a GitHub organization. It's a great place to just collaborate around the development platform initiatives because that's really getting hot right now in the Kubernetes space. Yeah, and I think it's there's the Janus blog there's also a bunch of GitHub, what you might call it, GitHub repos under a GitHub org called Janus as well. And so we actually have a question from the audience, which is, is Backstage a replacement for Dev Spaces? No, so yeah, Backstage itself is really just a portal. It allows a framework for you to build portals. Dev Spaces is a way that you can do active development. Yeah, I think and I think it's it's one of those distinctions which, you know, everyone kind of or I think a lot of people expect whenever you talk about developer tooling, you mean like an IDE, right, or like a place to code. But I think what Backstage is actually kind of setting out to do and maybe eventually there'll be plugins that do some of that. But really Backstage is trying to say, hey, here's everything you need to know about this particular project. And you know, and so it might have docs, it might have GitHub repos, it might have whatever, and then it has a bunch of plugins as well. But we've actually been looking at it. Why I kind of brought it up with, I guess, a couple of episodes ago was we as BU with the Spark organization that I'm affiliated with. What we do is experiential learning at BU, right? So we do student teams on projects for third parties. Obviously, like any other consultancy in a sense, right, we get repeat customers with repeat projects. And but we have, you know, my joke is we have like the worst turnover of any consultancy in the world, where basically we flip 300 employees every, you know, three months. And so as a result, trying to get the continuity of content between the projects or between the the cycles of the projects, right? It's the same project. It's very hard. And so we're hoping the Backstage will be able to give us kind of one place where we can do that content and have it so that, you know, it's just there. And we don't have to explain to every new team. Oh, you got to go to this Google Doc. You got to go to this GitHub repo. You got to go to this Trello board, et cetera. And we're really hoping that that's going to deliver for us. When you said security, though, for that, what what kind of aspects of security are you dealing with with Backstage? Yeah, so this is just a con and an issue that has really just open source mentality that I've worked with is that, OK, everything we just want to get working. But what about making it secure? And that's always very similar to like doc writing, come as a second or third or fourth. So Backstage, you know, there are integrations for, like, robust access control and integrations with other providers. But what I've been working on is adding support for a federated authentication using an OIDC provider called KeyClub. They're familiar with KeyClub as the SSL server. Develops an integration for KeyClub, bringing in not only a way that you can authenticate from Backstage to KeyClub, but also bring in data, like organizational data, like users and groups from KeyClub into the Backstage catalog. Yeah, that's super nice. And then, you know, the really nice thing coming from the developer side, not the security side, but the other nice thing right then is that I can say, hey, let me just put this user in this group, right? And then now I can have a plug-in to Backstage or something, which will actually go and manage access to all the things that they are supposed to access for a particular project, I presume. And what I love about KeyClub is that you can abstract the backends that your source of data. So you could actually have your users stored in GitHub or GitLab or even Microsoft Azure for AAD, but they can still all come through KeyClub as your single source of truth. So Backstage just starts with KeyClub and KeyClub does all the hard work integrating with the backend resources. Gotcha, gotcha. Yeah, that's really cool. Like I said, we're pretty seriously wanting to like implement it. You know, as you might imagine working at a university, it's a lot easier to do things in the summer. So we may not get to it for another couple of months, but you know, it is definitely on the to-do list. So the next question we want to kind of talk about. Daniel, do you have anything you wanted to add about kind of Backstage in general? Not really, but maybe it'll be helpful for us to understand who actually has no experience of Backstage before first. So sometimes the, so Andy, you got a lot of experience with the Backstage stuff. So you think that the Backstage is for targeting platform engineers only or even developers, they can use the Backstage for their daily job? I mean, they're still for developers. You're trying to make this environment available for your development team to start consuming. So that's really what we want to look at is making the developer process in Kubernetes easier. I mean, you look at Kubernetes and it was developed for a infrastructure mentality. It was never developed initially by Google for developers. It was just there to work on workloads. It does a great job doing that, but you have to build a developer experience on top of it to make it consumable. And for the first five years of Kubernetes, we always worked very hard to make that possible. And we made some strides, but it could certainly continue to be improved upon in Backstage and other IDP tools. I don't want to say Backstage is the only one out there. There are plenty of others out there in the space. I mean, I look at, I went to the last KubeCon and like everyone was talking about IDPs in terms of open platforms. I can only imagine what it's gonna be next month or a month and a half when we're all together in Amsterdam for the next KubeCon. How many companies that either exist now are gonna announce something at it? We have a lot of the vendors who have been in the Kubernetes space. Are they gonna announce something? That's always, I mean, it's always good to see when you have conferences like this. Yeah, that's cool. And then, so I've heard that you recently created a ham chart for the Backstage stuff. And then can you tell us a little bit more why you actually created the new ham chart for the Backstage and then what kind of benefit specifically platform engineers and SRE and even developer can earn some kind of really cool stuff from your ham chart on Backstage. Yeah, so when I first got into the Backstage community probably six, eight months ago, there was no easy and streamlined and holistic way to deploy Backstage. It was everyone in their brother created their own ham chart or deployment mechanism. So while working with some folks from the Spotify group, because no, they're the ones that originated Backstage, it was in their GitHub organization, it was initially, worked with them and the community to create a canonical ham chart because the Janice project we created our own just have something out there as well because like everyone we had to create one since there was no centralized one. So what we've done is we've worked with their communities to take some of the concepts and best practices from a number of the different charts and communities that were out there deploying Backstage using Helm and brought it into an actual chart that the community can start to leverage. So if you go to github.com back backstage slash, oh my think our chart, it's one of the two, you can go ahead and look at the repository for it. And it's also published to a traditional Helm repository so you can just add in the repository and just install it that way. And you can integrate this with any of like any GitHub tools like Flux or Argo CD, OpenKit GitHub and the OpenKit space that can easily be integrated into the Helm chart for Backstage. And what I've been doing is I've been demonstrating how you can take that chart that the community and all of us work to develop and deploy it using different paradigms. So everything from how do I just do a hello world Backstage chart to I want to deploy it to OpenShifter. I want to integrate it with some of the plugins that I mentioned earlier, the Kiko plugin. How can I do that and do it all with using the same tool but still be able to reuse the same asset? And that's really what I've been writing about. That's really awesome. So you know, so since the Kiko Detroit, I tried by myself to install Backstage on my laptop. I got to really be hard time to make that happen. But Helm tries to pretty much make me easier in my life. And then I think it's that all developer and even platform engineer have the same benefit to give it a try to Backstage experience on their own local environment even the Kubernetes or OpenShifter stuff with your head chart. I mean, to take a step back, let's go and move back from what we heard in the conversation where, okay, if you want to get started with Kubernetes, how easy was it initially? Okay, it's hard because I have to figure out what a deployment is, what a service is and figure out networking. And then how do I take dependencies from more applications and bring them together? So look at Helm. Helm helped bridge that gap by introducing the folks to Kubernetes. I can deploy, let's just say the most canonical Helm chart out there is WordPress. I want to deploy WordPress. WordPress is a multi-tier application. I need a front end and I need a database. Well, switching that out together manually is not easy. If you're getting into Kubernetes, it's quite difficult to take not only how do I deploy one application to a two and tie them together? What Helm does is it's very similar to how I just install a package in your operating system. DNF, Yum, install X, install WordPress. It goes ahead and if it's all the dependencies, installs and figures it out of the box. That's what Helm does. That's what we did for Backstage and look how it makes your life easier, Daniel, to get Backstage running in Kubernetes. A hundred percent agree. So I wanted to kind of ask you like a Helm kind of question, somewhere along the way I picked up, I want to say, it might have been from Thomas Tomacek using Makefiles, basically, to kind of build in and deploy my containerized applications. I've kind of played into, I've kind of tried around kind of getting into Helm to kind of do the casual things that I'm doing a lot. Another kind of similar concept right or whatever is like Docker compose kind of model. Every time I've kind of looked at Helm, I didn't find it as simplistic as doing either of those kind of choices. Obviously Helm is bringing a bit more to the table about like ensuring how those deployments happen. But do you think that like, has it started to get to that level of simplicity or like what can I do or where can I learn how to make it so that I can kind of, in my casual tool chain or whatever, or even less than casual, we've been working on deploying a containerized application to do basically data set management that we want to offer as part of our organization to the public. How do I get there kind of, or how do I get started with Helm such that I can make it as simple as, if I have some Docker compose experience or I use Makefiles or whatever, any ideas? I always just start with just getting started with Helm, looking at the documentation for the getting started experience, talking about how to install the CLI. There's a CLI that comes with Helm. So that's how you interact with Helm. Helm initially was a client server model. There was a server cyclopunk that you had to deploy to Kubernetes. And that was a bit of a security concern because you had to give it elevated permissions to be able to manage app, the Helm charts across multiple main spaces. With Helm 3, the most recent version of Helm, we've eliminated the entire server components. So there's just client size. So all you need is the CLI. But there are also, if you're really a developer, there are an SDK that comes with Helm, so you can integrate Helm into your applications and your workflows. So that's one of the other ways that you can get into it. Like you mentioned, you know, being from a developer standpoint, that's another thing you can do is you can look at how you can abstract that process within your own workflows. We're actually gonna talk about that. If you are going to KubeCon or if you're gonna watch KubeCon afterwards, there's gonna be a session that the maintainers are putting on around developing and integrating with Helm from a development standpoint because we talked about our last KubeCon talk from the maintainers track. We talked about the different light ecosystem around Helm and all these different projects that are working with Helm. But now we're gonna talk about, as a developer, how do I go ahead and leverage the tooling that Helm provides to make it easier to be able to work with Helm in your own ecosystem? Gotcha. Yeah, that would be, I think, really handy. And just while you were talking, I was looking for the link, but if you wanna go see his talk, oh, that's not a deep link. That's annoying. Oh, it's at least a link, it's in the ballpark. Yeah, right, right. So in the ballpark, if you, I searched for block and I found you as in the top four as a result in the schedule search, it's kind of like searching for Langdon. It's usually pretty easy. It's nice to also be at the top of the alphabetical order as well. Right, right, that does helpful. Yeah, white is not good for that. So, yeah, I am mildly jealous. So yeah, I think that's also one of those things where kind of going and listening to, or like kind of watching somebody demo it or workshop it or whatever can be really helpful. And that's exactly what we try to do is with these maintainer talks, we wanna make it so that we invited a welcoming community so that if you are involved with Helm itself, wanna get interested in not only active development, but just from a consumption standpoint, how do we organically grow the ecosystem around making it easier to deploy your applications to Kubernetes? That's what we really want for Helm. That's the mission of Helm is, how can we streamline the process for deploying apps to Kubernetes? Right, so my next question of course is like, because I'm gonna toss those guys under the bus, have you talked to the Podman team about kind of adding to their generate feature? Cause right now they'll generate QBML from Podman. So the nice thing about Podman, is that you can actually create pods in Podman intuitively enough. So it's a lot closer to actual Kubernetes than Docker is. And so, but it'd be kind of handy if I could say, Podman generate from the pods that I have set up, Helm charts. I mean, one of the great features of Podman recently is around Podman Play Cube. I love that feature, especially if you're coming from a traditionally, and we talked to earlier about Edge workloads. We have a lot of these applications that were originally developed for Kubernetes that run in traditional data centers. But let's say you're deploying your applications to Kubernetes and you wanted to deploy it at the Edge, and you may not have a full Kubernetes environment. Podman Play Cube is great because you can take those same manifest, those same deployment manifest, and bring those into the Podman world without having to necessarily have Kubernetes. But you can still, from a developer standpoint, use the same set of manifests. Right. And for people like me, that's a good thing. Yeah, it will frighten me in general. It can be worth, we can be talking about soap. That's true. I'm just saying, it could be worth it. Yeah. And I feel like HamChart is now some kind of mandatory technology and background knowledge for individual developer as well as operation team. So any recommendation on how to get started to run HamChart? If you could, you could actually promote your HamChart book as well, but I'm just wondering if you stick to it. You kind of took the words in my mouth there, Daniel. Yeah, but I'm just wondering, everybody wants to run the HamChart and where is the perfect place to learn how to get started, the Ham's learning paths. Yeah, so I'll put a number of different paths out there that our folks can get interested in to Helm. Number one is, if you want to just go to the Helm.sh, which is the website for Helm, that has a great getting started. There's a book that I and another colleague of mine have written called Managing Kubernetes with Helm. It really dives into a lot of the capabilities and features around Helm, Helm development, and CI CD. So looking at tools at Argo CD to manage it. And then finally, if you love Slack, like I do love Slack, I think I have eight work spaces that I'm in every day. You can join the Kubernetes Slack channel and inside there, you can have, there are two channels of interest. One is Helm-users and Helm-dev. So if you're really interested in learning about Helm development and actually getting introduced to the community and just getting help and support and just chatting, join those two mediums, they're great. And then obviously we want to be able to get more people involved in the Helm ecosystem because like any technology, Helm isn't done. Helm, we're just at a step in our entire development process. We want to make it easier. We have a number of thoughts around what we want to do for Helm 4, which is the next version of Helm. And I'm sure we might be able to touch upon some of those in our last 25 minutes today, but I'm sure our audience would be interested in where's Helm going? Yeah, that was actually, I think that is exactly correct. Yeah, so what do you see on the horizon for like Helm 4 and what kind of timeframe is that, do you think? So I don't, I'm going to leave timeframes out of it. I'm going to put my project manager hat on. I don't put timeframes out there, but a couple of them out there. You get released when it's ready. Yeah, exactly. Two areas, one, as I mentioned, security is an area that I really focus on. So I would like to look at integrating new methods of signing Helm charts into this Helm ecosystem. Right now it's using traditional GPG-based signatures, but I'd like to introduce other means. SigStore is the one that I would promote because SigStore is a technology that makes signing content easier. You can even use GPG with SigStore and actually we have a plugin in the SigStore community that allows you to take a signed Helm chart, a signed Helm chart in the traditional sense and make use of some of the SigStore technologies like RECOR, the transparency log. So you can go ahead and take a signed Helm chart and publish it to the transparency log and do verification in a similar process that you can do in container images. You can also just sign the Helm release artifact as well using the SigStore tooling, but there is some more integration that's direct from the Helm ecosystem. So aside from signing and security, probably templating in languages. Right now Helm uses Golain-based templating language to manage the templating capabilities, but there are others in the community that want to introduce additional frameworks. Hulang is one that I've heard of interest, but just providing a flexible way to deploy and create these type of resources in a dynamic fashion is something that I know a lot of interest in the community. I was just gonna post as well a prior episode we did about SigStore because I think that is definitely another one of those kind of up-and-coming technologies that is- Did we have Luke Kynes potentially speak? We did. We did. Excellent. Yeah, great, Luke's great. He's amazing at work with. Sorry, so I was typing, so I got a little distracted. So, yeah, so I think some of the, I think that kind of verification signature kind of stuff is super nice because I think that's where you can kind of, you know, the temptation is to, you know, kind of just kind of install anything, right? Sometimes, and so it could be really handy to kind of say, okay, here are my trusted sources, right? And I'm gonna go and approve those signatures kind of whatever. And then I can kind of fall prey to that, you know, trying things out and just kind of running them. But then if I get prompted, hey, this is a new one. Do you really want to go there? I think that's a really nice way to kind of give you that freedom of, you know, kind of exploration, but also, hey, this may not be as far into the woods as you want to get to today. And when we talk more about SigStore, you know, more into now how we actually gating what gets deployed to our art, to our deployment resources. So in Kubernetes, do I want this container imaged and these deployment manifest to actually go out to my environment? Do I want to do Providence and actually do some accessation verification before allowing it to be deployed? And we look at some of the technologies out there, we have everything from, you know, Red Hat has advanced cluster security that provides some capabilities for ensuring that your images are signed to other policy frameworks like OPA and Tiberno, Gatekeeper. There are a number of different projects out there that are attempting to address the same space and being able to then integrate that with like Helm is another way that you can do that too. So how we can bring all these technologies together to make it more secure to operate and develop for Kubernetes environments. Right, right. And even just regular, you know, traditional environments, I'm working with a number of teams within Red Hat, you know, we're doing some work around Ansible. So how do we make Ansible, you know, signing, you know, collections now? Ansible has some collection signing right now. How do we do the same thing for like some of your project data for if you're using Ansible Automation Platform? Right, exactly. I mean, you know, one of those things, you know, kind of speaking again, I kind of come much more from the developer background that I do kind of ops side, right? And so, and then particularly in consulting, one of my biggest paranoid fears, right, was that we are going to get through building this whole project and then find out that the ops, you know, team was not going to want to deploy it for X, Y, and Z, probably perfectly valid reasons, but knowing in advance, right, that what's available in my ecosystem, you know, as tools for when I want to deploy it later is a very nice thing. And I think these tools kind of, I think they get correct, you know, whatever like kind of value statements or whatever about how they're used in operations. But I think developers don't realize that it's also really useful for developers to kind of be able to get a good sense of what is the ecosystem that is, that the, you know, ops team is comfortable with and how do you have ways to know what that is? So you mentioned ops, but you've always started to scratch the surface. Ops is, you know, when you talk about DevOps, that is your first hurdle into bringing these two groups together. But if we talk more about the six-storey side of the world, we're also going beyond DevOps, we're talking about DevSecOps. So we're actually bringing in another group, security into this process. But then if you then add in a lot of the governance around how I actually determine what policies I want to employ and govern, which assets are being able to be allegedly deployed, that's actually going even on a step further than DevSecOps into more of a modern governance, automated governance side of the world. So really we're all going down this journey of lineage of how we bring these different various dispersed groups within organizations together to make operating our software more and more secure. Well, I think, I mean, you know, it's funny because we started the conversation about edge computing, right? And edge computing in some ways is really driving this especially kind of the S-bomb kind of idea, right? So it was to cure bill of materials or is the S officially secure, I guess? Software, software. Yeah, okay, so, but no, I thought there was also, isn't there one that's a, it's secure bill of materials versus like a, like unsigned or signed bill of materials? It's all but soup, so. So, but long story short is that, you know, when you're talking about deploying applications, right? At the scale you need to do to like do it. I found out that this building has 700 plus thermostats, right? That all run software, they do PoE for the internet traffic, you know, and like because this building was just built and it has, you know, it's very, very green. It's like the highest level LED certification. And so, when you're talking about doing deployment at that kind of scale, you need to be able to like completely automate all the things that we think of traditionally as having to happen in the data center of like getting software from point A to point B, but then making sure that what landed at point B is valid is what you expected to get over that. Now, another area that for those of you who are mainly, you know, develop data centric, data center centric is this convergence of another concept. And I would mention DevOps being really the focus, but if you're working with Edge workload, you're actually working with another convergence of IT and OT, you know, in your traditional information technology, but then operational technology bringing in these different groups together because they very much like IT, like Dev and Ops, they work in silos. Right, yeah, yeah, one of the, you know, it's funny, like, you know, you go to the Phoenix project, right? And you kind of wonder, it's like, okay, we need to see the rest of the series, you know, it's like, like where is security, you know, where is quality assurance? You know, we need the rest of the books someday, you know, to have the novels about the, you know, kind of the rest of these silos that we've built in the technology world, which, you know, I always go back to the technology, you know, particularly software, right? It's such a young field. And, you know, even in the early days for me, it's like, I had to know about basically everything to be able to get an application deployed. But then as we got specialized, everyone started to strive. And what we never really figured out is, how do we keep the collaboration that we had when it was all one person who knew everything in their head? But, you know, because you need all these different aspects, but you need these people to work together rather than thinking of it as like something you're throwing over the wall every time. I did wanna address, there was a question from the audience, which is basically our Helm charts fully switchable between OpenShift and Kubernetes. And could you take a Helm chart and just use it in OpenShift or vice versa? I think I know the answer to this question, but Andrew, please share with us what your opinion is. The answer is, it depends because a Helm chart will contain a number of different Kubernetes resources. Some of those resources may not be available on either environment, whether it be a Kubernetes environment, because if you're leveraging custom resource definitions, they may not be available on the other cluster. Because if you're just deploying custom resources, but not the definitions themselves, they may not be available in on the environment. And then OpenShift includes a number of resources like routes and deployment campaigns and other type of manifests that are not available in traditional Kubernetes environments. But a lot of the charts that have OpenShift support typically have parameters. So one of the benefits of Helm is these, we talked about these templates, but the templates, the actual values that you inject are called parameters. And they usually have parameters that have toggles for OpenShift. So there are features that will actually be enabled. Chart developers are actually creating opt-in and opt-out of OpenShift features. That makes it really easier. I look at Istio. Istio is one great example that has specific support for OpenShift. You can actually use one of the values to turn on OpenShift. You know, go ahead and configure the charts with OpenShift support. Gotcha. So like so few things in the technology world. The answer is usually it depends. So did you want to add anything else kind of about Helm or where you think Helm is going? No, I think that's at least the starting point. What I have seen, so I've been around Kubernetes for a while. I've probably been here since, I've been here since the pre-10 days. Helm had a slow burn in terms of the popularity. Operators probably about four or five years, about three, four years ago got really hot because it was this concept like, oh my goodness, I can manage my customer resources. It's kind of like when customer resources started taking off too. But I've actually started seeing Helm popularity. The uptick has gotten much, much greater recently. And I think it just means it's either because of one of two things. One, teams are starting to reflect where operators and other more complex workflows are because it's toils I mentioned to have to manage a lifestyle of an operator versus I just manage a Helm chart. But they also see the more and more, more not only developers are getting into the Kubernetes ecosystem, but more vendors are starting to distribute their applications in their contents via Helm. Gotcha. Yeah, that's I think one of the really nice things, right? I mean, in some ways that's what really made Red Hat Enterprise Linux take off, right? Was the ability for vendors to have a well-defined way of distributing even proprietary software to a platform in a well-known way that was enterprise ready, et cetera, et cetera. I think that when you have, it's like, it's always all about the ecosystem, right? So it's super useful if you have well-understood ways of building and deploying even proprietary applications on that infrastructure such that you can run it in ways that you want to as your enterprise rather than some custom mess or whatever that some proprietary vendor has decided is how they're gonna distribute their software. And especially if you're working with teams you know, like you have a one-lead platform engineer, well, if you either get to sit by a boss or play physical chairs and decides to head to a different company, then you're left with some proprietary way of deploying software versus Helm, very rich community, very easy to get support in terms of the community. Some vendors are starting to provide more support around Helm too. So being able to look at a tool that can be supported regardless of either your workflow situation or just the technology, you know, aptitude of your teams. Right, right. Although I will say I do prefer the wins the lottery versus the hitting by the bus metaphor, but the- And the wins the lottery works too, but you know, I can look myself in and say, what would you do if you win the lottery? Nothing, what would I do different, nothing? I know, yeah, I'm kind of with you. Yeah, it's hard being a nerd. But- You know what we do, sorry. Yeah, so we have to close out with, we know you travel a lot and we know where you're gonna be in April for KubeCon, but is there anywhere else that you're particularly looking forward to that's coming up for you? Yeah, I'm actually going over to Romania in a few weeks. That's the first time I've been in that part of Eastern Europe. So looking forward to that, I'll be in London early, early, early April, be in Copenhagen next week. So doing a lot of European travel, but then I'm probably gonna focus a little bit more on the APAC region in the U.S. summer time as I started to travel, opens back up and I'm starting to reconnect with more of the teams and customers in these different regions. I have taken a more of a newer role within Red Hat in the last year. So we had some of our work muted because of the pandemic, but as more of these regions start to open back up and more customers are looking to have more of the partners who work with them are being able to actually get out to meet these folks and actually hoping out with not only our customers, but also the communities. I love speaking at some of these developer days that are out there. So trying to coordinate my trip also with these different community days and seeing the Kubernetes community starting to have more of these Kubernetes community days out there and aligning with that. We actually talked about those last episode, I think, with Chris Short and like the contributor summit was the main focus of the episode, but we were talking about the community days and I helped to start the DevOps days in Boston and what was so nice about DevOps days was there was like a pre-packaged whole website and everything else and you just kind of like set it up and then you went out and kind of did the conferencing stuff, but you didn't have to do what I think a lot of people who want to run a conference find difficult, which is kind of the building all the marketing tools and that kind of stuff. And especially, you wouldn't have any idea how to run a conference, you only do that in August timeframe. Yeah, right, right, yeah. DevConf US, let's see, but DevOps days was actually before all that because I think I helped start that in, I don't know, had to be 10 or 15 years ago. Yeah, so it's been a while, but yes. Is it easier now? Oh yeah, well, particularly seeing is someone who you may be familiar with, Kravashi Mahani and who does a lot with Podman and Sally O'Malley who does a lot with kind of containerization. It's been, I think she's been doing a lot of kind of build materials and stuff too lately. I've been working with her and her team actually quite a bit these days. As I mentioned, I'm working with the six store community as well as some of the work we're doing here within Red Hat. So I work with Sally's team and Sally's head to house. Yeah, well, so I convinced both of them to help co-chair DevConf US with me a couple of years ago and they are really starting to run with it. I might be picking up that emeritus title soon. Oh, that's good. Yeah, I can just show up and look pretty and I don't have to. US, you can enjoy the conferences that having to run on and on like a different car cut off. Exactly. Yes, my running joke is for the first two years of the conference, I literally blew out a pair of shoes at each one from just how many, you know, and I've done, as part of the trivia, I've done how many steps did Langdon take over the course of the conference. So yeah, I think unless we have anything else we want to talk about, Daniels, do you have any other questions you wanted to ask? Maybe one question. So I think they add a lot of experience to customer meeting and the customer engagement so far. So is there any interesting story any your customer actually looking for to adopting backstage or some use case or the other stuff? I know it's a backstage and pre-order stage right now but yeah, it'll be very helpful for us if you share a little bit about the kind of the real story. Yeah, so there's a lot of interest right now around integrated development platforms and portals out there. I'm working with some of the folks within Red Hat to talk to some of our customers who are very interested in leveraging backstage within OpenShift. And some of the work that we're doing here within Red Hat is providing a way that you can, deploy the backstage ecosystem into OpenShift confidently. So working with our product management and engineering teams to help not only from the community side of things which I care very much about but also from our customers to also provide a enterprise grade experience that they can trust. Yeah, thanks for sharing. Sandra, I could yield next to session topic and compromise the backstage stuff. Yeah. Right, definitely. Yeah, it's actually interesting that you mentioned interest in developer portals, whatever. I actually have a student who is proposing an independent study that they'd like to do about looking at some of the research around. I guess there's some growing concern about ideas may be not the best way for new developers to start to get and maybe that it's like too hand-holdy which I thought was kind of interesting but just kind of in the same vein. So I don't know, maybe future episode we will have the output of her potential research into that topic. She's talking about doing it for the summer. So I think that might be really interesting because I think as a long-time engineer, right? I need all the help. I want all the help, right? But at the end of the day, I know how the code works. But I thought that was kind of an interesting perspective. Well, thank you. I really appreciate your time and obviously and also all you do for the community which is as many people know, quite a lot. And so, and you know, and I think we finally met in person, right? Like sometimes in the last like six months, right? It was definitely. Oh, okay, yeah. And so that was kind of interesting because I think we've crossed paths off and on for at least eight years, maybe 10. So that's kind of random. I know that I've met a number of colleagues of mine either in airports, hotels, or conferences. And they've actually lived in the same city that I have lived in. But I never see them except for one or two years. Exactly. So whenever I go to Chicago, I never ever met Andy at once. All right, well, thanks all. And we will see you in about a month. And you know, that'll be our last episode before KubeCon, I believe. And then we're going to do some more driving interviews at KubeCon. So if you haven't checked those out, you should go see our interviews that we did at KubeCon NA, where we borrowed a car from Ford and drove around people and did interviews about KubeCon. So the question is, are you going to go ahead and have instead of being in a car, are you going to be in a bike? Because it's Amsterdam? So that was the hope. But the problem is if you think about like a tandem bike, how do you interview somebody, right? Or you could do two bikes, but then you have to have both people able to ride bikes, which may or may not be, you know. So we did toy with that. We also toyed with, could we do a boat? But, you know, like a canal boat, right? But to kind of give it the same feel, that means I have to drive said boat, which doesn't sound like a good idea. Well, they do have some of the things where you'll have someone, you know, pedal the bike, but then you're sitting in back. Oh, that's true, like a pedicab. Yeah. Oh, that wasn't a terrible idea. I think, however, we might be doing another fancy, fancy electric car by a customer relationship that Red Hat has that I don't wanna mention because I'm not sure how confirmed it is. But I am now sorry we didn't think of pedicabs. So maybe for the next thing. There's always future sessions. Exactly, exactly. So thanks everybody for coming and we will see you next time. Thanks everyone, thanks guys. Thanks a lot.