 Hey, welcome to the fourth episode of Chatloop Backoff. Today is a special leap year edition as it is February 29th, 2024. This is a program that dives deep into the realms of the cloud native landscape. I'm your host, Jeffrey Seca, but many of you know me already as GV. Many thanks to Jeremy for guest hosting last week. I got to visit a much warmer climate than I have talked about previously. I got the pleasure to go down to KCD, Sao Paulo and keynote there. It was a great community event. I keep saying wherever there's cloud native, there's family. And that's not just a touchy-feely thing. Like I really do feel it, I really do mean it. Anyway, get aside, I am thrilled to have you here to join us today for today's episode. We're focusing on backstage. Backstage is an incubating project in the cloud native computing foundation or the CNCF. And it's a project focused on developer experience and building developer portals. But before we get into it, there are some housekeeping items besides my little puppy deciding to play ball right under my desk right now. This live stream is a sibling to the similarly named clash loop back off, which is an event we put on at various Kubecons. In that one, we put two community members against each other to accomplish some technical challenge of my choosing. It's meant to be laid back, at least for the audience, and generally fun. This stream is more of a self-induced version of clash loop back off. I haven't done any in-depth research on the topic that we've chosen. Instead, I kind of want to dive deep into the topic with fresh eyes. And I want to learn as I normally would off stream. So when doing so, hopefully we can all walk away having learned something new. During the live stream, please feel free to drop questions into chat. I'll get to as many as possible, but we're here to learn together. So if you happen to know the answer or something, please let me know if I'm stuck. Again, comments are welcome. That said, this is an official live stream of the CNCF and it is subject to the CNCF code of conduct. Please do not add anything to chat or questions that would violate that code of conduct. Essentially, respect all the fellow participants and presenters, don't be a jerk. This video will be put onto our YouTube channel afterwards. So folks that couldn't make the live stream can still learn along with us just asynchronously. All of that said, let me move this over there. Let's get to some news really quick. So some big news and this one is kind of long coming. Falco, a project that was previously incubating has now graduated. It was open sourced back in 2016 by Cystig and became the first runtime security project accepted into the sandbox. 2018, it went into incubation and, or sorry, it went into incubation in 2020, yeah. It has had just tremendous growth since joining the CNCF and it graduating, I feel, was a long time coming. So I'm very happy to see that. I personally have messed around with it and it is pretty cool. It is a project that lets you watch CIS calls and monitor essentially low level things on the host and you can generate rules to kind of alert or propagate up anything that happens. Next up, kind of along those same lines, Netflix actually has open sourced a project called BPF Top. So think like EBPF. This is like BPF Top. It sounds super simple. It's just a way to look at running BPF processes or programs, I guess, and look at the resource usage. But that kind of low level insight is super helpful, especially when debugging. And I mean, Netflix open sourced it, so it seems pretty good. And last but not least and highly related to today, there was an article published on the news stack that really stuck out to me and it was called improving developer experience drives profitability. Ultimately, the article is discussing research that's come out regarding performance efficiency boosts that occur when developer experience is the main topic in concern. Like I said, it's appropriate for today's topic, but it also just is kind of validation to me. I have always felt like if you focus on the developers and the people that are really hands-on keyboard and eliminate as much friction for them as possible, two things happen. One, obviously there's a level of productivity that boosts, but also there is increased morale because I don't know about many of you, but when I have had to bash my head against a wall for something that should be relatively straightforward in my opinion, or I'm stuck waiting for some sort of resource that hasn't been delivered to me, it just causes a headache. It makes me deflate all of my motivation, all of my willpower just goes by the wayside. So it turns out if you remove stuff that stops that, things get improved. But enough news, let's start thinking with portals. Ah, developer portals, right. Let me share my screen and then we can kind of get into it in a momento. Share screen, entire screen, boom. I am sharing a thank you magical wizard behind the screen. All right. So as usual, I like to create notes for every episode and I wanted to just kind of go ahead and create the notes for this one live. As far as I know, Jeremy has not created his for last week, I kind of bugged him about it, but let's see if he's done it or not. Oh, okay. Well, let's create today's and because I'm lazy, I'm just gonna copypasta this. Go from there down. All right, like I said, episode four is going to be backstage. This is TBD and then I will fill in the news. As we go. So summary, let's talk about first, what even is backstage, which is actually going to create, thank you, co-pilot, but no. It's actually going to pose the question, what is a developer portal? Then I did do enough research to see that spinning up a demo is going to be not impossible, but probably not worth the time. I can go into why later, and in fact, why a live demo with backstage probably isn't a good idea on stream when you're doing live. Let's kind of fill it in from there. So what even is backstage? I thought I closed you, Zoom. There we go. So backstage is a open platform for building developer portals. That doesn't really, that just kind of sounds like buzzwordy, but what it ultimately does is gives you a way to standardize access to resources and standardize ways to interact with said resources within the framework of a company or a project. More often than not, backstage is going to be used in a company setting, though I have seen backstage used for projects. It's kind of not normal, shall we say. And by resources, what do we mean? And it's kind of cool. It kind of means everything. What do I mean by that? I want to actually, I want to draw. Let's draw really quick. That's way too big. So let's say you are a developer and you've just joined company X. Company X has a whole lot of things underneath it. They have a GCP account. They have an AWS account. They have an internal portal for docs. So let's just call that like they have a knowledge base. They have a test environment. I am sorry, I can't write well with a mouse. And let's say they have an auth system. So some form of auth. So you come in, you need to be granted access to, you know, in a basic normal older kind of style of company. You need to be granted access usually to every single thing. There might be a way where some of these solutions have, you know, SSO behind it. Like say that you're using a G Suite. Well, you can tie G Suite to GCP. And so that would be SSO. So I'm gonna easily, it's possible but not easily be able to tie AWS into, you know, G Suite SSO. Let's say, you know, the knowledge base happens to be something that has Google. The test environment may be a Kubernetes cluster. So it could have SSO. And we've talked about SSO. That's the auth system, cool. The problem is AWS does not have SSO easily enabled. And what happens if that knowledge base also does not have SSO enabled? Well, usually you're cutting a ticket to your company IT or potentially, you know, cutting a ticket to like a different group other than the company IT. But it still, it slows this person down quite a bit. Now, especially if you're slowing down access to them getting into your knowledge base, that's where things really can cause, I don't wanna say a headache, but it can delay them onboarding, right? So that's kind of where this concept of a developer portal comes in. Where be you? So developer portal says, no, this model is false. We need to improve it. Yes, I'm gonna call it BS. Please keep all of your snickering to a minimum. So what backstage does is it says, no, what I'm going to do is create a portal. And inside that portal, I'm going to have links to your GCP. I'm going to have links to the knowledge base. I'm going to have links to the AWS account. And I'm going to have a way to spin up your own individual test environments in separate Kubernetes clusters. And I'm going to do this through a method of plugins. And then I'm gonna do all of this through, you know, the SSO, whether that is some form of G Suite, whether that is, you know, SAML, GitHub, whatever. It's all configurable. But that winds up being the way that you, you know, get onboarded. So suddenly, out of nowhere, you have a new employee that can come in, be granted access to backstage, you know, at the new employee level. And the, you know, they might not have access to the prod environment. They might not have access to certain other privileged resources that could still be granted through backstage. But they do have everything that you can possibly give them in order to succeed. And that is why backstage has become, in my opinion, and a lot of other people's opinions, so popular. It accelerates any developer to go from zero to actually being able to do meaningful work at the speed of their ability to soak information up and actually do work. I know in my career, I have gone to companies and I felt like I could not, I didn't have access to the resources. I was waiting on tickets to be resolved. I was searching for information across multiple different knowledge bases. And it took me, and I'm not kidding, almost three months to think that I was actually a meaningful, I could contribute to the company a meaningful way. I really wish backstage existed when I had made those switches because I feel like life would have been a whole lot easier. Enough with my horrible art style, let's actually look at some documentation of backstage. So backstage is a project that I had looked at a little bit, but it was probably a year and a half to two years ago. So I may have some outdated information that I personally want to kind of clear up. So one thing was, actually, I should probably go into how backstage is built. So let's see how, oh, they have a very nice architecture overview. Cool. So there are three main components to the backstage architecture. That looks better, cool. So there's core backstage UI, there's plugins, and then there's databases. This kind of goes back to what I was saying before with plugins. Each of these things, like we had, oh, I said I was gonna stop drawing, but you know, we had GCP, we had AWS, and we had some knowledge base. Well, each of those is gonna require a plugin. And when you think about just, I guess, current standard web application architecture, there's usually a front end and a back end. So that's probably what they're talking about with plugins and their backing services, the UI itself and then whatever, persistent services, sending things to the background. So an example with CircleCI, so there's a CircleCI plugin that manages CircleCI externally, there's service catalog, we'll get into what a service catalog is, lighthouse plugin, tech radar, all these interesting things. I should mention that backstage was originally donated out of Spotify and they have been doing quite a lot of work around this and there are backstage and Spotify curated plugins and then there are community plugins. So for most things that you might want to try and pull into this concept of a developer portal, you should be able to have a plugin. And if there isn't one, the building of a new plugin is, I don't wanna say relatively easy, but I will say it's, there are a lot of really good examples to base that off of. So my crappy little design right here kind of lines up with the screenshot. Backstage itself, you have the core components which is think the nav of backstage at the root level. Then when you click into any of these, you can get down to the core component of that nav bar and then there's individual plugins that surface up information. Each plugin typically makes itself available to the UI in a dedicated URL. That kind of makes sense. So lighthouse would be on slash lighthouse. Backstage probably does not work well under a, oh, what am I trying to say? Backstage probably works best at the root level of a domain name, not like some sub path. I'm not 100% sure, but if I was setting this up myself, I would not even try to put it under some sort of sub path. So circle CI is at slash circle CI in this case, makes sense. Plugins are typically installed as react components. So if you're doing any sort of plugin development, you're gonna be needing some front end expertise as well as backend. Here's a file that imports many full page plugins. Let's see what that shows. I'm not even gonna try and go through that right now. That's not gonna be worth our time. I do think this is gonna require a second session going into backstage because there is a lot to digest here. My main focus for this session was to only go into kind of a high level, what backstage is, why it exists and what all it can do from a high level, not like let's develop a plugin or let's set it up from scratch. Graciously, or thankfully there are demo installs that we can kind of click into and look around without me having to spin up a kind cluster and mess around with it. Cause as I mentioned earlier, why is a live demo probably not in the cards? It takes a bit to set up. Hopefully you kind of understand why at this point. It's a very, very large undertaking. So architecturally plugins take three forms, a standalone plugin, which I assume is just kind of a, a front end for something like this tech radar. That kind of makes sense. The entire thing resides in the browser. The data is owned by backstage and there's nothing else, there's no dependencies. Service backed, oh cool. I should just shut up and keep scrolling actually. So yeah, there's UI and then there's a tech radar plugin that will save stuff. Service backed plugins make API requests to a service which is running within backstage. Lighthouse is a good example. And then third party plugin which would be something like CircleCI. So the plugin is hitting a proxy internally that would go out to CircleCI. This all makes sense. So this line right here is actually one of the big reasons why a live demo from scratch, to be clear, like from scratch spinning up a Kubernetes cluster and whatnot would be somewhat difficult. Backstage relies heavily on NPM packages both for distribution and structuring of code within projects. Well, why do I keep saying this might be difficult? Well, earlier I said it, I messed with this maybe a year and a half ago. A year and a half ago, whenever there was a change that you made to your backstage config, now I'm not talking persistent data changes. I'm talking we have installed a new plugin or we want to reconfigure a plugin, things like that. It required a full rebuild of backstage. And one of the things that I wanna look into today with everyone is if they've improved that story. Because as it stands, I believe there is not a clean and clear way to take a backstage config with all of its bells and whistles and bring it into a, I know there are ways to do it but an easy way to bring it into a container and deploy it. And I hope I'm wrong. Like I might, I am not, hopefully at this point it is clear I am not an expert and this is really, I'm talking out how I think. So there we go. This though, this makes the whole thing worth it. Right here. Our goal is to provide engineers with the best developer experience in the world. A fantastic developer experience leads to happy creative and productive engineers. So for all that I might be talking about this might be difficult to deploy in a way that's a demo. Oftentimes you're not doing that. We have this, I'm gonna get kind of existential for a second. Oftentimes and lately it feels like I'm not talking like the cloud native community. I'm not talking about anyone in particular. I'm just kind of making an observation in general. We focus so much on how easy something is to spin up and get running because as in this case I will focus on cloud native engineers or just I guess modern developers in general. The easier something is to spin up because sorry, the easier something is to spin up more people think that's better because we want to be able to tear it down again and make one change and then spin it back up and then test our changes. And like that is iterative development. That is that instant dopamine hit of being able to make a change, see the change that I made that change works next thing. But there are some things where that doesn't make sense. Something like backstage is a good example of where that doesn't make sense because you're not getting the dopamine hit by spinning up backstage and then cool, I'm gonna make a change and then push it up. No, no, no, backstage is meant to enable the dopamine hit enable the developers that you want to make changes and learn faster and grow. So if it is a little more difficult than we are used to as a cloud native community to spin up and get running, guess what? There's a reason for that. You're building a whole mini ecosystem just for your company or your project or honestly, I've thought about running backstage just for like my group of friends for our little silly side projects. And that's okay. That's not like it's not meant to necessarily be a press button receive developer experience that everyone will be happy about. Instead, it's about building an ecosystem. And building an ecosystem is not as simple as kind create cluster and helm install backstage. You need to think and you need to plan out what things do my developers need? What services does my company have that I should try and provide a uniform overview for? Right? I know this kind of got a little deep and a little serious, but ultimately we need to think about that to really understand how backstage can accelerate the developer journey and the developer experience. Whew, I am sorry. I kind of zoned out there for a second. Let's look at demos. Demos are cool. So we can explore the UI and basic features of backstage firsthand and they host their own demo, which is awesome. Does this mean we're gonna be able to click around and get into a GCP instance? I sure hope not. But it should show us the layout. Let us click around and see what things backstage can do for us. So welcome to backstage. This is the software catalog. And I need to back up. I didn't actually talk about what a software catalog is. Software catalog is a centralized system that keeps track of ownership and metadata of all the software in your ecosystem. What does that actually mean? Well, in the world of modern microservice development, typically you have separate teams that may be working on separate microservices, all moving towards the same idea or product, correct? So you could have 15, 20 different teams all with completely different approaches and completely, in some cases, completely different languages and environments. But you're still trying to take all of these different components and then turn them into some form of a product that people wanna use. Software catalog is an approach to do that. And this continues to push, how do I put this? All of this is in service of a developer experience not trying to make a company happy, right? The ability to look at all of the different microservices that your company, company X happens to be, oh, I shouldn't use that anymore because there is an X, crap. Company A, all of the software microservices that company A is trying to ship, it's nice to be able to go, oh, I'm using a microservice that leverages this other microservice. I've run into an issue. I don't know who to talk to. And so you can, in a single unified interface, find out what they do, who they are, how they're doing it. And potentially, depending on if you have the backstage setup to do this intent, you can, you can spin up your own mini stack or test environment with their component to validate whether things are working or not. So the software catalog sounds kind of, I don't wanna say simple, but it is extraordinarily powerful. One thing that I will say outright, like I said, I'm not an expert, but also I am summing this up how it fits in my head. I highly recommend other people that are truly interested in more than just me rambling, go to these docs and look through them. Because these are some of the most well-written and well-thought-out docs I've seen. Because again, saying developer experience and developer portal sounds simple, but there are so many, like there's so much behind it in service of making the developer experience good. If you were strongly considering setting this up yourself, you need to be well-versed in some of these terms and some of these ideas. And I'm not the best person to reference it, the docs are. Okay, so you can, like I said, set up different components to the service catalog. And that's just, I guess, identifying what your microservice is. You can do it by hand, you can do it by software components, or sorry, components through backstage, which is like a template. So you click through, or you can point to a, I'm assuming this is a YAML file, are we YAML filing? A point to a YAML file already, you know, hosted within GitHub or, you know, web get thing of choice. And then backstage, we'll pull that configuration in and add it to the software catalog, hooray. So things like the component metadata, again, who's the owner? That's, in my opinion, that's kind of the biggest thing right there. You know, there's different life cycles that you can move components through, but being able at a glance to see who is responsible or who is the owner for microservice that may or may not be giving you trouble, that's huge. And also the great thing is that this is all done in a standardized way. There is no, in an ideal world, and hopefully in most instances of backstage, there isn't an exception. There are ways to pull in every single possible microservice. Even if it might not be, the life cycle might not be managed through backstage, you can still add the metadata into it, so it's referenced in backstage and you can, you know, do searches and kind of see the overall picture of the software you're trying to build. That was a whole lot of random chatting. Let's actually go look at this. So if you click catalog, let's see what kind of software stack are we looking at? I think this is meant to be not actually Spotify, but kind of a mini music application because we have an artist lookup microservice. We have Playback, Playback SDK, podcast, search, shuffle, and then there's a pet store, which is probably just a random demo and then there's backstage itself. But note with some of these, well, actually let's zoom in quick. So let's look at, you can, for each software component to find what system it's part of, so audio Playback system, podcast system. When you think about companies, this I guess would be almost like a business unit potentially and a business unit might have multiple microservices underneath it with sub teams working on it. The owners are defined and the owners can be teams or they can be individual users, which is fantastic. In an ideal world, there would never be a single person that's responsible for it. It would always be a team. Types, service, library, website, you can define things as needed, but here is something that I love to see, tags. So you can give arbitrary tags to each of these components. So you could do things like, I want to find everything that has to do with Java within our stack. Random idea that comes to mind, if there's a huge CVE that comes out for Java, I'm not picking on Java, it's just what I've typed in. Well, now you know that you have to look at these three different stacks to see if that CVE might be applicable to you or not, but there could also be mixed languages within the stack. Microservices written in Java, another one is written in Go. You can tag things with just Playback. This is awesome. So let's look at the shuffle API service and see what kind of stuff gets laid out here. So immediately, this is what I'm talking about by accelerating developer onboarding and also expanding the developer experience. We have a link to the source. We have a link, air quotes, because it's fake to the tech docs, that would be something that references it internally. We can click who the owner is and then get more information on them. We can click the system, I'll do that in a second, and it will probably expand out and show all the different components. And then it also shows all of the relations that this component has. So it's part of the audio playback system and it is owned by user guests. And if we click the system of, actually no, let's stick in here. So in this component, there's a CI CD, which okay, you need to add an annotation to your component, it's a demo, cool. But you can still, for each individual component, define what the CI CD system is and there would be a plugin that would tie it into this. I'm curious if there's any other example services in here that would show that. You can define both the APIs that it's providing and the internal APIs that it's consuming. The dependencies, which components depend on it, I'll find more of that. And then basic things like to-dos and docs, if there's a way to link to the docs. And so if we go to the system view, we see that the system, you can look at who the owner of the system is, the domain, so I guess that would be one level up of that business unit. Source of the whole system, which pulls all of these components together, again, tech docs internally. And then we can click into these different components. I'm kind of speed running to see if there's any other good examples. Catalog, let's look at a different system. Ooh, but this one was cool. This actually shows a little bit more of a complex relationship here. So the artist lookup component depends on the artist DB resource, and then multiple things consume the artist lookup component. That's cool. And then there are, each component can also have arbitrary links attached to them. So now if we click, so that's like all the stuff that is in the catalog part of backstage. But you can also define all the APIs that you're trying to publish, and I think consume. So let's click Star Wars GraphQL API. Team B is apparently owned, owns it, but nothing currently consumes it. Pet Store webhook. Oh, that's cool. That's super cool. Where was Star Wars GraphQL? So in this view, you can also click definition and then it will output the actual definition of the API that you're referencing. So in this case here is the API for the Star Wars GraphQL database. And so if we go to Pet Store and go to definition, this happens to be a Swagger API and has the Swagger API docs just embedded in here, which is sweet. Because the other thing is you can actually, for those that haven't messed around with Swagger, if you have valid authentication to the API and you can like click that and put that in there, you can actually use this view to try it out and run queries against this to test the API live. Swagger is kind of cool and could have its own whole episode to it, in my opinion. So that's the API view in backstage. There are different templates. I think I mentioned them. You have different templates that you can use to create new components. In this demo, those templates do not exist, kind of makes sense. But we have separate plugins here. There's the TechRadar plugin we were talking about before. There's cost insights and then there's GraphQL. TechRadar is probably what we would expect. So in this case, imagine you are a 1000 plus person engineering group. That's a whole lot of engineers that might be looking at different stacks and different technologies. So having a pretty basic TechRadar showing, these things are good, these things are interesting, but not necessarily good. Dear God, don't touch these, very useful. And in fact, the CNCF itself, I'm not trying to promote one thing or another, but the CNCF used to publish TechRadars for new technologies in our ecosystem. And I think we're trying to bring them back. I'll have to ask Taylor if we're trying to bring back TechRadars. So there's a cost insights plugin. I'm curious if this is, is this like, is it cost insights, cost insights? Interesting. Thank you, Luna. Am I gonna have to take away the ball? So does, I'm curious if, does this integrate with, well, I should back up. What does this integrate with? You can begin working with today on GitHub, include, all right, let's see what it says on GitHub. AWS Cost Explorer. Okay. I think this is cool. Well, no, this is cool. If this works, how I think it works. In your apps, in the app catalog, in backstage, you can actually assign arbitrary engineering cost for that app. But also, I'm gonna kind of jump up. Where was it? Create a cost insights client. Clients must implement the cost insights API interface. So if we think back to the plugin model, for backstage plugins, here actually, docs, architecture, plugin architecture. Right. So there's standalone, there's service backed, and then there's third party. I am curious if, I'm sorry, I'm jumping around. If you can implement this cost insights API interface into this, so that if there is a GCP plugin or an AWS plugin, and it happens to have the cost insights API, you can pull that into that cost insights plugin. Is the cost insights API to integrate with other plugins for cloud providers? So let's actually like kind of mess around with the cost insights plugin quick. So we can do cost overview. So this is looking at, I'm assuming the cost of development based on engineer time. No, it's not, sick. Okay, so this must interact with, I mean, this actually has to interact with more than AWS because BigQuery's GCP. So in this case, look, our BigQuery cost is increasing. I would say our cloud storage in our fictional company has slowly been increasing. But other expenses are increasing, but not nearly as bad from the looks of it. But BigQuery, BigQuery's blowing up. It's more than doubled, 2,000 down to 6,000, it's tripled. If we break it down by project, we can actually see, you know, which projects are spending the most money. Project bead, you're doing good. You're staying pretty flat. What, okay, so they provide action items. That's cool. Investigate the cost growth in an example project. View instructions, what do you do? Oh, I think this is just kind of default language. Convert to US dollars, convert to engineers. Convert to carbon offsets, that's awesome. So what happens when we click compute engine? Okay, so it's just looking at the compute engine cost. BigQuery, so it breaks down based on service, but from the very top, you can kind of look at more. I would love to see this actually installed. I'm kind of wondering if this is possible to install across some of the CNCF-like infrastructure. This could be useful. I'm getting ideas. I need to not get ideas right now. All right, last plugin in the navigation is the GraphQL backend. GraphEQL, I expand this out. Oh, good, I can, cool. Do I want to though? Oh, so this is just kind of like a raw editor that you can run GraphQL queries against. I wonder what that'll do. I'ma do it, I'm getting alerts everywhere now. Oh, oh yeah, I'm sorry, that actually makes sense. So that probably just granted, so this whole thing is GraphE, okay. GraphEQL is a GraphQL client, and I probably just connected GraphEQL to the GitHub GraphQL API with my credentials for this session. Okay, well if this is just a GraphQL backend, I'm not even gonna kind of waste much time on it. So if we go into the kind of default settings, you can set up authentication providers. Hey, look, it shows that I've logged in, provides authentication towards GitHub APIs, yeah. You can set feature flags. This is another interesting and powerful part of backstage. In case you want to test certain features within your account and you know, it has a dark mode, which let's be honest, that's the real thing. So what else do we want to cover around backstage? Local development, core features. We talked about software catalog. Actually, yeah, let's, Kubernetes and backstage. Okay, so this is Kubernetes is kind of a top level resource within backstage. And so you can install the Kubernetes plugin and then be able to manage different services or definitions of components that are Kubernetes resources. That makes sense. Tech docs, Spotify's homegrown docs like code solution built directly into backstage. Tech docs add on framework and make docs plugins. So I think from the looks of it, this docs page is almost built based on their own tech docs framework. That's kind of cool, that's dog fooded. But also there's a plugin for make docs. All right, let's actually see what it would have taken to install backstage into something like kind or Minikube. And then I think I will update my notes, wrap it up and then everyone can have an awesome Thursday. Or would it be a leap Thursday? So this example is using Minikube, create a namespace, backstage requires Postgres. For any backstage configuration secrets such as auth tokens, we can create a similar Kubernetes secret just like we did for Postgres. So GitHub tokens, GCP tokens, et cetera. Everything that I said in the beginning is a lie. That or this has gotten a lot easier. Katie, I'm straight up, I'm changing the script. Like in the next couple of weeks we are doing a session two and I'm just gonna spend five minutes and actually deploy this because come on. Like, yes, if you make changes to backstage you need to rebuild the container and deploy it that's not a problem. But it seemed so much more difficult before and this is a lot cleaner. So creating a backstage deployment once you have Postgres installed and running is as simple as a YAML file referencing backstage and you can even just, I imagine use their default for a demo, boom, it works. And if you want, yeah, you build your custom image within Minikube, that's one kind of advantage with Minikube. You can still do it with like kind and K3S, but that said, build image, deploy, and then you define your backend. You expose it with a service at this point we're just talking Kubernetes, cool. Okay, what is my Linux distribution? That, while not relevant is something I will happily talk about. So I use a Linux distribution called Bluefin. It is part of an open source family of essentially Fedora remix distributions called Universal Blue. So if you go to universalblue.org it has pretty much everything you want. They have multiple distributions, I will call them builds from now on or images, actually images is more appropriate. The whole idea here is they're building a Linux distribution in the same way that we build containers because it's a bunch of cloud native nerds doing it. There's a server version that's called U-Core. There is a gaming distribution that is even installable on Steam Dex and that's called the Bazite. And then there is Bluefin. Bluefin itself has its own separate website, which is not that. What is it, bluefin.com? Project bluefin.io, but ultimately this is, that's the distribution I use. It's awesome. A near and dear friend of mine, George Castro, he's been the, not the main developer shall we say, actually he kind of has been, but he's been, this is his pipe dream and it's wonderful. It's meant for developers. And yeah, everything is either flat pack or brew. And they have their own kind of custom tooling both around building this in a way and also building it, interacting with it, auto updates. Part of the reason why this is built off of a remix of Fedora. It's called Fedora Silver Blue. Everything's a container. And so they use, it's called RPMOS Tree. And that's kind of hard to read. But like this is, my operating system is a Docker container. It is a Docker image. And so it will periodically update this Docker image. And then based on like diffs, it knows what to update. I could actually do a whole session on this, except I'm trying to stick to cloud native stuff. This is cloud native, but it's cloud native adjacent. It's not a CNCF project. Okay, I think that's where we're gonna end it. I'm going to update the summary notes. I'm going to try and plan out a second session around backstage. And then yeah, thanks for hanging out. Thanks for learning with me. Thanks for watching me bumble through some docs. I love the idea of backstage. And if I ever go someplace, not the CNCF, I pray that they have backstage setup, regardless of my role, because it'll make my life a lot easier. So with that, thank you everybody. Have a good, wonderful Thursday. And some of you, you know, if you haven't registered for KubeCon, please do. I have a feeling we're gonna wait list soon. All right, take care. Bye.