 All right, I think we can start. There's going to be some people that come in. The good bit of this presentation is later on anyway, so it's OK. OK, so just before I get started, I want to call out that this is kind of like a short and down version of our talk that we're going to give on Thursday here at KubeCon. So if you're watching this in recording or in the stream, go ahead and check out the talk on Thursday for the full, unedited, extended cut of this talk. Let's start with some introductions. My name is Ben Lambert. I'm a senior engineer at Spotify and a maintainer of Backstage. I have also Patrick with me. Hello, my name is Patrick. Go by a Rugby Pong GitHub and also a engineer at Spotify and a core maintainer of Backstage. Cool. So first off, I want to call out some new project areas since the last time that we spoke in Amsterdam earlier this year. So we've got three new project areas to call out with some dedicated people taking ownership of these parts of the code. I should talk a little bit about the difference between the incubating project areas and I guess you can call them graduated project areas. Incubating is basically also going to be partly maintained by the maintainer team too. So we're going to help kick things off and still be a part of that project area until we feel comfortable from both sides that we can hand over more of the maintainer cost for that area. So there's an open API project area, which Aramis has taken over. This project area is looking at how we can improve the open API tooling and open API specs for the entire of the Backstage back-end plugin ecosystem. So this is things like providing typed clients for communication with back-end plugins like the catalog and also doing request validation and things like that in Express. There's a scaffold to project area. For those that might have seen me and Patrick speak before, you'll know that this is kind of a project area which is kind of close to my heart and I'm especially interested in. It's a heavily used feature of Backstage that's grown in complexity since we first built it basically. So it's great to get both Paul Cowan from Front-End Rescue and Bogdan Netshi Parenko from ball.com, who have been super involved as contributors from the early days of the scaffolder and its inception. So it's great to get them on board to help us maintain it and take it further. And lastly, there's also the permissions framework and all the permissions packages that go with it. This has been taken ownership by the Imaginary Goat Squad at Spotify, who are the original authors of the RFC and the actual implementation. Don't ask about the squad names at Spotify. They're all very strange. They mean nothing. We work in math mammals. Yeah, it's strange. But yeah. Cool, Patrick. All right, thank you. So that was a couple of project updates and now we are moving on to talking about what we have been working on for the Backstage Front-End Framework. So since the beginning of the Backstage project, you've always had to integrate plugins using TypeScript code. Typically, you'll find one or a few snippets in a plug-in readme that tell you to import a few things, render them, and so on. And it's fairly simple, but already there. There's a few points of friction, right? If you're someone that wants to set up Backstage and you're not familiar with TypeScript React, you might not even need to know how to code at all or be that familiar with it. That's a lot of friction. But even if you have that knowledge, it can get pretty complicated as your project grows. One snippet of code is at a time and eventually you're here. You're sifting through your entity pages, trying to figure out where to add that card for your Java services. So what we have been working on is a new way to integrate plugins into a Backstage application. In essence, what we want to do is get rid of these snippets of code that you need to use to integrate your plugins into the app. So we still want to keep the flexibility though of allowing you to customize your app for your own needs. Now, we call this overall solution, declarative integration. And it's worth noting that this is something that we work on for both frontend and backend plugins. But with the changes to the new backend system, there's a lot less work to be done there. So what we've been working on recently and what we're talking about today, it's completely the frontend bit of the system. So if you've been around, you know that we presented declarative integration as part of our roadmap at KubeCon earlier this year in Amsterdam. We've been working on the implementation since just after the summer. So what you see here today, it's work in progress and it's subject to change. So as we sat down to design this new system, we had a couple of goals in mind. So first and foremost, we wanted to remove the need to write TypeScript to integrate plugins. But we didn't just want to shift it all over to YAML or something like that. We wanted to make sure that plugins can provide defaults so that the only configuration you need is for the customization of your app. We also wanted to clean up the composition patterns, making it easier to build plugins that extend or compose with other plugins. And finally, we wanted to work towards the possibility of dynamic plugins as a first-class citizen of the backstage framework. So that essentially allows you to install new plugins in a running app. Now, to be clear, dynamic installation is not part of what we're building right now and what we're talking about today, but it's something we had in mind when designing the system. Okay, so that's some background. Now let's talk about the design of this new system. Now, this is a very high-level overview of the new framework. At the root, there is an app instance. So that's responsible for wiring things together and providing a base to build upon. You populate it with content by installing plugins, but plugins are also just lightweight containers and they contain extensions. And it's in the extensions where all of the functionality actually lies. It's when you take these extensions, wire them together when you actually get a backstage app that you can run and use. Now, if you're familiar with the current system, you know that a lot has really changed here. This is more or less what it looks like today and even apps and plugins in the new system are, they kinda look the same, at least from the outside. Now the big difference in the new system is the extensions and the way extensions talk to each other. So that's where we're gonna focus now. So let's return to our snippet of code for installing an extension in our current system and we're gonna look at how we've changed it with the new system. So in the new system, each extension provides its own attachment point. Now, attachment point is what we call the location in the app that an extension is installed at. So in this case, we're installing the GraphQL page as a top level route of the app. Now, in the new system, extensions can provide the attachment point themselves. So we can get rid of that. Extensions can also provide a default integration configuration. In this case, it's the route path that we want our page to be rendered at. So let's drop that bit too. And we have also moved away from React as a first class building block. Now, it was nice to easily be able to introduce extra React elements in places in your app, but it was also really confusing at times the way we used React to compose the app. Extensions are now just plain values that you import directly into the app instead. And finally, we have added the option to automatically detect plugin packages and import them into the app. So you no longer need this explicit import either. Cool, so that's all code gone. Great. How do we actually build an app out of that? So to understand the new extension structure, we'll start by looking at the structure of an existing backstage application. So what you see here is a thin slice of an app. It's a tree of React elements. They end up determines the structure of that. Some of these elements are provided by extensions. Some of them are provided by plugins. Sorry, some are provided by plugins. Some are provided by the framework itself and some are just React components. And as a side note, it's this mixing of different type of React components that we'd like to avoid. So we're instead going for a cleaner separation between the backstage core framework and React. But all right, so what does this look like in the new system? If there's no code, there's gonna be a lot of YAML, right? Yep, so to the right here, you can see the structure of the app or what it's represented as in the new system. You no longer have an explicit tree. Instead, you have a flat list of extensions with each extension declaring its attachment point. Now, it's not until runtime that this is all resolved into an actual structure and this tree structure you actually had before. Now, however, the good thing about all of this YAML is that you don't need to write it. All this entire structure is provided by us from the plugin and from the framework itself. That means that in an actual app, you'll have something that looks closer to this. If all you want are the defaults, your entire app can more or less be defined by just the dependencies in your package JSON. Of course, we want to be able to customize apps too, so we're gonna show you that in just a second. But first, I wanna talk about something else here. So one of the main strengths in our current system is how well it visualizes the structure of the app. You can look at this code and you can kind of see where everything is rendered, what routes and so on. And it was actually one of our main design goals of this system. And of course, now in the new system, we really can't have that goal because we wanted to remove code, so yeah. But we do still recognize that it's very important to be able to see the structure of an app and visualize it for debugging and just generally be able to work with it. So we instead plan to build tools for that purpose. Okay, so let me show you what that looks like. And the overall app structure by jumping into a demo. All right, so what we have here is a very lightweight backstage application, just a couple of plugins installed, we have a few cards on our entity page and so on, nothing too fancy. It is all running in the new system. So this is all extensions and whatnot in the new system. And we can see that using our visualizer down here as well. So this is the entire app, that's the structure of the entire app. All of the square blue boxes represent individual extensions in this application. You can also see edges between them. The rounded boxes are inputs in the edges and the inputs form attachment points. So every single extension in this app is attached to a parent extension to form this tree. All of them except for this root app extension here. You can see it has a couple of inputs, among other things, you can see that we have two different themes installed in this app. So extensions are no longer only react elements, we can also have other kinds of things installed as extensions too. Now we also have a more detailed view. So every item here is also an extension, it's the same view, but a little bit with a bit more details. We can see here for example, we have the catalog page extension. I can navigate to it here. So the catalog page extension provides all of the content that you see here to the right on this page, everything except for the sidebar. Now these colored dots here to the right on these extensions, they represent outputs. So you can see here, among other things the react page, catalog page outputs a react element. And this output is passed to the parent extension through the attachment point and this is how extensions communicate with each other. And this goes on all the way up. So you can see the routes here, also outputs a react element, all the way up to the root of the app where we output the single react element that we can render the entire app with. All right. There we go, magic key points for that. One second, I'm just gonna take a drink. So let's shift gears a little bit and talk about how you actually use this new system. So we mostly skip over how to set up the app itself and the plugins, because they're gonna remain fairly similar to existing system. It's that we'll focus on what it looks like to create extensions and how to configure them in the app. So let's return to our very simple example from previously. So far we're using the defaults, right? But what if we wanna change the path for the settings page instead of the preferences, right? So in the old system, of course, you can just update that in the JSX directly in the directory. But with declarative integration, you can instead change the path by passing configuration to our settings page extension. And we do this by referencing the extension by its ID and then passing the extension specific config to update the path. Now, this next example is a little bit trickier. So what we'd like to do is disable an extension in a certain environment. And in the old system, there was no real great way to do this and you'd end up coming up with your own switch or feature flag or whatever. And for simplicity, we're just commenting it out in the example. But with declarative integration, you can simply disable the extension by providing the config that you can see on the right. Now, these are just two small examples of how you can customize your app using declarative integration. Extensions can, of course, declare their own configuration schema to make any form of static configuration possible. And you're also able to change that attachment point of extensions and, of course, completely override any extension yourself. Let's look at how you can create your own extensions. So almost all of the time, you're gonna be using these predefined extension creators like create page extension here. This predefined extension creators come with a useful set of defaults and an implementation and all we need to do in this case is give it an ID, a path and a way to load the page component itself. Great, so we can create our extension but how do we install this in the app? So, as mentioned before, we use plugins as containers for extensions to install these things into the app. So the recommended pattern is to stick to one plugin per package and by making the plugin the default export of that package, it will automatically be picked up and installed in the app. So there's also a lower level API here, which is called create extension. So this is what you're gonna use if you wanna make your own extension creators or if you simply want a little bit more detailed control over an extension itself. So you can see here, or what you can see here is the definition for the core layout extension. So this extension is like the shell for the entire application which you can see in backstage and it renders like the navigation and the content for the page that you can see and it's all wrapped in the sidebar page. You can see that at the bottom with the factory bit. So you can see when using create extension directly, we've got a lot more control over what we actually want to do in the extension. We have to declare an ID, an attachment point, some inputs which are gonna be the navigation sidebar and the app content and then our outputs which is the react element. We could also define configuration schema in here but for this example, we kind of skipped over that to make it a little bit more simple. You should now kind of see it in your head how you can build up this tree of inputs and outputs and compose different extensions together to form the app. So obviously in these examples, we're taking inputs of react components and then output in a react component but they don't have to be react elements like they can be any form of data type, whatever you like. And again, this is a lower level API. You'll mostly be using these predefined extension creators and whilst we're not gonna be using create extension directly today, I want you to keep this core layout extension in mind because it's gonna come in useful in the demo, which I'm gonna go into now. Okay, let's write some code. Where is it? Here it is. Okay, so this is the example app that Patrick was just showing. I'm just gonna go ahead and go to the example website component. So this is the app config on the right. Can you all see that? Anybody not see that? Good, cool. So this is the app config that you just get with your backstage application. You might have seen it before. And we've got a list of extensions installed. So you can see here we've got the entity cards. So for extensions that are order important or that they depend on the order, we disable these extensions by default so that you can enable them in the correct order or how you wanna see them in the app. So if I was to reorder one of these extensions by doing this and then save, we should say yes, that it updates, changes the order on the left. If I wanna disable an extension, I can just pass in false and this disables. I can also provide additional configuration. So the entity content to do here, we can see this to do plugin. Yes. This is a extension which basically lists the to do comments inside your application. We have default config here, which is the tab title. I can go ahead and override that by doing some more config here. Oops. Oops, today's it. Config and then title. And then let's say we wanna override this to be roadmap instead. Cool. So now it overrides it. What else? Okay, so this is configuring extensions that are already installed in the app. But let's say I wanna add some more extensions, right? So I'm just gonna go to our themes here. So we can see, as Patrick mentioned earlier, we have the two themes that are installed in the app. So we've got the light theme and the dark theme. So let's say I wanna add a new theme. Let's just scroll to the bottom of this. Cool. I'm just gonna go and add, I think it's fine. Add our new internal theme package. If I could spell, there we go. So by adding this to the package, and what it's gonna do is it's gonna automatically detect there's a new extension and then bootstrap it into the app for us automatically. So you might have seen, you eagle-eyed people there, that we have now a new theme, which is the aperture theme. And it's already been enabled in the app. I can go ahead and click it and then see that this has already been configured for us and easy to use. I'm just gonna switch back to the light theme again. So the call layout extension that we were talking about earlier, an override in extensions. So what I can do here is also I can install another extension which is gonna replace the implementation for other extensions that are provided by default or provided from elsewhere. So I'm gonna go ahead here and add our new one, which is called top nav. So what this is gonna do is it's gonna replace the call layout extension and replace the navigation component to give us a top navigation in backstage. So override in extensions, super easy, super simple. What about removing extensions, right? So I can just go ahead and do yaw and remove at internal top nav. I hope this is gonna work. And then by magic, it just resets. That's all good. So this is configuring extensions and basically adding and removing. What about building our own extension? So in the index.jsx here, this might look a little bit different from what you have already. It's normally a lot of JSX. This is kind of how we would configure it in the new system. So we have a create app call here, which passes in a plugin to the feature detection. So because this is not a separate module, we can opt into discovering this manually by adding it to the features property here. And then we've got a plugin and our extensions are raised empty. So I'm gonna go and create an extension now, which is basically just gonna render, I don't know, some welcome content on a little page, right? And now we're gonna do that. So we're gonna use that create page extension that we talked about earlier. So I'm just gonna do const, welcome, this keyboard is so low. Equals create page extension. We need to give this an ID, so I'm just gonna call this welcome.page. We need to give it a loader, which is gonna tell us how to get the react component that we wanna render. I'm just gonna cheat a little bit here and just do promise, oh my God, not resolve, if I could spell again. There we go. And then h1, welcome. And then we also need to give it a default path. So this is the default configuration that you'd get from the extension. You can of course override this if you want, but we're just gonna provide this with a welcome path. And then what we wanna do is add this extension to our plugin, save. And then hopefully, if this is all worked, I should be able to go to welcome here and we get a welcome page. Yay, great. Yeah, so that's kind of what I've got for a quick demo here. I'm gonna pass back over to Patrick now. All right, so good demo. Now, as you know, we've had to cut down this talk down a little bit. So we're gonna skip this section on compatibility. I'll give you a summary. So what our idea for being able to migrate existing applications is a top-down approach where you can switch out a, in an existing app, you can switch it to the new system at the root and then use a compatibility helper that lets you use the existing react-based structure of an app and import it into the new system. Once you got that, you can start migrating plugins one at a time until your entire app has been moved over. But as Ben mentioned, on Thursday, there's a talk where we go into this entire section. Okay, so let's look back at this timeline. I know a lot of you are gonna be curious about this. Our hope is to be able to ship an early alpha release at the end of this year, but no promises. Now, this is still going to be a very early release and you should not be migrating your production apps, but there will be a complete system to try out and play around with and we are looking forward to your feedback. Thank you. Awesome, Patrick, Ben, thank you so much. Do we have any questions, the maintainers? So thank you so much. I think you kind of touched a little bit up on this. So are we saying that at some point when this is something that everybody can, like adopters can start using, we can have like a hybrid of like old system and new system? Yes, so both the core APIs that you use to build plugins will have forwards compatibility and work in the new system. So if you have old plugin code, the use API use route of those kind of things, that will work in the new system. So all your plugin code is essentially going to be migrated by code mods. But of course you have the glue of the actual extensions, the way you define how a plugin is integrated into that, that you can ship duplicates of in a plugin. And then inside the app itself, yeah, you can mix two systems, not really mix, but you can import the old system into the new. So effectively have a mix. Any other questions? Hi, great talk. So how quickly do you expect to mature this? We expect to refactor early in 24. So should we go for the alpha or would you calm down and, yeah, wait a little bit. Yeah, be calm I think, be calm. We don't know yet how, you know, all of the existing use cases and plugins and so on, how well they translate to the existing system if we need to make more changes to allow for things that plugins need to do that the system doesn't support yet. So we want to discover all of those things, make sure it's also a little bit for broader usage. Yeah, I think that it's also changed quite a lot dramatically over the past few months too. So it's kind of, yeah, it's a moving target I still think at the minute. Let's give it time to settle it and then we'll reach out. Sure. Any other questions? No? Thank you so much. Thank you. Thank you.