 My name is Thomas Grönbeck. I'm a senior manager on bank data and managing a few platform teams, one of them being the developer portal team working with Backstage. OK. So my talk is called, how did Backstage make 500-plus developers happy in a highly regulated enterprise? And when I said in December, we had founded our developer experience initiative. We were all high hopes. We wanted to conquer the world. And yeah, check mark with bold and flashy title for the talk. And then, are we there yet? Three months later? No, we are not there yet. Is it hard to validate and measure developer experience? It's freaking hard. So we are on this journey, and I want to share what we learned along the way and bring it to you guys so that you can learn from it. So yeah, that's what we are up to now. So just quickly about bank data, because it's a little bit important for this story. We are a regulated enterprise. What we're going to do, our mission is actually to build the future of a digital banking. We are operated out of Denmark. Any Danes in the room? Perfect. So yeah, the green dots, it's our eight banks. We actually owed by eight banks. These banks have close to 2 million customers, a lot of business customers as well. And we make all the products they need. We make all the products for the end user, for the Danish citizens, and all the products for the banking employees. And the light blue dots, that's bank data. That's where we are situated, close to 1,000 employees. And when you make banking application for a lot of Danish people, it is a critical business. So we are highly regulated. These are just some of the things we have to put up with. So all our teams need to be really, really certain about how they do the implementation of their solutions. And when you make banking applications for eight banks, then you also have these eight banks making sure that you make their business in the right way. So a lot of weight on our teams. We actually want to solve this problem with developer experience. We want to build and bake compliance demands into the way that we do stuff. So that is into our developer experience initiative, into our developer portal. And I'll come back to the two tracks that we have made, making sure that our developers are happy. And yeah, just a little more story. We bank data was founded when IT was starting to get important for banks. So we were started in the 60s. And we took all the way up to now microservices AI cloud. But we also have mainframes. So this is a big landscape. And that is also important to know when we speak developer experience, because teams own stuff on different tool stacks. I imagine that's the same with a lot of your companies out there. So why did we start that? It was actually partly because of all these different tool stacks. So this could be the experience for a developer in bank data. Because these teams, they might have an entrant responsibility for mainframe applications, maybe for mobile application, front end applications, and also microservice in Java running on our OpenShift container platform. So it's a wide tool stack. And as you probably know, when tooling are made at different times by different teams, the experience is getting fragmented. And we are there as well. So maybe you need to find your logging here. You're logging over there on another tool stack. That is complex for our developers. Of course, we went the DevOps way as well, putting more complexity into the team. You build it, you run it. Shift left on testing, security, compliance. So even more cognitive load. You could just see the heads. Little fuses exploding in the heads of these developers. We wanted to do that better. That's also why we speak about developer experience and backstage. So what did our developers ask for? They asked for this, among other things. So we wanted it to be easy to find documentation. I don't know you guys. You maybe also have confluence. Confluence it for everybody. So our centralized managed documentation was kind of lost in there, along with all the team documentation. Developers wanted that it was easy to find the templates they could use. They say, please make it easy for me to follow best practices and be secure and compliant by design. And also very important, it's that other experience with the cockpit. It's very difficult to onboard new people if you have a fragmented developer experience. So they wanted them to be able to take a new person into the team and that they could be effective and help the team faster. And of course, also reduce the complexity and the cognitive load. Help me feel effective in my daily work. So this was some of the demands. Some of the things we took into our developer experience initiative, yeah, good. So with this data, what did we actually do? So we actually already in 2023 started to look into how to set up an initiative around developer experience. And we got our upper management to approve this. We actually found funding as well. And we set up these five core values for our developer experience initiative. So we wanted to make sure that our developers would have a highly available tool stack. The tool stack needed to be there when they needed it to be effective in the daily work. So we speak a lot about SLAs, SLOs, SLIs. And we actually really put our efforts from the platform teams into giving good and stable platforms. We also wanted to reduce wait times. So if you had a ticket, you shouldn't wait for it. You shouldn't have that task switch. You probably know that when you wait for stuff, you go to do something else. We wanted to make that less than before. And we wanted to minimize cognitive load. We wanted to enforce self-service that's speaking a lot into backstates as well, and also enhance discoverability that it is exactly what backstates can do for us. So when we started to define these five core values, we could see, yeah, we can work with that. So what we did was actually to fund two teams to start on this journey. So our developer experience initiative is two teams right now. Two small teams, three person each. To make it more effective, we wanted to focus on outcome instead of features. A lot of other talks have been about that as well. Know your problem. Start by fixing one problem. That's what we wanted the teams to do. They should be working in an experimentation and way with fast feedback loops. And they should try to set up a north star, a guiding star for where they wanted to go. And last but not least, they should build one slice of cake at a time. So one problem, fix it, move on. And I'll come back to the lessons because when you take platform engineers that could build a whole portal and give them an approach like this, it's not easy. Let's come back to those lessons learned. So these are our two tracks in the developer experience initiative. We call them Automation Highway and our developer portal. I'll just give a brief introduction to Automation Highway. I'll go much further down in Developer Portal. So Automation Highway is actually the way to make the effortless journey for a developer. We create. We make it easy for them to create a service. Right now, we support a Krakos Java microservice running on our container platform. We make all these set up for them. So in two and a half minutes, they can spin up the shell for the business logic and they can put the business logic into the shell. With observability and Argo and SonarCube put in by default. So they really like this and they want more of this. So after Krakos, after we support Krakos in a good way, we will move on to probably spring microservices that we also have a lot of teams using. So Developer Portal, we use backstage for that surprise. And yeah, I'll go much further down to that one. So to make this a thing in bank data, we actually started to make some marketing. Because we had upper management support. But now we needed to get the support for bank data. Because yeah, these are new products. These are new ways of working. These are something new for teams to start to use. So the last Tuesday each month, we make a one hour review where we invite everybody between, yeah, we are five to 600 developers. So they can all come and see what we have built in the last month. And we also had the yearly summit here in February. So that was also an event where we did some talks about these two tracks, showed it. How can you use these products? What are we building? And we actually had 100 people joining in the last Tuesday in February. So we are pretty OK with that. And it really helps to pave the ways for adoption on these products. Good. So going into backstage a little deeper, our ambition for backstage is actually, yeah, one stop shop. So it's the end goal. And it's the picture we are painting. And it's the story that we are telling. So one stop shop, single pane of glass, where you go when you need to do your daily work as a developer. So I just want to share our track vision. This is a track vision that was founded in a workshop with the developer portal core team and stakeholders. So the product is for all bank developers. But we are especially catering for profiles that are new hires or new in a domain or new on a tool stack. And our assumption is that if we can make a great product onboarding them, then it can also be effective for more experienced profiles. So we are optimizing for onboarding. We make sure that they can find everything they need. So that could be the service catalog templates that can make it easy for them to develop. The developer portal is the one stop shop, the single pane of glass. And yeah, our product provides easy updated and sociable access to golden path. And golden path, that's important. That's actually what we want to optimize for. So we want to have a golden path for front end, a golden path for back end. And yeah, and we are getting closer and closer every day. So just to share a timeline on how we started this journey. Actually, in 2023, we used it quite some time where we had some interns, because this was before we got funded. So we did a lot of proof of concept. We looked at a lot of the different plug ins. We started to set it up, but we didn't put it into production. I didn't want to have expectations from 500 developers with interns and student workers. We wanted to have a funded team before we took that thing on us. And then when it was the 1st of January, we actually set up a new CTO area. And the CTO area got ownership of the developer experience initiative. Then we started the core team with backstage. And they set a north star metric. And the north star metric was inspired by Spotify and other people. This commitment to start to measuring time to 10 pull requests. And as I said, we chose that metric. And as you see in the end of the line, we still need to establish that baseline. It's been harder than we expected. We also said that what we are actually the first piece of the cake that is developer onboarding for front-end developers. So it was to set more, what can you say, one goal, not conquering everything, one goal. And that was make a great developer experience for developer onboarding for front-end developers. And then we built. And actually, our first release is planned here in start of April. And I'll come back to lessons learned, because I didn't expect that it would take three months to get here, because we came out of an experiment with a lot of knowledge. But back to the learnings later. So this is actually the timeline up till now. Yeah, our first release, as you can see, a little bit bigger slice of cake. So of course, we want to make the golden path for the front-end developers. This automation highway, our another track in developer experience initiative, we want to make the golden path for them as well. And we want to make a software template where they can trigger and start the GitHub action that built this Krakus Java microservices so that we can standardize the way that all the developers that can use that tool stack will do that. We will also have the software catalog in the first release. It will be a subset of services and APIs. We will integrate with Azure AD, bring all the users, the team, and show ownership. And then we hope that we can make the first inner sourcing model for pilot teams, because we really believe in the inner sourcing. We have a strong inner sourcing ambition. And if we are to create a great product with Backstage, we need a lot of contributors in bank data. So we are starting to get pilots up and running. We still have a long way to go. Yes, so what did we learn along the way? Let me share that with you. So we have this ambition to bake compliance in. And developers really want that. We actually thought it would be harder because we have quite some of our Streamaligned teams that are pretty deep into the infrastructure. But when we started to tell this story, they said, please, I can see you nodding. Please take this effort away. Let me be efficient bringing value to the banks within my domain. They really want that. Make golden path the easy option because both developers and our auditors love compliance by design. And if you are able to create a cool plugin that can actually solve bad developer experience and compliance, you might have a killer app. And yeah, we have an example. So we have mainframe, as I shared. We actually have user interface in terminal. And actually, when you create a lot of change management, we have a lot of automatic flows that will start and make that ticket, that change management number, as you can say. But the approval is actually inside this horrible experience. So we built a plugin. It was actually a Bachelor product for some students. And you can see up in the top, you have an approve button. And actually, pretty in a short time, when people approve changes, they don't need to go into that bad experience. They can use a plugin in backstage. And people are really looking forward to that. We hope this can go into the first release as well. But yeah, we're not certain. So another important topic that we learned a lot of lessons around is how can we deliver value with backstage? And I must say, also, developing for three months and still not having released yet, if you are in a regulated company like we are delivering value to banks, there are a lot of strict processes around security access and data. And it just takes time to integrate these different domains, these different systems into backstage. And you need to communicate this to your stakeholders. Because when you say that you are focusing on outcomes and not features, they want to see value. So communicate if you are in a domain where this is difficult, remember to communicate it. We also had a harder learning curve than expected. Because we are three in the team, and we got one person that was in our experiment in 2023. We took him in. He was an intern, so it's his first job. So having him rub off his knowledge to two more experienced people, it takes time. And people they know to get their hands dirty as well, there are a learning curve on backstage. So yeah, it takes time. We also had quite a big mindset shift going from doing features to working with outcomes. And yeah, this is my point of view. But when technical people, they were probably by default think about making a robust platform, making sure that it can scale. Because they know that they will get support the minute that it goes out the door. So they will probably optimize for that. And right now, we want to see that we optimize for value. So we need to find a balance there. And we need to do coaching on that as well. Yeah. Yeah, also a lesson around please remember to validate market fit and value. And do it early with your pilots, because you can get that feedback into the team and you can create a better product. The last lesson learned is around how to measure developer experience. I just, this is my, it's a very important message for me. There is no silver bullet and no simple formula for this. But there are a lot of great inspiration. And I think I myself has read a lot of it. But if I have good stuff, yeah. Get DX has a lot of good stuff on this. And Google also say good stuff about this. They also say this is hard. And they say that you need to find a balance around soft and hard metrics. Survey is fine, but also get data, hard data from the tool chains or the systems. So you need to find your way in this. We took the approach to set a north star metrics around time to tell pull requests. We haven't established it yet. It's harder than expected, but we haven't given up. So we, and we have its historical data in our BitBug and GitHub environment. So we can, we can also find the historical data and we can also find out when we on board new front and developers, how quickly can they produce the 10 pull requests. So this is still our ambition. And then this one stop shop developers, they really like this vision. So, and they want to contribute to this. And when we shared it on our yearly summit in February, we really got a lot of feedback. And we really validated by the support that this is a product to build. And we have a lot of people, almost too many, that are going to the core team and say, oh, I know what I want to put into the shop. So focus is also very important to stay the course. Yeah, and simple surveys can be a great start. And we have an example from our automation highway. And it's very simple. When they finished driving the highway, you can say, we just make a simple survey. Yeah, did this free up your time to deliver value to the banks? And right now we have a 3.8 rating on that. So people find value in the product. And you can also always start to get feedback in a way like this. And we also want to do that in the developer portal team as well. So this is a very simple way to get value and feedback. So that was the lesson learned. And right now I'm ready for questions. Thank you for that. A colleague from Nodia here, a similar place. So I have just a couple of short questions to you. First of all, do you plan to have one big instance of back state for all the teams? Or do you plan to create an instance of back state for each development team? So do you mean a server instance for? Yes. Okay, so yeah, for these 500 developers, they are in around 100 product teams. Right now we plan to make one instance. And then we know that maybe you can set up the catalog in one instance, your database in one instance. So I think we will skate it like that, but we will have one instance. So how you actually solve the problem of SODs and onboarding to the platform. So you have those 500 teams or 500 developers, each of them having like probably five or seven repositories in GitHub slash back a bit bucket. How do you register the information? Which team, who is the member of this team who should have access to which information that comes from each repository and which repository is it? So we have a few of our repos that are very critic and that we cannot show openly. So the first step is to get, let's say, repositories that can be open. That is like the first step. And then we need to find a way of solving that who can see what part of backstage. So we have not found a solution on that. We know that we have stuff we need to shut down for just yeah, someone to see, but we have not solved that yet. Sure. So the last one for me used something about automation highway. Do you actually use a scaffolder and templates from backstage or do you, how with the security of that? Because then it's everything goes directly from the browser front end to something. So like you let your user browser to create something in the bit bucket repositories. How does it work for you? So what we do, and it's still in pilot. So we take around, I think, 10 parameters and stuff like that. We call it with a JSON string to a GitHub action and that triggers the flow in the GitHub environment. And then he built the scaffolder of this microservice. So it's just the UI, you can say, the way you trigger it with the template. I've done exactly the same thing, but with the AWX underneath, not a GitHub action, but yeah, okay, yeah, that solves my question, answers my questions. Thank you. Thank you. Thank you for great questions. Thank you so much.