 Thank you. Thank you. Thank you. How is my voice? Is the sound okay back there? It's okay. Thanks Hey, thanks for coming. We are sorry if we tremble a bit that not only our first experience to speak in English abroad, but it's also very cool for us. So please be patient. I'm Gregorio Mello. Feel free to call me Greg. I'm from ThoughtWorks and I lead one of our business units that's dedicated to platforms and clouds in Latin America. Okay, my name is Marcos. I'm the CTO of B3 and I ask passion to you because I will talk a little in Portuguese and in Greg will translate to make more clear what I need to pass through, okay? But my English is not so good. So we want to tell you what our experience has been like introducing backstage to B3 and before we tell you how it has been so far, we'll tell you what B3 is about. Okay, so B3, a little in English, B3 is the Brazil Stock Exchange. We are the financial market main infrastructure from Brazil and we have a goal to make the, to contribute to better future for the market and the society of Brazil. That, what is it? And we have more than stock exchange. We have in our portfolio, finance portfolio, derivatives, fixed income. We provide some infrastructure for our clients and leasing vehicles and real estate services and our technology allows from brokers to offer diversification alternatives to the customers through Brazilians to make electronic transfers of money and lease of vehicles. So we have a huge importance in the Brazilian market and I will talk this history in Portuguese and Greg will translate because this is very interesting to know. We are over 100 years old company. As you can see in this picture, the negotiations back in time were done with a group of people. Together they would negotiate prices in a small room. As we were growing, we had to implement some manual technologies. By the way, we are a technology company. So in the room we had some people negotiating between them and one single person. We are wearing white and this person was writing the prices on the wall. And that was the price for the stock for the rest of the day. As we were growing, we were implementing more and newer technologies. The bottom left picture, you can see that's what we normally see as stock negotiations. A lot of people yelling, screaming, as the prices were matching. The brokers, the people negotiating, they would talk to people in red and white and they in turn would go to the computers and write the price. And the prices would be updated in the monitors. So we could then get a better variation in prices along the day. And now the bottom right picture is the way we negotiate now. No people, no yelling. And now everything is done electronically and under nanoseconds. So what does B3 want to be? Yeah, we transact over 21 million transactions a day. Around 1 trillion reais, Brazilian reais, like 1 dollar, 5 reais per month. So what does B3 want to be? B3 wants to be the best software development company in Brazil. So for that, in order to get there, yeah, so this journey, this slide is important because when we speak about backstage and implementing backstage, it's not only about backstage and its technology. But it's about how we improve the experience for the developers and their day-to-day work. So it's first step, people and communication. So it means that how do we prepare our people, our developers so they can become the best developers we have in Brazil. And together with that, how we communicate to the market and we retain and attract more professionals. The next step is the tech journey. So here we map the journeys in technologies, all of them that we have in the company. Because we are a 150-year-old company. There's a lot of unnecessary processes there. So we shrink a bunch of these unnecessary processes. So the next step, we try to combine the soft engineering pillar with the technology champions that we have in the organization. And together they define the best practices, DevOps, security, quality. And then we have standards and practices that we can create standards for us. And then we combine all this together and we go to the cloud and we start talking about cloud. We are transforming our applications and moving them to Microsoft Azure and OCI. Which will enable us to have more automation and have more fluidity in our software development processes. Then the next step is how we make better use of AI in our software development processes. We use AI for coding, for testing, an incident resolution, including how we can make a better usage of backstage. And at the end we get to backstage, which we now call as a product, Ultron. Why Ultron? Ultron, for those who don't know, is a villain from Marvel with some artificial intelligence. And we say that Ultron is the villain of our bureaucracy. He's going to destroy our bureaucracy. And with all of this together, it makes us reach our treasure. Which is productivity, quality and efficiency. Cool, thanks Marcus. And I'm from Thorworks. Thorworks is a software development consultancy. We combine strategy, design and technical excellence to create amazing solutions for our customers. We have a globally distributed presence. We have, other than Latin America, we have presence here in North America, in Europe, Asia and Oceania. And we have, you might have seen some of our content out there. We have Thorworks that have written over 100 books. We provide this artifact that we call TechRator every, twice a year. We give our inputs on recommendations on technologies we provided to the community. And the technology radar has been used as a guide for many of our clients and many companies that we see in the industry. And we got together. We actually met before B3. Marcus was in another client. We were talking about backstage there back then. It wasn't his responsibility then. It is now. And Marcus moved to B3. We started talking about developer experience. And now we understand that there is a lot of waste. There is a lot of friction. And there is a cost on this friction for developers. This is an industry average that we compute. And we normally use to talk to clients on the cost of developing software that we normally don't see at the cost of a bad developer experience. But you can take some of this math to your organizations if you are struggling to promote and get your voice to sticky holders like Marcus that can sign the check and say, hey, go ahead and start the project. It takes a lot of time for us to replace a professional when this professional lives in an organization. On average, we take 43 days to replace this person. It is a strong heat on our time to market. It can compromise the ability for teams to hit their marketing targets. And once you have a professional in the organization, this person might take too many weeks to get productive. We were talking about cases in Brazil that sometimes it takes a new professional and some of the organization three weeks to get a virtual machine so that it can start coding remotely. It is almost a monthly salary of a professional. And if you hire 400 people over a year, you'll pay almost 400 salaries. It's just waste. And it's not new. We've been talking about it for a long time. In 2019, the Accelerate folks provided a report saying that half of the people that responded for the survey for up to between 30% and 40% only was considered productive time. Not just coding, but making decisions, diagramming, coming up with documentation. The other 60% or 70% was considered waste. Sometimes waiting for other teams to respond to reply to your request, your tickets that you had to open, manual tests that could be automated. So we understood the issues that we were facing as an industry and within B3. We, B3 knew it wanted to deploy backstage. We had this partnership. We have this partnership with backstage. We have expertise. We got together and we started with Ultron. Before we joined, an API from its inception to production, it would take six months. It was part of this time was because development teams would need to open a ticket to another team so that they would check the code and verify whether the APIs were matching the specifications and standards. If you had a schema change of a database, as part of your pipeline, you had to open a ticket so that the database specialist could look at your database change and eventually make your comments validate and eventually apply. And when you decide to deploy software, even for the pre-production environments, you have to create releases. And normally it used to be a manual step. Create a branch, create a documentation, come up with the artifacts and move stuff around. It would take a long time. So then Ultron comes with the AI. Maybe not yet. But it comes as a feeling. So now you know why Ultron. I'm going to skip that. But the way we did that that we think is key for any backstage engagement, develop experience engagement, is to consider adopting a product mentality. You're not just adding a tool into a tool chain, your tool set. It's not just like you deploy Jenkins and now you do CI. No, there is a lot of culture changes and technical practices that you need to develop as a team. So you need to consider what is the plan for that? How are you going to measure the success of adopting this solution? How are you going to maintain it? Because the product is going to evolve. You need to upgrade that. There will be vulnerabilities that you need to patch. You need to see how it's going to scale for the amount of developers that you're going to have. How you monitor the health of it? Because if it is going to be your single point of contact for all developers, it has to scale for all of them. So you need to monitor and operate it. Consider it as a product. Consider it as a product. It's important to say that we have 2,000 professionals in IT department here with us. Cool. So 2,000 developers, 2,000 professionals overall, not just developers. So we decided that when we think of the success of this initiative, we would consider three pillars. Speed of the adoption. How fast are we adopting backstage? How good is it, not just the adoption and the product, the backstage itself and the plugins we're developing customized, but also the products at the end? And how is the community interacting with the solution? How frequently they use, how fast once they join the organization, how fast they get on board? And we chose some key metrics for us to measure the success for this application, for this product. Lead time for change. Cycle time. And time to first commit. We know that time to the 10th commit is getting more and more popular over time. But time to first commit is already good enough for us. So we decided to keep tracking that. When you look at quality, test coverage, we know that test coverage by itself doesn't bring quality per se, but the lack of it is a great indicator that you don't have. So we started considering coverage as a key indicator of quality that is being produced and the change failure rate, how frequently things fail when we are trying to deploy the software. The APIs and the products that are created and onboarded into backstage. And what's the network effect of the product, of Ultron? How many of the workflows are started? The pipelines that start and successfully finish, because you don't want developers that they start a pipeline. Pipeline fails somewhere and says, hey, this is a shitty product. We don't want to use it. You want them to see that there's a benefit for them there. And the growth of the number of repositories on GitHub that were created through backstage is a great indicator for us whether we are going on a successful path. If you see the proportion of new repositories being created through backstage, they hide the number, like when you compare to the outside of backstage, the better it is as an indicator of usage, of adoption. So these are the key metrics that we are using to consider whether we are being successful or not. By the way, these slides are already on the app, so you can download just as it is now. There's no updates from a few days ago. And then results, right? You don't need to see the details here, but just keep in mind that to your left, you see July, and these bars, they have different colors and they are grouped by week. So you see July, then you have four groups of different colors, and they are deployments per environment per week. And you can see that by September, the numbers have increased by a lot. So we're seeing that the frequency of deployment has more than doubled for many environments for the development environment that has skyrocketed. We are considering not using the development environment as a key indicator for us because we don't want to take an outlier and make the numbers look pretty for us in general, but we think it is a good indicator that people are using it, they're experimenting, they're seeing a lot of their products already making use of the workflows and pipelines that we have developed. So now the APIs don't take six months to go to production. Now they take a single month. Is it a lot? It is still a lot. The technology will enable us to deploy in under a day. But from six months to one month, we have reduced five-sixths of the total time to get to production. Now the APIs are validated automatically. So the teams, they are using a tool that validates API specs and the pipeline will break if the API is not up to the standards. So there is no ticket open, there is no time waiting for another team to provide feedback on my API specs. So that's one example of many of the steps that would manually happen after the team has considered dev complete, like security and vulnerability scanning. Now some of these steps we have moved a lot earlier into the pipelines. Now the database specialists, they don't need to look at the JIRA tickets anymore. They look at the pull requests and they cannot prove it. It's a lot faster for them, the whole process, because they have a better visualization. Not only they are seeing the schema change, but they are seeing the context around the change for that schema. And the tooling is much faster for us to just approve and they take their time to do something more strategic later on. And release is, well, this is something easy to automate, but maybe the points were not connected, the teams were not connected well enough to understand that there are multiple steps that were responsibilities of many different teams and now everything is combined into a single script that does the whole thing for them. In order to get there, you want to add anything? No. In order to get there, this was our journey, has been our journey. We started back in January with a few weeks to understand what is the context of B3, wait for the virtual machines to come up to be delivered to us, agree on key indicators, understand and learn what are the strategic goals for the organizations. Then we came up with a foundation, what's the technical solution, the deployments, the basic plugins, and then get some pilot teams to work with us, even deciding what pilot teams was already an important conversation for us that took a while because you don't want a team that is not critical at all for the organization to onboard because for the rest of the organization, it's like, it worked for them, but they don't have the problems we have. So we decided to pick a team that's like, there is a high criticality for them. And we are onboarding teams. We wish we were a lot further but I think we have accomplished a lot of things already together. We are at the moment that we are bringing some new content into the deployment of backstage that we have, bringing new areas, bringing new functionalities and enabling more and more teams. We still have the difficult conversations on what's the value of what we are doing because it's something very technical, but combined with the product mentality, we can get halfway through both sides and understand from the leadership, from the development team, they understand the value of what we are doing. Yeah, so hopefully this goes a long way. There's a bunch of good things that we've seen here today that we can bring back to B3 and see how 2024 is going to be for us. Thank you. If you want to talk more about our experience with backstage, we ThoughtWorks has a booth out there just outside. We'd be happy to talk to you. Thank you. Questions? Awesome, Marcus. We got you. Thank you. Any questions? No questions? Good. That's one. Hi, this is Jayant. So how did you integrate this U-tron with the existing process what you already have or existing tool which you already have in your company? Yeah, so some of the integrations, they were easy to do. Some tools that they already had at B3, we just bring plugins, connect, configure, they were good to go. Some others we had to come up with our solution. The API spec validation, there are two links for that, but we had to adapt our pipelines to check and run on that. And some stuff were really new, like the schema changes for database specialists. This is something very new for them. It was really good for us that they understood the importance of their time and how much we could make their work quicker and safer by bringing them into it a lot earlier in the process and consider using pull requests and approvals. So it was not just tooling change, but also changing mentality that worked. Many of the technologies that B3 already had, we could just integrate with existing plugins. But some others we had to come up with our solutions and do some sort of integration, like scripts, executing scripts. Thank you. Yeah, thank you. Thanks, folks. Any other questions? Cool. Cool. Thanks, Marcos. Thank you. Thank you, I appreciate it.