 I want to just kick off and share a couple of thoughts about why we're here and what we're doing. And as a former IT leader, I would come to these events not unlike yourselves. And I was here usually for one of a couple of reasons. Usually it wasn't for a sales pitch. It wasn't for someone to do a demo. I can usually find those pretty easily. And Lord knows if I talk to a sales rep, I can get the pitch. I came to these events because I wanted to meet my peers. Because I wanted to listen and learn from them about what they were doing. And I don't know what motivated you to come, but I hope it is these objectives, because that's what we've tried to put together for a program of sorts to talk about. Now, does anybody have other objectives here tonight? If you're looking for jobs, I do have a web page up of, we are hiring, I'll be glad to go through the open positions. We've got a number of them that are open. So we can talk about that. We are growing rapidly. So hopefully from an agenda perspective, this meets your needs. If not, let me know, because we can tailor this on the fly and make it flexible to meet your needs. And feel free to grab a seat. Welcome. A lot of times when I talk to people about the trends and the things they're experiencing, they end up talking about digital transformation. This is a cliche topic and it's one that a lot of enterprises and organizations are wrestling with. And it's one that when I've looked at the data and I've talked to people, I come away with some things that are sometimes disconcerting, especially when as many people are doing it. And when you look at it, digital transformation is not something that's just happening here. It's not something that's just happening with a few unicorn companies. It's happening in enterprises everywhere from old style insurance companies to the government with three letter agencies that we can't talk about. It's happening in countries around the world. And when we look at this, the World Economic Forum produced and they worked with Accenture. They produced a really awesome, got links to it here if you want to pull it up. But they pulled up and they produced this long detailed study of what does it mean when people say digital transformation. And it ultimately is about changing the business. It's about changing the way businesses interact with their customers. It's changing the relationships. It's about taking advantage of a networked economy, a digital economy to be more efficient, to innovate on business models. I think about how radically different it is when I come here and I get in a car driven by someone I don't know that a little app on my phone somehow is enabled. How many years ago was that a foreign idea? It wasn't very long that it was, you'd automatically get a cab or rent a car. Today, I get into a car shared by a friend of mine who I've just met moments ago because of this really cool mobile app. It's radical if you think about it. But that little mobile app is disrupting a massive industry. I've gone off and stayed at friends' houses. I didn't know them before I went to their house. But a little app allowed me to get to know them and then they shared their house with me. Airbnb doesn't own any land or any real estate, but they account for millions of nightly stays. Amazing transformation. It's a huge thing that's happening. And it's hard though. It's not an easy thing for Hilton or for American Express or for Goldman Sachs or for any traditional companies to go through this transformation. They have all sorts of legacy challenges that hold them back. And as a result, a lot of them are somewhat behind the power curve in getting started because as they go down this curve, they're trying to get started to move faster. They're trying to understand how can they get from where they are to being more innovative faster than the competition. Mark Andreessen wrote this series of tweets about cycle time compression and how that's a key between winning and losing. There's a series of tweets that he basically described how mobile apps and software as a service allow for faster innovation and therefore will win over time. He made the comparison that people don't buy cars anymore. It's not about buying cars, it's more about the phone. Phones iterate faster. The car is a feature for the phone. Think about that. How many times have you thought about, will my phone be compatible with the car I'm shopping for? It's a radical change and it's one that 10 years ago, I don't think any of us would have thought that was true. Mark, he saw that where it was coming and I think his quote is spot on. And as we try to go faster, this is where it gets to be hard. And looking at the data, Forester's written a report on this where Ted Shadler wrote a report talking about how hard it is to get started with digital transformation. IDC did this study about a year ago about where people were at. And I did the math. My addition to this was the math. 45% of organizations haven't started. If this is as big a deal as we all talk about it, as common as everybody talks about, why is it that 45% have not even started to figure this out? And the reason, and I think the reason is really clear. There are lots of challenges. There are lots of challenges from the different teams, the processes, the tools they're using, the silos in the organization, the different goals and objectives. And then think about it, most of these organizations have goals or objectives like we have to, I mean, how many of you are in an organization where the goal is to go faster? How about more efficient? Maybe more secure, lower risk? You're probably all three, right? These goals and objectives are out there and you're all trying to drive towards doing that, but you all run in. Okay, cool. You're putting on the helmet. I was getting worried about where I was going with this. Yeah, maybe you want me to speed up. But the reality is these challenges are out there. And last year, or earlier this year, we started talking to Chris about looking at some of the challenges that impact people and impact organizations with accelerating software delivery so that they can meet the needs of the market. And so with that, I want to turn it over to Chris and I'll double click the slides to PowerPoint so you can share what you've learned and what you've discovered. All right, thanks very much. Yeah, so thank you very much, John. Thanks, get lab for having me here today. So let's see where this goes right into. I'll just give you a manager tool chain before it manages you. So I'm Chris Kondo, I'm from Forrester Research from the East Coast. We work out of the Cambridge office there right next to Alewife. If anyone's familiar with that. I came to be an analyst about three years ago. Previously, I had worked at companies like Microsoft. And then a long, long time ago, a company called Digital Equipment Corporation. But at Microsoft is really, really I started learning about sole idea of automation, product focus. The idea that being focused on the customer, being focused on delivering an experience that is delightful to your user is very important when I was developing for them from 99 to 2014. We were in their New England Research and Development Office in Cambridge, right next to MIT. And we were in an offshoot of their Office 365 team. And so back then we had to write all our own DevOps tools because Microsoft had a no open source policy. And we did our own automation. We had to write our own deployment scripts, Azure didn't have any at the time. And so it was really kind of like a bootstrap effort. Fast forward to today and you see that there's just the plethora of tools available, but not everybody's using them. So let's just jump right into it. So the agenda is teamwork and tools, paramount to accelerating software delivery. We'll talk about the custom research that we did on behalf of GitLab and the survey that we ran. And then some recommendations about choosing a modern single tool chain can be beneficial. Interesting thing that kind of got me down this path is when my very first research project was actually part of my interview process at Forester. So when you want to become an analyst at Forester, they actually make you write a research paper. And then they actually make you deliver it in front of a bunch of analysts. And so throughout this whole process, which took several months, my wife kept saying, they're trying to steal your ideas, don't take that job, stop doing that. And like, if you walk in there, they don't offer you this much money, you're walking right out the door. But if you know where Lil Mass is, that's where she's from and you'll understand what I'm talking about. But anyways, so I went in and we did this whole report on DevOps transformation in the cloud. And then I interviewed a lot of teams and we spent 10% of our resources just managing our tool chain. We spend this much time managing our on-premise tool chain. If somebody could offer something, a cloud-hosted tool or a solution that can alleviate that pain from us, we'd be willing to go for it. So it kind of got me down this whole agenda. So a little bit more about Forrest to work with businesses and technology leaders to develop customer accept strategies that drive growth. So the whole customer obsession, focus on the customer. It's been something that Forrest has been behind for quite a long time. So let's talk about it, right? So teamwork and tools are paramount to accelerating software delivery. This is sort of like the metaphor for the modern DevOps team. Everyone's got a different role, but everyone has to work together. And they have to work fast, right? You only have so much time to get it right. You don't have so much time to keep the release moving along. And there's a lot of pressure on these teams, right? So John just talked about the digital transformation. He referenced Ted Shadler's report. Ted's a vice president, principal analyst at Forrester. And he wrote this report, The Sorry State of Digital Transformation, where he talked about the fact that all of these companies are under so much pressure and some of them actually think they're done. And he's like, the most ridiculous thing is that people think they're done with digital transformation. The idea is once you've embraced it, you realize it's just not gonna end. It's just gonna keep going on. Along with that, Forrester has a lot of other research that backs up the idea that customer experience is a real driver for revenue, a real driver for keeping your business up and running, and a real driver for winners and losers in this market. And so when you put those two things together, it gets back to that Mark Andresen quote, right? The idea that cycle time compression is really putting a lot of pressure on our feature teams. So, we took a look at some research and we said, what is the current release frequency? And we said, all right, well in 2000, we actually have newer data from 2019 that shows the same number, but for the last five years previous, this is pretty much the graph for release cycles for the people that we interview. So we sent out a survey to about, it's a little bit more than this now, but it's about 3,000 developers. And these are all customers of Forrester, so they're like insurance companies, banks, all these companies that are struggling with their digital transformation efforts. And by and large, most of them are stuck in this middle average once per quarter release. Only very few, five and 6% are doing a release a day or multiple times a day. And so the vast majority of the area under the curve is between a month and a year. So not really agile, right? If you have a bug fix, you're not getting out the door anytime soon. If you have to get something in front of a customer you want to try it out, it ain't happening anytime soon. And so by and large, the majority of customers that we deal with are struggling with this digital transformation effort. Now you can go look at like other research from like Dora or even the state of supply chain. They all have, I would say sort of like interviewing or surveying companies maybe at a broader sense. Maybe there's more smaller, more cloud native companies that are able to kind of push these numbers closer to these more daily release or more weekly release. But by and large, the people that we're interviewing are stuck and they're looking for help. Here's another thing, right? So this sort of calls up the 2018, 2019 data where it's showing some improvement finally in the customer base that we have. So apparently our subscriptions are doing something for these people and finally getting unstuck. But they're still not doing quite well. They're still kind of still over here. And then a lot of teams say they're more agile, but the reality is that it went from we think we're kind of agile too. Yeah, we're pretty agile over here. Most of them are saying that they're using agile, but a lot of companies still are stuck in this sort of fake agile world where even though they want to do agile, even though they've got all these scrum teams and they've got all these meetings set up, it's really just a big waterfall project chopped up into 10 easy projects, right? And the requirements never change, the backlog never changes. It only grows, it never actually shrinks. And so the whole idea of iterate, learn, and then readjust, just doesn't happen on these teams. What usually happens is they start with their first release, somebody realizes there's five more things that they want to add, but they don't care about the things that they might want to take off. And the workload just keeps growing. And that's how you get stuck with these people that aren't very agile and are releasing once a quarter or once every six months because they just don't understand what it means to really be agile, which is the ability to not only add things in respond to customer's demand, but to remove them based on lower priorities, right? To be responsive to what's going on and what's important. Here's another interesting one. I'm not sure if I even believe this data to be quite honest. Because it shows continuous integration that only 31% of the folks that we surveyed, 3,294 developers, claim to be using continuous integration. It's what if the data is, and maybe it's the way the question is asked, but it just seems absolutely ludicrous to me, right? And so when you look at this data, it just shows that there's a complete lack of modern techniques or modern tools and technologies being used. And maybe that is the state of things, right? When you actually start looking at the data, maybe the reality is that not everyone's actually using continuous integration on a daily basis. Maybe they're doing a batch build job at night, or maybe they're not doing it at all. And you have to feel bad for people in that situation. So teams tell forestry too many tools create friction, right? So this is one of the problems that they have, right? 71% agree that governance and end-to-end visibility of software delivery are major challenges, right? So one of the problems that folks talk about is that, like, well, we work in a regulated industry, and almost every step of the development process, we have to do an audit, or we have to do a manual check, we have to do a manual review. Somebody has to come in and look at stuff and approve things and sign things off. Each tool chain maintains security differently, complicating access control, right? So someone says, hey, I want to create a brand new tool. I want to create a new build pipeline. I want to get started on my project. But I'm waiting because the permissions have to be set up, and the tool chain is really complex. And so the operations person or whoever's managing the tool chain has to manually go and configure all these things. And it's really complicated, or I can't get access when I need it, many estimate managing tool chains consumes 10% of their development. That's what I said at the very beginning that I first learned about this whole problem where teams are complaining that, like, hey, I feel like I got 100 developers, and 10 of them are always managing the tool chain. I'd gladly buy something. I'd gladly rent something in the cloud if it meant I didn't have to do that, yes. This particular case was 10% of the development team and developers, right? So these are like they don't have a separate team managing their tools. They're actually managing it themselves. Does that answer your question? OK. And then 67% agree that handoffs between different teams using different tools slows down delivery, right? So this is another problem where a lot of different teams have their own, roll your own tool chain, or each team has a different tool chain. They've all incorporated open source tools. They've all built their own different bespoke tool chains. And now you're trying to transfer work from one place to another, and it's not going very well. So this is the respondents demographics. 100% in an IT role, respondent level manager, team architect, director, vice president, C-level executive. So a good meaty section of people that are actually doing the work, and then a nice smattering of people who want to make sure that these guys are doing their job. By and large, Western Europe, US, and I guess that counts as US as well, but I wonder how much DevOps is going on up there. Maybe a lot, who knows? Job function, IT and infrastructure operations. By and large, for most traditional companies, these are the people that own the tools and operate the tools. Those teams are trying to become more agile. Application development, and then it kind of goes down the road from there. And then the company size, by and large, 51%, 5,000 to 19,000. Oh no, I'm sorry, 1,000 to almost 5,000 employees. And then 20,000 or more is the smallest one. So a pretty good distribution, pretty good number. And it's 252 professionals. I don't know why that was in there. So teams have too many tool chains and too many tools to manage. So this is one of the interesting things that came out of the survey data. So to the best of your knowledge, how many different software delivery tool chains does your software organization maintain? So the biggest number was two tool chains, but the second biggest number was three to five tool chains. So one tool chain, maybe that's too unreasonable, because if you're dealing with multiple technology stacks, like maybe your team's big enough that you have a Java stack, you have a Microsoft stack, you have a mobile, like maybe Android, and you might want Android, and you might want native iOS. So you might want all those things. Maybe you need three or four different tool chains to manage it, but that just puts a lot of burden on these teams. And then you get up here in some of these other folks 21 tool chains, 11 to 20, 6 to 10 tool chains. This is just ridiculous. So those are obviously from the larger companies, but this means that the person who answered said they are managing this many tool chains. So it doesn't mean that just because a company has 20,000 employees, doesn't mean that, and they could possibly support 20 different tool chains, doesn't mean they're dispersed all over the place. It could mean that this team is actually stuck managing them for teams that are local to where they are. And then to the best of your knowledge, how many different tools comprise your current software delivery tool chain? The majority is 1 to 5, so a pretty decent backbone, probably some CI, CD, an agile planning tool, maybe a release tool, maybe a static analysis tool. 6 to 10 tools, though, is 30%. So now you're getting a little bit more complexity, right? I'm not sure if we actually dug in on the complexity. Maybe it's in the next slide. And then 11 to 15, 16 to 20. So if you take this box right here and you multiply that out, that's hundreds of different tools that are being managed by the same team. That means hundreds of different licenses, hundreds of different updates, hundreds of different interconnections. It's really quite a mess. And it's quite a burden, right? So how did teams end up with so many tools? Drivers to expand a number of tools within tool chain, organic growth and maturity of the CI, CD automation market, right? So they started with a basic tool chain and then, hey, this looks good. Let's add that on. And that looks good. Let's add that on. And then, of course, we need some static code analysis. Of course, we need SonarCube. Of course, we need all these things. But then that particular tool chain lives in one team. Then the other team is doing something very similar. Only they're downloading a different version of it, or maybe a later version of it. And then before you know it, even though the two tool chains might resemble each other on paper, in actual practice, they're two completely different snowflake environments because they're different versions of Jenkins. They're different versions of Sonar. They're different versions of agile planning tools. And the two things have incompatible APIs. And it becomes a real big mess, right? Greater awareness of test automation, security scanning, cloud-native technology, such as Docker, right? So people see these goodies and they want to add them. Why not, right? New is good. More is better. Drivers for the expansion for the number of tool chains, team autonomy in selecting their own tool chain. So this idea that teams should be able to select the tools that are best fit for their team, it's a nice concept. But what ends up happening is you end up getting a sprawl, right? And then you get clients that call me and say, we've got 10 different tool chains and I'm an enterprise architect and I work for a big bank and my job is now to wrangle all these things together and come up with a common set of tools. What am I supposed to do? Because all the developers are going to commit mutiny on me and I don't want to piss anybody off. And it's like, oh, you've got to manage that. We've got to figure out a way to keep everybody on board, give them what they want, but maybe standardize somehow, right? Diverse technology stacks lead to different tool chains. So we talked about that, right? Mobile apps versus web server. And then these enterprise silos. That's probably the evilest one, right? Where you've got these different business divisions or maybe you've got different divisions within the same business unit and one particular set of developers and managers and designers worries about one aspect of your application. Maybe you're an insurance company and they worry about how they're acquiring new customers whereas the other one worries about how are we actually processing claims, how are we doing all the forms? And they're in just completely separate worlds even though those two things are supposed to work together somehow. So that's how teams kind of end up with so many tools and so many different tools and so many different tool chains. Tool chain responsibility gets spread out. Which teams are ultimately responsible for maintaining your tool chain? That's the question we asked. And this is with a burden. 40% is on the development team. So when I talked about 10% people, that was an anecdotal number. I listen to clients, I interview people, they tell me, I don't know, it's roughly 10%. I don't have a survey that exactly says that. But then this actually came back in this survey. It was really, I guess it was kind of rewarding. I feel like sometimes I'm a little bit like Colombo where I just sort of stumble around and listen to people and come to the conclusion that sometimes doesn't seem like it has any basis but it turns out I was right. So development teams are really overburdened with these tools. Why do you think that is though? Well, typically it's a developer who understands how to automate things. A lot of companies have test teams but those testers don't necessarily understand automation. So they might have an IT team who understands how to script, who understands how to do some automation but not in a development sense. Not necessarily traditionally, not every traditional INO person understands how to code and instantiate infrastructure and use Java or YAML or other types of languages that can instantiate, understand how to compile and build code. So development teams, a lot of the burden falls on them mainly because they want to go faster. They're the ones that are pushing the rest of the team to go faster. So they take on this burden saying, well, we've got to do this. This is how we're going to manage it. Release teams and DevOps teams. That's a growing sort of like faction within teams where they say, well, this particular skill set is kind of unique. It's different from just software development. It combines software development and infrastructure. There are people that just would naturally gravitate towards that on both those teams. Let's put them together and maybe that'll be a good way to be a self-service DevOps team that can create self-service tool chains, that can create managing and help manage that burden for our development team, take it off the development team, kind of take it off the operations team and put it on this middle tier team. Well, that works pretty well but those folks are still, if they're managing 20 tool chains and each of those tool chains has five tools, that's five times 20, that's a hundred, right? That's a lot of stuff to manage still, even for a team that's dedicated to that. Not bad considering I've jet lag. And then it goes down to software tools and operations teams. So by and large, 70% is on these teams right here. So that's a lot. It's a lot of burden and what it really is is sort of like the cost and burden of managing these tools means that these folks who are working on this aren't creating business value. They aren't helping with the customer experience. They're not helping design the next architecture. They're not developing necessarily the next feature set. They're worried about just the plumbing and the fixtures that keep the whole business running. Now, when I went to a DevOps meetup at Wayfair, they have a really cool office, not quite like this but they're in Copy Square in Boston. Copy Square is a really expensive mall in the middle of Boston and in the tower are all these developers working there with this funky furniture that nobody bought, fuzzy lights and crazy chairs and stuff and all the street signs that point to different furniture departments or actually different development teams. And so they did a really cool DevOps meetup and they talked about the fact that they consider DevOps sort of like a differentiator. So they could sort of embrace it. Like, we're in this to win it and we're up against Amazon, we're up against Walmart and we're the little guy. Even though there's a thousand developers here, even though we make billions of dollars, we're the really, really tiny online retailer compared to these other guys. So they consider these things just the way they have to operate. It's a differentiator, the ability to release and own their own tool chain and develop fast is something that they realize is what's gonna make them a differentiator. So that's why a lot of these teams take these burdens on. Integrating those tools becomes a labor-intensive challenge, right? So what's the biggest problem? Our tool chain is integrated with a combination of plugins and scripts, right? So 40% of the folks who have one or two tool chains tell you that and three plus tool chains say that. So if you're installing plugins onto your CI server and then that becomes a unique installation, now that becomes a different server than the other server that has different plugins. They all become snowflakes. They all become independent points of failure, independent points of management, independent points of pain on the ass. Our tool chain is integrated via well-defined APIs. Well, that would be nice, right? So 18% have one or two tool chains and three plus tool chains is 30%. So that's good, right? So well-defined APIs, that helps you loosely couple of architectures but still a bit of work to maintain when you need to update those APIs. Our tool chain is integrated manually via hard-coded custom integration between tools. So these folks have a rather tenuous relationship with their tool chain, 23% and 17%. And then we have an out-of-the-blocks software delivery tool chain that is integrated into N. These are the GitLab customers, apparently. And then our tool chain is not integrated. So, oops, that happens, right? Or they're using another brand that we're not gonna talk about. Anyways, so then there's also this skill gap, right? So these are the other problems that come up, right? So skill gap emerges. Teams struggle to maintain a vast number of different tools and their integrations, right? So insufficient skills, expertise and resources to integrate discrete tools, right? 46%. So what does that mean, right? So it kind of goes back to the whole thing. The reason my developers are in charge of the tool chain is because they're the folks that typically understand how to integrate tools together. They're typically the people who understand APIs, who understand how these things all have to plug together and they understand how they want their software built. When there's an insufficient skill set or body of resources to pull from, you end up pulling from your development team. Insufficient skills, expertise or resources to maintain the tools, so that's another big problem. So it's one thing to build the tool chain. It's another thing to maintain the tool chain. And then the behavior change needed to maximize these tools, lack of executive leadership, and NA, we don't face any challenges in this area at all. They're just happy, having a great time. So that's 7%. So 7% of all the respondents say they're just happy. And so by and large, people are dealing with resource issues, they're dealing with skill set issues, they're dealing with the fact that it's just hard to maintain all these tools. So IT professionals struggle to maintain secure tool chains. Here's another one, it's kind of interesting that they came out of it. Which of the following process challenges does your team currently face with its tool chain? And difficulty ensuring security across the tool chain was 45%. So now, I think if we asked this question a few years ago, probably security wouldn't have been a big issue, right? But when you install the open source version of Jenkins and everyone has root access to your server, and that's the way that all these tools actually integrate together is they have an admin, hard-coded credential. Then when it's just amongst us friendly developers, no big deal, right? But then once you start expanding this tool chain and more and more actors start being part of it, then you realize, hey, guess what? We've got basically, there's a backend network of all these development tool chains that all have hard-coded credentials that anybody has access to and could access other assets within the company. Now people suddenly realize, after all the breaches that have gone on, like a target and other high-profile company breaches that security is a big issue. And then they started looking at these software delivery tool chains and realized every one of these tools has to be configured for security differently. Every one of them has different ways of managing RBAC, different ways of managing SSO, different ways of how they communicate with the next tool up and the next tool down. And that's when the concern really started hitting home that this is a really big problem, right? It's been sort of like something that's been sort of swept under the rug for a long time, but not anymore, right? Lack of visibility is the other one, right? So once you've automated a lot of these things, it sort of goes into a black box and comes out the other side, but you're like, all right, well, do I know how fast we're actually working now? Do I, where's all that data? Is it something I can look at? Is it something I can comb through? Is there any way I can understand what our cycle time is, our velocity? And so what's happening is all these tools are sort of creating like an opaque veneer around the entire software delivery process. And so these particular folks who answer this are like, we want greater visibility of what's going on, how the artifacts are being passed along, what's the handoff look like? Who's doing what? And some ability to get an audit because we're a bank or we're an insurance company or we're in the government and we need to understand who worked on what when, when did that work item get picked up? How long did it take for the work item to get processed? Things like that that we talked about at Forrester called Value Stream Management, or the whole idea of like lean software development where you try to really understand the full process from end to end and get visibility into that process. So those are two things that really came out of this survey that are really quite interesting that I'm not sure that we expected. I think we just expected it to be a maintenance nightmare and that was the end of it, but this is another problem. Poor usability across the tool chain is the third one. The idea that these tools are just difficult and they're kind of bearish to work with. That one tool might be really elegant, maybe you're using a really great agile planning tool like JIRA and you think it's great, but then you go on to your CI tool and like everything just falls apart. And so the idea that you get poor visibility, poor usability across the tool chain, I look at that as spottiness. Some tools are good, some tools are bad. And then back to these other things, insufficient skills, insufficient resources, difficulty implementing charge back models, and then we don't face any challenges, 4%. What's that? Charge back? I'm pretty sure that is just, yeah, actually, I think it's a financial idea, the idea that you wanna actually figure out how to pay for this particular thing. So you've asked an IT team to support you and now they want money from your development team for doing some work for you. And it's like, well, how do you manage all that? How do you work between cost center to the cost center? And so what can happen sometimes is you say, well, on the development team, I need you to fix this and they're like, well, we're not gonna do it unless you fund us. I'm like, well, we're not gonna fund you so we'll have a developer do it because we're sick of dealing with all the red tape inside our internal bureaucracy system. That's what I believe it means, that's my interpretation anyways. Yes? Okay, the audience agrees. IT professionals see benefits from an out-of-the-box tool chain. So which of the following benefits do you anticipate realizing or have you realized from a new out-of-the-box tool chain management system? And so when we thought it was gonna be like, oh, we thought we were gonna see, well, it's a lot easier to maintain my tools now. But reason to the top is improved quality. Improved quality of their products. Why? Maybe it's because they're able to focus on developing their products now versus developing their tool chain, right? Maybe it's because the tool chain now kind of works for them rather than them working for the tool chain and they're actually able to think about and get visibility into it and understand what's going on inside all the different software developing process. Improved security, improved developer productivity. So these three right here are all the sort of like sore points that people had earlier got solved by this idea of, this is what you can get from a complete connected and integrated tool chain. That's pretty stunning. It's sort of like a nice reflection on the pain that we saw earlier. We also saw increased revenue, increased compliance, improved time to value, and then all the way down, lower costs and cost savings of managing it, faster onboarding times, improved visibility, improved developer job satisfaction, increased co-chairing, and we have received no benefits, 5%. So this 5% is just, they just live on the island. I'm not quite sure what they're doing. They're always talking about how they're happy, but then they have no benefit. So I'm not quite sure what that's all about. But so it's really quite interesting and it's quite a way that interesting how the data kind of sort of handshakes it together. And so the ability to target any environment cloud was a top business benefit. This is something that, actually I'm gonna go back to my day of going to the Wayfair DevOps meetup. They actually hosted it in their building and they gave out DevOps t-shirts and free sandwiches. It was really great. It was one of the best moments of my life, I guess. Anyways, they talked about the fact that they host on Amazon, Google and Azure simultaneously. And to them, multi-cloud was extremely important because first of all, they compete with Amazon but they host on Amazon so they have this kind of weird relationship with them. And then what they don't wanna do is be beholden to one particular cloud vendor or have to suffer an outage and not be able to have a backup plan. So they have a dynamic system where they go multi-cloud. That was really the first time I heard about someone really truly buying in on a multi-cloud. Since that time, Forrester has done a lot of research not just this particular survey. Analysts like Lauren Nelson, they all talk about the fact that customers are asking for multi-cloud solutions. They don't wanna be locked into one particular vendor. They want the ability to shift workloads or do workload-appropriate things for each cloud. Each cloud has different strengths and weaknesses or they wanna be able to shift storage back and forth when they want it. Well, they wanna have some things on-premise, some things in the cloud. So the whole idea that they don't wanna be locked in, they wanna have the ability to move things around. And so we're seeing that come through in this particular survey as well. The ability to deploy any target environment or cloud. 25%, that was the biggest number. The next one is simplified maintenance. Even though people want simplified maintenance, they want to make the tool chain easier, a big business driver is the ability to be multi-cloud. And that was another kind of surprise out of this whole survey, which was a pleasant surprise. 77% agree that organizations is moving to the cloud and want to avoid cloud blocking. Our future is multi-cloud. So it's really quite interesting and kind of bucks the trend of what some cloud vendors will tell you. So why is multi-cloud important? Avoid getting locked in. I guess I already talked about it. Deploy and host your applications or components to any of the providers that best match your functional needs. I mean, maybe you don't wanna host on one of the big public clouds. Maybe you wanna host on Oracle, right? And so you should have the choice. And then public cloud outages are rare, but they happen. S3 outage, there was an Azure DevOps outage. So, do you wanna be relied on one particular vendor? If you're in the government or you're in a financial technology services, you're trading thousands of transactions a minute and you're gonna invest in a cloud vendor, are you gonna put all your eggs in one basket or are you gonna have a plan B and a plan C, right? You just can't afford to be down when being down means maybe millions or billions of dollars of lost revenue. So, and there's also the whole idea of like governance where many firms have to have multiple sources for their vendors. So, there's a lot of different reasons why multi-cloud is I think going to become a more important aspect for larger enterprises. For smaller companies, I think it's not a big deal, right? It's like, hey, we're, you know, 100 people were a startup or we're beyond startup and we're getting bigger. You know, maybe that's not such a big deal, right? Maybe it's okay just to invest in one particular vendor, but if you're a large firm, if you have a lot of different technology stacks, you know, you wanna be able to have the choice. So, what can teams do? Reduce the diversity of tool chains within the enterprise. Duplicated functionality is wasteful, right? So, if you have multiple tools, all doing the same thing and they're all working on the same stack, why bother, right? Is it just because it's some particular person's pet project or is it actually doing any value, right? So, the whole idea is measure the value. You know, does it support open standards? Is it, does the tool chain simplify security and risk compliance? Will the tool chain allow you to deploy where it's most advantageous? Will the tool chain be easy to maintain, right? So, you look at these factors and you try to think about, you know, it's great that I'm giving my developers choice, but you know, we're all working for a business to make money. And sometimes that means a little bit of standardization. Sometimes that means a little bit of, you know, trying to bring things to sort of like an even keel so that different developers can jump from different teams and could have mobility. You could have the ability to, you know, you can get all the tools you want, but you can just do them in kind of a standard way. So, in summary, software delivery automation is a business differentiator. You know, all the companies that are going through digital transformation that are suddenly realizing that, oh, guess what? You know, we are actually, our product is gonna be a software platform which we deliver our services on. It's not that, you know, the software's just there to be part of the claims management process or it's not just there, you know, to be our website for the banking. It actually is our product now. And we just happen to sell financial services on it. We happen to sell banking on it or the government happens to do your taxes on it. Government, I mean, custom best of breed tool chains provide differentiation, but they come at a cost and that cost gets multiplied when there are multiple tool chains, right? So, not every team wants to take on that cost. Not every team can afford to take on that cost. And if you have a choice and you don't need to take on the cost, why would you want to? And then choosing a modern single tool chain solution can be beneficial. It's less maintenance. Look for ones that have open standards that enable extensibility so that you, you know, you want, one of the biggest reasons why tool chains are the past, like if you look at, you know, IBM's Rational, you know, the original Microsoft TFS ClearCase, they were all closed systems, right? If you wanted to add a scanning tool, if you wanted to do something, you had to kind of do it on your own. You want something that's got APIs that allows loosely coupled architectures that allows you to plug in tools as you need to. So you can sort of have this backbone of a tool chain and then hang the other pieces off of it so you don't have to worry about managing all of it. Simpler security model, less time spent on automation, more time spent delivering value, and then freedom to host and deploy where you need, right? So that's the really big benefit that we see that we feel that the survey and the state is pointing to. So Q&A, thanks very much. Q&A for you. Oh, all right. So if you have questions, I have a mic, I'll walk around and do the MC thing. Anybody have questions for Chris? Someone must have a question. No, none. Yeah, none. Nope, here we go. If it works. But you had a slide where you were talking about how it was what people wanted to gain and also what they did, but that's actually two very different things. That's true. You mean like what we hope to get out of it and what we did get out of it? You mean that slide? Yeah. And so I just, like it made me curious, of the people who hoped to get it, what percentage of, how do they feel? I don't know if we cut the data that way or not. We probably have to go back and look at it. If they got it, yeah. Yeah, I don't know the numbers off the top of my head. We cut the data about 50 different ways, trying to, and we can look at, I don't know it off the top of my head though, do you? I thought we had that. I thought we had a view of sort of, I want to look at the report. Okay. We might. We'll have to look. Yep. Other questions? That is a good question though. Thank you. Of the tool chains, did you break down what tool chains were being used? So when you talk about maybe 10, 15 tool chains, were a majority of them, maybe five or six of them within security or specific to CI or deployment or anything like that? No, they were pretty generic questions. The survey itself was, it was pretty compact is what I would say. And so the information, there is a lot of information in the cracks that we didn't quite capture. Okay, no worries. Thank you. Yeah, it didn't get into naming specific tools. Yeah, the granularity that you're looking for, we didn't actually capture. And again, we were trying to talk to, the objective we were trying to talk to were managers and executives. And last time I asked to CI what tools they use, PowerPoint is usually the first one they tell me about. Yeah. You had, it was a middle of the lane sort of benefit that people said that they got, which was a shorter time to ramp. Okay. New people. Yes. Did you dig into that to find out if the reason that there was a shorter time to ramp was because they were surrounded by people that already knew the tool chain or because it was an industry standard chain so they could go look on the, they could go Google how to do things versus custom tools where, like, did you get any sense as to why using a single tool chain like this actually led to a faster ramp time versus whatever people were doing before? So not from this survey, but from my own client inquiry where customers, you know, so the way Forrester works is we publish research and then clients of Forrester, you know, click on your report and say, I want to talk to you and ask you a question. So these questions come up and it's like, you know, we're trying to, you know, figure out like, you know, standardize around a tool chain and then we get into the conversation about why and this comes up a lot where they say, well, because we have so many different tools, if a developer who understands this particular tool and how it works and there's rules and there's business rules encoded in it. And what ends up happening is sort of like, the tool has a surface that's like the tip of the iceberg and then underneath it, there's all these business rules. And so then they go to another team and it's like, oh, well your business rules are a lot different than the business rules I used to use. And, you know, we didn't do it this way. We didn't, you know, use that particular scanning tool, use that particular scanning tool. So what I would say is they end up just sort of tripping over these landmines over and over again and it ends up becoming frustrating, right? Because it's not consistent from team to team. So they're in silos and they're transitioning over or you know, you try to get two teams to work together and they just can't agree on how to actually do things together. And so that's where I think it comes, I don't necessarily think that one tool is less sophisticated or more complicated than the other, although some tools certainly can be. I just think it's that those teams then sort of further embellished them with rules and scripts and other types of code. I suppose, you know, you could have one tool that becomes really complex as well with all, you know, manner of business rules built into it. But if it's in one tool, I feel like at least there's a common interface to find out where those rules are and you know how to discover them and know how to understand them. And that there's a common knowledge base across the team or the organization that can help you with shared knowledge. When it's all different individual tools, it's sort of like figure it out for yourself. Yeah. Well, and so you're sort of saying too that you may stop to figure it out yourself to start with, but once you've figured it out once, you get to reuse that knowledge over and over again. Yep, exactly. Other questions? Any of what I have? Yes, over here in the front. Oh, sorry. When you're using it in an enterprise, is there a limit on a number of teams where you start to have problems and usability of the user interface or being able to apply governance across the teams? Are you talking about the tool itself's ability to scale? Yeah. So that's a question that in this particular case, for GitLab, I'll let John talk about that. But what I've typically seen is that people are having scaling trouble because they have too many different types of tools. So maybe it's one thing to say, is it one single tool that's one instance or is it the same tool just used in multiple places? Right, so those are two different kinds of things, I guess, two different ways to scale that. I guess that's a question I don't actually know a direct answer for. But my feeling, so if you look at folks that are moving to cloud-hosted systems, the main reason they're doing it is because they're just sick of managing a bunch of snowflake servers. And that's why they're moving to the cloud. They're sick of having to ask for someone to create a server for them or stand up a tool chain so they're moving to the cloud. If you look at people that have teams that manage on-premise installations, they're looking for one tool so that their team can master it and know how to create self-provisioning systems that can scale that they can actually manage. So there's two different things going on there. Teams that are okay and willing to invest in having an on-premise team to manage their tool chain, they want standardization because it makes it easier for them to scale out as a services team. Teams that don't wanna have to manage that team or can't afford that team are saying, I just wanna go to the cloud because I can't afford to manage it myself. Thank you. Anyone else? I'm gonna stand where I can see everyone now. Ah, there you go. Yes, sir? Chris, wondering if there was any big surprises in the research that popped up and maybe or any assumptions that you had that were validated with the research? Yeah, so the assumption I originally had was that developers were mostly managing tools and even though like a lot of the companies that Forrester that we work with, their INO team does, but my gut feeling was that most of the infrastructure teams are installing the tools but they're not actually operating and maintaining them. And so this survey kind of pulled out the fact that developers are typically the ones who are actually doing all the work on that tool, even though maybe some operations guy put on a server for them. The other thing is that, so that was an assumption that got approved. The thing that surprised me was that the people that talked about multi-cloud and security as really big driving factors that those are the benefits that they want to get from a common tool chain. I think that's really interesting in telling that security is becoming a really big problem for a lot of companies. They're really starting to focus on it and they understand that it's not just outside the firewall, it has to be inside the firewall as well. So that was kind of a surprise to me. I didn't think that was gonna come out. Thanks.