 Okay. Sorry. Thanks, Ekta. So for those of you who don't know me, Joanne Molesky co-authored Lean Enterprise with Jess Humboldt and Barry O'Reilly, who were my colleagues at ThoughtWorks at the time. I am currently employed by ThoughtWorks, been working for ThoughtWorks for seven years, started in Melbourne, Australia, now live in Canada, which is actually my home, and work for ThoughtWorks in North America. My roles within ThoughtWorks have been varied. I come out of a governance, risk, and compliance background. My degree is in home economics with a major in foods and nutrition. The economics, yes. Well, I worked in health care for about 12 years before I made a career change, and I did a variety of jobs. And then in the mid-1990s, I started teaching people how to use, if you know it, Netscape Communicator. And from there, I just haven't looked back. So coming into ThoughtWorks, governance, risk, and compliance, which is basically a foreign concept for ThoughtWorks at the time, and have worked in a variety of roles. First role globally was as the lead, the global lead for our continuous delivery practice in developing the capabilities of our internal people on all the practices related to continuous delivery. Not that I know them all, I just led all the other people who knew it all. Next, global lead for information security. Again, not that I know it all, just led the people into developing the capabilities. I currently work mostly in our internal tech ops management, and being more an advisor to our tech ops management about governance, risk, and compliance. And to me, governance is about visibility and transparency. We create the policies, the organization structure, the rules to create visibility and transparency. And often what happens as a result, we actually obscure what's happening because people find very creative ways to get around rules that prevent them from doing their work. So part of my role last year was to go in and look at one of our key accounts for ThoughtWorks and do an assessment on it to say, you know, we may have lost our way a bit, and we need to figure out how to get back on track with this account. So I worked with one of my colleagues from ThoughtWorks Australia, Meiju, and we said, how are we going to do this? Because in order to do an assessment, you really have to have some standards or criteria that you're going to use to say, are you doing a good job or not? And part of our goal was to give the teams on that delivery engagement something that they could work with after Joanne and May walk away. So we gave an assessment, we had to do the assessment, we had to have recommendations and findings for the teams and the management, and then we would just walk away. Well, that doesn't work very well. And most agile transformations when people are starting off with agile, this is the big problem. We bring in a lot of experts and consultants, show people a few rituals, a few ways of working, and then we walk away. So we came up with this framework that says, once we leave, here's something you can use to help you, all teams within the delivery project, to use to help you improve and stay with that principles and concepts of agile delivery. So this is mostly about delivery. I'm not talking about organizational transformation at this point. So I just want to make clear this is a little different than what if you've read Lean Enterprise, this is a little different. So a question here, how many people here are actually on delivery teams working on projects or product delivery? Quite a few hands up there. How many people would say they're in management responsible for people who are delivering software? A few more, some of them are the same. And how many people just don't have anything to do with software delivery? What do you do? I'm just interested in software delivery, because that's a big part of it. I can't move on the side of it. So this may help you. It may not, but if you feel you're bored, just get up and leave, and I won't be offended. So we didn't want to call it a framework, because frameworks have a negative connotation in thought works. So we said, oh, what can we call this? And this is really the Xenovagile delivery. I speak internationally quite a bit. And this year and last year, a lot of my presentations have been on the difference between doing Agile and being Agile. And there's a huge difference. Now, when you're a beginner and a novice, you do Agile, because that's the way you sort of learn the basics. Unfortunately, when we teach people the basics, we don't teach them the core basics, the principles and the values behind Agile. We just teach them, this is how you stand up. This is how you do a retrospective. This is how you do blah, blah, blah. This is how you do scrum, you know. All little pieces, but the most important thing about being Agile is your ability to recognize the single signals that tell you, you need to change. And most of us forget about that. So that's at the core basis of this. And when Maya and I started, we were going, oh, look at all the stuff you have to worry about when you talk about Agile. There's people who write about practices, oh, we got the DevOps and continuous delivery thing. Oh, now we got experience design and design thinking getting in the way. And then Joanna and Jess and Barry wrote about Lean, so we had to throw that in. It just goes on and on. And of course, there's the standard scrum XP, safe and less. And what I'm saying, none of these in and of themselves are bad. They all contribute in some way or all influence some way of allowing an organization and delivery teams to be Agile. Most delivery teams can't get that systemic view of how their work is affected by others and affects others. We don't take that view because we're focused on delivering whatever our product owner tells us that they want the next feature. So we're saying, how do we pull all this together? And I really want to say before I start again, there's nothing bad in and of any of these things. If you're beginner and you look at these things, it's important to read up and understand what they say. Take what makes sense for you at the time and try and apply it. Don't use any of them as the dogma of how I actually have to be Agile. And I say for many teams and transformations don't even say the A word, right? Don't even use a DevOps word. Just encourage people to get better at delivering working software. So we came up with, we call it the framework. I don't know if I still want to call the framework. I don't know what else to say. But basically, it's information on which you could hang conversations about, are we getting better? It creates a common language. But really, we looked at it and we said we have to talk about shared measures of performance, cross functional teams that can collaborate, talk to each other. You have to have fast feedback. We need shared goals. And that goes into, I'll go into the end-to-end product delivery approach. Here are outcomes. This is my lean thinking here. You want to reduce waste in the delivery software. And you want it to like customers. Now I said reduced waste. So I'm going to go through a couple of principles that are really important for everybody to understand and keep in mind no matter what your work is. And the first one is reducing costs is not reducing waste. And reducing waste is not necessarily reducing cost. Hopefully, if you reduce waste, you free up time to do things that have benefit and value to the organization. But a cost is a unit cost. Be that people, infrastructure, hardware, software, travel, training, blah, blah, blah. And it's usually related to numbers of units. So what's the cost per unit and how many units do I have? Whereas waste is things like wait times, cost to delay, communication overhead, defects. Oh, the operational support and maintenance. Just keeping something running that's a piece of junk. But somebody wants it somewhere. So I got to babysit the thing. And functionality that is neither useful nor used. And this last one, this doesn't seem to be working. Oh, there it is. Functionally, neither useful nor used. That is the biggest waste of software delivery today. We're creating stuff that we don't need to do or use. It's just wasteful. Now the other thing for product delivery is always focus on the decisions that you have to make. Not so much on how to do the work. But you need to look at these things when you're thinking about how you work. And I call these the four R's. Are we doing the right thing? Are we doing it the right way? Are we getting them done well? And are we getting the benefits? This is what value is about. And this is what governance is about. And if people tell you governance is about the policy structures, processes and procedures, I go, no. Those are outputs. This is the outcome you're looking for when you're looking at governance. And this comes from an old framework that was put up by a soccer called VAL-IT. I think it was in 2007. This is not new stuff. It's been around for ages. So the framework in and of itself, how would you use it? What does it provide? Well, we've already talked about that high-level governance model. And what is that anti-enview of the Agile delivery? From I have an idea to it's in production. The assessments, you could actually use this to help you assess your own teams. We could actually have external people come in and use it to say whether we're doing the right things or we're doing the right ways. Continuous improvement, this is really important in Agile. You want to keep getting better. You don't stop at doing the rituals. You need excellence in engineering practices. You need that collaboration and all the other values I'll talk about in a minute. And then the baseline of practices, the principles and concepts. We developed this internally within ThoughtWorks. We're testing it out and we're using it. But as we go through it, you'll see that we have about 42 different elements that we talk about within the frameworks. Being Agile from a technical aspect is not a simple thing. It's quite complex. And so what we have found the benefit of this framework is giving that common understanding of what we're talking about and something, the common language and then something on which we can have the conversations to say where do we need to focus. So why do you need it? I already alluded to that. You need that common understanding, the place to start the conversations within teams, amongst teams, with business partners, product owners, with the executive. You need to know how to discover problems. And because we have a concept of what good looks like, it allows teams to say, you know what, we're not so good there. We may have a problem there. And really, really important with anything that you do is you have to be able to measure your improvement of the outcome of your work. You can use it in delivery teams for conversations with stakeholders. So it's just a way of thinking about Agile practices and how your teams are doing or your team. What it's not, and this is with any framework. I want you to remember that frameworks are merely suggestions, okay? A bunch of people who think they know about something, get together and create a framework. Some people are really smart and sell certifications in the framework and get, you know, are able to support themselves. Or they create entire organizations to do that. Right? But most frameworks are basically some people who know something about something, have put together some sort of document that tells you, give idea to novices and people who are not as experienced, some suggestions on how they may achieve a better understanding and working knowledge of the subject matter. End of story. It is not a dogma that says you must do this. Merely a suggestion. And I always say to people, you'll read a framework, understand what's in it, take what works for you at the time or might work for your time, try it out if it doesn't work, toss it out, go on to another framework, use four or five different frameworks. Frameworks are not to be confused with standards. Standards are when an external body probably comes in or even sometimes an internal body comes in and says, you must do these things. Totally different concept. This is a framework. Okay? It's not complete, conclusive, prescriptive or exhaustive. And it will change over time as we learn more. So whatever I say tonight, take what you think might work for you, try it out, adapt it, change it. I don't care. We don't care. We want to share the information and get people thinking more holistically and systemically about actual delivery. Excuse me. Okay. It's a target. Bullseye, whatever you want to call it. I don't apologize for the graphics. I'm not a good graphic designer and we haven't been able to get it into our graphic designers yet, but we start with the goals. This is critical. What are we trying to achieve as an organization through the work we do, through the use of technology? It's important for, as us as technologists, working in any level to understand what our business goals are so that we can align our work to that. And ask the important question, are we doing the right things? Do they align with the goals of the organization? These should be clearly articulated by the executive of the organization. If they're not, make up your own and then go and get confirmation, seek confirmation from the people who are actually in the business operations of the organization. You always state it from a customer perspective because that allows you to broaden your own perspective of what the possibility for solutions are and what the possibilities are for the work that you do. And I'll give you an example. A goal for an organization in the financial institution is to sell more mortgages. They want to do that. And this is an example of my colleague in Australia, Gary O'Brien, uses quite frequently and one that he actually experienced in an organization he was working with. But when you think about a customer, who in the room wants a big mortgage? Not many people, right? If I could avoid it, being a Canadian, I would totally would. I would pay it off as soon as I can. Most of the world is like that except the United States for different reasons. Because do you know that my interest on a mortgage is tax deductible? Yeah, awesome. So I was encouraged people to buy bigger homes and get another big mortgage. But anyway, I digress. You don't want to sell mortgages from a customer perspective. I don't want to buy a mortgage. What do I want to buy? A home or a property investment or maybe my kid's education. You know, there's different reasons why the customer wants the mortgage. And when you start thinking in those terms, it's not about, you know, creating new operational efficiencies and selling mortgages and creating new incentives for selling mortgages. It's about helping your customers find a home that they can afford to find a property. It's going to be a good investment, right? To finance their children's education in a safe and reliable way. So that brings a whole new perspective on what the type of solutions are that you might deliver to the customer. So that's why we say it should be stated from a customer perspective. You want everybody in the organization to fit their role into this goal. Now again, that's why the statement of the goal is very clear. But if you don't have a goal, the team should make one up so that they have some focus on making decisions about is this a good thing to do or not? Because it is the foundation for good decision-making within the organization. It all starts with that. Wrapped around goals stand for delivery teams or values. Does anybody recognize what's up on the wall? A few heads shaking? Seattle Manifesto, right? And you know, if you listen to Martin Allister, the people who wrote the manifesto, what they'll tell you is that this has been abused in certain ways to people are misinterpreted where you say, we only do the things on the left and we don't do the things on the right. No, you have to do the things on the right in order to avoid chaos, complete and utter chaos. However, if you have to make a decision, your values between one thing or another, when you have limited resources or limited time, you would choose the things on the left rather than the things on the right. So individuals and interactions over processing tools, working software over the documentation, customer collaboration over contract negotiations and then responding to change over following the plan. I can't tell you the number of organizations I go into and said, we want to do an Agile transformation. What's the first thing they do? They build a big waterfall plan. You go, okay, let's just step back and talk about this for a minute. And then the other thing they'll do is we'll bring all the instructors in, all the coaches in, and then we're going to measure our success by the number of teams and people who have taken training, the number of certified people we have and the number of teams doing daily stand-ups, the number of iterations, the number of teams that are doing smaller iterations, two week iterations, blah, blah, blah. Nothing about testing the team's ability to get better and do change faster and responding to real data to make new decisions and change the plan. Now values with values also come what we call the guiding principles. And these are corresponding to the values in the Agile Manifesto. So we came up with six guiding principles that we said for our teams at the ThoughtWorks, this is what we want you to consider whatever you do. These are what you should value. One team, no matter where you sit, no matter who pays your paycheck. If you're working on a product, you're on the team. Now this was really important to us because we were actually looking at a distributed model where some people were offshore and some people were onshore. And the offshore people were being treated like essentially code monkeys. And we had to have a very frank conversation with our client to say, you know what, you're not getting value out of those people because they're really, really smart and they're being told to shut up and do what they're told. And if that's the way you want to work, you should just go hire somebody else who will do that because we don't want to do that. We're one team, we need to be treated with respect as a team member on the same level as all the other team members who are onshore. We needed always on communication. There was crossover in the time zones between the teams onshore and offshore. But we also had video conferencing on and people just walk up to the video conference and say is so and so around and you could have that conversation across the oceans. Right? Focus on business value. We found that a lot of our teams didn't understand why they were doing the work and how it aligned with our clients goals. Now it doesn't matter if it's client's goal or your own internal goals. You have to have that focus on the business value and understand what the business value is. And that involves conversations with people in the business who understand what those goals are. Fast feedback. Oh, geez. What can I say? We're agile, but we're only releasing once every nine months. I'll say it right now. The only time technology brings value to an organization is when it's in the hands of the consumer. Up to that point, you may learn something. You may do a lot of hard work. You may be really, really smart and learn things. But you're not delivering true value to the organization until it's working in production. So the sooner you can do that, the better. And the smaller pieces of work you can do to get faster feedback allows you to reduce the risk significantly of making really bad mistakes. Really, really bad mistakes. So that fast feedback is really important. And I say to some of my teams, they say I want it within a day. Sometimes they say within two weeks. And sometimes they'll say you got two months. But I rarely go past three months. Give me data that will prove to me that we should continue down this path that we're headed. We don't do that. We don't do that very often. And I'll take the example of we go into organizations many, many times. Are you doing continuous integration? Oh, yes. We have a CI server and we use Jenkins or whatever. How often do your devs commit? Oh, once a week, once over two weeks. How many branches do you have? Oh, five. That's not CI. That's something. That's waterfall. That's the way you work in waterfall. And you're not getting feedback. So if your dev makes a mistake on day one and doesn't commit in day seven, everything past day one has been wasted. So we always have to have these conversations and I'm being very frank with you. We never are quite as frank with our clients when they're doing things wrong. But that's, you know, basically that's the way it is. People trick themselves into being agile and think they're doing it because they're not checking and they have no definition of what is good. For say, it's just too much bother to go and figure it out. So they make it up. So we're providing framework that says, okay, here very loosely defined what good may look like. Measure the performance and value and I'll talk about that a lot more later and then the culture of continuous improvement. Keep going, keep going, keep going. So that's what it looks like. Goals, we wrap the values and the guiding principles around it and then we get into foundations. Foundations are elements required across all teams within accounts. And we were talking about distributed delivery on this. So it says distributed delivery and I have a cleanse to this slide. So there you go. It could work for non-distributed delivery where everybody's in the same room but they reflect reviews on delivering software and technology solutions, baseline standards. Everybody should do these. There's lots of them. Don't freak out. These are the ones we came up with and I'll be quite honest. They're the ones we came up with. We probably missed some. There's probably some of you go, we could sit and discuss this at Nozium and I go, you guys figure out what your foundations are. That's the point. But this is it for us. This is what our foundations is. This is the minimum we expect from all teams. Everybody will operate these ways and we look at the 14, I think it's 14 different elements around there. And we try to align them with the different values and principles that's the segments of the circle. And then wrapped around that are team practices because there's a lot of stuff that we do in agile that is largely, you know, how we implement it. It's largely dependent on the complexity of the work we're doing and the system we're working with. How new is it? Right? How many systems to integrate in? What's the architecture that we have to work with today versus where we want to go? The number of team members we have, their capabilities, their location, their size. There's many different factors that would affect these practices. And this is what they look like. Again, there's a whole lot of them. I think there's 17 here. The point is that you can do this yourself. Right? Some of these elements you may not worry about at all. Some may be very, very important and you say, well, you missed one that's really important for us. Put it on and deal with it. It's the way I look at it. So within each of these foundations and team practices for ThoughtWorks, this is what we did. We said, what's the goal of this particular element? What are we trying to achieve when we talk about this element within the team practices? What's the definition of good? And this is for our assessment to say, if we were going to look at a team and tell them where they could improve, they would say, okay, you're good when. And then we said, what are some related operational metrics you could use to help you measure your progress in this area? And again, you'll look at these. You might look at these and say, well, you missed blah, blah. Okay, okay, that's fine. Just put it on. And to show you one of the cards for something that's very near and dear to my heart, access management. Particularly because I find that delivery teams historically have never paid attention to this. They do now because every environment is now needs to be as secured as a production environment. Right? So everybody needs to pay attention to access. So we gave the goal. We said, here's what we are looking at. If we're going to judge, you know, are we getting good? Are we getting better? These are things you want to look at, things you might want to test out within your team, within your program. And here's some operational metrics you could use to help measure your improvement in this area. And that's what it looks like when it's put all together for ThoughtWorks. You could adapt this to yourselves as well. Now the other problem that we encountered was that people really didn't understand how this all fits in an organizational hierarchy. So we said, okay, you need roles and responsibilities. We'll give you roles and responsibility at a very high level. So the executive and the leadership look at the goals and the values and principles. And for us, the ThoughtWorks is people who do delivery of software for clients. What we say is our ThoughtWorks executive should be creating a relationship with the client executive to understand the goals of the organization and the principles and values. And of course we look for a very close alignment of clients to share the values and principles that we have. And that's the executive level to worry about. At the delivery management level, we say that the delivery managers, both at the client side and on our side, need to worry about principles and values. So that's where we get the crossover and the communication happen between executive and delivery management. And they also worry about foundations for the teams. Make sure that those foundations are in place or the culture is such that we can actually get the foundations in place over time. And that requires our delivery management team to create close relationships with a delivery management team on the client side. Now if you don't have that sort of partners that are doing helping you with software delivery, you don't have to worry about that, but you should do it internally. So delivery, excuse me, the delivery teams worry about the foundations and the team practices. So if I can't get the foundations in place as a delivery team lead, I would go to my delivery management and say I need help. There's a problem here. Delivery management are the ones who should work it out. What are we going to do to help the teams get better on those foundations? Does that make sense to everybody? Yeah, it's pretty simple, but it works. And surprise, surprise, a lot of our management team didn't think in this way at all. They had nothing to wrap their work around and as a result, things are starting to fall apart. Teams are feeling they didn't get the support they required. They didn't understand what the heck we were doing all this work for and how it aligned with the goals and the values. And we follow this internally within ThoughtWorks now. And when we talk delivery management, we also include people on our business operation side too in that. Yeah. What do we mean by foundations? The foundations would be the foundational practices of the delivery teams. So that would have been the, let me see if I can, the gray part in this circle. All those practices there. Does that make sense? Since I've broken the flow, can I also ask if you're going to get the slides? We could arrange that. Yeah, as long as they, you know, just make sure you attribute them to ThoughtWorks because this is, this is RIP. Yeah. Okay, measures and scorecards. I said I'd talk about measures. Tada. Okay. We're back to the four Rs. Are we getting better at delivering technical solutions? Are we delivering the right way? Are we delivering the right thing? Are we getting the value we expected? They have to be simple, easy to understand. They're not complex and I'm talking about reporting from a team level through to the executive. Now the team levels they can get pretty complex but the thing is to focus on a few things at a time so that you're not over measuring and just creating confusion. They're reported by every team, rolled up according to goals or bets and should be used to tell the story of our ability to improve over time. That's why you measure. They need to be based on outcomes, not outputs. Okay. Outputs are things completed, widgets produced, process followed. What is a feature? Is a feature an output or an outcome? It's a test. Who says it's an output? If I deliver a feature, is that an output? Raise your hands if you think so. Who thinks it's an outcome? Few people. The delivery of the feature is not the outcome. What you get from the customers and the value they get is the outcome. The actual delivery of a feature is an output. I produced a thing. Is anybody going to use it? What's the outcome of that thing? Does it just sit there like a pivot table in Excel today? You got to ask yourself that question. Just because you built it doesn't mean it's actually going to happen. Deliver value. Actually deliver value. And scorecards and dashboards. Oh yeah. I used to be a technical writer so I'm very precise in my language sometimes in English and it's really important because a dashboard tells me what the measurement or what the status is at any point in time. Whereas a scorecard tells a story and it shows the picture of change over time. And it should be very few metrics. Whereas a dashboard can have numerous things. And I come out of the world of security. And I tell you we got dashboards up the yin and yang. It is dashboard city. And you can have lots and lots of different dashboards on the team level like status of the build is a dashboard. The scorecard would be how many times does that build break? How long? What's the change of the build time over time? That's your scorecard. It tells a story. It gives you information that you should act on. So the scorecard example and you would map this over a period of time would look something like this. And this is what we came up with again not written in stone. First draft will probably change this. But here you have and I'm going to work right from left. Okay. Sorry. Where am I? Okay. These metrics on the left. Cost of release to production. We chose that as one of our key metrics that we wanted to report on for our delivery teams. Why? When do you get value from the use of technology? Test. Anybody know? I said it earlier. When it was in production. When customers are actually using it. Okay. The cost is from that time when we actually started the work for this release to it got into production. It includes all the wait time you have for other teams. Because if you're trying to look at continuous improvement one of the things you might want to do is reduce that wait time. It would include the build time. It would include the number of people involved. It would include their billing rate. Everything else. All the meetings they attend to. It is a total cost of release to production. Now this levels the playing field amongst teams. Because what you can do is say yeah but that team is located over there and that are to reduce rate, reduce unit costs. But as they take longer to give you something of value then you have a sort of leveling metric to say is one more valuable than others? Like if you particularly with the finance people this is a conversation to have. Right? You can say yeah but these these group work much faster and deliver much more than that group. And it also says if you want to reduce that cost you have to go back and start asking the five why's. Why, why, why, why, why and work through the problems so that you can reduce the cost. So it should reduce over time. The next one is the number of defects in production and anytime you do a metric and you're going to use this like one metric to rule them all. You need a check metric. There is no one metric. Right? Because I can game that metric. Everybody in the room probably knows that. I can game that metric really well. Just push it out. I don't care how many defects there are. So as you say okay one of the check metrics would be the number of defects in production. And a defect wouldn't just be a functional defect. It might be a performance defect. Right? There's a lot of things that you can identify as defects other than just the usual functional feature defect. It doesn't work the way I said it should. And if you want to go even further you would you would measure how long does it take us to fix the critical defects. So you can add stuff on top of this too at a team level particularly and maybe at a program level. The executives may not be interested in that at all. But those are the things you would measure to start having the conversation with product owners to say look we need to pay attention to technical debt. Because it's costing us more to release production because our builds are taking longer and things are breaking more often and every time we add a new feature we increase the complexity. So we need some breathing space to fix all this stuff. And you have a lot of stuff that's not working at production as a result. The meantime to release production we use this at ThoughtWorks because we're into the automation in a big way and we want to do continuous delivery. That and we're looking at DevOps and what does it take to do that last mile from I have the code complete the builds are all ready to go I want to push it into production. So our goal is to reduce that time so that we don't have to take outages to move to production. It's fully automated and we architected the system that we can push it to one area than another area than another area than another area and it goes really really fast. You might have something different that you want to put in there. And if you do I would encourage you to do so but don't just keep adding right because then you get people just look at it and go I have no idea what this means and there's just way too many measures here. You just confuse people so keep it simple. I usually tell people give me two metrics knowing that I'll get four and then the last one hard for a lot of teams to do this but if you're a really high performing team and you have access to production you can measure you can gather data that gives you the return on capital. And we actually did this we do this with one of our clients of Australia and China and we can actually say you know by adding this feature on the web page we actually increased revenues by x amount because we have access to all that information and we're monitoring it on a constant basis but that's a really high performing team so you and but maybe at a program level in the account management level or the business partner line of business they have that information and you can start to share. Now over on the left side you'll see different boxes. At the top things that the executive account management are concerned about are the service level agreements and compliance to contracts and agreements between different parties so for us as somebody who does software delivery for clients this is really important. If it's internal and you're a financial institution there may be something in there about software delivery how many pieces of software are out of compliance or we're out of compliance and have they been fixed right? Down below at the program and team level you have operational technical metrics so these are the things that teams should be thinking about measuring not all at once but if they have a problem these are measures that you would look at to say where are we today and how can we get better and I'll talk a little more about them right after I talk about this slide a special word about features points and burn down charts. How many groups are measured have how many teams have their performance measured by the number of points or features you develop or deliver? I see some hands going up nobody wants to admit to it and yet every every client we go into the only question the business asks on the iteration is how many points okay we need to start having critical conversations with the business and tell them to stop reading those frameworks that tell them to measure points the report on points you need to measure points they have no direct relation to value I could deliver a point or a feature it still hasn't got a direct relationship to value they're relative they're not standards there's nothing that says this is a point and a point for one team that has three people in a very very difficult technical problem to deal with is very different from a point that a team that has many team members with you know a relatively simple product to deliver you can't compare them from one team to another no matter how hard you try to standardize they are not going to be the same points the concept of a point in a burn down chart features is to give the stakeholders an idea of what our capacity is for the work nothing more and it is not a measure of performance now if your points are all over the map you have a problem with estimation that you need to deal with but that's not a problem with performance okay and if we focus on features and points usually what happens is you get that technical debt building up and bloat what we call the technical bloat accumulates choose up the valuable resources that could be put to better use so stop stop including them in your performance scorecards end it find something else to report on yes how do you measure about what's important to the customer right what's important to the organization so it could be does it increase revenues does it increase the size of a basket if you've got some online purchasing does it increase the sales that that you have from your website does it increase you know points to some other service depends what you're trying to achieve through the delivery of the product that's what you measure the points are irrelevant they're good for the team but they're not a value measure at all and the reason we use points as a value measure is because they're easy we're humans that's what we do we measure the things that are easy if i want to lose weight what do i do i get on the scale every day and i go hmm it's still the same oh it went up what do i have to do what do i have to measure if i want to lose weight well really did you complete that red circle today maybe not and what did you have for dinner you know what was the caloric intake to get you to say the leading indicators get you to what you actually want to do lose the weight so you have to do what's the goal figure out a measure that would say what would be a reasonable measure now if you're not sure about measuring i highly recommend that you read Douglas Hubbard's book how to measure everything you don't need to read the whole book read the first three or four chapters to give you an idea of how measures work and there's lots of different ways of measuring did you know that when you're looking for survey reasonable amount of people to survey is five if you if you ask five people the same question you have a pretty good indication how the larger population is going to respond randomly selected you learn things like that in his book okay it's a it's a entirely different talk about you know what the actual measures would be and they're very contextual so for me to say this specifically would be probably the wrong thing to say is that good enough okay um and and then when you're doing measures here's here's a really important thing benchmarks they're very very useful when it's internally used to define the starting point where we are today measure our improvement over time and indicators of successful activities oh these guys seem to be doing a little bit better in this area than than your team maybe you might want to have a conversation see if they're doing anything that you could try that's a really useful conversation they're very destructive and they can be used quite destructively when you start saying I get this all this time from executive so I just got to put it out there Joanne I spent x number of percentage on it what's the industry benchmark for spending I don't know how you got this job but you know you can't compare yourself to other organizations you can't compare yourself to the googles or the amazons or the spotifies or even your competitors because context is everything and they're probably gaming the benchmarks too and they're saying what are those guys over there doing and those guys saying what are you doing and they're not neither of them are telling the truth so it's quite it's a useless game competition amongst teams that team's better doing that team there are the winners again contextual you have to look at the complexity the architecture the technology base the number of people and the resources they have available to them you can't say that one team is better than another what we can say is every team can improve and then they're also very destructive when they use set target sets by executives imagine this one executive I want all my teams to deliver a hundred points per iteration guess what's going to happen every team delivers a hundred points I just changed my definition of what a point is this happens all the time you have to have very critical conversations with those executives so I said I talked about technical metrics here's some that we came up with and that's very important remember here's some we came up with this is not the exhaustive list or the exclusive list this is what we decided we would use and any agile transformation any agile implementation that you do it's very very important for the teams to decide what they're going to measure and where they're going to get better the important thing to remember is though they have to have focus so you don't want to target more than two things at a time to improve and I would say one is best per iteration and then over time you get to a target level that the team has said say we're happy if we get to this level when they get to that level or if they can get to that level then you have another discussion should we move on to something else that's creating problems or should we just try and keep improving what we find happens when the teams are allowed to set their own targets and figure out ways to meet those targets and choose the measures that they will use to determine if they're reaching those targets they often exceed them they underestimate their ability to do well and change for the better it's remarkable as human beings are quite remarkable somebody I went to a conference in Australia and there was a lady talking about change and she said you know the thing about change is we say everybody say people don't like to change and that's not true people change all the time think of all the changes you've gone through in your lifetime think about how many times you change your clothes change the decor change what's sitting on your desk now there are exceptions to this but what people don't like is being told they have to change and how to do it and i'm one of those people as soon as you tell me how to do it i'm going to say no not happening if you tell me here's a here's a really gnarly problem i'll help you figure it out we have to solve it it's a whole different perspective and i will change so the communication around this and the way you use the metrics is really important because if you say to all the teams all right we're all going to go for 80% automated test coverage in all our applications and you know what happens not happening it doesn't have people find ways to get around it so you have to allow the teams to choose what metrics are going to use and always go back to the five y's identify the problems that you have use some of the tools that are available to you so you all have cam-band boards how many people do value stream mapping to see you know where the waste is and how you could improve five y's y y y y y to get to the root cause of problems and work on that and allow people to take responsibility for the change and give them credit for the hard work they do i just want to slap people who are leads and say i did this i go no you didn't your team did it and they don't acknowledge all the hard work of the team they may say one or two individuals but it's a team effort and once you start acknowledging the individuals and and as a as a member of the team it's a totally different dynamic that happens okay so in summary if you're just starting on the journey by just do agile just get a framework and start playing with all the techniques and the suggestions that they make and see how they work for you and figure it out if it's not working like give it a give it you know a pretty good decent time because you're gonna have to work through it you know say we're going to try something new we're going to try it for at least two weeks and then see what we need to change learn from failure please stop saying fail fast i keep telling jess i wish you had never said that you know and like jesse i think jesse robbins was the other guy learn fast try and experiment and learn adjust that's what being agile that's where it starts be aware of your decision-making biases we all have them based on our past experiences you got to leave those at the door and that's a really hard thing to do and listen when other people are calling you out and saying yeah but particularly if you know leader or management position your role is to allow people to do the work and to listen to them and remove the blockers measure measure measure not too much at once but try and figure out what are the leading and lagging indicators that i need to look up to help my team improve set the criteria that triggers the decision points when we're going to stop doing stuff so a time box no value delivered we stopped doing it and we do something else and you have to be reasonable about this and use the context of your particular teams to help you determine when that will be right limit the work in progress oh my god we go into organizations we say okay what is your portfolio look like and i know one organization it was like 500 different new initiatives on the wall for one year because it's just like i yeah we'll take it in we'll take it in we'll take it in but nobody's doing the prioritization and nobody's making the hard decision about what are we going to stop to do stop doing to get all this done something about limiting the work in progress at the portfolio portfolio levels at the team level two what i find with teams when i go in for a lot of our teams is they're writing their stories too big and they have too many stories in play because they think that they can fill up the spare time by working on something different and then you get the top context switching and then what happens is that nothing gets done right or well so we say particularly if you have bottleneck people get them focused to remove the bottlenecks start do some cross-changing but limit the work in progress for the rest of the team as well because if you don't do that you will end up focusing on features again and not the technical stuff the team level you're talking about something like unbound yep no and that's where you need the points right that's where you need the points we said we could do 100 we know we can do 100 and yet we got 120 up there what has come off has this conversation or go back into the backlog but i find that most teams try to do too much all at once and there's no focus and even with this measuring metrics and trying to improve you have to limit the types and the number of improvements you're going to do at any one point in time to figure out the most critical one work on that when you're happy that you've got that into the flow and it's working well for you then move on to another one don't try and do it all at once it's like 30 40 different elements there you can't do it all at once okay it's an art form i'm going to say it different circumstances different context requires different responses and that's when people executives tell me how to do this and i'm going well i don't know your organization here's some things that others have done that might work no no tell me tell me an organization that's just like us how they did it i go well even if they would share that with me i don't think there's any organization that's just like you the people within the organization are the best people to make decision about where you should start and what you should do not some expert who flies around the world talking about this stuff tells you you have to take responsibility for it to make it your own and focus on the outcome of the work okay we've done that quite a bit anyway that's all i've got thank you very much for being most attentive and i guess we'll take questions now how are we doing for time or at eight o'clock when you're saying team uh what kind of role inside the team because from what i hear from you is they're doing quite a lot of things they doing the technical improvement as well they also need