 Okay, how many of you are doing Agile something? You know, scrum, flavored, hand-band, whatever. Almost everybody. So how did you decide to do that? Think about it. Did you make the decision? Did somebody else, a high-level executive, make the decision? And was it done scientifically? That is, did you look at the randomized, double-blind, placebo-controlled experiments that showed clearly, scientifically, that Agile is better? How many looked at those studies? Really. Okay, most of you are honest because the reason you didn't look at those studies is there aren't any. Well, Linda, come on, how could we do, how could we do a real experiment? We don't have time or energy or resources to do our projects once. There's no way. We could do a controlled experiment where we do them with two different teams using different methodologies so that we can collect some data and we could compare. It's not going to happen. And that is true. And that is why we are stuck with a very poor decision-making mechanism which says, well, we decide whether we're going to go Agile or not by whether we hear some good stories. Somebody tells us about another company. Or we hear about a team that increased customer satisfaction or decreased errors. It's a good story about some benefit of Agile. And on the basis of those good stories, you know, they're not even really case studies. Business schools like Harvard provide MBAs. They have a format, requirements for a business case study. We don't even meet that. We write up in many semi-prestigious journals stories about successes with Agile or other approaches. Everyone gets excited and says, we want to adopt this new process. And that's the way it's been for a long, long time, at least throughout my lifespan in our industry. We have a history of going from one bandwagon to the next. Never having any science behind our decisions. Never collecting any evidence. No data. So I get pushed back when I start talking about things like that because people say, but you know, in Agile, we talk about experiments all the time. Yes, we do. We use the word. We use the word experiment. And really, are we doing experiments? Are we using that word appropriately? Because if we are, then we should mean in terms of a scientific experiment that there should be, first of all, an hypothesis. An experiment begins with an idea that we want to test. And often there is no hypothesis. We just want to look at some piece of the practices or maybe a twist on a practice. We just want to try something. That's really what we mean by experiment. So there is no hypothesis. There's also no randomized trial. If we involve others in our organizations, it's usually the people who are already enthusiastic about it. Hey, who wants to try some Agile stuff? And those people who are already enthusiastic, well, they're going to make it happen. What we know there is plenty of evidence to show if you're already a believer, if you already like it, if you already think it will work, then you will make it work. It's called the placebo effect. And it's not just for medication. It also applies to process. And it's impossible to remove the placebo effect. You can't say, let's pretend we don't truly believe in it. Because as soon as we do, we know we will affect the outcome. And there's also no involvement of any kind of control group, a group for comparison. We just try to remember the way things were so that we can say now, oh, with Agile, it's so much better. In fact, sometimes when we begin to use Agile, we use the term common sense. Well, look at these practices, the values. It's just common sense, isn't it? And then finally, if we do any kind of remote experiment flavored trial, we get to the end and there is no analysis, there is no data collection. We just say, you know, this looks better than the way we were doing things before. And that's what we call an experiment. Some of you are perhaps in little development teams and you do something called A-B testing. Yes? Nobody? Yeah. Isn't that an experiment? Well, the answer is no. No, it is not. It never begins with n-hypothesis. It begins with a couple of ways of doing something. Let's change this web page so the A version will look like this and the B version will look like that. What's the underlying mechanism? What are we going to try to learn if we do that A-B test? There is no hypothesis and once we see the results of that A-B test, there isn't any movement forward on any kind of underlying theory. And in fact, we don't even know if those results will transfer from one organization to another. We don't even know if they will transfer to what we're doing next week. All we know is that right now the results of this A-B test show A is better than B. For what reason we have no explanation. We're not working toward an hypothesis verification or in the development of any theory. We're just trying something. We're trying two versions of a web page. So I'm going to suggest that we get rid of the word experiment because clearly we're not doing experiments. Clearly we never will do experiments. So let's pretend that we're not scientific even though we like to call ourselves engineers and instead use the word trial. We're just trying. In fact I'm going to point to a pattern from fearless change which is called trial run and there are lots of benefits to trials. So let's focus on what we can do instead of tying our wagon to a star that will lead us in a direction where we can't possibly go. We'll never be scientific. We'll never be able to do real experiments. But we can try things. So let's say trial, small trials instead of experiments and let's look at the benefits of those. I often hear that different teams want to do an experiment or a trial because they want to prove something. They're hoping that by doing a trial or an experiment they can prove to that other team over there or that other part of the organization over there that my idea is a good one and that it works. So even if we were able to do a real experiment, one instance of an experiment is never enough to prove anything. And if we're not doing experiments or we're just doing little trials, one trial certainly hasn't a hope of proving anything. So I have a message here that says, I just want Linda, you are out of luck. Can I have somebody help me with this? You know why it's me, man? Yeah. No, yeah. Okay. So I don't know what happened. We don't know. Okay. So isn't this fair? So what we know is that one trial doesn't prove anything and you should have a word that we don't often use in software development, humility, be humble about your results. Even if your trial is successful, you have to realize once and done, one instance, even if it were a real experiment, proves nothing. It gives you some ideas for the next trial. Some ideas for and hypothesis you might develop, but it proves nothing. So I feel like we're praying here. So we mentioned the placebo effect. It wasn't until the 1950s that medical doctors and people who worked for the pharmaceutical organizations realized that even though they were doing real experiments, that they had a controlled treatment group that wasn't sufficient. What they found is that after doing a drug trial that many times, even though the drug appeared to be successful in the real experiment, it didn't hold up over time and that people didn't show the benefits they had hoped for as a result of taking that particular drug. And they said there's something else going on here and so they added another group. In addition to the control, which gets no treatment, the treatment group that gets the drug, they added the placebo group. Now what's interesting about that placebo group, do you know what a placebo is? Yes, it's a fake tablet, fake injection, fake surgery. It's a treatment that has no benefit. But it looks like the real treatment. It looks like the real drug or the real injection or the real surgery. So the interesting thing is they found that people in the placebo group got better. In fact, a statistically significant number of them got better, about 50%. So the explanation for that is that the people who were in the placebo group didn't know that they were in the placebo group. They believed they were getting the real treatment and that belief, the power of that belief was enough to improve whatever the condition they were testing. If we knew more about the power of placebo, we might not need so many drugs. If we knew about the power of belief and could harness it, it's like a little pharmacy in our heads. If we knew more about how to use that, I think we'd be happier, we'd be healthier, but we also have to realize that it is always a condition in every trial. You can't discount the fact that people come to the table already believing in what you're doing and that if in a placebo trial 50% get better, what does that mean for your organization? 50% of the belief of people who come to the table, whether they believe it or not, is going to have an impact on the results that you get. If people believe or if they do not, they can have a real impact on whether or you're successful with agile or anything. That's how powerful you are. So what can you do with these trials? They should be, and I'm trying to create something that you might remember, they should be small. They should be simple. They should be fast. And they should be frugal. Do you know what the word frugal means? It means inexpensive or cheap. It doesn't cost a lot of money. Hang on to that, small, simple, fast, and frugal. And you should not use the same group of people all the time because remember you're going to be affected by their belief. You should bring in people who don't like whatever it is that you're testing. You shouldn't believe that if the trial works that that means you have proved something. You should use it as a little step in a long journey that has to do with learning. Maybe it's me. I'm beginning to feel bad. Let's try this again. Small is important because many times the trials are not going to choose the result that you want. In fact, the result that's going to happen is going to be something that you do not want. If it's as small as possible, the impact will not be so great. The mistake we often make with trials or experiments in our organizations is we make them too big and then we become too invested in them. And then when they fail, we ignore the results. We often won't even see it or we won't admit it or we will fight it. Because if they're very small experiments, then what we always want to say is they can never fail whatever the result happens to be. We're going to do some learning. We'll learn from whatever happens. In fact, in many organizations they don't like to use the word failure. So I say let's just get rid of that. We won't talk about failure. We'll present our ideas for trials as opportunities for learning and what will happen as a result of running the trial will always be a success because we will always learn something. And the question you should ask for all of your trial, can we make it even smaller? Can we make it as small as possible? But no smaller. Why simple? If it's easy, if it's so obvious, if it's just a little thing, then everyone feels free to provide an idea for a trial. Bring in as many ideas as possible. Always be running small, simple, fast and frugal trials. Let everyone's idea be heard instead of arguing it down or having endless discussions about whether the idea is a good one or not. Instead just say what kind of small, simple, fast, frugal trial can we run? And of course keep asking. Can we make it even simpler or small, simple trial? One quote from Rich Sheridan who's the CEO and co-founder of, I believe, the most agile company on the planet. Is that right, Shane? Indeed. So Google Menlo Innovations. And when I say agile, I mean agile in every sense of the word and that includes transparency. If you want to know how Menlo Innovations does anything, they will tell you. They will tell you how they hire. They will tell you how they do estimations. They will tell you how they do development. They will tell you how they do pairing. They will tell you how they do anything. They will share that information with you. So Rich says we want fast, frequent and inexpensive experimentation and assume they will fail. One of the most common phrases you'll hear at Menlo is let's run the experiment. Instead of having a discussion, instead of wasting time and energy worrying about whether the idea is a good one or not, let's just try it. Let's do that small, simple, fast and frugal trial and move on. Frugal means cheap. Frugal means don't spend any money on it. Frugal means it should be as easy as possible to run it so that you don't have to ask for funding. There is a bias that humans have called the sunk cost fallacy. And once we've spent money, time or energy on anything, then we're really reluctant to move on and say, I think I was wrong about that. We do the projects. We fail to cancel projects when they should be because we say to ourselves in the big meeting, you know we've already spent $10 million on this. We can't stop now. It's not going very well, but we've already spent a fair amount of money so we can't stop now. It's what keeps us sitting through a bad movie. I've already spent $20 on the movie tickets. I'm not going to get up and walk out. So one thing you can do to avoid the sunk cost fallacy, make them cheap. So you don't have to say, we spend a lot of money on this. We have a lot invested in this. No, it costs us nothing. And in fact, you should always ask yourself, can we make it even cheaper? Make it small. Make it simple. Make it frugal. And fast means to me, time box. Often teams will set up an idea for an experiment and a trial and the trial is underway and they don't want to stop it. They're either waiting for the result they hoped for or they weren't sure what they were hoping for. So if that fast, frugal, small and simple trial is not time boxed, it will run forever and you'll never learn anything. Say up front. And here's my very trivial example. This is a very small, simple, cheap experiment. Let's try having the stand up at 10 a.m. instead of 8 a.m. for the next two weeks and see if attendance is better. So there's something we can measure. See if attendance is better and maybe it'll be more productive when you come in the morning without having to worry about gathering everybody for the stand up. Time boxing will ensure that the experiment or the trial has a fixed end and can say up front what it is. They want to measure. What are you going to learn? How do you know what you've learned from the trial? I'm going to get this. This is a form of proof. We want to show management, for instance, that it really works for our team. So we want to show that those people who are opposed to our idea, we want to show them that they are wrong. Because if we run an experiment or a trial, we'll have some evidence. So here's the bad news and I'll be talking a little bit about this tomorrow. People, and that means us, people are not convinced by data. We like to believe that we are. We like to believe that we are influenced by logical argument, but the evidence is unfortunately that we are not. And in fact, there is often another bias that appears when faced with real evidence and that is we become more strongly entrenched in the idea that we have always held it's called the backfire effect. So don't think that you're going to show even the result of one trial as evidence that's going to convince other people there are better ways. Use some of the patterns from Fearless Change. Involve them. Bring them in if you're trying to show them agile works. Let them try it. Let them hear some good stories. Remember I started by saying how convinced we are with stories as though that were a bad thing. That is a bad thing on the one hand, but on the other you should use it. Stories are convincing. If people are happy with agile or whatever your idea is have them share those stories. Have them tell each other. Here's what worked for us. Not with the idea of laying out a logical argument but sharing experience. Here's what we tried on our customer was happy to solve a problem. What we know about our organization what we know about our teams what we know about us is that we are what's called the context adaptive system and that means it's almost impossible to solve a problem because as soon as you introduce anything as soon as you change anything you change everything and that side effect isn't predictable you have no idea what the end result of that small change could possibly be. So your only hope for changing something as complicated as hard to understand as your organization or yourself is to do some small things and watch what happens. It's called probe sense respond do a little trial see what happens watch and then on the basis of that result make your next small little trial and repeat I'm going to finish that slide I'll just start it again I'll just start it again I think this is not a good sign guys maybe we should do a small experiment what we don't want to do is get wound up with the answer to the problem there are very few problems we can solve in complex adaptive system we don't have the understanding we don't have the ability we don't have the will, we don't have the time we don't have the energy we live in a world now with a lot of problems I live in the United States I only begin to tell you about all the problems we're having right now and as far as I know the only thing possible that we can do is a little trial because ultimately it boils into this there are so many things we don't know and there are so many things we don't know we don't know and anytime we do an experiment there's risk involved and uncertainty and so we should prepare to be surprised I don't know if you know this gentleman Alexander Fleming? anybody? one and a half people good, okay so many of us would not be alive today if it weren't for Alexander Fleming because until the 1940s we didn't have the concept we didn't have the tools for dealing with infection and now we have antibiotics and we're overusing them and now we're going to face the same problem again very soon probably Alexander Fleming was not a very neat guy in fact he was very messy you can sort of get that idea from his desk he went away for two weeks on a holiday and when he came back he started moving little Petri dishes around and he picked up one of them and he said gee that's funny I thought I had cleaned these dishes and instead I see that some of them still had remnants of the bacteria that were growing in them but look here are some where the bacteria have been killed I wonder what happened now he didn't go into his lab that morning saying I am going to discover penicillin he was surprised by the behavior of the bacteria and the fact that his messy desk produced some results that he didn't expect and that's what you want you can't plan it out because there are so many things you don't know and in fact if you run every trial and you have the end result in mind you might not ever be surprised and that's not really a good thing my second favorite organization I hate to legal people here but Sipgate is a company in Düsseldorf, Germany and they are also transparent so if you go to their website you can look at all kinds of information about how they run their very agile company and they have a practice called Open Friday and they discovered it by accident they were doing trials and this is a quote from my friend Karina there who said we weren't even trying for that we were even looking for something called Open Friday it appeared as the result of a small trial and we learned so much that now Open Friday is a practice in their organization and that's exactly what you want to do prepare to be surprised prepare to learn now I often say you can't convince people with argument there isn't any reason for going into the idea of a trial or an experiment with the idea that you want to show those people but this is a good one we want management we want management to be on our side our team wants to go agile and what can we do to convince our management that does make sense because we know that management wait how many managers in the room oh come on come on alright we know management I'm going to say good things it's alright management management are smart people yes they are management are people who care management are people who want to support you but they do want evidence they would like to see what happened as a result of that experiment especially if it involves customer, suppliers interaction with others outside the company and you say things are better we can show clearly that our customers are happier our suppliers are happier we're working better with the testers things are better, measurably better they want to hear that send this long list of stuff here that I'm not going to this next slide is really dense and I'm going to make an offer to you that I do with all my presentations which is I know that you can download some of this at least from the conference site isn't that true somebody can tell me that but if you want to send me some email that was linda at lynda rising.org I will also send you the slides I will send you the PowerPoint and so you can give the presentation at your organization you can change it up and then you can read this very dense slide these are recommendations from books called the innovators hypothesis if there is one struggle that technical people have communicating with executives and managers they have a different way of thinking a different way of looking at the world different vocabulary concerns different pain points and if you don't address those all your technical arguments will go for naught yes they want to see your evidence but the things they care about are not the things that you care about as a technical person so you have to speak to the point of view of their world and if you don't it doesn't matter what kind of results you have what kind of evidence you have it's going to be as effective I found this quote from the vice president of product development at Shutterstock and he said when you have results from trials you kill off the hippos so a hippo is the highest paid person's opinion it's a good way to get to the bottom of a decision without relying on what some high level executive said at Shutterstock if a senior executive has an idea then everyone simply says let's try it let's test it let's not just adopt it or do it because it comes from the CXO everyone's idea is open for testing everyone's idea can be on the table for consideration not for discussion but let's do a small, simple, fast and frugal trial let's try it out it saves a lot of time here is the biggest danger the biggest problem our brains have with experiments even real experiments is that we have a whole list of biases that transfer what exists in the world that it makes sense for us and those biases never let us and there's no idea around them so our only hope is a bit of awareness so this is a report of a longitudinal study it was done in Hawaii it was done on a lot of men over 8,000 that's a big sign isn't it if the sample size was big it ran for 30 years that's good they were powerful they did this longitudinal study how many of you are coffee drinkers so automatically we can say you are biased are you not do you scan the newspaper or look on your phone when you're getting the latest headlines looking for benefits or drinking coffee come on, don't you do that yes, yes of course you do and probably journalists who write results of experiments are also coffee drinkers so when they saw this study which showed that at the end of 30 years those men who drank the most coffee were the least likely to get Parkinson's disease which is a neurological struggle that attacks a lot of people it's merciless nobody wants it the headline ran coffee protects against Parkinson's did you happen to see it if you had then you would have said yes now I can drink coffee and I can feel like I will be healthier and happier and Parkinson's free wow but they were wrong it's not that the data was bad nothing wrong with the method nothing wrong with the design of the experiment it was what the brains of the scientists did with the results and we do the same thing they said clearly the data show that if you drink a lot of coffee you will not get Parkinson's or your skin is minimized and they were wrong so I have my family carries the gene for neurologic researchers related to Parkinson's that is not Parkinson's my father has it or had it my brother has it my daughter has it and none of us drink coffee we drink coffee because we don't like it we have tried to drink it but we don't like it we really really don't like it we have an aversion to it even the sound and what finally we have learned is that those people who have tendencies for certain neurological disorders also have genetic dislike of coffee do you see the difference it's not that drinking coffee protects you against the neurological disorder it's that if you're going to get the neurological disorder you don't like coffee so the expression is correlation is not causation and when we notice that two things move together we want to link them in that way oh this must be a cause of an effect of the other when in reality we might have it backwards so the only thing you can do when you run your small, simple, fast frugal trials is always ask could we be misinterpreting the results could we have it backwards are we assuming that our results show one thing when in reality they show something completely different and that is so hard to do because we want our results to show something significant so we can have the headline coffee protects against Parkinson's that's what we want so if you carry nothing away today it should be this don't argue about it anymore don't have a meeting about it where you discuss it and you bring in a variety of opinions so that it takes even more time instead say what can we do here what small, simple, fast frugal trial what can we do that would help us answer this question and I don't mean this just for your organization or your team I also mean this for your life try it just try it the other benefit that will come from this change in mindset is that people will develop a culture of being curious instead of labeling I wonder if that could work of being open instead of labeling oh that's a bad idea well I don't know I wonder I wonder if it could work and that culture of openness that culture of experimentation will be the most valuable result of all your trials that that will change you into a creative organization that isn't afraid to submit ideas to try things to learn that's worth a lot there's a long list of things you'll have the slides so you don't have to go through all that and before you push back and say oh Linda, my company well we work with financial services and we can't always do a lot of experiments to that I say nonsense this first link is to a small organization that started out as part of the British government the British government is doing small, simple, fast and frugal trial if the British government can do small, simple, fast, frugal trials surely your financial services company can find a way and if you need some inspiration go to the behavioral insights team and now they're not only working with the British government they're also working with lots of other governments across the globe so if they can do it you can do it the other two links that I had there for sites that are not only collecting information about small, simple, fast and frugal trials they're going to try to give you ideas here are some ideas things that other organizations have run as small, simple, fast and frugal trials maybe you could change one of them adapt it to your organization and give you a chance to go and say what are others doing maybe we could do that too well I hope I don't have to do this more than one more time this is kind of a review for you take notes so I know the world has a lot of problems so when I think about anything these days I think about trying to solve some of those big problems and what I have to keep reminding myself is that those big problems don't have big solutions I think there was a man named Gandhi faced a lot of big problems and as I remember on one occasion he said what we're going to do is we're going to march to the sea and a march is really just a lot of little steps and it started out with a few people and as he marched more and more people joined and they all began to march together one little step at a time all those steps they led to something pretty big and that's the idea lots of little steps lots of little trials that's how we solve big problems that's how we all have to work together to solve those problems and you know you can see already I'm incredibly old I might not get there so you're going to have to carry on make those small simple experiments not just in your organization not just on your team we have to do that make the world a better place I hope we can I believe we can Namaste we have any questions we have 5 minutes for questions just trying to understand how do you differentiate from the Teota-Kata philosophy of doing continuous small what they call experiments compared to the trial kind of an approach that you are suggesting so I didn't understand the first part of the question so Teota has this Kata approach of doing improvement Katas where they would try and do small continuous increments, single factor experiments as they call it I'm trying to think if your approach sounded very similar what if there's any difference yes, no, I agree and this is not my approach this is sort of a combination of things that have evolved from a lot of sources and the realization that in my long history I have never seen anybody even take a remotely scientific approach to making decisions and when you think how smart we are and how much we care how poor we are in decision making when I know we can do a better job and we have severe limitations we cannot do real experiments so what are we going to do so it's not really my idea it's the combination of a lot of things including what you mentioned and all the books that was just a few of them lots of people have ideas about how to proceed and the good news is that this is happening I gave you some sites for the behavioral insights team many governments now around the world including my own at least until the president administration also had a behavioral insights team was also doing small, simple, fast, frugal experiments it's the only way forward for all of us because governments can't really do scientific experiments either is that okay? yeah