 What is deception it think of another word an equivalent for deception? Yes. Yes. That's the one I was thinking about lying lying Because if you're going to come up with an accurate estimate What would that be by the way a? Way to predict what you're going to do in the future that really reflects what you're going to do in the future Seems like an impossible task to me and It's not only impossible, but I'm going to throw out a lot of Cognitive science research that will show you it is impossible it is impossible if You don't get to talk to me about this Got my contact information and by the way several people have tried to send Email to Linda at Linda rising comm and it doesn't work very well So that's or Linda at Linda rising dot org and I'll be happy to send you any information about the papers That are in this particular presentation And I have something to give away so that you can read one of the papers that I'm going to talk about So this is an area that I care about I Have a PhD in computer science and I've been working in this field for a long time And I have come to face-to-face with a lot of Estimation problems, but the crux of the information I'm about to give you has nothing to do with Computer science or software engineering. It's all from the field of cognitive science or neuroscience Now I don't have a PhD in neuroscience. I thought about working on it In fact, maybe you've been thinking how is it possible that a woman of this age could have a PhD in computer science? Probably when she was going to school. They didn't even have a subject called computer science and That's true. They didn't So I started out life as a chemist and then I became a Mathematician and then I became a computer scientist and it's only in the last 10 to 20 years That I've become interested in things that are outside all the technical fields where I do have expertise and now that's my primary interest and One of the reasons for that is That the cognitive scientists do a lot of research that involves controlled experiments and so they make progress unlike our field Where we are still having debates that we have had for 10 20 no yay, even 30 years and Agile is just the latest version of a set of ideas that we think we believe Will help us, but we have no proof We don't do controlled experiments. We not really a science So I think I've been drawn to what we can see Who is doing science? So that we can see if there's anything out there that could possibly help us do a better job And this is not an academic talk So I'm not going to reference those papers But if you send me some emails saying Linda you talked about those people who believed That they could estimate the amount of liquid in a glass and the research shows that they can't would you send me that paper? And I'll be happy to do that. So this is what we're talking about Lying lying to others But the worst kind of deception and we're all guilty It's when we lie to ourselves We deceive ourselves about what we can do and That definitely gets in the way of an estimate Because if we're not clear and how we view ourselves and our abilities, then how can we possibly Put out a good estimate for what we can produce Even in a short period of time like the next couple of weeks It turns out that One of the interesting results from cognitive neuroscience is we're hard-wired to do this. In fact, we develop it over time Small children are not so very good at it, but they get better Because we teach them even animals Even animals deceive I've heard lots of good stories about dogs and cats But monkeys also deceive So how can we possibly Create reliable estimates Well, you know, there are people who make a study of estimation They always are trying to sell you something some kind of insurance and they've got a lot of data About how given your state of health and your age how long you are going to live They do a pretty good job of that And what's interesting is that most of us when we look at that information will distort it By saying oh no, that doesn't apply to me In fact on average most of us will say that we're going to live about ten years longer Than those actuarial predictions Now for some on a my age another ten years. Well, that's not really a whole lot I'm already running out of those ten years. I think that I overestimated ten years ago There are also statistics on Marriages how many of you are married? Okay, I know you do it differently over here. I've been hearing stories about that In the United States the marriage divorce problem. It's a dilemma Says the data says that about 50% of the people who do get married are going to get divorced So do you suppose most people when they get married are? objective in evaluating their chances of Staying married versus becoming divorced. Do you suppose they do some little decision tables? You suppose they carefully and objectively evaluate what their chances are No, it turns out that most of us believe that hours Will last forever. I have a good friend who's been married five times and Every time I've asked her Susan. How is it that you? decide to get married again and She always says the same thing. Oh this time Oh, this time I know This time I know that he is the right one Does that sound like any software projects? You know We know we know this time. We know this time. We'll get it right even when in the face of Those bad estimates if someone points out that last time our estimates were not so good. We can explain that away Oh, no, that was an anomaly. Oh, no, that won't happen again. Oh, no this time Just like my friend Susan. Oh, no this time all will be well We're hard-wired to do it. In fact if I really knew What you were thinking about me If I really knew What you were thinking about my ability to get up here and give you any kind of information if I really knew Whether you thought I was crazy or not. I Probably want to run home and cover up my head and never get out of bed again, but no I Believe you love me. I Believe you think even though I am an old lady that I do have a few things Might even be interesting worth hearing I Do believe that and it turns out that we all do that We all believe that we're just a little bit better than the average Now I use average, but as a former mathematician. I'm really talking about the median Most of us believe we are above the median. Do we have any Mathematically literate people here who understand the problem with that? How can we all believe we're above the median? How is that possible I heard about a study recently that was done in I Think it was Sweden and they asked drivers Well, are you better than the average the median? Driver and they all said they were Even though some of them were in the hospital Suffering as a result of negligence on the part of themselves as a driver in an accident They still believed they were above average. Oh No, I know what you're saying Well, that's other people those other people. Oh sure. I could see that they might believe But I'm rational I'm logical. I see the world as it is Turns out that smarter people are actually better and I remember the poll from yesterday. You are all smart people So I don't know if you've heard the story about Ignaz Semmelweis an Interesting man who noticed a phenomenon in an area of a hospital. This is in the 1800s Where women were dying in childbirth and so he started doing some little experiments Because he did notice that the doctors who were delivering the babies were going directly from performing autopsies Now that would be on a dead person Who had died for some reason to immediately without washing To go to deliver a baby So he had the idea well, why don't we make some Bleach solution available and tell the doctors to just quickly Rinse off their hands before they go deliver the babies and sure enough in his controlled studies He could show that he could reduce the mortality rate for women who had been dying Childbirth now, that's a controlled study. He had examined evidence. He performed a little experiment Surely we know scientists doctors. They're convinced by that Aren't they isn't that what science is all about? looking at evidence and if need be Changing your mind isn't it I hate this scientists are worse Than ordinary people at accepting evidence From experiments that don't confirm What they have already Published I remember looking at reviews of papers in Well, it was a long time ago. It was a different part of the world and the comment from the reviewer was I wouldn't believe this Even if it were true So I like this comment from max a New scientific truth does not triumph By convincing its opponents and making them see the light, but rather because its opponents eventually die And a new generation grows up That's familiar with it So if you look at the history of science it profiteem proceeds by jumps and starts Because there is a paradigm shift when the old scientists die and now we can make progress Until these scientists die and then we have to wait for them before we make progress again And then it's it's sort of like fits and starts scientists are worse In fright right now we're fighting a big battle in the field of economics Because the old economists are holding on to what was called classical Economics I was never an economist, but I do have I lack three hours of having a degree in it I'm old. I have lots of degrees So those classical economists if you remember Mary's talk this morning believe in the rational decision-maker And they have written books and published papers and done a lot of research around that and they're not going to give it up Even in the face of now behavioral Economics that says oh, no People are not rational decision-makers and we have clear evidence We have experimental evidence. Isn't that what convinces scientists? I thought this was really an interesting study. It says you're not only not rational But you're probably influenced by things you don't even realize you're influenced by including the weather So in the study it says that if you're going to go visit a college campus with the idea perhaps going there and If it's a really good school and it's a cloudy day, you're likely to go there However, if you go to a school that's well We have several of these in the United States They're called party schools where you go to have a good time at least until they kick you out And if the sunny day when you go visit you're more likely to go there So the weather Was involved in making your decision. Oh, well that's other people that's other people. Oh By the way productivity your productivity increases on cloudy days Or the productivity of other people so we know and I would love to have seen the Graduate students who were assigned to do this research on average There are three lies in ten minutes of conversation. So how long have I been talking? And that's remember average. We could have the average versus median discussion All of the high school students thought they were above average, but the college professors 93% believe they are above average and Since they're do you have online dating in India? I won't ask you to raise your hands The online dating population now is so big that a lot of social psychologists are studying it And they have discovered some very interesting things about people who participate in online dating But they're a younger population than average They're richer Well educated taller thinner and They love extreme sports. I wonder what the difference is So they notice there are trends That the women who are in the online dating population tend to be really thinner Whereas the man well they tend to be younger In fact, there are noticeable trends in age in men. There's hardly anybody there who's 30 But a lot who are 27 or 28 or 29 and then all of a sudden they become 32 It's like they finally gave up Lot of research around children How many of you have children So when a child gets a present a gift from grandmother or aunt or uncle and it's maybe not the present They wanted what do we tell that child? Say thank you Say you like it Say you love your grandmother Give her a big hug Do they do that in India? Yes, of course, they do it everywhere. They do it everywhere. There's an interesting study done with bars of soap Had little children participate in an experiment, but the experiment was really not about the experiment the experiment was about the prize The little children were told all through the experiment that they were going to get a special prize at the end And so they were really excited and when the researcher brought out a nice box With the big ribbon on top and they got started jumping up and down and they said here's your prize And they opened it up and inside was a Bar of soap and the experiment was really to see how would the child react to that bar of soap Parents were observing this through a one-way window And after the experiment was over and they were going through a debrief if the child had said, oh Oh, thanks Thanks, it's just what I wanted The parents said oh, I was so proud. I Was so proud of my child because he lied Well, I thought was interesting is that most parents around the globe say The lie they tell the most and I don't remember ever using this one Is that they will leave them behind if they don't keep up? Have you ever done that? really surprising to me is I always expect the United States to be at the top in any kind of bad metric We're the fattest nation on the face of the earth. We have high rates of obesity and diabetes I just expect the United States to be at the top. Well, we're right under China here 98% versus 84% Parents who lie to their children. I love this picture. It's a little Chinese child dressed in a snow white Princess costume playing a trap set at a mall in Beijing Well, that's another talk. That's another talk Surely there must be some things that we do well with estimates Surely for instance what we eat and drink every day we pay attention to that Especially some of us who are trying to watch our weight. Don't we do a good job of monitoring that? Turns out we have a bias for the vertical So in the glass on the right, which is the tall glass We assume that whatever is in there holds a lot more than it actually does The little short fat glass on the left we assume holds a lot less These two glasses in this example are holding exactly the same amount of liquid In controlled studies even professional bartenders get this one wrong When asked to pour a certain amount if it's in a taller glass or a shorter glass They're off because they have exactly the same bias for the vertical that we all do And if you want to come measure this later those two lines are exactly the same length When you look at them at first glance you think that the vertical one is longer Has some evolutionary advantage apparently to keep us from falling over cliffs We know that if we put a big plate or a big bowl in front of you, you will eat more So if you're trying to watch your calories One easy thing you can do is just use smaller plates It does work Distance works if food is within reach You'll eat more if you have to stand up and walk a few feet The amount you eat Read is reduced a slight amount if you actually have to go into another room to get it it goes down by another 50% So to avoid those second helpings Just leave the food in the kitchen and go eat in the living room and Put it in a little tiny bowl One of the problems we face today Not just in the United States Is that we sit Those of you were in my tutorial on the first day. That's my first recommendation now to software developers is stand up Stand up and move Sitting is the new smoking Plenty of research to show that sitting kills The longer you sit each day the sooner you will die Usually somebody will stand up at that point, but golly. No you just sitting there just sitting there So one of the things that happens if you sit all day is that your hormones are going crazy And there is a regulator of appetite called gherelin and people who sit all day Believe they're hungrier than they are because that gherelin balance gets out of whack So when you're not burning up a lot of calories, you're still as hungry still consuming in fact more calories So that feeling of hunger or what regulates your appetite is not Reliable it's not a good way to estimate How much you should eat we know that if you go into a restaurant and the name of the food is exotic And attractive You'll eat more If it's called giant chocolate lava to die for Instead of chocolate cake now the good news is That you can do that To encourage your children to eat things that you think are good for them Experiments with eating and with children have shown that if you call it willy-nilly dilly silly green beans They'll eat more of them I like this one. Where is it tiny tasty tree tops? Don't say broccoli So we are all susceptible to the influence of names and prices and Yet we continually underestimate how much we have eaten when you interview people after they come out of a restaurant Well, how many calories do you think you consumed? We're always under by at least 50% So we have no idea no clue if we can't estimate The amount of energy we're putting in our own bodies and we've been doing that since we were tiny What hope is there? For other very complicated estimates that we face in software. I thought this was really interesting think about that If you lean to the left The estimates going to be smaller you might try that on your team What do you think this user story Maybe and have some people lean to the right and I think somebody should lie down Lie down. I think it's probably about three story points, but I'm not sure You could experiment You might learn something. Oh, I remember that 9-11 I was in an airport. That's where I am a lot of the time anyway Never got off the ground. Nobody would tell us anything. They just said after a while go home No flights today, and it wasn't until I got home that I started watching that on the television Over and over and over planes Flying into the World Trade Center people screaming and running Panicky Watching horrible things happen Over and over and over Events like that change forever the way we think about risk And about estimates so a very interesting thing happened in the year after that People who had watched that Over and over came away with something about flying Flying is bad When I finally did get on a plane I was going from St. Louis to Minneapolis and there were only nine of us on a 767 which is a pretty big plane and it's the only time in the long time I've ever been flying when the flight attendant said sit anywhere you want Well, there was no business class, but ah, two of you were on the plane So we did we we sat anywhere we wanted no business class. So where did we sit? Where did we sit? emergency We all sat together Window aisle window aisle window aisle on both sides and also We talked to each other Well, where were you? What were you doing? And all of them said except for me When I get home, I am never flying again That was the message they had gotten from watching those planes Fly into that building was flying is a bad thing People are afraid of flying anyway because of loss of control but now Flying went way down now people in the u.s. Were still traveling So how did they get from one place to another if they weren't going to fly in there are no trains? Instead they were driving So some interesting metrics came out of that year air travel went down by 20% and all of a sudden passenger miles increased and that meant accidents and in that year 800 extra Deaths which is three times the number of people who were killed on the four planes on 9-11 So they didn't do a very good job of risk estimation. Did they? They said it's too risky to fly So instead we'll go out and drive and kill each other that way Often wonder if that's what the terrorists had in mind We will scare the people in the u.s. They'll never fly they'll get out and drive and they'll kill each other off It worked it worked we did it and even when shown that data People who are afraid or who makes their risk assessments based on fear and that would be all of us are not convinced By data we all do that Any smokers in the room One thank you, sir Smokers are also afraid of flying. They're not afraid of smoking Even though look at the statistic average life expectancy reduction for smoking five years Average life expectancy reduction for flying one day They're not convinced by data Do you have lotteries in India the lotteries we all think we're going to win the lottery Look at the odds how many zeros are in there Oh my god Chances of winning the big lottery one in one zero zero zero are greater than having a heart attack one in fifty But no one believes we're going to have heart attacks, but we're all going to win the lottery It's got something to do with that television thing Because when the big winner is on television, we all see that and that's what we remember Is the big winner? I think they should do interviews of the losers So my calculation says if they took 30 seconds to interview each loser that would take approximately nine years Tell me how was it losing the lottery? Well, it wasn't really too good. I'm not too happy. Well, thank you, sir Well, how was it losing the lottery? Well, no not too good. Thank you, sir. Thank you for that How was that losing the lottery? No, not too good. Thank you, sir if they did that That's what we would remember that might help us do a better job of Assessing our chances of winning I like this one too if you drive 10 miles to buy a lottery ticket You're more likely to be killed in the car accident than win Well the fortune, unfortunately, that We have this hard-wired biased We are very optimistic about what we can do our talents and our abilities and It definitely gets in the way When we estimate anything Not just in software But certainly it's there. We all think we're better than we are faster more talented Better able to do just about anything we have to do So I mentioned I was a mathematician Often especially back in the early days when there were giant estimation tools I would get assigned to the estimation group Had a special group of people who were sort of the mathematically inclined people and we would do the estimates for the other teams They'd send you off for training and I love those tools Because you could look under the hood and you could see incredible mathematics Wow and Data oh my gosh Data they have data any kind of project They got it You don't have your own data. That's okay. They've got data for you So they give you the data they give you the tools Who could ask for anything more? Just one problem It didn't work We spent all that money all that training all that lovely mathematics and it didn't work It was no better than do you know this one? Oh, I think it's about 25 man years It's very disappointing. I Was disappointed my manager was disappointed the company was disappointed Turns out that we just don't do a good job of Estimating what's going to happen in the future based on our experience So if you haven't read that paper, please don't tell Harvard Business Review But I have a copy of it and I will send it to you if you just send me some email and say Linda Would you please send me that Harvard Business Review article? In it was a study of managers decisions They ran a scenario of a software project and Things started to go wrong and the manager started to make really bad decisions So they did a little debrief and they said to the managers now look here here were your decisions Can you see that perhaps you could have done a better job? You could have made better decisions and they agreed. Oh, yes. Oh, yes Well, yes, we see now that we could have done a better job So the next day they re ran the scenario a different project different kinds of errors and What do you think did they do a better job any managers in the room they made the same mistakes Even after the debrief they made the same mistake. This is not to say that managers are stupid I have said that on occasion, but that's not what I mean That's not what I'm saying here. What I'm saying is that we all Do not do a good job of learning from experience even when we take the time to do an examination Of the past and we try to learn from it. We just don't do a good job witness the managers process That somehow if we do a good job of tracking our velocity if we somehow do a good job of building our team If we somehow do a good job of that we'll get it right one of the reasons it's impossible is that we produce an estimate a Number of some kind and then immediately we start to do inappropriate things with it Like add Subtract Multiply and divide now that estimate is a guess It's exactly like saying well, how do you view the future? How does it look to you and you would say oh the future looks bright and Your manager gets ahold of that and says well, let's see Suppose we add an extra person to the team. Would it be twice as bright? Would it be bright plus two? So the inappropriate operations even for your brain the part of your brain that does projections estimation Innovation is located in a different part from add subtract Multiply divide you can only add subtract multiply and divide on Things like numbers that really have some kind of meaning your number three or 69 or whatever it was that you came up Is a guess it's just like bright or green or Medium you can't add it We should be honest We should have the courage to stand up and say I don't know and you know what nobody else does either I need to have the courage to say that this is a guess and if you start adding Subtracting or multiplying or dividing to something. That's just a guess like bright sunny day You're going to get something totally meaningless and Then prepare to look for another job Turns out that agile does provide some hope Because it says don't sit down and try to estimate what's going to happen ten years hence I can remember meetings where we were doing that You'd have the customer come in and you say oh sure we can tell you we're going to deliver this on June 16th in Auton 6 10 years out. Of course we can do that They knew we were lying and we knew we were lying So I started calling that the dance of deception. Oh, yes, we can have that in June. I know Just believe me. I'm telling you the truth And they would say oh, thank you Thank you. That's when we want it will be so happy when your product arrived and then of course we were late And they weren't so happy. So is better if you don't Go down that road of ten years out if you say well I can do a pretty good job if we do a little tiny estimate. We are better at that Our estimates really are better For two weeks out instead of ten years. Oh, yes No surprise it seems so obvious, doesn't it? Finally and I'm going to give this away. I'm an editor of IEEE software and I brought my latest copy with me and sure enough here's an interesting article This fellow is now trying to put together all those cognitive biases that I just talked about and to see If there are some helpful tips That will let you kind of get around some of those biases and make your estimates better Brilliant finally we have a scientist who's putting those two things together So that article and a lot of other things about the requirements and architecture. So just Make that disappear. So I don't have to haul that home He's saying that here are some tips from this particular article if you have to look at a sequence of tasks My for instance in your next iteration Maybe you've got ten things that you think you're going to do and you line them up according to t-shirt size or whatever You're using right now and so you have an idea of the smaller ones and the medium-sized ones and the larger ones your cognitive biases Work best if you start in the middle Normally, I think most teams that I've seen they start with the small one and it turns out that the estimates in that case are Worse than starting with the ones in the middle and let those Drive the estimates for the larger ones and the smaller ones. So take that one Take that one home with you Look at the difficult ones first. I Know I worked with a lot of teams and they think no no no I know this one is easy. They'll probably only take me and Then they use that one to drive the Harder ones so the researchers say no no no do those difficult estimates first and then Because the order in which you think about things Does affect the results So don't start with the little ones start with the more complex ones. We know we're really good when things are close together So when you're laying out those tasks for estimation if there are big jumps Save those for last the ones that are pretty close together within an order of magnitude is what the research shows your Relative estimates are pretty good You can say well, I think this one is probably twice this one if they're very close together if they're More than an order of magnitude apart Your estimates are not going to be very good already talked about that one It's not enough to have the failure or even recognize it unless you actually take the pain in a retrospective to say Let's really Let's really do something To learn from it. What's our experiment going to be in the next iteration so that we can make sure we don't have that same mistake again And we know that we're more objective if others Are involved when you do this task of estimation on your own Then your own biases are the ones you have to overcome Normally if you have somebody else in the room even somebody who's not on your team Who can just give an opinion? But the old Delphi method depend upon an independent observer somebody else who wasn't Emotionally connected to those estimates could help yours be better lots of research around independence For those of you who speak multiple languages I thought this was really interesting because I know people who do speak more than one language find that their brains work differently So try having your estimation section in different languages Of course, I guess that would only work if you were all speaking the same other language wouldn't it? Actually, I have one team where they are Norwegian and Danish and they find they get along better if the Norwegian speak Norwegian and the Danes speak Danish Then if they all speak English They can understand enough of the others language Whereas if they have to speak English, that's in a cognitive load that takes away from the task in front of them So the result of this research says you do a better job of thinking if you are not in your primary language So you could try it as an experiment for those of you who are bilingual I gave a keynote speech at Agile 2011 it's online. You can go listen to it I'm going to do an extension of that this year at the Agile conference and it has to do with how you view the world Whether you believe that you're born with a set of capabilities and there's absolutely nothing you can do about it Or whether you believe well, of course, I am born with a set of characteristics. There is nothing I can do about that But I can always improve One of those is the fixed mindset You can't do anything about what you've got and that's it or the Agile mindset which believes yes I am limited. I may never be in Einstein or Beethoven, but I can always be better In fact, I can be better tomorrow than I am today If you have the Agile mindset your estimates are better You want to read more about that these References are all online You don't have to go out and read the book on the Agile mindset or you just Google on that Agile 2011 Linda rising the weird talk. I love this quote from Ron Jeffries Since the main thing to remember is This is not a fact. What will happen isn't just what you laid out there It'll be a lot like that but it'll be somewhat better or somewhat worse and once in a while You'll get lucky and you'll do lots better and sometimes you might not even come close However, the good news is you don't have to wait until the end The iteration plan every couple of weeks you have a chance To reassess and see how fast you're going so you'll have every opportunity To steer your project to success. Yes That's the Agile message It's not about getting it right It's about learning over time And having a chance at the end of each iteration to say stop What do we learn including what do we learn about our estimates? So for those of you been waiting for the silver bullet Here it is It's the estimation goat. Don't just go to estimate goat calm Anything you want to know the goat will tell you how many story points the goat knows Should we take on this project the goat will tell you Should I get married the goats will give you good advice So if you really want the best possible advice given the world we live in today The goat is the place to go So don't say there are no silver bullets actually there are no silver bullets, but there are silver goats So here's the good message. I Told you now for almost an hour that you're biased You're overly optimistic you believe your way above average when clearly that couldn't possibly be the case for all of us Suppose someone offered you the chance to remove that bias and to be objective and truly rational About your talents your abilities and your place in the world Suppose that was available to you. Would you take that? Opportunity would you don't do it. There is a small subset of the population that does see the world clearly Unfortunately, most of them are on serious medication and many of them are in the hospital It's not a good thing to see the world as it is. It's not a good thing to know what people really think about you It's much better to believe that we all love each other and that we are all way above average Don't take that pill Stay optimistic You'll be happier in the long run and isn't that better than an accurate estimate?