 Who was in Ninda Rising's talk on estimation and deception? All right, so none overlapping audiences. That's fine too, because none of this is going to be already familiar to you. I'm just two words about who I am. I'm a freelance consultant. I'm interested. I used to be a programmer, so I know a little bit about tech stuff. Nowadays, I'm into consulting, which means I'm also interested in the dark side consulting and management and all that. Who's a programmer? Who's not a programmer? Interesting. So let's talk after. I want to know what you're interested in in this topic. But for everyone else, I think we should be on the same wavelength of what design means. But I'm getting ahead of myself. So practices, that's the essential point of this talk. Now, shape of the S, that's a good transition into another S-shaped thing, money, value. What am I hoping that you're going to get from this talk? My intention is that you come away from this talk with a better sense of why in AdL, design is approached the way it is. Why, what is the relationship between things like per-programming and test-reven development? Is it coincidence that there are those two things in, say, extreme programming and refactoring? Did they happen to be thrown into a pot and stirred? Or is there a deeper relationship? That's the first thing I want to talk about. I would like you to come away from this session with renewed enthusiasm for doing some practices that you have in maybe doing, but not quite as much as you should or you think you should. Who's doing test-reven development? OK, so there's some margin, right? Who's writing unit tests? We didn't raise their hand before, Mr. George. I hope that you can also, for those of you who are already doing test-reven development, the others will be convinced to try it because they have a better understanding of how it works and why it might work. For those of you who are already doing that, I hope that you will find out about new things you can do to go to the next level. So things I will tell you about, which are not new, new, but new to you, but especially what I would really, ideally, if I had a magic wand that I could wave and have anything I wanted to happen out of this session, is for you, all of you guys here, to create guys and girls. Create new stuff, things that nobody even know exists because no one has thought of them before. Because you're applying IDs from a different domain to the domain of source code design. That would be pretty exciting. And then people will say, 10 years from now, this was invented in Bangalore just after the session that happened in Algeria, India. That would be cool. So let's talk about code. Because I want to, I know that most of you are programmers. I've done this with big audiences where people were also in other roles. But I want to clarify the term design. So this talk is about design in one specific sense, which is the structure of source code. Not other kinds of design. So one article that I think will leave a lasting impression in this field is an article by Martin Flower from the year 2000, which was titled Provocatingly, Is Design Dead? Now the context for asking this question was back then, agile, and the word agile had not even been formally recognized as being the unifying theme of extreme programming and scrum. That was the year after 2001. Interestingly though, if you go to IEEE, if you are members of IEEE, and if you search for the word agile in the domain of software engineering, and if you do a date filter so you can get only the publications from before 2000, you will find one article that has the term agile in the title, which talks about an agile development methodology. So the term was actually invented by someone before the famous meeting in Snowbird. You've heard about the Snowbird thing, right, when the agile term was invented, supposedly, because it turns out that someone else had come up with it before. So it was reinvented. There is nothing new under this. Everything is reinvention. But anyway, I digress. So back then, there was a perception that because scrum and extreme programming were kind of down on heavy documentation and stuff like technical design documentation, like UML diagrams, that meant that scrum and extreme programming were anti-design. We don't do design. And so Fowler wrote this article to address this question. Are scrum and extreme programming against doing design? And he came up, of course, with a resounding no. So there is design in agile. So let's talk about design. What's the first thing that comes to mind? And I may already have primed you to think in those terms when I say the word design as programmers. I don't have Kandii anymore to motivate people. I had Kandii in the previous session. So relationships between subcomponents. What's a very common way of representing that? So one of the first things that come into mind when we think about design, whether we like that or not, whether we approve or not, is I would even say a specific kind of UML diagram, a class diagram. Something that shows the relationships of inheritance between classes. That's one very narrow, very specific view. But for most people, if you want to give the impression, the appearance of design, you are going to create a class diagram. That's kind of a ubiquitous view of design. And before agile came along, I think that was a big part of what people saw as the whole of software design. And so one thing that I think agile has already accomplished is it has exploded. It has demolished some dearly held notions about design that people used to have. Not everyone, but the mythical universal representation of design, the caricature of design, if you want. So there had long been, and there still is, tendency to insist that design has to be visual. And in fact, if you look at some textbooks and some formal body of knowledge listings, you will see you're only doing design if you do a visual representation. That's not true. Look at those two guys. Well, as far as visual, what they're doing is not so hot. It's scribbles. But I think what's happening here, I don't know, because I was not there when this picture was taken. This is something I took from Flickr. But for me, it's kind of the new picture I want to have in mind when I think about design. It's people whiteboarding together. So they're having a conversation. That's a lot of what design is about to me now. I think of design, and I'm going to reference Dave's talk, which was conveniently just before. How many of you were in Dave West's talk just before? OK, so not much of a shared background with the rest of Uber. He talked about design as the intentional and deliberate alterations of some of the elements or the relationships between the elements within a system. And one of the key words there is deliberate. Because, so for instance, if you have a design question or issue, if you're trying to make a design decision, what does it take for you to be doing design? I know that's hard to answer. But let me try and narrow it down. If you have only one ID, then you're going to implement that ID. So are you doing design? Yes or no? OK, I'm going to argue that if you have only one ID and you implement that ID, you are not doing design. So how many does it take? Two IDs? No, if you have two IDs, what you have is? Sorry? No, what you have if you have only two IDs. Do you guys know the story of the donkey between two piles of hay? And he's, oh, this pie looks nice. But this also looks nice. And he starts to death between the two piles of hay because they're equally big. And he doesn't know how to decide. If you have only two IDs, what you have is a dilemma. And a dilemma is a bad place to be. So you are only doing design, in my view, if you have three alternatives. If you have three equally credible ways that you could do the design. And design is about articulating what the consequences of each ID is going to be and picking one with intention. And the visual thing is a secondary component. It doesn't have to be visual. You can do design in equations. You can do design in talk. Some people are going to be verbal. Some others are going to be like I attend to be. You can tell because I move my hands a lot when I talk, especially about conceptual stuff. So some people are naturally kinesthetic. So what's with the visual tyranny? Down with the visual tyranny. Anyway, I get carried away. Another thing that came from, actually, that's pre-agile, so from extreme programming, is refactoring. How many of you do refactoring? Great, everyone, nearly everyone. But the thing about refactoring, and do you remember how it's defined in Fowler's book, refactoring is improving the design of existing codes. So that's the definition from the subtitle of the refactoring book. Improving the design of existing codes. But what that means is the code has to exist first, and then we do design. And that goes counter to the old conception of software development as a sequence of activities in which design preceded coding, which may have been a good idea at the time. But it led to this silly notion of design is something that a bunch of people do in a group, and then they throw it over the wall as the expression goes, to the coders, and then the coders implement the design. Does that work? Have you seen it work? But did it work before? OK, we're not going to go into the historical debate, but I think the agile community basically smashed that notion to bits. It's dead. This one, I think we still have work to do in this one, so it's a work in progress. We were able to go ahead. No, OK, so I agree. I mean, I can see where you're coming from. I could put myself into your shoes and say, yes. But rather than say that way, take the notion that you are only doing design if you have three alternatives. Because I know some people who pretend that they have thought deeply about a topic, and then they have come up with a solution. But when you insist gently, don't go making people mad at you, they only have one possible solution. And they have one way that the design could be. They are not designers. They are fanatics. Maybe I'm making some of you mad now, but OK, let's talk more about that after. But the key point is we have at least one practice which gets us used to the notion that design is something that can and probably should, at least in some of the time, happen after the coding. And so we interleave design and coding rather than doing all of the design first and all of the coding next. So that's one thing that's already been kind of subtle, decided. I'm more interested in thinking about what comes after. What are the rest of the things that we have to rethink in order to have our design activities really incorporate what we have learned from 10 years now of agile. So we are starting to have this notion that design is not something that one solitary thinker, so that's supposed to be my rendering of the rodin thinker, but you can tell I'm not a very good artist. But as long as it gets the message across, it's not something you do in your chair. It's something that you do in conversation, in discussion and dialogue and debate with others. The problem is if you pick up a book on design and you try to see conversations mentioned in there, I don't think you will find much mention of that. So you pick up any book about software design. I have read a few, older ones, newer ones, the book on design patterns, some books from the old times. There's a very good one which is called Essential Systems Analysis, which although it has analysis in the title, it's as much about design. But what was apparent in all of them is kind of a single-minded focus on design as an individual activity. So I think that's something that's still in the future. Somebody is going to reinvent design practices that speak to design as a collective activity. And maybe that's going to come from the other design disciplines who are probably closer to that than we in the software development community are. And this one, I think, still has to be addressed at all. Design is not about properties of the code or it's not about documents. So that's a very deep-seated one. You can see it. Who's heard about cyclomatic complexity? Anyone using Sonar? I feel bad for you, because I think whatever version doesn't matter. I'm sorry. The problem with tools like Sonar is that they focus on the code as if code was something like metal or plastic or a substance with its measurable properties that you could predict if you can measure the melting point of plastic. You can measure the flash point of an explosive substance. And you can use that to make predictions, to say, if I put this in a liquid or in an environment which is at this temperature, then it's going to melt or explode or something like that. And what I think, what I see behind the ideas like Sonar, is kind of trying to have the same predictive ability. So if I have a piece of software that has this much cyclomatic complexity, that's its flash point. It's going to explode into a shower of sparking bugs. So that's what I call the code as a naturally occurring substance fallacy. But design is not about that. It's not about achieving that kind of predictive power about how this stuff that's out there is going to behave. Sorry? Let's save that, because I'm going to bring my side of the argument. And then when you have that in mind, I think we'll be in a good place. But because I'm already anticipating at least part of what I think is a reasonable counter-argument, which is, well, if it's not about the code, we spend a lot of time in there, down in the source code. So if it's not about the code, what is it about? And I'm glad you asked the question, if you didn't. But I'm glad you asked the question, because I think it's about brains. What's the salient fact about software development? It's that until they manage to automate it, and I think good luck with that, it's human creative activity. So it's performed by a specific piece of hardware, which happens to be legs and arms and a heart and a stomach. But possibly the most interesting part of that is near the top, encased in a protective shell of bone. And it's the brain. So I want to talk, we've talked about design. I want to talk about your brain. And so most of you haven't seen Linda Rising's talk, but I'm going to be talking about things, and you will think, oh, they apply to other people. So I'm using the expression, your brain with deliberate intent. So I'm going to be talking about things that happen in your brain, that 99% sure apply to you as well as to me. And hey, it's great, we have something in common. So what's the salient thing about your brain? It's a system designed by evolution. Evolution is not smart. Evolution is patient. Very different things. So your brain is the result of, I hope I'm offending anyone with this observation, but that's what I happen to believe. Your brain is the result of millions of years of evolution. And evolution is patient, but it's stupid. So your brain is riddled with bugs. And it's maybe a pretentious thing to say, but I'm tempted to hypothesize a law, the law of software quality at Saragant. But just to try that on you, bugs in the code always come from bugs in the brain. I know that's a strong statement, but I'm making strong statements so that they can be disproved, so we can argue about them. I'm not attached to those opinions. They are just my best guess at the moment. So everything we need to know about software bugs we can learn, maybe not everything, but a good deal of what we need to know about software bugs we can learn by looking at the bugs in our own brain. So you're going, what does it mean by a bug in the brain? Maybe some of you already know, but I'm going to talk about that. So you've seen Mary Popendick's talk, have you? That was the day before. So maybe not as much of an overlap between the management, agility, and the technical agility. And that's a shame. It's interesting to straddle the fence, to see both sides. But she talked about something that's come out of recent research, which is that the brain has two systems, a fast system and a slow system. The fast is heuristic. It's fast and frugal. It goes fast, but it makes lots of mistakes. And it has a slow, more deliberative, more analytical system. So that's one very broad way of looking at things. So I'm mostly going to be talking about mistakes that we make when we rely on the fast system. So one of them is what I call overinterpretation, the fact that we do not tolerate ambiguity. I'm going to talk more about that. Another interesting one is the mind projection fallacy. We project. What I mean by that is if we think something, if we experience something in our brains, we tend to believe that it exists, that it is in the world out there. And actually, that's false. So I'm just going to prove that to you now. I'm going to introduce you to my girlfriend. She's nice, though. She's fully clothed. I want to, so I'm going to ask you to raise your hand in a few minutes, but don't do that right away because I want everyone to do it at the same time. So instructions. If you see the dancer going around this way, pivoting on her left foot, please, when I say, not now, raise your left hand. If you see her pivoting the other way on her right foot, raise your right hand. Now, look around you. Keep your hands up. And you will notice something interesting, which is that. I didn't get that. You're not all raising the same hand. That's the interesting observation. But you're all seeing the same thing. I showed this to my children. So I have three kids, boys. And they all have the same reaction. They say that the truth comes out of the mouth of babes of children. And they were looking at the image. And they saw the dancer reverse directions. Who has seen this yet? Not everyone, but some of you will see it happen. It will kind of take you by surprise. And maybe you will have the same idea that my kids had, which was, Tandy, you changed the picture. Of course, that's the natural explanation. But no, I did not change the picture. So what's going on there? It's interesting. What's going on there is your brain is lying to you. This image is really, I mean, objectively, insofar as anything can be known objectively, this image is ambiguous. It has no z-axis information. It has no depth information. So some of you maybe do image processing. All the information is flat. So the image could be either the projection of a spinning dancer going clockwise or counterclockwise. It could be either of those. So normally, if your brain was in the business of telling you, so some of you are going mad now, trying to make her change directions, the tip I can give you is you have to narrow down your vision to just the foot. And you have to make an effort of imagination to see it as a left or a right foot. And it works. And when you master the skill of making her change direction and purpose, you will have learned something totally useless, but you will at least have learned something. So your brain is lying to you. It insists that you are seeing her spin in one specific direction. And actually, the reality is otherwise. So that's, I want to say, QED at this point, proof. Anybody disagree with that? Anybody disagree with me that your brain is lying to you? OK. Well, so you could retreat and say, it's not really my brain, it's my perceptual system. But the rest of me is fine. OK. So let's continue talking about bugs, which are those bugs in the brain are collectively known as cognitive biases. So I'm going to talk briefly about three of those, respectively known as confirmation bias, anchoring, and the Dunning-Kruger effect. Now, it's not just those three. There are many, many more. So just to start a list, groupthink, narrative fallacy, illusion of transparency, change-bindness, availability heuristic, learn-helplessness, conformity bias, bias-tender effect, there are many, many, many of those. There are many very solid studies which prove those biases, those bugs exist. So I mean, cognitive scientists, they would make good testers of software because they know how to really reproduce a bug. They know how to isolate a bug. They know how to really, really narrow things down. So they have definite proof that it's this thing that's going on and not something else which looks like it. So these guys are good. We could learn from those guys. But do we learn from those guys? Have you read any? So who's got a formal education in software engineering? I can't raise my hand. Well, I have a degree in software engineering, but I'm not a didact, which is why I get away with studying things I'm not supposed to. So those of you who have formal education in software engineering, have you read a paper about cognitive biases as part of your studies? And that's a shame, because we could really learn a lot from those guys. But the amount of ink devoted in all of software engineering as a discipline, it's not zero. But the amount of ink devoted to those topics is, I think, about as big as that dot on the exclamation point that we just moved away from. So very, very tiny. There are a few guys who have been saying for years and years that we should pay more attention to psychology and cognitive science. And pretty much nobody listens to them. So what can I do? I can only try and continue what I've started doing, which is to make you experience some of those cognitive biases. That's the next topic you're ahead of me. So I'm going to try and be quick with those. So we're not going to try this experiment live. I'm just going to tell you what it is. But any testers in the room? OK, confirmation bias. I'm going to directly address the equivalent in software. And then I'm going to tell you what this is. When you write a piece of code, and then you try to think about ways that it could be broken, you're going to think almost systematically about tests that verify that it does work. You're going to be thinking about positive test cases. A guy named Peter Wason did and repeated many times this experiment where he shows four cards, three k, eight a. So the original cards were actually, I think, a, k, four and seven. So I don't know why he picked something that spells the name of a gun. But I wanted to avoid the gun reference. And he asked people to check, to validate or invalidate a rule. And the rule was, so those cards have a number on one side and a letter on the other side. And the question was, which cards do you need to turn over to check whether the rule that occurred with an event number on the front as a consonant on the back? Or by Wason? OK, so there are different answers and only one correct answer. So some of you are wrong. Sorry to break this to you. And so he tested various hypotheses, like are people misunderstanding the question. And frankly, I did not take special care in formulating the question, so those of you who are wrong now, it could be not because of confirmation bias, but because I'm confused about asking the question the right way. But what matters is you can look at the paper and you can look at everything he did to make sure that it was this one thing and not something else that was going on. But let's try and reason this through. So do you need to turn over the three? No, why? Because it's not an event number. So it doesn't matter what's on the other side. The rule only talks about event numbers. So we do need to turn over the eight, because if it has a vowel on the back, then the rule is disproven. Do we need to turn over the K? Yes, no, yes, no. I kept your minds. No, we don't. Because whatever is on the other side will only confirm the rule, so anyone who thinks that we do need to turn over the K, I strongly suspect that this is confirmation bias talking. We don't need to turn over the K. If it's an event number on the back, then it's a confirmation of the rule. If it's an odd number of the back, as per the reasoning earlier, we don't need to turn it over. Because it's not covered by the rule. Do we need to turn over the A? Yes, because if it's an event number on the other side, then the rule is disproved. So you can see right there that it's a hard task. Interestingly, there is a way to make this task a lot easier, even though it would be structurally exactly the same. And can anyone guess? I'm sure you're going to be surprised by the answer. OK, so you're all programmers, right? So you know, you can tell me that. If I change the name of a variable, does the program behave in the same way? I change all instances of variable foo to bar instead. And there are no other variables named bar in the system. So there's not a problem of aliasing or conflict. You need to go faster. But this is so interesting, right? So we change a variable name, and there are no traps. Does the programmer the same? And the mind doesn't work that way. So if I were to change the system to say, instead of event number, I'm going to say a human aged older than 21, OK? And instead of odd number, it's a human aged younger than 21. Instead of a consonant, I say a human carrying an alcoholic drink, OK? And instead of a vowel, it's a human carrying a non-alcoholic drink. And the setting is a bar. So this presentation does include the joke. A man walks into a bar, and he sees a waist and test. It's not a funny joke, but it's still a bar joke. Now, if I ask you, who are you? So you see a young guy, and you see an old guy. Basically, if you ask someone, are you a good driver? Are you a good programmer? People tend to systematically rate themselves as being better than an average driver, for instance. So probably it applies to programmer, but I don't think it has been studied formally. People systematically tend to rate their own ability the higher, even as they are, lower in the actual performance. So that's called the Dunning-Rueger effect. Anchoring and priming, those are fascinating. The classic study on priming, sorry. And there's an experiment, but I'm going to skip that. I can tell you about it later. I want to show you something else instead, which is, I think, it's worth a laugh. Some of you may have already seen it. If you have, please don't spoil it. Because for anyone who hasn't seen it, it's a very nice surprise. But I'm getting ahead of myself. So priming, they did this experiment with a control group and a normal group. So it was a controlled experiment of students. And they asked the students to perform a word test. So put a sentence together. Very simple test of linguistic ability. But actually, what they didn't tell the student was that the experiment was not the word test itself. It was what happened when the students walked away from the room where the experiment was held and back to the elevator, which was at the end of a long corridor. And what the experimenters did, sneaky people. In one half of the word test that they gave to people to make a sentence together, there were words like pension, Florida as in the US. So that fluoridize where people go when they're retired. Dentures, false teeth, stuff like that. So what does that make you think of all people? And the control group was getting words like apple, bicycle, stuff like that. So what do you think happened? Excellent. Good prediction. But at the time when they did this experiment, that was extremely surprising. Because priming effects had already been approved and studied. But the dominant expectation was that if I'm talking with you and I'm sleeping in some words in the conversation, it can maybe affect your verbal behavior. So you're going to talk to someone else. And maybe you're going to use the same words in imitation. But not your behavior. People were sure that your actual motor behavior was not dependent on priming effects. And these guys proved that, indeed, motor behavior could be influenced by things you had no idea were influencing you. And so linearizing in her earlier talk was talking about estimation, which is an interesting problem for programmers. And she was showing how anchoring and priming effects could change the estimates that you give. So for instance, the weather outside may change the estimates you give in an estimator. That's something. We think we are giving only rational answers to rational questions. But actually, we're influencing all sorts of things. And all because I'll save the surprise for Christmas if you open the presents as late as possible. But I want to talk about practices. Penning poker is about conversation. So conversation ties back into how long is it going to take if we take this decision, take this other decision. And that's design. But it's not the main reason I want to talk about Penning poker. And then we're going to talk about future directions. And maybe that's also something we can talk about after what I mean by that is whenever you grab me in the hallway or at coffee or because you've been thinking or understanding is correct or you want to run some ID past me, and I will be available. The thing about the current design practices is you can pair them together with Penning. First example I thought of, which was the first inspiration for saying, yes, I really need to give a talk about this and tie the things together for other people so that we have more brains working on that particular way of seeing things. Because clearly it's producing interesting results. When I came across TDD, I thought, oh, genius. And then you write the code and then you refactor. But it's not what I'm. The focus is going to be on test first code later. So who does that? Not quite the same thing as a write code. And I also write unit tests. Because if you write the unit test after the code, you are not going to get the same good results as if you do test-driven development as it was introduced by Kenbeck, which is write the test first. Write the code after. Why is that? The answer is on the side. Confirmation points. Who said there were a tester? So do you know a guy named Landford Myers? So that has a list of things that equals the actions of something like this. I'm not sure if you remember axiom number four. But raise your hand if you have heard this. A developer should never test their own code. Who, wait, wait, so who believed it? I can tell you this is a little confession time. At one point, I was very loud when arguing with my management. No, I'm not going to test my own code. Everyone knows that's not something you do. It's stupid. It's a bad use of my time. We should have an external testing. And obviously, that's one of the things that has changed with this reprogramming in general. We're much more into developers that were designed. Testing the way a developer does it. Testing the way a developer approaches it. Go figure. Whenever someone says something like that, which is a very sweeping claim, and they have a because, in general, what history remembers, what we remember, is just why this inversion is so powerful. So that's the data figure that I was, oh, yes. That's the reason. And maybe that's not the whole reason. Maybe I'm just making a mountain between the world software and the world of cognitive science. And there are others. I think about their programming as something that is a huge antidote to the Dunning-Kruger effect. What happens when you have a young, novice, inexperienced programmer, and you ask them to write some code on their own? They think they're a hotshot. They think they're the best programmer in the world. That's the Dunning-Kruger effect at work. And so if you ask them to do a code review, what? No, I don't need a code review. My code is perfect. Who's seen that? Or something like that? OK, some of you are lucky. You have young developers who know proper humidity. Maybe I don't know. But nevertheless, I think that's a, and maybe the rest of you are already doing programming. So what you do is you pair a more novice programmer with a more experienced one. So the more experienced one checks that enthusiasm. At the same time, he sees transferring knowledge. But another thing interesting happens is even the more experienced programmer is actively learning something from the younger because he has to bring his ideas down to the concrete level where the younger one can actually understand. So teaching something is a great way to understand it better. Which is also why programming is done by smart people. Because you're teaching the dumbest thing there is, which is a computer. So whenever you have to transfer some knowledge from your head to the computer, you have to be very precise about what you think. So that's why programming is a great way to debug your thinking. We should do more of that. We should do more debugging of our thinking, which means we should be able to reverse engineer the bugs, reproduce them, know about them, install patches for the bugs that we cannot fix. Ah, planning poker. Planning poker is an interesting case. I wanted to at least have one example where the agile practice. Because I don't want to spend one hour saying, agile is great because it cures all confirmation bias. Who's going to believe me? Some agile practices are actually quite stupid when you think of them in the context of cognitive biases. And that's also, I think, why we should learn more cognitive science. Because there are some things we've been doing which are stupid, and we should stop doing them. And I think planning poker is one of them. So maybe some of you are mad with me now. But that's OK. Planning poker is problematic in the sense that it plays right into anchoring effects. So anchoring is a slightly different thing from priming, but it's the same deal. If I want to get a raise from my boss, so that's a tip. Maybe some of you will find useful, even though it's nothing to do with design. But I will gladly share it. You go to your boss, and you ask for a very high raise. You say, I don't know how much is a typical raise here, but you ask for a 30% raise. And that's your opening bid. And of course, you're going to, times are tight. I can do that. OK, how about a 20% raise then? And maybe you will get a pleasant surprise, because the boss tells you, OK, that's what I have to do to keep you here. I'm going to give you a 20% raise. What you have done is you have anchored the negotiation. It's a bad thing. Don't actually do it, unless you have to. You have anchored the negotiation at a starting number, which is higher than it would otherwise have been. And maybe your boss knows about it, so the second you step into his office, he says, I know you're going to ask me for a 5% raise, but forget it. We don't have that kind of money. You have been anchored. So it works both ways. And what happens in planning poker is that someone gives a number. I think this story is a 13. What happens? Cognitive bias kicks in. Everybody anchors on the 13. And what's worse, the rules for planning poker make this even worse, because they say, if there are people in the room who disagree with the 13, we do a second round, until everyone converge. What happens? The people who are the most convincing, even if they're wrong, they are the ones that everyone converges on. So instead of having a correct estimate, you are encouraging people to play for the leader, basically. OK, that's one of the things we can talk about. I have very little time to cover future directions, but I actually prefer to do that in the informal conversations. But something that I think is a good idea, you all do retrospectives. Try this. Do a design-themed retrospective. Do it the exact same way you would do a normal retrospective, which is don't do that with a code in front of you, don't do that over email. Get people in a group, in a room, and get them to say, what do you think is working well with our design? What could be improved? What have we learned about design in the past iteration? Try that. I think it's a potent combination, because it combats during rigor effect for one, because you have a variety of perspectives. It brings cognitive diversity, which has been, you can always write to me and email me, and I will send you pointers to research. Diversity is generally a positive thing, so it's an untied-out for group thing. One thing that I think works great is the coding dojo. I've seen people who participate in the coding dojo become better programmers over time in an impressive way. Perprogramming, I think the active ingredient of perprogramming, and we use that in the coding dojo, is what I call programming out loud. So that's where, instead of being in the zone, with your headphone zone, and you're just typing away, hammering at the keyboard, you try to explain to someone else what is going through your mind as you program. So that's what happens in perprogramming. But you can reproduce that in other settings. And so I think coding dojo as a way to practice programming and design moves is a great thing. So you put a group of people, 10 people in a room, and you have only one pair programming, projecting the screen on a projector. And everyone is allowed to interrupt as soon as they don't understand what's going on. That's basically the rule. There's a little more details, but I just want to throw that as a teaser. So that's the kind of thing that has already been invented. But really what I can't wait to see is what you will invent with this new weapon in your arsenal, which is, let's go see what the cognitive science people are saying about how the brain works. And let's figure out how we can apply that to software design problems. So to close, the way I want to summarize this whole talk and tie everything together is by saying to me, this is the new understanding of design. Design is about practices that help your brain deal with code more effectively. And that's where I'll stop. Thank you all. The answer is 30. It's a bit dark? OK, so it's easy to miss something that's dark on a dark background. Well, maybe that's posed the effect of it. Who knew about this one already? But anyway, at some point, he was walking in front of someone wearing white. Did you notice anything? So maybe there's, with the contrast, there's something of a confounding effect, as they call it. But this is such a fun video, I always show it. So another kind of cognitive bias, attention bias. I don't want to keep you from the coffee break. Thanks again. And feel free to come talk to me any time now or later to grab one of my cards, or to ask me for one of my cards. And see you tomorrow when I'm doing another talk, if you're interested. Thanks.