 Welcome everybody. The title of the session is don't fail fast, learn fast. Failing fast is an option, but innovating as fast as a must. And in this we'll be trying to be as interactive as we can. And we'll be trying some experiments, so we hope you'll enjoy the ride here. Yuta. Yes, also welcome from my side. So probably you are here because you have seen what are the goals of this workshop. And when we were preparing for the workshop, we found out, well, this is actually a lot of stuff, what we put in here as goals. And if you really closely look to the different goals, you will find out that actually indeed it all sums up to understand how to learn and innovate fast. So this is what we want to discuss with you today, what we want to address today. And here is our little agenda. Right, and the agenda besides doing the welcoming that we're doing right now is we'll be setting a context. What does it mean to learn fast and fail fast? And we'll be looking at frameworks for thinking and we'll be touching on something called transformative learning, which is different than ordinary learning that's cumulative learning. We'll be looking at something called probes. What are probes and breaking them down into designing an experiment and measuring things so that you can see the whole cycle of how you try to learn fast. And so Yuta, over to you. Yeah, so we would like to know from you, what do you regard as the biggest challenges for companies today? In order to do so, please go to menti.com and use the code 1701384. And if you go there, then what you do is you just enter, for example, a word, or also you can put like two words there or if it must be a sentence, but it would be better like key words here. And I believe you can put in up to three different words. And we are waiting here for you to hear what do you regard as the biggest challenges for companies today? I just saw that Cindy passed provided the link. And here is the code again. Yes, now I see. Yes. Okay, we have disruption here. Change management is a challenge for companies covered for sure. It's a challenge for everyone for the whole world, no matter if you're a company or an individual, I assume. What else? Again, go to menti.com. There are more people have got there. Yeah, Cindy was headshared the link above. Limit work and progress is one thing that people start to work on a lot of stuff at the same time. So parallel work is something that's happening all too often. Then we have sharing purpose across, which is kind of avoiding silo work. That's at least how I interpret it. Then, oh, the new norm. I recently heard people saying, oh, I can't hear that term anymore. Yes, adjusting to the new norm. That's quite true. Then connect or connection. Yeah, probably despite the thing that we can't see each other right now, creativity. I like that very much. It's nice accountability. Communication always a topic. I remember the late chair, Weinberg, who once said whenever you see a problem and people will say it's a technical problem. I'm sure it's always a people's communication problem. I guess that's true. We see more coming up. Empathy, innovation, modernity, and slow. Okay. Yeah. Culture, burnout, deliver value. What else? Skills for future. Yes. Is there anything that jumps to your eyes, John, that I have? One thing that jumps is that nobody is repeating the same word. It seems so. Yeah, there is. Repeat the same word if you want. Vote for that. Yeah, that's true. Yeah, it looks like everyone comes up with something different, which is also interesting. Maybe that's also a signal of the time. We also have a varied audience, I think. Maybe. So that this is why different things are coming up. It seems it's slowing down. And now, well, just because one word has been repeated, it's the learning part. I really like that. And well, of course, I guess we have to like that because the topic of the top workshop is don't fail fast, learn fast, right? So learning fast is really at the core. And if you are not interested in learning, I guess you wouldn't be here. So that makes perfect sense. So I think I move on. You can still keep bringing them in. That's fine. We keep those and we'll also share the slides, of course, afterwards, including your challenges that you found. And I want to move on to the next thing, which is actually probably it's more the point, but we used to name it before the pandemic. We were all talking about we are living in this WCAW world. Now, a lot of the things you were coming up with are kind of a different way of expressing this WCAW world, which is that, well, everything is just volatile. It's uncertain, complex and also ambiguous. However, being really honest with you, although it is John and I, and I know also a lot of other colleagues of us, they are talking about the WCAW world for, it seems like, ages. And it always had been, I don't know, almost a bit artificial to explain what it really means. Of course, we could say, or we said, well, there was like Airbnb disrupting the market or Uber disrupting the market and this and that. So different possibilities for different markets, but still people were like giving us that stare. And so if you will, the pandemic now is the best example for the world because companies have put so much effort into coming up with plans for 2020 and at latest in March, all the plans showed like there were a waste. All the effort that has been put into those was just a waste of time and money, of course, as well. So just think about like budgets for this year. It's just not anything like what people have imagined that would happen. And this is really probably the best example. And now we don't have that many problems anymore explaining what WCAW really is and does to us. Well, the pandemic is really to the extreme, but on the other hand, we are facing stuff like that all the time. So the WCAW world is actually asking companies to be agile. And what we mean with agile here is really not that companies need to use scrum all over the place. What we mean with that is that they have to be agile in the literal sense. And the literal sense means they have to be responsive, flexible, fast, timely, nimble, all of that. And so actually companies are asked, well, I said like being agile, but actually it's thinking beyond agile. And one way of doing that, thinking of beyond agile is looking at what John and I created, the Bossa Nova, which brings together different streams and allows people to expand their horizon by looking at, okay, is there a different way how we can do budgets? Can we do those without like fixing them one year in advance? So that's the B for beyond budgeting. Oh, is there a way how we can leverage the full potential of every employee in order to have the full innovative power of everyone by using the principles of open space? Or how can we use the power in a way that it's really shared and decentralized so everyone can make the decision about the stuff he, she knows the most. And then of course, there's agile, which brings in mainly the inspector. So this is kind of a glimpse of what Bossa Nova offers. But the key thing is thinking beyond agile, thinking about what other different streams, frameworks are offering and bringing those together in order to expand your horizon. Now we want to look exactly into this, into like expanding your horizon. Great. So I'm going to use a little story to illustrate that. And because everybody reads at different speeds, I'm going to actually read the story out here and let you think with me about what does this have to do about horizons. So here's the story. A clever little girl and her father were shopping in a little neighborhood store. As they packed up their purchases, the owner of the store offered the little girl some free sweets in a bowl. Get a handful of sweets, the merchant said to the girl. The girl just stood there looking up at her father. The owner repeated himself, honey, take a handful of sweets. They're free. Again, the girl did not move, continuing to look up in the face of her father. Finally, the father reached into the bowl and got a handful of sweets and put them in a bag. As they walked home, the father asked his daughter why she didn't take a handful of candy from the bowl. The girl, with a big grin on her face, looked at her father and said, dot, dot, dot, what do you think the girl said? Okay, so in the discuss, why don't you put in some thoughts about what did the girl say? Excuse me, what did the girl say about why she didn't grab a handful of candy? So use the chat for that. Yeah, so, how? We have a few answers here. I, and I'm, okay. And see, if you go to audience public, you see them. So one is your hand full is smaller than mine. That's hand is bigger. Very good. Very good. Everybody got it. Yeah, many people got it. Right. Many, many people got the answer. And many people don't get the answer. But what we have is, even though you got the answer, is perhaps a framework shift when you're, when the, you may have been seeing the little girl is like, what's wrong with her? And then when it's, you see that, oh, she, she had a purpose that you now understand. That's called a framework shift. You see it differently. And that's what we're looking for at the beginning of the cycle that is helping you learn fast, innovating fast is a must. So it's that framework shift that we're looking for. And we'll do another exercise a little bit later in the presentation to let you see if you are still, or show you a framework shift in a different way. Right. So we also would like to bring this a little bit back to where you might be able to use stuff that we are sharing here right away. And for doing so, I would like to do a little recap with you on retrospectives, which is one of my favorites anyway. So it's, I always love to do that. So a retrospective, as you probably know, means that we are regularly reflecting on what's helping us, what's hindering us and how we can get more effective and also more efficient in what we are doing or if you will, how can we deliver a better value to the customer and whatever better means in our context, it could be faster, it could be more innovative, it could be anything. And a retrospective, that's the recap, is defined by those five phases. If you haven't seen those before, I highly recommend to you go and get the book from Diana, Larsen and Esther Derby, where they describe actual retrospectives and where those phases are in. So you start with a welcome and set the stage, then you gather data of if it's like a sprint retrospective of your last sprint. But you can also look at a different timeframe, but let's stick with the scrum process. So we gather data of the last sprint, then we start generating insights. So what does the data tell us? So it's a kind of a reflection based on the data that we've collected. So people might have found that, well, the tests are always failing or whatever. And so we generate insights, okay, what does that mean and does this happen all the time or just occasionally and so on. Then we decide what to do. So it's coming up with actions and for those actions, we often then also have like a commitment. Is this really what we wanted to do? Is this the difference we want to make with, by implementing those actions? And then we close and, yeah, this part and we meet next two weeks later. Now, the thing, what we think is, well, maybe I go to this later, but I just thought, sorry, stay with me. So I first have a reflection for you. Think about three actions that have been decided in your last retrospective. If you're not doing retrospectives then I'm pretty sure you know other kinds of workshop stuff where you come up with any kinds of actions and just think of those. So think about three actions decided in your last retrospective. And if you want, we would be happy if you would share one of them or so in the chat, but you don't have to. So we know sometimes it's company confidential. Really give you a little bit of time to think of that. These are already wonderful examples that I see here. You all are able to see them as well. So I'm not reading them all out. I don't know. Let's take the one that I'm seeing right in front of my face, which is from Ratchiv, more time for refinement. So that's an action. Actually, I have seen more often that teams are coming up with this, which is also true for some of the others that I've seen here. Kind of similar things that we've seen in similar teams. So the kind of the problem that at least we are seeing here, that very often we gather data, generate insight, decide on actions like the ones we are having here, like more time for refinement. But what's actually missing is, on the one hand, what's our hypothesis? What do we think we will achieve by doing, implementing this action and maybe even more important, how can we measure that this action actually has made that difference? Or maybe it was the wrong action or maybe it was the wrong hypothesis and the problem is actually somewhere else and we need a different hypothesis. So the thing that's often missing, not only in retrospectives, but retrospectives are a great example for we are a way to often jumping into actions without reflecting on what could be our hypothesis and how can we measure if we made a difference? So the title of the presentation is Innovating Fast. But so why don't we just jump and do something? So this is a little bit of let's go slower to go fast. In that list that you just saw, you saw that there was a gather insights and that was kind of the point that I was making with the little girl and asking her father to quietly to be the person gathering the sweets. That insight shouldn't lead just right to action, not only when you're in a retrospective, but at any time that you're thinking together, it doesn't have to be a retrospective. Having the discipline to say, huh, that's interesting. What if is the beginning of developing a hypothesis. So you need to act, but you need to act in a very deliberate way and that's the key to innovating fast is to have this discipline that we're now getting into. What is a hypothesis? We've said using Rajeev's example, we need more time for refinement. Okay, then the hypothesis might be that if we take more time for refinement, then we will have a more effective sprint the next time. And now you need... Oops, there we go. So the first thing you say about, gee, what if we try more time for refinement is exactly what to expect to happen. Let's be precise and what conditions is it under. So we expect maybe that we complete the sprint completely or what is it we expect to happen because that then becomes the basis of how you're measuring. And under what conditions? Is it that everybody is present in the sprint all week? Is it that we don't have a change in the way the server is operating or whatever the conditions are? And so you need to come up with those kinds of statements to be clear what is it you really think is happening. Our hypothesis is that if we do this under these conditions, then such and such will happen. And then you have to design an experiment because you're trying out things fast, but you're trying them out small. So how is it that you're testing your assumption and how is it that you're measuring? Well, the first thing is that you need to measure before you start and be able to look back if it's needing time for refinement. How much refinement time did you have? How did you measure it? What constitutes refinement time? And then of course you have to measure afterwards and then it really helps if you have some way to compare. Like maybe you have a similar group that isn't doing this and you can compare so that you really have something to see and some way to know what really happened. And whatever you come up with should be small, should be very specific, and it also should be something that you can do. So it's not some grand idea. So this is what we're trying to do in the full cycle of things. Reflecting on your situation, typically it's what you might do in a retrospective, but it can be outside of that entirely. It doesn't have to happen with a programming team or in the context of a sprint. And getting the insights is when we saw the steps that you take for a retrospective is reflecting on your situation. What we're introducing next is to say design a probe. You've all heard of Snowden talking earlier in the conference about probing to make sense of what's going on. And you're dealing with complexity almost always when you're trying to think what do we do next. So you design a probe. A probe consists of a hypothesis and the design of your experiment altogether. So you have to design that experiment then try it out for the specified period of time. And after you've done the experiment there are two things that happen. One is that the inner loop here that you see, you have to have that lead you to the next time you get together to think, huh, we tried that experiment, we tried more time for refinement, and what happened? Maybe good things happened, maybe it didn't happen, but whatever happened happened. It may be that you didn't get what you expected, something else happened. But you also need to publish it to your peers. Now that can be just the group in your office. It can be in the company newsletter. It could be in the Agile Alliance's publication system. It can be a paper that you do. It can be something through your network. But it's really critical to also publish what you're doing to the extent you can, given company confidentiality requirements. Publish what you're doing to your peers. And why? Why do you tell other people? It's so that they can try to replicate what you're doing. And when they do try, you learn from their attempts to replicate it. It's kind of like asking, is this reliable, what we've come up with? Is it valid? How does it vary, given other situations? And if you're publishing to your peers, then maybe they'll publish to you and you learn from them what they're doing. So this is the full cycle of being innovative. And it takes some discipline that you're not necessarily used to. You tell me. Go ahead. We would like to look at how is such a probe defined? Well, again, as John said, you heard it from Dave Snowden and also what John was kind of, he was giving you a peek into that. So the definition of a probe in this context, in order to learn fast, is that a probe is defined as more safe-to-fail experiments and they're based on hypothesis, well, we have talked about hypothesis already, that are derived from reflection on the current situation as well as on theory. So most of it, I guess, should now sound a little bit familiar. So for example, derived from reflection on the current situation, that's actually what gather data and generate insight does for you in the retrospective. If you're using this approach, what we highly recommend also outside of retrospective for coming up, for example, with new product ideas or so, then you still need to have that thing where you have time for reflection on what's the situation, for example, for the market for a specific customer or maybe for yourself. And also in theory, so what is out there and what do you know from maybe science about that topic you're reflecting on. And then you use that reflection in order to come up with your hypothesis, so kind of a what-if thing and John was elaborating on that. And then again, based on the hypothesis, you come up with the experiments. Now, I think we haven't really talked about safe-to-fail experiments and there are different things that we mean with that. So one thing is, safe-to-fail is in a way that, well, actually in our opinion, an experiment embedded in a probe never really fails because if it's not working out the way you thought it will, the only thing it is, it's a success because it might invalidate your hypothesis or it will tell you it's the wrong experiment and that's a success. It's not a failure because you gain more knowledge and that's especially coming back to like these times, the WUCA times or the pandemic times, this is super important. So this is one way of looking into safe-to-fail. The other thing of looking into safe-to-fail means also about who is affected by the experiment or maybe even exaggerating a little bit or I hope it's a little bit exaggerated that the people you are doing the experiments with and again, remember that's exaggerating how I just phrased that because if you think of it this way, then you kind of look at the people as maybe guinea pigs and this is not what we mean here by no way but what we mean actually again, having it as safe-to-fail that it's safe for everyone to say well, I would really like to try that and no matter if the experiment then validates the hypothesis or invalidates it, the person is fine and also if somebody says well, I'm not really in for that I don't want to try that then there's also no punishment so no punishment for either taking part in the experiment or not and that sometimes I think ignored because sometimes, especially in the HR role I sometimes feel that scrum masters, HR coaches come up with ideas and kind of put them on the team and not really leaving the the leeway for them to decide what do they think is safe-to-fail so keep that in mind so this is another maybe angle to a safe-to-fail experiment I was just going to give an example of a safe-to-fail experiment let's say that you decide you're going to change you're doing performance evaluations of everybody and it affects how much money they might make and if they engage in the experiment what happens if the experiment doesn't work do you punish them or they get less money under the old system so you want to be really sure that people are not going to get hurt by what it is you're doing that would be an example of a safe-to-fail experiment very good, thank you so we would now like you to do another little reflection which could be also related to a retrospective because I assume a lot of people are doing here from the audience but again it could be also a different context than a retrospective so come up with a challenge and it can be key or not that you try, you can be you or your team probably it's more that your team try to solve more than once so think about something well the way it sometimes appears is that it is a topic that pops up in almost every retrospective and maybe never it's a trust or maybe it is a trust but it doesn't go away so next time it pops up again and I would like you to give a minute or so to reflect on it and again if you want to share anything about that in the chat while you're reflecting the chat is open and while you still maybe think about this and reflect on it so my question oh there's Dolly, hi Dolly challenge, have I held the backlog for example stories in ready state and can be accommodated from when two to three sprints is a challenge that came up in here yeah that's cool to see you oh well we don't see you so the question that we are having right now is what was it like thinking about that thinking about a challenge a challenge that you try to solve more than once and we would like you to raise your hand if you want to share what was it like thinking about that and maybe also elaborating on that challenge you came up with so if you raise your hand in the Q&A section then this would help us and at the moment I don't do we, do people know how to raise their hands maybe Sandeep you have to share Sandeep says there's a button on the right to raise the hand and now I wonder hey everyone once you raise your hand I'll keep you on the stage and you'll have to accept the invite to be able to speak so please don't forget to accept the invite once I put you on the stage yeah we try that but at least I don't find the raise your hand thing but I hope people do I don't see it either Sandeep oh there was Dolly I think Dolly raised her hand I guess let me see is that true Sandeep is it on the very right side of the screen ah I am hey you dying John yeah ah okay wonderful so Sandeep helped us very good so Dolly can you share with us like what was that challenge you tried to solve more than once and more important like how was it to think about that right now so what was your way of thinking okay the challenge that I wanted to share is it is happening time and again that when we are doing the skin planning we do not have enough stories groomed and brought to ready state which can be quickly picked up and put in the planning we end up spending a lot of time in this planning session more than is that is required and the suggestion is to have the refinements in a way that we do all the grooming and bring those stories to ready state so that we can save time during planning not to discuss everything from the scratch this is the issue that we are trying to solve and it is not just for one spring we have to use refinements in a way that we will have enough backlog items for the upcoming sprints as well so it's also kind of interesting because we accidentally achieved topic before which was more time for refinement so this fits in nicely so how was it for you now thinking about a challenge that's coming up again and again it is frustrating it is frustrating because we are identifying that as a challenge but there is no proper action or not enough action is taken probably you come up with actions did you come up with actions before yes we have set up more refinement sessions but there were challenges in the sense team members thought that there were too many meetings which was kind of blocking them from working on the screen to go so one thing that I am hearing is well it was frustrating and it's not the one hand that's the key thing and probably the frustration came by well you talk about this over and over and come up with one action after the other and probably you also implement the action but still that topic comes up okay I think this is this is good for now is it John you have any more questions yeah I think so no the refinement is a good thing to explore more to see how we can start coming up with something that gets a mental framework transformation so I hand it over to you okay thank you thank you Dolly we remember that we saw in the example of the little girl that we in order to understand what she was doing we had to think differently about what she was doing here she was being asked to do something very logical but it seemed like she was just being shy or something and so our initial thought was okay the little girl is being shy poor little girl but then when we say oh actually she's being very clever then we could think about it the whole situation in a different way so with the refinement question I see I hear that people are saying too many meetings and we then say well what's a different way of thinking about too many meetings oh I'm frustrated it's a waste of time what's really going on here is there is there a way that we can frame the problem or reframe it together so that we can get different insights and one of the things this chart doesn't say is it doesn't ask us to think about how we're thinking that's something called reflexive inquiry how we're thinking about this oh this it's it's too many meetings that we're having well really why what is the it sounds like the framework we're thinking of is that we ought to maximize our time really what happens if we don't try to maximize our time maybe we try to have even more meetings who knows but that that kind of possible hypothesis comes out of saying why are we thinking that way what is it that we're thinking as you think this way there are some things that you'll notice the you were sort of thinking about this today individually but as the team thinks together the with questions like that what happens if we have even more and more meetings you become boundary list and that it doesn't seem like it's your observation anymore but the team if it's really working effectively we'll just feel like hey we're all thinking in this together and you'll be coming up with wow what happens maybe two or three of you could meet even more often and does that lead to better refinement and so you come up with things that you can try again very small experiments to see what it is you're dealing with maybe maybe we do want to have even more meetings and then the you look for views that maybe discomforting or challenging somebody says we shouldn't be meeting at all what happens if we don't meet at all let's let the thing self-organize or maybe the amount of time meeting is not what is the problem maybe we need to be doing something different when we meet and that kind of looking for things that just you know say this is totally this is ridiculous what you're thinking right now what if we think of this other wild thing is the way that you build to the point where you may have a sudden leap of insight and so that process of thinking about how we're thinking is something that you need to do very deliberately in your thought process that that thing about in the steps in a retrospective about getting insights that's not just oh yeah let's you know let's think about this together and maybe throw some stickies on the wall it's very a very deliberate discipline and what you're looking for is not like okay we learned this last week or this last time it's some new framework some new mindset for how it is you're thinking about your problem and before we go into our experiment you two do you have anything to add no I think that's that's it okay we move into the experiment yes let's go into the experiment get set get set ready okay here are three statements you see them there and the question is how can these three statements be true and if you have seen that before then probably you should be quiet but if you haven't right so take a moment to look at this how can it be true I think runchart has already suggestion here yeah he says 6 plus 1 plus 1 is 10 6 plus 1 is 0 somebody else says 6 plus 1 plus 3 equals typo sorry typo okay all right but all three of these statements have to be true yeah but I think I can't follow runchart's thinking which is 6 plus 1 is 7 plus 3 is 10 and 1 plus 0 is 1 I think this is what he's saying well then how can number how can the second statement be true then oh see he's getting somewhere 7 plus 5 is maybe working out okay okay what about the last oh but now the last one he's saying 10 is then 1 last 0 yeah that's what I'm saying sorry runchart sorry yeah so what are some more do you have then also a solution for the third one with the 0 are there any more right so so runchart's idea holds true holds true for the first two statements not for the third one yeah anybody else have any ideas go ahead and make your comments this is something that children generally get faster than adults that's a hint another hint is if you think of the little girl maybe you have to shift the way you have been thinking yeah how can you show and that's me saying I didn't solve it so being very honest here I don't see any more solutions coming here we need to we solve it yeah maybe go ahead shall we try to tell them the answer do they want to know the answer do you think if somebody if somebody would like to see the answer please enter it in the chat oh here we have something oh now it's getting m plus 1 is 8 times i is 40 okay okay we're getting actually high as well so that's true for the third one but probably not for the other two so they should be all follow the same rule in order to be the same consistent logic yeah okay okay all right so let's think of a kindergartner here looking at this and the kindergartner maybe you could I don't know if they can see my arrow you too you might want to put your arrow over the six I think I'm not sure I'm I don't want to touch it to be really honest so it's working right now so look at the six in the first line this has a closed loop a little circle so there's one and the eight the eight has two closed circles that makes two and the one the one seven and five there's no closed circle so that's a zero but the zero has a circle isn't that interesting that's a contradiction so this everybody I think approach this as we have to put an operator in between the six and the one and the three or the one and seven and the five I need as well and that's that's the framework that you are bringing to this so the question to be asking when you're reflecting is well why are we thinking that way and it often helps to think about what would kindergartener think how would they solve that problem and that insight that realizing how you're thinking about things is the key to thinking innovatively fast and it's not necessarily how long you're taking to meet it's really digging down and saying what are we thinking about this we don't have time for refinements is time really the the essence of it or is it how we're thinking when we do get together and how is it that we're thinking when we get together this is the sort of thing that you have to ask when you're trying to get transformative learning you transform by getting a new framework and that's where your innovation comes from so go ahead you too there are some other ideas for solving it hahaha ordering and descending and descending order ascending order is zero I'm not understanding that and neither however the point was leave or try to understand what's the framework you are thinking from and when once you are aware of that you can also come up with a different framework of thinking and this is what we would like you to ask you now so we had earlier asked you to think of a challenge key or not that you have tried to solve more than once and so the example was coming up with the same thing once in a while in your retrospective or repeatedly in your retrospective and it seems whatever you do that thing doesn't go away like what Dolly shared so and hope people are still trying to find different ways of solving it and what we would like you to do now is think of that challenge that's coming up more often and ask yourself how are you thinking about the situation and the you or yours that how am I thinking about the situation you can really also expand that to how are we as a team actually thinking about the situation what are we assuming what would it be like if we wouldn't make that assumption and what do you observe then can you validate that and how would you do that so kind of what is your hypothesis here really and again the thing is trying to first getting aware of the framework you are thinking from and then coming up with hopefully a different way how to think about it so getting away from for example the operators and we would like first to give you a little bit of time to reflect on that and if you are willing it would be great if you could then also raise your hand and bring you up on stage to share how you are thinking about that and I believe we would need your help for that again if I understood how to bring somebody on stage the raise hand option should be on the right side where you are seeing the discuss buttons but the audience should be able to see that and once you raise your hand I will go and invite you on the stage so please don't forget to accept it to be on stage we can rely on you to invite people then on stage and you pick whoever you are the one cool thank you if you don't want to come on stage but maybe share some of your thoughts in the chat that would be fine too so kind of maybe you have an idea on that thing why is it you keep thinking in the same way about that challenge that's coming up again and again so thinking of how John already kind of pointed this out when he talked about the transformative learning thing if I take again the refinement thing often we first start thinking well actually that's a good thing also that's happening right now there's something we don't have enough exactly it's kind of the way I believe what we see a lot so we plan on something and then it's not working out and our first reaction often is we have to get better understanding it's kind of like what Dolly is sharing here in the comment that we don't have enough of those refinement sessions and again maybe that is true and so you can try and have more of those refinement sessions but if it doesn't lead you any more anywhere so maybe that assumption is wrong it's not the refinement sessions in the same way as it's not getting better at planning but maybe you need to do almost the opposite to let go or well here we have another one from Donnie maybe refinement meetings are not effective not productive enough for giving expected outcome so that would be a different way of thinking about not enough so maybe the way the meeting is run is the problem or here what Runchart is sharing how many action items are ideal after retrospective and maybe this is just an open question and actually I can only if I answer that trying to answer it right away fewer is always better a maximum of three because we all have more things to do and sometimes we feel like the more we think of doing the better it is however then maybe we don't have time to do it really in that way that we can observe if we really make a difference here Sean do you want to add something here I think that everybody is getting the point here about how is this reflection process work how does it form the basis of making a hypothesis and the I think it's the key to handling VUCA is to really think about how are we thinking we heard about Snowden this week there's also a complexity theorist named Stacy who encourages that way of thinking a different way of thinking about what Snowden is saying but if you remember that you're in complex situations and you don't know the cause and effect then that may help you really start to do the basic thinking that you need to come up with to do a hypothesis and having said that I think we maybe look at what's a hypothesis look like what are the details of a probe more about a probe and so we've gotten to designing a probe and the I think actually you can handle this designing a probe the discipline here is not just coming up with action items but actually doing the discipline of designing the action item or the experiment that you're going to conduct so you have to name it the better refinement might be something how can we have better refinement it's helpful to put it in a question and you need to put down the context the background of the situation where we're dealing with refinements we have sprints around such and such topic we're at such and such phase we have these people here with so and so only present some of the time whatever it's really good to describe what you're dealing with and then you need to say we expect that if we have more refinement sessions rather than the three we've been having we're going to have five of them and this is what we expect to happen that we'll have the refinement sessions we'll spend another hour in refinement as we did before and the impact will be that we end up with fewer errors when we present the product of the sprint and we'll define an error as whatever we define it and that so there will be more effective in the following way something that we can observe and the observation doesn't have to be necessarily objective it can be everybody feels better but feeling better is defined as you rank it five on a scale of one to five or however you're reporting that it can be an observation of subjective feelings and then the next thing to do is having defined your hypothesis is to define exactly how we're going to do an experiment and see if our hypothesis comes out and the experiment depends completely on the situation but it can be that we're going to split the team in two and have half the team have these extra refinement times and half not or maybe we invite the team next to us to try this out to just try doing the measurements but not doing anything different they then would become our control group and we're not going to do this the whole time in the sprint we're only going to do it halfway through and then we're going to check and see what's happening or whatever but defining that as precisely as you can and putting it in writing you'd be amazed at what happens when you force yourself to have the discipline to really put this in writing so that when you have your next sprint you can look back and say oh yeah that's what we thought at that point in time so this is a discipline it's a discipline that intervenes before having an insight and actually saying okay you have an action item to do such and such and it's the path to trying to explain things better as to yourself as to what's happening Mute, you may have things to add and I'll turn it over to you. Yeah, well actually what you were just saying reminded me of something else which is well we showed this, no I'm experimenting here I think you see the circle again a key thing is and now we just keep a direct perspective for just a moment the important thing is also that you go here and look back in your next retrospective oh what was the action we have decided upon what was our hypothesis how did the experiment come out what do we measure because that's all too often also a thing where we are we are coming up with an action and then there's an extra retrospective where we just start again from gather data, generate insights and defining actions so we also have to look back and see did we actually make a difference here and again I use the retrospective here or we both we are using the retrospective at the moment as an example but this is also true like if you are looking at coming up with a new product coming up with a new design and you are going through like any kind of and then you also here need to regularly come back and look at is the hypothesis actually has this been validated or not did the experiment really provide us the data we were hoping to get or maybe we need to come up with a different experiment so a probe is super useful but only if it's embedded in this circle and you keep coming back to well what was it actually we were thinking and what does it mean for us now so this is kind of what I felt when I heard you that we probably need to stress once more maybe this is the time for sharing a sample probe here and the sample probe is from Ashish Aghava maybe some of you know him and so this is actually an appropriation of the probe so it's I don't know maybe two pages long if you look into all the details and so the name of it is what John said in the slide before it's a question hinting towards the hypothesis how can we able teams to discuss openly their impediments and improvements in retrospectives and then the background is agile is being followed in most teams and organizations now they are also doing retrospectives but most of the time the right set of impediments and improvements are not the output of these retrospectives meetings as they have a fear that if they speak up so this is kind of an observation of the current situation or the outcome of a reflection really and then the hypothesis here is if we do not have managers in the retrospectives then the team will open up with their impediments and improvements as they will feel safe and actually well he's more putting another reflection on like what I would normally say that the last paragraph should be okay what is actually the experiment on the other hand the hypothesis provides already a hint towards which is like not inviting managers at least for some of the retrospectives and then see what's happening now one thing you might wonder is how is exactly this probe related to innovation at least I wondered about that after we have decided on this and then I thought well isn't this exactly the case that we see way too often that innovation doesn't happen because people don't feel safe to speak up so again it doesn't have to be something in a retrospective it's also if you think about being innovative around a product and even this can also happening in a retrospective so hearing various voices feeling safe so psychological safety are really big topics for being innovative and maybe you want to come up with a probe that helps you create that environment where that is possible now one thing that I definitely want to share here and to be honest I don't feel safe enough with the platform we are using here and therefore I'm just providing you two well I think I'm providing you two links so here's one link you can look at and maybe after the presentation and if you have questions about that we can also share that so there are two links where you can find different probes that have been developed by various people who are also well have used that time for writing it up in order to publish it remember the publish to your peers section in that circle so this also did happen and so this is all on our website where you can access it take a look at it and maybe you find the one or other probe that can help you in your situation or maybe you find the other one or other probe that can inspire you to come up with your own probe and this is actually the thing that to be very honest we see more often that people look at the probes look at what other people have tried and then coming up with what fits in their own context in terms of probe what's the hypothesis in my own context and therefore also what is the experiment that I want to try for how long I was going to say before we go on remember that this is just an abstract if you go to one of these two links you'll see the whole thing written up which is part of the discipline that we're recommending it's interesting that after they tried this and in fact get more sharing of impediments and they had a way of counting the number of impediments and the in-depthness of them they then experimented with bringing the manager back into the session and that's a whole other story but that's the kind of path that they took to make their thinking together more effective that came out of experimenting with new things I think that's all that I have to say I really encourage everybody to go and look where people have been publishing on our website some of these probes that they've been doing now that you're stressing this again I have to probably expand on that what you see and the narraton this is where it's really have been also expanded then under probes you find a few others and then I'm not sure I don't find the right link for the other thing there's also resources where you find the probes that we have published in our book however for those you only have the abbreviations and I will share the link later and also under probably if you find your way around that I'm pretty sure but I don't have it ready right now here I think we have definitely one big topic left although we touched it and another one that's also not small but in terms of importance and we want to cover that Sean right so we've talked some about measurements and we talked some about the pre and post measurements it's one of the biggest challenges of this process of being conscious about your hypotheses and carrying out an experiment where we measure it the it's important that you use controls because some of you may have heard of the Hawthorne experiment where just the process of measuring things changes behavior and so you really need to have somebody who's not doing those the changes to really see whether what you're doing is actually making a difference so if you have another group that's just doing the measurements and not doing the changes that you're talking about then you have some way of saying well huh they just did measuring and their results improved well that may be just the Hawthorne effect that people when they get measured get like all happy and starting to do things better than they might have before so the controls are really important if you can find a way to do that at all if you can only do in pre and post measure then you really need to be very clear what you've been doing beforehand so if it's looking at refinements since that seems to be our example have you been counting your refinements and maybe you just need to have a sprint what all you're doing is counting your refinements and then the next time you do your experimental step because if you don't have a clear measure as to what's happened before you're not going to get a clear measure afterwards in the example of having a manager be out of the room versus a manager being in the room they did have to actually do a sprint where the manager stayed in the room and they carefully counted what were the innovative measures that were made they had a team some outsiders looking at this list of measurements and estimating how important they were and so on so they were very careful about it and they had to arrange for that you may need to be doing estimates there's a book called how to measure anything that talks about the value of estimation so there's many things to think about it's not just necessarily something that you count you can ask people to estimate what's happening and then you need to have some training in how you're estimating stuff there's more to talk about on this we might ask you to think now in the things that you might have been thinking of about your continuing challenges how actually are you measuring the situations where you have continuing challenges and we might even ask people to put in into the chat if it's refinements or whatever it is that you put down how is it that you're measuring what's actually happening and so I'd love to and how can you now think of measuring it in case you're not measuring it either one just any comments about the measurement and so maybe we can go to the publication slide while people are thinking about that and we'll come back to it Yuta you remember that in the circular chart we had that we were saying that there's two forms of publication one form of publication here it is the inner loop here if the next time you get together to think it could be a retrospective or whatever you need to have somebody write up the experiment it's not enough to sit there and say well we think if this happened or that happened it's really very helpful to write it up so that you have that you have a record if you're going to try it again you have a record of continuing experiments that you've tried that you can look at later if you have two or three that happen and you keep failing well how do we start innovating and let's see what it is we've been doing because it has a common memory so it really helps to write that up that everybody can look at in writing when you have the next session and then the external publication the when scientists are out there publishing to their peers that's really critical in validating the results that they're doing maybe it's in your department whatever setting up relationships where you're publishing to each other what you're doing is a way of really accelerating the innovation if you're only trying this by yourself then it's going to take you much longer than if somebody else is trying to replicate what you've been doing and you're replicating what they've been doing the reason for that is that it's a kind of control the value of a control if we can replicate what you're doing then it seems like hey we're really on to something as opposed to hey we did this about refinements and it sort of seemed to work but you've tried it and it didn't make any difference at all so then maybe what we were doing was being caused by something that we're not seeing and so if it can be replicated I'll give you one example of replication I remember several years ago there was a guy who was a very well known person in the Central Intelligence Agency in the United States and he did an experiment in which he put lie detector probes on his philodendron plant and had a way of keeping a record of what was happening with the lie detector that was on the philodendron plant and he thought that he found a pattern where when he decided he was away from the home and decided to come back home that the philodendron plant had a little spike and it seemed to coincide with this decision to come back home and so he said my goodness the philodendron is reading my mind we have proof of extrasensory perception and he published it but it turned out nobody else found it, nobody else so his so-called proof ended up being well that's interesting that you're getting that but we're not able to replicate it so that publishing is the only way that you can be really sure that what you're seeing is actually happening and I want to add here or feel I have to add here we are also living in a complex context which might mean although at first sight you are at the same context as that other team but if you look closely you are not and that's maybe the reason why you cannot replicate it because things are different from one to another and you may not be able to prove cause and effect but at least it doesn't matter I want to stress the thing that you see on that slide as well because that was for me such a big thing when I started writing that I figured that by writing I really go through a different reflection cycle and I have another learning curve here because writing something up is really the thing that you have to take a different perspective you have more to take the perspective of the reader like how do you explain it to somebody else and this is often bringing at least to me another big learning so I guess John wouldn't be happy with or will not be happy with me when I'm saying now however I'm saying it anyway so even if you are not publishing it don't tell John writing it up already makes a huge difference publishing it to yourself oh my goodness yes that's kind of what I'm suggesting here sorry but that's the most internal publication there is yeah why not keep it diary that's very valuable I think writing brings you to a different part of your brain we don't have any comments about measurement well I suggest we kind of wrap it up and then we see if people have any questions though that we then go into Q&A if we use the Q&A or the chat that's up to you but let us kind of wrap it up and we would like for wrapping it up you to go to Menti again so go to menti.com and now the code is 7488988 and it's still the same link as before so I'm putting it in as well and the question is and you're seeing it what's the most important or surprising concept you learned today and let's say today in this session I guess you have heard and learned a lot of stuff today but let's focus on this workshop frame of thinking importance of publication oh yes so okay at least I didn't put you off with that maybe writing it up without publishing it that's good otherwise John wouldn't talk to me anymore I'm sure I think keeping a diary is important yeah that's true churnalling yeah anything else any other things coming up to you if there was no important or surprising concept you can put that down no surprising or important concept assuming that there was yeah we're encouraging some kind of of response here we go oh keeping team comfortable yes very too all too often I think we forget about this especially as coaches or scrum masters because we are so into change and not everyone is into change and we have to accept that respect that and therefore providing building purchase for them change your perspective I really like that so that's the transformative learning here mm-hmm anything else comes to your mind and again you can also say barring we probably take it personal but it's fine it's okay it would be fair yeah yeah retro action experiment perfect oh I like that yes yes actually this is kind of the thing that I believe right now that well I think I hope right now that this is what the pandemic teaches us that we have to take time to reflect and not always jumping into actions right what else do we see here retro action we have to reflect on situation probe in the chest yes publish learning reflect reflect reflect it makes me think of a hall of mirrors at a carnival ah yes that's true yeah you see you see different things I like it when they set the mirrors sort of at angles to each other and you see a whole series of copies of yourself mm-hmm yeah it's a different perspective right right so yeah kind of giving you a few more seconds oh probe in the chest is something that comes out now very good like that well probably I like everything that you came up with right now nobody said boring yet you too well they are not brave enough maybe they're being kind yes I like the keeping the team comfortable comment yes that's not anything we emphasize but it's yeah keeping it safe well the thing was about the guinea pig and not doing experiments on to someone right yeah respecting the people actually mm-hmm awesome that sounds better than boring okay we you can keep answering this safe for everyone yes and probe in the chest chumps up more and more and the importance of publication gets bigger too yeah you can keep answering and we are kind of closing it up here with like our contact data so you know how you can reach us on twitter linked in yet we also have a crew on that topic on linked in so that's the tiny URL slash at your boss and over which you can join and we have I think three more minutes for I have a question on the three more minutes Q&A assumes people have questions they may have comments so why don't we could use the Q&A for comments also reflections well we can use the chat for that that's true the chat for questions and for comments and for anything you want to share you see wonderful session learning innovation so in case there is anything that still puzzles you or something you would like to comment on this is the time to share it in the chat and there are only two minutes left I'm seeing now and maybe that's for me the time to stop screen sharing so it seems there are no other comments questions and therefore we really would like to thank you all for coming for joining us it was really great fun for us having you here sharing this with you and keep probing and John you're the one for the famous last words famous last words please let us know about what you do if you try this out send us a note because we'd love to hear about it if you have something you want to publish and you can't think of a good place to publish it we can certainly take it on our website and those get transferred other places sometimes or you can point to your friends and say there it is so we'd love to hear back from you