 everybody here for beyond user stories right if anybody's things that they're not they're at the wrong session don't worry I'm glad your subconscious got you the right place so I'm very glad that you guys are here so good morning agile India right I can't hear it good morning agile India and the breakfast is not kicking in has the dark side of collaboration kind of got you down is like no you're not gonna collaborate we're gonna sit on separate ways and we're not gonna participate right that's not going on right all right good good all right so today's session is about beyond user stories how do we take our how do we take our team to the next level of awesome in this session I'm going to be talking about a case study which I have kind of done it and I've kind of transformed the portfolio of team into a more outcome based model or outcome way of thinking all right so I have a lot of fun stuff to talk about so let's get kind of started so a little bit about me I am a culture hacker because I don't like the word agile coach so I think culture hacker is a lot more cooler so I kind of go with culture hacker I am also a speaker I speak at a lot of different scrum and agile groups I speak at a lot of meetups I speak at a lot of different conferences I'm also a guest lecturer at Arizona State University I talk about lean mindset and design thinking so I speak over there as well I'm also a doodler did anybody catch my sketch note on yesterday's keynote did anybody see that all right couple of people did all right cool so I love doing that I love doing sketch notes and doodling and when time permits I'm also an enterprise agile coach I treat a lot of good stuff on my Twitter handle at agile bright spot I'm also on the LinkedIn and it's the exact same picture on LinkedIn so you guys don't get confused who I am all right I also belong to a really cool bunch of guys I'm a part of intro edge technology services and we are a a scaled agile bronze partner shout out to my scaled agile team over there all right so all the scale agile people decided to show up over here so I'm actually very happy about that they're giving me some like they said they were gonna heckle but I think they're not gonna do that they're gonna support me so we do a lot of agile transformations we have a lot of different trainings training in leading safe safe BMO safe scrum exp and if you're not interested in that we also let you borrow our awesome coaches and culture hackers to bring into your teams and portfolios so we can bring our awesome mess and spread the awesome so that's a little bit about me so let's kind of get started with where it all started so what a year and a half ago I was called into this this portfolio and it is large financial organization now contractually I'm obligated to not say their names I'm not gonna say their name but it's a large financial organization it's a Forbes top 50 or a Fortune top 50 company so they called me in and it was a day-long interview believe it or not it was a day-long interview and I was supposed to meet with different segment of the hierarchy so I was supposed to meet with the leaders in the morning the VPs and so on directors or so on then I was supposed to meet with their scrum masters then I was supposed to meet with their product owners and so on their teams and so on and so forth it was a pretty exhaustive or deal and I was actually very happy about that that they actually wanted me to meet all this different segment and then we'll mutually make a call whether we want to do this or not so the very first thing when I went over there I met with their leaders over there and the conversation started by saying that kelpies we understand the importance of agile you don't need to sell us agile over here we've already drank that Kool-Aid we have no problems with it we have heavily empowered our teams we have gotten all our teams have gone through a lot of scrum training about a year ago we also brought in a bunch of coaches to kind of coach them kind of get them going on this journey and we are now ready to basically take our team to that next level so like alright it sounds like a plan and I said well okay well we'll see what we can do over here next was the scrum masters there were scrum masters and project managers so they kind of came in and they sat down with me there were four of them who sat down with me they were part of this interviewing committee and they proudly opened up their rally and they showed me that Kalpesh we are doing fairly good as far as a team is considered considered our burn down charts are I mean our velocity charts are fantastic look at this in the last 10 sprints we haven't missed a single story we have always been true to our commitments we are just based on our velocity we are just based on our capacity we have this down that's good that's very good but one of them said you know what but I don't believe in velocities I think it's all about throughput that's where it's at I said that's great and he showed me his throughput charts here look at this our team's throughput charts and we really monitor that and it's about how many stories your dish up velocity is just a number it's about throughput I said that looks fantastic I mean you guys have got it down that's really great so they showed me this throughput charts over time I said fantastic the other scrum masters said you know what this is all nonsense it's agile is all about time to market right who cares about what our velocity is who cares about how many stories that finish it's all about time to market and I'm gonna show you my cycle lead times so they showed me their cycle lead time and that was pretty darn good I mean they were finishing their stories in less than a week whether it is everybody know what a cycle lead time is who does not know what cycle lead time is all right cycle lead time is a time it takes for a story to go from in progress to accept it or done right so that's what they were monitoring so they're kind of monitoring that data and saying yeah I mean we have it we have it down our cycle lead time is fantastic I mean we have it under four or five five days we have it great so that's actually fantastic you guys have really good data you guys are doing some really good work over here and then they said you know what now let's let's take you to our floor let's take you to our environment and one of the masters said we'll take you to our Gemba so I was pleasantly surprised I'm like oh that's actually great you guys nobody talk is anybody know what Gemba is so Gemba is a it's a lean terminology or it's a lean word a lean word and it says it's a place of action where the action really happens right or the floor right where everything is happening right so they said we'll take you to our Gemba so at least told me that these guys were reading up on stuff they at least knew what was going on so we said all right let's let's look at your environment and they had all open collaborative workspaces now I know you guys came from the dark side of collaboration and some of you might have switched the side today I don't know but that looked pretty good to me I mean it is it's an open floor people were having their own desk but they were also collaborating everybody can talk to each other these guys are voluntarily collaborating around whiteboard so I was pretty impressed looking at that so that's great that's fantastic so you know what our teams are all co-located we are co-located teams they're all here so more power to you wait it gets even better we have co-located product owners beat that so that's even better you guys have co-located product owner that's fantastic what more could a team ask for and the teams are doing all their agile ceremonies together their daily stand-ups sprint planning sprint reviews retrospective they're doing it all together like all right that's fantastic and then in the back of my head I'm like I bet you guys are not doing anything about that CI CD and technical agility stuff I'm like okay I'll probably sell you on that and the next thing the dish out is like oh yeah we already got we're on to our technical agility to do we have a lot of automation testing in place we're already actively working towards that we have CI CD and one click deploy and action and we regularly address our technical debt we even have a negotiation with a product owner where 70% of our backlog is features and 30% of our backlog is technical issues so I was like all right that sounds like that sounds pretty good so as far as metrics and environment was concerned these guys were rock stars these guys were freaking rock stars I'm basically standing over there and scratching my head and if you guys are anything like me I am wondering over there why the fuck why the why do you guys call me why do you guys need me here you guys have it all figured out so why am I here so all right so I'm like okay I'll play along the day is coming along and now I'm wondering like how am I gonna justify my existence over here right it's alright I said let's let's play along let's let's meet more people right I mean what do you guys have going on so I met with the product owners or the product directors I said tell me more about what do you do it's like yeah Kalpesh you know what we take feedback from my users we come up with ideas and then we work with our design team and we come up with a solution but alright that sounds like the right thing to do no problem or there and the the technology team or the technology director said well we take that solution we do some release planning we have a release planning sessions we bucket them out we build it we keep our product owners involved and we we ship it in like you know three sprints four sprints however long it takes we kind of ship it like all right you guys are doing the right thing so you know on about two or three months a quarter or less than quarter we are able to dish it out we're able to launch our stuff to the market so the project is on time it's within budget done in an agile way so that's that's great the product owner is happy I mean they like what they see they've been very much in board and the product delivery director is happy because hey they're delivered on our commitment but there was one problem there was one problem the users are still not happy the users are still confused it's hit and a miss you know Kalpesh sometimes they love our stuff sometimes they don't love our stuff so I'm like aha that's what I had my moment I said okay I get it I think I know what the problem is because when the team is committing to a whole bunch of features or a whole bunch of stories or whatnot all they've got is to perceive your tool there is no room for pivoting right so tons of output but how are we measuring this thing I know you guys are doing fantastic job with your user stories you guys are dish and stories out like there is no tomorrow you guys are a mean green story machine I get it you guys are proud of it no doubt about that but do you guys know what the what what did do the outcome and they're like well not really we don't know that some of the higher up people knew what outcomes they were trying for teams that know cool about it I said all right I said let's look under the hood a little bit so at the end of the day I was at means able to give this smart comment and I was able to justify my place over there so we'll bring you in you said something really smart we'll bring you in I'm like all right let's let's spend some time with your teams so let's look under the hood right and I asked where the stories come from so I actually want to ask you guys not you but think about your teams if you ask your teams where do stories come from what would this is yes is product owner I love it anybody else where do stories come from end users okay I love it anybody else wants to take a stab at it where do stories come from oh they won't even come I don't know where they just they come from somewhere they won't even come we just figured it out he started working on it all right so here is when I asked the team that where do stories really come from I heard all kinds of answers well the product owner the PO finds it for us the PO brings it for us right they some of them said they give it to us right that doesn't even come I don't know but this was my my hall of fame answer was dude just just give me the story and I'll code for it just give me the story and I'll code for it that's the answer I'm like oh my god so all right I said okay let's let's check along I said okay let's take it a level about right and see what's really going on so I asked them what's wrong with your product so what is wrong with your product right I asked them so a whole bunch of them said well we need more refactoring we need more automation we need to use react and not angular we need to like a lot of you have a lot of dead code going on over here and one of the guy hoping that I was give him some price said we need more lean architecture I said okay I get it but nobody really talked about the product nobody did talk about the product nobody when I asked them how many of you have actually seen your product being used by the users or by your users in their habitat and he guesses how many people 0% 0% had actually seen their product in action and some of these guys have been working in that company for almost 10 years very proud of it they had those plaques over this thing that five years 10 years 15 years or whatnot never seen their product in action but I asked them well do you know what what what proud what part of the product your users love and what they hate no never really we don't know that that's where the problem was they never knew they've never seen it in action that no idea what what the product will what they loved about it what they hate about it how they use it where the problem is I have no idea just give me the story I'll code for it I've got the code I've got the story part down I can do this thing all day long right so that was the problem was so I'm gonna digress a little bit and I'm gonna share an interesting fact with you guys so I was reading this book from Steven Steven Coe on 8th habit and he says only 37% in the company and he actually interviewed like 23,000 people from different company and said the founder only 37% said that they had a clear understanding of what their organization is trying to do right only one in five are enthusiastic about it only one in five said that they had a clear line of sight between their task and what the organizational goals are and only 15% felt that the organization fully enables them to execute the goals and 20% full only 20% fully trusted their organization just a bunch of characteristics I can't connect to this there's 23,000 people so let me bring a little bit more context to your world so for a moment what I want you guys to do is think about your team as a soccer team alright so put on that put on a thinking hat and think for a moment at the scrum teams that you are belonging to is actually a soccer team so is everybody imagining that your team is a soccer team just raise your hand saying yes we are doing it we are imagining our team as a soccer team alright so think of your team as a soccer team alright only four of the 11 players on the field would know which goal is there alright only two of the 11 would care right only two of the 11 would know what position they play and exactly know what they're supposed to do right and all but two players would in some way be competing against their own team members rather than their opponent and we actually know that what it's called right we've seen a lot in IPL right we've seen that in IPL right we know how this goes down right this is what's happening with your team and when I showed this to the leadership they were like hell this is where the problem is yeah they're all playing the games but they have no clue which side of the ball I'm just called into kick the ball I will just score the stories I will code that all day long I'll just keep hitting the ball but dude where is the ball going I'll be scoring any goals or not I'll be winning the game or not I don't know that I'm just supposed to hit the ball right that's where the challenge right so okay so so you can now take the hat back or are you guys still thinking about your team and the soccer team this is true how many of you actually connect to this thing did I did anybody felt like oh my god my team is doing that so and I actually got this data from made to stick books I mean that's what I got this data from and I like this is so true this is so true right so that's where I kind of went around and I said okay I can I'm kind of I'm kind of I'm on to something over here but I want to put my finger on this thing so what I did was I I drew the scrum framework on the board and I asked the teams tell me what your perception is right so what do you think happens here before stuff comes into the backlog right so I want to actually ask you guys not you but think about what your team would say right if you ask your team well what do you think happens before the stuff comes into the backlog what do you think your team member would say I don't know the portfolio managers okay I love it I mean this is the scale natural guys are gonna be like wait wait wait yes that is right the portfolio good the product owner comes up with the requirements magic happens before it's magic and actually one of the guys told me the product owner consults the oracle there's a magic eight ball they shake it and they find okay this is the next thing that we got to work on that is what's happening that's how they perceive it to be right I said all right this is this is good now we're kind of going somewhere with this thing what do you think happens here once you dish stuff out once stuff has gone into production what do you think your teams would say I love it but other guys said production issues happen they actually said production issues production issues happen right that's a skater it's a black hole I don't know what happened I dished it out right and actually I'm gonna give it over to you guys so these this company has changed one of their interview questions so now when the interview engineers they actually asked a question now to whoever is coming in said where do you think your responsibility as an engineer starts and where do you think your responsibility ends right you don't have to answer it right now this is their way of now gauging like I know who are we really getting right this is what's happening I mean yeah what happens afterwards production issues simple as that production issues come we fix it it goes back into our backlog we size it and it goes through our cycle again all right so that's when I kind of thought well okay I think we gotta we gotta work we gotta look at this from a different angle so that's where I kind of got an aha moment my aha moment was delivery feels a lot more better to these guys they're in the mode of delivery they're more they're a mean green story machine that feels better for learning feels weird learning felt very weird to these guys it was basically they were always asked the question can it be built but I needed we need to now shift our thought process it's not can it be built it shouldn't be that is what we should be focusing on that is where that is where you will kind of go on the soccer field and you will be kicking the ball in the right direction that's the question we should be asking it shouldn't be there right so they had phenomenally optimized for the build part they were superbly optimized for built but had no clue about measure and learn this part of the loop completely did not exist for those guys it just did not exist for those guys right so that's where that and that's where I kind of started off with those guys so I again put the scrum framework and I ask the question let's open the boundaries of the scrum frame let's open the boundaries a little bit and let's expose these guys and make them participate or take ownership into the stuff that happens before because when I asked with the product owner part owner what do you think happens before this of all we define our outcomes we have a lot of hypotheses that we generate we have user labs will do empathy mapping we have AB test designs or whatnot and I said okay well let's make our team a part of this process they need to be exposed to this thing so they understand why I came to work today right actually you won't believe it there was a whole analytics team that was observing calculating stuff on how what like how the user's reacting it was in India and they did not even know it existed nobody knew that team existed there was an experimentation and test and learn team that was in place nobody knew that team existed that no idea the team existed only the product owner knew about it they were basically operating in silos they were agile but they were agile in silos that's what the challenge was so I said you know what let's open it up let's make them part of the outcome definition so I I told my product owner says you know what let's play along with me let's actually bring these guys in and let's say let's make them part of the outcome definition and what I did was that I'm so sorry the picture turned out to be very bright but what they're doing over here is they've we basically mapped stuff out and we put things over here so we kind of listed an outcome over here who can help us with this how they can help us and what features we will create so as a team we basically mapped the journey we map the journey I said okay well we're gonna put the outcome on the left hand side this is what the outcome you're trying to achieve now together as a team we are gonna brainstorm how will we reach the features okay so you are connected on what the journey is the very first session I had all the engineers came they sat down and the product owner kind of went about it that's not how we do it right so very first session the engineers felt the team felt so uncomfortable with this thing they're like I'm not able to come up with a lot of ideas so the first time around the product owner did a lot of driving the next session we had the team slowly started coming to the table okay I get it I know how this exercise work let me participate in this stuff they were now talking stuff out say okay if this is the outcome this is who can help us with it this is how they can help us with this stuff and this is the these are the features that we need to put out if we need to achieve this this outcome the very first time I did this it was just they all that all of them together it's just one team and and doing it next time around our experiment I said okay let's split into two teams right a team you guys do this exercise and you guys do this exercise and what was happening they're coming up with all these great ideas but well how will we test this how will we measure this so I kind of experimented that again I said okay in this mix I will call that person from analytics and that person will be a part of this exercise as well so now they start understanding oh before I put a feature out I need to figure out how this will be measured right so they start interacting with these guys right so we slowly we started changing the definition of our scrum team is choice I change the definition of a scrum team so they started now and after that we had a design team so we start to add a design team member to this discussion right so now it was a designer it was an analytics team and the engineering team who are sitting over there and figuring out how do we go from our outcomes to actually a feature they were addressing the question is this feasible is this doable can we measure this thing can we test this thing it's all being done over here and by the way do you know how long these sessions would last make a guess one hour one hour is all I need because when I kind of put this pitch this ideas do you what's the first question I would have been asked by the scrum master project managers what do you think they would have asked my resources will be wasting their time over here right I said all I need is an hour an hour one hour is all I need let's see where this takes us and this idea really caught on so once we have figured out what features we wanted what we did was we started doing design studios I said all right we've got a features now actually let's do design the by the way do you guys know what design studios are or UX sessions are how many of you know what they are so a couple of you so a design studio is where the the design team member and the he's our our hipster designs design design guru over here he's a designer we actually flew in from New York he actually now flies in every once a month to sit with the team and do this exercise with them but at first I kind of cracked the deal said can you do it once a month or once a quarter with me so this guy flew in and what they do is they actually put the problem on the wall they were so good this is the feature that we want to create can be designed couple of options over here so what he was doing was he was actually sharing design thought process or design thinking process with the team members he kind of showed them and what they did was they start coming up with three different options for that feature if you have to create this feature let's come up with three different options how are we going to create this feature right and here is a product owner she's taking notes over here these are two of the engineers there are part of the team there are others over there but they were a little camera shy so they said okay if you're going to put this on a conference screen or what we don't want to be a part of it I said all right no problem so these were engineers and again how long did it take an hour I don't need more time I just want an hour if you're taking more than an hour to figure this out we're doing something wrong we are going into analysis paralysis and that's fundamentally wrong so they just can we can we quickly come up with and these are paper prototype it's a big easel pad and we kind of draw it out refining it was my my my designer's job he sat there and he came up with nice little prototypes after that all right so we start making a part of design studios we were doing this together and after that and this is the real kicker this is where we were having a rapid user lab sessions so this is where we kind of said okay our our rapid user lab sessions that the teams actually got to see you the users interacting with the prototype they had created the three prototypes that I kind of showed you with this picture over here they actually now dished it out and they're actually getting customer feedback and if you look at this game we'll spend some time showing the the way this is set up so this is a screen this is the prototype that's over here they're able to see the the user over here and there's a chat going on with the person who's moderating this session so the chat going on with those guys as well what we did was we listed the features over here what features are we actually dishing out what are the hypotheses that we are trying to validate right and if there are any notes we're kind of capturing that over here right now I want you guys to actually take a good look at this picture and I want you guys to pay attention to the body language right this is the guy who actually built who came up with the idea of the feature of the prototypes looking at the car at the user's reaction this guy couldn't even sit down he's like do this right there why are you not clicking come on man look at this stuff that guy couldn't even sit down that's how involved this person was he couldn't he couldn't fan them that the stuff that he had created or was about to build the users are not getting it the user were just not getting it this guy right here he's my scrum master project manager any comment on his body language the guy is super worried why why would he be super worried what this is the stuff I had lined up in my backlog two sprints from now it is getting thrown out and what's the biggest worry of a project manager funding an amount of work if I don't have that what will my team do I don't have a team I don't have funding what will my team be doing over there this guy was worried scared sitting over here like oh my god what am I gonna do because our stuff is getting thrown out of the walls right now right but one very interesting stuff happened this lady right here she's a QA person right and what happened was she made a very interesting observation while these rapid lab sessions are going on one of the features that we showed them was actually a mobile view of our application of our product and as soon as we showed that prototype to the user the user did something very interesting the user stood up and he actually took that thing on their hand and they started moving around anybody wants to guess why that happened because that's how I use your product I'm never really sitting down and using your product I'm always on the move if I'm seeing this on my mobile screen if I'm seeing this on my tablet screen I'm actually on the move I'm never sitting down since this lady saw that her testing she never sits down and tests she actually walks around and tests because she knows if I wanted a true user acceptance testing is actually the way the users would be testing this thing and this is where I justify why your test is really need to see how the users are interacting with your product right it's not about how many defects I can find it's about what environment our our users actually using this product in and I need to simulate that right actually they were very funny comment that the user actually made is that all you guys get from me while I'm using a product is one eye and one finger that's all you guys get from me when I'm using this thing and that's from then on that's how our testers actually test our mobile views that's how you'll be testing it right they actually go outside the building now and they walk around in the test it or the connectivity is bad that's how they tested there was a fundamental shift now in the way they were looking at their product right but that's not even the biggest price the biggest price was the first session they were just watching the next time around when we did this rapid user labs or this user lab sessions right what happened was the engineers while they were watching this stuff and one session got over what they did was they quickly called the designer and they said let's refine this based on what our users told us can we modify a few buttons here and there and the designer kind of went at it modified it and the next user that came after an hour we actually modified up and we started to get some feedback from there the third user that came we modified it even further and we showed them that by the time we had our sixth user we had pretty much nailed it down on what our users wanted all in a day we went from a idea that we don't know if our users would like to a set of features that we are darn sure that they would love it in a day but the real price for me was this guy actually this guy he actually said wait Kalpesh the build in the build measure learn loop doesn't really mean I have to write a code built over here could mean that I built a prototype really quickly and validate it because that's the purpose of doing this thing isn't it I almost cried a little bit give me a hug let's bring it in you guys I literally cried a little at that time because that's what the goal was do you know what's a measure of the most innovative organizations is the true measure of the most innovative organization is the time to first prototype that's the true measure that the organization that can is the fastest to create a prototype wins because you are in front of the users the earliest actually I have has anybody heard of plural site the company plural site yes Nate walking Shaw is the chief product owner of that company and I was I met him at one of the conferences and I was sitting with him and I was gonna ask him how do you guys kind of go about doing this and he said like we go from user story mapping user story brainstorming to high fidelity wireframes to getting it in front of our customers do you know how much time they take from user story mapping coming up with user stories or features to actually getting the prototype in front of their customer do you know how much time they take four to six hours that is fast that is fast they literally start desperate with that we start as print but this is the features what we want that day we will know what we got to work on and the rest of this print is where we actually build and dish it out that is true agility agility wasn't about what are my velocity charts look like what do my burn downs look like what do my throughput or my cycle lead time look that is not agility agility is how can I get the right stuff to my users the fastest that is true the ability of these guys to spend that day and dish out different prototypes and refine on the solution that is true agility that organization wins right so once we actually saw that what we did was we're empathy mapping does anybody know what empathy mapping is so empathy mapping is what we do is we actually observe our users and we go back to the drawing board and say well what did they say about our features what did they think about our features what did they do with our features what did they feel about our features right this was actually done and and as Richard yesterday said and is a keynote yesterday they have an anthropologist right we actually said let our teams do this thing and our teams kind of went to town with that based on whatever they observed they actually started writing that stuff down look at that number of sticky notes over there these guys that started really started observing what the users were saying when you have tuned in the way your users thinks the way your user feels about your product thinks about their product the way they use your product when you have an engineering team who can do that you've got a potent mix that team is dangerous that team is dangerous so they started doing empathy mapping and by the way if you were wondering why is everybody wearing the same t-shirt we actually got them t-shirts that had their outcome on it so we kind of invested into that let's let's get them t-shirts that they're wearing their outcomes on it so that's why they have the blue t-shirt fun but the team actually started doing empathy they were really connected and again how much time does this thing take an hour I don't need more of your time I just need an hour right so they started doing empathy mapping but once that was done that was just within the team that was tribal knowledge right okay so we got it but we are and as I said earlier in my slides I'm a culture hacker so I designed a culture hack over here so what I did was I said okay let's do something we have a couple of options for this features that our users kind of gave feedback on it's time to actually throw it out in the market and validate this thing right has anybody heard of what multivariate testing is or ab testing is who knows what ab testing is alright ab testing is where what you do is you have two versions or three versions of the same feature and you dish it out to the different population and you get feedback and see which one is the winner so you open it up for like 1% 2% or 5% of the users you open it up for them and you see which one is the winner which one is the population or the users like the most and then you basically scale it from their own right so you've got a winner and then you basically build that out completely and dish it out for 100% of the population right so what we did over here was we sent out a mass email to all the teams so all the tele product teams that we had all the leadership that was involved we actually sent out an email saying that we are creating tell us which design will win so we actually had this which design will win we told them what our objective was what what is it that we are trying to accomplish with this feature we told them what our target segment was this is where personas kicked in this is where they came to know what personas are really targeting so we told them what our target segment was and they had to come up with key success metrics how would we know that this is a winner how would we know that this is what the users really want we know that is cost cross the threshold remember the team is extremely involved they know it from the get go what the success metric and then you kind of listed out what what is the outcome we're trying to accomplish right and we dished out the two options option a option b and we actually had a voting thing over here so we actually put voting thing over there so people could actually vote and once the people people start voting and we actually gave them a bribe there was the design which would win and whoever selected that design would actually get a price we actually would give out price for every design that we gave out right so we used to give out like you know you will get a bunch of candy bars or you'll get like a little toy Batman or something like that we had a little toy store that we kind of put out of the place and people who kind of gave the right answers would win and what we did was we actually started publishing the before and after this is what you thought was going to win and this is what you actually users winner came out to be this is where the entire flow all the product teams kind of come to know about okay slowly understanding what our users are really like and the teams were accountable actually it's very funny the conversations in the hallway started changing the team members when they were interacting with each other start actually asking here the stuff that you that that you dished out in the previously which was a winner what was the winner did it win did that feature win that's the conversation that's how to happen and that was the true thing everyone discussed they said okay no well no it didn't really win well why did it not only when well because the users the engineers were now talking about and that is when we shifted the conversations from should we use a reactor angular to actually the conversation shifted towards oh well this feature didn't really the users didn't really like it because of this reason and this actually this feature actually did off the wazoo they did really good everybody loved it we killed it that's how we changed the conversation that was it was all one simple it's one voting system that we sent out and that fundamentally changed the conversation with the so once that happened and stuff is out in the production so I said okay that's great now let's talk about what happens in the black hole one stuff is released what happens after that right so no release or launch announcement actually how many of you wants to release stuff into production send out this big old email saying that yes we launched we are in production how many of you actually send out launch announcements or release announcements pretty much everybody of you right and you know all this email chain that kind of goes back and forth say congratulations great job you guys are the best do you guys get that like 40 emails will go and the leader will send out an email to saying great job you guys nailed it or whatnot right I said that's nonsense I said that's basically saying that so how many of you actually have kids good or you guys where kids once upon a time when you guys actually went and gave a test and you came back home did your parents celebrate saying that great you gave a test this is awesome we should celebrate we should give out sweets or did you actually celebrate when you got good grades it's when we got good grades right this is what that's not let's stop this launch announcement because that's the fundamental thing that we are rewarding over we are rewarding a behavior of a project on time on budget that is what we are rewarding over here this was fundamentally they are reward system so I said no launch or release announcement and you have no idea the amount of hate I got you have no idea the amount of hate I got so sorry so I'm running out of time but I'll kind of quickly go so what we did was we said you know what no launch announcement only learning announcement so we basically sent out saying okay our latest progress latest learnings and our game plan to address that that's the email we started sending out once a month we dished that stuff out right so we kind of call out say okay this is the latest progress latest learnings and latest plan we kind of listed out what is the purpose of our team and then you said progress what all are the features that we have released so far what were our recent learnings give me the exact numbers yes or something went up by 30% we had 142 bps per increase in this thing we had 149,000 brand new recommendations or whatnot right this was tangible outcome tell me every month how much have you moved the outcome I don't care if you were on time I don't care if you're on budget or not did you move the needle or not what did you guys learn that is what's the and this is actually been sent out to the entire organization right and based on this learnings what is your game plan how are you gonna adjust yourself and when we started sending out this and of course the thank you everybody wants to thank you right so we start thanking all the teams or whatnot for doing this thing but we started celebrating learnings and then the emails changed we started getting two kinds of emails hurrah you guys got us this much money you guys got us new new new new users signed up you got our digital engagement up by something and then emails kind of coming up see why did that not work out for you guys what really happened can you guys share the learnings or not that's the kind of email the teams start stopped thinking about are we on time did we release on and that the whole conversations had now shifted towards outcomes they actually wanted we actually wanted celebrations of what not on how much the outcomes we had accomplished which features and which team was accomplishing which outcomes by how much is what was being talked about and that is what is what is being rewarded now how much did you move the needle on the outcome all we had to do was simple email a simple email and this is my perfect culture hack that I talk about we fundamentally change the culture we fundamentally change the mindset now the engineers are not worried about how hey can I finish this thing or is it technology that or will be making into production or time or not no now they are worried about the questions they ask the product owner is how much will this feature increase our outcome by because that is what I'm interested in I am no longer interested in how much how quickly can this be built or we can be we can probably accomplish this in two sprints or 10 sprints no how much will this feature move our outcome by because that's what I'm getting rewarded on I want to see thank you emails on that can you give me that right and that's where the whole mindset has shifted now now these guys talk about the outcome teams need to be measured by the successful experience in the hands of the users and not on successful delivery of the product that is a true measure of the team you don't measure a team by how many features they finished you measure a team by a successful experience in the hands of the user that is a true measure of the team you could be dishing out stories left and right day and night you have a workshop going on all day long that doesn't do jack for the organization if you didn't move the outcome at all if the users are still not happy that doesn't cut it you're in the wrong business right so in fact we were so much around the outcome that we actually put our outcome results right on this you could not even come to the floor till you have actually seen how is your stuff affecting the outcomes you cannot even come upstairs you can't even come into your office till you don't know how you're affecting it's basically like damn it man we really didn't do anything in the last three weeks it didn't do anything at all but they're like hell yeah I actually had you see this chart you see this yellow my team did it there was pride over there and that's what even the leadership started rewarding on that's actually great information we know that this is what the team has accomplished there was a tremendous sense of pride and there was a lot of trash talking to because the teams had trash like how we've accomplished so much of an outcome and you guys haven't done jack so there was a lot of that as well but that's actually good because they're talking about they are no longer talking about release and deadlines they're talking about outcomes and value and that's what the conversation is right so the aha moment was that many teams with many dreams but to succeed we needed one team with one team right we had analytics team we have test and learn team we have design team we have engineering team we have PO teams they were all there and they were all trying to get their job done but they were all need to be working towards one team that's where the success if they were all all their efforts were aligned towards that one outcome they had to come together they had to deliver on that outcome that's where the success is right so I call this as the Holy Trinity we had scrum teams when we started it but then we kind of fundamentally shifted the composition of our scrum teams our scrum teams actually now have product owner designers and engineers that is what the team structure now looks like right it was first it was product owner we have front-end engineers we had back-end engineers and that's about it and tester that's our scrum team go rock your word now we're saying that we don't have scrum teams we have product teams right and we have supporting teams around we actually also played with test and learn and analytics what happened was the work that test and learn was doing we pulled it into our own backlog right so team actually started designing those experiments themselves and they started taking it to their backlog at all so that team kind of went away and now the teams are experts in test and learn and then analytics we kind of played with it and kind of brought them in but we kind of brought them out we want dedicated one person who comes to us tells us the data and walks away I actually show and tell the sprint reviews have changed in our show and tell our sprint reviews we actually talk about the engineer showcase the features the product owner actually showcases what was the impact what was the value delivery that we had from previous releases analytics actually comes and shows us the numbers this is how much we increase by this is how much we decrease by and designers come and show us their design vision this is what we are taking our design to work that is what is our show and tell now it is no longer just engineering team put on the spot now show us what you did it is actually all of us coming together and saying this is what we did towards that is what our show and tell right so we are towards some teams but we are now well product right and that is a lot of difference between just a scrum team and a product team a product team is centered around outcome achievement how quickly can we accomplish that outcome right so you would think all right all this stuff happened right so did you guys so what you changed all this stuff so what our velocity didn't increase throughout this whole process velocity didn't increase throughput didn't increase we are still dishing out the same number of stories that we were before throughput did not increase there was no increase throughput speed to market our cycle lead time hasn't improved any better that's still the same right so you like all this stuff you were harping about for last 40 minutes what the hell dude right so what happened was our hit ratio does anybody know what hit ratio is the success of the feature in the hand of the user was tremendously increased increased to the point by almost 80% 80% of the work that we dished out was loved by the user they loved it initially it was probably 10% 20% 30% hit ratio at tremendous and so was the motivation the motivation of the team was skyrocketed why because now they were actually decision makers they had a part they had a stake in the outcome that we were driving there was the equal by equal seat for them on the table I actually say that our product ownership is just another skill on the table for us product owners engineers designers analytics we all are product owners this is our product we succeed together so the motivation is off the charts for these guys I actually read a book on the flight while I was coming over here and that book was just published in March it's called smarter better faster it's a book written by the guy who wrote power of habit and that's what he says that when you give the team the power of decision-making the motivation is off the rocket and when I was reading the freedom like ah I knew this this is what we did right so beyond user stories your teams are not just limited not it's not just about user stories we actually expose them to the entire journeys they knew who the target segment they knew what the personas where they knew they're a part of the problem scenario design they're part of the value proposition and assumption they're part of the customer discovery and experimentation and they are a part of the decision should be pivot or persevere it's a decision made by the team it's not somebody higher up up there or the product owner came we as a team decided should we continue with this feature or should we not continue with the future pivot or persevere is a decision at the ground level and not by somebody up there right so basically fundamentally shifted that so our manifesto is different now we say product over software is what we work towards it's not we are not in software business we are in the product business we are an experienced business right and experiences over functionality we are an experienced business what the functionality we 저도er izhtout what they do for our user and with that said I'm actually taking this platform and I'm launching a campaign right agile India this floor with you guys I am body campaign that says no more backlog lumber tags we are not backlog lumber the engineers you have in your team they have ideas they have suggestions they have recommendations they are a smart bunch of people utilize them if you're keeping them to sit there and write user stories, that's a waste, that's a colossal waste of talent. That's a colossal waste of time. That's a colossal waste of effort. These guys are smart. They have a lot of ideas. They can contribute to your product like there is no tomorrow. So that's why I'm saying they are not backlog lumberjack. They are equal partners on the table. By the way, does anybody know what lumberjack is? How many of you don't know what a lumberjack is? So lumberjacks are basically people who basically chop the wood. So you see, put the wood over here, chop next. Put the wood over here, chop next. Put the wood over here, chop next. That's what they thought. Actually, an Indian version of that would be kolukabel, right? That thing is just circling around all day long. I don't know where the seeds came from. I don't know what happened to the oil or whatnot, but I just keep circling around all day long. I'm really good kolukabel. I circle around 50 times. I'm faster than everybody else, but dude, I have no idea what happens. So no backlog lumberjacks, right? And that's my campaign. I actually have laptop stickers for you guys, so please don't go grab your stickers or whatnot. And how many of you guys were so amazed by what I was doing over here that you forgot to take the notes? So if you forgot to take the notes, I have a sketch note for what my talk was, so you guys can grab it. Everything that I talked about is in this sketch note. And I actually have, if you don't be a frenzy of taking pictures, I actually have sketch notes printed out for you guys, so you guys can grab it. I'll leave it up over here. And I also have stickers for backlog lumberjacks, so please proudly put this in your laptop. And every time, when somebody tells you, just finish the story, tell them, I am not a backlog lumberjack, all right? But I'm not done yet, I'm not done yet, I'm not done yet. Hold on, I'm loving the attention. Guys, when I went and implemented this thing, I am not empowered in any special way. I don't have special authority when I go into these things. I don't have special place in the teams when I go into this thing, right? I am as much empowered as you guys are, right? I don't have any special authority. And just like you guys, I use my good looks and charming personality to make the change happen, right? And that's what you guys can do as well. Everything that I mentioned in this presentation today is extremely doable by you guys. All, it's all about having one session with your team. It's all about changing the emails. It's all about launch announcement. It's all about not talking about stories, but about outcome. Everything that I've mentioned in the session over here is doable by you guys. So I empower you guys, go out there and please stop the backlog lumberjacks. Thank you, Agile India. Thank you for listening to me. Thank you, thank you, thank you, thank you. Any questions, any questions, thoughts, suggestions? Yes, I have a question over there. And please, before you leave, I have stickers so really if you can help me pass the stickers around or whatnot. And I do have these awesome sketch notes. So please grab and spread them around. I'll actually pass a whole bunch over here. I'll pass a whole bunch over here. But questions, so if you have any questions, please. Questions, somebody had questions. Yeah, come on over here, come on over here. Come on over here. So.