 Welcome to Agile Roots 2010, sponsored by Version 1, Rally Software, Vario, Amirsis, Agile Alliance, and Xmission Internet. Agile Team Performance Management by Arlen Bankstone and Sanjeev Augustine. We have something called White Speed here. We have an Agile Coaching Consulting Training Company, and Sanjeev and I've been working together about 10 years. My background is user experience design and Agile Coaching and Development of all various sorts, certified scrum trainer that I'll do in the industry. I've had a lot of experience in a lot of different sorts of companies, and that's what a lot of this is coming for us. Thanks. So, I'm Sanjeev Augustine. I've been in the industry for the past 20 years, about two decades, so a little over two decades, and I've been playing with this thing called Agile for over a decade. I got started, I think Jeff happened this earlier this morning, talking about XP. He got started on an XP program, an XP team, and likewise, so did I. In fact, my first experience with Agile was the fact that in a situation that I was coming into a company as a project manager, and in this company, they were asked, they said, well, you know what, we have these three projects for you, extreme programming projects, and have you learned about extreme programming? And I said, no, I haven't, but I've been interested in learning new stuff, so I've been willing to go on and learn the... So, we were, you know, I joined this company as a project manager that said, we need some help managing these projects, and there's this thing called extreme programming. I said, well, it sounds good, but we don't need to find out about it, although I learned about extreme programming. And in those days, there was the first book by Kent Beck, Extreme Programming Explained. Anyone read the book? All right, yeah. Great book, and I went off and read it, and thought it was a really easy kind of nice read, right? So, both of our art pages, so I said, well, my own background was as a developer, I got started as a software developer, and did that for a number of years. So, the many techniques of technical discipline and real strong software craftsmanship were kind of filmmaking to me. But, at the very end of the book, Kent said, you know, where's the stuff about the project management in this book? So, I thought maybe I'd miss something, so I went back to the index, and I looked for project management. It pointed me back to one paragraph in the book about the role of a project manager, and there wasn't a lot, but there's one line that I can remember. It said, the role of a project manager is to remove obstacles for the team and to bring the team pizza. Okay, that's a good place to start, so it's, you know, this good sort of job description, I can get started with that. Anyway, one thing led to another, and I sort of ended up writing my own book about project management, managing action projects. Freddie's home in 2005, and they're so inclined in that direction. But, in recently, what Arnold and I, and others in the industry, are doing, is that we're exploring more, sort of beyond the boundaries of Scrum, beyond the boundaries of action project management, what it is to create a true management system. What is a performance management system? And we've heard a number of speakers today, and you'll see a sort of these threads running through, when Jeff talked, when Jeff happened to talk, when Diana talked, and some others spoke. Now, how do we set up an environment, a true environment where we can truly manage a project? So, I'm going to jump into our sort of agenda, we're going to keep it in right here, this smaller crowd, so we'll maybe take a couple of questions as we go along. We don't have to rely on the phone Q&A at the end. But I want to talk about what's wrong with performance management, and actually more specifically, what's wrong with performance measurement today, because a lot of times we conflate performance measurement with performance management. I'll come back to that talk that's coming up. So, I'm going to talk about some current issues, some emerging trends that are pushing us in the opposite direction, where we can actually get our whole performance management system better aligned with agile teams, self-organization, true, certain leadership from project managers and other managers as well. So, we'll talk about an agile performance management system, and we'll talk about some goals. I don't want to speak about intrinsic motivation, and over to all of these, we'll talk about extrinsic motivation, these cool trends that have been starting to come into prominence now, especially with the work of Dan Pink, you know, with this new book called Drive. Who here has heard of Dan Pink or read his stuff or, you know, so if you haven't made sure you want to look at this, lots of stuff on YouTube, on Google, it is certainly Dan Pink has this new book called Drive, and a lot of, you know, the fair part of what they're talking about here is being inspired or involved like this. All right, so we'll talk about some of the current issues with performance management. Who here has actually either delivered or, you know, the receiving end of a performance appraisal? All right, anyone who not? Put your hands up if you haven't, either received or delivered from our school, you're a lucky guy. Who do you work with? Right now, this is a dependent consultant. Right, but over the course of a year, he has several different software companies. Right, so over the course of a career, I'm sure, you know, each one of us has encountered a performance appraisal, right? They have some sort of 360-degree feedback system, and if you're a manager, put your hands up to your manager. Manager, senior manager, director, all right, you guys, you all start at the left of your eye over here with us, actually. Okay, if you're a manager, you just put your hand up. Tell me, put your hand up if you really loved delivering that performance appraisal. Who really loved it? Okay, so we got, and we went to explore that, but you see the numbers kind of thing down, when you say, do we have a good performance measurement and more importantly, a performance management system? We don't have performance appraisals, and we kind of do these things de facto, I think the de facto standard in the industry, standard image, 360-degree feedback, but the sad truth of it is that our performance management systems are broken, and the reviews are probably one of the worst ways of doing that. People get demobilized, they get demotivated, and a lot of times we end up as managers talking more about the things that are wrong with people than where the strengths are. So, you know, there's Marcus Buckingham, you guys, who here has heard of Marcus Buckingham, right? First discover where your strengths are, actually, what does it do? First break all the rules, now discuss your strengths. If there's this whole sort of thinking that you've been saying, it is much more effective to discover and amplify people's strengths than it is to continuously sort of bump them off the weaknesses. Of course, if you have some egregious weaknesses which should be able to fix those, we have to do this part of our members of the civilized society so that we have a productive team, but let's tap into people's actual strengths, and that is much more productive than actually spending more time talking about the weaknesses. Look at that graph over there. However, the performance appraisal process actually meets us in the opposite direction. It meets us to spend much more time talking about people's weaknesses and how to fix them and move them to some sort of medium of sort of performance management. Many of these techniques tend to be looked at in what we call management 1.0. Spread big trailers, industrial things, management. Not suited for management of knowledge workers. Peter Drucker coined this term knowledge workers for people who use their minds as much as they their hands. And certainly within software development of any sort of knowledge-based profession, the way we manage knowledge workers has to be quite different from the way we manage industrial workers. So, management 1.0 tends to overemphasize the role of the manager. If you think about the typical review, it involves collection, feedback from a whole bunch of folks, the manager collects it and then gives his or her interpretation. There's an interpretation and an overemphasization of the role of the manager. That's one of the issues of this whole performance of praises that is set up. Let's look at some of the other issues. Traditional reviews bundle three things. Feedback. You're supposed to get feedback from a performance of praises. That's supposed to be the point. Do we do a performance review? Put your hands up if you're doing annual performance reviews. Put your hands up if you're doing 6 you know, a performance review every 6 months after you. You guys are getting better. Anyone doing the month of quarter? You guys are way better. And more frequently than month of quarter, maybe once a month. I got a company joining a company. Tell us which company you own it. wannabe. That's the name of the company. I certainly want to talk with you because there are very few companies that out here that understand that a performance of praises is primarily about feedback. And it's actually less about compensation and merit pay. So one of the things we need to do is to figure out how to unbundle these things because they actually tend to get performance of praises getting done once a year. And feedback once a year is what? Good, bad, ugly? Really? It's feedback once a year is just too little, too late. So the more frequently we can get feedback, it's the better. And of course, if you want to apply someone that we got to legal cover, that's sometimes the goal of a performance of praises. There's some sort of normal distribution that we are usually told that we can only have 10% of people who are doing exceed expectations. You can have only another 20% or a median expectation. And therefore, once we work with this appraisal process, we have to be able to fit some people into this normal distribution and our hands are tight with that. So there are a bunch of issues over there. We'll get several speakers that quote Edward Deming and we certainly don't want to be left out of the game. So we have our own Deming quote over there. And he's looking at what he tends to think about annual ratings. We're starting to see a number of trends in the opposite direction. Anyone here, let's say under the age of 35? Yeah, you guys, 35. You guys qualify for Gen Y in the later generation. I think for the first time in corporate America we have Gen X in the ballroom and Baby Boomers in Gen Y and Gen X together in Workforce. So Gen X kind of executives driving Workforces that have a mixture of Gen Y, Gen X and Baby Boomers. Also, it turns out that the values not to paint everybody with the same brush but they tend to be generational differences in expectations between folks who are in the 35 or under the Gen Xers or people in the 40s like myself or Baby Boomers who are you know, been around for a while. And we start to see that there are things like the quality of life movement where people are looking for more meaning as part of their lives. So my father's work was a professor of sociology and he worked in the university for 25 years. Everyday he'd go up and you know he'd teach everyday but when he came back and he did that for 25 years. And he started meaning a little bit to his teaching profession but a lot of it through his extracurricular activities outside of work. Well, it turns out that in today's workforce many of us are seeking meaning in our lives that combine what we're doing at work with our personal passions, with our personal ambitions, with our personal motives. And so what you see is that there's a you know, especially among the Gen Y it turns out that they will more often be driven by personal passion than you know in a punch in the pocket and way from looking at work and their activities. So I want to sort of jump into what we're proposing here today. We're talking about an agile performance management system that will start to define an overall management system not just a measurement system which is what the performance appraisal is the performance appraisal is just one way of HR sort of influencing one sort of measurement with a whole lot of disastrous consequences. So we're talking about a more holistic look or a more holistic system that really draws everybody and Andrew used his term into the circle of evidence. We talk about agile in the circle of evidence but now our circle of evidence is fairly restricted to or it was fairly restricted to developers. Agile grew from out of the development community. Therefore, you know, if you're a developer you're sourced where maths might end at the minimum of the circle of evidence. If you were a project manager you're kind of left out of the party. Well, now we're starting to make a remedy that now project management is strong and all of that has been sort of being brought completely into the circle of evidence at least on the bridge. If you were a product order, I remember Jeff some years ago sort of lamenting the lack of usability techniques within the agile community. Well, we're starting to make live up to our own promise if you will that agile teams are holistic they're one team and they integrate all the disciplines. Well, we want to come up with a performance measurement system that combines intrinsic motivation and extrinsic motivation. So, intrinsic motivation is about what drives each one of us as individuals. How do we find meaning in our work? What excites us? How do we get passionate? How do we get into the zone? Into this flow there's this guy called Mihai Jixson. Mihai Jixson behind me. Thank you. We'll get her in a grand block where we'll talk about flow and passion. How you can get into the zone and really get highly productive. We'll be talking about it a little later. But intrinsic motivation is about that. And it's not all about ourselves. We have to anchor what we're doing and align it with the external system. Behavior is a function of the person within the system. So, therefore, we need an extrinsic motivation that says, here's the final set. These are external outcomes. These are external outcomes. These are things. What are the results? We can come up with the best kind of system that works for us that helps us align our passions. We can truly align unless we're talking about what are the results we're able to achieve? What are we achieving? How is the market just us? It's ultimately about results. It's not about the process. It is processes simply in mean stores that end. And it's about the results that the end said we're shooting for. So, we're going to be talking about each one of those things. There are some companies actually that are very involved in cultures. And those cultures are actually rooted in their performance management system. Or the performance management system. You see some examples up there. Not that we're calling out one or the other, but these are different strokes for different forms, right? Google might appeal to us in this whimsical idea factory. People end this 20% time that helps them to work on things outside of work. Who here is sort of the Google 20% rule? What is the Google 20% rule? 20% is your own time, which means you do want to spend... You work on everything you want. You work not quite what if you want something, but something whatever you want that an entrepreneur can add, result back to the company. So, you can go on things like the first holiday, occasionally Google would permit or sort of break from the simplistic rules and they allow this holiday theme to come up, right? Halloween and little pictures and all that. I think it's fair to play later on with something in a component's way back into Google's actual product. Senco, a Brazilian company, our Brazilian example of years is this company where everything is extremely democratic right up to the point that there are two places reserved on the board of directors for employees. At any point, employees can walk into a board meeting and sit down and observe so that employees can be participated in the board of directors meeting. Different models. If you're not performing and you've fallen into that about 10%, you will be fine. Now, certainly not a model that we would want to adopt for ourselves, but it kind of drives a type of culture and type of behavior. General Electric is sort of financially successful and quite undistinguishable. You can tell from this culture work what it is. What kind of system do we want to set up? We want to set up a system that is much more in alignment with the agile ethos. I actually haven't seen the new purpose that Diana put up for the agile alliance. It's about the sort of transcendent purpose that we would. It's about productivity. It's about helping companies be more productive and more humane. And what is the third one? Sustainable, I think. Product humane and sustainable. I think that's those are three goals or principles that we should be looking for. What are some of the goals that we want to set up? First of all, we want to set up a true performance management system, not just a performance appraisal system. Or it's got to be all encompassing and encompassing those roles that we talked about. We talked about unbundled feedback compensation of merit aid. We talked about those issues with actually having them bundled and doing the feedback only once at the end of the year. But let's begin from team members accommodate different levels of performance. People perform at different levels at different points of the career. You might have the same top performer if you move people to a different environment. So different people perform at different levels during the two years. And most importantly we want to make sure that ours is an adaptive system and what makes it adaptive we have to be collecting feedback and delivering feedback every step of the way. So we're going to talk about a few of these things. I'm going to go through this slide on intrinsic motivation and how we're doing that. So when it comes to intrinsic motivation we're going to draw on three things. And these are the three things that Dan made sense to watch out for look out for. Autonomy, master and purpose. When we're striving to motivate people and we're striving to help them find meaning within their jobs and within their lives. Not quite within their lives but please within their jobs. We need to be looking at these three things. Autonomy, master and purpose. So let's start with the first one. Way back in the 40-50 years about the bureaucracy a few years at least but here's the one who first proposed or coined this term knowledge workers. He said we need to manage knowledge workers differently from the way we manage the industrialized world. He also said that measuring productivity is actually a pretty difficult thing to do. And he said quality is the optimal not too high productivity. How do we actually get high quality and thereby high productivity? Knowledge workers need to determine their own goals. They need to be involved in the measurement criteria. So this is all about intrinsic motivation and then here sort of anchoring back, anchoring it back in extrinsic motivation is that we're saying that knowledge workers must understand what is our business? What kind of business results should we do? Are we expected to deliver? What are our customers? What do they want? How will they pay us? What does the customer consider valuable? These are the extrinsic but the stuff at the bottom of the slide are the extrinsic driving factors and the stuff at the top is how are they expected myself and maybe my own intrinsic motivation factor. So I can get to autonomy if I have my knowledge workers focus on the top part of the slide and myself and my other fellow managers can work with the knowledge workers to answer the bottom of the slide. That makes sense? Any questions there? We'll keep moving on. So Alistair I guess is another room. So Alistair's famous for many things and one of the things he's famous for is for coming up with this taxonomy of learning progression where people follow this path of shoe-on-wee a beginner or a practice level a journeyman level and a master level every craft every profession follows some sort of progression. We don't all start out as experts. In fact the path to becoming an expert is actually pretty long and arduous. Anyone here who's reading books like Malcolm's Level I think it was in Outliers Malcolm Gladwell said that you need what in order to become an expert. Anyone come up with this? Ten thousand hours. It turns out that many of the child so-called child prodigies actually just had an early slowly start and it actually worked out to ten thousand hours which is exactly what in our work base we talked about five years of solid hard work you know really practicing and working towards mastery and if we understand that there's a progression towards mastery and then we also understand and the gentleman at the back will lean on you for the process again Mihaly, six cents behind six cents behind six cents behind thank you very much he says people are most happy when they are in a state of flow on this journey of mastery when you're in the zone working writing some code designing a screen of your manager and what managers feel happy about coming up with a great system like this so whatever your role is it's how do we achieve that journey or how do we get into mastery because mastery is in some senses a journey without an end we have a destination so let's get let's understand that we need to cater to people's autonomy let's understand that we need a journey towards mastery and then also let's understand that people are looking for some meaning and purpose in their lives so it's not just about getting that reward it's actually going to motivate me here's my reward, I'm going to get 10 bucks if I build this kind of code and I'm going to sort of run like the hamster away let's part of it we need both but I'm also going to be for something a higher purpose for my life how can I anchor or teach my work that I'm doing to something larger than myself the Agile Alliance is not just about a teaching process it's about coming up with a you know it's a valuation or a system of community of team members that's driving sustainably in the programming software driving that's right so autonomy, mastery and purpose is all part of interest and motivation and I'm going to sort of leave you with this one I've talked in a fairly high level in terms of gear but I want to talk about share this one slide we have a given amount of time but here's an example of how you might do to take a personal objective kind of process a personal performance abrasion process and rather than just say deliver a line of code such as such or let's take some of these and analyze and direct market research you might want to do something like this this is called job crafting where people can look at the jobs they're doing and in those great shared of areas are their activities and this one person has basically looked at the sites of these activities represent the time she's spending on them if they're a little smaller writing quality reports less than analyzing and directing market research you see two roles over here but basically what we're saying is now let's talk about what this person does more actually one who give them a word to write themselves and if you want them to list the activities that they do the work job and responsibilities start with their passions what are your passions what are the motives if you have a passion for teaching wait 5 seconds if I have a passion for teaching others I didn't do that another 5 seconds if I have a passion for teaching others how can I align that with the work that I'm doing I have a strength of one on one communication now my strength can be aligned with my passion for teaching somebody else and I'm looking for meaningful relationships so how do I take my passions plug into my strengths align those with my motives and then you see the two roles over here one role by me and part of my team to do is best and there's another role up there and then I can start to prove maybe do this very visually put it in context with the rest of the team but at the end of this process there's several objectives that are not also allowing me to identify my own interests and motivations and also I'm looking for some really straightforward objectives by which I can measure as part of the role of the system with that I'm going to hand over to all of you who are talking about it so we have Sanjeev has described a system where we have intrinsic motivations for the things we're doing in other words we care about the work we're doing we like it we find it enjoyable it's rewarding is this enough have any of us have any of you ever started companies that you had to stop at some point because they didn't work financially what we're talking about here are in fact that life is not so simple it's doing a job you love and it'll all work out there are elements that come into play and that is the extrinsic realities in the world the fact is we need a little of both of these elements we need to be passionate about the things we're doing we need to build products that we love if we don't do that they don't come out right and their life can fail this has really been getting in all of the others as it's staying all along it's something the Agile Movement has certainly pulled the mark very closely and knows well if we think about the other side of the coin the extrinsic motivations are there any that matter and we would argue they're not where you should start you start with the intrinsic stuff but at the end of the day the extrinsic comes into play as well and if you fire this summit up what we're really saying here if you're doing things that you love you find a way to work with that complete you have to be good during processes you also need to be good during products Jeff Patton's you know this morning as well Jeff and I have known each other for a long time I think very much alike in a lot of ways what he's basically saying is we've been Agile has been a developer-convenient thing or maybe not just developers but over time testers have found a way to be included user experience background UX people have found a way to be included it's a gradually growing sort of ten big guy here and now what we're trying to do is extend it out and say also we're going to include everyone in some sort of an effective manner and here's where some other things come into play if you look at basic extrinsic motivations think of safety those of us who have been coaches always hear about this you can create an environment of safety for your teams such that they can feel comfortable to learn and experiment and fail and to see the case and in order to do this you've got to get paid enough to pay your mortgage and support your family and upgrade your life every year and again when things get a little slow for you this means that basically you have a base salary that is fair that is set by the market that is not greatly below those peers of yours and why does this matter because this is the way people are people are relativistic and they can care one another to one another keeping up the genesis things that people next door make more money than me this makes people unhappy and it's unfortunate that people are that way but they are so we need to take that into account and say alright we've got a base salary and it's fair and I think we can agree that's not anything new but there's also the idea of something above and beyond that that people are really passionate about you create a product and you've had fun doing it and everything but then let's say it succeeds wildly and you get no piece of the pie and somebody else takes all the credit does this eat at you a bit I mean despite the fact that you love the journey it was a fun journey at the end of it you'd also like to say well I created something marvelous and I'd like to share the reports there and if it doesn't happen we'll begin to feel jilted and that's this lack of justice will sink the mightiness of efforts and over time it will make even an agentically rewarding system unsustainable so what we're saying here is you need some of the extrinsic reports in order to be able to I don't think that's a good compute it's a projector but that needs to be able to need to be able to look at the extrinsic side of things and basically include people with the rewards of the efforts that they've been that they've been working with so if you look at ways to actually build this into a system what we're really saying is you've got different sorts of roles of course there and these you've got here a few hysteria types you have people who are in a strong team what do they try to do to create a good product help their team members and then report to the creator team and then do things collectively that are good for everyone so you can say that perhaps in a sense it's like video game development companies well there's something that is certainly intrinsically and extensically motivating in different ways a common metric how many of you can fill with metacritic if you've heard of this site it's a site that gathers reviews from various outlooks video games, movies and whatnot and actually there are a number of game companies out there that often are incited by their publishers to get high scores on metacritic because a lot of people look at that and when reviewers are writing things in the press it's just something that the market books that it says you've even got your succeder that you haven't based upon that score so that might need one of those extrinsic motivations now as you're going towards that you would say that if they succeeded they would be able to reap the benefits of that so there would be an element of team results how do we do that as a team to basically deliver that product and there may be some portion of individual results as well depending on your particular role in the project and what you're able to do that might come into play the basic point of this is none of these are meant to be said in stone that the individual reports should be subsumed by the larger ones and then not that they should ever exist because in some cases they make sense but they should be subsumed and then the larger system of helping your team in fact to do good stuff there's a up in this invisible blue slide up here where we're talking about are some examples in fact of different sorts of measurements that you might find that would be examples of extrinsically measured of extrinsic measures and these are mostly real examples from various projects we might say that there's a project out there to try to keep a you know a website to be designing a website because people start a process to sign up for your service and they never finish and we've got to drop out a 96% and that sucks so we want to get better than that and that would be in fact an extrinsic motivation and the things that the team would suggest ways to get there is this the product that there's job well they're there to spearhead the vision and they provide ideas and ways to get there but it's everybody's job and everybody has to say well that's our goal and we're going to play our little part and try to get a little bit closer to that goal as we move forward but it gives us data to which we can turn and decide if we're improving or not improving as we go forward I did a lot of work in it Lean and Six Sigma and things like this actually after I had learned Agile so when I was learning all of that I was doing it in the context of in fact working with the Agile teams and it led to a lot of this what we call discovery sessions these project kickoff events where basically we try to decide what these measures were and have teams share in the description of what those measures were so that decreased dropout rate would not be something handed down from on high it would be something where the team would collectively come to the realization that that is in fact to be a role model of what they're trying to do and it would be a good way to demonstrate success in the proper direction an x-ray take part data measure and it's one that you would apply a number of different solutions to that everybody has a part to play and once we come back to what you're going to see is basically then all we're really talking about is a more holistic process Agile has an era of inclusivity to it in the language but in practice it's not always inclusive it's inclusive with any smaller containers as Diana put it that slide up there so nice even the team it's inclusive but not necessarily inclusive in the broader context so how can people play a more effective role basically managers can be leaders they can try to set up systems and again I'm saying exactly what Diana was saying previously set up environments here where in this sort of a system can actually live where team members feel safe to mode their process through retrospectives improving it and all that and the product that is in product owners they're not just about the product and the result in fact they're also interested in the process because it will help them to realize a better end product so then you've got this an entire inclusive system here which involves basically managers and the team and who are you missing here the customers and users and again if you look at Jeff you got to do a little detail about what I really mean by this we're not talking about having one customer ever since it had come we're talking about actually having users and customers actively involved and basically they're telling you what their problems are or you're looking at them even better at figuring it out for yourself you need the entire team not just the product you're going out there and finding out what's happening and coming back but in fact the entire team determining what the issues are and understanding them and it's difficult to read there but then this leads to both these intrinsic and extrinsic objectives intrinsic basically we as a team feel like the work we're doing is good we can't come up with better ideas how to do it you think about a tentacle metaphor here for a moment you might say that the team would go out there they would select the seeds and they would choose the plant we think that given the environment out there what we understand about our customers and the market and the competition these would be ideas that would probably succeed led by a product owner perhaps but again with involvement in the entire team and they would decide how the process they felt would best achieve that in life scrum a dash of XP and some con models as a tool set perhaps and then they would go out there and at the end of each sprint let's say we're doing scrum if you go and you show some things to customers and users you let them use it if you look at your experience and mention you say are all the things we're doing working or aren't they is the product all it needs to be or isn't it and if it isn't perhaps we haven't chosen the right things to do perhaps our process needs improvement but again it brings that into perspective as well not just about the process but about the product capability as well and so this is basically a system that it is definitely an old loop it's happening all the time from both the process and product perspective we're building things we're testing how they work we're seeing how we did them and if we can do it better and we're moving forward over time so that's basically the essence of what we're talking about find the entrance and motivations and matters of team find things that they're passionate about and basically have them understand and involve directly customers and users in the entire process not in demonstrations of dog and funny shows but in actually rolling things out operationally letting people play with them and get some experience and you can see what's happening I'll go back to you because what inspired me to come up with a lot of this is really just pulling together ideas with different spheres there were sparks of brilliance up here that nobody else ever came to but basically I was talking about video game development because you would go with it and say a video game is interesting because it's so much more subjective and inherently it's much more difficult to say here's an idea for a game and it will work the fact is you never know it'll work and if you look at the most successful companies out there they're ones that have good ideas at the start and then take them and they just they nurture them and evolve them and pivot them and change them until they're at a diamond sheet and if they don't then it'll succeed it's not a game that's the sort of thing anybody needs to buy so it's extra important that you build something that people will want to buy and we can look at any of our products and there are elements of this in those things that we build but that's extremely compelling so for example a team might say I want to build something that would be a full voting environment that creates tension in the player and makes them nervous so a product owner, creative director game designer to come to us with all the answers we've been involved in everyone and say what do you think can make them work from a music perspective what do you think can make them work from an animation perspective how do we measure full voting you know how we're testing with people how do we know if it's full voting we'd ask the team you know come up with a way to measure this maybe to survey maybe there are very concrete measures okay so it's not all form and fuzzy and in abstract we need to get into something that's measurable and that's not always easy but it is usually possible okay so even for very abstract things like games and so just to sort of sum up the basic idea there are lots of different things floating out there we're not going to get into any detail about these right now although the browser may talk about it offline but this idea that there are different levels of feedback and different types of things that happen and I'm sort of talking about this idea of group discovery and that's what Jeff was talking about this morning as well involving everybody in the definition and ideation and iteration of whatever it is we're building a product group feedback on a product there's this idea of micro feedback and things like that which are mostly meant to be individual again process feedback about performance of an individual we'll twitter review you know Arlen you're too long-winded I'll take that as a note for my next talk the short things up a bit there are these ideas of majority assessments it's sort of an evil a much diminished thing and as a service to many people they say oh but we don't want to be assessed for maturity but what is it used for is a key question if a team says there are elements that we can see in our engineering practices or in our product discovery practices where we're more or less mature according to various benchmarks and our own interpretations then that can help us get better and as long as we're not measured and then our pay suffers because of it or what not then that's a good thing it's feedback that helps us improve our process and that's marvelous so lots of different tactics and techniques and tricks how to go about implementing this but the basic idea we need to find the intrinsic motivations that actually drive the goal and we need to make sure that we are linking these to measurable extrinsic outcomes and that we're also involving the teams and product trainers and everyone else and in finding and defining those extrinsic measures and the intrinsic ones people decide how they can best contribute they come up with their objectives teams and product trainers decide what's best for them to succeed and they test it and if it doesn't succeed they change it so an adaptive inclusive process here I'm not going to try to read all that up there but basically there are lots of problems and you see real problems having to have a coach consultant the ones you see most often in agile teams these simple ideas and values can help you address but being more inclusive that's my job we don't say we don't stay out of a room manager we don't say product manager that's just you we're all trying to build something quality here and there are various parts that we can play in it and HR needs to be part of the game too and it's an element that has been long outside of the equation and it's time for it to come back in and with that that's pretty much it I just wanted to sort of round up here and ask questions apologize for the undigivory presentation and it's that they have no privilege for the next thing yes I think the basic idea comes down to boundary context in the game if you have contractors and they're involved in a specific effort and you can say that they've got roles and responsibilities ask contractors to the team ask temporary employees of the new organization in some sense that's part of their job responsibility all we're really saying here is you need to basically help to define in fact what your responsibilities are as a contractor and to help the team define what the product or service or capability is that we're building needs to do and it's just part of your job it could be built in as part of the contract in fact as you when you get the person in terms of actual extrinsic reward measures did that make a complaint or something of that nature involved it may not necessarily be I mean it could be something that's just you know you participated in retrospectives you participated in the discovery session you give us your feedback here's the role you play in it it's just another team member it all got that way I'm not too good at this other question any of you think you've already got a system that's just like network we're actively so I'm going out there and looking in the community examples of various parts of this trying to help companies grow into the other part so this is you know we love to hear your stories basically the things that you think have worked about working your environments sort of trying to take this knowledge and spread it around it and help people find find their own way especially what happened with the proposal we already have so I think you have one hand up there for general talk about a system that the most amazing system that you've built has worked really well so I'd love to hear from you if I could place the easiest and we'll put the disappearing presentation on the slide for you to see alright thank you very much thank you always