 Welcome to Agile Roots 2010, sponsored by Version 1, Rally Software, Vario, AmirSys, Agile Alliance, and Xmission Internet. Enough Design by Ion McFarlane. I hope this turns into something generative at the end. We'll spend a little bit of time and we'll have some highlights to begin with. I want to start off with a question about how important design actually is. I don't know what that could be doing. This is an icon. Which one do people actually use? What's the difference? This one has more features. This one has more features. The difference is the design. What's the difference between this and this? The difference between this one and this one? Or is this hard? I'm scared of you. I like that one. You do simple tools like planning tools, like a game chart. I find that not easy to close up a track or a bias. But sometimes products are successful in spite of design. I use a verbal line not because it's the most awesome user experience. Mint.com is a much better user experience. The fact is, I get this because I'm not just buying user experience. I'm locked into user experience that's whatever Wells Fargo chooses to give me. Of course it's also a factor in my choice. But I like to choose a bank based on your likelihood that I'm still having my money in six months. Services that provide, quote unquote, availability that you can't clear by the time. Other factors. But my argument here is that design is really the highest leverage thing that you can do in terms of creating great products. So the problem comes in when you're looking at how you design an agile content. And I think really the core of that is trying to get at what's enough for design. How much design is really enough to get you the product that you want? Is it possible to over design? I think this is kind of extreme from a mature perspective. Really we're serving in form more than we're serving in function. And this is the outcome that we don't want. So the problem is you can also, one of the problems is really in over design is really more invalidated design. What happens is we make design decisions and we don't test them. We don't look at them and say, did we make the right design decision? And as a result we get a whole bunch of design debt. If you don't check until the end of what your design decisions are, you end up with problems like this. People familiar with Gallup and Gritty. Bridge that collapsed only a few months after it was built. There were design assumptions that went into it that weren't tested until it was much too late. One of the things about design is that we do make incorrect design decisions. There's a time when we make a design decision and there's a time when we discover the design decision. And all decisions that we make in that window are wasted effort. If that's a month or a year, that's not. If that's after you've launched, that's really bad. On the other hand, if you could design everything perfectly, if you would all up front, it would really be actually easier to do the development. Waterfall as a process isn't broken except for all its assumptions. If you could have the design all figured out from the beginning, genuinely would be simpler to come. That's something we could sometimes ignore or forget about in the actual space. But it really would be simpler just to say, okay, you're building stuff that we need to build. But even in that ideal case, if you knew exactly what you wanted to build, change what had happened during the course of working on the project, and your assumptions, even if they were completely correct at the beginning, wouldn't be valid anymore. And that results in a bunch of ways. A lot of that waste, that's the genuine. I think that's part of the reduction of your work. This is actually dangerous. Some of that creates problems down the road that are sort of in landmines. So the real problem that I want to talk about most is unvalidated. The thing is, this process is natural. We're going to make design decisions, and some of them are going to be wrong. And the problem isn't so much that we are making more design decisions. It's that we're spending a lot of time between when we make bad designs and when we correct them. When we correct a bad romantic suit. That's bad if this is a week or a day or a minute. That's just part of a healthy design process. This is obviously something that's not new to any of the animals. The idea of short iterations, and the shorter you can make iterations, and the shorter you can make the feedback cycle being critical, is endemic to the whole habit mindset. What you don't want to do is sort of go wandering off into the desert with your project. If you're a little startup and you have a certain amount of money, you don't want to be sort of now this guy wandering off into the desert if he only has water for like 4 days. You kind of want to know that you're going someplace useful. That you have enough means to get there. You don't want to end up like him. The journey is itself valuable. If you know exactly where you're going, you miss sometimes the opportunity to find a place like this. As you're wandering, sometimes you find really interesting things that weren't what you set out to find. It's certainly true of these guys. They went off, they convinced the Queen of Spain to send them off to find a trade route to get to Cuba and Cun's China. That's not what they found. It was probably useful for us since we're all in the place that they found, but they didn't go off on that journey. Even though that wasn't what they sent out to do. So there's a tension there. There's a tension between wanting to have enough to figure it out so that you're going into a place that you can get to, but being willing to make changes along the way as you discover new things on the journey. This is an example of a website. It's a social shopping site. It's Cochlead. Cochlead is actually a collaborative Twitter client for people to do brand management, and the reason that they started doing this is because as they set out to do their social shopping site, they realized as they were trying to support their brand, that working on Twitter was really, really hard if you were more than one person. And of course, being a company, they had more than one person actually collaborating on things. So they discovered this hidden need that they haven't really set out to find. But they were open to it. And I think it's really important to sort of be open to these new things that you discover on the way, even if that's not where you set out to be. So I mean again, there's this tension between going in a direction and then being able to iterate and find your way to the thing that ends up being the right thing for your company. To go back to the waterfall example, one of the things that's a really strong tension between the design community and the actual development community, this is a genuine case study. We had this one project where through market pressures, et cetera, the clients decided that they were going to use the design firm, but they had a finite amount of money to spend on it. So they gave them two months to come up with complete specs for the whole thing, and the result was this 500 page project. It was a genuine 500 page project for our design team. And they gave it to us and it was actually pretty good. This was actually an example of a fairly successful waterfall design during the course of development. So they gave us the first version of the PRD. We started developing for a while. Then they did a little bit of work on the other project. They came back with some changes and they gave us a new version of their 500 page PRD, and it was really hard to figure out exactly what to change. And then there was again the third version. The thing is, again, this was an incredibly well thought out 500 page waterfall PRD. There were only a couple big problems with it, and the site that came out of it was actually pretty good. This is the site, this is the site that we did four years ago. Unfortunately, one of the problems was the navigation. Basic navigation, this is a fairly user-generated content site, a whole lot of content. Basically three layers of content. And the navigation that they worked out just fundamentally didn't work. There was this little tiny nav element that you had to mouse across, maybe traverse 600 pixels inside of 10 pixels high boundary in order to get to the content that was very important. It was so pervasive, it caused an awful lot of rework on a lot of the fundamental assumptions of the site. This is a good example of how that unvalidated assumption, a single unvalidated assumption, even when everything else was done really well, had a huge cost associated with it. One of the problems that we see a lot in the design world is that this is the fault of the Rockstar designer. There are a few that some people do an incredibly good job about front design. John and I did a really nice job on the IPOD and the IMAC conversion. Of course they do iterate, but in longer cycles. The problem is, even if you can do it, that being able to give the feedback makes it so much easier to do this well. And the fact is, most people aren't John and I, most people can't come up with the right design from the beginning. It's so much easier to actually test things in smaller increments as you go along. There's a movement of Lean Startup which takes a very different perspective. By testing your way to design happiness, they take a very rigorous feedback and factual-based approach to do you like Coke or Pepsi? Does this design work better? Or does this design work better? But the problem that sometimes some of those we think that we can start out of any problem, this is a generic case of that. I don't think this startup is going to go any more. The fact is, you have to have, in order to be able to test anything, you have to have some assumptions that are meaningful to test. Am I a teenage girl? There are so many possible things. When you test things like two different strings in a piece of marketing you could test an arbitrary set of strings. But the fact is you're still doing some design to say, does this little Friday work as a marketing slide? Or is this phrase better? There is design that's implicit in this process. What are we getting at here in the back? I think we need to get to a point where we're respecting the design that happens in the little improvements. But still validating it and validating it with feedback and with metrics and various other things. We know that research is important. We know that doing market research and understanding our customers is really important. And yet, there's an awful lot of sites that are out here where there's no evidence at all that they've done any research. We're going to do a double cracker. It's an advertising tool that we've built. It's awesome. This is a tool that we've built. Sorry. The projector just went down. So in any case, we know the design is really important. And yet, there's a lot of products where research is important. We know that understanding marketing is really important. And yet, so often there are products that we find in the marketplace that are close that are really successful. Where there's no evidence that they've gone and done in their marketing circle. The double cracker is one where all we were doing was scratching our own eggs. We needed an agile planning tool. We happen to be a perfect target customer for that because we do agile. For many, many years now. So a tool that works for us is probably going to be better than average in terms of a tool. But that doesn't necessarily mean that there's a market for it. We haven't validated that there's a market for it. In the case of cracker, we were just building it for itself so that doesn't matter. It became a product because we sort of let it out into the wild. But if you're a startup and you're trying to figure out what should I spend $500,000 building, it's probably really important to find out before you get started that it's worth spending the time and effort on. This is another good example. It doesn't do a whole lot of different research. Google has lots of great products. This came up in the last talk. A bunch of these products. Really very good. They're unique products in this space. So there are some that are not as good. Google Video, they spent a whole lot of money licensing content. They had exclusive able to CBS. I think millions of dollars spent getting access to content. They didn't even bother to have different icons on the different shows. The top five shows on CBS. They'd be like they've lost, just had a great time. Another not so great example of Google design was approval. What's interesting about the designs that were successful is that these are kind of all things that a software developer could envision and could product manage. Google is a culture that is very focused around things very technical. Where the design is emergent from just being able to use the products as you go. There's some other interesting Google groups of something that's kind of in that space. So of these great products, another thing that's interesting is a lot of them are acquisitions. They actually innovate more by bringing products in. Things like cool video they had to replace with YouTube because it wasn't actually working for them. Only a few of them actually generate revenue. So when we think about something like Google as an example of an incredible company, it's still an incredible company and they're doing some amazing things. Actually Android is just starting to generate revenue. But what they're really good at is stuff that a developer could envision and could execute on. There's a big difference between them and Apple. Apple is creating consumer products that aren't just for people like them working in the company. But the thing is there's some hidden design going on. Even though they're not designing it, the people who work there are capable of creating very good products in the segments that they have a natural tendency to. I'm really just trying to show that that design is still happening even though it's not an implicit task in many cases. When I talk about design, what do I mean by design? This came up actually in Andrew's session too. There's an awful lot of different kinds of design. Instead of not talking about software design, I'm talking about product design or not code design, but product design, software design. There's an awful lot of different kinds of design. There's visual design, information design, experience design, interaction design, user research, usability design, usability testing, product design, branding, demographic research, psychographic research, AD testing, professional design, et cetera. So again, one of the problems that we have when we talk about how much design should be done is that we tend to inflate all these interesting things and say, well, should you do a brand design or should you do incremental design? In that, we sort of miss the thing of how much research should be done, how much visual design should be done, how much usability design, how much branding and how much all of these are different things and we need to look at them as separate. So again, when we think about how much is enough, all of these factors sort of play out into it. Each different kind has sort of a different lead time associated with this. I didn't think of it software design much like development is very practical for anyone. What I usually see as working well is where we have enough upfront market research that says that this is a product that we actually want to build. Upfront thought about what the branding is and what the market opportunity is and what the voice is so that we're not just going off into the wilderness, you know, roughly what it is we're going to build. But now we can refine as we go and then be iterative as we go and get feedback and test our assumptions and come up with big hypotheses and really build the right product. I think in an animal frame there's a bunch of things that work differently. We need different tools than we did prior to animal in order to make design worthwhile. I think there's a real tension between structurally the way design firms are used to going about design. They're used to this sort of fixed fee, you have three months' builders of design and then you're done thing. And the way agile teams like to work and usually time of materials, usually short iterations, usually lots of feedback and usually lots of flexibility. The designs for an agile team are really modern principles, really rule-based. I think it's much more useful to be able to ask questions like if we have a domain model for example, we can ask questions like where should this piece of content live and come up with a reasonable answer even without somebody sitting and looking at that specific thing. We know that we're in a trading application, we know this is trade related so it goes from the trade stage. We know that this is account related so it goes from the account stage. We know that it's this kind of module so it's going to have this kind of topography and this kind of placement and this kind of boundaries. These kinds of rules are really helpful as we also go and iteratively develop these things because we have some rules to apply to it so that the first time we're implementing it, it looks reasonable. I think user interactions are much more important for us to get in terms of design than pixel positioning. When it works really well, but for us what we see from a design firm is a whole bunch of wireframes that talk about what the edge cases are, like what does the minute, you know, what does it look like when there's nothing in the photo scroller, what does it look like when it's full, when it's partially full, when there's 20 pages, what does pagination look like? That doesn't mean something we find is really well done just by sitting down with a designer and having them just do it together. Something that works, you know, a parent works right from a developer, but it also works really, really well from a steady state design and development perspective. One of the nice things that it sort of implicitly manages to get all this knowledge transfer from the designer to the developer and back about what's expensive, what's cheap, what works well, what doesn't work well, both from a design and developer perspective. Low-fidelity sketches are so much more valuable than high-fidelity PSDs, that 500-page PSD, PRD that we got had lots of beautiful pictures of every page, but you had to look at it really closely to figure out what was the newly information on the each page. And so to me, the artifacts that are most valuable are the ones that where every line on the page tells you something you didn't know before. If we can get to where the artifacts that we're producing are just the essentials, just the very minimum things that tell us something new about the application, those are the kind of artifacts that I think are really, really useful. So getting close to the end of the talk, one big question before we go through that, what's the lightest way to actually get the job done? To me, this is a great question to ask yourself from when you're evaluating a designer, when you're trying to create a designer. The whole notion of enough design to me is there's a tension between doing too little and doing too much. There is an amount that is very clear that projects work from not having enough design, but also they can be we can get that lag down to the node and we can have these lightweight artifacts that really tell us something new with every line on the page. I think we get a long way to eliminating a lot of the ways that are endemic in traditional design documents and traditional design processes. So the framework for this whole talk came out of a session that actually had us put together back in January in San Francisco. There's a discussion that's just starting to really happen, I think, in the design community and in the actual community, about how these two pieces really get together a lot. And so this talk is sort of the beginning of what sort of call to us to talk about what are the things that really work effectively. We know what the problem is. I think this is sort of an explanation of sort of what the problem is and sort of how we can think about the problem. But I think we can also come up with a bunch of best practices. I think we have a good set of people in the room who started this discussion about what are some of the best practices that we can apply so that we can learn and teach the design community what we learned in the actual development community and learn to all be better at creating really good products and integrating the design process more fully into the development process. One of the things that we've done is we've taken a lot of work sessions. I know Andrew is doing meetings and I've also been doing a bunch of sessions. Let's go to Sirius and talk about how do these two pieces together. One of the questions that I ask a lot is what are the techniques that you use to develop the design and what are the artifacts that you generate. So why don't we just sort of start throwing out some things that have been found that have been useful to design artifacts for us to design techniques that work for you. I'll give you a bunch that we have come up with along the way. So the one that we used in the session this morning although in a very microcosmic way was Design Studio. Which I think it pulls together a lot of different components of agile thinking into which activity. So Design Studio is collaborative sketching but it's structured collaborative sketching so it's divided into passive aviation to get what's called like Ideation Clearinghouse and once you've done the Ideation Clearinghouse it's getting all these out of the air that you're envisioning. Sort of like what they were talking about in the earlier session about this gentleman who is the poker guy or whatever. So to get that thing out of his head we want to know what's in your head that when we build something we want to make sure that we're doing that with the awareness of what you build. So that's the first step. The second step is now let's iterate on that. Now we have all our ideas out of the table. Let's do rapid iteration. So everybody gets to sell out whatever else that he is to another and then you iterate it and then you wrap it so you have something that you feel has value. So people who want to do that you should be that you should concentrate on what I call formative usability testing which is usability testing that is helping you define design decisions. A really good place to start if you want to take a look at techniques is something that was published by Michael Medlock at all when they were at Microsoft Games. It's called the WRITE T.E. that's an acronym. So I would advocate WRITE testing as a way to go. Can you talk a little bit more about WRITE testing? Okay. Well I'm going to compare formative versus what is commonly called summative usability testing. Summative usability testing is basically checking to make sure that a product that you've built hits certain usability requirements. So formative usability testing is more exploratory in nature it's basically trying to it's looking at doing the minimum, creating the minimum level prototype and then driving people's behavior to it through it. So you're watching behavior and you're using a prototype but it's a paper based prototype for example something that's very easy to iterate on but what you're checking is a user's behavior and what you're trying to uncover really quickly are the humongous blocking problems right? So one of the things that's very different about WRITE testing than what formal testing is that you can make changes in a prototype after one user between sessions for example and honestly it's really hard to encapsulate this for a great so yeah. Again what you're getting at is you're trying to expose the big design problems right? These kinds of problems are like dams that block further design process once you've kind of you can't discover other ones until you take care of the huge design problems. Right. And this sort of gets at that sort of fractal nature of design and development. There are these really big things it's not even relevant to talk about mind scale questions until there's something resolved and there's really course scale questions. So being able to create hypotheses and test them cheaply, to me that's the essence of what sort of agile design is about. So I mean techniques that you mentioned, other techniques you've learned? Maybe you learned that while you were giving the presentation but you should find that design is made by a contractor and all at once and then just send them generally. So the technique that I've seen working is very simple but it's convincing the design of the contractor to a designer of a team member which is very very hard. So I did one of the things, one of the big tensions there is if the contract with the design firm is structured in a way where they can give you time and materials and engage with the team over the course of the project it's very hard to overcome that. So part of this I think is about educating the design community and the customer community about how valuable it is to have the designer fully integrated into the team. The thing I found is that the suppression is sort of like lots of managers using resources to be saying why would you want the team to come up with these kind of ideas at multiple times. The same thing I heard about designers that just need to work or leave and walk away, which is all true. I think kind of the testing upon the idea of cross-pollination of competencies. So like Jeff Patton said the closer you get to someone you know proximity and increase in proximity improves fidelity. And so working with developers and developers with user experience designers you know you have that open communication and they become in essence a part of your world and you kind of a part of theirs. And you pick up skills you know from them and they pick up skills from you. And obviously this can be taken not just user research but care programming like I believe you had mentioned. Just sitting down with the developer while he's going through things. And I think that right there goes a long way, a real long way. Especially working as a single team and a single unit. And something that Andrew touched upon is with the restaurant metaphors. He talked about the chef or whoever the owner that was successful in all the restaurants. He knew what was going on in every aspect, every part of that restaurant and he understood what was going on. And I think that's important in any team for everyone to kind of have a pretty good idea of what's going on on this side of the team and this side of the team. I totally agree with that. I see the integration here next. I've seen the integration of the designers into the core team at the large scales. Absolutely critical. But also having the pair with developers, that kind of communication that proximity that you get from them is so valuable because they really get insight into what the design constraints are and into what artifacts would communicate. And often you don't need artifacts to compare with the developer. All the fine scale stuff we see done most effectively by again by having the designer sitting down with the person who's actually doing the CSS or the HTML and saying that's a little too far to go up and then they can try it and again, they get the feedback cycle down to seconds, not days or even the iteration. It's extremely valuable. In terms of techniques and other exposures here, I like what's been said about sketching. And if it's not known, one really great resource on sketching is put by Bill Buxton entitled, Sketching User Experiences. He's now at Microsoft. Don't hold that against him. Didn't have some great people. But he goes into a lot of great methodologies. I missed the great talk on design studio, but a lot of really great ways where Andrew, like I've heard said, that a few sketches are better than one high fidelity VST. He said, I would expect that any designer on my team would be able to build something beautiful. But to be on my team you've got to build five sketches rapidly and we can quickly get a great solution collaboratively. I think it also depends on what medium you're using. You do a lot of Silverlight development in Microsoft, using Microsoft technology. In Silverlight there's a great tool that sits in between the sketching and the actual production called Sketch Club. Which is great for expressing a vector form interactive design that allows you to quickly step through a product and quickly get a sense of not just the look and not just the wireframe, but the behavior of an application. And you get a lot of great feedback using a prototype tool like that or a device that can sit in the middle and you can publish kind of what Jeff was saying earlier this morning, to a wide audience and get that feedback pretty rapidly as opposed to having to just be paired locally or all this travel. Along the same lines that you were talking about, I noticed that there's a lot of UML and modeling going on for the past few years. When you throw a lot of UML design out there and give your use case scenarios or sequence diagrams, everything, it doesn't work when someone is sitting at one place and sending out this diagram and asking other people to understand the design. I've seen people heavy on UML and how they share those designs and everything. In my experience I noticed that that does not work very well. It is better for people to raise from a little team and put up their ideas in the same terms. And then finally come up with something. I think your point on sketching was very much during that. I think the low fidelity thing that you can do in a room with a whiteboard, one of the techniques that we collected out of the place was taking photographs of the whiteboard. Making your paper prototype by doing whiteboarding is a great thing. This is a user interface, this is what user flow is going to be like. It's so cheap to do and you can iterate on it so quickly and solve problems like in the end in like a minute instead of two weeks to do the Photoshop comps and they looked at them and they didn't really work and etc. Exactly. A live example of how our experience at work was when we were designing a screen for some video solution and we were showing the police officer's car and how the video comes in and all kinds of stuff and then a guy sitting in Chicago where the other branch office is, he actually sent a design, banged out the GUI and sent it with people here. Developers were like, no no we cannot do this. There is a big technical gap on what could be achieved and what someone looks at in terms of from a pure user experience perspective to what the developers look at from a technical perspective. So people...