 Hi everybody, thank you very much for having me here. It's really an honor to be invited. Thank you to Kaladar Bapu. I apologize if I, which were the name. It's very challenging for me, especially on West Coast time. So it's, we are now 12 and a half hours in the future of where I work. So I feel that if I could do my research here, I would actually have better insights than most of my teams back home and I could tell them what the future looks like. So thank you so much for the wonderful introduction. I really appreciate it. The talk today is really about embracing a team approach to research in the agile world. And before I get into that, I want to just tell you a little bit about myself to help explain a dilemma that I find myself in day-to-day, although not as much now in my job at Salesforce. I'm a kind of classically trained human factors engineer, cognitive psychologist, and systems engineer. My degree is in engineering psychology from the University of Illinois Urbana-Champaign. Perhaps maybe some of you have heard of it. I started my life out doing basic research, building cognitive models and studying visual perception and understanding how people understood their world when they read and also when they looked at scenes. We did a lot of eye-tracking research back in the 1980s to build models of human information processing. Over the years, I got more interested in decision-making, started looking at display design, started exploring how people understand data and making insights from data. That's where I work now. I'm an analytics user experience doing research, which is just a fascinating world to be in. I've always had a lifelong interest in learning. I study how people learn and how people understand things. Because of this background, I think, I have a dilemma. That dilemma when I work with my product teams is that I, as a classically trained researcher, am coming to teams that have a number of pressures on them every day. Many of you know this because this is the world that you live in. In Agile at Salesforce, we work typically in one and two-week sprints, sometimes longer, but kind of a traditional sprint cadence. We have, in my mind, that's a pretty short development cycle. We have cross-functional teams. They all want a greater understanding of the end users and their context of use and their goals and their pain points as Adam so eloquently stated in the opening keynote today. Also, for me, the challenge is they want just-in-time answers. They have requests that say things like, we need to make a critical decision and, oh, by the way, we're going to start working on it in the next sprint, which starts next week. And we're going to close that sprint in under a month from now. And so we'd like to have answers by then. As a researcher, that just makes me crazy because I want to do research that's comprehensive. I want validation of my results. I want to minimize bias. I want to improve my likelihood of actually understanding what's really going on. So that's really, really difficult. Meanwhile, my team just wants to implement before we fully understand the problem. It's like, hey, just give us a hint of what to do. It's like, no, we can do better than hints. So how can we improve research and understanding for our teams, especially working in rapid development environments? I'm going to ask you to look at an image. It's an illusion, so there's nothing hidden here. For those of you who've seen this before, please don't shout it out. I'll ask for your help in a moment. For people who haven't seen this before, I'd like you to think about what you see from this image. So just take a moment, look at the image. Now, show of hands. How many people who don't know what this is think you see what this is? Okay, so under half, it's kind of hard for me to see this entire side of the audience because there's like a sun in my face. So I trust that some hands are probably up. So for people, somebody who raised their hand, can you just say what you think it is? Dog. So we have a lot of people saying dog. How many people do not see a dog? Awesome, that's great because the illusion actually is supposed to be a dog. Now for somebody, I heard dog from over here. Can you explain where you see the dog? Where's the dog located in this screen? Right side toward the center, right side toward the center. Do people who don't see it start to see a dog? Anybody starting to see a dog? What if I told you the dog was a Dalmatian, right? It's that white dog with black dots on it. What if I told you the head was toward the middle and the head's kind of down, kind of like grazing almost like a cow, which is awesome to see here in India. This is still my most favorite thing about visiting India, seeing cows on the streets. And for people who can't see it, if I maybe put a shadow around it like here, to get a sense of where that is, how many people now are pretty sure you see a dog here? If you don't keep your hands down, it's fine. Okay, so with a team approach, we can improve the greater overall understanding of the team. If everybody's involved in research and understanding our research needs, we're never going to have 100% understanding, but we can always improve how much understanding we have. We can get a greater understanding of our user needs, goals, context of use, also the opportunities that we have as designers of solutions for those folks, and a better understanding of what we don't know. Say, oh, this person in QE still doesn't see the dog. What can we do to help her? This person, the person in design doesn't see the dog. How can we help him? So here's another way that research can improve thinking. We're going to do an activity. Anybody who's taken a college psychology class, you're about to experience something you've already experienced before. I want to ask you how accurately you can read. Specifically, we're going to look at these colors. So let's just do a favor and if we can start at the top and go to the bottom. So let's start reading. So it's yellow, red, blue, and black. So now you know what the colors look like. All right, we're going to do this a little bit more colors, a little faster. Go to the left-hand side at the top and just start up there and we're going to go down. I'll start with you. So it's yellow, red. Okay, now go to the right-hand side and continue. Red. Okay, great. And I'm not trying to preach a sermon. I don't need everyone to be in sync. I just want you to feel what that was like mentally as you did that. Now, most of you know what's coming. And even though you know what's coming, it's not going to make it any better. I'm going to ask you to say the colors. Don't read the words. Ignore the words. Just say the colors. Let's start at the top. Yellow. Now let's go to the right-hand side. The first color is green. That is awesome. So I don't know what your experience was like, but I heard a lot more uniformity at the beginning. And then when we got to this screen, everything kind of fell apart. It's like there's lots of different speeds and the speed was inconsistent. It's like yellow, green, wait, black. What are you doing to me, Steve? Why did we invite you here? So this is, I think, a great example of how engaging in a team approach can help us by improving what Daniel Kahneman, a Nobel Prize economist and psychologist, calls slow thinking. It's the system in our cognitive processing where we deploy attention, where we deploy resources to look at logic and apply things that go beyond our biases and our natural associations. It counters fast thinking, which is kind of that automatic, almost immediate process that you felt when you saw the colors. And then when you saw the colors but the words, most of us read more than we name colors. Like naming colors is not a task you do every day. Reading is probably a task you do every day. So it's easier for your brain to do. So that's a much more automatic process. Similarly, when we engage in a team approach to research and we don't put our researcher off in a little corner or we don't make somebody just take care of, for example, usability testing, we improve everybody's awareness of what the problems are and we process more effectively. We are no longer just victim of our biases and relying on heuristics. So how does this apply to Agile and how does this whole teamwork thing apply to Agile? I would argue that Agile is all about teamwork. I'm often shocked. I teach at UC Berkeley part time and I'm shocked with my students, especially those who work in Agile. I'm shocked that they've never read the Agile manifesto. Has anybody here seen the Agile manifesto? If not, you're looking at it. It's on the screen. It's not long. It's really short. There's a set of Agile principles behind it that are a little longer, but not much. So I'll be citing some of the principles for the rest of this keynote. But I'd like you to just be aware that the majority of Agile is about working as a team to make change faster. And at the top here, it says that regular intervals, the team reflects on how to become more effective, then tunes and adjust its behavior accordingly. That's all about working together. It's all about what Adam just talked about in terms of taking a collaborative approach, idea number one for his strategy to be effective. So Agile requires teamwork. So at the beginning, I advocate that in order to engage in a team approach to research, you need to surface your concerns and your worries early. And I think the Agile principle here that goes with this is the most effective and efficient method of conveying information is with a face to face conversation. We don't have to be face to face. It could be remote. You could be engaging through even email. But I think it's important to surface our concerns. And a psychologist and researcher, Gary Klein, who studies expert decision making amongst people in high risk environments like smoke jumpers. These are people who jump into forest fires and then put out the forest fire. That's all around them. They make split second decisions all the time and they don't die. They've made these decisions so quickly over so many years that they're just innate. It's just like reading. It's that conflict you felt between reading and color naming. Their decision making is so great. And so what we need to do is we need to conduct a strategy to focus us on what we think the risks might be before we go into the process of developing our product where we're making those split second decisions. Gary Klein suggests that having a premortem is a great way to do this. A premortem is simply pretending that your project has failed before you've even started on it. And the strategy is really simple. The tactics are simple. Imagine it failed, sit around in a room, brainstorm why, only take a minute to come up with different ideas and then talk about them and then create plans to say, yeah, if what Papu said is correct, how can we prevent that because we haven't even started? The benefit here is that you are able to counter bias. You encourage dissent on the team. You create a safe environment where if a person has a concern that maybe you're taking on too much in your sprint, then there's a safe space to talk about it. And it also sensitizes the team to these problems. So if you see them crop up, the team can say, hey, remember when we had our premortem? I think some of those concerns are coming up. So let's take action to address them. Recommendation number two is understand your questions. For those of you who attended my workshop on Monday, you'll apologize for the repetition here. I think this is about the Agile principle that says simplicity, the art of maximizing the amount of work not done is essential. I love this, right? I come from a world where we have to write pages and pages and pages of information to prove to people that the research that we've done is based on a solid foundation. In this context, though, when we're looking for the amount of work not done, we're actually looking for ways to avoid doing unnecessary work. Again, like Adam said, we're avoiding coming up with solutions to problems that don't exist. So I think a great way to engage in this is by building a fishbone diagram. Psychologist Jeffrey Soro refers to this as a cause and effect diagram. Basically, all you do is you identify your main concern. You brainstorm causes. So maybe a main concern is people don't buy our product. And you brainstorm the reasons why people might not buy your product. Ideally, it's better to do this early, but you can do this late as well. And you group them into categories that make sense, maybe usability concerns, maybe functionality concerns, maybe quality issues. And then you use that to drive your research. This helps to illustrate what you've covered and also what your gaps are. If you have nobody looking at or investigating, for example, pricing, and you have a complex pricing model, maybe you should think about deploying some resources for that. It also facilitates prioritization. When we're working in Agile or at Salesforce, most of us work in Agile Scrum, we have to prioritize. We have to ruthlessly prioritize. And if you don't ruthlessly prioritize, you're at risk of shipping nothing. And so we have to figure out what is our top priority for the next two weeks. But what that means is what are we not going to cover for the next two weeks? And it's really important to be aware of that. And so just an example, and all of these slides, by the way, are available on SlideShare. So if you just go to SlideShare.net, look for my name, Steve Fadden, and I'm from the UxIndia. They should pop up right to the top. This is an example of what Fishbone looks like. And so you can create your diagram. You say, maybe there's some usability concerns. Maybe users don't know how to start. Maybe users think it's too many steps. It's an inefficient process. And then you can highlight, okay, this will be the current research focus for this sprint, or for the next month, or for the next release. You can use that to drive a conversation with the rest of your team. So they know if there's a pricing question, we're not looking at it right now. If that's a key concern, you might want to revisit that as a team. If that came up during a pre-mortem, hopefully you haven't prioritized in a way that gives it short shrift compared to the other items. So my third item, and for those of you who are in my workshop or who are familiar with the work that I've done in the past, you know, this is my area. This is a thing I love. Implement flexible methods. Agile is about welcoming changing requirements even late into development. Researchers, designers, developers, I think hate this because that means you're going to change stuff. So if I was working on code, or if I was doing some designs, or I have some schematics, or I started creating my red line specs, and last minute we're going to change it. Now I have to change everything. So, gee, that sounds terrible. But I think it's really great that we now have kind of access to tools that are cloud-based and tools that we can deploy around the world 24-7 asynchronously. There are research methods that we can use without having the person in the same room with us. In fact, without even being at the same point in time, we can get research answers to questions. Not all of the questions, not all of the answers, but enough so that I can deploy some studies to start collecting answers while at the same time maybe doing a site visit, maybe engaging in in-depth interviews, maybe doing some observations, maybe doing some classical usability testing. Right now I actually have two studies running right now, and it's great. Every morning I go to in my room, I open it up, I refresh the browser, oh, 157 responses. That's what I saw to one of the studies this morning. It's really great. I have some bad news to give one of my designers, but that's okay. I was able to do this research while being here at a conference, as opposed to saying, sorry, no support this sprint. And the risk for me as a researcher when I say no support this sprint means, eventually they may decide, I don't need Steve. And so what I would argue is use a team approach so that everybody understands what these methods are and the benefits of them, so you can collaborate and brainstorm. How do we best use our resources to answer the questions during this period of time, this sprint, this release, this year. These tests are listed here. Some examples are click testing, where you kind of ask a person, where would you go if you needed to add a reminder to your calendar. Here's a result of a click test from a fake UI that one of my interns built for me. Very grateful. There's a commenting text on the right where you basically just show a person a UI or an image or a scenario or a set of screenshots and you say comment on things that stand out. Positive or negative. And so there's some helpful information there from a user who participated. Involved customers often is my fourth recommendation. Agile is about engaging developers and other stakeholders. So in Agile they refer to it as business people. I would argue it's everybody, it's not just stakeholders, it's also your customers. Those are stakeholders. So involve them daily throughout the project. I know that's hard to do. So what we need to think about is doing what Steve Blank, he's the many credit him as starting the lean startup movement. He's also a professor of entrepreneurship at Stanford. He says create Goob opportunities. What's Goob? It's not a new snack. Goob is a get out of the building opportunity. Get out of a building. If you have a startup, get out of the building to test your value proposition against as many people as possible to give it a chance to fail or give it a chance to succeed or move closer to success. Same holds true for design, for functionality, for product solutions. You can do this through field visits and remote observations. You could just set up a Google Hangout or a WebEx or a go to meeting session and say hey can you just share your screen and share your audio so I can hear and ideally if you can maybe just point a web camera to the office just so I can get a sense of what your work looks like for the next hour. Just ignore me and I promise I won't do anything with it. We're just using it to give the design team that I work with a better understanding of the context in which you work so we might be able to build some empathy for better decision making. Also have brown bags. Invite your customers in. Give them freebies. Hey, free license to our software. Free gift card. And then also consider JAD and participatory design. JAD actually has been around for several decades. JAD is joint application design and JAD basically says create a council of customers and have them engage with your team at periodic elements. I think it's unrealistic to actually embed customers in our groups because A how many customers work for companies that are comfortable saying sure yeah go just be at sales force for the rest of the year. But having them in maybe for a set of two hour workshops over the course of six months could be a great way to get continuous feedback from them as you're planning what your requirements are and what your designs might look like. The fifth item is develop customer empathy. And I love Chloe Walker. I think sorry if I forgot your last name Chloe did a great workshop yesterday on building empathy. Empathy is really about understanding people's needs and emotions and feelings and philosophies. And I think that really ties to the agile principle of building projects around motivated individuals. If your developers quality engineers etc. Understand what the issues are then I think they'll have a better chance and a better opportunity to design solutions that matter. The way you can build empathy is by communicating compelling stories. Nancy Duarte is one of the leaders in the field of storytelling and great presentation giving Jennifer Acker is a management professor at Stanford and she looks at the role of storytelling especially in marketing. And they've identified that good stories have a big idea. They depict the customer as a hero. They give customers a voice and those customers use that voice to highlight gaps that need to be closed ideally by your team. Stories are memorable engaging and emotional and when we communicate compelling stories we create lasting feelings. Maya Angelou now deceased poet and activist says people will forget what you said they'll forget what you did but people will never forget how you made them feel. I think that's really what empathy is about. You want to make sure that your entire team is in your customer's shoes. Before I get to my last recommendation let's do a quick pop quiz. There's some research that shows that people procrastinate but people are least likely to procrastinate when tasks are represented in a way that make them most and then there's a set of choices here. So I would like you to vote with your hands. Let me know if you think that people procrastinate least when a task is most is represented in a way that makes it most abstract. So abstract tasks least likely seems really challenging. How about attractive when a task seems attractive you're least likely to procrastinate it. A few more hands. About concrete when it's task seems really firm. A few more hands. How about difficult. I'm going to challenge you with this really hard task. And how about important prioritize the heck out of it. Okay so good news for all these who said those of you who said concrete. Concrete tasks are routinely and repeatedly shown as being the most likely to get completed. Faster to get completed on time to not be delayed. If you celebrate new years and you have new years resolutions. Typically people have these really vague abstract ambitious attractive but not concrete tasks right the task is I'm going to lose weight. Maybe I'll even lose 10 pounds. But instead I might want to say I'm going to lose weight by eating one fewer cookie every week. Well eating one fewer cookie first of all I need a baseline how many cookies I'm eating. But if I eat just one less cookie. It seems a lot easier it seems a lot more attainable. And so this leads to my last recommendation which is making solutions actionable. Agile is about maintaining a constant pace indefinitely and that means people need to be working on bite sized chunks. Concrete. I think researchers struggle when we want to provide you with a thesis that proves the work we did with is amazing and we have all these findings when in reality we should be providing brief recommendations that are achievable and prioritized. And prioritization really comes back to what Kimberly Weaver at Virginia Tech calls the presenters paradox when we give presentations we think that by providing more information it's better. But in actuality less information is better because people who receive information value it based on the average of the messages that they received not the sum. So if I give two high priority recommendations my audience is more likely to come back and say it sounds like we have some really high priority work to do. But if I give two high priority recommendations and then one medium priority recommendation the average is that of that is medium high it's not as urgent. So avoid the presenters paradox prioritize ruthlessly. So let's recap team approach surface your concerns early understand your questions implement flexible methods involve your customers often develop empathy for your customers and make your work actionable. And I think by following or doing our best to follow these types of principles with our teams we're able to address the dilemma I talked about earlier and thank you very much for your time.