 talking to you guys about how do we build for people not quite like ourselves? How do we build for others? But before I get there, I want to give you a little bit of a background on me. I always like to know who I'm learning from and if I should be learning from them. So I am an applied anthropologist. I'm also a senior user experience researcher at Google. As of last week, I was research lead for Taze. That's part of our next billion user or MBU group. And when I leave, you guys, I'll actually be starting a new role at Google working around machine learning and how we apply that to help care in order to improve the availability, the accuracy of medical care. So I'm kind of a study in the adaptability of UX research methods. And what I'm going to be talking to you guys about today is about bringing user experience research into your product development cycle. So more specifically, how do we develop user insights for users who are far away or who have a different culture, a different lifestyle than we do? So typically, when I'm asked to speak at events, I'm invited as kind of an expert on emerging markets or increasingly as Google popularizes the term next billion user as like an MBU person. But what I've really come to understand is that when people are asking about techniques, when they're asking about best practices for working on MBU or doing international research, at the heart of that question, what they really want to know is how do I deal with difference? How do I approach difference, learn about people who don't share the same assumptions, who don't share the same background? And so today, that's what I really want to offer you. And I want to be very clear about that and what we're doing today. What I'm going to show you, not just about how do I build for the next billion users, how do I work in an emerging market, but how do I think about difference? How do I avoid self-centered design, this tendency to default to designing for ourselves and the people like us? How do we get out of that box? But before we dive in, before we go there, we have to start off by understanding what do we mean when we say UX? What do we mean when we talk about user experience? So I might do a bit of a roll call to begin to think about our background. Sometimes our backgrounds impact how we make that definition. So we're going to think first about roles. So who here is on, can I lose it? Just kidding. Top secret information. There we go. Thank you for whoever was pointing it out. All right. So before we dive in again, and I will try to keep a closer watch on that, let's think about roles. Who do I have here who is a dedicated UX researcher? Let's get a sense of the room. Okay. This is looking like the workshop yesterday. Who do I have here who is a UX designer who considers themselves a dedicated UX designer? Okay. So got a lot of design. Anybody on the product side? Any PMs? Founders? A couple? Great. So we do have a good mix. And so as a user experience researcher, one of the things that I like to do, particularly if we're talking to people with different backgrounds, is that we always want to start off by operationalizing our definitions, right? By making sure that we're all defining our terms in the same way. So I want to give you my definition for UX. What do I mean when I say UX? This is the definition that I really want you to try out today. That UX is really about improving the design of anything that people experience. Anything. So we're going to move outside of the app box here. And really what we're going to be thinking about is we're going to be thinking about a product team, research, design, PM, all of our stakeholders working together in order to enable our users to be two things. One, I want my user to be effective. So I want them to be able to hit the target to get done what they want to get done. Effective. The other thing that I want to do is I want to make sure that my user is delighted. That they're having a positive, delightful experience wherever they are, whatever they're doing. And in particular, if that thing is something that isn't naturally fun. So UX, improving the design of anything that people experience by making those users effective and delighting them. Yeah. Effecting them. Making them effective rather, delighting them. So let's take an example. Who here loves waiting in line? I love to queue. I love it. Not even one. This is an honest group. Usually there's a troll in the front somewhere, right? Yeah. No one likes to stand in line. No one likes it. But it's a positive user experience at Disney World. Who's been to Disney World? You know what I'm talking about here? Disney World, I got a couple. Yeah. It's a great experience. You don't mind as much, right? They've created good UX at Disney. Why? What makes that good UX? Well, think back to what we defined this as. We said good UX, anything I do where I'm both effective and I'm delighted as a user. And at Disney World, my lines are hidden, right? So I don't see a line. So I don't feel as demoralized. Instead, I'm winding through all of these areas that are filled with distractions that are filled with special effects. You think back to Space Mountain is the one most people love, right? Space Mountain. Guess how many different game centers I go before I even hit the ride? 87. 87. So we've got hidden lines, we've got these game stations. And so as a user, I'm delighted. The other piece I need to be is to be effective. And so the other thing that you find when you go to Disney is that I'm informed about the weight lines. So I know how long it's going to take for me to get to the start of the line. For each of those rides, I get a free app to help me keep up with it. And so it allows me to be more effective, right? It allows me to plan. It lets me maximize my time. I'm effective. I'm delighted. It's good UX. And we take that kind of, we want an experience like Disney because I want to get us out of the box and really be thinking about we can have good UX for any experience. Whether it's waiting in line, whether it's navigating traffic. And if we take the example of Taze of my last product, right, trying to send money, trying to pay a bill. No one likes paying a bill. But we want to make sure that it's a good UX that you can be delighted and effective when you do it. So how do we create that good UX? And by extension, how do we create really great products? Well, that we're going to focus on the user. And in fact, that focus is one of the guiding principles or beliefs at Google. It's number one among our 10 things that we know to be true. So what does that mean in a practical sense? Focus on the user and all else will follow. What it means is that we use data throughout our process. So we start off with data in order to set goals for our product. And those goals are always going to be phrased in terms of what is my user need. Then we're going to use data to find out which of the possible solutions is the best fit for those user needs that I've identified. Then we're going to think about using data to fine tune our implementation of those specific features. And finally, we want to use data to measure our success, to make sure that we are in fact getting it right, refining their feature set and improving over time. So data at every step in the process. And that, in a nutshell, is UX. And I want to show you that UX process here at a higher level. What I'm going to do with the rest of my time here with you is I'm going to run you through each of these stages in a little more detail. I'm going to do it somewhat briefly. But I do have a task for you. As we talk about each of these, as we go through discover, explore, iterate, and design, I want you to think about where you are with your product team. Where you are with your product. And which of these stages best defines your current focus? So what am I doing now? Where am I with my team? Because if you can connect to this model, then it's going to allow me to give you a more tailored, concrete advice on your research solution. So when I describe the phase that fits where you are with your team, I want you to just jot that down. Because I'm going to ask you about it. And even in a big group, right, you still want to have an answer. So we're going to go through each of these, ask yourself, me, my product team, are we discovering, exploring, iterating, or all the way into that design. So at this point, if you read the description, right, you may be asking yourself, where is this focus on people we don't know? Why are you talking about all these general UX processes when I came to learn about designing for emerging markets, for MBU, for people not like me? Why are we doing general stuff? Well, the reason is because user experience, as a discipline, develops as a way to help us focus on the user exactly for that, when that user isn't you. That's why we develop UX, right? That's the heart of it, if we take it far enough back. And that issue, building for those who aren't ourselves, it's not exclusive to populations who are obviously different. Not exclusive to the next billion users. Not exclusive to international research. It's just more obvious there. So one of the biggest mistakes that we tend to make as a product team, is assuming a common shared experience. We talk a lot at Google about the curse of knowledge. If you guys know back in the 90s, this was an experiment at Stanford, this idea of, we are very bad, we have a cognitive bias that keeps us from understanding the experience of someone who doesn't share our assumptions. I'll show it to you later if you want it to tapping exercise. It turns out that people always overestimate their ability to explain a song to someone, because they assume that their knowledge is shared. It's like charades. Who's played charades? You know that think about hone in on that feeling that you got when you were up doing all this? It's obviously Titanic. Why are you not getting it? You have it, it's so clear to you. All these things are so clear to you. The answer is so clear to you that it's very hard for you to project and get into the mind of someone who doesn't share that knowledge, that information. So we are always designing for people that we don't truly know. Because it is very, very rare that in fact, we are our user. Right? That's a big step. That's a big point. And rather than assuming a common shared experience, we really want to push towards putting our assumption about user needs and pain points to the test. Right and so everybody's going to remember in 99.9% of the cases, you are not your user. You are not your user. Give me a bit of a personal example. I started my career in Mountain View working with corporate engineering. Those of you who left Google, corporate engineering is internal tools. So it's designing tools for Googlers. And even there, as a Googler, I couldn't simply design for my own needs. Our team couldn't just design for what we wanted. We needed to understand the experience of people managers, of new Googlers, of new transfers. Even as Googlers designing for Googlers, we couldn't make that jump. Right? We had to confirm those assumptions. And so what that means is that the larger learnings that you're going to get out of today are insights that are going to help you build better products in any market. So how do we build for people that we don't know? There's a process for that. And that process is the UX process. Okay. We're back here again. And hopefully people feel a little more comfortable that you're getting what you came for, right? That we are talking about building for people we don't know, because that's what UX is all about. That's what we're doing every day, not just when your job role says NBU, right? Even when you're working with local populations. So this is where we're headed. You're going to get ready for that deep dive. You're going to remember your goal. So your goal is to figure out what step in the process your team is at today. Where's your current focus? All right. So everybody ready? Got your goal? Ready for some knowledge. All right. Excellent. So we'll start off by talking about the discovery phase. The UX process really starts with discovery with understanding the problem. So sometimes that problem is vague. It's undefined. And we're going to need to use exploratory research or design tasks to formulate our users' needs. This is the stage where we often talk about doing foundational research. So we use discovery techniques. We're doing things like immersion or contextual inquiry, observation, user interviews. But those user interviews are much more likely at this stage to be ethnographic, to feel much more open ended. We think about focus groups. We could use photo voice here. You think about the last presentation. Photo voice is that idea of using photography to help my users focus in on their experiences. So instead of asking them to express themselves in words, I give them the opportunity to do that with images, right? A homework assignment about collecting images themselves. Diary studies is also a really good technique at this phase. So discovery at its heart is about being there with your user. The goal at this phase is to really get into their own environment, to get into the places where they live, where they work, where they're experiencing problems, and to understand those problems, and also what role your team could play in solving it. Someone's discovery. If that feels familiar to you, this feels like what you're doing. I'm trying to find user problems. I'm trying to figure out how our team might be uniquely positioned to solve some of those problems. Then you're going to just jot down discovery. That's where you are. What comes next? So after I figured out the problem is, I want to move on to exploring. So the problem is clearly defined. And so now I'm ready to start to gather more detailed requirements and to try out multiple idealized solutions in the form of concepts. So when we explore, the focus is not on implementation. We're not ready to think about UI. We're not ready to think about what it looks like on the screen. Instead, we want to think about the idea. So don't skip this step. This is very important. Once I've figured out what's the right problem to go after, I want to think at the concept level, at the ideal level, what are all the solutions that we could create? And which of those idealized solutions would work best for this population? So we're testing the concepts with users to explore, to evaluate that possible solution without getting stuck yet on the details of implementation. So if that's you, you're going to jot down explore. So after we've found our problem, explored a range of problems, settled on the one that fits both with the user and with our team's unique capabilities, we thought about the widest set of solutions possible. And we figured out at the conceptual idealized level, which one of those makes the most sense, then we're ready to move into design. This is really where we start to focus in on usability. We know that we have a good concept. And so now we need to think about how do we best implement that idea with design work, and with prototyping. And so now I'm violently stepping into the stages where I'm putting it to paper. I know that this idea works. But what's the best way to convey this idea to my user? So if you see yourself here, I'm ready to work on the nitty gritty of the UI, I'm ready to think about how to implement that solution. You're going to write down design. And then once we've got the right implementation, we can put it into practice. So we're implementing. And we're not stopping there. We're also fine tuning that solution. So once our solutions launch, the teams then also going to use feedback and data to iterate and to improve on that solution. In collecting these images, I realize that Google takes a lot of water to get this done. So at this phase of the presentation, I want you to have a good sense of where you are. So where am I in that UX process? Let's just do a quick poll of the room. Who here is in the discovery phase? So I'm looking for the right user problem. Hold them up high because it's a big room. And have a look around and see the number of hands that are up for discovery. So you can prove this to yourself. Okay, great. So thanks, guys, for the discovery. We'll put those hands down. Next, I want to see who's at exploration. So I'm thinking about concept level solutions. Kind of look around, get a feel for the room there. Thank you. What about design? I'm actively creating mocks for feature ideas. Okay, got a lot more hands there. And what about implementation? I've shipped my live product. Fine tuning it. So think about if you saw what I saw. I saw a lot of hands towards the end, right, those later stages, those later processes. UX research, it can, it should happen throughout your product development cycle. But before we go any further, I want us to think a little bit about where we saw the hands and why. So the likelihood is that you did raise your hand for either design or implementation. And it's not just this group. In fact, this group is a little more balanced than average. Almost always, when I talk to groups and we think about where we fall in the UX cycle, we're really pushing towards the end of the cycle. Design implementation. Okay, why is that? Why would we be doing that? Well, if you follow it to its logical conclusion, what we're seeing is that the standard product development cycle has this near exclusive focus on designing and building solutions. We're very solution centric. And that's not a good thing. I want to give you a plug here to really rethink where you're investing your efforts. Because a solution can only be as good as your understanding of the problem. And anyone to push on that, I'm hoping you'll even kind of get a little more involved with me on this point. I'd love for us to just kind of repeat that and take it away as a fact. Right? So again, she said with me, a solution, a solution can only be as good as my understanding of the problem. Super. One more time. See, take it away and tell it to somebody put it on Twitter. A solution can only be as good as my understanding of the problem. Awesome fact, right? And so do you think about where you are? Fair enough if you said, yeah, but we did those already. And now we're ready to solution. You're in the right place. But if you gave short shift to understanding your problem and really exploring at the conceptual level, all possible solutions, that's a discussion you really want to have with your product team. So that said, I am here to meet you where you are. So I want us to go back to defining what to do with that. We know where you are in the UX process. Let's talk about the type of research that you're going to need. And we're going to do that with ice cream, because I love ice cream. And it photographs well. All right. So at the most basic, basic level, the key to determining the research that you need, your really biggest decision is going to go back to what is my primary research question. So ask yourself, what's our biggest question right now? What do we need to know? And then you're going to take that question. And you're going to ask yourself, is this a usability question? Or is it a usefulness question? Am I asking if my product, if my feature is usable? Or if it will be useful? So one of those is a, can they use it usability? The other is, will they want to use it a preference question? So let's take it to the ice cream. If I'm differentiating between usability and usefulness, melted ice cream is a usability problem. Bear? So melted ice cream isn't usable, not from anyone. No one likes it. Nobody wants it. No matter who I talk to, it's a problem. So those findings are going to be very universal. If I contrast that with something like ice cream flavor, flavors are a usefulness question, right? Do you like it? Doesn't match your preference? And you could start to see that it really matters now who I ask, and whether those people are representative of the target user for my product, right? You can prove that to yourself, who here loves bubblegum ice cream? Awesome. Who here like myself thinks this abomination? So this is not a universal question, right? It depends. Think about it. Who I choose to talk to might give me very different answers here. Compare that thought experience, what would happen if who here loves melted ice cream? I love to eat my melted ice cream right off the floor. Yeah. Well, always one. Yeah. So if I ask you, do you like melted ice cream? I'm getting at, is melted ice cream usable, right? I'm trying to figure out if there are any problems. And I should expect a universal response. So it doesn't really matter who I talk to. If I move from usability to usefulness, is this something that meets a need for you? Now I'm talking preferences. Now I will see a difference of opinion and who I talk to is essential. And I have to put in the time and investment to make sure that the people that I'm talking to that my sample is reflective of my potential user population of the folks I want to target with that product, all right? So super valuable because it tells me how to sample. And it's always a garbage in, garbage out thing with research, right? If I start with a bad sample, if I talk to the wrong people, then my research results are not going to be helpful and useful. In fact, they're more likely to be misleading. So for usability, and this is the great thing, because that melted ice cream is never usable not to anyone, anyone can spot that problem. And so sampling for usability is really easy and straightforward, right? And it doesn't take that many users. So if you think about Jacob Nielsen, kind of classic recommendations there, perfect, he's going to say five, based on modeling of problem discoverability. So five users are going to be able to find about 85% of my usability problems. Given testing session, you can think about more recent work, if you want to take it up to the kind of Jeff Sorrow, James Lewis, they pushed really hard on the fact that not all problems are equally easy to discover, right? That we have variable levels of discoverability. So even here, though, even if not all my problems are as obvious as melted ice cream, if I go up to 10 users, I'm still going to be able to find all common issues, and nearly all moderately occurring issues. So if you think about for most UXRs, myself included, I usually shoot from six to eight. You're new to the job, you want to impress the boss, you could go to 10. Anything over 10 is really a waste of your resources. So that's usability. What about if I'm thinking about usefulness, about the preferences here? Now things are going to shift and change a little bit, right? So guess who we need to talk to now if we're going to assess usefulness and preference? We have to talk to people who match our target users. So we have two options to create that match. We can do a random sample, or we can do a propulsive sample. So with the propulsive sample, what I'm doing is I'm looking for people who are deliberately selected to represent variables of interest. Things that I know from the beginning, a priori, should really impact a person's experience in the domain that I'm building for. So a random sample is going to work for you when you have big numbers. But it can be problematic a lot of times, especially if you're working for a smaller company for a start-up, those big numbers can be out of reach, right? Hitting that many people. But the other concern is that it might generate the wrong kind of data for you. So if we're looking at big numbers, we're often ending up with quantitative data, not the kind of qualitative data that would let us generate new ideas. Now the good news here is that if you think in advance about what are those variables that are likely to change how my users think about the product, and if I deliberately test with users who represent the extremes of that variable. So we're talking about sampling for heterogeneity. Then I don't need a big random sample anymore. A well-chosen, proposed sample is going to give me equally valid results. So think about it. For my domain, for the place that I'm building, think financial products, right? What are the two or three things that would really change how someone experienced a financial product? Probably where they are from their career, right? A student is going to have a very different experience with finance and spending that someone mid-career will have. And so I make sure that I'm sampling from both ends of that extreme. I compare those experiences, and if they're the same, then it's a good guess that there are not going to be a lot of differences in between. Yep. And we've statistically checked that. You can see a lot of work from a hand worker that proves that for you. So if you think to yourself, what are the things likely to change the experience, deliberately sample for those, then you often don't need that big random sample anymore. All right? So how do I know, though, if I'm doing usability or usefulness? That's going to come back to your stage, right? And so one solid indicator where I am in the process. So if I think back to my folks who are doing discovery, who are doing exploration, yeah, discovery, exploration folks, this is really nice because all of your work is going to be about preferences. Everything you're asking at discovery and exploration is really focused on figuring out what are the pain points that I want to go after? And what are the solutions that best match up to those pain points? So the thing that you really need to keep in mind at this point is that you can't solve for everyone. So you're going to have to work it with your team, come together on who is my target user, and consistently both build for and test with those folks. Okay? Perfect. If I'm on the design side, if I'm evaluating mocks, if I'm iterating on a solution that I've already implemented, working with a live product or feature, the job becomes a little bit tougher because some of my questions are going to be usefulness. Some of them are going to be usability. Okay? Because once I have at least mocks, I do have the ability to focus on usability. Could someone use this? So the better that I can separate, once you start to separate out what's a usability question from what's a preference question, you're not going to waste time for a usability question worrying about the sample, right? I'm going to grab six to eight folks who I can corner, and I'm going to run them through the major tasks in my product. If it's a preference question, now I've got all the time that I've saved doing quick usability to make sure that, yes, my sample does reflect my target user group. So I have to ask myself again, anytime it's preference, can I trust that this group represents my users? And making that distinction is going to be one of the easiest things that you can do to up your UX game and ultimately build better products. So as a takeaway, as a final, good UX is universal. So whether you're working for an MBU population or not, your UX strategy is really going to be the same. You're going to focus on the user, you're going to think about variability, and you're going to utilize and adapt patterns. So we spent a lot of time on the first two today. I'll leave you with this site if you are working on MBU populations and you want to have a look at some of the MBU design patterns that Google's put out, check those out, and those are going to be people you don't know because in all honesty, an MBU population is not an APAC population, right? So if you're in this room, that's not you, right? So have a look at our patterns as well. Thank you guys so much for listening. I appreciate your time and feel free to follow up with questions.