 Let's reflect on what worked and what didn't work and what challenges you all had and What your experience was like would anyone like to share and we should probably get the mic Make up if it's possible Yeah, anybody like to share your experience Either interview we interviewer as a team that was planning. What happened when you started? Yeah, one second. Mike's coming. I thought it was really interesting how Although we didn't ask specifically to get to the core value We really understood based on the questions. We had an excellent interviewer and an excellent interviewer awesome and The family was very very important. You could tell from the the answers that were given and and it's I never really thought that Interesting you can do that without explicitly. Yeah getting to what what the real? I love that comment because it so in the next section. We're going to be talking a little bit about listening techniques Indi young she wrote practical empathy. I talked about that in the presentation It's amazing when you don't necessarily have like an interview type of approach What stuff comes up and if you don't have like a hardcore agenda? You'll actually learn a lot more about those values and those principles and those reasonings that people have so that's really awesome Insight appreciate that as far as the Planning was concerned. It was pretty okay in terms of just putting down points. Yeah, but when it actually came to the interview Because there was no pattern in the the travel Person who traveled because he also liked he once traveled alone and once traveled along with the entire family So the experiences differed in planning in the purpose So there were different layers in in in terms of the same traveler. Yes who has traveled two different What to say in modes in terms of experiences? It's wonderful. So the planning also differed Yeah, one was a impromptu thing which he didn't care how we traveled and in a case of with family It was a long list of To-do list structure in terms of arriving at the destination and the way to travel So you found a difference in context based on what people were trying to accomplish Yeah, values change based on the context of use so that's extremely Important to get and you often won't get that if you just ask somebody what's important So how about problems anybody want to talk about problems that they had or challenges they encountered? And if you can if you're at a table that's still doing the interview, please stop Um Just to make it easier for us to hear and also I'm starting to lose my voice and I can't hear with background noise I need my bell Yeah, can we folks can we just yeah, let's start focusing on the conversation now Sorry, I love that you're like involved and animated the goal here though is to activate you to these concepts Not to continue to solve the problems that I've given you So I want to hear what are the problems people had either in planning in execution or Asked after you started asking questions. What was that experience like? I? Feel we got quite a mix of like problems We got from people so one of the main important point was Communication gap between the travel Elton and the person they were expecting few information beforehand But a lot of times they got the surprises in the last moment that somehow this didn't work out and the prices went very high Why didn't you tell me before so those kind of issues which you normally think it's already sorted those came in a very last moment So that affects the whole plan your budget your pocket everything So I think that is the one major factor people went through I think that's the major communication gap is the main thing Yeah, communication gaps and little gotchas little like complete reversals that happen as a result of the questioning experience Really can lead to new insights, but also problems as the interviewer. Yeah, thank you Any other product up I up here. I'm gonna slowly make my way to the front to just so I can get back to the slides so Our what we had to do was figure out what people we wanted to keep the Research really broad and not really go into them, you know Interviewing people deciding what we want to find but then here are multiple things right so when we started We had like a really lengthy discussion about how Our ideas about what kind of questions classify as broad and narrow. Yeah, I think the 80% of the time was trying us trying to decide what that what those questions would be But then we kind of reached a common ground and he was the interviewer. I'm the interviewer And while asking questions like we wanted to keep it broad So the first thing was what does this person feel about travel? Yeah, and the second question was what do you? Can you think of a tell us about a time when you thought? When travel was some give you you walked away traveling and then remembering something something that stuck with you Yeah, it can be good or bad. So while having the discussion. I I knew I made a mental note That I'm not gonna lead this person into thinking that was a good or bad because they are immediately gonna go there Right, but what I did while I practiced it while I was I do this also for my job When I started interviewing him You're also an emotional person at the end of the day. So depending on how that person reacted I felt like I needed to guide him a little bit So I'm like making those calculations in my head at the same time I'm like I shouldn't have said good or bad because now I've directed him So this again with in my experience using I mean I do a lot of interviews for design research And this is something that you just have to keep doing it to get better. Yeah It's I really appreciate that point and so starting at the beginning you were mentioning that it's very challenging when you're trying to go broad because For us especially in in the world of UX or any kind of product design or any kind of engineering environment You want to get at specifics fast, right? Like because that's how you solve problems But this isn't about specifics. It's kind of like the very first comment that we had from from from the table here There's a lot that will just come out of the conversation if you trust the process and It's tough because the interviewer and the interviewee you're both on edge You both have emotions those that you know the interviewee It's terrifying to be the interviewee because you don't want to give a wrong answer Because you have a feeling like oh, they're looking for something specific. I now need to find out what that is It's really a co-discovery experience and as an interviewer you need to get the right answer, right? You're you're a journalist you're finding the story, but just trust the process it will come and I love the the idea that If you start with something broad like You know tell me about a trip that you took and Some of your memories of that trip or some of your impressions of that trip. That's really a wonderful open question But as soon as a person I can see myself doing that as soon as the person starts going down a path Or I'm like, oh, yeah, this is a pain point. Yeah, okay I'm gonna start nodding more and giving more affirmation That's just gonna lead to more of the feedback. That's exactly like that so you need to kind of almost go into Observational mode and it's like, oh, okay, that's interesting. Oh, please say more You know nod your head periodically, but don't do it so that you're like specifically drawing out specific points And that's so hard. So at the end of the day all of these techniques and the fact that you've spent some time Practicing creating laddering questions and critical incident questions You can now use these anytime if you take uber or Ola or if you're at you know Cafe coffee day talking with the bird the cashier the barista you can you can whip out a critical incident question It's not a big deal. You can do some laddering, you know, the driver says. Oh, yeah Well, I'm only doing this because of blah. Oh Why is blah important to you like you can you can bust out these techniques any time So that it'll become part of your muscle memory and it'll feel more comfortable when you're doing it in a quote research setting But then, you know, you don't need to we need to get out of that like as interviewers get out of that mindset where You know, we're formalizing that event. Yeah, and more of like just like I'm doing this Like I was telling these guys when I walk home every day They're sent when does and whatever and just go have a regular whatever conversation with them because sometimes It might not occur to you at that moment But then later on when you think about things we in our subconscious is making a lot of connections And it just pops out another time. So this is a practice. I've been doing and it's been kind of helping me Map emotions like others in my own. Yeah, thank you very much Yeah, you need to get out of the idea of solutioning and just into listening and I think that's really really hard for us And so as much as you can practice, that's that's ideal. So, you know, it one more back here. Yeah I'm Swaraj and thanks to Swatika. It was a very good interview from her One thing I would like to share that while asking the questions there should be a balance like okay It's good that we are trying to know what the interview is liking about the traveling But what didn't work actually? Yeah, like Nowadays we book our travels and they provide us a package But sometimes it happens that people I would like to do it on their own like as per the budget their time availability But where we Personalize that experience for them like if they are trying to plan it on their own can we help them also? From from a company or from a app side. So that's not happening current currently and That's what I I see that the question should be balanced between that what they likes and what didn't work Basically, basically then we can bridge the gap and fix the solution for them That will help more. Yeah, absolutely And I think so you're and it sounds like you've identified and highlighted some needs that are in the opportunity space But keeping in mind for this activity. We're not really thinking about design. We're not really thinking about the opportunities We're really thinking more about What does that conversation look like right and what are the values from that conversation? So I think I think understanding what's driving your user population to value Yeah, we'll help you design when you finally do implement those new features What's the right way to do exactly so basically that will come up when we will ask that what didn't work during the travel Yeah, basically they may you know might have you know planned something I mean, let me just let me get it feels when you're saying what didn't work You're already going into pain points. You're already going into solutioning stay up here and stay broad Okay, we're not at the solutioning level right now We're not saying how would you feel if I am Clint implemented a scheduling system in this app? No, definitely not because you want to be able to understand. What do you value, you know family? I value family. Okay. We're thinking about implementing this new feature in our app How could we tie that back to this family thing as a value, right? So don't be focusing so much on the pain Focus on the value. Yeah, that's what I get to the balance between both something if it but I would argue There's no balance here at all. We're doing value focused interviewing. We're not doing pain point interviewing That'll come in the next activity. Thank you Yeah, good observation. Yeah, let's do one more Sorry, I'm I was raised with lines and cues and so I'm gonna just follow the queue I'm interviewing a little closer. I'm my interview. Yes, interview. And then I was asked critical reasoning Questions critical incident question. Yeah, and then I was asked the why questions So what I thought that it would be better if we mix them hold the mic uncomfortably close to your mouth. Unfortunately. Yeah So what I Found is that if these questions could be mixed with the techniques. Yeah, it would work better Yeah, absolutely So the reason why I had all of you planning both of these types of questions is because these questions come in concert You never know when you're having a conversation even an observation where you realize. Oh, I wonder why this is going on I'm gonna ask why and then the answer should always be unsatisfying So you need to ask why again and then the person might say well, you know I really it's really important for me to travel to see my mother because she's in fragile health and actually the last Time I traveled it was a real disaster. Oh, can you tell me more about that that disaster as you mentioned it Tell me about what happened. What did you do? How did you feel? What would you wish went differently? Like you can swap the critical incident question and the why question all the time and get more valuable data that way Especially for things like needs and values and emotions. Yeah, it's a great observation All right, we'll do one more Can we do one more from a different table? I'm sorry. I don't want to I don't want to Say no, I just want to make the volunteers walk around a lot Hi, actually my question is some problem with it within so Everyone talk everyone likes to talk each other and you know the human Feels you know interacting with each other. So my question is a little different So we have a tool a survey tool. So how can we get more out of it? You know the questioning that you just because we have a millions of users. Yeah, and So how we miss a question, you know in a human way So that would be help us so if I can paraphrase it sounds like you you have a survey tool And you're wondering how could you maybe implement activities like this? Yeah in a tool like that. Yes So this is so critical incident question. Oh says reflect. Sorry the critical incident questions are actually Questions that I start almost every survey I do with The challenge there is data analysis, right? But it's extremely beneficial if you truly are trying to identify What kinds of problems or values your user population has it's beneficial to have that grounding question? You know before we start the survey Please in two sentences or less talk about the last time you had to block. Yeah, and Unfortunately, there's no good. I found no good text analysis software So you still have to eyeball it but the cool thing is that a it helps you identify. Is this a real person? That's part of my target and then be what kinds of things are they talking about you will learn so much there And it takes a lot less time to analyze even hundreds of responses than it does to do ten interviews interviews Yeah, so you can do a critical incident question really easily the laddering questions are much harder in a survey Yeah, they all are contextual, right? Yes, and so I don't I don't have a good solution for that all I Can say is that every survey that I give typically closes with a Why type of question and an opportunity for additional feedback and information, you know Any questions you wish the survey asked but didn't you know again? There's going to be text analysis associated with that But if you care about these things that are driving your user population, it's worth that amount of time I would not plan a survey with a thousand people with these open text open-ended types of questions But if you're doing a survey, let's say with 40 50 60 people totally doable using critical incident Maybe throwing in one or two wise. Yeah. Yeah. Thank you question. Thank you. Thank you Okay, I'd like to pause the questions here because on in terms of timing we have We technically have three more activities to do but I'm gonna merge one of them So we still have two and so the way I see it. We have an hour and 20 minutes left. Is that ish? Correct, correct ish Okay, so I leave it up to you. Do you want to continue on or do you want to do Q&A? I'm happy doing both. I learn more through the Q&A But I'm guessing some of you are gonna complain because you want to hear more stuff and do more stuff So let's take a show of hands continue the questions Okay, so yeah Please hold your questions. I'm happy to talk to you when we're done And like I said, I'll be outside in the hallways at the booth and stuff like that for the remainder of the conference So I'm here all day tomorrow and my the best of my knowledge. I'm not doing anything tomorrow So I'm totally fine hanging out in the halls and chatting. So hold your questions. Let's talk about them more I drew I drew the app as a monster because I put stuff into it and that stuff disappears. Oh Okay, say more about that. You might want to bust out the why questions there Or I drew your I drew your product. I drew this product as a superhero because it actually it literally saved my life last year Please say more. These are extremely emotional stories if you can create the right context that you can Elicit from folks because it's a time-consuming process. I have done this effectively online You know, you could do this using some kind of online meeting tool But it takes time because you want your participant to take the time to draw and you also want them to take the Time to tell their story and then you want the follow-up interview time Works really well if you're doing like a group activity a group workshop participatory design focus group But you want to make sure that the participants the users who participate in this are really representative of people that you're concerned about You wouldn't cast a broad net. You'd want a really tight target And so the advice is typically think about this in terms of extremes So if you had let's say you've got a user population using your product You can say if I had to break them into two different groups What characteristics would define one group versus another oftentimes you'll find like novice users versus expert users? For like this travel it might be people traveling on a budget versus people traveling for business or People traveling because they just got a ton of money and they want to you know to do a dream vacation and You find people based on those extremes and have them tell their stories Because it's really not a numbers game. We don't want to you know We don't want to collect data from you know 25 people it's fine to collect it from three and the other thing is because they are so time-intensive and intensive and emotional They don't play that well in like a report deck It's better to make sure that your key stakeholders are there witnessing and observing the experience You could either do it the creepy psychology way with the one-way mirror Or they could be literally sitting in the room as part of the group or you could ideally take a video and then be able to play that back so drawing and Drawing and telling stories can be can be very rich So something to think about really easy to I love the I love these prompts because you want the prompt to be disembodied enough That if you're in a situation where the participant knows you work for the company You're talking about someone else Right. Tell me about someone else having a problem not you of course, but someone else Tell me about what this product would be like if it came to life It gives a little bit of distance It makes a little bit more psychologically safe for the participant to speak their mind Another technique we talked about this earlier is listening and some of you did this during your you're laddering in your critical incident again indie young she wrote the book practical empathy and She out. She lays out the entire approach to doing this. It's excellent book. It's totally worth it however, if you aren't if you're on a budget just go to her website and She actually makes most of her stuff available for free online. She's really she's really great But the idea of listening is it's kind of like flipping the idea of an interview on its head What you're doing is you're trying to develop empathy and understanding the things that drive your users based on what they talk about and a listening session is all about understanding why what are people reasoning about what are their reactions what are values and their guiding principles and Basically, you follow some interview practices in terms of you know You greet the person you make them feel welcome and that type of thing But you don't necessarily have a prepared list of questions to answer specific things You have broad questions where you're just trying to get a lay of the land and so Indie talks about You know have your have a question. That's like a prompt and it's like let's start broad in many many cases It's probably analogous to what you all asked for your critical incident questions Don't take notes that one's hard Right because you don't want to have an interview type of idea going on in the mind of your participant Let the speaker control the topics the interviewee is the one who's driving here If the interviewee goes in a different path Let it go take it follow that path and see where it goes because the path you want them to follow about your pain points Kind of like to the point I was trying to make earlier That might be irrelevant compared to this other area that's so much more important to this user And I hear your your product team saying yeah, but we're not building features for that Well, maybe you should be thinking about it then Right, won't it help to the listener? I'm really serious uncomfortably close to your mouth. I can't hear you Okay What I'm saying is that if someone mentioned about their personal experience by telling I also did that Won't it help me the person connect with a person so that they can say more things? So if a person says something like oh, yeah, me too. I did that won't that help build a connection Sure, it'll help build a connection your goal isn't to become this person's friend Right your goal is to have a listening session where this person's driving as soon as you say me You make it not about them. Okay, so do not do that, but maybe some things they won't know my mind Yeah, thank you. It's a great question, and it's why a listening session is so hard. It's not about you It's not about you building a good rapport with your participant It's about the participant having a spotlight on them and them talking to you like they have a diary and they're sharing their secrets to their diary As soon as you remind them that you're a different person Well, they might close off and shut shut down. Sure. They'll be your buddy and they'll go grab a you know coffee later That's not your goal though. Your goal is to get to their most innermost thoughts on values needs and emotions So keep that in mind. I know it's a tough message, but it has to break us out of that interviewing mindset It's a great question So fewest words possible Right Again avoid I statements. We're trying to make this all about them. Ideally. We want our participant to just go on a monologue for one hour Just talking talking talking about their thoughts and their values and their emotions. We're not even there We'll do a tiny bit of steering But it's really tiny so if you can say it in in very few words, that's good again You have to practice that how would I ask a follow-up question to this and I like to say fewer than five words In a five-year words if you must but fewer is better You don't really want to remind them that they're there. They already know you're there They can see you when Indy Young talks about doing listening She talks about it actually being easier to do this remote because you don't have to worry about all the body language stuff You don't have to worry about you can kind of put your phone on mute while the other person's talking So they can't hear you cough or take a sigh because they might read it a certain way again It's all about getting to that inner monologue and then finally be respectful So just like in any interview don't judge, but if a person says something truly shocking you might not be judging me Well, you might right like you might have that surprise face or that that breath That just reminded that person that you're there. So I Bring listening up. I don't necessarily recommend trying it during our next activity, but you're welcome to if you wish but listening is Incredibly hard because research as researchers and also just as people who consume media, right? We see interviews There's a script for how an interview should work. You're the interviewer. I'm the interviewee. You're gonna do this I'm gonna do that Listening is totally different. It's kind of like being a friend, but not being a friend, which is why it's so hard, right? So listening is a really great technique to try out. Yeah, can we take some questions? Wait for the mic Maybe I'll merge the last two activities into one activity so reaction cards For those of you who consider yourselves interviewers and researchers, this will make you feel really happy We're outside of that uncomfortableness of listening reaction cards are on the face a really simple technique and They're they're Developed by a few researchers over at Microsoft and the idea is after an individual has an experience with a product or service You want to assess what values they they would associate that product or service with and so what they do is let's say It's like a usability study They could even do a usability study do a bunch of tasks with your product and then say I want you to go Through these cards and for each card I want you to put it aside if if it resonates with your experience and Discard it if it doesn't these are the cards. These are the words that are each card. You look at these one at a time It's a slow process But it's very valuable and this is available on Nielsen Norman group. They've got this list So for example, the person is just going through each card one at a time Like you know Confusing dated difficult disconnected and in general they're kind of discarding most of the cards But the ones they put aside are interesting and then once they have this pile of of cards You ask them pick your top three So maybe they'll have 15 or 20 cards Excuse me Then say pick your top three and Stack rank them so you have a number one and number two and a number three and then tell me your thoughts and your reasoning behind that It's a really great way not great way not only to get a good story But also understand if you have like values that you want your product team to follow You either a have a set of ideal values that are identified by your participant your participants Oh, if I was using this app, I would like it to work this way these three things. I want to be fun I want to be controllable and I want it to be Sophisticated let's say and to me these things mean the following you could adopt those as team values fun I forgot what I said before, you know, whatever those three things are those are our team values every design decision We make goes through the lens of those three things and then You say here's the actual product or here's a prototype or even here's a scenario that describes how the product works I want you to go through these cards and pick the ones aside that match The experience you just had and if that those cards don't match the values your ideal values are set to be You've got a gap you have a delta now you have some work to do So it's a great not only for assessing current values, but also ideal values So reaction cards and they're very easy again to to manage You present a scenario and present a task have the participant review the terms one at a time Select all that apply prioritize the top three stack rank and then describe their rationale It's not a card sort, right a card sort would be Create a bunch of categories. That's not what these are. We already created the categories positive and negative They just need to pull them out. So don't confuse this with a card sort However, if you have access to card sorting software, you can do this with that software You can have cards with all these terms and just say drag all the cards that match your experience To the match bin and drag all the others into the trash and then from the match bin now Drag cards to top one top two and top three So you can use a card sorting tool to do that and many of those tools are available online for free So card sorting is sorry reaction cards are really really robust approach to get at values What you're not getting is you're not getting that rich story. You're not getting an understanding of that whole participant It's not alert listening session. It's barely a critical incident session But it's very targeted, right? So if you're somewhere down the design path already or if you're just beginning and you want to understand What values should it be striving for? Reaction card methodology works really well So any questions about that so you would do this with one user The the user needs to go through all of the cards Yeah, please so then user will pick up any one of the card and then he or she will talk about the they go through all the cards and Each one that matches their experience goes into one group each one that doesn't gets discarded Then they take the top three and they tell you a story about why Okay, okay. Yeah, you don't want them to tell you why for each card they pick out that could take forever Yeah, we're going through 120 cards, right? And so if I have 30 cards that yeah, yeah, it was kind of fun. Yeah I really I liked I liked how this happened when I did a thing and yeah, I was it was kind of friendly It's okay. It's like I get it There's a story behind every card. You really want the story about those top three Okay, you just don't have time sure. Thank you Hi, so since we're kind of arriving back to top three emotions that they're feeling great How is it any different from going to them and asking? Hey, when you think about product Just named three things three emotions that come to your mind first. Yeah, so how is this exercise different from that? Yeah, how I mean, how is it different? I kind of turn that back like What what happens here? That's not happening when you ask somebody to you know, tell me the top three emotions that come to mind They're going through painstakingly every single emotion they could possibly go through and then the choosing versus yeah It's a recognition versus a recall activity, right? So when I say top of mind, what are the top three? Yeah, you have three things that are immediately available, but they might not be representative of how you feel overall It's just how I feel literally right now if I was Let's pretend I was 30 minutes late to this workshop, right? And I and I'm running here and I'm sweating and I'm reading and you say oh Steve What give me the top three emotions that come from the alarm clock app that clearly didn't work? I'll have three very strong emotions about that, but they might not be representative of my overall experience with that clock It's a deliberate process and instead of forcing me to recall top of mind I can now just recognize terms that resonate. So it's more deliberative Yep, that doesn't mean that there's anything wrong with simply asking a person that's great data to get to but that's highly contextual This is more generalized Yeah, it's great question So let me continue on All right, I need your vote. This is a choose your own adventure here So you have two more activities? We definitely don't have time for them And this is why I keep going back to UX India. I love the dialogue and the discourse that we have here But that means all of these great plans that I had just aren't gonna happen Same thing happens when I teach to So would you rather by show of hands go? merge the next two activities or Do you want to do this one and then probably forfeit the one after so the question is So merge to which means I'm gonna talk about slides a little bit more. So that's the vote right now So keep going on the slides and we'll do one big activity at the end or okay peep and Yeah, I know And you can but that's kind of voting for the former isn't it? Well, okay, like most hands just went up so I'll just show you the other stuff that we're gonna talk about and then you all can decide how we'll how we'll manage it The activity we're going to do is the travel app company wants you to identify the values users have about their app You would then have somebody use an app like literally use an app Ola calendar Flight search whatever and then they would either go through reaction cards Or you could do a listening session or you could have them draw something to talk about the values they associate with it Okay We can totally do that in the next activity to we can just include this as a set The only thing is I won't have my cheat words up here So I'll need to do a little or you could just you could download the slides from slide share So let me go through Won't reflect. Let's talk about survey because this is the most robust area in terms of stuff. That's out there It's also the easiest the cool thing about surveys is they're super easy to deploy So survey methods Help you understand emotions There's actually quite a bit of research that shows that survey data correlates well with emotion Not all the time not for all surveys, but in general when you ask people to indicate their feelings That seems to work better than a lot of other methods You have to consider the granularity of the experience. Are you on are you worrying about how people feel about an Interaction are you worried about how people feel about maybe your home page or how about how they feel about your entire app Or how about the entire space that your organization is in your company? You have to think about that granularity the cool thing about a survey is like during the presentation yesterday If you decide I want, you know a simple question to understand how emotions change over time You could do like a one-question survey over time Super easy, but if you want to know the complex constellation of emotions a person feels that's really tedious if you're going to do it Let's say every three minutes Right your user will never complete the task so the single item Self-report is I would argue probably the best bet for most of us in terms of capturing the emotions of our stakeholders I'm sorry of our users It helps you understand the valence So how do you feel positive and negative and many versions also help you understand the intensity? So how much emotion are you feeling? You can use it Repeatedly it's really simple and in general the simplest versions of these basically literally say how are you feeling and it's There's nine-point scale that's out. I'll talk about that in a moment, but really the scale doesn't matter too much It's just the valence. Do you feel negative? Do you feel positive or do you feel somewhere in the between and? And how intense do you feel a little bit a lot and there's a now an organization called UX emotions This was published in UX PA and I believe presented at Kai I might be wrong about that latter part But it looks kind of like this. I think they're in beta And so I think it's available for anybody who wants to use it right now for free. I think But the idea is that they have so this is a survey It is it's one question even though it looks like a lot of rows You just put a box in one of the a check in one of these boxes So you're basically asking do you feel super positive? Do you feel super negative? Do you feel somewhere in between and then how much of that feeling do you feel? And so you might put a box for example, you know, oh, yeah, I I like that product I'm satisfied and I'm really satisfied. I'll put it right here But you could also say I'm a little bit satisfied So maybe you'll put it way up at the top under the satisfied column, but it's just one check Okay, it makes sense So it's a single item Getting both valence as well as intensity Great approach you can administer it multiple times the authors of this have done a lot of research on it They found that it correlates really well With how people actually feel post-hoc. So you have people to do it a thing they Check the survey you bring them back like in a week show them a video and they're like, oh, yeah Here's how I was feeling they're really accurate about judging that seems to be doing it seems to do better than biometrics They submitted the images of participants to a company that read the face And they found that actually what the person marked was closer than what the machine guessed Maybe the person was lying, but I don't have any reason to believe to believe that There's another approach sorry the formatting is a mess I had to convert the aspect ratio here But it's basically you're just asking people opposites. So this is called the semantic differential This specific approach is called the bipolar emotional response technique or Burt Which is cute There's not an established set of antonyms you pick what antonyms you want the recommended guidance is up to seven So you decide with your team what emotions are we most concerned about and then what antonyms pairs would assess those emotions So friendly hostile clear confusing surprising predictable and then the participant just puts a check To where they feel and you get a more a multivariate understanding of where they feel You could also just literally put you know the set of emotions that are there But I think this is a better approach because it's a little bit more well-rounded The Geneva emotion wheel this is this is wild right look at this thing. I believe this is available for everyone to use under creative commons The GM Geneva emotion wheel has had quite a bit of research around it and Is great if you are looking at getting that multivariate picture of how people are feeling It's probably not great for repeated administration like at the beginning the middle and the end of a task in a study It's just kind of too much to process But if you feel that your participants your users are going to have those compound emotions You can get those components So you don't have to put a box on all of these things You just put a box that corresponds to the emotion you're feeling and the intensity and so if I feel let's say Half pleasure I could put a check in that middle circle on the pleasure radial But also I feel some Fear because I'm not sure what's gonna happen. So I'll put a little check in that middle fear and For this study I use the product. So I'm super interested. So I'm gonna put a check in the interested and Also, I am gonna get paid a thousand dollars and I don't know what that converts to multiply that by like 70 Right, and that's a rupees So I feel a ton of joy because I am gonna go buy something amazing after this study So that's how you use the Geneva motion wheel and all the others that don't apply just leave them blank And then if none of them apply I've got a non-option and if something's not there I can just write it in so I can get a sense of how my my participants are feeling Okay, and then what if you're worried about bias? What if you're worried about people not understanding words or interpreting the the term you use wrong? You can use something like pick a mood pick a mood uses only icons and the icons show you in fact If you're worried about cultural bias, it can be like robot icons So unless they're biased against robots and then then you've got a problem and so and Things like pick a mood have been shown to to work really well across cultural so you don't have to worry about translation and They've been embedded in various products. I think it was originally developed for education purposes So like getting assessment of children and how they feel but I really love this approach because it doesn't tie you to a specific language You don't have to worry about localization So so those are the service any questions about the surveys So in the activity, I'm gonna encourage you if you want to practice survey practice the single item self-report survey How are you feeling right now? Bad good. I think that's that's a great one to practice There are no questions. Oh y'all are great. Okay, cool Facial expression observation. So all the caveats about biometrics still apply here But if you're with a participant like doing a usability study, you can see them on a web camera You yourself can assess what emotion you think you're reading It's not necessarily going to be as accurate in terms of what the algorithms are picking up because the algorithms are Tracking all those face points and we as humans don't have the ability necessarily to capture that plus you all Talked about kind of that gut reaction you had when you saw something and then kind of like the the follow-up feelings So it's a little less Valid but could still be enough in terms of getting a thermometer of how your participants feel and so There's a site called science of people calm Where actually the author she trains you online like not hardcore training Which just gives you some some some landmarks to look at and they're indicated by the different facial areas here Totally check it out. Obviously don't have time in this workshop to do it But if you're going to have face-to-face contact with your users, it's probably good to start reading understand at least gross measures of emotion from a face and Then also there's body language Body language I think can be even more valuable because a lot of times in a study people have that neutral expression That was mentioned. I think that's what I got you mentioned it Body language might be a little bit more revealing and it's a great indicator of arousal So there's this idea of status And so if a person has low status you can often detect it because they have a small posture So if they're sitting at a desk, they might be hunched over They might be fidgeting they might be obviously distracted They're they're basically checked out and If you it's interesting I have my students do observations at a local Subway stop and they watch people like tourists coming in trying to figure out how to use the ticketing system Where you put in money or a credit card and it kind of spits out a ticket that you can use on the on the train And it's totally you can totally see when people are completely like dejected You know they come up to the system. They're like and like you can just kind of tell there's like an emotional conversation happening with the ticketing system And and you don't need to understand how to read the language. It's just so robust So as a gross measure, it's really helpful on the on the flip side if a person is really big and You know stable and powerful. Maybe they're staring. They're looking right at you They have a gaze as opposed to fleeting around. That's an indicator of higher status So they're they're feeling empowered here. So it's an arousal indicator. It's not necessarily an emotion in the game Maybe they feel empowered because they're angry Right, but it is something to take note of it's kind of a free thing that you can look for when you're doing your studies if you have access to a person or a video so Hmm All right again, another boat. So those are those techniques So the travel app company wants to understand how users of their app feel as they use the app Okay, so remember you still have I apologize. I don't have this on the slide, but you still have listening You still have drawing You still have reaction cards. I have a few set most of my reaction cards Were taken yesterday to be fair I told people they could take them and they took me on up on it So I think I only have two full sets of reaction cards and they're actually it's a subset It's a smaller group because there's just way too many cards to do But if you're interested in a team doing react reaction cards, we've got a few Otherwise I want you to think about an App think about a task have a person do a task on an app and then figure out a way to assess their emotions Ideally before during and after they do the task. So you get kind of like three points So unfortunately listening won't works too well During but you could still have a listening session after they experience it, but you're gonna have to figure out How do I craft my script to ask questions about the beginning of the task the middle in the end? And how do I do that without being an interviewer? That's really tough Okay, so I would kind of recommend not doing a listening session for this So does that does it make sense? Does your charge make sense? And there's enough time here to do more than one technique I'd recommend everybody trying a survey and just see how a person feels at the beginning the middle in the end using a survey and then think of one other technique and Drawing could be good Observation sure that comes for free right because you all can see the person there same rules as before Pick a participant from your table. That's the interviewee from the last exercise I mean doesn't have to be the same person, but that's what we call the interviewee Have them perform the task and gather emotional data at the start the middle in the end Again, there's the the some hints about survey and observation any questions. Oh Yeah Thank you Hi, I would say he was really good. He was really grumpy So when you don't expect your user to be the way, you know You would normally say that think that they would give you the feedback and he was like, yeah, I don't use apps I don't like net. I you know, I make other people do my work. I just don't want to do it So okay So when that happens and you know, it puts you in a spot where how Would you make this user actually even try using the app because they themselves are so Putting that you know a screen in front of them saying I don't want to do this. Yeah So, you know Yeah, it was just not in the mood or you know, maybe they're just only for the money right the compensation that they're gonna get Short answer is I don't know Longer answer is it depends on context and I mean they showed up so you have to kind of meet them part way Were you able to do the task? Oh, we were but the whole point Say, you know what just just call your agent and do it because it just felt the agent was so much better So you had such a good persona that just it became really really hard. Absolutely. All right. Very cool. Thank you Anybody else? observations experiences Hi so We kind of took like a different approach with the process So we asked our interview to think out aloud while using the app. Okay so that gave us more insight about how we felt rather than just After completing the task we asked him how he felt before while and after but because of the think-out allowed We could have like a lot more insight about how he was feeling. Yeah, where his emotions changed so it kind of took like a path from Feeling that he's in a hurry to complete a task which is going to be really easy Yeah but then when he starts doing it and there is like a lot of information to process and choices to make it becomes a little problematic Then he Makes he has to make the choices so he gives himself the Validation by reviews and photos then eventually he's Satisfied with his choice confirms the booking and feels the satisfaction and relief eventually Okay, do you for the emotions that you got were they based on the verbal protocol the thinking out loud? Or was it also based on survey questions looking at the face looking at the bilingual? How did you assess that the emotions were the right ones so thinking out loud goes along with reading the Body language. Mm-hmm. So it took advantage of some of the visual. Yeah cues that were being given Yeah, so even if while he's fidgeting with the up and down so you know, it's really hard to make the choice So fidgeting is a good example of low status too. So it kind of shows an arousal problem, too. Yeah that but it would kind of sometimes do like a Complementary scenario where the user says that he does not mind making any choice But he's still fidgeting while taking the choices. Yeah, he does not have right. Yeah Sometimes the body says something different than what the words are. Yeah, and so yeah I love that example if you're you know, if you're just doing simple verbal protocols, right think out loud that can be extremely beneficial It's just that the challenge is Sometimes we don't say what we actually feel so you have to make sure that there's ideally Another stream of data so because we can see people and kind of see their body language see their faces We can say okay. He's saying that it's okay, or it's a great usable usability experience But what I'm seeing is completely different than that and so that might be an opportunity to kind of just make a note That's a that basically says user said things are okay. I did not observe that You know and and that's a great conversation later for the team. So great opera. Yeah, that's a great opportunity to capture that data Thank you. Yeah, so I was into interview. Yeah, you were a grumpy person, right? Yeah Yeah, I was the grumpy person and they were trying to interview me and I Deliberately became more grumpy But the thing the good thing for me was as an interview. I learned a lot about When the things went wrong, I refused completely that I'm not going to listen to you guys then I Saw them changing their tracks there initially They were just saying why don't you do this and do that then they changed their tracks and they offered me some You know midway point where I could meet them So these were the things that I as a designer learned that you know, you can't just Bombard people you have to offer something. Yeah, it's kind of a bargain and to and fro and give and take that was for me That was awesome. That's a great observation. Yeah, unfortunately as researchers when you're running a session You're you don't have the power position that I have here. You can't just barrel in and say, okay, you're done Okay, let's talk okay move you have to go with the participant because you have to honor their time You have to honor their commitment and you have to honor the fact that they actually have valuable data for you to get if only you Can figure out how to get it. Yeah, really great Let's do let's do one more question because I just want to show you the remaining slides and then make sure y'all get lunch So microphone Now the back is shy Thanks, so I figured out that the persona of the person was very important You asked him to be deliberately grumpy. Yeah, but I found out that if the person is like is natural he can be like Act differently like if I'm asking a person who hates my app to give me some some kind of feedback He'll give it differently But if I'm asking someone like who just hates using other apps, he will respond differently Yeah, so just want to make a point context of use is really important and context that the user is coming from is really important I think it's great. It's a great distinction if they hate your app or your company That's very different than hate using apps to do this task and the level of data that you can get is very different Which kind of shows that like the critical incident and the laddering questions We had at the beginning can also help you kind of set the stage for what kind of user this is So I think that's really valuable. So in the interest of time, let me just show you the other slides So, you know what's here. I'm more than happy to you know Continue chatting with you all here at the conference and on you know online socially etc So the one piece that we're not you know going to have time to do today is mapping and that's that's totally okay But mapping is is this whole concept? I'm sure most of you have tried or dabbled in journey mapping Raise your hand if you've done journey mapping Yeah, so you're all pretty much at least familiar with journey mapping if you haven't done journey mapping Or if you haven't done it and it worked well Think about ways to make it less complicated When you look at this list of possible things that can go into a journey map It's paralyzing and when you think about it as a stakeholder Like if I'm a director of engineering and I look at a journey map that has all that I might say I don't even know where to look So think about doing something really simple Think about narrowing your focus just say this is a journey map for this type of user in this very specific kind of context We're only focusing on You know the main phases of using our product so like we did today before during and after That's a beginning journey map and then over time as you flesh the journey map out It can turn into well before actually has like pre-awareness and then awareness And then during is really like onboarding configuration production etc And so you can break it down over time But you need to do it as part of a social process that involves all your stakeholders And a way that makes sure that the data you're capturing is actually meaningful and for us today that includes emotions So when you're building journey maps and this comes from a um a blog on medium posted by one of my former co-workers at Salesforce Her name is Richard Prajapati. She did a great project with um with some teammates to identify How do we tie emotional information to the journey and she did some exploration of The different kinds of data you you could collect and she noticed that stakeholders reacted in different ways based on how the data were depicted So if you have only qualitative data You might want to only use qualitative indicators such as like smiley or sad faces Because if you use something that looks like a graph people are going to look at it and say oh Well, show me the data. This is based on it's like. Oh, no, I only did 10 interviews But that's not the point the point is people in general when they experience this part of the product are super happy And here they're super grumpy If you're doing a mix you can usually have some kind of graph with quotes, right that are embedded And if you're doing something purely based on quant like maybe usage data You could just show those indicators or survey if you have a survey indicator of 1 to 10 You can just show where that survey number is going So think about how you could tie this to your journey map And we're not going to practice mapping. Sorry um We're not going to reflect So my last my last um second last slide is these are the resources I talked about yesterday I strongly encourage you guys to check them out Seductive interaction design is great for designers looking to create emotional experiences It's it's one of the hallmark Texts of its kind. It's it's it's easy to consume book as well Patrick Jordan wrote what I consider to be the bible of research for emotions. It calls it designing pleasurable products Really excellent book that reviews all the different types of techniques. There are Easily a hundred techniques So many of them are variations on the themes that we talked about today But if you're really wanting to geek out about this check that book out Don Norman, of course emotional design Why we love or hate everyday things. It's kind of the the the seminal work on Why designing for emotion matters? Why emotion will often carry you your users through the bad times, right? So check that out and then finally, um, erin walters book designing for emotion Is a really nice design guide in terms of principles and practices and techniques that you can consider when designing for emotions That's a little bit more Practical I would argue than anderson seductive interaction design the seductive interaction design is more about global Strategic thoughts and and concepts. All four of these books are awesome. I would strongly encourage you to check them out Um, I leave you with this quote from turkish designer. Um om. Kobanly Kobanly Good design touches you great design touches your soul. That's sweet So as you think about the next design project that you're working on ask yourself How can we make sure that this is touching our user's souls and how can I do research to prove that? Okay, so I know people are taking pictures. I'm gonna leave this up here for a sec All right. Thank you very much for your time. Please reach out to me You can if you like watching what I eat and pictures of what I drink go follow me on twitter All the slides are on slideshare and you can reach me at linkedin Just send me a message if you use linkedin mobile I've learned that it won't let you send a message. So then send me a direct message on twitter saying Hey, I'm not a bot. Um, you know, that's who I am. Thank you so much for your time. I really appreciate it It's awesome