 usability testing is all about like for me exposing the problems and exposing what works really well for users and understanding why. Research is not really like that so they're just different tools it's like it's like saying is a hammer better than a screwdriver depends on the thing you're trying to find out. So one thing I get asked a lot about is people like the idea of research they know it's important it's like flossing everyone knows you should do it but they know don't know how how does someone start getting into doing research like if they know next to nothing about it. If you're thinking about your product as you're sort of developing it at that stage of research then it's just great to get people using it for the tasks that you're planning it for so even if that's friends family even if that's people in the office it's just great to get your product exposed to people like that so that you can watch what happens and in usability testing we're really looking for the problems for people fall into and what works well for them and why it works well for them and so having anybody go through your product you will see those things as they as they work through over time you'd want to you know start testing with a broader range of users and get out of those kind of biases that come in with friends and family but to start off and gain confidence and start small it's a great place to start so when you talk about usability testing is there anything and the biases is there anything specific that you're supposed to look for so you've designed your app you've given it to your friends or family is there any like I suppose like golden rules of things to look out for or is it not really work like that yeah well what you make sure you do is kind of set up the tasks that you want to be watching for so basically you have people go through the tasks and they speak out loud as they do it often depending on what you're testing for and so you'll be identifying your sort of most critical user journeys usually and have them walk through that and often it's something like some language you use we call that content you know the words you've got on the button or where you've positioned something that just don't make sense for most other people it totally made sense for you in your design work or your or development work but it doesn't work for other people and so you're looking for that and you've got them speaking out loud so you can really understand kind of why they're coming at things in certain ways and that's the value of usability testing you know as opposed to looking at data from logs or something you really understanding the why of why they they get into certain situations where they can't move forward or certain flows work really really well I mean is there like a scale of research so like usability testing feels like it's like the midpoint of a product like that but is there like a before and after because I know there's also the debates of user groups that you have where you're asking questions or surveys I mean is how would you say is like the cycle of doing research what was the first thing you would you recommend before even building the thing or maybe when you've got early stages yeah so right at the beginning we tend to do what we call foundational research that's when we're really understanding the domain we might use also you know secondary research and see what else other people have done but we might also do foundational research often it involves more sort of field research techniques going out to where people are doing that activity that you're interested in supporting in your product so field visits doing observational work contextual inquiry is a particular technique where you're with the person throughout the day and asking kind of questions as you go along to really understand why they do certain things in their environment so that's the whole like foundational research stage at some point you might want to understand sort of broad representations of data you might do survey work I'd always recommend doing some sort of qualitative work to begin with whether that's you know your contextual inquiry your interviews so especially for surveys so that you've got the range of results that you would want to ask for a particular survey question if you come back and you've asked this kind of closed-ended question with different question answers next to it but you've missed a couple of really significant things out then you're you're there's real problems with your survey so doing that qualitative work to begin with is really really helpful and then you referred to I think you were meaning probably focus groups yeah sometimes get a bad rap yeah so focus groups aren't used that much in user research occasionally I've used them and it's really been more for my benefit of finding out a domain I kind of haven't used them for rigorous you know representative data but conducting a focus group helps me understand like these are all the important topics that matter to people within this domain and then I can go forward and do my other techniques so it's more about boosting your own knowledge rather than finding information out about that's how I've used them so you have like this kind of always versus mentality in tech but I mean do you think there is a better way of research whether qualitative or quantitative or are these is that like a pointless thing to be asking that it's really about using those to just get a better understanding of the problem that you're trying to solve so is one better than the other in your opinion or is that it doesn't really work like that they're actually very complimentary and you need both so when it comes to the stage of doing usability testing oftentimes we might when we're really into sort of initial use of the product or we've got it out there to to beta test us or we've already launched and we're doing studies then then you can collect logs data logs data is like really very very useful you can see where people are dropping off on a page for example and why they're not converting for example but you can't necessarily always understand why and so partnering that with usability testing is really valuable so that we can really understand the why and that's you know our quantitative data has you know huge numbers in it or large numbers as large as we can get and the data is as representative as we can get we really are very concerned about that in usability testing it's not so much of a requirement to make sure that your participants are truly representative of the population because you're looking for pain points and you're trying to understand the why it is that people have these problems so that you understand their mental model and you avoid designing in that way again yeah i know one criticism of usability testing especially like we say sometimes a sweet spot is between five and eight people right is that enough like to really solve the problems you're trying to solve or is it not usability testing is not really like that it's like it's like you're not comparing it to say like surveys where you're doing a hundred people that use a bit of specific pain points that's exactly right so in surveys you have to be really sure that you're getting as good as a sample as you can as as representative of you can of the population that's of interest to you and there's various sampling techniques to do that correctly otherwise there's going to be problems with your survey as we've often seen in voting and polling and that kind of thing with usability testing we're doing something else quite quite different really we're using usability testing to understand what problems people have with the design as they go through as they try and complete their task and so what happens is often you see you start seeing the same problems again and again and you really learn that you know oh this is kind of a problem from everyone in fact there can be problems that you see right at the beginning of running your studies say after even one or two people and you're like yes of course you know i should have seen that of course people think of it in that way and of course they're dropping off at this point and you don't need many many people to prove that and so you tend to find after five or eight then you've seen the majority of the problems if you you've probably seen 80 percent of the problems if you carry on testing you will see more problems over time but you'll have caught all the main ones in the first five to eight and it will be definitely diminishing returns on that so say i'm designing a product is there a point where i'll pivot in the research so like say we've done two tests on an app or website these really obvious problems the UI is not obvious or whatever comes up do you stop and say right we're going to adjust the UI and the design thing and carry on testing again or does that muddy the waters i mean is it okay to do that yeah no it's totally okay to do that and that's actually a form of testing called write usability testing and there we schedule two or three people to come in and then we have a break like half a day or a day where the team get together and talk about what they've seen and how they can adjust it and then you'll get two to three more people come in and do the same again and iterate on it and it's it's a great idea really because you don't really want to or need to see kind of five six users all you know fall into the same problem if you all agree as a team that that problem exists and needs fixing there are other problems where you know you're not sure somebody somebody you know goes one way and somebody else goes another and it's really there might be a few different mental models out there and you kind of have to see how it goes over time to see what what the main issues are and what your team feels need fixing but certainly um there's another good reason to do it as well that those obvious ones that you really want to fix might be sort of hiding other problems that wouldn't be exposed if you didn't fix those ones during the course of the flow okay the other thing um I remember you you did the research project called the 25 principles I have to say somebody's known as the gov principle web web first we did web and then we did app and we did a whole set on retail and I remember speaking with you it's like uh it's almost like my mind just was blown when you talked about you do a pre-study before the study to test to see whether the study is actually good I mean that kind of like of course you do um could you explain a bit about like what um is it called a pre-study or pilot study right it's called pilot study and so for usability testing we um yeah exactly like you say we're testing the test we want to make sure that um the different tasks that we ask people sort of lining up properly and make sense for them we want to make sure that those flows work as we think we should at the time in in the product um it's absolutely crucial to do pilot testing beforehand yeah and you can do that with all techniques of user research so there are many techniques that we haven't also talked about uh diary studies as something that's sometimes done we can test the questions that we're asking people in the diary study even surveys it's important to test your survey instrument out first and we often do what we call cognitive pre-testing which will sit down with someone and go through the questions and make sure the questions mean to them what we think they mean yeah um so just even going back to the beginning so you want to test you've done some test with your family what's the next thing I mean is there do you really at that point need to get a good researcher to help you plan these things or is there something that small teams and startups can do now like professionally to test their yeah um well there's absolutely you can do it google ventures puts out some good um guides for doing usability testing and different sorts of user research testing so that's a good place to look for how to do this from a startup perspective uh you know as a user researcher I personally don't think companies uh hire user researchers early enough in order to get that kind of foundational research done and really understand the space really need to be thinking about doing that hire earlier than we're doing it now what tends to happen in companies is that people hire eventually they realize that they can't do design without a designer so they hire a designer and then the designer says all right it's hard for me to do design without the research findings and to really understand the context I'm working in so that's the kind of like pattern it tends to happen in it would be nice from my perspective if it worked the other way around but that's not to say that you know people shouldn't be doing user testing themselves obviously you can get better it's not particularly advocated that you should work with family and friends it's just a good way to get started and you will find issues with your product but um it's better to uh try although you're not aiming for in five to eight people a representative sample of the population you do want to try and get the same sort of people that you that are you you're aiming your product for so if you're creating a a music app for teenagers then it'd be great to test on teenagers that are the sort of target users for your product it's also important to test for demographic as well like in terms of the way users will behave in india will be very different to where they behave in america right right and they're under different sort of constraints and different different contexts and different conditions with you know the kinds of phones they use um the different um connectivity they have all those kind of cultural things which may be like how they response to ui and whatever yeah and how is there is it just basically you need to get out in the field i mean i think one thing people do is they test too much in front of their computer when they're they're you know amazing mac book or amazing wi-fi connection but like how do you really empathize because i think research is more about empathizing with the person you're trying to design for right right absolutely and so i think you know we need to take account of the technological constraints that we just talked about you know making sure that we're if we're aiming for those kind of markets that we're potentially i know in india we have a lot better connectivity nowadays but in certain parts of the world there's a lot of the world that's still on 2g connections so taking those kind of considerations into account is important but also you're right that the the cultural and different contextual factors can be really important so what if you're testing for um users preferences over to a design so if you've got two separate things how do you know you're testing for the right things or the person's opinion and what are they or preferences they have yeah that's a really tricky one i think that you know i'm not greatly in favor of testing uh using usability testing for preferences because it's such a small sample potentially the sample might be biased if you asked with eight different people you might come up with different preferences especially if you were just adding up the numbers i'm really not in favor of that because it has those problems you know usability testing isn't really for that purpose so i'm not uh against asking about preferences in order to understand why people have those preferences i think that can be really useful in usability studies so understanding you know that they need particular things say in maps when they're navigating they need particular landmarks yeah that's really really interesting but you know just like trying to collect raw numbers about they prefer design a rather than design b they prefer blue to red i mean is that and people lie right when you ask them questions they might tell you what you they think you want to know absolutely so this is why i'm much more into like understanding them completing a task and understanding like what works for them and what doesn't work for them about completing that tasks it's really all about getting that deeper understanding in usability studies so yeah don't add up the numbers for preferences like but use that um to understand why the particular design is preferred over another one and what we can do for future designs to make it more useful for people and it's really understand the context rather than the personal opinion of yes well that's right yeah that's right and understanding why their context leads to that view i try as much as possible to avoid black magic and like whenever i'm reviewing any code that any of the designers on my team are writing we like try and avoid anything that's maybe it's a little hack and it makes it slightly more performant but the truth is like if we want to evolve the material design system we need to be able to build on top of the code and each layer of that code matters