 I too appreciate that you came in, although if you had not I wouldn't have felt bad because it is lovely and sunny out and I almost want to do this talk outdoors. How's the feedback? Is that me? Do I need to run? I'm good? Okay. Hey. So I have slides. I think I have no control over making them appear at this point. It's magic. Thank you. So thank you. Look, I'm giving this talk. There is a URL. These slides are not available yet, but they will be available after my talk at r-o-s-e-l dot l-i slash w-c-l-d-n. And I will show that URL again later on. But you're in the talk, Inclusive Usability Testing. If you think you're at lunch, you missed it. So there. I have a slide dedicated to my ego. This is just so you can say, this is a real person. You should probably block me now on Twitter. I have a website where I write a lot of stuff about accessibility and usability and general code ranting. Look, there are some things I've done, whatever. Level setting done, I think. Okay. So this whole concept of inclusive usability testing is about integrating people with disabilities into your testing process. So if you're already bringing people in, if you're already sitting them down with software and trying to get a sense of how usable something is, folding people in with disabilities is a great way to figure out what other usability hiccups you might have. For context, I've listed different types of disabilities. Hearing impaired, blind, low vision, physical impairment, and cognitive. I probably should have added a bullet for speech because speech is a more and more important interface, one which we're coming to rely on pretty regularly. This is the part where I now show you the entire talk I did last year, selfish accessibility. I just have to hit the play button and it's only 40 minutes. So this will be good. That's actually available on WordPress.tv so you can do that at your leisure, sort of like a homework assignment. One thing I want to do, I want to start with this quote from Wendy Chisholm from Microsoft because I think it's a really important quote and I think it speaks to where people might struggle with making that first initial step to including people with disabilities. I encourage you to fail in interacting with people with disabilities because you will learn a bunch. You will learn what not to say and what people care about. You'll learn about where the obstacles are, both the designed physical barriers and the constructed emotional ones that exist within yourself. Very much she's saying, don't panic. Just start interacting with people. People with disabilities will correct you if you say something or do something wrong. You'll learn about, hey, you know, maybe you'll offer your elbow to somebody who's blind to guide them. Maybe somebody will correct you not to pet the service animal because it's working and you don't like it when you get pet at work. I'm just saying. I mean, I do. Anyway, so I'm going to run through these basic nine categories. And there's overlap in these categories. Not everything is cut and dry. But just general, we'll talk about some of the concerns, planning, payment, which I meant to rename as compensation. Sorry, I said I did it and then I lied to you apparently. Venue, recruitment, accommodation, technical stuff, process, and privacy. Or as I understand, many of you say, privacy. It's a much better way to say it, by the way. This photo comes to us from a program called Save the Dream. This was a national sport day in 2017 in Qatar. And the idea was using high profile athletes to promote disability awareness. And at some level, this is a little bit like disability tourism, like the dining in the dark and putting blindfolds on and, you know, trying to enforce empathy exercises is a little bit tricky. But I think this was still a really important exercise, particularly in Qatar, because it sort of reframed this notion of accessibility and people with disabilities. Anyway, I have a bunch of these little images that I sprinkle throughout that don't directly tie to anything. So you don't need to make notes on those, I suppose. So big concern. Before you start doing any usability testing, make sure your accessibility remediation is complete. You are not bringing people in to do accessibility testing. You should have already done that. Your audits should have happened. Your bugs should be fixed. Everything should be ready to go. This should be your accessible version that you're putting sitting people down in front of. To reiterate, it's not accessibility testing. If you do bring in people to do accessibility testing, you're wasting their time and you're wasting your budget. So please don't do that. And be clear on that with all of the stakeholders. Make sure that they understand. You're not trying to uncover accessibility issues. You're just here to uncover usability issues. I'd mentioned disability tourism. You're not here just to see how somebody with a disability uses your software and be like, wow, that's fun. That's interesting. You're doing this because you're trying to gather good and valuable information. You're bringing in people for their knowledge and their expertise. That's the whole reason you're doing it. So limit the scope of what you're doing to just what's needed for testing. Don't start to fold in other things. You won't have time anyway, but don't fold in other things. As I said, be clear with all the stakeholders. You don't want to have to have a fight with them after the fact. As you're trying to fold in the results of your study, that can be pretty painful. And then be wary of anybody in the organization who is using this strictly as a PR thing. Like, hey, we brought in people with impairments because we love the community and we want to show you how much we care. You have an outcome. You have an output objective. Make sure that you honor that and you're not just bringing people in for a PR stunt. Planning. I like this image a lot. This is somebody who is taking a sign language class who kept drawing a floor plan to a house over and over and over because the person who was teaching it never set the reference point from one room to the next. And the person couldn't quite figure out where the kitchen is in relation to the living room because that's pretty important when you can't see. As the person said, I never know whether the kitchen is down the hall from the dining room on the other side of the dining room. It's rather confusing. This whole concept of making sure that everybody understands that reference point. And I think American sign language education is a nifty way to show that. Plus it's just a hilarious sketch. And I think I would live in this really weirdly stretched out house where the foyer is twice the length of my house. I'm just saying. Make sure in advance that you review your tests. You review the format of those tests. That will help you prioritize the questions and the steps that you're going to do. Especially since you might not get through everything. So you wanna make sure that you have it all broken down. Are you doing a prototype? Is this the live platform? Are you testing a specific feature that you've launched? Are you just gauging responses? Make sure that you're not preemptively imparting some bias in your study. Are these tests structured? Are they informal? Are they remote? For the scope of what I'm talking about, I'm not talking about remote tests. I'm talking about in-person onsite tests. But you have to have this stuff planned in advance. You have to know if you're just doing questionnaires or full deep dives into software. Also, make sure you know how many people are participating. Are you gonna have three people? Are you gonna have a dozen people? Have you budgeted for that? Both for money and for time? And then all the recruiting efforts that have to go into that, especially if you've never done any recruiting like this. Please make sure, though, that you budget for compensating participants. And I go into a little bit more detail in the compensation section, which I previously had listed as payment. I'd like to point out gift cards are awesome because if you give them gift cards with cash value, not to specific stores, participants can use them wherever and whenever they want. They can use only a little bit of it. They can use it to pay for a service they need. They can basically treat them as cash. Be careful to basically avoid cards that are required that have a venue requirement, like you have to use it at a terrible restaurant or you have to travel across town in order to put it to use. That's a real barrier already, so don't do that. Make sure it's more than a free coffee that you give them. And if it costs them $20 just to get to the venue to put it to use, you've already kind of messed up there. Fun fact, you can also use these as signature guides for your blind participants. You just put the card underneath where they sign and they sign above it and then they get to take the card home and start spending. It's important, it's critically important that you plan on paying participants more than you usually do. And I use the word pay here and it's the wrong word. In the US at least, once you start paying participants, it can be considered income and it can impact their ability to get any benefits or any aid packages that they rely on, which is why this whole section is called compensation. But make sure you set aside more money for them. There's an outsized cost for them to participate. There are additional burdens that a typical participant might not have to endure. One of those is transportation cost. In the States, it's no big deal for somebody to hop in a car and just go. We're very used to it in many American cities. But if I'm going to bring in somebody who's blind to do a study, that person's not hopping in a car. That person has to hire a service or has to wait for one of the paratransit lines or has to go through some other accommodation. And what might take me 15 minutes to drive could take somebody else two hours of door to door transport services between hopping services, scheduling, queuing everything up. Even bus routes are going to require longer wait, sometimes just to get the accessible bus or the accessible train to come through. And they might be borrowing or using their own or somebody else's mobility aid. It's also worth knowing that a lot of people with disabilities are generally underemployed. And I'll actually even jump ahead to that one. That doesn't mean they're unemployed. It just means that they are not making the same wage for their skills that many other people might be making for a variety of reasons. And they're often not in salaried positions. They might be in hourly positions. And you have to account for that time off of work because they might lose an entire day's worth of wages. Just to participate in your study, that could be the cost of one day's worth of wages. And you should be comfortable compensating them for that time, maybe not their entire day's worth of wages, but enough to make it worthwhile, enough to make it worth the effort and consider that transportation cost as well. Some participants might have aids that they need to bring along with them. It could be anything from sign language interpreters to people who just carry their stuff, whatever it is. And they might need that help just to go out in public, just to maintain a schedule, just to arrange transportation and there's a cost associated with that. It doesn't mean you don't want to include these people. It just means again, consider that cost. So think about gift cards that are accepted where they shop. Maybe you don't even get a generic gift card off the rack. Maybe you identify a regular service that they use and you just set them up on that service for a while. Maybe you know that there's a mobility aid company in Boston and one thing we did once was we just gave them some money here. Here's a pile of cash. Here's credit at the facility rather. Go and pick up what you need. Upgrade your cane, do whatever. And if not, that's okay too. But make sure that you have that conversation in advance. You want to compensate people fairly. You might also be overwhelming them with all the information you provide and a gift card is a pretty easy way for them to not have to panic too much. Because it's just there. It's just in their pocket whenever they want it. Then you. I hear some chuckles. I need to qualify this image. I love to use this image, but this image is a lie. These ramps that you see on the stairs are not for wheelchairs. They are not for wheelchairs. That's actually for a cart, a dolly, to wheel supplies up and down the stairs. However, I have been to venues where their approach to making it wheelchair accessible is to just drop ramps on the stairs. And what you end up with is murder stairs because nobody can wheel up a ramp like that without falling backward. And if you try to go down that ramp, so make sure your insurance is up to date, I guess, at your venue. This photo is actually from the Czech Republic and it's a train station. And what's important to know is some of you snickered at this. You know at a glance that this isn't right. Now people who rely on these kinds of aids and services and features, they're gonna know before you when anything else isn't right. You might see a ramp that looks perfectly lovely. It might, you know, it's integrated into stairs and it's pretty and it's got handrails. And that might be a death trap and you just don't know it. You might look at a facility that looks great. It's big and open. And so a blind person is not gonna struggle by running into plants, but an open space is terrifying to a blind person because there's no way to orient. Wayfinding is all removed. So you have to consider these kinds of things and remember you might not be expert at identifying if your venue is truly accessible. They will know it's a token effort anyway. It must be accessible. I was gonna have that slide repeated over and over, but yeah. And it's not just the venue, it's not just the building, but the entire route. If you rent a facility where you do the studies, you know, on those fancy ones with the two-way mirror and all the cameras pointing down at people and the microphones and everything, great. Make sure they can actually get to it somehow. Make sure that there's a bus that runs there and it's a bus that supports wheelchairs. Make sure that the paratransit line goes there. Make sure it doesn't drop them off two blocks away and then leaves them stranded. You have to be able to not just get them in the venue, but you have to get them to the venue and they have to not get killed while they're doing it. Think about sidewalks, parking lots, parking situations, things like that. Bus lines and transport services, these can be really expensive if they need to hire out a transport service. So bear in mind the cost, and bear in mind just because you found an accessible bus route doesn't mean that they're comfortable using it. They may have had a bad experience. I have a friend in Canada who will wait for the accessible bus, the bus with the wheelchair thing and he'll be there with his dog and his saddlebags on his wheelchair and the bus will say, we don't have any room for you and just keep going. And it's a lie and he's proven it, but this is why some people aren't even gonna rely on some of the services that are there and they'll have to figure something else out. Meet them at the door. Unless you're transporting them yourself, meet them there. Meet them at the bus stop. Meet them wherever the transit line drops them off. Don't leave them stranded. Walk them in, take them to the front desk, have somebody at the front desk know what to expect. If it's a straightforward route, make sure that that's explained to them in advance and that the person at the greeting area can help them. If there's an elevator, make sure it can handle wheelchairs, make sure there's a good turning radius. Are there bathrooms near where you're doing the study? Are there bathrooms right inside that entrance? They may have been traveling for two hours. Make sure that they can take a break before they go in there. Make sure that the lighting is good and variable. Make sure the place isn't, you're not walking in through a Yankee Candles store to get into your building. Do you have Yankee Candles here? Think of somebody murdered a perfume store. I don't know how else to describe it. Don't ever go in one. But things like smells and sounds and lighting, those can all be potentially confusing and it's different depending on your audience that you're bringing in, so bear that in mind as well. Definitely make sure you have a relief area for service animals. It's bad enough when somebody has to bring their dog on a bus for two hours, but then the dog can't take a break and when they're in there doing a study with you for a couple hours, what's the dog gonna do? And it can be as simple as just having a safe tree in the parking lot or puppy pads, depending on your comfort level with that. And then of course, make sure that when it is time to wrap everything up when they're exiting the venue, you don't leave them stranded again. Walk the person out. Walk them to their transport. Maybe wait for the transportation. Think about when you're releasing them. Is it during rush hour? You know, there's extra chaos involved with rush hour. Are the buses still running on that route? Are taxis still coming out there? I did a study a couple of years ago with a young woman who was blind who got there via a bus. The bus dropped her off a half mile from the facility, from the venue and she walked down the street, literally in the street because there was no sidewalk, no pavement, pedestrian, what is it, is it pavement? There was no pedestrian way. So she walked in traffic for a half mile with a cane and her phone telling her which way to go as she went, which she had to hold up to her ear to hear. And when she wrapped up the study, because I didn't know this was the case, she told me she could get there just fine, my mistake. I walked her to the bus stop, walked her a half mile to make sure she got to the bus stop and stayed in a pedestrian walkway and we got her there. And thankfully we'd built enough time in between the sessions. If we had not, it would have thrown off the next session, she could have been killed. It was just, it was a terrible experience and that was partly my fault because I didn't identify the route in advance because I was in a foreign city and the venues were out because they didn't do their job, they're due diligence of making sure that people could actually get there. So it's not just getting them there, it's also getting them out. Kind of bad form to kill your participants, I'm just saying. Maybe don't do that. So now that you've got things sort of set up and maybe you've identified we're gonna do it, you have to think about how you're gonna recruit people. This photo is from NASA. Their Goddard Space Flight Center brought in the blind camp of Maryland and what they did is they brought in blind and deafblind people and they set up all of these spacecraft parts on tables and they just let people walk around and pick them up and feel them and ask questions. There were no kids because if I was there as like a 10 year old, I'd be swinging one of those around and that would be a mess. But it was the first time many of them had gotten to experience a lot of these things which was awesome. Recruiting for them of course is very easy. They just went to the blind camp of Maryland and said, hey, come on in. You'll probably have some different things that you're going to want to do. So let's talk about that. Again, we've covered some of these fundamentals so we need to figure out what you have to do to get the right participants. In particular, you need to know in advance what kind of skill level you're looking for, what technologies you're looking for, what experience you're looking for, just like you would for any usability study. You know, do I want only Mac users? Do I want only Windows phone users? Because there's like two left. What do I do? Who do I recruit? What's that skill level? Still applies here. So define those characteristics up front. Now here's the really important thing. Lean on community or support organizations. They can do this heavy lift for you if you've never done it before. Many cities have a blind association, just as an example. Right? You can just give them a call. Think about where you're running this study. Find out what that city has to offer. Figure out what organizations are there and what their constituency looks like. And once you find one, ask for names of others. They often have a lot of overlapping constituents anyway. Often a person who has some sort of disability doesn't have just the one. There's often something else associated with it or linked to it. So they're getting different services from different organizations in many cases. These organizations can help you nail down the demographics that you want. Everything from age and gender down through skill tools, the kinds of jobs that they have. So if you're testing a travel application versus a sewing application, you're gonna have different users that you want to get involved with that. And these organizations can help define some of that. They also tend to have some brand recognition. Getting an email from Adrian Roselli, that guy, saying, hey, how would you like to come to my house and participate in a study? I mean, that's lovely if you reply, yes. And a little bit alarming. But realistically, if it's Lighthouse organization in San Francisco, or now I've forgotten their name in Boston, they have name recognition, even if I forget it. They're going to get a higher return rate on participants. They're going to get people to participate in a way that you will not. They will assume that the study is pre-vetted. This can only work in your favor. So you want to ally yourself with these organizations wherever you can. And remember that they have existing relationships. They already have people that they probably have participated in studies. They probably already have done studies before and can even introduce you to other organizations or find specific participants. So lean on those organizations where you can. You're not an expert in that specific disability. They also can provide contextual support. Things like skill level, ability, things that might trigger as a little strong of a word, but things that might cause a lot of stress for the participants. They can give you a lot of help on that. They can also help you with venues, with messaging, with structuring the test. In my case, they did a great job by giving me maps because I did a lot of in-home studies and they mapped out all the routes for me to take to get to the people's houses, which was awesome because you can't ask a blind participant how to get to their house. What road do I take? Ideally, let the organization do the recruiting for you. They can send out the emails. They can make sure that the language is good. They can vet everything. They can make sure that they're sending it to the right people. Let them do that heavy lift. Some of them do it for free. Some of them charge a fee for it. Be happy to pay for it. Be happy to pay for it. It's something you don't have to stress about. Can you hear when it gurgles on the microphone? It's this, it's not my stomach. Anyway, let the organization recruit. It pre-validates this study to a lot of the participants. Accommodation, accommodating your participants can be kind of tricky. There's no real story with this picture other than I hope he stopped before he hit her because that would be awkward. For those who can't see it, it's a toy scooter with a toy girl in front of it. No humans were harmed in the making of this photo except the person whose toy fell off into the crevasse. Yeah. Anyway, you might find really quickly that a scooter and a dog can overwhelm almost any testing facility. A dog alone can send some places into panic, so be aware of that. Make sure that you're building extra time for every task. Every task, bathroom break, extra time, drink of water, extra time, spending time on a screen to click the one button, that one call to action that you have, that's the big orange button in the big white field, extra time. Always allow extra time. You don't know their skill level. You don't know how easy it is to use that screen. That's why you're doing this study. Surfing with assistive technology in particular is a bit slower than you might be accustomed to for a variety of reasons. So don't rush them. Allow them to take breaks. Allow them to rest whenever they're feeling fatigued. This is not a pressure test. If you're doing that kind of study, that's a whole different study. There are lots of ways you can do that that's incredibly painful to watch. I'm not gonna talk about that here. Allow them to leave early. Allow them to start late. Just allow that kind of flexibility. You might have discovered that they need more time, and then they might have separately discovered they need even more time. Like I said, allow them to be late. Remember, they're at the whim of everything from transportation services to your poor instructions and everything in between. So don't stress if they show up late. Do the best you can. This is a, this single sentence, there's a lot to unpack there from experience. Service animals, canes, wheelchairs, you name it, they do not play well with tripods, and they do not play well with cables. I have had a lot of equipment unplugged and yanked off tables over the years, even when I thought I taped everything down and wrapped it all up. And it's, people are really amazing in their ability to mess up my poorly set up environment. But yeah, just remember that you're gonna run into struggles and considerations you had not considered. It's gonna happen. Also remember, don't pet the service animal. Service animals are there to do a job. I'm gonna have to, I'll stress this a few times throughout here, but don't pet them, let them do what they're there for. Make sure that the service animal has a clear space under the table. Under the table is a great place to put cables, for example. My first experience with a cable mishap, the dog got up, the monitor went down. Anybody want a slightly broken monitor, by the way? So make sure the dog can lay at the owner's feet. If it's a dog, whatever the animal is, typically a dog. Leave that space free of power bricks as well. Those can get pretty warm and can be uncomfortable for an animal. Ultimately, the owner is gonna tell you where to let the animal go. Some will sit to the side, some will want to be under the chair, some at the feet. Sometimes the owner will take the dog off of the harness and say, go nuts. In which case, now you have a dog to manage. So if you didn't pack your lunch properly, the dog is not working anymore. The dog is like a four-year-old. So be aware of that as well. They will get into everything. Consider different seating options for your participants as well. You have to really think about different body sizes and shapes, and you might wanna have a number of chairs available. Some people will prefer the stability of solid feet like the chairs you're in now. Some people will want wheels because they'll need to move around. Not everybody who comes in in a wheelchair is going to want to stay in the wheelchair. I ran a study where one of the participants was bigger than we had expected, and the chair that we provided was tiny. I could barely fit in. It had these little arms on the side, and it was on wheels, and she sat on the very front of the chair, the very front edge of the chair, and it kept rolling back from her. So it was a terrible, uncomfortable experience for her. I scoured the building to find a chair for her, and we only found one toward the end of the study, but it also kept her distracted for the entire study. My mistake, you know, valuable lesson there. It's amazing how much the seating condition can really, really screw up a person's ability to focus. I actually saw that. Thank you for that. This is very meta. He told me I have 10 minutes to go. I'm going to go over. Don't you guys worry. I got plenty more. Also, as I said, people who come in in a wheelchair or a scooter, they might not want to stay in it, so make sure there's a place for them to park it. So think about the size of your room now. You have a wheelchair that's got big saddlebags and a dog that's roaming around, and you have five extra chairs for the different people to accommodate them all. At a baseline, you need a room about this size for one person. I mean, that's hyperbole, unless the dog is really wired. I don't think anybody uses a greyhound as a service animal, but if they do, yeah, okay. Technology. This is a setup that probably won't work very well, but it's a general concept that I wanted to at least show you. This is somebody who's playing on a mobile device and is also being recorded and broadcast on a screen up here. Also, consider the mobile devices. People might be holding them to their ear or not as well, but the general gist here is you want multiple ways to capture what's happening on the screen, just like you would in any particular usability study, but there are some special accommodations to consider, and I'm going to run through some of them. The most critical, use the participant system. The person who's coming in for this study, if they can, have them bring their own kit. It's already configured, they know how to use it, there's nothing special they have to do. By the way, if you're making them test over a mobile connection, when we talked about compensation, remember you're hitting their data plan, so there's some fees associated with that for compensation. You don't want to waste time getting a user, getting a participant acclimated to your weird setup. You're running JAWS at normal human speaking rate, and they're just going to pass out out of boredom. Don't waste time doing all of that. If you're testing at a community organization, they've already got systems configured, you can typically just lean on those. If you're doing any mobile testing, don't use those mounts where the phone rests in the hand and they have a little camera that points over it. Think about how many blind or low vision users rely on the screen reader on their phone. Yeah, that mounts going right into their skull. So you can't rely on a mount because they're going to use the phone in ways you did not expect. A lot of them will interact maybe with the screen down, and they control it this way, some sideways, some of them are doing it like this. So you can't have that phone that they're using clamped on with gear, that will be a huge problem. Do not mess with the user's configuration, period. Do not mess with it. Remember, this piece of equipment that they're bringing you is their lifeline. It's how they do their job, it's how they interact with their family, it's how they get food, it's how they do everything that they need to survive. And if you mess up their configuration and they leave there and they turn on that system and it doesn't work, these are not typically highly skilled people. Just like the rest of the world, they're not typically highly skilled people. They get the system configured the way they want. This is why every time I go to my parents' house for a dinner, the first half hour is me spent reconfiguring their system. It's not unique to my parents. But that will also taint your results and it will frustrate the users. If it's absolutely necessary though, so there have been times we've had to turn down the speech rate on a screen reader. Okay, make sure you make notes of what you did and you bring everything back to where it needs to be. I see you panicking with that time, saying I'm panicking now too. It's fine, it's fine. There's no more talks in this room, right? But definitely return it to the way you found it when done. You have to make sure everything is functional or you've just damaged their ability to live their life. Process. You might send them any documents and agreements in advance. You might have not formatted them well. They might be inaccessible. They might not have read them like normal human people. So be prepared to read them out loud. Spend some time doing that. Wherever possible, point a camera at the user and all of the interactions. Ideally two cameras. If you can drive a second display plugged into their system even better so you can watch it. You don't wanna miss where the assistive technology is guiding the user. As I said, driving a second monitor. Do not interrupt the user when using assistive technology. Particularly critical for a screen reader. Screen reader when it comes to a page or an application announces all this really critical information at the start and if you interrupt them, they have to start all over. Don't do that. Always let the assistive technology do what it's doing. Let it happen. The user will make mistakes. It's your fault, not theirs. Doesn't matter what they did, it's your fault. They knocked over a monitor because they wave their arm. Your fault. Their dog got into your lunch. Your fault. The application caught fire because they pressed the wrong button. Your fault. It's never the participant's fault. They're already going out on a limb to participate. You wanna get good feedback from them. Make sure that you're getting all the feedback that you can get and be comfortable taking the blame for everything. Users will apologize for finding errors. Whose fault is it? Thank you. I tried. I really tried. Privacy. So in the States, we have something called HIPAA. I don't know what the equivalent is here, but the general rule of thumb here is, treat it all confidentially. Personal health information will be revealed. You don't need to ask what their situation is. You can ask the technology they use, but don't ask them why they use it. Ask about the technology. You're gonna find things out. This person's blind. It's gonna be kind of obvious. Accept that it will be revealed, but be prepared to treat it as confidential. It's about respecting their privacy, making sure that you're not asking too much information. In any reporting you do, anonymize the data. Please anonymize it. First names, simple information. Don't give up too much detail, especially if you're doing video clips. Clip out things you don't need. Crop out stuff that doesn't belong in the photo. Avoid photos that show unrelated parts of the scene. If you're coordinating with a recruiting organization, they will help with this. They're gonna have some familiarity with the preferences from their participants, things they want to share, things they don't wanna share, and they'll give you some guidance on the best language and the kind of language you can use in materials as well. How am I doing on time? Good news is we're wrapping up. So I'm gonna still go over, just so you know. CSUN, conference for, conference of assistive, what is it called now? Assistive Technology Conference. They changed the name a couple of years ago. Joe Doulson, he's on the WordPress accessibility team. Sat down with Leonie Watson, who is with the postiello group. She's blind. She's also a developer and she's an expert screen reader user. He sat down with her to test Gutenberg. Gutenberg is not ready for straight usability testing. There's still a lot of accessibility issues. So I would never suggest that we do everything I've talked about yet. That all needs to be hammered out to do the usability testing. But this setup was very simple. He did it in a bar. They weren't even drunk yet. That might have been a mistake. But it was a great opportunity to get some immediate feedback on how Gutenberg performs with a screen reader. It can be very simple to set up these kinds of tests depending on your participants, depending on the context. A lot of ways to do it so they don't feel too restricted. I will be posting these slides later. These are some links to some other resources. That first one is an article that came out in March. Well, after I wrote this talk, just so you know, he stole everything of mine. I will be speaking at WordCamp Europe and WordCamp Buffalo, you should come. We have chicken wings, two different topics. And again, my slides will be, they're not there yet. At r-o-s-e-l dot l-i slash w-c-l-d-n. Yeah, I think I'm done, right? Am I right on time? I've got a minute to spare. All right, do we do questions or what? Yes, we do. Thank you. Thank you. So like two. Cool. Questions? Any questions? Sammy, are you just doing it as a courtesy? Thank you. I appreciate that. When I do the studies, there's usually two people running the study in the room and one participant, and sometimes an animal. One person is running the study and the other person is the one who's making sure all the equipment is doing what it's supposed to do. Keeping the camera working, making sure if the laptop acts funny we can fix it or if they drop the phone we handed to them. And it allows the person who's focusing on just running through the script or the study itself can focus on just that. So two people plus a participant and then behind that mirror, anybody you want. How many overall, oh, I'm sorry, how many participants we have? Overall it depends on the client. Typically we'll do an hour long session with an hour break. So we'll get four to five people in per day and we'll do it over three days. So what is that, 15 people roughly? Yeah, but it depends on the client and how much they're willing to spend in the venue and everything else. Sure, it again depends on the client. So we've had clients who've said, hey, we've built this thing, we've done some custom JAWS scripting. And so we'll find JAWS participants. We've had other clients who've said, we just wanna know how this works and we'll bring in a variety of disability types and we don't ask what their assistive technology is in advance. Sometimes it's not relevant and sometimes it doesn't matter because we're not gonna get specifically the ones we want. So each study is unique and each one has different outcome objectives, output objectives, so it's highly variable. Did I see another one? Hi, I'm interested in what kind of training I could do to get into this area. What kind of, where do you start? And thank you for your talk, it was great. That's a great question. I gotta tell you that I got into this accidentally because I worked with a lot of agencies over the years who would run regular usability studies and then I would start to fold in people. So I learned how to do the usability studies from traditional ad and marketing agencies and now I work with a guy, David Sloan, who's out of Edinburgh. He works for the Possello Group. He has a PhD, which I guess is an impressive thing and he's a researcher who does lots of studies. He's written some good stuff on this. Sarah Horton has written some good stuff on this. As for training, I don't know what to tell you because I didn't do a structured formal training. I just kept following people who were smarter than I and they told me what not to do because I kept doing it wrong. It's pretty much the best advice I can give you. Screw it up a lot and when people stop getting angry at you, you're probably doing it well. How I've survived.