 Good morning and welcome to this week's edition of Encompass Live. I'm your host, Krista Porter here at the Nebraska Library Commission. Encompass Live is the commission's weekly webinar series where we cover a variety of topics that may be of interest to libraries. We broadcast the show live every Wednesday morning at 10 a.m. central time, but if you're unable to join us on Wednesdays, that's fine. We record the show every week as we are doing today and is then posted to our archives for you to watch at your convenience. And I'll show you at the end of today's show where to get to all those archives and to navigate through those. Both our live show and our archive recordings are free and open to anyone to watch, so please do share with your friends, family, neighbors, colleagues, anyone you think might be interested in any of the topics we have on the show. For those of you not from Nebraska, the Nebraska Library Commission is the state agency for libraries in Nebraska. We are the state library as what is called in other states. So we provide services to all types of libraries. So you will find things on our show that will run really the gamut of anything that is library related things for publics, academics, K-12, museums, corrections, et cetera, et cetera, et cetera. Our real air only criteria is that it is something to do with libraries, any types of libraries, something cool we think they're doing. We bring in speakers from inside, from Nebraska, from outside Nebraska to talk about things we're doing at their libraries. Sometimes there are sessions here from the Nebraska Library Commission staff on products and services and things that we offer to our libraries here in the state. So you should definitely be able to find something on our show that would be useful to you. Before we get into today's show, I just want to briefly talk about something we're doing here at the Library Commission. We are still in the deep in the throes of the COVID-19 pandemic right now. And at the Library Commission, we are trying, are providing resources as we can to our libraries in the state. If you're not on Nebraska Library, check with your state library or your state library association. They may be offering the same kind of services, same kind of information. We have a blog poster that's pinned to the top of our website. So it's always there at the top for you. Everything else comes in underneath it. We also have a list we are trying to keep up with whether libraries are open, closed, special accommodations they're making, Wi-Fi in the parking lot, extending due dates, et cetera. And now we do also have libraries re-closing because of opening to Zoom. We try to keep that up as well as we can. We have a web forum that libraries can submit to us to let us know what their situation is. Or we do also have staff here that proactively go out and check out libraries, webpages and Facebook posts and whatnot to keep what's going on. We have a specific subpage here on different topics related to what you might be dealing with in your library. People asking about homeschooling children, unemployment, et cetera. I want to highlight specifically what about my library. This is where we gather things specifically for you as a librarian that you might find useful as you were trying to navigate what you're going to be doing during the current pandemic. We have a lot of resources on here, links to different things as new information comes up. We add it. If there's new webinars or events that happen that you can attend or recordings of them, we add them to here as well. So a lot of this is general information that anybody can use, of course, from, you know, ALA, CDC example. The realm project being done specifically studying about library materials. But some things are specific to Nebraska. So just if you're not in a Nebraska library, you pay attention there. So I just want to make sure our Nebraska libraries know this information and resources are out there for them. Let us know if you do have any questions about anything that you don't cover here if you need more information. So on to today's show. I am going to hand over control to you right now. Jessica, so you get your screen up. So you should see the pop up saying show my screen or share my screen show screen. Yeah, there we go. All right. Yeah, yeah, we can see it. So today we're talking about drive-thrus not the kind of drive-thrus you go and get your lunch or dinner. But user testing with your library. Jessica Gilbert-Redmond is on with us. Good morning, Jessica. And Kalisha Christensen. They are both from the University of North Dakota and they've done some great research and had some good information for you. So I'm just going to hand it over to you guys to do more introduction of yourself and your present to do your presentation. Okay, thank you. So yeah, Kalisha and I work together at the Chester Fritz. Well, University of North Dakota where I was the web services librarian at the Chester Fritz library. And she was the web designer. So we work together a lot on a recent site redesign. And we both discovered that we have a love of user testing and usability and getting to know our users. And so we thought this might be a little bit fun and interesting for other people. And we called it drive-thru user testing because we wanted it to be quick and easy. So things that don't take a lot of money or a lot of time. So if you're a single librarian, you know, an only librarian, you can do these things. I mean, there is some time involved with preparation, of course, but it's doable. There we go. Sorry, my slide was not changing there. And as you can see, there's Kalisha and there's me. If you want to tweet about this as we're going through and Compass Live has a hashtag, hashtag incomplive. And then if you want us to see it specifically, if you use hashtag drive-thru UX, I'll be checking that later and we can get back to some of your stuff. And as I said before, we're at the University of North Dakota in Grand Forks, North Dakota where Kalisha is web designer and developer. And I'm now the online resources and services librarian at the School of Medicine and Health Sciences. I kind of moved over a few months back. So our agenda today is, first, I'm going to give you a short introduction where we're going to talk about research in general and what that means and what it looks like. And then I'm going to bop it over to Kalisha and she's going to talk about accessibility as a part of user experience. And I know some of you might be surprised to see that on there, but it's a very important part of user experience that is easy to do. It's just a lot of people forget or think it's not as important as the other user testing. They get very excited about the other user testing and they forget that there are tools and things out there for them to help all users and to make sites usable for all users. And then we're going to get into some quick and easy user testing that you can do no matter what library you're at. So I recently finished reading Erica Hall's Just Enough Research and it really struck a chord with me. It's not just about user research, it's about research in general. But she gave a list of things that research is not. And I just kind of want to go through some of these because I thought they were very interesting. Number one, it's not asking people what they like. You're not asking someone if they like this shiny object on the page. You want to know if they can use it, not if they like it. It's usability first and design later. And I know Kalisha's a designer, so do you have anything to say about that, Kalisha? Yeah, just that a lot of times design is subjective. But the usability of it may not be. So yeah, you need to look at the functionality and then the data eventually. But the look can sometimes, yeah, it's just subjective and can be personal preference. And that's true. So I mean, if people can use it, not 100% of people are not going to like what it looks like. As we found out with the website redesign at the University of Dakota, it's usable. People get where they need to go, but not everybody loves how it looks. And that's fine. You can't please everybody. Number two and three kind of go together. It's not about looking smart and it's not about being proven right. This is something that sometimes people go into user testing thinking they're going to have a aha moment. Like, I think it should be this and everybody else is against me on this or everybody else thinks something else. I'm going to prove them wrong. And it should never be about that. It should be about the user. You should go into user testing and any kind of research trying to find as the data that you need to make a connection between what you have. And if it proves you wrong, it should, you should be okay with that. It's not a personal thing. And four, it's not better just because you have more data. I feel like we kind of fall into this idea that if we have more quantitative data, then it's better. If we can put out a survey and get 700 people to take that survey, it is better than doing other types of user testing with five to 10 people. And that's not necessarily true. And we're going to talk about that a little bit more later. And part of that is, and we're moving into assumptions are insults. So you're, you're assuming that what you're making or what you're getting from the survey is 100% the truth. And that's an assumption that you're making. You might not know what you think you know to know what users are thinking you have to talk to the users and simply sending out a survey isn't talking to them. It's asking them very specific questions where usually you give them very specific responses and you're telling them, I want you to pick a response out of these. There are ones where they're just fill in the blank type of surveys, but even those people might still be trying to tell you what you want to hear or what they think you want to hear. And there's, there's nothing like actually talking to the people. And you're also assuming things about the people that that brings in some biases. So you may think that you know who your users are, but when you go out and start talking to them, you may find that you have users you didn't even realize you had. You may find that you have more varied you group of users than you thought you had. So just kind of keep your assumptions to the minimum and talk to people and go into it, not thinking you have the answers and you're proving them right, but that you don't know the answers and you want to find them out. Except no substitute for listening to and observing real people who need to do the things you're designing a thing to help people do. That just underlines what I was just saying. You, you really need to see people doing the things that you're expecting people to do on your site. And that's very important. And this isn't just technological user testing. This goes for pretty much any kind of user testing. If you're doing physical space user testing, you can't just put chairs out there and just say, well, you know, five people told us they like those chairs, we're going to buy 100 of those chairs. And that's it, you know, you know, by observing that people like a different type, different types of seating space. So you want to give people the options and observe them and talk to them and find out why, why are you sitting here instead of there. So it's not just technical technological online user testing. It's any kind of user testing that you need to talk to people. Usability is the absolute minimum standard for anything to design to be used by humans. If a design towards the intended user who attempts the intended use that design is a failure from the standpoint of user centered design. So a lot of us have heard of mobile first when we're designing websites and things like that. And that's on a technology scale. What we really need to be thinking about on this end of things is a user first. And again, it goes back to what Kalisha and I were talking about with the shiny and the pretty. You can you can come to that later, and you can always make that better and you can always change it up. But you really need to make sure that your users can find what they need where they expect it. And if that doesn't happen, then your whole entire design is a failure. It doesn't matter if it's pretty and shiny. If your users can't find what they need in an intuitive way. Okay, I'm going to turn things over to Kalisha. All right, let me skip my screen here. Kalisha, you should see the pop up now to share your screen. Yeah. One thing we didn't, I guess we must not have done this when we did our test because I have to do one quick thing here. Sorry. Right. Yeah, because you had said that with Jessica would do it, but I thought I did. And, but let me guess you're on a Mac. Yeah, it's going to it's going to request that you give it you need to give it permission. And then it says I have to restart Jessica. Would you be willing to? I guess so. Yes. Yes, I'm going to show you. Let me. Can you still can you guys see that I just want to see. Yeah, we've got the accessibility human slide up in that problem. Okay. Okay, here we go. All right. So yeah, I want to talk about accessibility for in regards to user protesting and usability, because it's really a foundational piece. And as far as our kind of concept of drive through, you want to be able to do it quickly. You want to we want this to be easy usability testing. Well, a really easy place to start is with automated tools and accessibility testing. You can take care of a lot of low hanging fruit and a lot of potential usability issues by addressing accessibility issues first. And then I kind of want to talk about going beyond that super easy level of automated accessibility testing and then going the step beyond into accessibility testing with people. And I found some a recent and very interesting and informative article that I will be mentioning and sharing with you guys as we go along here. So Jessica, thank you. The one I mentioned that accessibility is really key. There's this neat graphic called the US honeycomb. And I kind of reworked it for this slide and sorry about the train if you can hear that. I live right next door to a train crossing. Then before it's actually not very loud. It's just a little background. But so I kind of reworked this a little bit and just to pull out the fact that accessibility and and having an accessible service or website or whatever it may be is a key part of user experience along with these other items. Jessica's really talks about her slides and her focus is really on the findable and the usable useful like all of those other things, but accessibility and accessible can be a really key component or a base layer for a lot a lot of that. Next slide please. So for that for the basics for that foundational layer of accessibility and as it relates to user testing and user experience. So you're going to want to do that automated stuff that takes a lot of the workload off of you later on. But just keep in mind that that doesn't solve all the problems. And you really are going to have to turn back to the user centered and use in person or like in person virtual user feedback to really make sure you're in touch with your users and serving your users. So as far as automated tools, there's one called a wave, which is from web aim, which web aim stands for web accessibility in mind. And it's just always good to keep that in mind accessibility web or otherwise. But that's a really good tool and they focus on having their tool focuses on issues that have a broad impact on accessibility. So it's really that low hanging group quick, easy fixes. If you fix these things, you're going to be in pretty good shape and have a pretty solid base for anything. Any other issues or things you hone in on or when you want to get into a more granular level, you'll be able to have a good base if you use this tool. And then style guides accessibility and inclusion and diversity and equity are all very tightly related. They're all kind of the same concept of user centered design, inclusive design. So if you can get familiar with a diversity, equity and inclusion style guide, if you don't have one, create one, find one, get familiar with it and always just keep it close to your heart and keep it. Use it as a touchstone as you go forward and like do your testing. You know, just to keep that in mind that don't make assumptions like Jessica said about your users. Yeah, it will help guide you a lot and help you progress and iterate quick more quickly and more effectively. And then there's this really handy toolkit called the Microsoft inclusive design toolkit. And it has a bunch of resources and a bunch of information and it's handy it's like a PDF they also have a website and there's just lots of good information there so I included a link there just because it can really help you key in to your users perspective or how it helps you figure out how to key in to your end users perspective. I want to mention to all around this slide where you've got those the different links there. I just want to let everyone know that these slides will be available after the present after the session as well. So don't worry about trying to scribble down all these different URLs and keep track of afterwards we post up the recording of today show will have a link to these slides as well so you'll have access to them through there. Jessica next slide please thank you. So one thing though it's kind of a red herring when you start talking about accessibility. People start to think about people with disabilities oh I need to focus on this and yes you do need to be inclusive and you need to have it in mind and have it be a focus but it doesn't necessarily be need to be the focus you want to. You don't want to separate out. People with disabilities or accessibility issues from the broader scope of your accessibility testing and your your goals of having a user centered design or website or service. So really represented a representative users means all users and and you but you do want to still be inclusive purposefully inclusive and that's where that style guide comes in to play in that toolkit. So. They raising the awareness there. Next slide. Okay so I mentioned earlier that there's this series of articles and it's from the patch yellow group and I've only read that so sorry if I did not say that correctly but they are the accessibility experts. And they have a lot of good resources and they recently like within the last couple months have been putting out a UX series and since they focus on accessibility it's kind of. UX married with accessibility and it's very very interesting and so. There there will be a we will share a link for that as well. But. They have a lot of good content and so. One of the interesting things is this concept of early evaluation of digital products with disabled users means we can identify and resolve accessibility issues before it's too late to address them. And being too late to address an accessibility issue means. You're not doing that drive through quick and easy testing you're going to end up having a much bigger lift later trying to fix and reverse things for decisions or just slightly incorrect decisions that were made early on. If you have to pivot later to fix accessibility issues so if you can get those identified early. Great so that's why it's good to do that automated testing. Then you still really want to do that human centered in person testing where you're seeing somebody use an assistive technology or you're you know really observing what they're doing. So that you can make those appropriate pivots early on. I think next slide. So. How to go about accessible user testing here so there's some considerations. And these are paraphrased from that patch yellow group UX article. Don't make your user start at zero do that automated testing first and fix any known issues first as well you just don't want to be placing that extra burden on your users when they're. When you're trying to focus in on that higher level of usability issues. Hopefully the basics are covered try to get those all taken care of first and you might be surprised at. What issues you think are are not basic and they end up being like a basic issue that you could have fixed earlier it's like oh that dang if I would have just done this thing we wouldn't have been. Trapped or stuck or you know have to go so far back later on down the road. You're going to want to check all your materials for accessibility or make sure that you have appropriate alternatives. You want to make sure that your physical location or technology for remote sessions is accessible. Because you don't want to inadvertently be excluding users just with that focus of inclusivity of everybody. Yeah just keep that in mind and then recorded data should have context. For instance which assistive technologies did they use were they using jaws screen reader were they using a zoom magnifier software were they using a zoom magnifier. Who knows it could be anything so you'll want to know how that impacts your data that you're gathering. Next slide. So kind of a process is to be sure to be inclusive with the users early on. And then evaluate by observation after you've taken care of all the low hanging fruit then you want to dig in and really get an idea from the user's perspective. Iterate build on that do this in smaller batches with small small amount of people so that you can do it more frequently because then you're going to hone in and get more granular and more effective data. As you go along so yeah the more iterations of the process the better the better results you're going to end up having. I think next slide. So automated testing is kind of like take out versus dine in just to reiterate that again with automated testing you get a meal you get it it's delicious and it's not a lot of effort and it it fills you up you know you get a full bill at the end of it. But you you end up still missing a lot you miss a lot of context you miss a lot of information and perspective that is important to have so that's why it's you kind of need both. Yeah. So just kind of a fun way to think about that. Thanks Jessica. So how do you you're rocking that. How do you do this accessible user testing again the pet the pet yellow group they have a part of their series is specifically on user testing with accessibility in mind. And one of the big takeaways is observe mentioned this a few times Jessica mentioned it as from the research book from the intro. You what really want to observe your users a survey or interview may not reveal your users to perspective or problems they can have bias the survey can have bias. They may focus more on the questions ask than any barriers they encounter during the session which could be a much bigger deal and more important issue to solve. Observing your participants reduces the effort required by them to have to think of what kind of information are they trying to get out here. So that yeah that's another piece of observing your users is key when you're trying to take it to that next level of user centered design. So I do have a question about this you said that this group has been putting out a series of things just recently about this. I assume I hope there's something in there about the fact that some libraries are not actually open to be able to in person observe that you know related to the pandemic is that do they have any sort of like. How would you do this now if you're not able to actually see what you know people are not coming the way they used to. Yeah I think they do have some information on the virtual kind of user testing. And I don't think that the series is complete yet. So I think I don't know how many more they plan on doing but I would hope if they haven't addressed it in one of the previous ones I don't remember. I know it's been mentioned but I don't know if they have a specific like here's what you do. I would hope that maybe they are going to come out with that. You know some it's here in Nebraska. Our libraries are all over the place with whether they're open closed partially open. You know you can only come in and use the computers for half an hour an hour. So you know the observing would be. Difficult or awkward or at least different than the typical which I guess is a good thing to evaluate to I mean you guys have been talking about you know in general like in a normal situation you're trying to figure out what is working what isn't. But doing this now when things are not the normal. How is this current situation we're going to working or not you know at this moment. Not even necessarily for our final version of our website or final version of how we're doing things in the library but right now we need to really be know really find out is what we're offering working or not. And I think this could be so do you think some of this could be modified to you know in our current situation do this with the thought of just because of how it is now not what it will be later when everything is over. Yep, I think that the observing we say the word observe yeah kind of makes you think of in person observe observing standing over your shoulder but it also can mean like like what we're doing right now I could. Screen sharing observing I mean there's going to be probably more difficulties and barriers depending on what kind of if there is an assistive technology or something involved. But I think that there are still good methods and ways of doing observing that's not like standing over shoulder or person to person I think you can do and I would hope that that. And I would hope that that series might touch on that since it is timely. Right yeah like virtual things and you know the web web statistics website statistics and things like that out who's using what and what they're you know accessing. I know we depend on that a lot here at the library Commission of what's working what's not should we change how we're getting people to this particular page what are they actually searching for when they end up on our website because that's not what we thought yeah things like that. Yeah and one interesting thing and it's kind of stretching because I am not an accessibility expert and I'm new to this I just find it really fascinating. But something I encountered recently that's interesting is that as far as tracking how people interact with the website and this is a good thing. Assistive technologies are not you cannot actually know that somebody is using an assistive technology you're not going to know if they're using a screen magnifier you're not going to know if they're using a screen reader. Generally you're not going to know that information it and as they should it looks like any other user is interacting with your website so you are getting you know that is this usable but it is important to make sure you're still accessible to people who have maybe using an assistive technology. Because you don't know what they may be how they may be trying to use your site yeah absolutely yeah. So yeah it is a it's very interesting and definitely new to me would like to learn more about that part so yeah. I think that's it for my slides. Okay, now we're going to switch over to the user testing part after you've done your accessibility checks. And again as Glisha said, when you're doing user testing you're getting a lot of these students without even knowing it and that's it's the same as you know the web statistics. You're getting a lot of users with differing abilities and you don't know who has a color deficiency necessarily you don't know who has like a processing disorder or something like that and it's not you may not need to know what you need to know is how our users interacting with this and is it getting people where they need to go. And so the accessibility part of it is super important and I was I was kind of thinking about what you guys were talking about just now and with the screen reader one thing that I made sure to do when we were doing our. redesign for the library website was to use NVDA a free freely available screen reader and go through the site and make sure that it made sense like it was going to the headings the right way that it was easy to navigate the site with that so there are free tools there as well. And it kind of goes you know it goes along with this first slide that I have, you know, starting easy and look at what others are doing. Your website doesn't exist in a bubble or a vacuum. So one easy way to start testing and looking at your site is by testing and looking at other websites that are similar so your public library look at other public library websites of like a similar size like you don't want to look at, you know, a public library. Big consortium libraries website if you're a small roll library because you're doing different things and your audiences are different. So kind of think about your audience and find library websites that are similar to the audience. So I'm in an academic library so I was looking for academic libraries that were similar, because it's similar in as much as we could decide, or as much as I could find out, similar in research level faculty staff levels student levels and potentially budget levels because sometimes you're you're looking at it and I have remember don't just look at the shiny. Sometimes you get so involved and looking at the cool new features that another library has on their website that you forget that that's not why you're there. You're not there to find all the cool new features you're there to see is their site accessible. They may have full new features and their site is completely unacceptable inaccessible to users. So, I also did use screen readers on other people's sites. I actually started with our own main university website and found some interesting and funny things there which cliche talked about. Some things that you know, like hashtags that read weirdly and you didn't realize that they were reading that way to screen reader people are people who are using screen readers. Go ahead. I was just going to say that, and we can use those screen readers ourselves, but we don't know what it's like to be somebody who uses it on a regular basis. So that's why again it's important to not assume and to observe somebody really doing it. Because they're going to have such a different experience than we can we can kind of test the waters a little tiny bit and just get a little glimpse but we still aren't really able to evaluate a true experience of using that on a regular basis. And it's important that if you're doing some of this user testing that we're going to talk about. That you have these tools available to the user. So if they don't have their own laptop. And it's somebody who needs a screen reader you want to have a screen reader available and go in with the bill with the mindset that it may be a screen reader they're not used to so if they're using jaws. And all I have access to is NVDA. That's something to kind of think about to is it may not be the site it may be the familiarity with the tool that they're using as well. So there's all these things that we need to check our assumptions as we've been saying the whole time. But when you're looking at other sites just kind of focus on what if, if you have scenarios that you're thinking about for your own website and one that I was thinking of because I've done the scenario is can students find their subject librarians at an academic library. And so I set the scenario myself. If I went to somebody's website, another university, and could I easily find their subject librarians could I could I figure out how to do that intuitively, without backtracking and looking everywhere and that kind of stuff. So it's important that you test your scenarios yourself. And one good way to do that is by going to someone else's website, because of course you know how to get around your website you might be the one building it. So if, if you're not sure if a scenario makes sense go try on someone else's website. And see if you can do it on a similar library site. And Kalisha touched on surveys as well. They're so tempting they're like easy delivery or drive through where you don't have to put a lot of work, you feel like you don't have to put a lot of work into it. But there can be unseen consequences or costs. So if you lean too hard on the on that on the quantitative data. You're not doing the actual work of talking to people, and paying attention to what they're telling us because your survey could be very specific, like you're looking for very specific things. And you know that you're looking for those things so you build your survey that way. And there's a whole chapter of this in Erica Hall's book that in her second edition, she put in there just, I almost feel like she wrote the second edition just to stick this chapter in there about surveys and how, you know, they're not necessarily the best tool, even if they tend to be the easiest, because all of us have probably some kind of survey tool. We have Google Forms or we have Qualtrics or you know we have access to something. And so we think that they're quick and easy and they get us a lot of data and we get 700 responses in, you know, a week this is amazing. But again, it's not about the amount of data that you get it's about the quality of that data that you're getting. And bias can creep into your survey, both when you're building the survey, and when you're looking at the data itself, and you may not even be aware of that. A couple of examples that Khaleesh and I had been talking about is, if you're looking for demographic data, which a lot of surveys will ask for demographic data. There, it's too easy to creep into assumptions and biases about so many things related to demographic data. So, how you choose your age groupings can be assumptions. I know at my age sometimes I fall into one age group sometimes I fall into another. Sometimes I'm right on the cuss sometimes I'm in the middle of a group. So it's always interesting to see that or when we think about gender and you're asking for gender. So male female other doesn't really cut it these days so there's a lot of information out there about how to make surveys better. But the problem with surveys is a lot of times people aren't doing that because they know what they want to know and they're going to put the survey out there so if at all possible, just kind of leave surveys. Unless it's something you absolutely have to do I know that we've SMHS we are School of Medicine and Health Sciences we had to do a survey because we were told we had to do a survey, when in reality, I would have rather talk to people about things and again you know because we're not in our offices and with COVID-19 a lot of surveys are going on I'm sure we're all seeing them on our listserv and things like that. But don't be too tempted by it like it shouldn't be your go to UX method. And as Erica Hall said it's much harder to write a good survey than to conduct good qualitative user research, something like the difference between building an instrument for remote sensing and sticking your head out the window to see what the weather is like. So it's, it's kind of like you're building this thing and you're putting all this work into it. And you said hey what's the weather like or looked outside or put your head outside you would know so ask the people who are using it. And one type of user testing that I found very helpful, particularly when you're when you're organizing a site, or you're organizing your navigation or you're trying to make the movement of your site, more intuitive to a user or card and you can do paper based card sorts of course not if you're distancing for COVID-19, or you can do online ones. And there are some low cost alternatives for online survey or online card sorts that exist, for example, and I've done both so one of the online ones I used was optimal sort which has an option for to describe, or you can just do a one off survey and I think it was, you know, like $99 or something like that and you have access to, or not survey a card sort, and you can, you have access to all that data and everything and perpetuity. So like, I did it a couple years ago I could log in today still, and still see the data from that, and still pull the data from that. So there are some low cost alternatives. And a lot of this will depend on what kind of sort you want to do if you do want to do my online one was not an observed one, it was, you know, they took it. I got the results, but there are some card sort tools that allow you to do it live and you watch them and you take notes while they're doing it. But there are, you know, you're going to have to weigh the pros and cons because there are some more costs with that. Whereas with paper based if there's if you can find a way to observe people doing it. That is, you know, you just have to make your cards and, and it can be moderated it can be unmoderated so you can be sitting there watching them or you may not necessarily have to watch them so if there's a way to get the cards to the people you can have them sort it. You can do the paper based ones with a moderator but when we're in times like these you kind of have to think about what you can do and maybe adapt a little bit. And card sorts, you know, to keep with our theme of drive through is more like a buffet you have so many options like we just talked about paper based online, moderated, unmoderated. And then we'll talk about a few more here like there's just so much to choose from. And there are so many ways to do it. There are examples of paper based ones it kind of in person ones. On the left, there's a post it note one where they just stuck it up on a wall. If you have a whiteboard space that you can have people do it. That would be great and you can just. I know that when we've done something like this we had people move things around at a set time like here's the time that you're going to go in and do it I would go in and take a picture of it. So I could get the data, and then I would reset, basically just put things in a random order over on the side again the next group would come in and I would take pictures. So I didn't have to get the data down right then in between people I could get groups in, get the close ups of all the information of how they had sorted it and go on and then do my data sorting later. The center one is kind of it's another one. This is what I did with our wiki that we had a staff wiki that I did a paper card sort where they use the post it as their groups, they came up with their own little headings and groups and then the content cards for the the information. Another example of just a regular card sort using paper. And then here's a quick look at optimal sort. Since that's what I was familiar with, you'll see one on top where the user gets to name their own heading, and then one on the bottom where they don't get to name their own heading, which leads me into the types of card sorts there are three types of card sorts there's open where you have the content cards and the user will come up with all the names. So if you're doing something for navigation. You're like here all my pages. Now where should they go on the website what what are the headers, or the main, the main labels that are going to be on the headers. There's a closed card sort where you have all of the headers and headings figured out and they only get the content cards and have to put them in specific orders, or in the order that makes sense to them. So I don't recommend those headings, I don't usually recommend that one unless for some reason you absolutely cannot have an open or hybrid which we'll talk about in a second, because it removes so much choice from a user, and you don't even, you won't know, especially if it's not a moderated or facilitated one, you won't know that they had trouble putting this card in that you'll just get the card in that stack. There's an open or hybrid which they can create their own, but there are some so if you know like oh I want to I definitely want to have an about section, because you know, we want to have our basic about stuff in there. You could have a card that says you know a heading that's like about. And then they'll put the content stuff that's under there but they also have the ability to create their own headings and create their own little labels beyond the content. So here's the step by step. First thing you want to do is get your cards ready. And like I said if you're doing a close card sort you'll need all your heading cards in addition to the content. One recommendation that I have is try not to hand write the cards if at all possible. Because depending on who you have hand write them I, one of my paper based words someone else set it up and then was like here you're going to run this and I was like oh okay people could not read the cards. Because of the handwriting. So if I had yes I know I admit I have the worst handwriting I have other people to read have someone else do it on my behalf thank you. And the more that you write the more that I write the worse my handwriting gets like I might start out with it being legible but by the end I'm like oh my gosh I'm on the 30th card. And you don't want to write anymore so what I normally have done is just do a spreadsheet do a quick mail merge with like six cards per page you can use label you know like what would be a label and then just print them and cut them. So that's probably what I would have done, because then I wouldn't have had to write it. And you also don't want to have tons and tons of cards because that's just going to be. It's going to take too long because they're they're making a decision for each card they're going to pick up a card. They're going to see what is on it what label is is there and they're going to choose where it goes. And you have to assume that some are going to be quick. But some are going to be like, Oh, I don't know, it could go here, it could go there it could there's going to have to be thought processes and discussions. So I would say like if you have more than 50 or 60 to try to figure out a natural way to separate those out into other groups so that you can do. Kind of like cliche second multiple multiple groups and even iterative groups and see if that makes sense like here's top level stuff. Okay, now we're going to do based on those groups, some lower level navigation or something. And then you'll set up your sort area if you're doing a physical sort they need room to move around. If there's multiple people. They need to be able to move around each other and move the cards around as they're talking about them. You can do it on a table you can do it on a wall you can do it on a whiteboard depending on what you're using. If you're doing in person sorts. I prefer like I was saying before to have a moderator facilitator, where you explain what's happening. You tell the the user to think aloud, all the users should be thinking aloud. So if you see someone picking up a card and going remind the facilitator would remind them Oh what are you thinking. What are you thinking about this card, and they're taking notes while this is happening. They are not there to give hints because the label on the card is the label that a user on the end of the site like when you put it out there that's what they're going to have. And if it's confusing them you want them to think aloud and go I don't know what this means. You would make note of this label is not working because they don't know what's under this label and label should be intuitive to lead people to data. And if they're looking at it and they're like, this makes no sense to me. Then you know that that label is not going to work and let's say that they put it somewhere and it really does not make sense there you know that they are really confused about it. And you may have to go through and rethink the label. I mean so it's not just one and done, like Alicia said it's it's iterative you may have to do this a couple of times to get things right. And I recommend doing it with small groups or even separate individuals so my one of my paper based card sorts was groups of two, and I had five groups of two. Remember when I did my online one it was, it was 15. So it was three groups of five individuals so like one group of five did it. Did some changes one group of five did it did some changes so it's it's keeping it small is best because then you'll be able to keep track of the discussion. And then just if you did the paper based prepare a spreadsheet of what data you got out of it so that you have all that data ready to go. Like from your pictures if you took pictures of it, or download it if you did an online one and start looking at your data that's what you were there for and what you wanted to get so hopefully it makes sense and if you're confused by it. You might want to especially if you did online on moderated you might want to figure out if there's a way to do it online and moderated so that you can hear that that talking. And there are some online ones that let you track where cards removed. Autimal sort didn't allow that but other ones that we looked at that cost a little more like, you could see if someone put a card and X slot and then moved it to why slot. So there, there were some that have additional tracking so just kind of look at your budget. See what works for you, like we said it's, it's drive through so what works for you at this moment and fulfills your needs. Go with it and do it the best of your ability. And finally we have something that's called gorilla testing it's also called public testing hallway testing on the fly testing. And that is where you set up shop somewhere, and you find some way to get people to come to you and do the testing. Now this doesn't that's does need to be in person of course because you're just setting up in a traffic area. A lot of universities have been doing these over the past few years, and they're setting them up like cafes. So like if possible offer coffee by water snacks tea for their time and it also helps draw people in coffee especially a lot of these are tiny cafe user cafe us cafe. So collision I were talking and this is the cafe experience they get to come in, you get to sit down a little table, and they get to have a nice little, you know, quiet conversation with somebody. While they're, you know, sipping their coffee, or they get to take their coffee to go after they've done that you know. So recommend food as a bribery for things anything like that yes. Yeah, we were looking at gift cards or something and we were like, Well heck we could just do granola bars. It doesn't matter if they can eat it. They'll eat it chips. I mean, I'm always surprised when people have chips in the morning at morning things for students and the students are just like taking away Cheetos and eating on the morning and it's free. It's what matters. They'll eat it anywhere anytime just give them food and and coffee is is big. I so many people come in so many students just want coffee all the time. They don't even, they don't even care what it tastes like they just want coffee. One thing that you'll definitely want to do is create very specific scenarios. So this is kind of like what I was talking about when you're looking at other people's test out your scenarios and other people's websites. Because you want to put them in the situation where they feel like they are going to do a task that they would naturally do. So if you're saying something like hey do you know how to find your subject librarians you don't say do you know how to use your subject library. You give them a little scenario like if you were in a class for your major area and your professor said that you needed to meet with a librarian about a paper or a topic. How would you find the library that you need to meet with. And so you see I'm kind of very careful and that's not a perfect one but that's just kind of a off the top of my head type one. You're not using the word subject librarian to lead them to that label. You're not you're giving them a scenario where they might find themselves in real life you're not saying how do you find your librarian. Or how do you find the librarian for your major area you're making them think if I were in this situation and I had to do this. How would I do it. And so you really want to make sure that it seems natural to them and not force. It's going to be the observer effect right where particularly when you're in person they know that you're standing there. They know that you're you know like watching them and sometimes. I've noticed that people think more when I'm watching them like if if you've ever observed students when they don't know that like they're not in user testing and then just be like click that's the wrong thing back click that's the wrong thing in a in a user testing situation they usually go they read everything carefully like well because you want them to think aloud again. Maybe this one or maybe that one and a couple of times I've had students just act more naturally where they just click and go up that was wrong and they hit the back button click. Oh yeah that okay yeah here it is, which is probably more true to life because I'm sure there are some that agonize over every click but not as many as you right answer. Yes, right, just like go to the eye doctor. It's really important that you say that because that's one thing that I've always there is no right answer. They're not being tested. My site is being tested, and if my site if if whatever they do they can't get to it. My site failed, not them. I want you to try and break it I want you to tell me what's not working that's the whole point of this. I'm like, use it and break it please because if you break it I know what isn't going to work, or if something doesn't make sense or if there's a dead link I want to know that before it goes live right. So please just be testing it and using it and it's not you that's being tested if you can't find something. Even when the site is live if you can't find something it's my fault right I'm the. I'm the person creating this website. It's not your fault that something isn't easily found. So that's really important to reassure them on that point. You also want to train going back to the steps train your note taker and or facilitator I prefer to have two people one person who is kind of the facilitator who can say like if they stop thinking out loud, you're just like, can you can you tell us what you're thinking, you know kind of lead them into thinking out loud without leaving them on the site. And then also your note taker would be someone who talks about that keeps notes of where they went what they clicked and whether it succeeded or not. And again it's not a failure because it's not a failure on their part. So one thing that I've done is I don't write fail or failure if if we're doing the note taking with people because then they failed if they see that, and it's not so it's just you know, I use it like an s or an n succeeded or not. And then they don't know, you know, exactly what that means. Unless they ask and then I'll tell them like, oh, it didn't succeed, but it's not a failure on your part. And then just set up somewhere where there's a lot of people not overwhelmingly because you don't want to create a bottleneck and test one person at a time and no matter what test you do you don't want to do more than five to 10 or five to 15 minutes at a time. So if you have like three scenarios that are five minutes each, that's fine. But keep in keep in mind that at the end of the five, even if they haven't succeeded, you should move on to the next thing. And that's a that's a point of knowledge for you that this student tried for five minutes and never succeeded. So you need to rethink about, you know, think about how people are are moving through the site. And that's, it's actually a good thing, because it helps you make it better. If something fails, and you want things to be as good as possible. Kalisha, do you have anything to add as we I did have one thought that as far as web goes there is kind of I've seen this pop up a little bit here and there some kind of form of gorilla on the fly testing, which is where and it would be kind of virtual or distance and the website will just pop up like hey do you have a minute to fill out this one question, answer one question, you know, or something like that and so you can kind of get a taste of that gorilla testing on the web. So yeah, there, there is some virtual way of doing it but I, I have no idea how successful that kind of testing is because I know I don't particularly like having any pop ups or any questions when I'm trying to complete a task on something already. So if you wanted to do something like that, you would, it would, it wouldn't be as as much like gorilla testing, unless you did drop in, which I don't know about you but I don't know very many students where if I put it out even across all our social media everywhere and was like hey drop in and take some user testing online. I can't give them coffee I can't you know through the computer. One way that I got people for our card store or virtual card sort was gift cards. So that could be an option we had gift cards to targets, or to the local coffee shop across the street from the library. So there are some options with that but you would have to do by appointment I think, and have some kind of incentive. That's true people's time is valuable and they are helping you with your service or your product and or your project. And yeah they deserve some kind of compensation for that and virtually or with that random pop up I have no idea how what you would do with that. They get points that they can redeem. Does anybody have any other questions I know. Lisa and I talked a lot about what to include because there are so many types of user testing. So much more we can do with accessibility. But we wanted to focus more on a few things that have been most helpful to us. So if you have questions about anything else or we can try to answer them. Yeah, I'm sure if anybody doesn't have any questions or comments or anything you've tried out at your library that you want to share about type into the question section. We are a little bit after 11 o'clock but that's okay we started a little bit after 10am central time just want to let you know. If you do have questions will go as long as that takes what we will not be cut off because we're near the end of our official official time, and we'll go as long as if you have anything want to know. Nobody has any questions during. We didn't have someone who just said that hey they love coffee too so sort of anything like that yeah. I think I think this is great I'm so glad that I was able to get both talk about this yeah see coffee I've got mine. user users testing and usability is has been a submit I'm very adamant about that libraries need to do too many times I think they're going in just what you said earlier assuming I we know what we need to put out to the people that use our library so we're just going to do it, and they don't go back and look at and evaluate I tried it you know teach our libraries this is something to do. Look at your community needs for public libraries look at your, your page needs and you know, it's going to be a back and forth there's going to be changes nothing going to be in stone. So I think it's very importantly of all these resources and there is a lot to go through here definitely so I think hopefully that people have gathered a lot of resources and tips and ideas. And, like you said earlier to figure out which one's going to work best for you at your library with your staffing that you have or your budget. Lots of ways of doing this has been records explained here today. And I think not all it's going to be for you, and that's okay. You use the ones that work best for you. Yeah. And when you're when you're creating stuff this is something collision I have also talked about. And I think it's in the nature of people and librarianship to put they want they want all their knowledge to go out. Like, I want them to know all of these things. And we talked about a little bit like it doesn't matter what you're creating if you're creating a hand sheet or a handout or worksheets, a web page, it's, you need to focus on the need to know type of stuff instead of pouring all your knowledge out because on the user end, that can be really overwhelming. And that's something that I've seen time and again when talking to users is, there's just so much and I know that was one of the things with the web redesign. What the UND was doing is making things easier to find by kind of this is going to sound chunking it up right like this is this is a section on this page, it's easy to find people are used to scrolling now like on their mobile devices. So it does kind of make it easier to find things I think on the UND website by breaking them out into little digestible pieces. And then to where they need to go instead of just throwing everything in their face and seeing what sticks. There's so many things that libraries offer and that university is offered that you need to somehow break it down that way you can have everything right out there. Same thing with us as a state library agency, we as I guess at the beginning we serve all types of libraries and so many different things that we do for them. And it's been a struggle to what goes where how do we get them to the right thing where should it fall under what would make sense. Yeah. All right well it doesn't look anybody had any desperate questions that they typed in right now that's okay. We got a lot of information here so that's awesome. So there is Jessica and Clecia's email addresses there so please do reach out to them if you do have any other questions that you do come up with as you are trying to do this yourself. So, thank you everybody for being here I'm going to pull presenter control to me and get my screen up here again. There we go. All right, so thank you guys for being with me to your time so that we will get you on today and thank you everyone for attending. I wanted to yesterday the show has been recorded and I'm going to go back here to our main and compass live website if you do Google and compass live or use your search engine of choice. We're the only thing called that on the internet so far. So nobody else can call themselves this and you'll find our pages. But here on our upcoming shows we have listed but right underneath that is our archives and if you go to our archives page, you can see all of our previous shows. Today's will be at the top of the list. I'll also mention here, previously Jessica was on the show taming of the site helping users find what they need where they expect it. So that is kind of a companion piece to this one. And also following right after that you're talking about accessibility we actually happen to have a session on that creating accessible materials for library instruction. So if you're looking for a lot more. It's a one that totally is all about accessibility that session would be good to look into as well. She gave a lot of things are specific to my instruction but could definitely be expanded anything you're doing that's accessibility related. We do I also mentioned you have a search feature here we can search our archives you can search the entire show archives or just most recent 12 months. Because this is our full archives not going to scroll all the way down but and come a slide premiered in January 2009, and we have all of our shows here so it's over 10 years worth yes it's a long long page. Don't try and scroll through all of it use the search feature there at the top, but pay attention to the original broadcast date of anything. Everything is a date on it so you know when it first was done. The topics will stand the test of time you reading lists things like that, but some things may become outdated information maybe have changed over the years about a certain service or our or topic. Some things might not exist anymore links may be broken and URLs might not work anymore. But we are librarians we archive things for historical purposes we'll always have our full history archives on there. So just pay attention as you are going through. And if you do want to watch any of our archives there. Like I said today show be at the top of this here should be ready with the archive and the slides by the end of this week. I will email everyone who attended today and everyone who pre registered for today show to let you know when the recording is ready. Jessica would or please either one of you guys can email me your slides I think last time you had a link to Google. I sent both I sent a PDF and the and the link just in case. Yeah, you can send me both of those and I think the previous one I did. Yeah, I linked your, your Google docs yeah. For that one. So, send me that when you get a chance and we'll have that added into there. We will also post out to our various social media you're mentioning earlier about hashtags and whatnot we do use Twitter. If you are a Facebook user give us a like over on Facebook we have a links to their off of our main page. We do reminders reminder to log into today's show information our speakers when our recordings are ready. So we do try and update things here on our Facebook page as well, or if you're out on other social media, Twitter, Instagram we use those, excuse me and come live is the hashtag for our show. So that will be for our future shows. I hope you join us next week when our topic is pretty sweet tech teaching kids machine learning using scratch programming. I've heard about scratch a lot. Pretty sweet tech is our monthly show where our technology innovation librarian Amanda sweet comes on. It's a pretty sweet tech she's got a brand to there and talks about anything tech related. So if you are the techy type person at your library. This is definitely the show to keep an eye on she. It's the last Wednesday of every month we've got a schedule for unless there's some weird scheduling issues. You guys look for that and next week show will be about learning scratch programming and how you can work that into your library. So please do set up for that I've started working on our shows for September so keep an eye on our schedule there we'll get more as I get things finalize you'll see more of the what our upcoming topics are. So other than that, thank you, Jessica English ever coming here with us today. It was great having you on the show. Thank you everyone for attending. And hopefully we'll see you on a future episode of encompass live. Bye bye.