 Hello, good afternoon. Happy Wednesday afternoon everyone. I'd like to start very soon here with my session user testing, your Drupal administration process. I just wanted to get started by introducing myself and letting you know a little bit about my background. My name is Vanessa Turk and I'm a usability professional and information architect. And I started off my Drupal career actually creating content in Drupal 4.6 for a community type portal for young women who've had cancer. After some experience doing that, I got a job training people with new Drupal sites, showing them how to use their site. And through that I started to get to know information architecture and asked some really interesting questions of my developers that I was working with of how I could make things easier for some of my clients that had a pretty steep learning curve when they were starting to learn how to use their Drupal sites. Here's my presentation outline. All seven sections are not equal. A couple of them go by pretty quick. But I also wanted to get an idea of who you are. How many of you are usability folks and who have done usability studies in the past six months or so? Great. That's wonderful. And how many of you are not usability people or wouldn't consider yourself a usability person but would like to get into it or start doing some usability studies pretty soon? Perfect. Okay. Well, I hope you find this helpful. This information is... information that I've gathered over the last six or seven years of doing a little bit of this for all of my projects and a larger chunk of these studies recently as I get more into it. There are things that I find work for me but not everything always works for people. So afterwards please by all means come on up and tell me what's been working for you because I like to learn at these events even if I'm speaking just as much as when I'm going to sessions. All right. I'm going to get started then. Starting off with, you know, why study usability and why do we do this at all? And I think it comes down to the fact that we like to be efficient. We like to make things work better and be optimal with our processes because, I mean, frankly it saves money if we're struggling with things if we're wasting time, if we're doing things over again and if we have to go and ask people for things over and over again we're wasting money. So a lot of the time these things happen because we're trying to make things better in order to save ourselves time and money. So when I'm approached to do usability study the first thing I want to know is, okay, what are we studying in terms of a website tool and application? What does it do? Just really on a high level is it a brand presence? Is it some sort of task application? Does it have e-commerce? If you sent me a URL are there things behind a login that I don't see that I'm not really aware of that's there until I get under the hood and have a look around? Then I want to know a little bit about the organizational focus because when I'm looking at the site it may be that it's a brand promotion site but they really want to do e-commerce or there's a little tiny piece that's e-commerce and it's not very successful and I really want to focus on that. I'd like to know if the organizational focus is on programs and services or is it on the community or is it on some very key high profile people whose branding, whose voice, whose content production really need a focus? Then I want to find out a little bit about the organization. It might be that I'm talking to the IT department or the marketing department or the communications department or maybe dozens of other departments involved, some of them in the web administration process or the content creation process that I haven't been talking to yet. So to get an idea of what the company actually or the organization actually looks like from the top to the bottom. And then I want to know where they're at in terms of their organizational site admin. Is there a single person who's everybody's sending documents and PDFs and information for them to manage the site? Are there a group of editors? Are there sectional content managers like a faculty or department? And do they already have any kind of access restrictions and rules that they're trying to follow? I also want to know if their content ever comes from submissions or if there are members of the site or members of the public who are submitting content in some shape or form that will find its way onto the site. So once I know that, then I want to know okay, who's all on the site? Who's actually involved in this? Sometimes the policy and decision makers don't even have access to the site and they're not creating content. They're just having a look at it and having some meetings and giving feedback. Sometimes they're the person who run the show and they're the ones who actually manage all the content. They're subject matter experts or maybe a single site and these would be people who are in an educational institution or faculty and they're creating content to describe events or resources that people have. Content moderators who might have a role to correct voice and tone, grammar, facts or even just formatting content that people create for the site. And then there might be people who simply report on something. They create a blog or an article or they collect some information about events and they don't really have knowledge about the organization as a whole. They're just kind of working as writers. And then there might be some content custodians and these might be people who are tasked with just freshening up content, looking for spelling errors, removing content that's inappropriate. Just something along the sidelines where they don't fit into any of these other numbers here. The content consumers I've included these because these are people who may be members of the public, but they're contributing in a way because they're actually commenting or flagging or favoriting some content on the site. So really you could have a lot of people in the organization that you need to talk to. People from marketing and the communications department. It's very helpful if it's some kind of transactional site to talk to people in customer service to see what kind of questions they're getting and what kind of communications they're having with the rest of the team because often they'll be right in the middle. But I find that when this happens they're usually at a certain place in the process of improving the site. They're either moving from a system that isn't Drupal because I specialize in Drupal into Drupal or they're moving from something like just a flat HTML site where they're starting to understand what a content management system is, but they're not fully comprehending it. Or they're already having a Drupal site. They think they're doing okay with things they're just refreshing and sometimes when I do a usability audit we find a lot of inefficiencies that we can actually target next time we do the refresh and redo architecture to be a little bit more efficient for them. And now before I move forward to note about feedback I've seen usability surveys where people are asked very subjective questions how much they enjoy the site or how much they like the process or what's their favorite part about the site and I think that this can be helpful if you're asking a lot of people but probably not as effective if you're asking administrators of the site. Because right now if I were to get you to rate your shoes for example you're going to need a little bit more information from me if I was just going to ask you tell me on a scale of one to five how much you like your shoes and that might confuse a few people because you might want to know color or comfort or what else is there that I'm trying to find out and this is exactly the point I'm trying to make with this it's much more effective if we actually go deep into the weeds and identify some of the things that we want to know before we start asking them to comment especially in that subjective way So moving on to actually learning how to organize a usability study we want to find out the purpose of the site start to segment our audience groups find out the purpose of the study usually they're coming to you with some sort of a question they have an answer that they need to find a solution they're looking for and then the goal of the study might be to find that solution but to also find some other things on the periphery that people may mention during the study and then there's usually a list of suspected issues and an initial list of contacts because you'll always have a few people that you've been told you simply must talk to because they're just a wealth of wisdom about the site and then starting to understand how different roles may interact with the site you're going to encounter different sort of people people that are looking to the site for information so these might be people on an intranet people who are using a site to complete some sort of internal task or process and then people who are creating content and once you're able to start identifying those different segments and applying the different processes that they are responsible for you'll start to see things like these are the people that are responsible for product configuration or tool setup and this person runs the sign up and registration and some of these people will overlap and some people will you might find are not even in the correct role for what they've been asked to do you'll want to go a little bit deeper into those tasks and understand what they're actually made up of on a granular level what are they actually doing when they're creating pieces of content and managing areas of the site are they curating things are they submitting and archiving are they moving information around somehow so if you get a little bit of an idea of what's actually involved then you can start to create scenarios and scenarios are usually made up of the audience member the segment that you're studying what they would like to do and why they're doing it because there's usually some kind of value involved and this is an interesting piece of it because sometimes the value is not really very helpful for what they actually want to do the value may be the queue that gives you a way to find a better way to do something because they might be wanting to do something in a certain way just because they've always done it that way and that is the danger from moving to from a existing kind of proprietary system into Drupal where things may work a little bit differently and so they're asking I really need to do this particular thing but the reason that they're asking to do that is a value that we can actually accomplish in a completely different way for them so if you're not familiar with user stories this is the template this is the script that you can create that will help you define these stories for the different types of users so as a store administrator I want to change prices so that I can create a sale or something like that it's very important to understand the publication workflow and I want to caveat that by saying that the publication workflow in real life may be completely different than the one on the site and sometimes people try to get a little overkill on the site because they want to lock down their publication workflow and they end up making things a little bit more complicated than they need to be a really good way to uncover the content management needs is to use the seven C's which are conception where does content come from does the marketing manager decide that something needs to be written press release say graphical element decides that new content is needed and are there segments to that are different decision makers responsible for different types of content then who writes the content who sources images who creates the media for it and how do these pieces all come together how is it organized if I'm creating a particular kind of content and it's got many different facets attached to it who decides which tags go with which content then how is it confirmed the publishing workflow doesn't need a process where I'm sending it to an editor sending it back to subject matter expert who needs to confirm that it's actually correct and then if things change how are the changes tracked is there a revisioning system and how does it work and if we don't need a revisioning system and things need to be archived or go do they stay on the site or is there a place where they stay and then sort of any kind of community involvement is there social sharing is there commenting are there feeds where the site goes into different areas and anything special about the consumption are there access control rules are there certain fields that aren't able to be accessed by certain people are there inside notes for administrators any special analytics or tracking that happens as people move through the site and consume the content so as you go through this you'll start to pull out little notes for yourself things that you'll need to check when you create a script for some of these testers to go through you'll want to see if they are adding a huge amount of content are they able to batch are people checking each other's work do they need to check each other's work and are there different ways of categorizing the content so that some of it is internal and some of it is external or does it need to move from being internal to external and one of my personal favorites naming conventions because sometimes it's not a technological issue so this is an example of a testing plan that I had for a client where this is a set of fruit and vegetables and they had internal people that they would like wanted to have tested their staff they had wholesaler food service people and the community that remembers of their site and so they gave me lists of people their time zones and we created some questions and an interview date and then we put together a list of questions that were to me they're not written in stone they're asking me to ask certain questions in a certain way and be able to move back and forth between questions to see if I can engage them in conversation there are certain tasks that I would ask them to do but other things that are just sort of conversational that I want to know how they feel about so usability testing can be done remote or it can be done in person I've done all kinds of screen captures of people who are further away from me every tool imaginable I think it's just really important to have backup because sometimes tools don't work I've phoned Chili in Venezuela for different kinds of interviews that I've done and I've also just gone to local cafes to meet people and sit down with my computer and use something like Silverback where I'm able to talk with them and have them go through the scenarios that I've set up for them you'll want to explain a little bit about yourself and your role I've sent a sort of a very brief email that they're improving the site and somebody wants to talk to them and they're not even sure who you are and what you do and sometimes they're not really familiar with usability and they have to be told that they're not being tested it's the content, the site, the workflow that's being tested sometimes the information that you've been given about them is not fully accurate you might need to ask them to explain what they do in their own words the things that might be responsible for or it might not fully make sense what they've been asked to do with their role and then you'll go ahead and ask them to walk through their typical use of the site and complete several tasks and then also there might be things that come up and they usually want to be able to give you some additional information of how they feel about the site so I usually allow 25 to 45 minutes some people talk longer and so schedule a test every hour things to watch for clicking between lots of screens cheat sheets if they're not able to do what they need to do on the site and they're using other sort of supporting documentation and using external tools for things so an example of this might be there is a there's bookstore that on their ordering system their sections of the site were just numbered and there were people who had to refer to another sheet to see what the numbers symbolized so when we test we're actually also reviewing some of their information architecture components because we want to know if the organizational systems labeling navigation and search systems are working for them if they perhaps need to reorganize their information architecture to be a little bit more effective both for internal and external use so this is a fun example if somebody is looking for a particular piece of content or resource internally has it been tagged appropriately in the way that it might be searched for by several different people a more realistic example might be something like this so this is an HR form the human resource manager would know exactly what form it was but somebody else may call it something different holiday or vacation or application so those are the sort of things that you may need to ask people to do the same task in different ways and also ask for feedback on if they ever get help questions if people can't find certain things what are the things that they're not finding then look at those pieces of content and see if they might need to be labeled in a different way another good thing is to look at how files are organized if people are able to find what they need to find in their own company or organizational because if that's not in order then it's very difficult for them to find the kind of information that they need or the files and assets that they need to create the content they're looking for one of my favorite things is a naming convention naming conventions can be extremely helpful when trying to find reusable assets images and certain kinds of documents PDFs forms things like that if they're labeled differently and different people are looking for them it's very difficult to find so also that is one of those non-technological solutions in real world labels can be created by different people because different people think about words in different ways and this is just something that's come up in many different situations for example I did work with money exchange currency exchange where every location was called slightly differently and it took me a little while to realize that they were all the same all of these ways of referring to their location wasn't any different they were all the same kind of store a really interesting thing to look at is if they have any default instructions or help text that might be a little bit confusing because maybe the help text was created by a development team that didn't fully understand their content creation process perhaps the site was created by people in a different country or different language and maybe the help text doesn't quite make as much sense as you think it should so it's interesting to see if some people, if new users of power users are using the site a little bit differently when you get to the point where you're documenting and reporting you're going to want to review your notes and try to find ways of grouping information into categories and multiple categories are fine because there's a lot of things are going to repeat from section to section so often I'll go through and I'll comb through and I'll try to group things into users, features, tasks until I come up with a list that has and it's okay if things are across multiple lists that has a little bit of order to and it's not always the same sort of structure then I use a type of heuristic checklist to go through and identify what these kind of issues actually are and there's lots of different templates online that you can use to identify best practice and you might use different ones depending on what kind of site it is so it's so for example this is one of my favorite ones Jacob Nielsen's 10 and this is probably a list that you've seen many times but it's kind of when you think about it does apply to many different areas of the site and it's something very very helpful for if you're taking your report up to a group of stakeholders who may want to have a little bit of supporting documentation around it so rather than just giving them a list of stuff you have to change if you're in a situation where they're asking you to support that recommendation with some evidence this is the kind of evidence from research well documented research that has been around for many years so something like a visibility of system status this is something that comes up sometimes and you'll know this will manifest itself if there's been something perhaps custom created some kind of workflow where an administrator pushes a button and then they're not sure if they've published something or they're not sure if they've affected something so then they have to go and check and see if something has been changed so this is something that you might see if the developer or the person designing the interface didn't create some sort of system status matching between the real world and the system so this is if I were to refer back to the bookstore people the different sections of the site on the e-commerce site didn't match the different sections of the bookstore and they had their reasons for that but it was there a solution was to try to put both of those sections together so if you were a user of the site you could see both of them beside each other and so you did have that real world match if you needed it and it was very helpful for some of the newer employees to see for user control and freedom this is a navigation there will be a lot of situations where there's navigation within the administrative interface that might be created in a default out-of-the-box way that could be improved upon because there's people that are doing repetitive tasks in certain areas of the site that may take two or three or four steps to get that piece of content and sometimes it's just easiest to make a quick link for them because they're trying to access the content frequently again with the consistency and standards there's I think one of the most common ways that this is a problem is when they've created the site in kind of a technical way where the IT department has applied labels from their best understanding of the business of the organization but it's not really words that the rest of the organization uses so the best way to do that is to go through and comb through all the words and labels and see what people are calling things to see if there's any kind of inconsistency there because that definitely is a place where people get confused especially when they go in and rename it and then there's two of them error prevention is a good one because sometimes people do things and they're not sure what they've done wrong because they're not getting feedback on it so there are ways of designing certain kind of interfaces to help people not make errors that are really common so if you see people always you know several of the testers making certain kinds of errors then it might be a good place to target for improvement with recognition rather than recall some there are sometimes there's categories long long lists you know 50 plus keywords for a specific kind of resources and a nice way to help people find those keywords they don't start creating new keywords because they can't find what they need is to have them start to fill in by themselves so that auto complete so they're able to locate the keyword quickly and easily that is a kind of accelerator so there's other accelerators that people could use something like a little quick link to a particular area of the site where they need to edit something a lot of the time this sort of thing will present itself and you'll go oh okay right this is one of these issues and that'll help you start to create a list sometimes the administrative interface has some sort of default theme that is a little bit jumbly and it could be as simple as just switching to a cleaner theme it could be that they're using an older one or they just didn't realize that something really nice and clean was available and if they do switch it's a little disorienting but sometimes it's easier to use something that's a little bit cleaner and then with errors we all know that sometimes errors are extremely alarming to some administrative users and the errors aren't really that urgent at all and then there are others that are quite urgent so this particular one for me is a bit more of a training issue to help people understand what is actually a problem and what is just kind of an informational message for them so when it comes to help and documentation occasionally the issue is that people just haven't been educated and they haven't been trained on the site adequately they may have been thrown into their role maybe people have already been trained on the site they're coming in late and they didn't realize or they didn't realize the benefits of a formal training program or maybe it was never designed for them so sometimes the site lacks documentation it lacks help and the solution is not even a usability issue it's actually getting together with the staff and retraining on the site so there's different ways of categorizing different issues if you are creating a formal kind of report you can create something where you're dividing things into feature disclosure, lack of affordance and indication and an obscure mental model in a blocking mode or state a feature disclosure is quite just as simple as that the user can't find what they need to complete a task it's either hidden or they're not recognizing it and they're not able to navigate the menus they're expecting something to be in one place and it's actually somewhere else lack of affordance is that there's a link but nobody can tell that it's clickable or can't tell that something's enabled and they didn't know that a certain element was something that they could use in a particular way so that's often the most difficult thing to do when you're sitting beside somebody who's struggling with something and they're looking to you for help and you're trying to give them just enough of a hint so that you can go on with your test and obscure mental model is when they just don't they make incorrect assumptions about how something would work and or it doesn't match how they need to use the site or the help instructions are misinterpreted completely occasionally there's blocking modes where the site is not in a state where people can edit that they need to edit and they find themselves in a section area of the site or an interface where they think they can complete something and it doesn't happen and this is often a place where people they send in help tickets I tried to do this and it didn't work and sometimes it's very difficult to reproduce because they've placed themselves into a situation somehow where it's a little confusing how they even got there so this can be one of the most difficult ones to actually investigate so you may need to go back and do it a few more times and in the end sometimes things are just sometimes things are just not reproducible which is actually a good thing because then it's probably not going to happen again just a note on creating reports I always create reports with screenshots and annotated just kind of helpful hints about what I think would be useful so you can see in this particular picture it's a store where the order number is it's not in the order of the of the numbers so six is all together rather than actually being the order of the numbers themselves and there's also certain kinds of pieces of information on the interface that aren't required by the staff so it's as simple as kind of going in and saying okay well when you look at this page what kind of information do you need and what sort of information do you not need is there something missing where do you need to look to find the information once you have a list of all the user stories and the features and you've tried to explain what the issues are you've grouped them and you've tried to give solutions then it's time to meet with the development team I like to talk about what happened and what I need rather than how things should be solved because I feel like there's such a group of talented developers out there who are much more creative than I am in being able to suggest solutions I just want to be really really clear on what it is that I'm actually looking for and I think that's actually a lot more helpful so when development planning it's really great to have a big brainstorming session where you're understanding what those needs are and then prioritizing by level of effort and business value and if choosing solutions is an issue there's definitely a lot of different ways that you can go about refining and sorting through to see if it's a budget issue or if it's a capacity issue what sort of what would be the best way to actually decide what kind of changes to make so now we're at a place where I've created a few checklists that you can use and I'm just going to go through them with you here and then we'll have a short period of time for questions so once again planning for the study the purpose of the site is clear you've done your user segmentation the purpose of the goal of the study has been documented and you have any kind of suspected issues that people have brought forward and a preliminary contact list so this might look like this example here that there is a content source for educators they're interested in resources for teaching here are the list of users that they have so far that are administrating the site they're looking for ways to streamline content creation and publication workflow and they just want to identify some tools and processes to help them be more effective sometimes they come to you with suspected issues and they're absolutely bang on there and we're just reinforcing the fact that they realize that there's an issue and they need to do something about it but they're not sure what yet and sometimes the issue is not the issue and there's a whole bunch of underlying things that need to be sorted out I think it's really important also to try to get some people that it might not be the ones with the loudest voices involved because there's a lot of people on the sidelines that are vital to the company or organization that have some really important things to say maybe they're not as confident in speaking and coming forward but I do recommend trying your best to try to find everybody who might have something valuable to say or that has a really pivotal task on the site and try to talk to everybody even if it means getting up early or scheduling something on the weekend because it could be the nugget of information that you're looking for so the study checklist would be the updated content list complete list of people who you're going to be interviewing scheduling them it's really nice if you have somebody that they know that's scheduling them that you're not just calling them out of the blue because that way it's if you don't know them personally it's a little bit more difficult I think sometimes to create meetings with them sometimes you need some sort of consent form or nondisclosure agreement and then a planning spreadsheet either one that you make yourself or something that you download with all of your testing scenarios survey questions and etc a lot of people have favorite tools for screen capturing for surveys and for having meetings and then a document of some kind this is probably something that needs to be created so if you're interested you can use the users or the different sections of the site as the container as the different organizational pieces but I think it's very helpful to have a summary on the very top because some of the higher level stakeholders are just about they're just about reading a brief on it and then sort of any analysis tools that you want to be using checklists, guidelines books that you've read and then for the reports you've got a summary of goals summary of issues and a summary of recommendations so you'll have them organized by the types of content by tools, by process, by priority and then any kind of annotated screenshots video clips and quotes quotes can be very very helpful if you've had someone say something that's just wow that's fantastic either attributed to them or with initials and then recommendations if you have some organized again by type by section and then any next steps because I think sometimes you've got stakeholders they're fully bought in all they want to do is how do we get started they just want to know what you do their next thing once you hit the ground running so any kind of additional test or studies would be important to note at this date so if you want to come back and test again in six months you would like to group together another set of users to test that's also something helpful to include and then when it comes to development planning you want to make sure that your issues are mapped to user stories and that your user stories are kind of mapped to different sections and features of the site group together and prioritize and priorities that can change depending on who you're talking to of course if somebody is an events coordinator then the events section is going to be priority so it's really important to have somebody who's some kind of product owner somebody of some position a vision where they can make those kind of difficult decisions on an organizational level of what's the important thing for their website how the website what role that it plays in their organization and now it's time for questions I have a microphone in the middle of the room and I've been encouraged to use that so that the questions can be recorded and heard by all oh you're so exciting I know it's at the end of the day and it's the middle of a long week so I wanted to also play you a fun video and if you don't have any questions I'm going to tell you that this is a video that is made with my friend Ann and it's a video about working from home in the usability industry so let me just get out of this Vanessa I just got off the phone with our client and we have a problem they have a lot of click here all over their homepage in bold and red and still users can't find out where to go what are we going to do? the site lacks visual focus I think what we need is a great big action button that's exactly what we should do put a button on it Vanessa on this site membership registration is down I think it might be a usability issue walk me through the registration process is there a sign up? well this is a graphical element but no not really okay well that's the problem we need a button yes let's put a button on it this is a social media integrated enabled site but people are not sharing well there's the problem the like button is tiny what if every time you liked it it would get bigger like a cloud tab yeah put a button on it we've got a situation our client calls they want a mega mega menu that is integrated search okay well I think we can just make a big button in the shape of a magnifying glass on it yes let's put a button on it so we have this situation the support button is constantly being overlooked people just keep calling the support button needs a help button yeah let's put a button on it I got a call from Ray the CEO he's really quite concerned he wants to see more web 2.0 elements on this flat design sure let's just put this great big glossy jiggly button that pops on it yes let's put a button on it how are the users going to find the categories in a hierarchical structure interface we could use buttons for the facets yes let's put a button on it audience segmentation 15 users each with 12 sub users okay um dashboard with button button button button yeah let's put a button on it bounce rate reduction what are we going to do to keep people on the site maybe they're leaving by accident what we should do is put up a modal window where they have to press a button to confirm that they want to leave cancel for yes and go for no yes let's put a button on it the client wants a single place to publish content to multiple sites what do you think we should do oh they just need a syndication deploy button on the admin interface yes that's it we just have to put a button on it the marketing team is looking for ideas for subliminal persuasion for their whole action we could use a white button on white background with a white text what a great solution put a button on it we just got in this RFP they're asking for all kinds of things parallax, pop, digitized, monetized tag clout, NED reference deployment pipeline, ooh high performance what are we going to put down proposal put a button on it Ann it's really that simple we need to jump on a conference call right now can we use Skype? Skype is down, Microsoft is down what about Google Hangouts? they don't do the Google what about FaceTime? I don't think they're Mac users WebEx? that never works for me go to meeting? that one either download join me hello hello it's only I had a f***ing button thanks everybody have a good evening do come up and talk to me