 Okay, it's 10.45, so I'm going to get started. Good morning, everybody. How are you feeling? Yeah, whoo! Okay, so I want to start off with a little bit of background about my company, Digital Loom. We're a full service agency in Cambridge, Massachusetts. We work exclusively in Drupal and have since the year 2006. Most of our work is for higher education and non-profit clients, and I founded the company in 2000 as a loan freelancer working out of my apartment in my sweatpants and didn't really know that it was going to turn into a company, but it did. So one sort of goal that I had early on and probably my main goal in starting the company was to take great care of people. So at first that was my clients, and over time as I hired staff, it became my coworkers, and I felt like first and foremost that was our mission. Second to that was we had to make the web a better place. And I felt like that would naturally flow from taking care of people because if we're making things easier for our clients and for their users, then the web is going to be more beautiful and intuitive and engaging and dynamic and all those good things. And of course being an agency, we have to make money too, but I felt like if we do number one and two well, hopefully we would make money and the money would give us the fuel to go back and do number one and two over and over again. And so far that has been the case, which is great. So a big turning point for the company was in 2006, not only was it then that I officially incorporated and started hiring our first staff, it's also when I was introduced to Drupal specifically Drupal 4.7. Does this bring back fond memories for any of you in the room? Okay, all right, good. A few of us. So my first reactions when I saw Drupal and started playing around with it, where I had kind of two at once, the first one was, wow this is awesome, this could be really useful for our clients. A lot of them had started asking about content management systems and Drupal and WordPress were kind of first kind of becoming mature at that point and I thought, wow there's a lot of potential here. And the second equal reaction was we can't give it to the clients like this. This is going to confuse the heck out of them, they're going to be really bewildered by the back end, so we need to fix this. And of course as you know the beauty of Drupal is we can make it do whatever we want it to do. It's very modular, very customizable. And so from the beginning we started finding contributed modules out there that could improve the back end experience. We also started writing a lot of our own customizations and getting it into a position where we could show it to clients without shame and hopefully let them have a good experience. And then over the years we really iterated on that so every time we trained a client in Drupal we would notice where they stumbled or where they were getting hung up and we go back to the drawing board and find or write more modules and get it all streamlined. So fast forward to the Dries note yesterday. So I was sitting there silently going, yes! Yes! Like to every point that he said because to me the overall message was we need to make Drupal better and more usable for all kinds of audiences. So he touched on a lot of different kinds of people who are going to touch Drupal websites in some way or use them in some way. So he mentioned the technical and non-technical evaluators. So if someone's coming from an IT background or a marketing background they might be evaluating Drupal for use in their organization. He talked a lot about improving the user experience for site builders. So those of us who are in there putting Drupal together making it do what we want for our clients or for our organizations. And then he also talked about improvement, a lot of improvements that the Drupal community is prioritizing for the kind of more like client side users of the site. So the site administrators, the content contributors and of course the end users who are going to be ultimately using the site or application. So I was pretty psyched about that because it's something I've cared about for a long time. So I want to take a moment and get a show of hands from you guys. When you think about your day to day work and the ways that you're using Drupal, which of these audiences are most important to you? Who are the people that you're serving? So who here feels like their work is primarily geared toward the technical or non-technical evaluators? A couple of hands, I figured that would be the minority. Who here feels their work is more focused on people who are building sites? So if you're creating modules or working on Drupal core, anything like that. Okay, a few hands. And who here feels their most geared toward building sites for this bottom row of users, the site administrators, contributors and end users? Okay, cool. Alright, so we're in that same vote too. Most of what I'm going to be talking to is geared toward those audiences, so we're good. However, all the techniques I'm going to talk about can be used for any usability testing that you might want to do. Okay, so the big question is here's our people, how can we take great care of them? And as Therese was talking about, how do we build excitement about Drupal by delivering a really great user experience for all of these folks? So, here's a concept. Get ready. What if we watch the actual users perform the tasks? Yeah, great concept. So that has a name and it's called usability testing and I'm here to tell you that it's way easier than you think. In fact, it's so easy that we're going to do it today in this session. You are all going to have a chance to try it out on each other. So, again, get ready. It's going to be awesome. Okay, so before we get into that, let me give you the rundown on how it works. So, I can break it down into pretty much four steps. So, first of all, you're going to write the tasks. So, what is it that you want people to achieve when they're on your site? Second step is you're going to round up some testers. Third, you're going to watch those testers perform the tasks you just wrote. And four, you're going to learn from their struggles. And I say learn from their struggles, but you might be pleasantly surprised that some things will go smoothly for them. And so, that's a place to learn too. If something isn't broken, don't fix it. Just leave it the way it is, but hopefully you'll see places where you can improve your tasks as well. Okay, so let's start by talking about writing tasks. So, there's kind of an art to it and I'm going to give you a few tips. The general format that I like to use for writing tasks is something like you are an X in situation Y and you want to do Z. So, you're really trying to think of who is the person who's trying to complete this task and what's their motivation. So, you want your tasks to kind of capture that. And a few tips as you're thinking these through. Hopefully, before you build anything, you've thought about what its purpose is. What are the objectives that people are going to want to achieve or what do you want them to achieve? So, you want to make sure you have, you've figured out the most important objectives and that you have a task for each one. You want to keep them short because you're going to see when we try this people have to understand what they are and parse them so you don't want like a big long paragraph. And you want to be careful not to use the exact words that are on your site or in your face. So, for example, we do a ton of work in higher ed and higher ed often tends to have a lot of acronyms or like weird proprietary naming for things. So, they'll say things like, oh, see if you can download the on-site student visitor registration form. So, don't call it that. Call it, think about what's, how is a user going to think of it. So, you might say something like, oh, you're a student you want to visit campus. Figure out how. And then see if they can figure out what your crazy form is called. And then I also like to make users dig a little bit. So, if you're testing the front end of a website, don't have them find something that's right there on the homepage. Make, set it up so that, let's say you're having them look for a certain event that's coming up. Have them look for something that's maybe like, would require them to use the search or would have them like dig down a couple of pages. But don't make them dig down to like archived events from 2003. You know, keep it within reason, but make them dig a little. And when we do these tests, we usually do them in about 45 minute sessions. And we shoot for around 8 to 12 tasks. That's usually a reasonable amount that you can accomplish in that timeframe and not kind of burn out your users, your testers. Okay, so let's try that right now. So, I'm going to give you all two minutes to pick a website. It can be one that you've built or are working on or care about or just any old random website. And I'm going to have you write one usability testing task. And again, you are going to try this out on someone later. So, it can be whatever you want. So, two minutes starts now. Go. One minute left. I should have brought some dramatic theme music. Okay, how did that go? Would anybody like to share what they wrote down at the microphone? Ah, come on. Yeah, alright, that's the spirit. Figure out what compilers are available on X machine. I couldn't think of one of the machines anyway. One of the supercomputers that we have. Okay, awesome. Who else has one? This is cheating a little bit. Concert events in museums seem to have like the worst websites ever. So, find out what classical music events are playing in Edinburgh between May 27th and May 31st. Beautiful, I love it. Yeah, that's great. Yeah, I think that's almost verbatim a usability test I've done on the Institute of Contemporary Art in Boston because they had a very similar problem and wanted to make sure if people are visiting during a certain time frame, they can search for events within a range by what kind of events they're interested in. Can we get one more to share? Lodge a complaint that the bus that you are trying to get on was not able to be accessible. Mmm, great one. So, you're thinking of a usability test where you would what would be the task though? What would someone be trying to do? Find a certain bus that they want to get on? No, someone's angry because they tried to get on a transit bus and they couldn't get on there because the ramp wouldn't work and they needed to notify an authority about it. Okay. So, I think if I were going to translate that to a usability test, I would say angry because you couldn't get on the bus and you need to notify someone that the ramp did not work. Correct. How would you do that? Okay, great. Alright, those are awesome. So, you get the idea. So, again, you're getting in the mindset of the person who you're targeting writing a task that's kind of in their own words and then seeing how they do. Okay, so next step is of course you need to get some testers to try out your tasks for you. So, a few tips on that. So, the best thing to do is think about who are your current customers, members, clients, colleagues, whoever you think best fits the profile that you're testing. And so, if it's people already in your circle that you're already working with, that's great. You can just reach out to them. Of course, using your social media networks is a good way to meet people who want to volunteer to be testers for you. We've had good luck. So, when the first two fail and we can't organically find enough testers, we've actually had good luck posting it on job boards including Drupal.org or Indeed or Craigslist or any of those kinds of things. So, that's another avenue. And then there's paid services like usertesting.com that will help you recruit people within certain demographics. And if you're using one of these two methods like just posting out on job boards or usertesting.com, you might want to write a little questionnaire for people to fill out. And my experience is write the questionnaire such that they don't know who you're going for. So, say like, what's your current education level? What's your profession? That kind of thing. And then you can filter out people that you think are a good fit for your tester profile. Very important. Always offer an incentive to get volunteers. We typically will do like $50 Amazon gift cards for an hour of testing depending where you work and what your relationship is with your testers. For example, if you are a museum, you can offer like a free event or a free membership. If it's students, we'll sometimes just like $10 Starbucks gift card works or a t-shirt or something like that. It works fine. So, think about who your audience is and what they would enjoy. And we do a lot of our testing remotely using a web conference because we can see their face and we can see their mouse and what they're doing on screen so that works just as well as in-person testing. So that's just using a web conference, no fancy software. It can be, you know, whatever you like using. So, one question I get a lot is how many testers should you be getting? So, I have read experts saying anywhere, anything like, oh, five testers is all you need. That's going to help you smoke out, you know, 80% of the problems. And I've read others saying, oh, no, you really should get like 20 testers because, you know, like five isn't really statistically valid and you should really have a lot more. So, my experience is I think 8 to 10 is really the sweet spot. That's where you start. It's enough testers where you really start seeing patterns. But the important thing, if you forget everything else I've said, the important takeaway is make sure it's greater than zero. So, I think my goal is to make usability testing accessible to all of you. And it's not hard, it's not difficult. So if the idea of rounding up 10 people and doing an hour of testing with each is just so daunting or impossible to squeeze into your schedule, don't sweat it. Grab one or two people that you think are a good fit. You will learn a lot even if you just do it once. It's very very eye-opening and so I would encourage you do anything. Anything is better than nothing. Okay, so next we are going to line up some meetings with our testers. We're going to watch each one of them perform all the tasks that we've written. So this is the fun part. And here's where I'm going to ask a super brave volunteer to come up and be a usability tester for one task for me. Yay, come on up. And I promised an incentive so you're going to win a bona fide digital loom clear your cash t-shirt like this one. Yes. Okay, come on, you can come on up here. Yeah, come all the way up. Hello, what's your name? David. Hey, Victoria. Nice to meet you. Thanks for volunteering. Okay, so we have our tester David. He's going to help us out here. So when you start off the test, usually your testers are kind of nervous and they're worried that they're going to do something wrong. So your job is to just set them at ease. So first off I'm going to explain the process to David. I'm going to say, hey David, I've got a site that I'm working on. I want to see if it's usable. So I have one task that I'm going to give you to try and I want you to just do your best. Okay. It might go smoothly. It might go horribly. Whatever happens, we are testing the website. We're not testing you. Oh, that's good. So you can't fail. So I would tell him, okay, just try your best. It's okay to give up at any point if you want to. And I'm also going to ask David to think out loud as he's running the test. Because that way I can get a better sense of what's going through his mind, what's he looking for, what's he thinking about. So as you're doing the test just sort of give us a stream of consciousness. Alright. Any questions about what we're going to do? I'm going to try and find or do something. Right? Yes. You got it. And I like to record the sessions. That's not necessary, but just in case you want to go back and look at something or if, you know, in our case, our clients sometimes like to watch them, they're boring to go back and watch. So I wouldn't recommend it, but that can be helpful. Okay. So David, I'm going to show you a website in a moment for the Harvard Graduate School of Arts and Sciences. And I'm going to have you imagine you are a PhD student in the economics program. You want to find out the requirements for your dissertation and how it will be evaluated. Okay. Any questions? I probably should have some, but I don't. Okay. Great. That's the spirit. Okay. So I'm going to bring up our test website. Here it is. And I'm going to have you step up and try to complete that task. Alright. So I'm at the Harvard University site and I'm trying to just scroll down and see if there's anything down there. Oh, that's news. Oh, that's pretty cool. Social media, that has nothing to do with what I want to do. So I'm going to go up here and look at the programs of study and my degree programs and click on it. Oh, man. I have subject areas. Okay. Is there economics on this page? Probably should filter this, but I'm not going to. Oh, economics. Doctor of philosophy. Okay. That confused me. I didn't know that was what Ph.D. stood for. Alright. I probably should know that if I'm a Ph.D. Economics. Yes. I like them. And then there's all these texts and here's the require the special requirements. That's applying for it. That's the contact applying to more and one admissions requirements. Is that filling out a program requirements maybe? The first two years here are the courses. Here's the written field examination. Is that a dissertation? Maybe. Taken in two areas of concentration. Am I mid marching for the second year? Field exam, field exam. I guess I have no idea what any of this means. I should if I'm going to be a Ph.D. Dissertation, committee and research plan. Maybe this has got what I need. Part of the beginning of the spring semester of the third year, the students must assemble a dissertation committee consisting of at least two faculty members and must complete a preliminary research plan of at most five pages which is signed by the dissertation committee by the beginning of the fourth year. Maybe that's it. I'm not entirely sure. Oh, there's the dissertation committee. There's the time and place, date and time for the defense and special examination. Determined by the student after consulting with the dissertation committee date. I'm spending a lot of time reading up here. There's so much reading on the web. I didn't know they still did this. Where's my video? Look, this is 20 HD Ph.D. Okay, it should be Ph.D. and HD. Let's see. Date should allow me. Is that it? I really have no idea. So I'm going to pretend that maybe that had something to do, but you know, if I were a Ph.D. I'd probably be like, yeah, I did it or no, I need to find something else. So if I didn't find that, if that was not it, mission, FAQs, that doesn't seem to have anything to do with anything. There's the requirements for financial support. Three dissertation. These, there we go. Okay, I just missed it. Oh my, what? Again, I never want to be a Ph.D. Are my reading people's dissertations? Yes, I am. I don't want to do that, but I want people to read mine. Okay, that's the social media stuff I'll tweet about. I can't find anything later. I guess that was it. I don't know. I'm going to stop you there because this task is pretty much impossible to complete because the information isn't really there. You gave it an amazing try. Nice work. And a very entertaining try. Thank you, David. Okay, so after, so we watched David perform. We saw where he struggled. We thanked him. So now we get to learn from his struggles. So what did you guys notice?