 I am Renota. I am a UX researcher. I work for URA design to improve usability of open source software. I'm also a part of open source design and I started my UX research, doing UX research in 2016 through Outreachie where I did usability testing for GNOME. I am also a part of Fedora's diversity and inclusion team. So, designs are assumptions on developing with-hand users. What I mean by this is basically, when developers and designers are building a user interface, so what they have in mind, they build it on based on how they think. It's the best way for the users, but in the end most likely that's ends up being proved wrong by the users and so it becomes very hard for them to use, to actually use the software. So, in that case, what people do is they usually blame the users who are not knowing how to use the software. So, they label them as beginners, non-technical people and stuff like that, which is, and this keeps just going on and spiraling, so it becomes like a rat race where, and this happens also in every kind of software, but especially mostly on open-source software usability. So, there's a way how to stop the rat race and that is doing user research. What user research basically is, is connecting the end users with designers and developers. So, you just find a way to translate the user's needs and to show them to developers and designers, so they take them to considerations while building the interface. So, there are a lot of ways that you can do user research. There's AP testing, interview surveys, card tours, tree testing, and the list goes on, but today we'll focus on usability testing, because that's the main, the main method for conducting user research and it's the most effective one. So, but before we get into me explaining how to conduct the usability test, I just want to show you some use cases. So, on software that I've been working the past year, one of them is HTTPS Adware. So, on this side you can see this is the basic, the first design that they did and they had it for years. It's basically, it turned out to be not very user friendly. It's a bunch of turn boxes and just lengths and we did few iterations of user research and usability testing and we did some surveys and we did, after four iterations basically, we concluded this design which is the final design and will be implemented soon by HTTPS Adware. Another example is Thunderbird Preferences redesign. So, this was the original state of the design. When we did user research and through the usability testing sessions, we saw that the users could not be able to use the horizontal bar very well and the side bar was not very user friendly because it has so much components going on and they did not know how to distinct them. For example, they always messed up the security and privacy bar. They did not know which one has which options. This is after the design. This is a mock-up that we did. So, we narrowed down the options in the side bar. Privacy and security are not together and we got rid of the horizontal bar and you can see the interface is much clearer and it turned out to be much easier to use. This will always also be implemented. The third use case, I chose Briar. Briar is a secure messaging app. This is not actually a redesign. They just came up with a new feature which is adding contacts to the music. So, we had to come up with a whole idea for the design. So, we did user research first and we came up with the first prototype which is that one. After that we tested it and that did not go well on user testing because users do not know how to navigate through the screen very well. Then we decided to split the screen. This is the second prototype, by the way. This was better but we are now coming up with the third version which should be the one that gets implemented. So, hopefully you are convinced that doing the test and research, it is good and you want to know how to test it. So, how can you do it? It is fairly easy to do it. So, most people have a misconception of thinking that to do it is to do it. Testing the session, you need big labs and a lot of resources actually, a lot of testers and experience people to test them. But that is not the case. I think everyone can do that. You basically need just a few things. I am going to show you how you can conduct a usability test on only three steps. So, the first step is just choosing personas. What I mean by that is you should know the target users for the software that you want to test. For this example, I wanted to take software, because it is fairly easy to explain. But you can apply those things to whichever software that you choose. So, the first step that you should think of is who uses GNOME software. If we were to test right now again, we would think, okay, we should test with designers. So, we would include in our test our beginner designers, more experienced ones. But in this case, GNOME software should be used by everyone. So, we should make sure to include in our testing diverse range of participants. So, after we have come up with the target users, and now we can move on to scenario tests. What scenario tests are? They are basically tests that you give to each participant that they can accomplish. For example, so for GNOME software, the first task might be you choose an application and you tell the tester to install that application. And you watch how they do it, and you see you get to observe if it's easy for them to find the software that you're looking for and to install it. A second task might be to remove an installed software. And now, again, you can just watch them, how they navigate through the screen. And if they can find out how to remove the application, how do they find the option to search for the application and then remove it? Do they go to the installed application? Because there are a lot of ways that the user can get to the finish point, but you should just observe each user how they do it. Another task might be just to update preferences. So, here you can observe if the user knows and understands the text here. If they know what this means and how to use the screen. Another task might be just looking for updates, but you should be careful while doing the task, so you should not give hints to the users. So, when you make the task, on this case, for example, do not use the words like update, find an application, find updates. You can just say, like, can you find out the version of this application and then they should find out by themselves. Because if you give hints, then there's no point in testing usability of that. You can do how many tasks that you find using them. Usually, just five testers and five tasks, five to ten tasks, it's more than enough to uncover more usability issues. But if you want to get more specific and to get more details, then you can do more than that. This is another task that you could give, for example, if you want to find out if they know the version of Chrome software or how to contribute. So, after we are done with all the tasks and we chose our participants, now we can move on to the actual testing part. The tools that we need for testing are fairly simple. In our case, with known software, all you need to test is a quiet room with internet connection, a laptop with the software installed, and that's it. So, basically, you sit down on a chair on the side of the participant and it's important to not be remotely because that's a totally different way to do usability testing. This is more effective. And you get to observe the user while it's accomplishing the task. Another thing you might need is a notebook so you can take notes during the test. So, because it's important that when the test starts, you shouldn't start the test. So, you just set on a timer and measure each test through how much time each task is taking and you do not interrupt them. All the questions that you might have, you just write them down and wait until the testing session is done and then you can continue and ask them. So, after you've conducted all the tests, this is the most crucial part of the testing. So, it's finally getting the results. So, this is the connection that I've talked earlier. With the results, you're making the connection between end users and developers and designers. So, you want to find a way that all the usability issues that you found out in the testing that you made, you translate them to other people and make them understandable so that others can work on improving them. There's a lot of ways that you can show your results. So, the first one is a formal paper. This is basically you getting on details of everything that you got from the usability testing. So, screenshots and then explaining everything and also giving recommendations on how to improve those. If you do not feel comfortable or don't know how to give recommendations on improvements, you can easily just let them know about the results and the issues and then maybe someone else might have an idea on how to improve. Another format that you can do is a semi-formal paper or shorter paper. You can do a live presentation. You can also just file an issue of bug trackers and explain each usability issue and then other people can comment and help. So, a great tool to help with showing the results is actually visualization. So, you can use screenshots and just monitor points that have issues so we're not very easy for users and after you mark them, then explain in a text format underneath what the problem was. You can also use charts or tables or whatever visualization tools that you think would fit your own results. Heatmaps are a great way to show results also. So, basically, a heatmap is a table of each task and how each test is performed on each task. So, on one side of the table, you have listed all the tasks that you gave to the user and the other side is filled with boxes. So, every box, every color has its own name. So, for example, the green boxes represent the tasks that were very easy to accomplish. Then the yellow tasks are basically where the users had more trouble, more issues. The red ones are when they had the time limit. So, they took more time than they should to accomplish a task. And the black ones are basically the tasks where the user just gave up and could not get how to do that. And this is a great way to get the results for people who are not interested in reading the text format of everything, what everything went wrong. You can just look at the heatmap and get an overall feeling of what went wrong and what went right. So, hopefully, by now, you are able to and you know how you can contribute by doing user research and usability testing. We've talked about how to do user research, what is it, why it's important and most importantly how to present the results. So, hopefully you'll contribute to Fedora's experience. And thank you so much. If you have any questions, let me know. So, Fedora, I'm not very involved. I think Fedora has come in for the design and the user experience and usability testing and also accessibility go under the design part. So, it's not the same, I'd say, but it's under the design. So, I'm aware of the technical bugs like process and so on, but it seems to me that Fedora and I think improving user experience is not visible. So, do you have any overview or idea about the state of the affairs in Fedora? I'm not sure. Again, I'm not very involved. In the design for Fedora, I'm more open to the design and it's the same. I mean, the first thing, but maybe someone here has my experience in the design and they can answer. No? I do think they have a special pleasure instance like you design one and they are checking everything over there. There are some calls or some applications when you're testing, for example, when you conduct them, they made some user testing. And I think most European is done only by non-economic. Yeah, that's also true. But, yes, sometimes sometimes I'm not sure if they did well. Because they see the idea to just have active media people do just user testing. And because, like I explained, it's not very hard to do. It's fairly easy, minimal amount of people. So, by people, you don't need much things going on. So, it would be a good idea to have just how it was visibility was. So, the main problem is that you have to involve community. So, you can ask your colleagues if you're working there. But we are all programmers and it's not ideal to have only one sort of people. So, we have to ask community and mostly still, like, we did something like that on the university, but still programs. The problem is you have to involve some of the community. So, when we have, like, open house, you know, I think that they do, from time to time, they do some, like, one-side testing of, maybe, design. It's a problem that it's difficult. Yeah, I understand that. But the other thing from another view of you, that's why the reason why we just talked, to just show everyone that it's not that hard to do it, you don't need to be an expert, a designer, or something. You can just be a developer of whatever you do and still do the testing. So, you don't need any expertise to do this. This approach is the problem that you have to do. Because in Arizona, on the side, we had a lot of societies. And it's a compensator for that. Like, people who are doing goods and explaining it again and again that you don't have to wait, just ask the people who say you don't have to wait and if you want to help, there is a set of tasks. And then you get people to see that it's something for 20 minutes and you can't say again if you don't have to wait. But you get people who are not ready to go home and are not feeling like a city student looking for free food. That's the problem. Open house. And you can get people from all over the world. I like the logo. Just check people's health. That's true. There are some problems like, there is part is to be used as a process stuff. If there would be someone who would, there would be a computer and we don't take the session. There are ways, but I'm not sure. If there is a need-out or something, you can just organize it, take your laptop with you and whoever wants to contribute, they can do it. So I did that before open house. I just went with my laptop and I got some notifications with them and they were welcoming and they wanted to help and participate in the on-cast volunteer. So you can also do that. So when you choose it, does it happen where you are using this green and people manage to do it or... Do you need a kick-ass? Yeah, because there is always a problem with design, but did you ever get seen somewhere where nobody had an issue? Did everybody have an arm who always had an issue? So you mean the colors on the green boxes or when you were testing basically, you were sitting next to the user and you... No, no, no. You seem to show that there is always an issue. Did it happen that nobody has an issue? Yeah, I don't... From how much I've tested, I've never had an application that didn't have an usability issue. So there's always green and red, maybe black boxes, no, because they are very easy to use, but green, red, and yellow were always on the list. So I don't think there's such an application that's perfect and very easy to use for people, but we are striving to make it more green and yellow friendly and remove the red and black boxes, you see. Do you know, are there any open-source applications which do use automated use, experience, like measuring whether people take it and it's easy to accomplish something and so on? No, I mean, there are a lot... I don't know if there is an open source, but there are a lot of tools to do that, but my recommendation is always to do it in person and not use other tools, other people's tools, because in person, you get other people's emotions, so you can just know when they are feeling frustrated with the task or if they are getting nervous or angry, and with automation, you cannot feel how the users are reacting to a certain task. So the best way that you can get out the most out of the usability test is just to be in person. Even if you remove this tool, that thing is not as effective as the first one should be, but there are a lot of tools to do. Are there any open source applications which do this? I agree it's definitely best to do it in person, but it seems we have no such... we definitely don't regularly use it for doing it. So if there is any application which is doing it automatically, I'm not familiar with the areas and open source applications but there are like design patterns, for example, as you know, has a human interaction with design patterns, they are on their website, and you can just, when you're building the application, you can just look at the components and realize they are useful for which use case they should use, so you can use that as a guideline to be a designer and to go to open source. I was thinking about something like the building application functionality which will allow you to say, I like this, I don't like this, I think I'm hoping for the automated material. So I'm not aware of any of the source applications doing this. No, no. They are not, but the point of the test link is to not let the users tell you what they like or what they don't like, but you should see it from that, because most of the clients, that's how you get the real experience and you realize what you're describing. There are tools to help reduce the risk. I think, for example, it was eye tracking, I know some of the tools that are open source that do eye tracking. I can't remember the name right now but if you're interested, I can tell you the name. They make the goggles and the software because you're going to just do eye tracking. That's all open source. They are also another two-side survey tools which you can use and they are open source also, but not particularly the normal software. Any other common questions? Thank you so much for coming.