 Related to the West Ham, and Rob West Ham is down, I think it's very interesting. We hope that, and, yeah, please welcome our speakers. To the standard thing, and then Muldron told me that he's going to be here. Please, the body will be much more interesting if he tells us things. So, this talk is going to be about the maintainer's perspective in engaging with open source design, okay? So, if you've ever spoken up Boston, you have been asked to review the video. Your own talk using this thing, sorry. And over the past year, Walter and I have been working together to redesign this interface, okay? So, the purpose of this session is to fold here about the sign from the maintainer's perspective and to give you the chance to witness a semi-structure interview, which is a research, a qualitative research technique that you can use with your users or with your contributors to find out a little bit more about who they are and why they're using your software. To do one of these interviews, you need to prepare a discussion guide like this, but the technique gives you a lot of freedom. So, you don't have to go through all the questions, and you can give a lot of leeway to the interviewee to take the conversation where they want it to be. So, that's what we're going to do today. Yeah, if everybody's having with that. Cool. So, this is the interview. So, Walter, at some point you decided to call on to open source design and you submitted this post over here to our jobs forum. So, why did you decide to do that? Well, the interview that I wrote was a piece of software that was working quite well, but I noticed that people were misunderstanding things. Because, I mean, I'm not a designer or user interface designer, and I made a few mistakes and I was hoping to find somebody to help me, and then I found open source design. As it happens through another talk at FOSM 2016, which mentioned that open source design existed. So, I submitted a talk, a job there, just to see what we can do and things went from there. And my original idea was that, well, things would just need to be tweaked a bit because it's mostly there, but that's not quite what happened actually. Before we go into what happened, can you tell me a little bit what were the things that people were getting confused about? What were the things that people were misunderstanding? So, the things that people were misunderstanding and getting confused about was mostly things like the original interface wanted you to use some JavaScript buttons which would then update the form down below, and then you had to submit that form without making any changes. And some people misunderstood that. They were like, I clicked the buttons now, so now everything is fine. And they go like, they chose the everything is fine option, which meant that all the changes they made were thrown away and not used. Which is not very useful, of course. Those were the most extreme ones. A few other things was also with the audio options that was really not clear. People had to enter a number rather than to choose options, and that was a bit confusing to you. So, there were a few things like that that were very confusing for people and that they misunderstood. So, let's go back to what happened. So, I replied to that post, right? Right, yeah. And what happened next? That's been a while. I think what happened was that we had a chat about what the system needs to do, and I explained the background of how everything works a bit to you, so that you had a good idea, and I showed you how the current review interface worked, while the then current review interface worked. And then you went to work, and you came back with the proposal. Yeah, so you get access to the system, and I came back with the proposal. I actually sent you these. I sent you two things. Do you remember what I sent you? Yeah, so the things you sent me was, one was indeed a prototype, which was a web interface version of this one. That didn't actually work, but it showed how things were supposed to work. And you also sent me a video with an explanation of how things needed to work. I mean, this is just a mock-up, of course, that's a video. The mock-up was quite nice because it allowed me to interact with them and see what I thought was right, and then the video explained the idea behind it, I guess, right? What's that what you expected when you started to work? It was absolutely not what I was expecting. This is a complete overhaul of the older interface. It's completely different. I have to say that it's much better than what it used to be, because what used to be was everything, whereas right now it's like a layered thing. Well, you really help, you go through it step by step, and that's really good. I thought, well, we'll just need to move things around a bit, and that'll be fine. That's what I was expecting, but it's completely different. Did you freak out a little bit? Maybe for like two seconds, and I was like, well, yeah, it's better, so whatever. So not really, no, not really. It was, I didn't really freak out of that at all. I think it was a very brave decision of you to say, yeah, I'm going to follow what this crazy lady is telling me, because as you say, it was a completely different approach to the vessel than you had in place. Yeah, it was indeed a completely different approach. But at that point, my lord also said, if you came up with something that was going to need too much work, I was probably going to say, well, I'm going to keep this in mind, and probably not for this for them, but for the next one. But I got around to it. It was not that much work. It did mean that a few other things that I had planned for this edition have now had to be postponed. But that's fine. I mean, this is actually the most important part of the system, the review interface, because that's what other people need to work with. And to have that work much better means we don't have to use the administrator interface as much, which is the other part that I wanted to update. So that's fine, right? That's all the problem. So I sent you this stuff. And what happened next? Do you remember the type of process we found? Yeah, so I looked at that. Like I said, it was different from what I was expecting. I immediately saw a few things that looked like there could be problems, because the way the database is written, it had some issues that I couldn't directly implement, or which require a lot of work that I thought I wouldn't have before false them. And so we talked about that, and then we made a few compromises here and there. In one case, we said, let's try this, which then afterwards we decided, no, that's not a good idea. So we went back to the original one. In other cases, we actually kept it at the compromise that we did, that we came up with. So it was a bit of an iterative process where we improved. You made changes upon my feedback, and then I gave you feedback again, and so we came up with something that we thought was fine, right? What about the tools that we used to communicate? So what we used, well, we've used mostly email. You sent me an email with updates, and then I looked at what you prepared, and I looked at your video, and I gave it some thought, and then a few days later, we would do a live video of what was it again that we used? I'm not sure. One of those, one of those video chat, yeah, JITZ is what we used, right, exactly. So we did some JITZ session to talk about it, and that actually was a very good idea, because if we had to do everything over email, it would have taken much longer, and doing it over JITZ allowed us to interact a lot quicker, and that's actually good. So, yeah, we went through a few iterations. That's the, these are the notes that I made from our one-hour calls, by the way, and do you remember any disagreements, any of the things we kind of had to negotiate on? We negotiated mostly about the one thing that we ended up going back on, which was the other brokenness thing. So maybe just a little bit of background. If the review system, it doesn't allow you to change everything in the video. Some things you can't change in a web interface, and for that system, the old interface had a, like, well, we can't fix this with a text box that allows you to enter some free form text, and that would mark the talk as broken, and the administrator has to go in and see, okay, what's wrong, how can we fix this? The original interface kept that as part of the review. Like, we have to make changes, and that was just an extra thing, but then the talk doesn't get marked as broken, and that makes it difficult to see that it was broken. So then we came up with three selections at the first, but then we tested the interface afterwards that turned out to be a very bad idea, so we went back to the two option one. We'll see what happens there. Yeah, that's the other brokenness there. So you mentioned something interesting in there. We tested it. Yes. Oh, sorry, did I run out yet? That's fine. We tested things. So you, well, we, awesome. This is how we see video online. Okay, sorry. This is just in place in my video. Nobody really cares. Sorry, I can't mute it. It's fine. So, yeah, the way that worked was, we asked for a few volunteers, I think we found five. Some of them who had been speakers, had fallen before, and others had not been, which I think is a good mix. And you changed the mock-up so that they would be relevant to that speaker. Usually it was just to show the talk by that speaker, and then you sent them an email and you asked them to pretend that that was actually a working interface. There were a few things that didn't actually work, but that was fine. And then they went through it and you asked for feedback from them. You also recorded those sessions and I could review the recordings, which was actually very useful as well. And then you made some notes based on that so that we couldn't then look at together and use that to improve the design even further. Yeah, that's what you did. That's exactly what happened. Thank you. I'm just going to give you the mic because I keep getting to repeat what you say. No, no, no, it's okay, it's okay. I think we're good. Sure. When I was in college, we had some theoretical classes about the whole process. That's 20 years ago. I haven't actually done that in practice ever, so it was really new for me. I had like a very vague theoretical idea of how that works. It's ages ago without a feeling, remember? So yeah, no, I haven't. What do you think of the process? I think it worked quite well. And most of it was because the original design you came with was a really good one. If it hadn't been them, we would have... No, I think so. If it had been, it would have taken a bit longer, it would have been more complicated. But I think what you came up with originally was quite nice. And then iterating over that and improving it further and further after seeing what's technically possible and what works with tests that we did with uses, that actually worked well, I think. This is the outcome of the test in session. There was a little bit of analysis. There were some categories on the left and then some notes relevant to each test. And then this is the suggested design changes. Do you remember any of the suggested changes? It's been a while. Don't remember the... Yeah, we had... At some point we had... In the review page we showed how far down the process it was, which I thought was a very good thing. It looked really nice and it gave people a better understanding of how the system worked, I thought. But then we tested it, we found that it was actually confusing people. And because that page... If you see all four options there, it would have been good, but you don't really only ever see the second option. And then it's why is that there, do I still need to do something? It was confusing, so we removed that. I thought it looked really good, but if it's not functional, then it has to go. This is one of the interesting things with this process, the things that you think are going to work, then they'll work at all. Right, exactly. Then I started writing the code, I think. I mean, I had expected to not need as much because I thought we were going to change that much, but since you basically overhauled the whole system, I really had to change quite some things in the back end. And that took some time, and in fact today I noticed that there was a bug still in there. It's fixed now, but this morning's code, this morning's talks haven't been reviewed yet because I had to fix the bug first. But we, I mean, I implemented things, and then there were a few minor things that I still found that I asked some feedback from you, but you were quite busy by that time, so in some cases I just invented things myself. I think it looks good still though. I think this is one of the problems. I'm going to ask you next, what do you like about the process and what would you change? And I think one of the problems we've had is that I was a bit too busy, so I abandoned you a little bit at the end, didn't I? I still have something to say. It's fine, it's fine. I mean, yeah, you did indeed, but I mean, I understand. I mean, I couldn't get busy too, it happened, it's not a big deal, right? And what is it that you liked about the process? Have you liked anything? Is there anything that you found particularly interesting or? Well, nothing in particular. I think it worked well. I think the iterative process is a good system. It works quite well. It, working with mock-ups also allows me to understand what you're meaning, what we're talking about. It gives you something to deal with, rather than some hypothetical scenario that may or may not do something useful. So that's actually a good way of doing it. So, of course, translating that mock-up into actual stuff that works with the rest of the entire system, that was some work still. But not that much because much of the mock-up was already, the JavaScript part of it was already written, so thanks for that. So that was fairly reasonable. I think it worked quite well, and there's not much that ever changed really. Is there any aspect that you think we could improve? Can't really come up with anything right now. It might be useful at some point. We could maybe use some system where you can file bugs more easily or something. You'll use GitHub issues. We use GitHub issues, and that worked kind of. But it might be useful to have a bit more structured, I think, but that's about all. I also wouldn't know how we would make it more structured, so it's a bit complicated. But yeah, I think it worked definitely quite well, so I don't think there's much we can change. You're giving another talk after this, right? Yeah, I'm giving another talk in the open media this room about the technical part of how it all works, back and how that works. So if there's anyone who's interested in that, you definitely go there. I think it might be at 3.30. I'll check out the talk. Better make sure not to miss it, right? But yeah, it's definitely a good talk there. So we have three minutes to go. Will your people have any questions? And I just want to make a comment before questions. Let's be excellent to our speakers and not leave while we have a question session. Let's listen to questions. I'll tell you three minutes, okay? Thank you. Go ahead. I got one question regarding the process myself. Does it stop once everything is delivered, or do you have a way to improve the design with a bit of steam or the kind of stuff that we all see back from the interview? This all have to leave to you, because some of the things you just asked, I don't know. Well, I think what he's asking really is, what's going to happen next? Because we have a live testing of this interface going on in the next two days, right? Well, there is that, yes. Well, currently, I'm definitely monitoring things and making sure that everything runs well. If things go wrong, we do leave a contact, a way for people to contact us and say, hey, I don't understand how this is working. Actually, multiple ways. If that, we do get some feedback from that. I'll definitely expand on this, and we'll see how we can improve that even further. I'm not going to make changes to the design for this year. I'll definitely keep that in mind and then use it for next year, but we're not going to make live teams to the live interview place because I think that's a bad idea, right? But other than that, I don't think we'll have quite as many issues as we had last year, and last year we had about 90% of people understanding how the system worked, so it should be good. That is for sure. Different centrally, you've used mostly rapid prototyping to test these interfaces, although you've used bench and mock-ups, but I don't know if that's like... Mock-ups, prototyping, we can't see that. Okay, but do you also see events made before that? Do you create prototyping or just images for design use? Absolutely, yes. What happened in this case is that we have a very contained design problem. We didn't have to go through that many prior work or that much experimentation because the problem was relatively simple, it was well understood, and it was very contained. Most of the times design problems are not like that. So in those cases, definitely paper prototyping, mock-ups, draw the hell out of it before even bothering in prototyping it, as I did, definitely. This case was a little bit special in that. Ask him questions that you understand what you can do. For example, if I am a developer of sign-ups, then I'm going to check the user interface by you, and you say, oh, I should start from the beginning, and say, is this hard work? Is this not a good response? I would like to see at the very first, as your first response. No. So how do you make your mock-ups based on knowledge what the underlying software can do, perhaps the design is bad because the underlying system is bad and you cannot do much more? No, that's that's... What is it that you learned how to make your mock-ups? That's actually a good point, but I think from my point of view, it actually made me reconsider some things in a good way. And it's true that I had to rewrite some of the backend because it didn't match what you wanted to do, but I think it's an advantage because if she would have limited herself to what the system could do, then we would not have come up with such a great design. Shesher. So first of all, we did have a conversation before I sat down to design, so I could understand a little bit of what was going on behind S-Review. Yeah. But it is my role to challenge that as well. So I did, and he took up the challenge. So you are going to be challenged by your designer, and I think that we need to live with that, and we work together iteratively to balance that challenge and come up with something that was feasible. Right. Absolute very first one had a few things in there that were not workable, and we talked about it after the first iteration and we worked that out, and we made a few improvements. I think we had three or four iterations in total. So yeah, definitely, I mean, if it's just the first one and that sets or the highway, that doesn't work. That's all we did. We had a discussion about it first, then she made a mock-up, then we discussed that mock-up, based on that we made a few more changes. So it's an iterative process, and that way it does work. But I do agree, if you just make a design and you don't care at all about the backend, that's problematic, but that's not what happens. That's definitely not what happened. Folks, we need to wrap up. We're out of time. Thank you very much for being here. Thank you. Thank you very much.