 Everybody, yo, hi, oh, you're so friendly. We're starting a few minutes late. Okay, so, hi everyone, my name is Sarah Wiseman, and I'm going to be talking to you about scientifically ruining computer interfaces in movies and TV. If I do my job right today, you will not be able to watch any movies and TV shows in the future that involve computers without going, oh, no. So, hopefully I'll have ruined things for you in the future. So, before I start, I really need to talk to you about myself. Why do I think I'm allowed to come up here and talk to you about this subject? Well, I have a PhD in human-computer interaction. When I describe that to people, I just say it's like computers and psychology kind of mixed together a bit. When my grandparents describe it, they just go, computers? It's kind of difficult to describe. However, I know that me coming up here and telling you that I have a PhD is not really what you want to hear. I know that the post... Yeah. I know that post-Brexit, you guys don't want to hear from experts, do you? I know that. So, I have an alternative qualification to set forth to you today. So, some people aspire in their academic careers all their lives to become a doctor, a professor, a senior lecturer. I have something better than that. There's no way to mince those words. In 2012, I was quoted in the mail online as a scientist. So... You have a Daily Mail accredited scientist standing before you guys, so... I know that's what you want from me. Okay, so now we've established why I think I'm allowed to talk to you about this subject today. Let's introduce the topic itself. So, I'm talking about interfaces. I'm sure a lot of you know what interfaces are. Interfaces are the things that allow us as humans to talk to computers, basically. They're a barrier between our human brain and the microprocessors and things happening within a computer. The key thing that I want you to realise during this talk is that interfaces act both as an input, as in things that we as a human want to tell a computer and as an output, like things that a computer wants to tell us as humans. So, I want to talk about these in terms of films. Now, the first film I want to talk about briefly is The Martian that came out last year. I'm sure literally everyone in this tent has seen The Martian? No! I'm so, so sorry. I'm so, so sorry for making that assumption. Okay. Lots of people in this tent have seen The Martian. The Martian was a film about Matt Damon... LAUGHTER ...being in space, and he was trapped on Mars. Now, for this film, it was very... It was quite scientific, really. The Mark Watney, yeah, that's his name, was trapped on Mars, and as a result, he was having to use a whole array of different computers and pieces of technology. And you may not have realised it, but like a heck of a lot of effort and time went into designing exactly how those computer interfaces should look when you were watching the film, even though you may not have noticed them. So, some of these computer interfaces were for the rover that he was driving. Some of them gave them information about the spaceship. Spaceship is not a technical term to use, is it? Shuttle? Rocket? Whatever. Some of them gave them information about that. There was information for the NASA control room. And all of this... All of these interfaces were built between, as a collaboration, between a design team who make interfaces for films and NASA themselves. So, we've come a really long way in the most recent years in representing computers on film. You can't just pass off any old hologram in films these days. We are a savvy audience, and we need things to look realistic. And these interfaces, like I said, on the Martian, were built with NASA in collaboration, and that's why they look so good. This one, in particular, is for the MAV. Oh, my gosh, it's been such a long time since I've read this book. But this interface here shows you a sense of urgency in the film. You can see that this vehicle that is on the Mars surface is tilting at a dangerous level. And so, a lot of effort was put into these interfaces to make it more interesting for us as viewers. If I can talk about another film in Avengers Age of Ultron. Like, I'm going to be... I'm going to level with you here. I haven't necessarily seen this film, but I'm going to talk about it as though I have. So, this was another film that a design agency was brought in in order to create some realistic and useful-looking interfaces. And again, they've used these interfaces to actually represent the characters that they're associated with. So, in Branagh's lab, the Hulk, his interfaces were all very green, huge, and very squishy and squashy because he was kind of interested in biology. When you compare that to the interfaces that Tony Stark uses, which are orange and black and a bit more... The contrast is a bit harsher. You can see how they've used these interfaces to reflect the users of those said interfaces. Now, although this team that created these interfaces tried really hard to create beautiful things, they've also created stupid things. This is an interface that Tony Stark uses. And I just want to draw your attention to the keyboard on that interface. Now, it might not be immediately obvious to you why I hate this interface. And so, together, we're going to use science to destroy this interface. So, throughout this talk, basically what I want to do is I want to introduce you to a few design patterns and science that we use when we're designing user interfaces and explain to you how people in the film and TV industry have messed this up massively. So, what is issue number one? Issue number one is ignoring good design patterns. Now, to best illustrate this, so I should warn you, I'm about to show you quite a gory scene from Die Hard. So, just prepare yourself. I'm really sorry. I'm going to have to leave that up for a little bit longer still. I'm really sorry, everyone. So, for those of you not familiar with Die Hard, this is the computer interface that John McClain, our lead character, he has to use when he comes into this office building and he looks up his estranged wife, I suppose is the proper term for it. And he has to use it to find out, he has to type in her surname. He types in his own surname. She's changed her surname. And he finds out that she's changed her surname. So, it's a problem for Bruce Willis in that sense. However, this is a massive problem in general and it is something that I think that I and many of the people in this room will have come across in their lives. So, let me ask you. The last time you were getting a train anywhere for any length of time and you were rushing up to the station, did you come across this interface? Going the interface, I love it, that's fantastic. My people are here. So, this is the interface and you'll see some parallels between this interface and the interface that John McClain has to deal with in Die Hard. And the parallel is that they are arranged alphabetically. Now, as people that use the English language and are familiar with this alphabet, you might think this is an incredibly straightforward and sensible way to arrange letters. I mean, you'll run up to that interface and think, yes, I've got this, I can enter my reference number and somehow you fail and somehow you start moving slowly despite knowing letters for most of your life. And the reason is, the reason is semiotics. So, semiotics talks about how we take signs and how we take cues from the environment around us. And it turns out that when we are faced with a computer interface and we see that it has letters and we see that it's arranged in three or four rows, we instantly go into autopilot and our mind goes, don't worry, I've got this, this is QWERTY, we can do this. And so, when designers break this rule, we're screwed basically, we come up to it and despite knowing the alphabet, we cannot use it at all because these letters are arranged so differently to these letters that we're more familiar with when it comes to entering text on a computer. So, there's no way for me to fix that trained interface for you, but, well, before I get onto that, let me just point out that the way that Tony Stark has arranged the letters on his keyboard, you'll see it does start QWERTY, but there are actually only two lines of letters. He has completely mangled the keyboard if you compare it to this one. If you know where the Z, X and C are on this one, they're kind of up in the middle. Basically, what I'm saying is Tony Stark is dumb. And if he ever, ever comes to you for a commission saying, yeah, I'll cover your interface, do not accept that from him. He has no idea what he's doing. He's created a text entry interface that no one can use. So yes, the only solace I can give you from this is that the next time you run up to a train and you're late and you're having to enter your reference number, just think to yourself, oh, I'm kind of like John McClane right now. You can try and tell people around you that you're kind of like John McClane. I don't think that'll go down very well, but you'll know, you'll know from now on. So that's just one issue and thing that I get upset with in movies when it comes to interfaces. The second issue. The second issue I have a problem with is overly specialized GUIs. GUIs stands for Graphical User Interface, so it's what a computer screen shows you when you're interacting with it. To fully demonstrate what I mean by specialized GUIs, I want to introduce you to a concept that I have coined, that this I hope will ruin most films for you. This is called Exposition OS. (*audience laughs*) Exposition OS is what happens when film writers and TV writers can't really be bothered to let you know through clever plot and dialogue what is happening and instead choose to overly explicitly tell you what's happening via a computer screen. So for example, we've all seen this pop up on our computers, haven't we? Inspector General's Office. This computer is being audited by the Inspector General's Office. Every day that pops up on my screen. No, it doesn't. So this is what happens when we use interfaces in TV and movies just in order to progress a plotline. What would you do if you needed to show someone exactly who they're emailing for the plot progression? You make the two line absurdly large. No one's email client should look like that. That is not what the most important thing is, so you'll see that in movies a lot when you need to know who they're contacting. They'll make the two line very large. What do you do if the hardened detective that you've been following for eight episodes of this series is slowly dying as he needs to tell his son that he loves him. What you do in this instance is you create a situation where they record something on their phone and the phone tells you fail to upload recording to ChadVelcro at gmail.com. That is again, not an alert we'd ever see in the real world. This from True Detective 2, which although not as good as True Detective 1 was still quite a good TV show, but they still failed on this part. One of the things that we often see people using computers when we're watching movies and TV is when they're hacking. So this is from the film Masterminds. And if we're talking about overly specialized GUIs, this is the king of overly specialized GUIs. Hacking is something that shouldn't necessarily be happening, but the people that have designed this computer system have thought, well, just in case it does happen, we'll create a lovely GUI for them so that they understand they shouldn't be doing this and maybe they will stop. And on the subject of hacking and castles, I want to talk briefly about the TV show Castle. So in this situation, the hacker is actually being counter-hacked by someone and desperately wants them to stop what they're doing and has tried everything possible to them. And so has obviously resorted to the cybernuke, the cybernuke, you know, how will you all use cybernukes? You know that thing. Obviously we know what that is. And so in this situation, the hacker has to, he launches the image of a nuclear missile on three of his screens in order to press one of them with his hand. Now, why is this egregious to me? Why am I upset by this? Well, when we look at the literature, it actually turns out that when people are interacting with programs they're very familiar with and that they're expert on, the best thing you can do when you're doing that is to actually use the keyboard in front of you and moving from the keyboard and using a mouse in any way to interact with the GUI that's in front of you is completely ludicrous and stupid and inefficient. And so in situations where we see a hacker on the TV or movie using a GUI and having to actually move their hands from the keyboard and touch something, you can assume that they're not very good at their job. So in this particular research paper, it was looking at how people use word processing, which I know is not as cool as hacking, but let's pretend they were checking how people hacked things. They tested out three different versions of interacting with these particular common activities they would need to do. And it turns out that using the menu, going to edit copy, for example, is much slower than using the copy icon. I didn't know there was a copy icon, take from that what you will, which is again much slower than using the keyboard. So yes, to conclude, I really think that hackers on TV shows that are probably being paid loads of money and drugs and stuff. I don't know how hackers are paid. It sounds like I'm protesting too much. I don't know how hackers are paid. The fact that you see them using GUIs and using the mouse is absurd. It means they're not very efficient. It means they're not expert and you could have got someone better to do the job, basically. Whilst we're on the issue of hacking, I really, I need to get into typing behavior during hacking. I think maybe 50% of you know where I'm going with this. Anyway, so my PhD was actually in typing. I know that sounds boring as anything, but it genuinely was. I spent four years of my life looking at how people typed. And that means that during hacking scenes on TV and movies, I get very furious. I think the most, okay, I'm just gonna play the GIF. This, what you can see here is four hands on a keyboard. That's two people, both somehow making windows appear on a screen. I have no idea, there is, okay. There is one thing they could possibly be doing, right? The person on the left could be typing sweat addresses. And the person on the right could be typing monimolimnion. Those are the two longest words you can type just using your left hand and just using your right hand. That is the only thing you could be doing if two different people are typing on a keyboard. I'm gonna be honest with you, that is too absurd for me to even touch with science. I'm not gonna talk about that any further than that. I think that's just absurd. The hacking scene I do want to talk about is this one. For anyone who has watched Swordfish, no, I haven't done experiments into that. That's not what I'm gonna talk about. For anyone that hasn't watched Swordfish, make sure you don't do so in front of children. So in Swordfish, John Travolta, baddie, is trying to get the hacker Hugh Jackman who is a baddie but a goodie but a baddie but a goodie but he's a hacker, is he good, what are the ethics? We don't know. He's trying to get him to hack into something. John Travolta tells Hugh Jackman, he says, people can hack into this system in 60 minutes. I need you to hack into it in 60 seconds. What? What's Hugh Jackman gonna do? And also there's a gun. So that makes things a bit more edgy, doesn't it? And also there's something else happening. So let's look at this. There's no way. If most people take 60 minutes to do this hacking job by typing in loads of words and numbers, if they mostly people need to take an hour to do it and John Travolta needs Hugh Jackman to do it in a minute, there is absolutely no way that he, oh okay. Yeah, no, he manages it. Hugh Jackman manages to hack into this computer system that should have taken 60 minutes in 60 seconds. Let's look at the typing research and see if that's possible, shall we? Now, if we assume that the person that takes 60 minutes is actually, we'll assume this person is actually a novice typist, okay? So this person has never really used a computer keyboard before, whereas Hugh Jackman represents the expert and has definitely used a computer keyboard before. There are a number of things that make an expert faster than a novice when you're typing. And these relate to your motor skills. So it's a combination of both experts knowing where the individual letters are on the keyboard. It's likely that I'm just gonna make another sweeping assumption. Most people here can type without really thinking, oh, I need to move my forefinger, my left-hand finger to the S and this finger to the Q or whatever. It's likely we don't need to do that because we have this muscle memory because we're all expert typists. But it's not just that we have this ability to find the individual letters in the keyboard. We can also find pairs of letters, common things that happen together, like the THs, the ERs, the EDs, the QUs. We're very good as experts at knowing those pairs of letters and that makes us a lot faster too. If we were to type complete random gobbledygook that had, I don't know, S followed by a D, does that ever happen? I don't know, a Q followed by anything but a U, we would slow down quite a lot because that's not something we're familiar with. So, as an expert typist, this gives you a five-time speed boost. Compared to people that cannot type, you are typing five times faster generally than the novice typers. So we'll give Hugh Jackman that. We'll assume the person that took 60 minutes is a novice, Hugh Jackman, as an expert, is gonna type five times faster. Let's look at what other boosts we can give him. There is a boost that you get when you have knowledge of the material that you're typing versus the lack of knowledge. So what I'm talking about here is not the motor experience, it's not using your hands, it's using your brain. When you're typing text that you're familiar with and that you have stored in your long-term memory, you're far quicker at getting to those letters than when your brain has to process it, maybe from copying it beside you or, I don't know, reading it from somewhere else. As someone who doesn't, if you don't know the material you're typing, if you're not familiar with the words and the numbers and the codes that you're using, then you're actually gonna type a lot slower than someone who does have that stored in their long-term memory. The point being here that there is a boost from both your motor, from being an expert, and from your memory, from having knowledge of that material. That knowledge will earn you kind of a two-times speed boost. And I'm kind of afraid that's all we can give Hugh Jackman here, so let's think about it. If it took 60 minutes for someone who was a novice with no knowledge of things to type, if we say that as an expert, they, I'm just checking nothing's on fire, as an expert they took 12 minutes because they were five times faster. If they had knowledge of the material, they took six minutes because they were twice as fast. Ultimately, this comes down to not less than one minute. So what I'm telling you is, no, it cannot be done. Hugh Jackman could not have hacked into that if it normally took 60 minutes in one minute. What I'm saying is, I don't think swordfish is very scientifically accurate. I'm sorry to break it to you. I'm sorry to have to be the bearer of bad news, but don't watch this as a documentary. I think elements of it have been fabricated. So ultimately, I hope I've taken you through kind of a whistle-stop tour of just how you can ruin films for yourself and those around you. By assessing and questioning and looking at all of the, oh no, what has Exposition OS put up here? Oh, I don't need to tell you, but my time is up. Thank you. I think perhaps I have a little bit of time for questions. If not, you can find me like throughout the rest of the festival. I will be so happy to talk to you about typing, you won't be, I will be. Is there any questions right now? Oh, I think there's a microphone coming towards you. I don't mean that in a threatening way, it's cool. Hey, I was wondering if the whole sort of staggered keyboard layout actually adversely affects or positively affects typing speed. The staggered, you mean when it's split in two? No, I mean like the way that the rows, like the columns are not straight. Oh, interesting. The way that the columns are not straight, but the rows are, but your fingers are straight, but not all the same length. I wouldn't be able to immediately comment on that, but I know that there are certain speed gains to be had by only having to move your finger a small amount, so either one step left or one step down. And so a staggered keyboard gives you two keys that are one step away from you, either up or down. So that would definitely give you some sort of speed gain from it. But I wouldn't be able to comment further than that. But that's a really, that's an interesting question. I've got a 40% grid keyboard in my bag if you wanna see it later. I know what I'm doing later, cheers. Really cool, I should just put that right now. I just want you to... So despite you proving that, it would be physically impossible for Hugh Jackman to have that system in under a minute. How did he actually do it then? Right, I have my suspicions that Hugh Jackman probably had someone on the inside, maybe a film producer or someone who was able to slip in the codes that allowed him to do it. I think basically what needs to be done is someone needs to draft a university ethics form that covers both having a gun to your head and the other thing that's happening. And we need to find out through experimentation. I don't know anyone that could draft an ethics form that would cover that though, so I don't know. I mean, more funding, yeah, sure. Follow up question to that. Have you considered what would happen if you gave him something like a stenographer's corded keyboard, because you can type at the rate of speech on those? That's such a good question. So the question was about corded keyboards, which allow you to type faster because you're not just typing letters necessarily, you can type phonemes, and you can use a single hand to do so, and you can type more, you're able to type quicker as a result of using that. Flip, yeah, I don't know what the speed boost is from that, but yeah, I think if Hugh Jackman had demanded something like that, he probably would have sped up. That's really cool, that's really interesting. What about if Hugh Jackman actually just spent the minute developing that screen that said access granted just to put it by with the gun? Hugh Jackman is just a graphic designer in this film. Entirely likely impossible, yeah, yeah. Cool. Are you suggesting that those of us who don't use standard keyboard layouts are dangerous subversives? Hang on, do it again, do it again. Are you suggesting that those of us who don't use a standard keyboard layout are therefore dangerous and subversive? Yes. Are we allowed another one, there's one back there. Let's call this last one, I guess. Okay, test. I have a question about the show Mr. Robot. Have you watched it? I have problems with that on an acting level rather than an interface level. Should I watch it? Do I need to? Yes, geez, guys, okay, I'll watch it. Okay, so if you haven't watched it, I have no question, I guess. Thank you everyone, I mean, catch me if you see me, yeah, cool, thanks.