 But thank you for having me. I'm really happy to be here. I am Jessica Outlaw, and I will be talking to you about embodied cognition and these new design interactions that are going to be coming as virtual and augmented reality, embeds itself in our society even further. How many of you came to my workshop yesterday, like show of hands? All right, so for those of you who weren't in my workshop yesterday, this is a little bit about me. I have experienced developing and designing virtual reality apps. I'm trained as a behavioral scientist, and I feel like my goal is bringing all of these social science insights to the development of spatial computing. So I've been involved in studying online worlds. So maybe some people are familiar with Second Life. There's these new worlds where you can go into social virtual reality, and you're fully embodied. You have full body tracking. And it's very different from Second Life, because Second Life was only text-based. And what's happening is that people are being harassed. And it's a very complex problem, because in a way, people are safe because they're safe because they're in these virtual bodies. But at the same time, they're also being harassed in their homes. And so it's a very complex issue that a lot of companies are looking at. So I've been working in virtual and augmented reality for about three years. And the purpose of my talk today is to talk about the future of interaction design. So as part of this, I would like to do a little bit of an exercise. So I want everyone to practice this with me now. I want you to raise your left hand when you see this symbol. Left hand? OK. Everybody do it. All right. And then I want you to raise your right hand when you see this symbol. All right. You ready? OK. Put your hands down. All right. And go. Go. Go. Go. OK. How was that? OK. All right. Let's do another one. Let's switch it up. So I want you to raise your right hand when you see this symbol. And then I want you to raise your left hand when you see this symbol. All right. And go. Go. Go. Go. All right. How was that? What was the difference? It was natural. It was the exact same symbol. It was the I was giving you very similar instructions. It's just that you came to this talk today with an existing action perception coupling that I just rupted. And the thing is is that as experienced designers and interaction designers, we now have a very unique opportunity to leverage people's own bodies and their existing associations in interaction design because the technologies that exist today allow for full body tracking. And I'm going to talk about the consequences of that. And also, I just want to say I'm agnostic as to whether or not you should use the associations or whether or not you should disrupt the associations. I think that every case is different. And I think it's just more important to be aware of the design choices that you're making. So we're going to get into more about embodied cognition. But first, let's define embodiment. So this has to do with it has to be perceptible by the body. It's something that the mind understands through bodily experience. So there we have embodiment. And then cognition, very simply, things related to thought. And I think that everybody probably walked to the conference center today in the sun. Like, how did you know that you were in the sun? Did your skin pick up signals and send information to your somatosensory cortex in your brain? And that's how you knew? So there are other information that you had? So the thing that I'm trying to convey here is that there's this interplay between our brains and our bodies that we may not be consciously aware of on a moment-to-moment basis. But it's something that the future of experience design is going to be leveraging very, very heavily. Because I would like to convince you today that the human body is an interface. And the potential of virtual and augmented reality is that we will be able to increase the capacities of our own bodies and our senses and what we're capable of, which I'm really excited about. And then as a result of being embodied in these avatars and having these experiences, it's already shown to impact attention, memory, problem solving, decision making, in some really profound ways. I'm going to talk about an example later where people are going to be making decisions about what's fair and what's not fair based on random assignment into being when to what type of avatar they're placed in. I don't usually present graphs from 1997, but I did in this case because this is my fourth day at the conference. And usually I just hang out with fan boys and fan girls of virtual and augmented reality. And I have really encountered some skeptics of spatial computing while I've been here. And so I went to CNET. CNET still has graphs up from 1997. Is anybody working in graphic design in 1997? This is probably the oldest graph presented today. So who is really dominating the sphere in 1997 for ISPs? AOL, who were the other big players in 1997? CompuServe, Netscape. I would like to convince you that VR is much more at the level of AOL CompuServe in 1997 and that you may not be fully convinced of it. But in 1997, it was very driven by enterprise. And eventually consumer adoption came. So you may be skeptical right now, but I would like to persuade you that there's big things coming. So for example, here's an enterprise example where Walmart purchased 20,000 headsets for their employee training centers just in the United States last year. And there's already so many examples where people are using virtual reality to do simulation training. So this is at Children's Hospital Los Angeles. Having mannequins for this type of pediatric resuscitation is extremely expensive. And you can't actually do pediatric resuscitation training on real children. So this is a case where it's a very high-stress situation. And VR is providing this way to allow people to get the muscle memory in place so that way they can learn these very important skills. So there's a lot of infrastructure in place already. And there's already a lot of research showing the potential of this technology. And I'd like to talk more about what that means for designers who are thinking about the human body as an interface. I'm going to give a couple examples of how people can do things in virtual reality that they could really not do in this physical reality that we're here in right now. And I'm going to talk about this concept called homuncular flexibility. This is a picture of me. If you don't recognize me in the pink. This was my first virtual reality photo shoot. And in this case, I was meeting one person on the, I was on the West Coast of the US. This person was on the East Coast. And then the person taking our photo was based in France. And the three of us were embodied in these avatars and goofing around and in very different time zones. But just to give you an example of the commonality of jumping into these avatars. And there's some really weird research that's going on on how being in certain types of avatars changes your behavior. So in this case, we look pretty normal. We've got one hand. I've got two hands. I've got no arms. But other than that, I look pretty normal. In this case, this is a research study where they gave humans a tail. And they wanted to see if they could train humans on how to move a tail. And the only thing that they did was they gave people visual input. And so they had people move their hips. So you can see she's moving her hips. And then this blue line represents the tail. And the task that she has to do is she has to use her entire body to stop the, it's like a video game. She has to stop the flow of green, I don't know, like the green lines from getting past her. And so she has to move her entire body. And her arms and her legs are incapable of stopping the green lines that are on the farthest perimeter. And what they're able to show is that she got proficient enough with moving the tail just by using her hips that they actually think that this is based on the vestibular structures of humans and chimps, like how we used to have tails. And that's what I meant by homuncular flexibility. But it actually gets weirder because this study was done in 2013 and then some people at Stanford decided that they wanted to give people a third arm and see if people could learn how to control a third arm. And they were able to do that. And then they said, why are we even looking at humanoids? And then they tried to make people feel like bats and people very quickly adapted to feeling like bats. And then they decided to look at lobsters. And again, people were very, very adaptable and began to act and move like crustaceans. And so there's this phenomenon that's happening that we are conforming to the bodies that we are in. And so what's gonna happen when we're getting all of this information? I mean, there's one more point that I wanna make about this slide. One of the main reasons this is happening is that the person is seeing information. Like they're being tracked by their body, by these sensors. And then as they're being tracked, they're getting information back to them and it's being, like the information about how they're moving is being rendered to them and the information that the person in this study is getting is like, you have a tail. You are successfully navigating this tail. You're moving this tail. And that makes the person feel that they have the tail. And so the reason this technology is special is because there's massive amounts of data streams that you can use and then you can take all of that data and then you could represent things to people. And that's why this study is so important. And that's just with one type of data. What they did was they used positional tracking. And the thing is is that this is existing data streams that are available right now for anybody who's really interested in collecting it. So you could have full body tracking for motion and gesture. You don't have to have controllers. You know, you can have, there's like a lot of precision involved that the sensors can pick up. There are heat maps that show what you're paying attention to. So I can see what you're looking at over here. How long are you looking at it for? Are you paying attention to me while I'm talking to you? Are you zoning out? There's ways to track the size of your pupils. And that's another way to detect if you're paying attention or if you're zoning out. And then your boss can get that information if you're comfortable with that. But yeah, the sky is really the limit on what you can use these data streams for. Spatial mapping and awareness, let's see. And then you can also add this into heart rate monitors. So there's other health implications to this as well. So the takeaway that I want people to have from this is that when you're able to control the type of embodiment that you have, you can really radically control like how people are, who people are identifying with. You can also get them to increase the earned capacities. So we could all be controlling tails. And the other thing is that this is for people who are able-bodied. This is also opening up huge potentials for people who don't have the same capabilities and who might be managing different disabilities. So this is another reason why I think the potential of this field is thrilling. All right, so we talked about how avatars can shape our behavior. Let's talk about another way that this is frequently happening. So this is a study that was done in a social psychology lab. This is not a virtual reality study, but I think it's really, really, it's an important study that I think people should be aware of. So what happened in this study was there were groups of students that came to this undergraduate lab and half of them were given, well actually all of them were given coats, the exact same coats. And some of the students, half of the students were told these belong to painters. And like just house painters like just run of the mill, nothing special, but can you put this on and perform these cognitive tasks for us? And then the other half of the students were told that these exact same coats belong to doctors. And as a result, they worked so much harder. So why would that be? They outperformed the same people who had been randomized into the painters group on attention tasks. And that was likely because there's a symbolism around physicians and how hard they have to work and how many years they have to go to school. And there was something about that symbolism of the physician's profession that really extracted more effort from people. And what this is called is it's an example of enclosed cognition. So we've been talking about some examples of how the body can impact the brain, but just like the symbolic, the symbolism of the clothes that we wear can also impact it. All right, so I'm gonna talk about another study. This comes out of the Virtual Human Interaction Lab at Stanford. If anybody's really interested in some of the VR research, I would really recommend you look up the Stanford Virtual Human Interaction Lab. They're doing some really, really interesting research. And what makes them unique is that they really try to study not just people's experience, but they try to measure people's behavior after virtual reality experiences. So this is a study. I apologize for not converting this to the metric system. So it's just like short, average, and tall. All right, so this is a case where there were, there were people who were embodied in short avatars. And then there was somebody who worked at the research lab who was always in the average height avatar. And then other people were randomized into the tall avatar. So this is how the study went. They had like a social conversation and then there was a negotiation task at the end. So people were either matched up. So people were either randomized into the short avatar or into the tall avatar. And then this is the negotiation task that they had. How many people are familiar with the ultimatum game? Any econ majors? One? Okay, we got a couple more. So this is how the ultimatum game works. So one person gets $100 to split and then they have to, you have to offer some split to subject B. And subject B can accept or reject. And if she rejects, nobody gets any money. So that's why it's the ultimatum game. Like it's one shot and it's over. So what do you think happened? When people who had been randomly assigned into tall and short avatars played the game with an average sized person. So the short people, the people in the short avatars, they were very fair. They were like, you know what, let's just do, let's just split this 50-50. Makes sense. But the people who were in the tall avatars did not behave that way. They were like, you know what? Why don't you take 40? I'm gonna keep 60. And the research confederate always just accepted because they were more interested in finding out what people were going to offer them. So that's one example of how random assignment into an avatar can affect behavior. But then let's see what happens when the research confederate plays the ultimatum game back with them. So what do you think happens? Because in this case, in this case that I just presented, the people who were the research subjects offered money, but now they're going to be offered money and they have to decide whether or not to accept a fair offer or reject it. So in this case, in the final round of the ultimatum game, the research confederate made an unfair offer. I think it was like a 70-30 offer. And the people who had been randomized to be in the short avatars were like, okay, I'll take it. Like it's fine. However, the people who had been randomized to be in the tall avatars were extremely, were like much less willing to accept an unfair offer. So the delta is like 72% to 38%. I mean, that's a really significant difference given that these are people who were randomly assigned into a tall avatar or a short avatar. So why is this happening? It's likely because they're acting out a belief that relatively taller people have confidence and power, which is kind of very like 20th century. Like why do we still have those beliefs about ourselves? But there it is. That's what the data showed out. So there's more theorizing that could be done around exactly why this is happening and exactly why being embodied in this avatar is affecting your behavior. So one is that you could, so behavioral confirmation, that would have to do with I'm perceiving that this confederate's expectations of me are that if I'm in a short avatar, I need to conform to whatever offers they make me, no matter how unfair they are. Or it could be self-perception and it could be this whole idea that well, the way that I'm seeing myself right now is I'm in a low power position, so I should just accept whatever offer is made to me. Or it could be de-individuation. And so that could just be people are not necessarily feeling like an individual and they have lower self-awareness in what's going on in this task. So there's these various explanations, but I think what's really, really important for people to understand is that these design choices about just the height of an avatar can have really important effects later on. And it really poses the question of like, when do you want to leverage the existing associations that people have? So in this case, they leverage the association that height equals power. And at the beginning of the exercise today, we did left and right. So there's these associations that we have. So when do you want to actually use them? And then when do you actually want to challenge them? So the human body is an interface. I hope I've convinced you today that these technologies around virtual and augmented reality, they do have the capacity to increase our own ability to connect with ourselves, to connect with others, to change the way that we make sense of the world. And there's already a ton of research that's showing how just small changes in the environment or in how we are embodied can affect our attention, our memory, our problem-solving, and our decision-making. So I brought in the last part, I just want to go over a couple quick examples of types of design choices that people could be making. That could affect the overall experience of your users. So even how somebody is going to hold an object is going to make a really big difference. So I tried really hard to find somebody holding a pencil and a hammer at the same time. This was the closest that I could get. But think about the types of muscles that you need for holding a pencil, versus holding a hammer. So if you're going to do something with a pencil, you know, you're using like this part of your body. And it's going to be really good for precision tasks. It's going to be really good for detailed work. And then when you're holding a hammer, you're activating way more of your entire body. And you're in this like extremely powerful pose. And so you're going to be, I would not recommend creating like objects for people to hold if you're trying to get them to focus on the details or to think about having them do precision work. So I think even how people hold objects can affect the overall experience. This is one of my favorite examples from virtual reality, which is this whole idea of like, how will you get people to navigate? So this is an experience called the sky is gap. And the way this works is it's basically a game where you travel through an entire narrative of someone's life. And there's no controllers. The only way that you can move in the experience is to lean forward when you want to go forward in the person's life. And you want to lean backwards to go back to when the person as a child. And so it's activating these associations that we already have, that the future is in front of us and the past is behind us. Because if I went up to someone else and I was like, oh, I want to tell you about a story of back when I was in high school. Like that's a pretty natural experience that we have. And so this is a case where people are using the existing associations in a really successful way. How will people learn? So baseball outfielders are like one of the prime examples of embodied cognition because what they're trying to do is they're trying to move to catch the ball. They have to like navigate everything on the field. And it's something that you could never learn to do by sitting at a computer. It's something where everything in your body is focused on a single task. And it's so focused on solving this particular problem. How do I get my glove right underneath that ball? And so that's a very specific type of problem, which is how do I catch this ball in the outfield? But what if you have a problem that is not related to sports? Does anybody know what this is? You can shout it out. Yeah, yeah, it's a map. It's a map of this area of London where there was a huge cholera outbreak in 1854. And there was no germ theory at the time. And so people had no idea that cholera was spread by water until this one physician named John Snow. He, until he actually created a visual representation to show that there was this concentration of deaths around this one water fountain in this area of London. And it actually was a huge contribution and it ended up, excuse me, it ended up saving a lot of lives because what the city did is they shut off the water pump. And if they hadn't had this map as evidence, they would have kept the water pump on. And so the reason that I'm bringing this up is because I think like the point of this is that it represented a new way to look at data that no one had ever thought of doing before. And what's gonna happen when we have these new ways of having embodied learning experiences? And my theory is that with these new capabilities and these new experiences of being able to, to see data in interesting ways and novel ways, it's going to bring new insights for problem solving that we haven't had previously. So this is just a recap of my talk. I really hope that I persuaded you that embodied cognition is the future and that you have some very powerful tools at your disposal for affecting user behavior in this way. Thank you very much.