 My name is Tiara Freitner and she is a postdoc at the Department of Computer Science. Her main interest as a computer scientist lies in the human-computer interaction. And she explores mixed reality approaches for remote assistance in collaboration with the legal group and also with the systems. So let's give a warm welcome to talk about the role of our bodies during interaction in virtual environments to Tiara. Thanks a lot. Is this working? You really can't hear down here. It's amazing. Okay, so I'll be talking about the role of our bodies in particular of our virtual bodies during interaction in virtual environments and how such virtual bodies could actually be transformed, how they could be changed to allow new ways of interaction. For example, a virtual arm could be extended to allow you to reach out to objects in the environment that are out of your reach and allow you to maybe control them, interact with them. So for me, VR is still very much alive because for me it's a very valuable tool for research. VR allows me a very controlled experimental environment where I can control exactly what people see, what they hear, what they're allowed to do and I can track very well what the participants are doing. I can track their movements, I can check where they're looking and I can also make them experience the space around them in a very realistic manner which means I can control how they behave because if I put you in a different room you might behave differently. So for me, VR and the especially immersive environments are a very valuable tool and currently the way to interact with VR is most often with the use of controllers like these HTC Vive controllers that you see as virtual controllers in the virtual world as well and you press buttons and pull triggers to interact with these objects around you. Now this is actually a really awesome way to interact especially in situations where you're already holding some kind of tool like a gun for instance that has a similar shape as these controllers. So this is the space pirate trainer if you haven't tried it, I highly recommend it. You hold this gun in your hand that has the shape of the controller and you pull the trigger to shoot. It's a whole lot of fun and despite games like this I think there are other cases where controllers aren't suited as well. For example if you look at this pressing a button in VR using a controller, I don't know if you can see but what you have to do here is you actually have to pull the trigger to press the button so you put the controller on the button and then you pull the trigger. To me that's counter-intuitive and I would really prefer something like that which is more what we do in real life and generally I think our hands are the most powerful and versatile tools that we already have without having to pick up something new and there is already hand tracking technology out there that allows us to do this and this can actually enable more natural and versatile and creative interactions, more expressive as well. So with hand tracking virtual objects are literally at our fingertips, we can play around with them like in the real world but still what you see here is only hands and I'm wondering can we go a little bit further because in the real world we don't just see our hands, our hands are connected to our arms, our arms are connected to our bodies we know we're there, we perceive our body in a peripheral vision, we perceive our shadows so there's always a bit more to us than in VR and I think that this disembodied state is unnatural and will always remind you that what's around you in VR isn't quite real and that the virtual world can't affect you and I think for the virtual world to affect you what we need is body ownership. Now body ownership that's a concept from psychology and what it basically means that if you consider your left hand you know it's your hand, it's not someone else's. You can feel things with it, you can manipulate things with it, it's part of your body and you feel body ownership normally for your entire physical body but under certain circumstances you can actually perceive body ownership of body external objects and that's then called a body ownership illusion and a very famous or probably the most well known body ownership illusion is the rubber hand illusion some of you are already nodding so you know it in this experiment a rubber hand is placed on the table in front of a participant next to their real hand but the real hand is hidden from you and then the experimenter starts stroking both the rubber hand and the real hand at the same time in exactly the same manner and what you see and what you feel coincides so you feel strokes and you see them on the rubber hand and if this hand is threatened you will react in a similar way as if your own hand had been threatened in that moment and that's because you are under the illusion that this rubber hand is somehow connected to you that you can perceive with it and illusions like this are also achievable in VR, maybe even more easily achievable for example this person here can see the virtual hand move like her own hand and when she sees this ball hit her finger she feels a slight vibration, there's a vibration mototape to her index finger and she feels that so we have multi-sensory stimulation, whatever happens in VR becomes so much more real because you can actually feel it and see it and it responds to your movements so taking advantage of this illusion has enabled a lot of interesting research and researchers have found that our bodies define the way how we feel and how we react it's like if you imagine wearing different clothes that will change your attitude what they found here is that if you embody a body of dark-skinned body wearing casual clothes this will influence how you play the drums so you might actually play drums more vigorously because you think you're a black person or you imagine that and they compared that if you embody a white person formally clothed then you would be a bit more timid and a bit more rigid another example is that embodying a body of different gender will actually increase empathy in this study they let men embody a woman so they had a woman's body and then there was this virtual man verbally attacking them telling them how ugly they looked and they looked into this mirror and they saw this woman's body and they felt more empathetic they felt more understanding of her mimics they could suddenly imagine how she could actually feel in this moment this is I think currently being used for domestic violence treatments in Spain a very different area I guess for application of this is when interacting with abstract cursors versus a virtual hand this might actually change how you move how you behave so if you just watch him reaching around the cell blade making sure that the virtual hand doesn't touch it and compare that to how he moves the hand when it's an abstract hand it just goes from right to left because there's nothing there the saw can't hurt him right so if we just consider how if we gave ourselves bodies in VR how that might actually change how we behave and what we do so based on this research and since I have a background in human computer interaction I wanted to explore new interaction techniques by virtually extending the human body to overcome some of its physical limitations kind of giving us superpowers and make this virtual extension feel like it's part of the own body that was a really important point for me and today I'll present two of my projects really briefly in which I aimed to achieve this the first one I would like to show you is this prototype of an augmented reality system where you can extend your arm you already saw the video in the beginning very briefly you can extend your arm to reach to something that's out of reach and you can interact with it so he sees this long arm extending and then he can reach this tilting surface and simply press it up or down and so wearing this this is an Oculus Rift DK2 this was also before the leap motion that's why I didn't add hand tracking so I turned it into a video see-through H&D so through the camera that's on the front you can actually see your real environment in the headset and then I tracked the okay so the furniture that you see there the desk the curtain and that tilting surface those were all actuated so through an Arduino I could control the electric motors that made them move or open or close and I used optical tracking all these glowing dots you see are markers for the optical tracking system so I also tracked the user the user's hand movements and based on the distance of the user's hand to his shoulder I extended the arm so if the user's hand was close to his body the virtual hand would move with his own hand but as soon as he reached a bit further than a certain threshold the virtual hand would actually extend and it would go out up to I think four meters length is the limit I set and that would look a bit rubbery like that and with this arm they could reach out to this desk and push it up or down and it was really important to me here to make this interaction as uncomplicated as direct as possible so I tried to avoid using any buttons or any memorized gestures so just allowing them to actually press on the surface they wanted to manipulate they could also open and close this curtain by simply pushing this handle open and close I kind of had to invent a metaphor for a handle here but the point in this was that even though this virtual arm behaved very differently to how our own arms actually behave they could control it and they felt as if this arm was connected to them it wasn't as realistic as their own arm obviously but it still felt like it was theirs it was theirs to control it was theirs to interact with and in an evaluation of the system I found that the participants were actually able to use this long arm without prior training so they were able to simply put the headset on and start interacting and moving the furniture around as they wished and of course the furniture is just a placeholder for whatever we might want to control you could imagine if you have a disability you maybe can't leave your bed or just can't access some things that normal people could some setup like this could allow you to interact with the world through your body or through your new virtual body and in the second prototype that I want to show you I aimed at actually reducing the strain of overhead interaction so this is a little bit different so the blue thing you see is an overhead target and the yellow outline you see is a virtual arm so that's where the person is currently interacting or is supposed to interact and I wanted to allow them to do that while holding their arm at waist level in a very comfortable posture and this of course results in a very large offset between your real and your virtual arm but my goal was to still keep this feeling of body ownership for this virtual arm despite them looking up at a virtual hand and having their own hand here I wanted to keep this connection the setup I used here was an HTC Vive this is a bit more recent and I placed two leap motion sensors on the front which are hand tracking sensors and I needed two of these because I wanted to let them interact with the target at the top first and then slowly transition their hand downwards so one tracker would track your hands in line of sight and the second one would do that same thing in front of their bodies so I'll just show you what this actually looked like you'll see on the left side this woman interacting with the target she's looking upward that's where the target currently is on the right you see what she is seeing and just watch and try to pay attention to where her right hand is you see that it's moving downward this is fast forward so this is all happening fairly slowly she's still looking up but her right hand is actually at waist level now she's still interacting with the target overhead we'll just watch it again it's looping so the thing is that this transition happens so slowly that she barely noticed it happening she kind of when she was around halfway she noticed oh there's something a bit wrong here but it wasn't jarring it wasn't disrupting and most importantly for me it actually didn't break the solution of this virtual arm being her own it didn't break the expectation of her thinking that she might actually be able to feel things with this hand if I gave her the virtual ball she would feel that on her finger so while this prototype aims to ease overhead interaction a similar approach could for instance be used for stroke rehabilitation where you kind of forget and have to learn again how to use your arm if you see movements that can help reconfigure the connections in your brain to learn this again or an inverse approach could be actually used to help with physiotherapy which has also been done or is being done in Barcelona at the moment so just to summarize my research shows that for direct manipulations of objects that are difficult to reach we can modify the virtual body we interact through or at least the body that we think we have at that moment while still perceiving it as our own body and this is possible because what our bodies are in our minds that's very malleable we can change that we can shape that and I mean one vision that I have is that if we augment our environment with actuators robots maybe we could lift up cars with our fingertips because the robot could be doing it but what you perceive what you're doing the actions you're doing is that of just closing your fingers on that car and lifting it right so I think there could be a discrepancy between what is happening in the world and what you're perceiving but to me what you're perceiving is the important thing and I want to add that I'm not saying don't use controllers not at all use controllers wherever it makes sense I'm just saying consider whether it makes sense to use controllers even when you use controllers consider whether there might be a body anyways because that actually might change how we interact with the world and most importantly if we have virtual bodies we can transform them and we can give people superpowers to interact with the world that's all for me is your computer mic all set up? ready to go? so I just want to ask you Tiarra all the potential and the superpowers that we can get what kind of potential do you see who could benefit from changing gender or becoming an animal or some kind of superhero what kind of psychological benefits could there be of this? that's a really good question so there's one benefit for research itself because we can actually learn how people perceive how people perceive their identity or other people how they could identify with an outgroup perhaps like someone of a different gender or a different skin color so I'm not sure there was so much on what a single person could profit from that but more what the society could in general because we could actually let people experience things they wouldn't ever be able to experience in real life would we be able to measure and increase in empathy for example? for example this one study I didn't show it but the changing skin color is a study they did in Barcelona and what they measured there was a reduction of implicit racial bias after experiencing having a black body for I think 45 minutes so there are ways of evaluating that and the results show that it does have an effect on how we perceive the people around us and ourselves and I am also thinking of the potential for people who feel like they're born in the wrong body gender-wise and they could actually go in there and be themselves to have that experience there's a lot of potential I also ask you for a view towards the future what do you think are we going to have bodies in virtual reality and can these bodies represent us will we have physical avatars that we can walk around I'm sure you've seen Ready Player One is that a world we're going to be in where we can just log into this world and walk around with yourself? I think one of the things that might actually keep VR alive would be if it becomes a social thing because that's what Ready Player One shows it's social VR you actually perceive each other in VR you interact with each other in VR and the obvious thing and what we're already doing in games or in online platforms where you create a profile is we adapt what we look like we adapt what others perceive of us so I think that might be a direction that we might be going into and interesting the science challenge in this area I think we'll be having eye contact and if we're doing FaceTime or a sort of video conferencing very often we look at the camera but not at the person and that is something that we need to overcome if you want to really sort of relate but I think even that you wouldn't notice if I'm looking at the top of your head or at your eyes so I think even that is easy to overcome given that we give people eyes in VR if you have a virtual avatar without eyes there's nothing I can perceive about you but if we use eye tracking or you enter your eyes somewhere I think this is doable Tell me what an avatar without eyes would be It's scary What would that be? Something like that The design of avatars is also a very complicated thing and we'd have to put a lot of thought and a lot of research into how we perceive each other and so far I think the less features you put in and the less realistic you make something the better because our imagination will fill in the rest and if you make it realistic we'll just be in the uncanny valley and it will just seem like a horror movie Scary simulation Thank you so much