 And we're very happy to have you with us tonight, Christian and Arthur. So please give it up and have a lot of fun with the talk. Yeah, thank you very much. It's really a privilege to be here. This is an amazing event. So we're going to introduce the project and talk a bit about the scientific aspect behind it, as well as sort of the journey we've taken over the last three years that we've been working and how our methodology has kind of evolved over that whole process. So really the main question behind the project that we've kind of been addressing the whole time is what would the world be like if you could see through the eyes of someone else? If people who are in conflict on different sides of a war could experience what it's like to be someone else, their story, their family and so forth, how would that affect our ability to connect and resolve conflicts? So really what we're looking at is the relationship between identity and empathy, how you construct your identity and how that inhibits or facilitates empathy. So here's an example of some research that is happening in this movement we're seeing in neuroscience in the past 10 years in embodied cognition and around empathy. This, for example, is a study that showed that there was a stronger correlation between in the neural representation of threats. So you would do a brand scan of a person being threatened to be hurt and you would record the same person looking at someone else being threatened and when those two people were closed that had family ties or friends, there was a stronger correlation in the representation of threats. So in a way, familiarity was promoting sort of blurriness of self of a representation on a neurological level. Basically the difference between you and the other person neurologically is not that different than if they're being threatened and you feel close to them. So we have also Giacomo Rizzolatti who's investigating mirror neurons and mirror neurons are specifically neurons which fire when you both perceive an action and when you take that action yourself as well. And there's a nice quote from him there. We're going to kind of go a bit fast because we have I think too many slides. So one of the kind of illusions that you can induce is a very well known rubble hand illusion that when you see a plastic hand being touched and you feel your rear hand is why it's feeling behind the blanket rub at the same time in the same area, you can induce the feeling of owning a plastic hand. And this kind of effect is further investigated by Erson Lab in Karlinska Institute in Stockholm, Sweden using replacing visual feedback through virtual reality headsets and cameras. So in this sense, in this example, they were studying how owning a plastic body, so using the same technique of touching the two feet at the same time would affect the perception, for instance, of the world around you. So yeah, the size of your own body determines the perceived size of the world. So it's interesting that with that, it's basically that your senses don't operate independently of each other, they affect each other all the time. And so from that, you can hack your body's perception about itself and override your own sense of body ownership. So we see also another event lab in Barcelona led by Mel Slater, and they've done some really interesting research of what that means also socially, how you can change your perception about stuff. Using embodiment effect and in this case virtual world, they've seen that if you put a white person in the skin of an avatar with black skin, that actually reduces their implicit racial bias. So this is not the kind of bias that you say you have, but it's the bias that they detect through some sort of rigorous testing afterwards. So this is really the inspiration behind the project, and we decided to take this and put it in a different context, put it in an artistic context, investigate the ethical implications of it and other possible uses. So this is an early slide or an early picture of how we started out. We're using like very low budget stuff as things we bought from China, virtual reality glasses from the 90s that we kind of got from the university somehow, and really just started experimenting with these ideas and experiments, trying to recreate them and see what would happen. So the kind of first system that we have been using, there's been further iteration after that, was kind of like clearly inspired by the research we're showing before. Except that we tried to replace a virtual environment by sort of a human element. We're not using virtually generated environments through computation, but rather using another person's sensory motor contingencies to transfer the body. So here we're using the Oculus River, we're using other headsets before, which is sent to several mounted camera, which follows the head movements, and the performer will be copying the movements of the user, and we use identical objects and identical spaces so that they can get appropriate touch feedback. So really what we're looking at here is how to use this in, how to develop this project in different contexts. So one of the first places that we started doing this was in Les Trucs, which is a performing arts center outside of Barcelona, and they have an interesting association with the Indignados movement in Spain. So there's a lot of different community groups and activist groups within this place. So the idea was to find a methodology that would allow us to investigate this without hierarchy. So we're not just saying, oh, we're the technical people and we're going to tell you how to do this, but rather say, here's how the system works, here's the technology, here's the scientific aspect. What would you use it for? What kind of stories or situations would you like to explore using this? And so working with anthropologists and community organizers, women's groups, dancers, hackers, a bunch of different people, we tried to kind of create a bunch of different experiments and situations and see what would come out of this. This is one of the first ones. Here you see basically the two identical, oh yeah. Here you see two identical spaces, identical objects, and then as Arthur was saying, the user will reach out and pick up an object that they see. The object is there and at the same time it's in the other space. So they can feel that and then at the same time this is associated to narratives. So we would have a prerecorded narrative of the person that they're embodying associated to different objects. So you can branch the narrative then in sort of embodied theater kind of way in non-linear form. So that's really the first iterations that happened there. Other experiments we did, maybe you want to talk about? Yeah, so that was Sarah and her mother Anna who's an artist and they reached out to us and talking about the kind of difficulties that had in their relationship and they were trying to figure out a way to do some kind of performance around that. And Sarah made a very beautiful story about, it was very poetic and very interesting metaphors for such a young kid. She was talking, the story was about the girl with the red tears, with the blood tears, and talking about through the story about her difficulties relating to the other people at school and her mother was, so and once she was telling the story she was drawing it and her mother was kind of like copying the movement to get this embodiment effect into the world and story and feelings of her daughter to hopefully get a better understanding and it worked in both ways, I think, both as a sort of self empowerment tool for Sarah and Anna together like a sort of insider connection into that story. So it's really like the kind of experiences that people, I mean we tried the experiment and it was like oh this is interesting but once we started sharing it with people and they were creating their own experiences it became, it seemed very powerful, they came back weeks later and were like wow that's amazing that she's doing better in school we have a better relationship now and this kind of stuff was really what motivated us to keep on working because we weren't being paid, this was entirely self funded and we're like okay well let's see what other things we could do. So one of the other people we worked with was Yusefa who was an illegal immigrant in Spain who had come over illegally on a raft and had been living in the street, had learned to read, learned to write and eventually became a dance instructor and he wanted to share his story with people in the local community where he lived. So we worked with him over the course of several weeks and figured out different protocols and recorded some audio and had different objects that people could interact with and then we presented this as a public performance for people in the town where we were and it was amazing like you see like this is really I think what became interesting it wasn't that the technology was like providing something that was needed you know like you don't need the technology what it is is it provides a context where you're in this liminal space you're not quite yourself, you're not quite the other person and what happens in that space is the starting point for dialogue afterwards that's really how we try to use this technology like whatever video game equipment and stuff and bring the folk co-opted and bring the focus back towards something human when you take when they take it off they're actually facing them the person that they just were and before that they were just interacting and experiencing themselves from the perspective of the other person and that's really what seems to be a powerful experience after taking it off the people who don't know him you know he's a big tall guy you know or like just wanted to reach out and hug him and it was kind of these sort of experiences was kept on confirming to us that that we're sort of onto something there yeah we've been also working with an integrated dance company which featured both people in wheelchairs and able-bodied dancers and through exchanging with them one of the suggestions that came out from wheelchair users what they wanted to see themselves dancing standing up which was like kind of a thing I mean nothing would have come to our idea to propose that because it seems kind of weird but after that we developed some kind of choreographic language where using his hands he could there was an improvisation happening with a dancer which was used to create to get to add the movements of the rest of the body and so they could see themselves dancing through the camera of the of the person standing up yeah there's all these videos online we don't really have much time to show them now but you can just look online and find documentation about these projects but one of the interesting things about this is how it relates to say if we were doing this in an academic institution or scientific institution it'd be very very difficult to get an ethical approval for this because like some of these people had never walked what if they had a traumatic experience or they became depressed afterwards you know it'd be very difficult to get ethical approval to run this kind of experience or experiment but when you're working in artistic context it's a lot more flexible and also it allows for this kind of horizontal collaboration that's more egalitarian you don't you're not they're not your research subject they are co-creating the whole experience with you in that process and then yeah we this this video which kind of brought us a lot of attention like quite unintentionally I should I should say we just posted the video and it and it went viral it just happened when we thought the project was actually about to end because we were all so we all working together in Barcelona was four of us at the time and we all that went to different country to to make a living but in the meantime we a lot of people can start and reaching out to us to after seeing this video and it kind of served as an introduction to all the work we had been doing before and it yeah it also happened through what the idea I think came out of the suggestion uh some people yeah we were sitting one of the open open because a lot of the stuff happened during this open hack labs and so one couple came to us actually they're two friends I think male and female and like oh we really want to kind of explore each other's bodies can you just give us the equipment and we'll you know go off into the room on our own like yeah sure okay no problem and then they came out afterwards and they were like wow that was totally amazing like okay so how can we so thank you so the idea was so we thought okay how can we expand on that so this setup is different than the other one rather than having a user and a performer you have uh two users basically they're swapping perspectives and in order for the embodiment effect to work they have to stay in sync so that means that all all the actions that they're doing have to be agreed like kind of non-verbally and it creates like this interesting dance of agency and consent you know where every like if you know I mean you know whatever the stereotypical thing that you might think is like oh well I breastnut I can go like this but I mean like if you do that and they don't want to you're not going to see it you know you're just going to be an asshole so it's so the idea is like it just changed the whole relationship then between the between the two people who were experiencing it and uh and yeah that was that was really interesting so we got loads of attention from this like unexpectedly um we didn't even promote it just got found on somewhere and yeah and then the problems that we had to actually show it in installations yeah so they invite us like do an installation now it's like we don't normally work in installation but we did we started developing stuff and we've been invited to a bunch of different events so we went to some uh medical hacking day thing and started then really because lots of people from different fields contacted us like cognitive science researchers performing our people people involved in conflict resolution all this stuff and so we started saying wow actually this idea this concept this sort of stuff has applications and lots of niche fields how do we expand this so you know of course it's open source and stuff we thought well let's just just share it and try to go to all these places where they're inviting us and see what we can get so this is one of the experiments you want to talk about this yeah that was some um sort of uh pre-experiment we're trying to see the design the system could be used in this way uh it's kind of like a replication with our system of um the virtual virtual environment therapies for burned victims where they they found out and it's actually used in hospitals sometimes uh that for severe severe burned victims when you put them in the video game where they like throw snowballs at pink winds and polar bears it kind of distracts them from the pain like from pain that's chronic pain that they cannot get rid of and with this kind of distraction work so we we have been looking at this kind of um so we put we we measure the pain tolerance with the hand like how long you you you can stand with your hand in the bucket of eyes with controlled temperature with and without the the system so doing this kind of respiration exercise focusing on your body while being another person uh it seemed it seemed the the semi control the result environment results seem to indicate that there was a positive effect yeah it was in collaboration with two different uh neuroscientists from different universities this is like a pilot study so in addition we also went of course to some different art exhibitions this is in Einhoven that uh queer art exhibition called gender blender and then we realized that showing the art in these contexts provides us an opportunity to do to do user testing in a way you know like you're exposed to a much broader section of the population than you would if you're in a research institution where you're dealing with mostly 20 to 30 year old white males with a technology background so if all of a sudden we're being invited to all these places we thought well let's go and show it and really be there and do it as kind of a technical and anthropological research at the same time um so this is this was some workshop at a retune conference in September in Berlin where we had a workshop with people showing him how to use it uh yeah and then there's a series of pictures with uh this is in Sao Paulo in Brazil as well uh this is in Spain again working with dancers this time there's lots of different iterations of the project that we've sort of gone through but there's more documentation online uh yeah this was interesting this is uh one of the people who contacted us actually after the video went viral uh it works in the united nations and the alliance of civilizations which is a small kind of department that funds and research's experimental projects and conflict resolution so he'd actually had the same idea using neuroscience embodiments virtual reality storytelling performance to address conflict specifically but he hadn't found anyone to work on it didn't have the skills himself so when he found us he was like wow and we're like oh cool this is amazing so he just said yeah join the team you know like it's you know it's open so since then this was a project that we we sort of participated in it was a retreat for Somali diaspora storytellers so that would be journalists and artists and poets and writers and activists and documentary makers uh looking at intertribal conflicts so this is of course it's a tricky situation you go there and you don't want to be like oh we're the white european guys with some technology that's going to fix all your problems you know you have to kind of go there and say all right we just have this stuff we've been working on what do you think of it how would you use it what's the context how do we create a context together that this might be useful maybe it's not useful but we you need to try to have this this dialogue so i think that's really like where we're at now um we're you know we've been invited to mit for a year to do some research there as well uh we're also got invited to two events in uh israel and so we decided well we're going to israel let's see if we can connect those two events and stay in israel for a month and then we're like well if we're in israel we might as well try to go to palestine as well okay there's no funding how do we do that but let's run so with the ideas we're going to run two concurrent uh simultaneous sort of workshop development periods with members of the community activists artists hackers in israel and palestine and tried to network these people and interconnect the results across the border as well looking everyone with the same common goal of like how do we address conflict and just see what comes out of that maybe it's to do with the machine maybe it's not but using it as a tool to really build networks of people who work and collaborate on a horizontal level and find the tools for communicating in an interdisciplinary manner is really what we're what we're interested in doing with the future of the project so yeah we have we have like we're talking about before open open source platform we have this in three printed hardware that was designed for us by fablab Barcelona we always work with really low budget technology so this is this jacket you're seeing is a bunch of three printed pieces uh PlayStation cameras and Arduino in the back the really most expensive piece of hardware is the computer in it i guess and uh and the alcolos and um we i invite all the quarters in in the room to check out our github and reach out to us if they have any idea how they would like to contribute there's a lot of things that we need help with uh on the on the technical level as well and um yeah i think yeah that's um yeah i guess like also just thinking about this sort of uh context and way we're working we're also interested in other people who are maybe working maybe with different projects but also an interdisciplinary manner like how useful is it like to design stuff like if you're working with like pgp adoption or something like that you know working directly with users and stuff are people in the field to actually co-create these tools and methods of usage maybe there's people working on that here maybe there's stuff that we've been doing over the last few years which can contribute to these kind of things in terms of formalizing a methodology for kind of collaborative development of projects that are interdisciplinary um yeah i think that's that's basically it so thank you very much yeah thank you christian and atha the good news is we still have like 10 minutes for questions and answers so please line up at the microphones here in the hallway um we start with a question from the internet hi um the question from the internet is more related to um yeah how you can use this technology also in other fields like would you say you could help people with body image issues like anorexia or something to recalibrate this perception of their own body that's part of the thing we have been talking with some uh neurologist that's working exactly in that uh at this um hiking medicine event in Madrid that was a possibility you could do that in virtual reality right now the way it's done is to assess if someone has a bad badly assessed body image is that just show a bunch of pictures to a person and um and on every picture they say oh yeah this is bigger than me thinner than me bigger than me but you could be easily imagine doing that uh with with someone else right and giving like all the agency and like that really the the the strong thing about the the machine is that we're using the real world and you the it's actually i think so far it's uh strangely my thing it's it's more convincing to use a video feed with like uh some movement with maybe some approximation and some latency than just a virtual environment uh it's easy to imagine how it could be better quickly but then the the flexibility of of using just really simple technology and the and the real world is i think allows us to to to explore a very quickly different thing and that was uh something we uh and we possibly yeah we actually met with someone who found the project and was saying i've i've been on rex for 20 something years and i would like to come and meet you so they actually traveled to meet us at one of the events we were doing and we're able to try the system and make some recommendations i mean you know we said well we're not you know doctors we don't we can't promise you anything but like no no i just want to go and and participate and see what i can you know kind of offer in terms of that so there was like there has been some initial kind of dialogue about that and working with people you know patients and this kind of uh horizontal sort of uh structure did you know any medical papers on that um medical papers on anorexia and virtual reality yes um there could be uh i can't think off the top of my head but if you want to email us or whatever we can talk later as well all right thanks for the input next question for microphone three yeah hi um i've read that there are sort of two different ways that different individuals process uh visual input and reconcile it with how they're you know where they are in space and that uh one of those modes makes people more prone to things like motion sickness from virtual reality equipment and it happens that those things break down across gender lines it's more common for women to be uh motion sick from vr technologies and i was wondering if you have seen much of that in your observations of your project if you have um many women involved uh working with you regularly i know you commented about you know the typical white dudes in technology problem um so if you could speak to that i would find that really interesting yeah um in terms of uh we've not necessarily done a a quantifiable analysis of everyone every participant that we've worked with i've worked in this field before as well and yeah you're right there's differences in how this is perceived by people also across age as well i was working in uh vr rehabilitation for people with stroke which is mostly elderly as well um so yeah there's definitely a big difference um in terms of motion sickness i don't think we've observed any kind of effect that would that that's that's interesting i didn't know that was that was happening do you think that that could be related to the fact that you're actually you know coordinating with another person and so that the timing maybe works out better or is it just um you have no idea because that's a totally valid answer there's there's there's there's some very technical stuff about why um we haven't had too many issues with motion sickness we actually cut off uh we're changing the orientation of the camera so you have greater up and down orientation and less side issues so that there's uh there's less motion sickness because it's not your entire field of view that's that's covered in this case um so anyway but uh all right thanks next question for microphone two well all right yeah this will be a question about the setup we've seen in the slides um it looked like there was just one camera essentially did you get a proper stereoscopic view enabling death perception on the oculus we have been working with a bunch of stereo cameras uh is it the setup we were talking about yeah this is this is mono yeah uh we just duplicate the signal but on yeah one of the issues with using stereoscopic cameras in an installation setup is that you have to kind of calibrate per user with interpupillary distance so it makes it a lot simpler if we just you know for getting people in and out or whatever it's chaos then you know it's easier we do have some that's like a mechanical way of actually adjusting the cameras and then you can also do it with software too but we don't generally use that in whenever we do on yeah on this one for example we we had a bunch of stereo cameras this one was trying to build our own was a bit complicated this one we're not available are not available anymore but we have we've had some yeah it's it really makes the immersion stronger of course especially in close quarters when you can at your hand and your body it's like sort of less of a difference in like after a couple meters but uh yeah it's actually important definitely thanks okay it seems to me that there's another question from the internet yes there is so how well do you think is the immersion controllable by the user what's like the risk of abuse for this technology like for torturing or anything in this direction I think there's that's one of the main reasons that we're interested in kind of taking the project ourselves to these different contexts like we've been to art things we've been to documentary we're now at MIT which you know doesn't have a very ethical record in a lot of ways and it's like it's interesting to go to this context and kind of sort of bring the ideology that's kind of brought the made the project what it is so far into these contexts and see how we can sort of keep that going I think there's obviously uses for torture with this or whatever but I mean we're not so worried about that I think the co-option that could happen is that you know you have this like as a disinversive experience that you have on your own really what we're focusing on here is bringing creating a context that brings you back towards dealing with someone in reality you take off the the gear and then the experience is really then the meaningful part happens in that in that way that's really what we're looking at is creating a kind of a social context with this is just the starting point all right next question for microphone three how do you measure increased empathy and what kind of methodological issues have you seen arising with that yeah we haven't actually started quantifiable empathy is yeah that's not something that we've been doing so much at the moment we've been working at setting up some different collaborations with some other universities doing comparative studies with this and other things with regarding virtual reality and empathy now the environments we're working in are quite chaotic so having you know a measurable scientific it doesn't really work so there's methodologies that we can use some of them don't necessarily work but some of the things we try here I think are useful to bring back into maybe a more clinical environment and investigate there that's one of the things we're going to be doing while we're in MIT for instance is trying to you know make a more rigorous setup of some of the stuff we've tried in the field and the question from the internet yes two short ones actually like one would know how along the longest experiment with two people's thinking was and the other is whether there will be a demo or a workshop with you the next day actually yeah good question because we forgot to mention we're are running a workshop here but we haven't been able to book it because the wiki was down so if you want to check our twitter then we can we will have that organized at some point I'm thinking tomorrow or at some point but if you see us around and there's people who want to hack and you have a space already at your desk we have all the gear here already so just let us know we can get stuff going in terms of the other question which was I forgot I didn't understand the first part of the question was there I didn't get the question the first part how long the thinking of two people lasted yeah I think the longest one we did was we kept one of our research one of our team members in there for two hours I think but it wasn't with one other person in this case it was lots of other people swapping with him and he didn't take it off the whole time so he was quite disoriented by the end of it all right since you mentioned your twitter account maybe you want to go back to the last slide so people can note it down and I think this would be the very last question for microphone too okay thanks my question is about you said this is more like an art project of yours but you already mentioned some universities so are you aware of some scientific research already in the making right now is your setup or are there already papers maybe we have some people asked to explain them like how to replicate it inspired by like kind of research by Erson lab in our own but it's it's still in them it's still in the making there's other research that's similar and in some ways like we mentioned some of it there in terms of using our setup specifically we the first one that we're working with is MIT there's other universities that we can't really mention yet because it's not official so but we're already in talks with researchers within there who are seeking funding for having us there and developing the project and using it as a common platform maybe for sharing research between institutions as well since it's open source or whatever and if people are making some advancement with the interface or with the technology and one thing it can be fed to the broader open source community that's one of our motivations for working with these large technical and scientific universities as well all right thank you so much also for taking the time for q&a and thank you for this super interesting talk thank you so much for being here thanks