 And I'd like to welcome you to a talk titled Media Disruption Led by the Blind, which is by Daniel Sabio. Daniel is originally from Puerto Rico. He's now a new media artist and other things, what you probably will talk about as well, in Berlin. And one of the topics he will be talking about is Infinite Observer, which is the first narrative VR experience designed from the ground up by the blind for the blind. Hi, I'm Sabio, or the glad scientist. And yeah, there's my full long name. And yeah, I want to talk to you about some of my projects. But first I want to try a little experiment with you guys. So I'd like for everyone to close their eyes. And as we all close our eyes, I would like for just 30 seconds to listen. And listen deeply to the room around you. And as you're listening, create an image of the room around you. Create an image of how many people you think are in the room. An image of all the people who may be walking into the room right now, where they're coming from. And just do this. Now you can open your eyes. And as you open your eyes, imagine those 30 seconds as 30 minutes or 30 hours or 30 years. And then imagine this as your status quo perspective of the world. Now imagine the thousands of years spent attending to visual persuasions and how you'd personally interpret everyday computing with this newly gained perspective. Hold on to that for a little bit, because we're going to go on a tangent. So real quick, I just want to give you a long story of my life. It's basically like three conversations. So permission is granted. Please, you know, if you feel like you need to interrupt, you can do that. So when I was five years old, I told my mom I wanted to make video games. She was like, okay, whatever. I was just super into duck hunt. And, you know, 10 years later, she's like, what do you want to do in life? You know, it's that time. I said, I want to make video games. So that led me on a crazy path. And I ended up going to Georgia Tech, studying computer science. And then I ended up in Barcelona, met this guy who changed my life, who I literally saw in the hall earlier. And I said, hey, man, you changed my life. And he was like, yeah, okay. But basically what he taught me was that I could make art with code. And that literally just blew my mind. And I realized that everything around me was code. And that everything around me was, you know, manipulatable. And everything could be an art project or an art, anything, whatever you want to call it. And it was like this crazy achievement that came unlocked in my life. So I spent about eight years after that. Didn't really even care about video games. And just wanted to make stuff with code, installation, art, VJ, projection mapping, blah, blah, blah, all that stuff. And then, you know, we're talking about VR though. So then I realized one day, oh yeah, there's this immersive art that I'm making all the time. I'm making this crazy projection mapping stuff. There's all these logistics involved. What if I just make a VR experience and put people into this? Wow. And it hit me so late that it also blew my mind. And then I'm also a musician. So when I got the VR headset, I wondered if there was a way to make music in VR. So I figured that out. And I laughed a lot while I was doing it. And I performed on stage a ton doing this. But that's not really what I'm talking about. As I was doing that, people would come up to me after every show and say, hey, you make music for VR, right? I'm like, no, I make music in VR. But I realized that, hey, I had to learn a new skill set because people are going to continually ask me about this. So I quit my job. I moved to Berlin. There's a bunch of stuff about how to make stuff for sound in XR technology. If you're interested, slides will be up, all that good stuff. And I moved to Berlin. And I took a course with someone who's actually sitting in the audience and learned all these different technologies about producing sound in a spatial audio system. And I had this question of, how does a group audio driven VR experience look? So I created an experience with a group of amazing people where we had four people blindfolded. And they're in the Monome space, which is in Berlin. And it's a 4D sound center. So there's spatial audio. And you can walk up to sounds. And they exist in the space around you. And you can really interact with sound in a different way. And it was a good opportunity to take people into a sound focused and tactile focused VR world. So I did that in that setting. But the issue with that setting is that it lives there. It lives in Monome. And you can't take that space anywhere else. But what happened during that, and lots of gratitude to everybody that I worked with, is that I realized something that I just kept on thinking about what it feels like to be blind and how I could never really fully empathize with that experience. And so I want us to close our eyes again. I want us to just try one more time. But this time, as you close your eyes, I want you to just feel. I want you to feel your fingertips wherever they're laying. I want you to feel the bottom of your elbows, the insides of your knees, the bottom of your feet. And I want you to see if you can detect vibrations around you and see if you can feel the actual reflections of those vibrations in the room. Now open your eyes. And you can imagine the same things again and again and again that these are minutes, and these are hours, and these are years. But at the end of it all, what we come to is that that's our perspective from our limited experience in this and that we need a deeper perspective. So people often tell me what VR is. People in the VR community try to shove things in different places, say that VR has to be this, VR has to be that. And well, honestly, I'm kind of like F that noise. There's a lot of different things, and we're creating our own paths in this world. So it's not really virtual reality or visual reality, it's virtual reality. So if you think you need a headset necessarily to create a virtual reality experience, maybe we could think about it in a different way. And I came back to this question again and again, and what I realized is that it transformed into what would a VR experience be like for the blind? And can this experience actually challenge visual culture? And that led to the creation of Infinite Observer. So Infinite Observer is a VR experience designed for the blind. And as I started this project, I thought, okay, I'll make a VR experience that's for the blind, but also can be experienced by the sighted as well. And people in the VR community are just sitting there calling me crazy, non-stop. And I'm like, okay. So now we're gonna get to a lot of slides with just text because I actually feel like I want to embody more so a less visual tone for this part. So there's a lot of research to do. As soon as I really seriously decided to pursue this project, I went to, so there was that realization. I went to actually live with a blind couple in the Netherlands, Janne and Tina Visser. And they run a foundation called the Walk-In Foundation and they actually bring people into a dark experience, which we could say is a multi-sensory enhanced version of the glimpse that you guys participated in during this presentation. And so they have this multi-room experience that you go into, there's literally no light. Your eyes try to search for it for the first few minutes and then they literally just get tired of firing the cones and rods or whatever that is. And then you give in and you start to use your other senses and you start to discover all of the powers that you have beyond just sight. And so I spent about five days living with them, just night and day, really learning so much. And what I ended up learning from them was, first of all, these people are incredible. Janne and Tina were both fire walkers, teaching people how to walk on coals. Tina was teaching people ancient songs for years and Janne actually was a hacker. He was building his own ham radios and has built multiple Linux machines and also has his own distillery in his house, which his liquor is really good. And as they shared their experience and their blind experience with me, I also shared with them a ton of VR experiences, which I thought, okay, I'm gonna take the best audio VR experiences, I'm gonna show them to them, they're gonna have a response, they're gonna say, yeah, this is good, yeah, this is bad. Honestly, there's zero UX accessibility for people who are blind and VR at all. There's like nothing to do with voice menus, there's no thought whatsoever. And yeah, it made me pretty sad at first, but I also realized that there's a great opportunity to make a change. And I think that as I was with them, I really got inspired for the narrative of the piece as well. Because of the background and as we were talking, they were saying, well, what if we had this group of underground hackers who are interested in overthrowing the dominant visual culture? And that seemed to resonate a lot. So yeah, there's a little reminder. So that really sparked the story building. And this is a, oh, you can barely see it, cool. That's fine. So this is the basic image that sort of led our guidance in that it's smoky. There's an unknown figure. And you're welcomed into this group of hackers who want to subvert these dominant forces of society and they wanna use vibrational forces to do so. So instead of typical ways, they may use echolocation to navigate. They then use molecular vibration to actually teleport. And then they use by mapping the tectonic shifting because they're actually underground as well. They're able to detect movement above and also detect things which sound can actually disrupt visual vibrations in the visual spectrum. So these were some of the things that we talked about and actually became parts of the game and were able to implement already some of these into the experience. So as I mentioned, echolocation, we also have a frequency matching gun where you're able to hold it up to different materials and match a frequency. And that's able to explode, open up new pathways in this underground world. And really what the narrative is about is kind of telling a meta narrative and how can this actually include a real society of blind hackers potentially through just this experience in this art. So how can we take a site-specific installation, for instance, that takes place at a specific festival or at a specific community event, focus on the blind and have this merging of, okay, at the end of the experience, someone in the narrative is inviting you to join this group and as you leave the experience, maybe you have a text on your phone with a link to a website that now is accessed to this community, these sorts of interactions, and then how is it feasible to create an audio only group community platform that would be easily navigable by the blind? So these are a lot of the things that we're thinking about in this project now. So the technical side of things is pretty interesting. So I teamed up with a audio designer who specifically specializes in spatial audio and he works at Magic Leap and with him we've been building out the audio environment and so we're using Wise, which is a popular spatial audio sound engine for unity and unreal and because of the plugin that they have, it's able to actually detect the real-time game geometries and create realistic reflections of the sound in the space. So as we test this, we're looking very strainlessly at how realistic this sounds for the blind, like is this passing the test? And so far it's been questioned and we're wondering, is technology not there yet? Is this something where we have to continue deeply researching more and more and maybe use some cutting-edge research institutions to reach that point? And yeah, as I mentioned, we're working in unity and originally we were focused on deploying it to headsets but I think that with the small enough market of the headsets that blind users actually have access to, it makes more sense now to look at both PC users as well as mobile users and in using some of the Wise plugins in a game-edge and setting, it's easy to be able to deploy it to both and be flexible in that way. So those are a lot of the different things that we're considering and can an audio version only exist? Can we create custom solutions that can travel, that maybe combine all of these into installation formats as well? These are all different things. If you haven't noticed, this is an in-progress project. And so this is the team, that's Yanni and Tina and that's Kedar and this is Chris who's been guiding a lot of the interaction and creative along with me. And really the goal of this project is to empower the blind and create a heightened appreciation for our senses beyond sight. And through the narrative, hopefully it will instill some clarity around the dominance of visual culture and the ways that all of these tech giants have been using visual culture to greatly manipulate people and it may inspire some new ways to circumvent this. And in a bigger sense, I would love to create a voice for the blind as we develop new technologies. I really do see immersive technologies as being like the next big way that we interact with computers for the next 10 years. And I would love to see this be potentially a wake-up call or along with a slew of other projects be a wake-up call that this needs to be done ahead of time of projects coming out, or sorry, hardware and these cycles coming out. And then yeah, creating a community for interaction between real life blind hackers to be able to discuss anonymously and even maybe practice some of these techniques together that are inspired by the narrative. Yeah, and I do have a video. So I'm gonna show that now. So this is a video of an early prototype. Members usually find us, track us through the echoes, but you, I know, you're wondering why we're all here. So let me break it down for you. We have never had a voice. We're always just an afterthought. And they think we are powerless. And until now, we believe them. But look, that's not a problem. That is our advantage. They've made us invisible, so they'll never see us coming. We'll start with the US by finding their single point of failure. We can break down the greater infrastructure. Look, the US controls the entertainment market. If we are able to infiltrate the system and corrupt it without detection and insert our beacons, we can start to find others like us and accelerate our bigger initiatives across the world. So that was the video. So yeah, I'm happy to open up to questions now. And I actually have a few questions for you guys because I know that probably, just say all of you are smarter than me. So I'm very interested to know if anyone has experience using sound or noise in the context of hacking and what people think of the community extension of the piece. And if there's a research out there that could possibly help this project that I'm not aware of. If you have a question, please walk through the mic. One of the microphones. Number two? Have you given any thought to the fact that currently as far as I understand you need a big installation, right? So you cannot have this experience at home where if you just, you know, if you do, if you're visual, you can just buy the headset and that's it. Have you given any thought into building a audio VR headset? Yeah, so this is something that I was trying to actually talk to Oculus about doing and basically extrapolating like the sensors that they have into using it for head tracking because when you're in like an ambisonic world the main thing that lets you interpret it as a 3D audio is the sensors being able to know when you're turning your head or when you're moving within that space. They kind of just brushed me off and didn't give it much thought. But it is something that I'm considering and also that's kind of why I mentioned looking towards like some AR focused libraries where it has the same spatial audio features and you're actually able to scan a room with your phone and that may just be a more readily accessible to where, okay, someone's wearing headphones and they're able to scan their room with their phone and then get audio cues and also have voice recognition where they could guide their character through the space as well. Which is something all of these people are super familiar with is talking to their phone and having those interactions. Thank you. Okay, microphone number one. So thank you very much for doing this and for the talk. It was really interesting. I have some experience of listening to 3D noise, 5D music. They call it as well, which is easily done with Ableton in a normal headset. And my question to you would be, since I think it's not developed yet, you don't have a final product. If you ever thought of doing something that actually blind people could learn how to listen to reality in a better way, increasing some ways to pay attention to some kind of noise or feedback that would only require a normal headset because closing my eyes once and listening to the sounds of the guy shaving your head like the VR experience of that, it's quite amazing what you can do with normal headsets. Yeah, that's a really interesting idea to have like an educational element in there. Yeah, that's super cool. And I love that you brought up the binaural Barbershop example because I actually showed that to Yanni and Tina and it was one of their favorites. Beyond all the VR and ambisonic stuff, they're like that one. It was crazy. So binaural definitely has its place and even talking with K-dars, like this expert in spatial audio, they've been trying to explain it but there's some psychological phenomenon with the way that binaural works with our brains and it just kind of psychologically fakes us out in a cool way. Yeah, so thank you. And if you never watch the Barbershop for audio experience on YouTube. Again, microphone number one. What kind of equipment have you used for haptics or have you played around with haptics? Yeah, so all that we've used so far is just the basic Oculus Touch controllers. I've applied to a couple of grants with haptics research teams to look into that more. We were going to talk to HardLight and then they went under. What's HardLight? HardLight is like a full suit for haptics which also has its pluses and minuses, especially in an installation setting, like switching over between people that can be pretty crazy. But something that I'm looking into is something more simple. It's like a wristband that creates basic vibrations that actually send impulses to all of your hands so you feel like you're feeling things. And then the ultrasound stuff looks super cool. Or do you have experience working with haptics? Yeah, I played around with haptrators. They basically create an illusion on your skin that you feel like something's, like for instance, dragging you through the air. There's a psychologist who's working with visually impaired people called Vincent Hyward. He has a lot of really crazy experiments. Basically, it's like a vibrating motor that it's hard to explain, you have to feel it. But if you hold that thing, it just kind of drags you through the air and you can do a lot of really crazy stuff with it. But yeah, just Google Vincent Hyward, he's a crazy guy. Awesome, thank you. Before we switch to microphone number two, there are questions from the internet. Oh cool. Take my angel. Hi actually, some results can't really simulate an entire career with this special patient. What are the consequences? Yeah, definitely. That's why ideally, this is an installation-based experience where you are able to integrate even real life objects as haptics and also scent, possibly taste, which is something that I did explore with the blindfolded experience, and all that I'd love to bring into the installation format. But yeah, a lot of it is technology becoming more accessible, like with the haptics. And the more that those devices become more prevalent, the more that they can kind of enter into the work. Microphone number two. Hey, thanks a lot, a really inspiring project. And I would really love to hear a bit more about the collaboration you have with the three other people. How do you work together? Are you in one place? And can you share a bit more of these experiences? Yeah, definitely. So we had the initial idea, which was kind of in my brain. And then I emptied my brain to Yanni and Tina. And we really started to go crazy on the conceptual side of stuff. And I talked with Kader about it, who had made a VR experience for the blind a few years ago. And then we started developing. He lives in San Francisco. And so we would work over the internet together. And then Chris, I knew from living in Atlanta and pulled him onto the project. And it's kind of been me traveling to all these different places for really intense hackathons. And then us working remotely as we can. The next question again for a microphone number two. Because you asked for research that could help you with your project. You probably know, but just to be sure, there is an existing community of audio games and people who make those. And there's a researcher in Vienna who's called Michael Urbaneck, who does research on them. That might be interesting for you. Cool, yeah, I'll definitely look him up. Thank you. Okay, any other questions? Sigma Angel, do we have new questions from the internet? No questions. Do you have new questions for the audience? I... I just want to actually get us to do this one more time. If you guys are up for it. So this time as we close our eyes, I want us to really take a moment to imagine ourselves having been this way before as we have experienced this before. And I want us to use feeling and listening, and scent and taste. And to try to create an imaginary world around us and to have that feel as real as possible. Thank you so much for your time and all your everything. And enjoy all of this.