 So good afternoon everyone. As you all know, this is going to be an interesting talk for sure. It is titled Sense Without Sight. Our next speaker is Sai, and without too much ado, please give him a big round of applause. And he's going to be here shortly. Sai walks across the stage, stops in front of a jacket, and in front of a cane. I think that I am daredevil. I'm not, sadly. Probably better for my health. And I do not have any magic powers, unfortunately. I just have extra disabilities. He produces a cane and puts it onto the ground. Nevertheless, one, two back, yay, live demo. Feels the floor. Where did that go? No, I'm not going to fall off the stage. You're going to be falling soon. Picks up a second cane from the floor. I can walk around without my cane. It's a little hard to precisely target it. But as you can see, I don't fall off stage. So just to make this clearer, I am not faking this. And the way you can tell is I have a blindfold on. This blindfold is for real. This is a sleep mask. I can't see anything right now. So you don't need to ask how good my sight is or why. Because I can't see. The issue that I have is light blindness. Basically, I can see if it's really dark and I can't see if it's really bright. And lights that are as bright as these lights here actually hurt my eyes right now through the blindfold. So I definitely would not be able to do this talk with my eyes open and my glasses off. I do a lot of different things. Blindness is not one of them. My shirt, my slides, my website. There's all sorts of stuff that we can talk about. Let's please talk about that stuff. Not the fact that I'm blind. Except during this talk or during my workshops when you're... It's okay to ask questions about that or talk about that. Exception is if you want to work with me on stuff or if you have feedback, I welcome that. But let's talk about something else, like how to analyze a few terabytes of court data. Anyway, I have assistants for this talk. One of them is what I'm going to be calling the helpful asshole. AKA my Harold. You're welcome. Come up. This is on the stage. It comes onto the stage from the side. Where are you? Good. And I have someone running the slides for me because obviously I can't see my own slides. Fortunately for you, that means I'm not giving the talk like this. What have I just talked about? He says turning around, looking at a slide. Make sound if anything. It's helpful because I can hear where you are. So you might wonder how I walked onto stage without my cane. I'm going to go over that in a second. But first, the cane is really, really useful. So it gives me a number of things. For instance, right here. He has two canes in his hand. There is a difference in the floor. There is this carpeted section here. If I just brush this with my feet, I can feel it. That's how I walked on stage. But if I do it with my cane, hear that whack? You can say yes in response to questions. Hear that whack? Yeah. So even if I'm lightly walking, I just keep that at a known degree and I can easily walk alongside of it, I can also feel the texture. So one thing you may not notice, but you may as well try now, is that the texture of the carpet under your feet, under your feet, the people who are live in this audience. I don't know about you people at home, sorry. But this stage I have checked out and it is smoother from the front of stage to the back of stage and is smoother from the right of stage to the left of stage. And I believe that's true for most of the floor in the audience, but it's made out of separate squares. So some of them may be oriented a different way. So feel with your feet, for real, like right now. Feel and see if you can feel which way it's smooth and which way it's rough. With the cane, when I drag it like this, it's smoother and when I drag it like this, it resists. And especially if I'm holding it like this with a firmer grip, this is nice and smooth and this way it sort of resists, it bends the cane a little bit. What else? There's sound. So something that I can do really well with the cane that's hard to do with my feet, but I can. I can do that. But I can also do this. Who waxed the floor with his cane? Those two techniques sound completely different, right? Changing between the wooden floor. In fact, walks to the back of the stage and tests the sound in these firmer areas. And then suddenly it's echoey. So that helps me tell just sort of what ground I'm on. Similarly, actually in the front of the stage. Sorry about your mic. The stage edge, it's really sort of resistant. Whereas there, it's got much more of an echo to it. One thing this is very useful for is echo location. I am not a bat, but it is the same principle. There are some people who are much, much, much better than I am about echo location. I'm not that good. But what I can hear and what you can hear and what you're going to try to hear right now is the size of the room. So I'm going to whack the stage with my cane. What I want you to do is while I'm doing it, close your eyes. You're not going to miss anything. The slide's not going to change. I'm just here. So close your eyes and move your head from side to side as I do it. And you can even sort of look behind you and listen for the sound of the size of the room. You'll be able to hear how large the room is around you. And you'll hear the echo of my cane from the back of the room. Kill the PA for a sec. He smacks the cane where it is time. Quite hard on the stage. He moves a bit around the stage and keeps hitting the floor. Okay. So that's a simple version. Another version is these stage curtains. How do I not face plant into them? Well, there's a few ways. For one, I can just run into it with my cane. Turns back towards the curtains. That's a speaker. Sorry about that. And that is curtain. Nice and soft. Brushes it only with this cane. There are several other things about this. I'm going to show in a bit. But also, if I'm... Sorry about the speaker. If I'm walking through this and I walk through the side curtain, sort of behind... There are speakers on stage who sometimes interfere with the sound. Psy walks past them. Psy is now in the middle of the stage between the curtains. They've added extra stuff to the stage. Notices something lying on the stage. A jacket. Psy walks on towards the left of the stage. It absorbs the sound when I am tapping near it. Yeah. There's... Where's that? Bit, where are you? Looks like you're lost. Do you need some help? Sorry? Do you need some help? Looks like you're lost. Psy, I made a few turns. Thank you, dear angel. No, I do know where I am. So... My dear signal angel, or my herald, is representing that person who is trying to be helpful to me several times a day and kind of annoys me. Or ranges from annoying to actually dangerous. I obviously know where I am because I'm navigating the stage. It's just fine and you saw me without even my cane. So thank you, but no, I don't need directions. He's in the middle of the stage facing the audience. Howdy. Someone has approached now. Someone who's right next to you like this. How can I tell that he's there, right? I'm not touching him with a cane, although I can. Psy has touched the other person on that person's shoulder. I can attack you in the ankle. Sorry, but that's how I can tell you're there. Especially if you're standing really quiet. It's really hard to tell that someone's there especially in a crowd like this. I forgot it. So yes, I'm going to hit you in the ankle. Too bad. That's how I can tell you're there. But in this distance, I can feel his body heat radiating. So from about this distance away, yeah. This distance, around 80 centimetres there. There's his shoulder. This is cheek. Psy has touched the other person briefly. So to turn to the person who's next to you, ask their consent first. I have asked bits consent for everything that we're going to do. What I'm going to do is put your hand up next to them. Try to almost touch their cheek without quite touching their cheek. Some people are doing this. And then deliberately do so. Psy demonstrates. With your eyes closed and then trade off. Yeah, go ahead. New interactions are sparing throughout the audience. Yeah. Psy is sitting at the end of the stage. He jumped off the stage. This is on walls. This wall, for instance, the front of the stage. There's actually wind coming through these little holes in them. That wind is colder than the ambient temperature of the room. If I get my arm really close to it, then at about this distance I feel the cold air from it and then at about that distance there's actually sort of a sheet wind that is really tight close to the wall and then there the hair is on the back of my arm tingle. And there I can touch it. So we have people He describes the relation to the AC in the stage. They're going to drive through the aisles. I want you to close your eyes and see if you can feel the wind generated by them moving past you. Sai is now sitting on the stage, on the edge of the stage again. There are people driving around with scooters and skateboards through the aisles to the back. So you feel how when they pass you there's a breeze in their wake. This happens a lot in Congress. There are lots of people on scooters and hoverboards and motorized couches or whatever. There's three people on scooters and one on a skateboard. Or someone just walking really fast like me. They all generate a wind in their wake. Another thing you'll notice in Congress there's these nice tunnels. So for instance from CCL to hall two there's this tunnel through the glass hall. When you walk through it the wind characteristics of the room completely change. You start feeling a crosswind so if I'm walking this way and the tunnel is on my right then suddenly I will feel a crosswind from the tunnel that is colder and wasn't there before. Similarly with the wall it just sort of stops. Next. So I walk to the back of the stage. Another person approaches him. Oh yeah, yeah, yeah. Yeah, so everyone please close your eyes. And I do mean everyone. Oh, not the audio angels. Or the video angels. But everyone in the audience please close your eyes. If your phone number ends with the number one please sign Niels in such a way as to make some wind. If your phone number doesn't end in the number one point to the person who's making some wind. I can't see you so I'm not going to judge you. There's flapping in the audience. So do you feel that? Yes. No. How about people whose phone number ends in one stop people whose phone number ends in four just make some wind like wave your arm in front of you feel that? Try pointing to them. Well, not the people who ends in four. You can point to yourselves I guess. People whose number ends in seven. With all the others some are waving, some are pointing some are just not doing anything. When people move by you it generates wind when there's a tunnel when there's an air differential like the air under this stage is colder and that's why there's a wind out of it. Actually do you know what I'm going to do? Si moves to the left of the stage where two people are waiting. Let me hold your hand let me get that for you. Ah, your text. One of the people touches his arm one of the people touches his arm his herald Oh yeah, more stuff. Si approaches the chair touches the border of the stage with the cane so they found the steps walking downstairs Si vanishes from a line of sight If I open this door and I walk past it there's this crosswind I was mentioning a second ago about the tunnel so here there's just sort of flat to my right then here I walk a little bit forward suddenly there's this crosswind from that opening Similarly when people were biking by you similarly if I crack my cane I don't know if you can hear that very well but try hear how there's there's a sound off to that side that sort of ends at the door and then if I knock my cane here there's a little reflection through that door and yes I do know how to close the door we should explain that we are sitting in a cabin above the stage 1 floor up and our line of sight is restricted so we cannot see everything that's going on I can feel the that I'm close to the wall or not tingly, not tingly what's next? I guess reached the left stairs walks them up slowly, steadily somebody approaches him ah yes so it talks to the person um uh oh yeah you've added more to transfer you uh no sometimes that would be helpful but at the moment not so much because if you were to move it when I've already found where it is um then suddenly it wouldn't be there anymore and it'd be hard for me to orient one of the things that is difficult about being blind is you have to have quite a lot of memory so the things that you probably don't notice because you outsource your memory is how much of the environment around you changes so as a blind person they probably hate construction work I do and if I try to go sit down for instance for that matter looking for chairs on the state touching a chair sitting on it really hard for me to take a seat he doesn't smell that bad but some people have a pretty significant smell referring to this person perfume or cologne or axe or something like that the helper is peeling an orange on state public transit terminal and you've taken the elevator maybe you know the aroma of the elevator in public transit definitely lets you know you're in New York City so next group start moving the helper eats the orange my assistants in the audience are moving through the aisles with something in their hands what I want you to do is close your eyes the helpers are moving through the aisles with concealed objects what is it point to them see if you can point exactly where they are yummy orange it's five people walking through the audience at various points always different objects so if I'm walking past a coffee shop for instance hazardous things coffee a drug store smells like alcohol and cleaning solution a clothing store smells like leather and this weird perfume that they put in clothing stores for some reason I don't know why but they do and if you ever visit one hopefully you'll notice it now so let's see next still people walking through the audience reacts to people walking by dear asshole you know your lines you have a special line for the smelting a bit four people on state do you have your blindfold on no I don't have please do I don't have an extra one also one of the helpers is looking for a blindfold to continue to the next part of the talk fuck it's bright sigh Boris his blindfold now the helper is standing blindfolded on state first off say would you like some help sigh and give the person next to each other don't just randomly assault people just because I have a cane does not mean it's cool to assault me then if you're going to touch them do so with the back of your hand this way he knows I'm not just randomly grabbing him and yanking him around so I can offer you my arm we can walk let's go have a seat they walk towards the seat together there's a row of seats on the left that they're approaching there we go there's your seat they've arrived at the seats some people will like grab the hand and put it down and said I can just say here's the chair sigh guides the hand with his arm towards the chair all the other person grabbed and tried to push down I'm not quite sure now they're walking back towards the middle of the stage it's a little crack it's a stiff interaction do you hear the back of the room where that is asks the helper the other person is now wearing a blindfold so one thing you really don't want to do when someone is possibly trying to figure out their orientation is to randomly come up to them and grab them and pull them dear asshole a second person approaches the two let's pretend moves past sigh the other person he thinks he wants to walk towards the audience let's walk towards the audience a little bit both sigh and the asshole are now handling the other helper bit pulling him in different directions confusion I'm pretty sure you are you look lost fuck off please asshole moves away for a few steps please point to the back of the room blindfolded bit tries to find the back but is pointing into the audience instead this is what happens when you mess with someone's orientation please don't be that asshole if you want to be helpful you can be helpful I need you back to the chairs now maybe like a hand they hold you you don't hold them you can leave them the blindfolded person is holding sigh by sigh's hand and they are now both moving towards the seats at the left again and the blindfolded person is sitting down having been guided by sigh you can tell this is improvised and here I am again walking without my cane the blindfold is now off again and sigh is moving back towards the center of the stage I'll show in a sec another helper is now standing near sigh but moving by him by them oh wow sigh can walk across the stage or talk slight says just doing normal things isn't inspiring or impressive I wish I were kidding but that does happen pretty regularly can we clear the stuff off the stage please so instead of that I'd like to show you something that actually does involve a little bit of skill emptying the pockets namely about a decade worth of aikido for those of you who don't know aikido is martial art from Japan Anna a helper is now entering the stage moving towards sigh years or so learning to use a cane Anna has spent well more than that she's doing aikido and is a second degree black belt I am not however I'm wearing a blindfold and she is not so what I want to demonstrate for you is how I can tell that she's about to grab my hand there are about five steps apart and Anna is now approaching sigh her hand but still about three steps away there the vibration as it sort of rotates towards me with a nice little fist attempt here there I can tell she's starting to lean back and there I put her to the ground now Anna has fallen or been guided onto the ground by sigh they're approaching again airflow, heat slow head they're close to each other both hands in contact crook of the wrist doing the motion slow motion and he again puts her to the ground and faster the blindfold hasn't come off this isn't magic again I'm not daredevil I just can't see I didn't get extra super special hearing powers my hearing is pretty good but it's no better than the average person I'm actually a lot worse in the rain so that thing isn't a really a thing so just to prove that it doesn't just work in this sort of deliberately set up a special thing Anna is going to just attack me however she wants and I'm going to throw her around a bit Psy has now their back turned towards Anna is approaching from Psy's side they are interacting and Anna is on the ground again getting up again focus on her Anna has approached Psy again they are again interacting in an aikido way Anna is on the ground again Anna is on the ground and grabs Psy's arm they struggle with arms to each other Psy levels her to the ground another helper approach trying as well Anna is on the ground now they are both attacking Psy at the same time struggling arms held they are trying to stay up but both are getting to the ground there is quite fast movement all over the stage while both are attacking Psy once again Antra is from behind and is again on the floor the other helper is again attacking and now on the floor again and attacks again Psy defends himself and on the floor the helper attacks again and defense moves back and forth on the floor Psy now both help us and Psy bow to each other to help us move away Psy is in the center of the stage no I didn't tell them what to do in advance that's real yes I am a little bit out of breath uh oh it's been a while since I actually exercise well enough and again I'm still facing you I haven't forgotten where this line is between the wood and the carpet and in fact I can knock it with my ring sometimes against the walls a little better we all were quite surprised with the fighting it was impressive so if you want to not be an asshole uh please remember first off ask I may not want your help or need it and what you think may be helpful to me probably is completely wrong so listen to what I say yeah I would like you to please lead me to X uh that does not mean grab my arm and start dragging me to X uh I prefer to follow people by sound so I'm just going to tell you to scuff your feet when you walk follow you by sound or to just keep talking uh some people to prefer to follow by hand a bit help approach from the side so the side takes the helper's arm or upper arm let's go lead me to stage center please they both walk to the center there you go tada not that hard um if you are going to touch someone a bit here unless you've invited them to attack you because you're doing an icute exercise don't just randomly grab them and you anchor them around demonstrates touch them say hi my name is sigh would you like some help hi sigh I would thank you very much touch the helper with the back of his hand at his arm are you lost thanks I'd like to go to your place thanks maybe later and one important thing to note when people do simulations of disabilities they sometimes try to sign heels by just pretending to be blind or pretending to be in a wheelchair or something like you sit in a wheelchair for a day and then you think oh it must be so hard to be in a wheelchair because my arms hurt so much and I couldn't figure out how to get to the second floor that's because you've done it for a day and someone who actually uses a wheelchair has done it for probably years and years and years and has much better arms than you do where everything is and how to get around similarly I know how to get around without using my eyes you don't if you just try to put a blindfold on and grab a cane or let alone those blind experience museums where they just put a blindfold on you and don't give you a cane and have you walk around actually one to one once and they want to take my cane away from me like hell no that ain't happening I actually lent it to my partner so that my partner could not feel afraid of the surroundings of like accidentally walking into something accidentally walking off, well not a stage but off a step or something so you don't know what you don't know most of the things I've showed you in this talk you probably have never realized that they're even a thing like have you noticed how many different floorings there are in the mess hall touch just the floor again how many different sounds there are smells, the wind the ceiling heights you probably don't pay attention to that I do and without the experience you're just not going to be able to learn that that said I am teaching workshops you're welcome to come to them if I still have the energy to run them one is going to be right after this talk and the rest to be determined my volunteers could you please come up on stage if you're willing to be on camera help us are approaching and gathering in the middle of the state if you ask the 35C3 content crew and the VAC crew and everyone else it's kind of a pain to run something like this because it's a stereo recording that doesn't affect you in the audience live because the PAs are mono but hopefully those of you listening at home you've been listening to this with stereo headphones on and hopefully you'll hear it from my perspective what it's like when I crack the cane and you can hear the echo in the room so thank you very much thank you very much take a bow please 8 help us on stage audience coming in 9 people are on stage taking the applause bowing, the legislators can't be there very interesting talk we also have a Q&A slide please so if you have questions and you're okay with being on camera and possibly being asked to come on stage so I can answer your question depending on what your question is to the camera that's on stage right and otherwise there's cameras or there's microphones we can also just play in questions if you don't want to come on stage you can just ask your question from any one of the four microphones in the room Si looks for help a bit microphone number two which one is two it's for the person asking which one is the sorry which one is the one on stage right you can ask them on stage let's see if they want to come on stage which number microphone number two is number two is the microphone you want to come on stage I don't know yet number two is in front of you on the island number three behind there number one left on the and the two is walking number four if you're willing to be on camera so the question is right here thank you thank you for the really impressive talk my question is the following I want to know how your imagination works so when someone says imagine a box what's that you think of is that the material or how it sounds, how it feels just how does it work well there I would actually go to my psychology talk that I gave it 27 C3 and a box is a category theoretic concept that has a prototype possibly it's like asking imagine a furniture hmm imagine a box, a mobile kind of box is a shipping box, is a box of mate that are those little crates that I walk into etc. imagine a box is it an elevator yeah I don't know how to answer that kind of question so maybe I got that incorrect but what I want to say is that when I try to think of something I don't know any idea any project plan or etc I imagine how it looks so how I am at certain place how I for example see a box if someone is talking about the box so my brain works visually creating visual pictures I was interested in how that works for blind people well that depends on the blind person I have I've never really had a visual thought process so when I talk to people and ask them what is it like to think for you people in the audience think about this if you're imagining what you did over the last day try it okay many people feel that as sort of a quasi audio monologue it's like a voice in their head it's their voice it's not other voices many people have it like you have it of sort of visual experience some people have it as scrolling text could I get a clap for anyone who has their thoughts as text anyone okay two or three people who have that for me it's purely abstract but if I want to think of a box yes I can think of a box I can imagine it visually I can imagine what it feels like etc also please remember to see in dark light I can't see here because when looking up it is painful with my eyes closed under this blindfold there's enough light coming through that it hurts I definitely would not be able to open my eyes and I could take my blindfold off temporarily but that was because I was willing to tolerate some pain so with if it's a dark room then I can see just fine so thank you so the next question is from the internet so they won't be able to come on stage yes how do you feel about the app be my eyes be my eyes it's actually not bad there's be my eyes tap tap see be specular several for iPhone which I've never used because I've got an android though iPhone for what it's worth has and Mac products in general have really excellent support for blind people tap tap see is useful it's not as useful for me because at home like if I want to identify a can of soup or whatever I will just look at it with the lights down so I can see it myself and it's too much of pain to use if I'm out like if I'm grocery shopping that's something that's a real pain to do and an accessibility issue for those of you thinking about such things if you only if you only support the operation search or the operation look up and say I walk into a grocery store which I have done shock and they come up to me and say hey how can I help you what do you want I say well I don't know what I want what do you have I would like a savory snack that's vegetarian what do you have well what do you want name it but I don't know especially if I'm in Germany I have no idea what I sell here they still ask me that so browsing is much harder and with tap tap see and with be my eyes and so forth it's really for identifying a specific object one thing that you may not be used to here but in more barbaric countries all the paper currency is the same size and has no distinguishing features except for the next print run because the American Federation for the Blind sued the US Treasury and won so the next generation of bills except for the $1 note which the Treasury is not allowed to change by law but the next generation will have blind accessibility features here they have different sizes if you feel your euros they have a little strip on the edge of them and some versions of them have little braille dots that you can tell what they are if he has the dots be my eyes is more useful for someone who's blind at home as well another question another question for microphone 4 that's to the right of the stage they can ask from there but if they want to come up ask first I'm going to sorry for all the it was very impressive how you pick up on all these things that I normally don't so I was really curious when you meet a new person what do you pick up about them how do you assess the person how do you know they are interesting because I use visual cues for that so so I'm going to answer you assuming that I'm blind at the time if they come up to me and say how much can you see then they're probably not a person I want to continue interacting with if there's someone who comes up to me they've read my shirt about all sorts of different things and they have a conversation about that substantive and they say hey how would you like to implement liquid democracy or how do you make a non-linear writing system or something like that and they're more likely an interesting person or if they introduce themselves and are nice how do you evaluate people like you can't evaluate them just by what they look like I can tell your age and your approximate gender and your approximate place that you were raised linguistically and if you were at the same floor level I can tell your height I can probably tell roughly your weight just from your heat the sound of your voice the kind of pressure that you're making on the floor depending on what kind of floor it is the stone out in the mess I'm not going to get anything from that blind people can be racist too if that's what you're thinking so it describes everything with his hands everybody can be racist we're all a little bit racist dancy same way to anyone else so we'll do one last question for microphone number two the best center aisle so thank you for your talk I think that was very inspiring I was wondering most of the message that you showed were the analog so have you also tried something like maybe a smart cane so we have some more technical approach to detecting surfaces and obstacles or other things what's your experience hopefully my slide is still up there where it says I'd be interested in collaborating on that Eric Boyd at Sensebridge of Noisebridge where I used to be made a nice anklet called the Northpaw which is basically a series of pager buzzers that vibrate to point to which way is north I sadly broke mine and if any of you have a working one I would really like to have it absolute orientation is really difficult so relative orientation like what I was doing with the IKIDO I can tell where someone is I can focus on them but then it's easy to lose track of which way you're facing so something like that is quite useful there are some products that do this on demand so I forget what it was called but the LHZ Desden had this product which is a tactile feedback compass and clock so you click it and it'll vibrate in different ways to tell you which way you're facing but that's doesn't really work very well what you would want is something continuous I've heard of canes where they've like attached a camera to either the cane to the grip and it beeps when there's something there I've never used one so I can't comment on that directly but from blind friends who told me about experiences with them they tend to be engineered by sighted people who have no idea what it's like to be blind that said it is true that tree branches are a real obstacle because if I try to sweep something and something is above this height like a tree brand roughly one meter from floor or a metal sign like in Boston or DC or New York or really anywhere not so much in London they're pretty good about that except for the one time I walked into a building and there was a monitor fixed to the wall at this height head height hit the column I knew there was a column there and I was just walking past the column and walked smack into the monitor this was in a government building sadly demonstrating with this head and I haven't found any like that here but trees there's always trees bus stops often have sort of the sign on the side there's like this gap that might be say this high 30 centimeters from ground canes are usually anywhere between sternum and nose mine is just at the edge of my nose if I'm standing up tall it's quite long because I walk really fast so mine has a lower angle and that means that it's more likely to go under that and hit it like here when I'm almost at it tangles up my cane it's kind of a pain so it's possible that something that detects a higher range would be useful one thing that I'm specifically interested because it's something I'm bad at is absolute orientation so if you are an engineer interested in collaborating on something like putting little things in the cane grip or in a hat or on the side of eyeglasses or something like that that would tell me which is north constantly please get in touch or for that matter if you can make glasses that will take standard lenses and have zero light leakage please get in touch because it turns out there aren't any such frames on the market for some reason that's the technology but it ain't there so I think we're out of time thank you Sai for a very interesting talk big general applause thank you for listening this was the live description