 So, good afternoon everyone. As you all know, this is going to be an interesting talk for sure. It is titled Sense Without Sight. Our next speaker is Sai, and without too much ado, please give him a big round of applause. He's going to be here shortly. So, you might be thinking that I am daredevil. I'm not, sadly. No, probably better for my health, and I do not have any magic powers. Unfortunately, I just have extra disabilities that make it harder for me. Nevertheless, one, two back, yay, live demos. Where did that go? No, I'm not going to fall off. You're going to be falling soon. Nevertheless, I can walk around without my cane. It's a little hard to precisely target it. As you can see, I don't fall off stage. So, just to make this clearer, I am not faking this. And the way you can tell is I have a blindfold on. This blindfold is for real. A sleep mask. I can't see anything right now. So, you don't need to ask how good my sight is or why, because I can't see. The issue that I have is light blindness. Basically, I can see if it's really dark, and I can't see if it's really bright. And lights that are as bright as these lights here actually hurt my eyes kind of right now. So, I definitely would not be able to do this with my eyes open and my glasses off. I do a lot of different things. Blindness is not one of them. On my shirt, on my slides, on my website, there's all sorts of stuff that we can talk about. Let's please talk about that stuff and not the fact that I'm blind, except during this talk or during my workshops when you're, it's okay to ask questions about that or talk about that. The exception is if you want to work with me on stuff or if you have feedback, I welcome that. But let's talk about something else, like how to analyze a few terabytes of court data. Anyway, I have assistance for this talk. One of them is what I'm going to be calling the helpful asshole, aka my herald. You're welcome. Come up. I also have bits. Where are you? Good. And I have someone running the slides for me because obviously I can't see my own slides. Fortunately for you, that means I'm not giving the talk like this. Whatever I just talked about. It is okay to make sound if anything. It's helpful because I can hear where you are. You might wonder how I walked onto stage without my cane. I'm going to go over that in a second. First, the cane is really, really useful. It gives me a number of things. For instance, right here, you might not notice it, but there is a difference in the floor. There's this carpeted section here. There's this wood section here. If I just brush this with my feet, I can feel it. That's how I walked on stage. But if I do it with my cane, hear that whack? You can say yes in response to questions. Hear that whack? Yeah. So even if I'm lightly walking, I just keep that at a known degree and I can easily walk alongside of it. I can also feel the texture. So one thing you may not notice, but you may as well try now, is that the texture of the carpet under your feet, the people who are live in this audience. I don't know about you people at home, sorry. But this stage I have checked out and it is smoother from the front of stage to the back of stage and is smoother from the right of stage to the left of stage. And I believe that's true for most of the floor in the audience, but it's made out of separate squares. So some of them may be oriented a different way. So feel with your feet, for real, like right now. Feel and see if you can feel which way it's smooth and which way it's rough. The cane, when I drag it like this, it's smoother. When I drag it like this, it resists. And especially if I'm holding it like this with a firmer grip, this is nice and smooth. And then this way it sort of resists. It bends the cane a little bit. What else? There's sound. Something that I can do really well with a cane that's hard to do with my feet, but I can. So I can do that. But I can also do this. So those two textures sound completely different, right? In fact, here the sage is not hollow underneath. Here it's firmer and then suddenly it's echoey. So that helps me tell just sort of what ground I'm on. Similarly, actually, in the front of the sage. Sorry about your mic. The stage edge, it's really sort of resistant. Whereas there, it's got much more of an echo to it. One thing this is very useful for is echolocation. I am not a bat, but it is the same principle. There are some people who are much, much, much better than I am about echolocation. I'm not that good. But what I can hear and what you can hear, and what you're going to try to hear right now, is the size of the room. So I'm going to whack the sage with my cane. What I want you to do is while I'm doing it, close your eyes. You're not going to miss anything. The slide's not going to change. I'm just here. So close your eyes and move your head from side to side as I do it. And you can even sort of look behind you and listen for the sound of the size of the room. You'll be able to hear how large the room is around you. And you'll hear the echo of my cane from the back of the room. Kill the PA for a sec, right? PA? Okay. So that's a simple version. Another version is these stage curtains. How do I not face plant into them? Well, there's a few ways. For one, I can just run into it with my cane. That's a speaker. Sorry about that. And that is curtain. Nice and soft. There are several other things about this. I'm going to show in a bit. But also, if I'm... Sorry about the speaker. If I'm walking through this and I walk through the side curtain sort of behind, they've added extra stuff to the stage. Yay. It absorbs the sound when I am tapping near it. Yeah. And there's... There's that. Bit, where are you? Looks like you're lost. Do you need some help? Sorry? Do you need some help? It looks like you're lost. Thank you, dear angel. No, I do know where I am. So my dear signal angel, or my herald, is representing that person who's trying to be helpful to me several times a day and kind of annoys me or ranges from annoying to actually dangerous. I obviously know where I am because I'm navigating the stage just fine and you saw me walk on without even my cane. So thank you, but no, I don't need directions. Bit, howdy. So with someone who's right next to you like this, how can I tell that he's there? I'm not touching him with the cane, although I can. And I do. So if I whack you in the ankle, sorry, but that's how I can tell you're there, especially if you're standing really quiet, it's really hard to tell that someone's there, especially in a crowd like this. So yes, I'm going to hit you in the ankle too bad. That's how I can tell you're there. But in this distance, I can feel his body heat radiating. So from about this distance away, this distance, I can definitely feel it on the back of my arm there. There's his shoulder, there's his cheek. So actually, I would like you to try doing that. Turn to the person who's next to you. Ask their consent first. I have asked Bit's consent for everything that we're going to do. And what I would like you to do is put your hand up next to them. Try to almost touch their cheek without quite touching their cheek, and then deliberately do so. So almost, but not quite, just by the heat, with your eyes closed, and then trade off. Yeah, go ahead. So another place that I feel this is on walls. This wall, for instance, the front of the stage, there's actually wind coming through these little holes in them. That wind is colder than the ambient temperature of the room. If I get my arm really close to it, then at about this distance, I feel the cold air from it. And then at about that distance, there's actually sort of a sheet wind that is really tight close to the wall. And then there, the hair is on the back of my arm tingle. And there, I can touch it. So we have people who are in the audience on scooters and such, and they're going to drive through the aisles. I want you to close your eyes and see if you can feel the wind generated by them moving past you. And also the sound. Some of them are louder than others. So you feel how when they pass you, there's a breeze in their wake. This happens a lot in Congress. There are lots of people on scooters and hoverboards and motorized couches or whatever. Gods know what people motorize around here. Or someone just walking really fast, like me. They all generate a wind in their wake. Another thing you'll notice in Congress, there's these nice tunnels. So for instance, from CCL to Hall 2. There's this tunnel through the glass hall. When you walk through it, the wind characteristics of the room completely change. You start feeling a crosswind. So if I'm walking this way and the tunnel is on my right, then suddenly I will feel a crosswind from the tunnel that is colder and wasn't there before. Similarly with the wall, it just sort of stops. Next? Oh, yeah, yeah, yeah, yeah, right. Yeah, so everyone please close your eyes. And I do mean everyone. Oh, not the audio angels. Or the video angels. But everyone in the audience, please close your eyes. If your phone number ends with the number one, please move your arms in such a way as to make some wind. If your phone number doesn't end in the number one, point to the person who's making some wind. I can't see you, so I'm not going to judge you. So do you feel a yes, no? How about people whose phone number ends in one stop? People whose phone number ends in four. Just make some wind. Wave your arm in front of you. Feel that? Try pointing to them. Well, not the people who end in four. You can point to yourselves, I guess. People whose number ends in seven. Okay, so that's movement. When people move by you, it generates wind. When there's a tunnel, when there's an air differential, the air under this stage is colder, and that's why there's a wind out of it. Actually... Let me hold your hand. Can we get that for you? No? You good? Yeah, let me, please. More stuff. So, down here, if I open this door, and I walk past it, there's this crosswind. I was mentioning a second ago about the tunnel. So here, there's just sort of flat to my right. Then here, I walk a little bit forward. Suddenly there's this crosswind from that opening. Similarly, when people were biking by you, you could tell that. Similarly, if I crack my cane, I don't know if you can hear that very well, the trie. Here how there's a sound off to that side that sort of ends at the door. And then if I knock my cane here, there's a little reflection through that door. And yes, I do know how to close a door. I know. Amazing. Similarly, for these walls, I can feel that I'm close to the wall, or not, tingly, not tingly. What's next? Ah, yes. So... Oh, yeah, you've added more random crap. No, please. That would be helpful. Sometimes that would be helpful, but at the moment, not so much, because if you were to move it when I've already found where it is, then suddenly it wouldn't be there anymore, and it'd be hard for me to orient. One of the things that is difficult about being blind is you have to have quite a lot of memory. So the things that you probably don't notice because you outsource your memory is how much of the environment around you changes. So, as a blind person, they probably hate construction work. I do. And if I try to go sit down, for instance, for that matter, there's a chair. This had moved a lot. It'd be really hard for me to take a seat. Now, he doesn't smell that bad. But some people have a pretty significant smell because they're wearing perfume, or cologne, or axe, or something like that. If you've ever been in a public transit terminal and you've taken the elevator, maybe you know the distinct aroma of the elevator in public transit. Definitely lets you know you're in New York City. So, next group, start moving. So, my assistants in the audience are moving through the aisles with something in their hands. What I want you to do is close your eyes, smell, feel the wind current, smell what is it, point to them. See if you can point exactly where they are. Tell me your orange. Sorry, now I'm really hungry. So, if I'm walking past a coffee shop, for instance, it has a distinct smell. It smells like coffee, shockingly enough. A drugstore smells like alcohol and cleaning solution. A clothing store smells like leather and this weird perfume that they put in clothing stores for some reason. I don't know why, but they do. And if you ever visit one, hopefully you'll notice it now. So, let's see, next. All right. Dear asshole, you know your lines? Do you have your blindfold on? No. Please do. I don't have an extra one. Did you move his blindfold also? Should I lend you mine? Ah, it's bright. So, if you are going to try to help someone navigate, first off, say, would you like some help? Well, I'm pretty confused now, so yes, please. So, don't just, you know, randomly assault people. Just because I have a cane does not mean it's cool to assault me. Then, if you're going to touch them, do so with the back of your hand. This way, he knows I'm not just randomly grabbing him and yanking him around. It's really annoying. So, I can offer you my arm. We can walk. Let's go have a seat. There we go. There's your seat. Now, some people will, like, grab the hand and put it down and said, I can just say, here's the chair. Ta-da. So, follow me for a second. Could I have a sidewalk beep in the back? So, let's do a little crack. So, do you hear the back of the room where that is? I think it's in front of me, a bit to the right, maybe? So, one thing you really don't want to do when someone is possibly trying to figure out their orientation is to randomly come up to them and grab them and pull them. Dear asshole. Yes, please. Can I help you? Can I take you somewhere? Let's pretend that you think he wants to go down the stairs. He thinks he wants to walk towards the audience. Let's walk towards the audience a little bit. So, here the stairs are right this way? Yeah. Okay. No, but you're looking for the stairs. No, not really. You look lost. I'm trying to go elsewhere. Yeah, fuck off, please. Now, can you please point to the back of the room? Yeah, not quite, is it? This is what happens when you mess with someone's orientation. Please don't be that, asshole. If you want to be helpful, you can be helpful. You can be nice. Shall I leave you back to the chairs now? Would you like a hand? If they hold you, you don't hold them. And then we can leave them back to the chairs. There we go. How was he? May I have my blindfold back? You can tell this is improvised. And here I am again walking without my cane on the crack. So that's not just a party trick. I'll show in a sec. Next. Right. So some of you may have the response of thinking, oh, wow, so I can walk across a stage or talk or make decisions. I wish I were kidding, but that does happen pretty regularly. Can we clear the stuff off the stage, please? So instead of that, I'd like to show you something that actually does involve a little bit of skill. Namely, about a decade worth of Aikido. For those of you who don't know, Aikido is martial art from Japan. Anna. So I've spent about five or four years or so learning to use a cane. Anna has spent well more than that doing Aikido and is a second-degree black belt. I am not. However, I'm wearing a blindfold and she is not. So what I want to demonstrate for you is how I can tell that she's about to grab my hand. So there is the sound of the feet on the ground, making this sort of sound. There's the hand approaching me, the wind there, the vibration as her hips sort of rotate towards me with a nice little fist attempt here. There I can tell she's starting to lean back and there I put her to the ground. So again, so sound, movement, airflow, heat. Little head, vibration, wrist, the crook of the wrist, full speed. So the blindfold hasn't come off. It's in magic. Again, I'm not daredevil. I just can't see. I didn't get extra super special hearing powers. My hearing is pretty good, but it's no better than the average person. It's actually a lot worse in the rain, so that thing isn't a really a thing. So just to prove that it doesn't just work in this sort of deliberately set up special thing, Ana is going to just attack me however she wants and I'm going to throw her around a bit. Note that I can still focus on her. I can still pin her if I want. And no, I didn't tell them what to do in advance. That's real. Yes, I am a little bit out of breath. It's been a while since I actually exercised. Well, and again, I'm still facing you, right? I haven't forgotten where this line is between the wood and the carpet. And in fact, I can knock it with my ring sometimes. Against the wall is a little better to generate a nice echoes hand. So if you want to not be an asshole, please remember, first off, ask. I may not want your help or need it and what you think may be helpful to me may be, probably is, completely wrong. So listen to what I say. And if I say, yeah, I would like you to please lead me to X. That does not mean grab my arm and start dragging me to X. I prefer to follow people by sound, so I'm just going to tell you to scuff your feet when you walk so I can follow you by sound or to just keep talking. Some people prefer to follow by hand a bit. So if it comes up and I want directions to stage center, what so? Lead me to stage center, please. Ta-da. Not that hard. If you are going to touch someone, like, bit here, unless, you know, you've invited them to attack you because you're doing an IQ exercise, don't just randomly grab them and you anchor them around. Touch them and say, hi, my name is Si. Would you like some help? Hi, Si. I would. Thank you very much. And then you can offer or you can say, what would you like? Or are you lost? I'd like to go to your place, thanks. Maybe later. And one important thing to note, when people do simulations of disabilities, they sometimes try to do it by just pretending to be blind or pretending to be in a wheelchair or something, like you sit in a wheelchair for a day and then you think, oh, it must be so hard to be in a wheelchair because my arms hurt so much and I couldn't figure out how to get to the second floor. Well, that's because you've done it for a day and someone who actually uses a wheelchair has done it for probably years and years and years and has much better arms than you do. And they know where everything is and how to get around. Similarly, I know how to get around without using my eyes. You don't. If you just try to put a blindfold on and grab a cane or let alone those blind experienced museums where they just put a blindfold on you and don't give you a cane and have you walk around. Actually, one to one once and they wanted to take my cane away from me. It's like, hell no, that ain't happening. I actually lent it to my partner so that my partner could not feel afraid of the surroundings of accidentally walking into something, accidentally walking off, not a stage but off a step or something. So you don't know what you don't know. Most of the things I've showed you in this talk, you probably have never realized that they're even a thing. Like, have you noticed how many different floorings there are in the mess hall? How many different textures there are? How many different sounds there are? The smells, the wind, the ceiling heights? You probably don't pay attention to that. I do. And without the experience, you're just not going to be able to learn that. That said, I am teaching workshops. You're welcome to come to them if I still have the energy to run them. One is going to be right after this talk and the rest to be determined. My volunteers, could you please come up on stage if you're willing to be on camera? This talk is complicated. If you ask the 35C3 content crew and the Vok crew and everyone else, it's kind of a pain to run something like this because I'm doing stereo recording. That doesn't affect you in the audience live because the PAs are mono. But hopefully, those of you listening at home, you've been listening to this with stereo headphones on and hopefully you'll hear it from my perspective what it's like when I crack the cane and you can hear the echo in the room. So, thank you very much. Thank you very much. Take a bow, please. Thank you, Sai, for a very interesting talk. We have some time for questions. Can I have the Q&A slide, please? So, if you have questions and you're okay with being on camera and possibly being asked to come on stage so I can answer your question, depending on what your question is, please queue up to the camera that's on stage right and otherwise there's cameras... There's microphones. We can also just play in questions. If you don't want to come on stage, you can just ask your question from any one of the four microphones in the room. Bit. So, first question for microphone number two. Which one is two? It's for the person asking. Which one is the table one? Sorry? Which one is the one on stage right? And then I ask them on stage. Oh, let's see if they want to come on stage. Which number? Microphone number two is... Number two is the microphone like in the... You want to come on stage? I just want to know if I can... Number two is in front of you on the island. Number three behind there. Number one left on the island. Number four right. Okay, so go to number four if you're willing to be on camera. So the question is right here. Yeah, go ahead. Hi, thank you for the really impressive talk. Thank you. My question is the following. I want to know how your imagination works. So when someone says imagine a box, what's that you think of? Is that the material or how it sounds, how it feels? Just how does it work? Well, there I would actually go to my cognitive psychology talk that I gave at 27 C3. And a box is a category theoretic concept that has a prototype possibly. It's like asking, imagine a furniture. Imagine a box, a mobile kind of box. Is it a shipping box? Is it a box of mate that are those little crates that I walk into, et cetera? They're all different kinds of boxes. Is it an elevator? Yeah, I don't know how to answer that kind of question. So maybe I got that incorrect. But what I want to say is that when I try to think of something, I don't know any idea, any project plan, et cetera, I imagine how it looks. So how I am at a certain place, how I, I don't know, for example, see a box if someone's talking about the box. So my brain works like visually creating visual pictures. So I was interested in how that works for blind people. Well, that depends on the blind person. I have, I've never really had a visual thought process. So when I've talked to people and asked them, what is it like to think for you? So if people in the audience think about this, if you're, if you're imagining like what you did over the last day. Try it. Okay. Many people here feel that as sort of a quasi-audio monologue. That's, it's like a voice in their head, except it's their voice, it's not other voices. Many people have it like you have it of sort of visual experience. Some people have it as scrolling text. Could I get a clap for anyone who has their thoughts as text? Anyone? No? Okay. I bet like two or three people who have that. For me, it's purely abstract. But if I want to think of a box, yes, I can think of a box. I can imagine it visually. I can imagine what it feels like, etc. Also, please remember, I can see in dark light. I can't see here because looking up is painful with my eyes closed under this blindfold. There's enough light coming through that it hurts. I definitely would not be able to open my eyes. And I could take my blindfold off temporarily, because I was willing to tolerate some pain. So if it's a dark room, then I can see just fine. So the next question is from the internet, so they won't be able to come on stage. Yes, how do you feel about the app Be My Eyes? Be My Eyes. It's actually not bad. There's Be My Eyes, Taptap C, Be Specular, several for iPhone, which I've never used because I've got an Android, though iPhone for what it's worth has, and Mac products in general, have really excellent support for blind people. Taptap C is useful. It's not as useful for me because at home, like if I want to identify a can of soup or whatever, I will just look at it with the lights down so I can see it myself. And it's too much of a pain to use if I'm out, like if I'm grocery shopping. That's something that's a real pain to do. And an accessibility issue for those of you thinking about such things. If you only support the operation search or the operation lookup and say, I walk into a grocery store, which I have done. Shock, yeah. And they come up to me and say, hey, how can I help you? What do you want? I say, well, I don't know what I want. What do you have? I would like a savory snack that's vegetarian. What do you have? Well, what do you want? Just name it. And if I name it, they'll get it for me. But I don't know, especially if I'm in Germany. I have no idea what they sell here. Yeah, they still ask me that. So browsing is much harder. And with Tap-Tap-See and with Be My Eyes and so forth, it's really for identifying a specific object. One thing that you may not be used to here, but in more barbaric countries, all the paper currency is the same size and has no distinguishing features. Except for the next print run because the American Federation for the Blind sued the US Treasury and won. So the next generation of bills, except for the $1 note, which the Treasury is not allowed to change by law. But the next generation will have blind accessibility features. Here they have different sizes. If you feel your euros, they have a little strip on the edge of them. And some versions of them have little braille dots that you can tell what they are. So that's not as bad here. But yeah, Be My Eyes is more useful for someone who's blind at home as well. But it's a great app. The question from microphone four, that's to the right of the stage. They can ask from there, but if they want to come up. Do you want to come up? Ask first. I'm going to... Sorry for all the... Hey, so it was very impressive how you pick up on all these things that I normally don't. So I was really curious when you meet a new person, what do you pick up about them? How do you assess the person? How do you know they're interesting, attractive? How do you... Because I use visual cues for that. So I'm going to answer you assuming that I'm blind at the time. If they come up to me and say how much can you see, then they're probably not a person I want to continue interacting with. If there's someone who comes up to me, and I've read my shirt about all sorts of different things that we can have a conversation about that's substantive. And they say, hey, how would you like to implement liquid democracy? Or how do you make a nonlinear writing system? Or something like that. Then they're more likely an interesting person. Or if they introduce themselves and are nice. How do you evaluate people? You can't evaluate them just by what they look like. I can tell your age and your approximate gender and your approximate place that you were raised linguistically. And if you were at the same floor level, I can tell your height. I can probably tell roughly your weight just from your heat, the sound of your voice, the amount of pressure you're making on the floor, depending on what kind of floor it is. If it's the stone out in the mess, I'm not going to get anything from that. Blind people can be racist, too, if that's what you're thinking. Everybody can be racist. We're all a little bit racist. Same way to anyone else. We'll do one last question for microphone number two. That's center aisle. Thank you for your talk. I think that was very inspiring. I was wondering most of the message that you showed were the analog. Have you also tried something like maybe a smart cane, where you have some more technical approach to detecting surfaces and obstacles or other things? What's your experience? Hopefully my slide is still up there and I'll be interested in collaborating on that. Eric Boyd at a sense-bridge of Noisebridge, where I used to be, made a nice anklet called the Northpaw, which is basically a series of pager buzzers that vibrate to point to which way is north. I sadly broke mine, and if any of you have a working one, I would really like to have it. Absolute orientation is really difficult. Relative orientation, like what I was doing with the IKIDO, I can tell where someone is, I can focus on them, but then it's easy to lose track of which way you're facing. Something like that is quite useful. There are some products that do this on demand. I forget what it was called, but the LHZ had this product which is a tactile feedback compass and clock. So if you click it and it'll vibrate in different ways to tell you which way you're facing. But that doesn't really work very well what you would want is something continuous. I've heard of Keynes where they attached a camera to either the cane or to the grip and it beeps or something when there's something there. I've never used one, so I can't comment on that directly, but from blind friends who told me about experiences with them, they tend to be engineered by sighted people who have no idea what it's like to be blind. That said, it is true that tree branches are a real obstacle because if I try to sweep something and something is above this height like a tree branch at about this height or a metal sign like in Boston or DC or New York or really anywhere in the US not so much in London they're pretty good about that except for the one time I walked into a government building and there was a monitor fixed to the wall at this height I cleared the column I knew there was a column there and I was just walking past the column and walked smack into the monitor this was in a government building sadly but usually they're pretty good about it and I haven't found any like that here there's always trees bus stops often have sort of the sign on the side and there's like this gap that might be say this high because I have a fairly long cane canes are usually anywhere between sternum and nose and mine is sort of just at the edge of my nose if I'm standing up tall it's quite long because I walk really fast so mine has a lower angle and that means that it's more likely to go under that and then I'll hit it like here when I'm almost at it to hang those up my cane it's kind of a pain yeah so it's possible that something that detects a higher range would be useful one thing that I'm specifically interested because it's something I'm bad at is absolute orientation so if you are an engineer or a hacker who is interested in collaborating on something like putting I don't know putting little things in the cane grip or in a hat or on the side of eyeglasses or something like that that would tell me which way is north constantly please get in touch or for that matter if you can make glasses that will take standard lenses and have zero white leakage please get in touch because it turns out there aren't any such frames on the market for some reason that's a technology but but it ain't there so thank you very much thank you Sai for a very interesting talk big round of applause