Seeing with Sound (sensory substitution for the blind)

[jjovereats]

Oh I see!

[batfly]

Why cant we just use a simple 1 to 1 pixel to tone conversion?

Each tone would be a continuous tone and it would be spatially orientated in surround sound... depending upon the depth the pixel would have a lower volume...

Depending upon the color it would have a different pitch.

Use Left and right camera... Every color pixel in the video would be converted...

They could also plug in their camera to there laptop...

[smartsight]

@batfly A pure tone for a pixel on your left and a pure tone of the same frequency for a pixel on your right would together sound as a single pixel straight ahead. Surround sound unfortunately does not help with such ambiguities because the issue is mathematical: the sum of two sines of the same frequency becomes a single sine no matter what their respective phases are.

[batfly]

@smartsight

Unless you give that pixel tone a unique on off pattern?

[smartsight]

@batfly Unique patterns become perceptually confusing when there are many of them. Think of having a dozen unique patterns for a dozen different pixels sharing the same tone frequency. Many image-to-sound mapping schemes work as long as there are only a few pixels of interest in the view but break down when there are many pixels, as with common environmental views, the shape of a function plot, etc. Distribution in time is then a powerful general means for disambiguating information.

[batfly]

@smartsight

I wonder why unique patterns don't become perceptually confusing when there are many of them in vision?

[smartsight]

@batfly In vision by eyesight, the "unique pattern" of a pixel is just the position on the 2D retina, where each position has a separate neural connection to the brain: it is mostly 2D already, versus 1D for sound where two sines of the same frequency but different phase do not have separate neural connections to the brain but need to be distinguished via some other trick, such as not sounding them simultaneously.

[batfly]

@smartsight

Thanks for your response.... This idea sort of came to me out of the blue... And it makes sense to me what you are saying... though I still have some doubts in my mind... because I know the ear can distinguish plenty of notes while playing... say piano... and listening to an orchestral composition with multiple instruments... But I can totally understand the listening sense is seemingly limited when compared to vision.

[batfly]

@smartsight

What about combining like a grid on the skin of the back to create instantaneous little prickly sensation corresponding to pixels on a video? It would be like braille a vision?

[smartsight]

@batfly That is what Paul Bach-y-Rita worked on for decades, starting in the 1960s. Nowadays his company Wicab is opting for electrodes on the tongue. Visit vision dot wicab dot com for more information about their BrainPort Vision Device.

[8legsFreak]

there is a much better solution: camera captures data, that is transmitted to a netbook and then to a tiny vibrating pad glued to someones ear. The close the object is to the ear, the stronger the vibrations. Moreover, camera pixels can be represented by different micro niddles in the vibropad, so the vision is far more detailed.

[WildKatSpeaking]

I've been blind since birth. I wondered how she was able to navigate streets with traffic. Personally I'd rather hear what's happening around me (sounds of nature, traffic etc.) than swishing and other noises. Also how would she have a conversation with someone or hear someone speak to her with all of the noise. Again just my opinion. I know this could work well for others.

[idontevenhavestuff]

@WildKatSpeaking i guess you could wear it in one ear and the other ear you would hear normal sounds? not as good hearing, not as good sight but that might work?

[jjovereats]

@WildKatSpeaking Totally or legally?

[sweetbhikkhuni]

Wow, I bet that spycam will do her a lot of good, huh?

[winstonws]

@sweetbhikkhuni you seem to have missed the point, try watching the whole video...