Blind man visually picks up objects through visual-to-auditory sensory substitution (PIP version)

Early blind male user of The vOICe picks up small objects (AAA battery, paperclip) by listening to the camera-based soundscapes. The camera is in his special sunglasses.

The PIP (picture-in-picture) display of the first-person view via the camera glasses was manually aligned with the main video, leaving a slight temporal misalignment between the two views.

This video was recorded in 2010 during the NSF-funded SBIR Phase I project 0946035, "Viability of a sensory substitution device giving blind users sight through sound", executed by MetaModal LLC in Pasadena, California. More information at http://www.metamodal.com and http://www.seeingwithsound.com

More information about grasping exercises can be found at
http://www.seeingwithsound.com/training.htm#grasping_exercise

This project made use of The vOICe Learning Edition software for Microsoft Windows, which can be obtained from
http://www.seeingwithsound.com/winvoice.htm

Category:

Education

Tags:

• yt:fmt=18
• sensory substitution
• seeingwithsound
• blind
• vision

[smartsight]

@PacoBell Edge detection and enhancement tend to lead to cluttered views, because computer algorithms cannot reliably tell the difference between object edges and textures, shadows, irrelevant edges as with a tiled floor, and so on. The vOICe includes edge detection as a user option, but by default it is off and it is left to the user to decide when and where to make use of it: it is a mixed blessing for which the choices cannot be fully automated. In most cases one is better of without it.

[PacoBell]

I wonder how much faster the subject would discover the objects if the video stream were simplified with edge detection algorithms?