Sensory substitution for the blind: a walk in the garden wearing The vOICe





The interactive transcript could not be loaded.



Rating is available when the video has been rented.
This feature is not available right now. Please try again later.
Uploaded on Apr 16, 2010

The vOICe gives blind people live detailed visual information about their environment: the live view of a head-mounted camera is scanned from left to right every second, while associating height with pitch (tone frequency) and brightness with loudness. A rising bright line thus simply sounds as a rising tone, but complex views yield very complex "soundscapes". In this clip, use is made of camera glasses and an optional wide-angle (fish-eye) lens. More information about wearable setups for The vOICe and where to buy camera glasses is available at http://www.seeingwithsound.com/camera...

Normally, knowledge of context helps greatly with visual interpretation, especially in familiar environments. Here we compensate with a coarse annotation by time point for blind people who watch this video clip (ideally a screen reader would access and speak the closed captions):

0: I walk across the terrace towards our neighbor's house, with our own house of white bricks on the left side. Bright sky is at the top right.

10: a white garden chair shows briefly at the bottom left, and the tonality of the bright cement of the brick wall of our neighbor's house becomes apparent as I walk towards it until it is within arm's reach. Then I look left and right along the brick wall such that you get the changing rhythm rates associated with visual perspective.

37: then I take a closer look at the white garden chair, and pass it to walk towards a wooden fence, with a white football lying on the ground in front of it. The football gives a low-pitched beep while the wooden fence gives a slowing rhythm as I move closer.

1.08: I turn left to face the wall with white bricks of our own house, giving a more noisy texture with a tonality from the darker cement, and I move a bit closer to the wall.

1.25: I walk around a kind of white brick pillar at the corner of our house, and approach it from the side that has a rain pipe in the middle, and move up close and put my hand on the rain pipe.

1.55: I turn a bit left and take a close-up look at a red table cloth with white dots on a garden table. The many white dots give a peculiar sound texture.

2.12 I turn further left to take a closer look at a corner of our house, with windows on the left and right right side of the corner.

2.22: I turn left and walk over the grass to another wooden fence, giving a characteristic rhythm that slows down as I move closer.

2.33: Turning a bit to the right I take a look at some nearby foliage giving a kind of random texture.

2.45: Next I take a look at the roof with its regular pattern made up of the rows and columns of stone roof tiles.

3.06: I walk across the terrace to a white door in the fence that faces the street behind it.

3.31: I briefly turn right looking towards the white brick wall, and then turn left for a close-up look at some other foliage, and turn further left to effectively make a U-turn after facing another brick wall.

3.58: Then walking back across the terrace I turn to the right to have a close-up look at an empty rabbit's cage, which here gives a characteristic simultaneous tonality and rhythm at close range, corresponding to a rectangular metal wire pattern.

4.08: I walk further across the terrace to take a second look at the white brick pillar with the rain pipe, again putting my hand on the rain pipe at close range.

4.38: end of video clip.

After extensive use, brain plasticity should help make use of The vOICe "second nature", with less focus on the sounds and more focus on the visual content carried by the sounds.


to add this to Watch Later

Add to