NAVI -- Navigational Aids for the Visually Impaired

http://hci.uni-konstanz.de/blog/2011/03/15/navi/?lang=en

NAVI (Navigational Aids for the Visually Impaired) is a student project aiming at improving indoor navigation for visually impaired by leveraging the Microsoft Kinect camera, a vibrotactile waistbelt and markers from the AR-Toolkit.
While the "white cane" is a good tool to improve navigation for visually impaired, it has certain drawbacks such as a small radius or that it just detects objects that are on the ground (during typical use).
We wanted to augment the visually impaired person's impression of a room or building by providing vibro-tactile feedback that reproduces the room's layout.
For this, depth information from the Kinect is mapped by our software onto three pairs of Arduino LilyPad vibration motors located at the left, center and right of the waist. These pairs of vibration motors are hot glued into a fabric waist belt and connected to an Arduino 2009 board. To increase the impact of the vibration motor they were put into the cap of a plastic bottle. The Arduino in the waist belt is connected via usb to a laptop that was mounted onto a special backpack-construction, which has holes for cables and fan.
To support point-to-point navigation usually a seeing-eye dog is used. This dog however must be trained for certain routes, costs a lot of money and gets tired soon. In certain research projects GPS is used to provide this point-to-point navigation, however GPS is not applicable for indoor scenarios.
We wanted to utilize the rgb camera of the Kinect, so we put several markers of the AR-Toolkit on the walls and doors of our building thereby modeling a certain route from one room to another. The markers are tracked continuously all along the way and our software provides synthesized auditory navigation instructions for the person. These navigation instructions vary based on the distance of the person to the marker (which we get from Kinect's depth camera). So for example, if you walk towards a door the output will be "Door ahead in 3", "2", "1", "pull the door" where each part of the information depends on the distance to the marker on the door.
The software is written with C#/.NET. We used the MangedOpenNI (https://github.com/kobush/ManagedOpenNI) wrapper for the Kinect and the managed wrapper of the ARToolkitPlus (http://code.google.com/p/comp134artd) for marker tracking. Voice synthesis is done using Microsoft's Speech API (http://msdn.microsoft.com/en-us/speech/default). All input streams are glued together using Reactive Extension for .NET (http://msdn.microsoft.com/en-us/devlabs/ee794896).

Category:

Science & Technology

Tags:

• human-computer interaction
• interaction design
• user interface design
• microsoft kinect
• ARToolkit
• Arduino
• vibrotactile user interface
• blind navigation
• university of konstanz
• information engineering
• user experience
• augmented reality

[ezfuzion]

Looks good but I didn't hear any "HEY!" or "LISTEN!" at the start of each sentence that came from the artificial voice.

[carrett99]

Awesome job! My daughter is visually impaired & when I see things like this, it just makes me feel better about what advantages technology can have in her future.

[Visthashiznit]

Does it go "hey listen" all the time

[ronensv]

autonomous SEGWAY with better 3D camera than the KINECT:

[zombie0Breath]

i know they are blind. i was just saying that's a lot of weight and gear. but its a cool invention if u could get sponsored and maybe lower the weight and all the gadgets to a smaller more compact fit. i like this a lot dude nice!

[BlackRushNL]

@zombie0Breath they are blind...

[zombie0Breath]

u look like a noob of some sorr with all that gear XD

[21Kurenai]

Great * *!!

[nanoradio]

@xXeeDeeXx HA!

[97Marcopolo]

Super Arbeit Jungs :))

[Raptor747]

That's incredible; a specially built device would be much more compact and practical, and given enough time for development and miniaturization, this could be a major difference for the blind in the future.