ShopTalk, Wearable Shopping System for the Blind, at Lee's Market Place in Logan, UT - Trial 2

A supermarket trial of our
ShopTalk system at Lee's Market Place in Logan, Utah.

ShopTalk is a proof-of-concept wearable system designed to assist
visually impaired shoppers with finding shelved products in grocery
stores. The system is based on a simple conceptual formula: Independent
Blind Shopping = Verbal Route Directions + Shelf Barcode Scans.

This individual is a cane user. She has selected a product, and is now
following a set of verbal instructions generated by the system from the
topological map of the supermarket. The route in this section of
the video starts at the store lobby and ends at the aisle where the selected product resides.

When the shopper scans any shelf barcode, the system begins to use its
database, called the barcode connectivity matrix (BCM). The database
associates each shelf barcode with an aisle number, the side of the aisle,
a shelf section number, a shelf number in that shelf section, and a relative
position on the shelf.

When the shopper scans a shelf barcode, the system can tell her, "You are
in aisle 5" or "Move 2 shelves down" or "Scan five barcodes left" or "Walk
10 feet down the aisle" etc. The shopper receives these instructions
through a head phone. Notice how the shopper first finds shelf barcodes
by touch and then scans them by aligning the barcode scanner with the help
of two plastic stabilizers on both sides of the scanner. After the target
barcode is scanned, the shopper reaches over, gets the product, and puts
it into the basket.

The system's hardware consists of a computational unit, an OQO model 01,
a Belkin numeric keypad, a wireless barcode scanner and its base station,
and a USB hub that connects all components. The user wears a small CamelBak backpack to carry the equipment. The numeric keypad is attached by a velcro stip to one of the backpack's shoulder straps.

To qualify for these experiments each participant had to give us evidence of
independent travel: independent walks around the neighborhood and
independent use of public transportation.

We were following each shopper with the Lufkin wheel, a distance
measuring device. We used this wheel to systematically estimate the
distance walked by each participant. We noticed that the more the
participants used the system the smaller was the total distance they
walked in the store.

A key finding was that verbal route directions were sufficient for our
sample of independent travelers to navigate this supermarket reliably and
retrieve various products. The more they used the system, the less they
requested verbal route directions. As soon they heard the aisle of a
selected product, they would just go there independently.

If this finding generalizes to the entire population of independent
visually impaired travelers, it suggests that verbal route directions
may well be sufficient. The important point of ShopTalk is that the
store may not need to be instrumented with any external sensors, such
as RFID tags, Wi-Fi routers, IR transmitters, etc.

We would like to take this opportunity to thank the late Mr. Lee Badger,
the founder of Lee's Market Place. We met with Mr. Badger in early 2004
when this system was just bunch of drawings. He grasped everyting right
away and gave us permission to use his store in Logan, Utah.

For more information go to http://digital.cs.usu.edu/~vkulyukin/vkweb/research/accessible_shopping.html

Video Narration: Vladimir Kulyukin

Category:

Science & Technology

Tags:

• ShopTalk
• assistive technology
• rehabilitation engineering
• accessible shopping
• orientation and mobility
• verbal route directions
• accessibility
• wearable computing
• barcode localization
• barcode detection
• visually impaired
• blind
• john nicholson
• vladimir kulyukin
• eyes-free shopping