Upload

Loading icon Loading...

This video is unavailable.

Memory-Efficient Image Databases for Mobile Visual Search -- David M. Chen

Sign in to YouTube

Sign in with your Google Account (YouTube, Google+, Gmail, Orkut, Picasa, or Chrome) to like IVMS Admin's video.

Sign in to YouTube

Sign in with your Google Account (YouTube, Google+, Gmail, Orkut, Picasa, or Chrome) to dislike IVMS Admin's video.

Sign in to YouTube

Sign in with your Google Account (YouTube, Google+, Gmail, Orkut, Picasa, or Chrome) to add IVMS Admin's video to your playlist.

Published on Dec 21, 2013

Many mobile visual search (MVS) systems compare query images captured by the mobile device's camera against a database of labeled images to recognize objects seen in the device's viewfinder. Practical MVS systems require a fast response to provide an interactive and compelling user experience. Thus, the recognition pipeline must be extremely efficient and reliable. Congestion on a server or slow transmissions of the query data over a wireless network could severely degrade the user experience.

We show how a memory-efficient database stored entirely on a mobile device can enable on-device queries that achieve a fast response. The image signatures stored in the database must be compact to fit in the device's small memory capacity, capable of fast comparisons across a large database, and robust against large geometric and photometric visual distortions. We first develop two methods for efficiently compressing a database constructed from feature histograms. The popular vocabulary tree is included in this framework. Our methods reduce the database memory usage by 4-5x without any loss in matching accuracy and have fast decoding capabilities. Subsequently, we then develop a third database representation based on feature residuals that is even more compact. The residual-based database reduces memory usage by 12-14x, requires only a small codebook, and performs image matching directly in the compressed domain.

With our compact database stored on a mobile device, we have implemented a practical MVS system that can recognize media covers, book spines, outdoor landmarks, artwork, and video frames out of a large database in less than 1 second per query. Our system uses motion analysis on the device to automatically infer user interest, select high-quality query frames, and update the pose of recognized objects for accurate augmentation. We also demonstrate how a continuous stream of compact residual-based signatures enables a low bitrate query expansion onto a remote server when network conditions are favorable. The query expansion improves image matching during the current query and updates the local on-device database to benefit future queries.

Loading icon Loading...

Loading icon Loading...

Loading icon Loading...

The interactive transcript could not be loaded.

Loading icon Loading...

Loading icon Loading...

Ratings have been disabled for this video.
Rating is available when the video has been rented.
This feature is not available right now. Please try again later.

Loading icon Loading...

Loading...
Working...
Sign in to add this to Watch Later

Add to