Computer vision makes mixed reality applications possible

The Saito Lab is researching next-generation visual information processing using a computer. This is called computer vision.The aim is to enable computers to understand three-dimensional images, in the same way that people process information from their visual sense in their head, and to create new visual media.

Q. For example, I think that in the future, 3D TV will become very widespread. In the 3D pictures used for that, the 3D shapes have to be estimated from pictures, and processing to do such estimation is the most important aspect of this computer vision technology. If this kind of technology enables 3D content to be used skillfully, in the future, people will be able to watch 3D TV.

By establishing 3D image data, its possible to obtain more realistic information than with 2D image media. However, enabling a computer to understand 3D information requires a very high degree of complexity and precision.

Q. When an ordinary picture comes in, its just a 2D array of colors. From the 2D array, just recognizing that the subject is a person or a car is difficult. Thats the actual difficulty we run into. Ordinarily, it seems like a simple task, but when you really think about it, its very hard. I think this gap is one of the difficult aspects of our work.

We expect that one future application for this research will be mixed reality. Because the amount of information on the Internet has grown too large, in some cases, its hard to find the information you want just by doing a keyword search.
But if mixed reality is achieved, people will be able to automatically search for and display information thats related to, say, a photo taken with a camera-phone. In this way, people will arrive at accurate, essential information in an instant.
The Saito Lab is also developing several other applications. One is a Virtual Try-On System, where photos are taken of a person with markers attached, and the textures of their clothes are replaced. Another application is a performance support system, which enables people to learn to play a musical instrument efficiently, by showing how to play the instrument directly in the picture. Also being developed is a 3D laser detection system, to locate accident survivors buried in rubble.

Q. If our research on these technologies goes well, access to information will be possible just by looking, without even thinking. And the accessed information wont be just text, but 3D information that has a volume, so people wont even be aware that theyre gathering the information via the Internet. People will be in the Internet world casually and naturally. I think our research may make that sort of thing possible.

3D information processing technology using computer vision is expected to have a range of applications. By pursuing this research, the Saito Lab is building the foundations for a new kind of information society, where people can gather information more smoothly.

Category:

Science & Technology

Tags:

• Keio
• University
• Saito
• Group
• Computer
• vision
• new
• visual
• media
• 3D
• image
• processing
• guitar
• sense
• information
• technology
• engineering
• science
• tokyo
• japan