High-Presence, Low-Bandwidth, Apparent 3D Video-Conferencing with a Single Camera

Demo presented at WIAMIS 2009.

Authors: Timothy R. Brick, Jeffrey R. Spies, Barry-John Theobald, Iain Matthews, and Steven M. Boker.

Abstract:

Small digital video cameras have become increasingly common, appearing on portable consumer devices such as cellular phones. The widespread use of video-conferencing, however, is limited in part by the lack of bandwidth available on such devices. Also, videoconferencing can produce feelings of discomfort in conversants due to a lack of co-presence. Current techniques to increase co-presence are not practical in the consumer market due to the costly and elaborate equipment required (such as stereoscopic displays and multicamera arrays).

To address these issues, this paper describes a real-time, full frame-rate video-conferencing system that provides simulated three-dimensionality via motion parallax in order to achieve a higher level of co-presence. The system uses a deformable 3D face model to track and re-synthesize each users face using only a single monocular camera, so that only the (few tens of) parameters of the model need be transferred for each frame. This both provides motiontracking for the simulated 3D experience and reduces the bandwidth requirements of the video-conference to the order of a few hundred bytes per frame.

Bandwidth and processor usage for the algorithms are reported. Possible implications and applications of this technology are also discussed.

Category:

Science & Technology

Tags:

• video-conferencing
• 3d
• wiamis
• bandwidth
• demo

[TableWolfMusic]

Is there a version with a higher polygon count around the lips for lip reading by humans? as speech recognition is rather unimpressive to date, regarding accuracey, could the 3D modle around the lips area be used to increase that accuracey by means of the computer storing lip movements as gestures in conjunction with the sounds to increase accuracey of speech recognition?