Pose Estimation Based on Sylvester's Equation - Virtual Camera Application

In this system, we introduce a method to jointly track the object motion and estimate pose within the framework of particle filtering. We focus on direct estimation of the 3D pose from a 2D image sequence. Scale-Invariant Feature Transform (SIFT) is used to extract feature points in the images. We show that pose estimation from the corresponding feature points can be formed as a solution to Sylvester's equation. We rely on a solution to Sylvester's equation based on the Kronecker product or Hessenberg-Schur method to solve the equation and determine the pose state. We demonstrate that the classical Singular Value Decomposition (SVD) approach to pose estimation provides a solution to Sylvester's equation in 3D-3D pose estimation. The proposed approach to the solution of Sylvester's equation is therefore equivalent to the classical SVD method for 3D-3D pose estimation, yet it can also be used for pose estimation from 2D image sequences. The LHS shows a video clip captured by a Real Camera. The RHS shows the corresponding synthesized video produced by a Virtual Camera (from a different view point) as a result of the 3D Pose Estimation of the captured video [Reference: Chong Chen, Junlan Yang, Dan Schonfeld, and Magdi Mohamed, "Pose Estimation from Video Sequences Based on Sylvester's Equation", in the IS&T/SPIE 19th Annual Symposium, Electronic Imaging, Science & Technology, Visual Communications and Image Processing (SPIE-VCIP'2007), San Jose, California, January 28 - February 1, 2007].

Category:

Science & Technology

Tags:

• Pose-Estimation
• Particle-Filtering
• Sylvester's-Equation
• Lagrange-Multipliers