Simple Augmented reality with 3D-2D correspondences and optical flow (L&K algorithm)

An example of augmented reality application which draws a virtual cube above a real cube. The method used is based on finding 3D-2D correspondences and follow them using optical flow (Lucas-Kanade algorithm).
The accuracy is not too good, but is enough for a conceptual example.

Category:

Science & Technology

Tags:

• OpenCV
• augmented reality
• optical flow
• kanade
• computer vision

[Neon360]

neat! I want to see more action/movement. what happens if you pick it up?

[dehabu]

@Neon360 This example app is based on "one-time detection" of features and then the rest of time is used an optical flow algorithm. So, you shouldn't occlude keypoints. If you occlude them, then the virtual scene disappears.

I'd like to remark this is only a conceptual example of using motion flow on augmented reality applications. If something more robust is desired it would be necesary to use natural features descriptors (SIFT, SURF, Ferns, etc) combined with more elaborate forms of tracking.

[mrgermanros]

@dailydols We get the camera pose from the motion by using tracking (optical flow). Then we only apply the usual geometric methods to obtain the pose.

[dailydols]

what do you use for pose estimation? POSIT?

[mrgermanros]

@dailydols Actually the intrinsics are just an initial estimation, but the main problem is that 3D point estimation is not very good.

[dailydols]

looks like the camera intrinsics are not well calibrated. you might want to check this.