Content-Preserving Warps for 3D Video Stabilization

This video describes our research on video stabilization to be presented at SIGGRAPH 2009.




We describe a technique that transforms a video from a hand-held video camera so that it appears as if it were taken with a directed camera motion. Our method can adjust the video to appear as if it were taken from nearby viewpoints, allowing for 3D camera movements to be simulated. By aiming only for perceptual plausibility, rather than accurate reconstruction, we are able to develop algorithms that can effectively recreate dynamic scenes from a single source video. Our technique first recovers the original 3D camera motion and a sparse set of 3D, static scene points using an off-the-shelf structure-from-motion system. Then, a desired camera path is computed either automatically (e.g., by fitting a linear or quadratic path) or interactively. Finally, our technique performs a least-squares optimization that computes a spatially-varying warp from each input video frame into an output frame. The warp is computed to both follow the sparse displacements suggested by the recovered 3D structure, and avoid deforming the content in the video frame. Our experiments on stabilizing challenging videos of dynamic scenes demonstrate the effectiveness of our technique.

More information at:
http://www.cs.wisc.edu/graphics/Galle...

• Category

  Science & Technology

[Aseem Agarwala]

Hello, all, a public release in Adobe After Effects has been announced. Look at the YouTube video with ID ATuUTFP5ck4 ("Sneak Preview of New Motion Stabilizer for After Effects").

Thanks,

Aseem

[Iamgercornholio]

awesome, we could use it on Colverfield! You would ba able to tell what is going on^^

[genewitch]

It's because they look like they're dolly shots. Even steadycam isn't that stable.

It's actually really impressive technology; and it looks like it's a plugin, it's "rendered" so that means you have to wait to see the output, and i bet there's a thousand different options you can play with, but there's liable to be an "easy mode" where you let it use saliency maps to find the subject and track on it.

All in all, this looks AWESOME!

[athanasiothegr8]

What is the crop factor of the original image and the final render, and what parameters are mandatory to have a simple stabilization?

Looks realy good btw!

[stevens37y]

Youtube stabilisation function does something similar.

[TiagoTiagoT]

Btw, i think part of the issue is motion blur. Though, since you got such a detailed insight on the motion of the scene, can't you apply some smart deconvolution or some other technique to recover the lost detail in the blurred frames?

[TiagoTiagoT]

There is still some sort of subtle but eerie artifact, specially in footage shot while walking, i can't pinpoint what is wrong, but can notice some sort of cyclical change that happens more or less at the same frequency as the steps, feels a bit like that uncanny valley effect.

Nonetheless overall this is an amazing technique; 'grats on the amazing work!

[amfgrt]

Such artifacts occur in almost all test videos. This is because the camera is moving along the vertical axis while walking. The stabilization can only compensate rotation and viewing direction of the camera and lens distortions. But there is no way to change position of the camera, its path within the scene will remain the same unless we have a full 3D model of the scene.

[xxlray]

Are you cheating a bit? The comparison scenes are much less zoomed then the scenes for you approach.

[Marie-Claire V.]

cool!!!

[Just90sPop]

when the fuck is going to be this software out !!!!! i need it like nowwww ,i want it !!!!!

[blackfuse27]

so where do i get this?