3D Facial Performance Capture using Kinect

This video shows an example of how 3D facial motion performance can be captured without markers using a Microsoft Kinect camera (data shown in green). The resulting facial animation is shown in purple. The color video is just shown for reference and not used in the process.

This is work by M. Breidt http://www.kyb.tuebingen.mpg.de/nc/employee/details/mbreidt.html , H. H. Bülthoff and C. Curio http://www.kyb.tuebingen.mpg.de/research/dep/bu/ce.html at the department for Human Perception, Action and Cognition of the Max Planck Institute for Biological Cybernetics, Tübingen, Germany http://www.kyb.tuebingen.mpg.de/?id=43

Thanks to Joachim Tesch and Johannes Lächele for their help in recording raw data from the Kinect!

Category:

Science & Technology

Tags:

• 3D
• facial
• performance
• markerless
• capture
• animation
• kinect
• face

[MPIVideosProject]

Just a short note that I cannot properly monitor the youtube comments, so please get in touch via email if you want a reply. Thanks!

Martin

[ohmss006]

Hey man, I think this is really great what you have here! Can I ask, how did you do this exactly? because everywhere i looked regarding Kinect + Facial Capture, NO ONE has done exactly what you have! Is the facial capturing done through the Kinect???

Probably can you do another video but including the actual video reference for the motion capturing?

I hope you dont mind me asking, and hope to hear back! :D

[MPIVideosProject]

@ohmss006 - the video is included in the top right. That's the color video the Kinect sees, just for reference, not used by the system at the moment

[MPIVideosProject]

Thanks for the friendly comments. Right now, the code is not publicly avaliable but we would be interested in suggestions for possible collaborations.

[acf2802]

Cool and all, but what is the point of sharing this video publicly if you are going to sit on both the code and the implementation details?

[funkboxing]

I have a lot of experience with the Blender 2.5 Python API and I would love to help get this implemented. Please contact me if you would be interested in collaborating. Thanks for this- amazing.

[imegatrone]

I am very happy to see the vidoe after you give this This video shows an example of how 3D facial motion performance can be captured without markers using a Microsoft Kinect camera

[bundawartini]

I Love The Video It Can Increase My Knowledge This video shows an example of how 3D facial motion performance can be captured without markers using a Microsoft Kinect camera

[NganaJHone]

Steady I Really Like This video shows an example of how 3D facial motion performance can be captured without markers using a Microsoft Kinect camera

[Ondelendo]

Good, I like that you share this video, I wish success always This video shows an example of how 3D facial motion performance can be captured without markers using a Microsoft Kinect camera

[bebeheuy]

Nice Video That You Share , So Very Nice Thanks You This video shows an example of how 3D facial motion performance can be captured without markers using a Microsoft Kinect camer

[AntoMelta]

I Really Like The Video From Your This video shows an example of how 3D facial motion performance can be captured without markers using a Microsoft Kinect camera

[willamricard]

Your Video Is Very Useful Sharing This video shows an example of how 3D facial motion performance can be captured without markers using a Microsoft Kinect camera

[xChrizmax]

The middle one at the bottom looks like Abe Lincoln :P