Automatic Reconstruction of Personalized Avatars from 3D Face Scans

We present a simple algorithm for computing a high quality personalized avatar from a single color image and the corresponding depth map which have been captured by Microsoft's Kinect sensor. Due to the low market price of our hardware setup, 3D face scanning becomes feasible for home use. The proposed algorithm combines the advantages of robust non-rigid registration and fitting of a morphable face model. We obtain a high quality reconstruction of the facial geometry and texture along with one-to-one correspondences with our generic face model. This representation allows for a wide range of further applications such as facial animation or manipulation. Our algorithm has proven to be very robust. Since it does not require any user interaction, even non-expert users can easily create their own personalized avatars.

More infos @ http://www9.informatik.uni-erlangen.de/publications/publication/Pub.2011.tech...

Category:

Science & Technology

Tags:

• kinect hack
• kinect face scan
• kinect avatar
• kinect 3d scan

[abram730]

@roidroid Sounds like fun..

I just had an idea.

you could do a kinect fusion like systrem with a plenoptic camera.. I don't know how much processing power it would take though. It would work outside in the day.

It's a light field camera.. works sort of like an insect eye.

[roidroid]

@abram730 it works really well with space photography. Giving you an apparently indefinite "virtual" shutter speed with no motion blur and increasingly less noise.

It would be interesting to experiment with an HD video camera's highest iso setting, highest fps, and lowest shutter speed. Then run hours and hours of the resulting blindly-dark, noisy, high-light-sensitivity, low-motion-blur footage through a super resolution algorithm. I wanna play with virtual shutter speeds :D !

[abram730]

@roidroid Thank you. I just thought in my head how I would do it.. I couldn't find the actual terminology, so thank you. I'd imagine that this sort of thing started way back with FBI, CIA, Military... people entering in markers by hand to improve video on magnetic tapes.

[roidroid]

@abram730 "super resolution" is the name of that term.

(A few weeks ago i found myself googling like mad trying to remember it, glad i wrote it down this time)

[abram730]

You should use many frames from the RGB to create a higher detailed texture.

Similar to how you used many frames of depth data.

[inkanus5jg]

Can't wait for this technology to be implemented in gaming. Imagine a game that uses your own face as a template for the staring role of a game.

[roidroid]

1:51 can't this non-rigid registration be used on it's own, straight away? Why do you bother with the temporal smoothing and gauss filters first at all? it looks like the non-rigid registration is all you'll ever need.

[smoke9206]

all the textures are 'not sharp', bad quality, don't know how to say that. Can it be fixed? You know, to make textures better quality. If yes, why you are present us version with bad quality textures? :P anyway, the idea is great.

[FurryEngine]

great project!

[bricemathieu]

they need to make it a standard. not just one game. we need to add our acquaintances inside every single game. its not so hard to do. i never liked the miis anyway. the morphable model is the key, in that it can make everyone look beautiful and still similar to the original :)