3D Video Capture with Kinect

By combining the color and the depth image captured by the Microsoft Kinect, one can project the color image back out into space and create a "holographic" representation of the persons or objects that were captured.

For more information and source code, check my Kinect project page at http://idav.ucdavis.edu/~okreylos/Res...

Based on the reverse engineering efforts of user marcan42.

• Category

  Science & Technology

[Tariq Iqbal]

Hi Oliver,

Great job! Really exciting. I am new to Kinect, and facing very basic problem. I am using Kinect SDK 1.6 and VS2010 C#. I don't know how to record RGB and Depth video separately at 30FPS. I can show them, but if I try to write them as separate .png files, then the synchronization breaks and ends up with 5/6fps. Is there any standard way of recording videos and skeletal data? I also don't know if it is possible by multi-threading, and how. Your help will be highly appreciated.

[okreylos]

I can only speak in generalities, but writing PNG images has significant overhead. If you do all in a single thread, your CPU might max out at 6 fps. Most importantly, use multithreading. I use one thread per Kinect and processing stage (capture from USB, background removal, conversion to RGB / depth image, compression, writing) for a fully parallel processing pipeline. Second, instead of PNG either use an uncompressed "raw" image format like PPM, or write directly to a compressed video file.

[okreylos]

In my code, I use a custom lossless delta+Huffman encoder for depth frames, and a lossy Theora video codec for color frames. Both can keep up with 30 fps streams if they run in their own threads on a four year old Intel Core2 duo. On a Core i7, the code can process 2 Kinects at 30fps in real time. 2 Kinects is not an upper limit, it's just the most I've used so far on a single PC due to the USB bandwidth limitations.

[Tariq Iqbal]

Thank you very much for your reply. Is it possible to download your code from online. So, that I can take a look, and modify my code, or use your method for video recording. Thanks.

[okreylos]

My project page with source code is linked from the video description.

[Arron Parker]

wow incredible my names arron im a street magician designer/inspirer i was wondering i need a good camera and my webcams so slow to capture could i use my kinect to record videos from my pc some how heres the video id like to record again thnks support from you regarding this matter would be great

[okreylos]

You could; but if you just want to record regular video, there are much better-quality webcams for less money. The Kinect can only do 640x480 at 30 Hz, or 1280x1024 at 15 Hz.

Newer Linux distros have built-in Kinect support, they treat it like any other webcam.

[Benjamin Kingsley]

if you were to set up multiple kinect cameras you would be able to make a complete 3d re-image of your room??, if you could do that would it be at all possible to apply it to something like Skype?? and then from that area would it be at all possible to create a hologram??

[okreylos]

Yes; the video is recorded into a 3D geometry stream, and can be played back from any point of view, and on any type of display (including stereoscopic). My videos "Picnic Day Exhibit Test - 3D TV" (watch?v=wM65Gx08A0M) and "Exploring the El Mayor-Cucapah Fault Rupture Using LiDAR Viewer" (watch?v=_e2DsZ40sqg) are already in 3D, and should work on your 3D TV depending on your YouTube viewer settings.

[Tha Pixeltripper]

Whoa. That's awesome.

[ethan ivory]

Cool

[osearth]

proof of a 2D universe masquerading as more

[Toan Nguyen]

This is Amazing !!! I've searched through Internet for a good way to merge color and depth, all are pretty good but this is the best. Yay you man!

[Tariq Iqbal]

Thank you very much. I will try it.