2 Kinects 1 Box

First test of merging the 3D video streams from two Kinect cameras into a single 3D reconstruction. The cameras were placed at an angle of about 90 degrees, aimed at the same spot in 3D space.

The two cameras were calibrated internally using the method described in the previous video, and were calibrated externally (with respect to each other) using a flat checkerboard calibration pattern and manual measurements.

New software version with support for multiple Kinects is available for download at
http://idav.ucdavis.edu/~okreylos/ResDev/Kinect

Category:

Science & Technology

Tags:

• Xbox
• Kinect
• 3D
• video
• multiple
• cameras
• merged
• reconstruction
• calibration
• checkerboard

[DrKaito10]

Nice work. Are you still using manual measurement for your external calibration or have you found/made some software?

[okreylos]

@DrKaito10 My development branch has a new calibration method (intrinsic and extrinsic), but it's still semi-manual. You have to manually fit homographies to a sequence of images of a semi-clear rectangular grid in the 2D depth and color images. This is because the depth image is so fuzzy that automatic grid detection methods, at least the ones I've tried, don't work well enough. But the process is quick, and the results are very good.

[LegendaryAdrenaline]

What you are doing here is amazing to me (from a film technology standpoint). If you had 3, you could theoretically shoot a scene in a room, and operate the camera without getting in the way of your actors, no?

[okreylos]

@LegendaryAdrenaline You would need a larger number of cameras to avoid occlusions, i.e., object shadowing; but otherwise, that's exactly the idea.

[demonicowl]

The problem with multiple Kinects is that you must have multiple USB Controllers. Does your computer have this configuration or did you find a way around it? I'd be very keen to hear how you did this!

[okreylos]

@demonicowl Most desktop PCs have two USB busses to the outside; typically one to the front panel, and one to the back panel. That's what I'm using. Alternatively, you can buy additional PCI/PCIe USB cards, OR you can buy a USB 3 card and hope that the drivers work. A single USB 3 bus can theoretically support around 15 Kinects.

[okreylos]

@bengui122 So you started with a flawed premise (who said anything about xbox?) and jumped to a counter-factual conclusion (you're seeing it here, and I'm not faking it, ergo it must be possible). In other words, stating that you work for Microsoft was redundant.

Sorry, I kid. No offense. This was the perfect setup. :)

[Shititude]

BUT WILL IT BLEND ?

[bnanacheese2]

WOW! I have had a glimpse into the future!

[dwightaperry]

Keep up the good work, amazing

[esopbelac94]

So uh, is it posible for a 90-110degree ish wall of kinects all lined up to get perfect 3D mo-cap?

[tazbo28]

So is the main issue with the infrared sensors on teh kinect? It seems like those sensors which throw out those little dots all over everything are not high enough resolution. the noise in the result makes it look like its not getting enough depth data. i wonder if or when the sensors can be upgraded to allow for a finer resolution. similar to the way normal cameras work. is it hard to get multiple kinects to work together?

[ThisCanDestroyYou]

Dude, you are the man, keep it going & bring us the future!

[LegendaryAdrenaline]

@okreylos I look forward to seeing that idea implimented. Keep up the good work!

[BurningdiverUK]

@arethemad i would imagine it would be 4 and above depending on the rooms dimensions, the potential is amazing, with this use of multiple Kinect's you could do all sort of crazy stuff

[mgw31660]

could this work with motion tracking? I know that you can use one kinect, but would to be any better?

[brunolealxavier]

this is the video im talking about: /watch?NR=1&v=1Qx8NzuSSJ4

[brunolealxavier]

okreylos, i could see in one video by another author that the kinect can "recognize" mirrors, so i wonder if it could help you on this or another experience.

good luck (and sorry for my poor english)!