@martync1988 If you're referring to the non-lipsynched audio track, that's due to me not correcting for it during capture.
That said, latency of the 3D video is quite low. The Kinect delivers 30 images per second, so it has base latency of less than 30ms for internal processing, and a few ms to send a frame pair over USB. Then there are a few ms of unpacking and reconstruction time on the PC side, plus display latency. I'm estimating overall end-to-end latency in practice to be around 20ms.
Why not put a porarising filter on the ir emitter of one of the kinects, to polarise the light, and then put a polarising filter on the camera lenses of the OTHER kinect so that it can't see the first kinect's ir light?
Using multiple xbox kinects would be possible in a phased array configuration. Like he said, this would require additional hacking of the Xbox kinect. You would have to "chop" the signal so only one xbox kinect were on at a time. Ok internet, get to work!
2Kinects, is possible, but I think a large amount of work would have to be put into combining the two's point data into a cimbined, averaged, mesh.
Of more importance to me would be the ability to use two cameras in the motion capture setups. Though I think this would require quite a bit of tweaking to the NITE code... is that even opensource? I really want to see it happen though.
would it be possible to create an anti-blackshadow buffer? I mean. to fill the blaknes with the last non shadowed 3d images. this would not work for your body but it would for the background and anything still... kind of a 3d video compression: Only showing and what is changed (exept of non-data-shadows) in real time and what´s not just live it like it was.
I am sorry I did not read all 300 or so comments to this video, but let me suggest a hack for the multiple Kinects scenario: the basic idea is to fine tune the bandwidth of the infrarred emitter, so that every kinect emits and detects only a particular bandwidth of the infrarred spectrum, in a similar way that channels work in WiFi. Whether this can be done done through the API of the opensource driver of the Kinect is unknown to me, but the modifications to the hardware/firmware would be subtle
It's probably far fetched but I was wondering. If you were able to alter the frequency of the projected pattern of infrared in different kinects and correspondingly limit the cameras to that particular kinect's frequency, would that be an effective way of eliminating the interference that you described?
If full 360 degrees becomes possible, we can use it in movies and we can see movies in all angles as the watcher desires. I can watch it anyway I want! Damn... that would be tight... if I watched Transformers or Iron man, etc... with it, it would be Jesus!
Interesting! I wonder if small games studios could use these as motion capture for 3d model animation, rather than large, multimillion pound camera rooms?
Maybe it would be theoretically possible to use multiple units by polarizing the infrared light the kinect units send out and using corresponding polar filters on their detection units ?
This is so awesome! After you moved the view around, at 0:27, you are actually not looking into your camera, right? But since the angle is moved in the computer, it still looks like you do?? That's very confusing =P.
@Anto0044 Good question, I don't recall. But judging by the direction of the "shadows," I am looking into a virtual camera that's maybe a foot off from the physical camera.
You're getting 100FPS from a single unit, right? So if you have four Kinects, 90 degrees from each other, can you not strobe each Kinect individually at 25 or 30 FPS? This way each Kinect is timed so that only one is active and projecting its pattern and capturing data, thus no interference. Because the results would be interpolated it may give funky results with fast motion, but overall I think it would work well.
@gge5 The kinect sends depth and color images at 30 fps each. The higher frame rate displayed by the program is because the two types of images appear independently, so the program updates the display at least 60 times a second. And since the data arrives "bursty," and the software median-filters displayed frame rates, it shows up as higher.
@okreylos - Is the laser diode rotatable? From my observations of ifixit's teardown, it looks like it's just a tiny component that could probably be rotated 90 degrees. That way, with polarizers in front of the camera, you could have two Kinect's capturing 3D video of the same scene at the same time. The cameras are not polarization-sensitive, so it should work.
Maybe you could programm kinect to save the images in a queue and if they get this 'shadow' and the software has this 'shadowed' area saved, it loads it and display this image until it gets a new image of the 'shadowed' area not 'shadowed'.
I'd bet we can change out the IR source and filters for each connect to run with it's own Infrared frequency so that two to three will not interfere with each other.
You know the composite shaped (like at mcdonalds and banks and such) security cameras that seea rounded image? if you could get them to record inwards and fully cercular and get into one (a very big one) you should be able to get both a 3D and holograpgic image (to be viewed fron different prospectives)
you´re a Genius, its possible and ones retarded when say me "Microsoft copies Nintendo" but not´s true, i´ts something original when must made it, sorry 4 my bad english, over 9000 thumps up!
Possible solution for multiple kinects: Just switch around, so that at the time only one infrared pattern is active, then switch to the next kinect etc.
That is not true , ROS guys have connected two connects together looking at the same seen and it works well. There is still interference but very little
you got a 3d glasses! check the free izd3 driver to convert anaglific games on your pc, and then try to cobine whit this, im very interesting in that technology too greetings from mexico! :D
@getsdhc Not necessary; the Vrui toolkit underlying all this already does many kinds of stereo, including anaglyphic and several modes compatible with 3D TVs.
what if you triangulated the cameras out doors at a distance? would distance fix the rays from interfering or would it be harder to get a 3d reconstruction at a distance?
I'm not sure if someone already suggested that, but wouldn't it be possible to adjust the function of the Kinects to switch off and on in a quick sequence so that there's only one being on at one moment in time, kind of like how 3D TVs work?
The Kinect coupled with image to sound translation could be an amazing aid for visually impaired people. It could be couple to patterns of reflective studs on items and bar or QR codes to identify objects @davidfcox
a device could use modulated infra-red light to project a pattern and tuned receivers which would allow multiple devices to work at the same time. @davidfcox
Maybe this has been said already but is it possible to move the camera around an object and capture images at different times to recreate that object and still allow it to move freely in the environment? I'm think capture multiple images of a person at different angles and the recreate that person as they move.
Another idea: perhaps you can use some form of time-slicing, ie turn kinect projection on/off at fixed intervals, or even take sliced input from different kinects and combine
Could you not Cycle the kinects? Like Kinect 1 would take up a milisec of recording data than shut off, and kinect 2 would turn on and record and turn of and than Kinect 3 would turn on and record and shut off. and just repeat the cycle. The may be some delay, but I would be nifty to see if it could be done.
this technology combined with the google earth street views cam used by a game company with great funding like rockstar could ammount to an actual real-sized 3d reconstruction of a major city for a next GTA game... wouldnt that be something?
If not polarising the light, what about alternating the kinects at a fast rate, ie, like a shutter system for both the lens and the IR (hardware or software)
About polarising the IR, it would definitely be worth trying. The only problem would be if the polarisation of the IR dot changed too much when it hit certain surfaces. Perhaps not a huge problem?
@okreylos, maybe if you had 2 kinects both aimed toward each other from opposite sides of the room, they would work fine and not see each other's IR dots?
I expect there would still be a gap around the sides of each object where neither the front or back kinect could see, but perhaps that could be interpolated?
Registration of the two kinects to know their relative positions would be interesting. You could point to a thin object in both or just manually specify the distance between them.
But really, I was thinking that if you altered the spectrum input/output from infrared with a different Kinect, you should eliminate interference by making sure they're emitting/picking up different wavelengths. That would entail some fiddling around with the hardware though. (Warranty voiding!)
You may want to consider mentioning the names (or at least usernames) of the people who give you these ideas (the mirror), simply because people get all crazy and sue-happy over this kind of stuff. Aside from that, it's just nice to give credit to the people with the ideas...I'm sure they would appreciate it.
Just a thought from a non-techy person who enjoys seeing what you're experimenting with. :)
@lindseykei1 Interesting point. Of course, proper credit needs to be given. One of the nice side effects about having the discussion here is that all suggestions (good and otherwise) are listed in the comment thread, with the user name of the person who made the suggestion, and a (relatively accurate) time stamp.
I hope that's OK for people; otherwise I don't really know how to handle it.
All these people talking about alternating the two kinects or modulating them are talking bobbins.
Use a polarising filter for each kinect, and make sure that the two kinects have polarising filters rotated through 90 degrees. That way you could have two running independently and capturing data at once.
Maybe it would be possible to use multiple Kinetics by modulating each one at "different frequencies"... Well, it is not modulation per se, but you get the idea: You could turn-on/off each knetic by a few microseconds so their infrared rays won't interfere with each other... then you can reconstruct the full image aftterwards... It may cause some jitter, and it may require a firmware change as I don't know if it is possible to have control over the infrared from "outside".. but it looks doable
This would probably escalate the complexity of the programming a bit, but.. wouldn't it be a possibility to achieve full 3D projection with multiple Kinects by making them record the frames sequentially? ie. if Kinect A records frame 1,3,5,etc and Kinect B records 2,4,6,etc, or at a larger span of frames and both "strobing" on and off. That would eliminate the IR interference problem and produce two separate captures from different angles at minimal time difference.
Here is an idea for running multiple Kinects simultaneously which may require mods of the Kinect software.
Put multiple Kinects around the room. Have Kinect 1 capture a 3D image of the room. Then shut down/block Kinect 1and take a frame with Kinect 2. Repeat with any other Kinects. Use your software to merge all the images.
This solves the problem of having the infrared light interfere with each Kinect without major hardware mods.
P.S. Do you now have the right to patent your ideas?
In theory, couldn't you attach a GPS unit to each Kinect? That way you'd have a real world reference point for each Kinect, and could parse the data correctly based on those distances between the Kinects? I think you could definitely build a 3D model of a room and not have problems as long as the Kinects had reference points of where the other Kinects were. (Obviously, an alternative would be to just hardcode it assuming Kinects were all exactly 10 feet from each other)
@okreylos, surely you could use multiple Kinects using polarising filters? If the two kinects used a polarising filter over the IR array, and a polarising filter in the same orientation over the sensor?
You could rotate a second polarising filter through 90 degrees for kinect number 2, and they shouldn't interfere with each other.
@MisstraD Good idea. You can find Polarizing filters in every lcd display. It's possible that you need extra IR-leds but they are cheap. Good project.
@HvdHaghen When the camera uses a lazer, the light must be already Polarizeed. Than one of the cameras must be turned 90 degrees. Good chance there is a Polarizing filter on the camera. Easy to find out when you turn a Polarizing filter in front of a working camera.
Can you just process alternate frames from the Kinect when building up your depth map and image?
If so, why not have the lights on two Kinects set up so that they're only on when the other Kinect's lights are off? Do this with 4 of them and- while you'd lose out on update rate- you could have an all-round view in 3D (at about 25fps, which isn't particularly bad).
another idea I had was an Xbox360 accessory(like Kinect) that was placed above the gamer (in the same position as the light fixture that more than likely was already there when the home was manufactured)It could be hard-wired there replacing(or along with)the light.This unit could project IR downward(a constant beam of UV light spinning like a fan blade).This light(along with the Kinect looking directly at the gamer)could increase the 3D/4D image of the gamer....just another thought,lol
Just spit-balling some ideas. Please do not stop your journey of improving the Kinect, you may just revolutionize the gaming industry. Good luck sir!!
Keep in mind that you could also use "flex mirrors", they are a plastic type of mirror. Maybe these can be used to concave/convex the image? They make them in all sizes (including wall size). Also a "tint" could be applied, similar to the tint on car windows, but you could use a UV filtered type of tint. I think if two Kinects are looking right at each other they will blind themselves (like looking directly into a flashlight), but maybe if the IR were two different colors they might worktogether
This is just an idea coming out of a mind that has no experience with infrared light but could you possibly have each connect using a slightly varied frequency of light, only picking up that frequency so that each connect does not interfere with each other. Maybe have a lens that you put over the projector to change the frequency, then another lens for the receiving camera to return tot he original frequency to avoid any major hardware modifications.
Awesome work man, I'm totally impressed! Can you put opposite polarizing filters on the two Kinects to separate their IR signals? It would cut the light down by 50%, but that might still be okay for short ranges…
Wikes and Wilson built a body scanner using mirrors and fringe imaging, so my guess is, it can be done with mirrors, I also think it can be done with filters.
Based on testing by Jaques from openkinect team he did not see a great deal of interference when kinect was 6 to 7 feet and 120 degrees apart. I did test mirror and you can pick up reflected dots but maybe difficult to calculate if they were a reflection or not. The scan room and build 3D mesh models seems like a great solution and the wearable kinect idea or the robot kinect that scans room is a good low cost option. ROS team as working on this and I believe have 6 kinects hear
How about one kinect and multiple video cameras which are placed at a specific depth and location. So you know where the images are in respective to the kinect. Also why not block the IR projector of the other kinects and then use the IR from the first kinect to see for the rest. Although this could cause issues with objects blocking the IR. But I think using multiple cameras with one kinect should work.
@awasteofaustinstime multiple cams with one kinect would help fill in some of the shadows but would not supply depth information for a perfect 3d environment. Multiple kinects using the same ir beam would give distorted depth info and since the processing is done onboard i don't see a simple software solution either. I think.
You could connect an arbitrary number of capture devices and go through them round-robin so they don't interfere with each other.
It might also work fairly well to make a model of the space, a model of the subject, by moving around the capture device and then use the single device in a fixed location to animate the models and patching in live-data for things like facial expressions.
Something as simple as a polarizing filter on light source and camera might also work up to a certain number.
Actually, if you use a mirror, shouldn't it think the mirror has depth, and make your program make your room longer? Mirrors are not the same from all angles.
@Beeh0 Think of a mirror not as a glass plate, but a portal into a virtual world where left is right (actually, where front is back). So (without having tried), a Kinect would see my reflection in a mirror simply as another Oliver who is sitting behind the mirror. Now here's the trick: if I know where the mirror is, I can flip that virtual Oliver back into real space, and can make it match the missing half of the real one. Tricky math and programming, but could work.
I do have 1 idea for you when it comes to using 2 kinects. I noticed the frame rate isn't perfect - which is a good thing. If you were to put 2 kinects on opposite ends of the room, you could make them strobe infrared light asynchronously and try having them only process the frames in which the light is on. So in other words, in 1 frame, one of the kinects will flash an infrared strobe. In the 2nd frame, the 2nd kinect will flash its strobe. Then it reverts back to the first kinect
@okreylos The only problem with that mirror stuff is that if it is painting an IR grid on everything the grid would not extend into the mirror in a regular way.
@okreylos Are you sure? It wont get any depth perception from the mirror, its a flat object, the image is virtual. It would give the camera a different perspective, but the IR detects a flat surface. The programming would have to mix the origional 3D image with a 2D image. Im guessing upscaling 2D to 3D doesnt work out easily, hence the need for IR stuff.
@lfo98 Yes, as long as the mirror is IR-transparent (not all glass is), the Kinect wouldn't be able to distinguish virtual objects in the mirror from real objects. Think about it. For a camera, or anything that's based on visible light, a mirror does not appear to be a flat object, but a "portal" into a virtual world.
@okreylos Wouldn't the mirror reflect the IR light back into the room and interfere with the capture of the room itself, or would that reflected light only reach the surfaces not visible from the Kinect's angle?
@Beeh0 This won't be possible. Kinect projects a structured pattern and then uses the deformation of the pattern to learn the depth of every pixel. There will not be any distortion of this pattern on the mirror. Our brain can perceive depth through the mirror but unfortunately Kinect won't be able to do that.
Part of my comment was removed so I'm having to add it here,
As long as you ensure the kinects are atleast at rightangles to each other, they wont be able to see each other's projector IR field. (even with something like a sphere in the middle of them, they wouldn't be able to see each others IR dots).
@JonnyCasey I think you're forgetting something; someone else please check this. Even at more than 90 degrees apart, the two could still see each other's'points. Imagine the two projectors forming a V, and the tip of the V is the point where their lines of sight intersect. Now place a flat surface at that point, which is parallel to the baseline of the V, i.e., like this: V __ Then according to mirror laws, the two Kinects will project right into each other's cameras.
@okreylos Ah very true. Having two facing each other would not have this problem am I right? (if you also incorporate the blind spot technique I mentioned bellow)
@okreylos Ah very true. Having two facing each other would not have this problem am I right? (if you also incorporate the blind spot technique I mentioned bellow)
This would only give a back face though, It wouldn't help with the shadows would it. :(
If the project IR field of the kinect is less than 90° then you should be able to possition 3 or even 5 kinects so they do not overlap.
However, You can simply add a tiny blind spot to each of the kinect's IR projectors for the opposite kinect to reside in so as to prevent them from blinding each other. Then intorpolate the Z depth from the surrounding Z values around the blind spot (much like the human eye works). Use bluetak to add a blind spot to the IR emitters.
OK, I'm a bonehead. We're discussing the ins and outs of using multiple Kinects for full 360 degrees capture, and I forgot that I have an old video showing our previous 3D video system. I want all of you to watch the "Immersive 3D Video" video I posted as a video response. It shows what full 360 degree capture can look like. Don't be put off by the quality; it's an old renderer using only points and no triangles.
@okreylos You can simply add a tiny blind spot to each of the kinect's IR projectors for the opposite kinect to reside in so as to prevent them from blinding each other. Then intorpolate the Z depth from the surrounding Z values around the blind spot (much like the human eye works). Use bluetak to add a blind spot to the IR emitters. As long as the projectors are more than 90° from each other you wont have a problem with them seeing each other's IR projections.
I should add: the reason I'm so impressed with the Kinect is that I already have a bunch of other 3D cameras in my lab, so I know their limitations very well. In comparison, this thing is awesome.
What about using just another standard Web cam, to feed more informaiton into the 3D scene? It would be cheaper than a second kinect too. Maybe you could have the web cam at a KNOWN position to the kinect, or use a cube of known size to help with the calibration. Best of luck. I hope I am helping this project, just a little bit. You are the true genious, doing the hard work. OR, polorized lens on the IR emitters, and sensors. 1 kin sees through an up down filter, the other see left-right filter.
@Shakespeare1612 You might not believe me now, but that is one of the first things I thought when I found out the Kinect uses active sensing. There are pure stereo-based 3D methods using multiple regular cameras (I have two clusters of those in my lab), but they have serious problems. Using the Kinect as a baseline should give the regular cameras a better starting point, increasing quality.
If a second kinect throws out an interfering IR pattern, what about using just another standard Web cam, to feed more informaiton into the 3D scene? It would be cheaper than a second kinect too. Maybe you could have the web cam at a KNOWN position to the kinect, or use a cube of known size to help with the calibration. Best of luck. I hope I am helping this project, just a little bit. You are the true genious, doing the hard work.
just wondering...
if the IR light disturb each other then why not use polarized light?
3D cinemas uses that technology and it works at least better than those R/B glasses.
oh and the glasses cost 1 $ each(or free if you go watch a 3D movie) so 2 $ for a set of 2 glasses that probably work better than using no lenses
MrHack4never 3 months ago
Comment removed
MrHack4never 3 months ago
could you use multiple camera filters on the Ir emitter and camera to get a diode effect? a shift of just a few NM should suffice.
disconformist 5 months ago
This has been flagged as spam show
▬▬▬▬▬▬▬▬▬▬▬ஜ۩۞۩ஜ▬▬▬▬▬▬▬▬▬▬▬▬▬
ＴＨＩＳ ＣＯＭＭＥＮＴ ＩＳ ＦＡＮＣＹ！
▬▬▬▬▬▬▬▬▬▬▬ஜ۩۞۩ஜ▬▬▬▬▬▬▬▬▬▬▬▬▬
theblange23 6 months ago
This is fantastic, i'm curious to the audio delay, what is it that causes the processing delay?
martync1988 6 months ago
@martync1988 If you're referring to the non-lipsynched audio track, that's due to me not correcting for it during capture.
That said, latency of the 3D video is quite low. The Kinect delivers 30 images per second, so it has base latency of less than 30ms for internal processing, and a few ms to send a frame pair over USB. Then there are a few ms of unpacking and reconstruction time on the PC side, plus display latency. I'm estimating overall end-to-end latency in practice to be around 20ms.
okreylos 6 months ago
@okreylos This half-frame delay is nasty, and I consider it nearly impossible to compensate it in realtime. Really annoying...
dailydols 2 months ago
Why not put a porarising filter on the ir emitter of one of the kinects, to polarise the light, and then put a polarising filter on the camera lenses of the OTHER kinect so that it can't see the first kinect's ir light?
theguymjp 10 months ago
2 Kinects facing each other with polarizing filters.. no big problems with reflected light. Then do it with 4.. No strobe hack required..
You could ask Microsoft about a Kinect revision with this feature.
Why wouldn't they want to sell 4 per costumer?
Simple "Hey Microsoft do this and we will buy 4 Kinects each"
abram730 10 months ago
Using multiple xbox kinects would be possible in a phased array configuration. Like he said, this would require additional hacking of the Xbox kinect. You would have to "chop" the signal so only one xbox kinect were on at a time. Ok internet, get to work!
mhoyer01 11 months ago
2Kinects, is possible, but I think a large amount of work would have to be put into combining the two's point data into a cimbined, averaged, mesh.
Of more importance to me would be the ability to use two cameras in the motion capture setups. Though I think this would require quite a bit of tweaking to the NITE code... is that even opensource? I really want to see it happen though.
WACOMalt 11 months ago
Doritos crash couse 4 the kinect would be EPIC!!!
Flair9279 11 months ago
Comment removed
UberSecret 1 year ago
looks like borderlands
mrlolbob12 1 year ago
would it be possible to create an anti-blackshadow buffer? I mean. to fill the blaknes with the last non shadowed 3d images. this would not work for your body but it would for the background and anything still... kind of a 3d video compression: Only showing and what is changed (exept of non-data-shadows) in real time and what´s not just live it like it was.
tomatoso27 1 year ago
I am sorry I did not read all 300 or so comments to this video, but let me suggest a hack for the multiple Kinects scenario: the basic idea is to fine tune the bandwidth of the infrarred emitter, so that every kinect emits and detects only a particular bandwidth of the infrarred spectrum, in a similar way that channels work in WiFi. Whether this can be done done through the API of the opensource driver of the Kinect is unknown to me, but the modifications to the hardware/firmware would be subtle
Aswarp 1 year ago
@Aswarp Ok, now I watched the other videos through and I know that you made it. Congratulations! BTW, I obviously meant wavelength, not bandwidth...
Aswarp 1 year ago
@Aswarp see my comment below. Wavelength is, of course, inversely proportional to frequency, so this is basically the same idea.
beelzebobo 1 year ago
lol one kinect camera makes 2d sex
bigcheese123kid 1 year ago
It's probably far fetched but I was wondering. If you were able to alter the frequency of the projected pattern of infrared in different kinects and correspondingly limit the cameras to that particular kinect's frequency, would that be an effective way of eliminating the interference that you described?
beelzebobo 1 year ago
@beelzebobo that sounds like a sweet idea! Seems like that would solve the problem
impulse255 1 year ago
Strange.. I had a dream about this many years ago.
shawn106 1 year ago
Mean need buy other a Kinect for behind background? What game can effect during play ? its very weird
ironheartbj18 1 year ago
isn't that sorta how they filmed those scenes from the matrix?
spacegravity4me 1 year ago
why do you look like the guy from lost??
TxJ0k3r 1 year ago
the bigger question is... WHO THE FUCK CAN AFFORD 3 OR 5 KINECTS!?
Bronx1875 1 year ago
how do you use kinect to record ?
declanhay 1 year ago
microsoft should hire this guy.. maybe learn something from him..
karatzification 1 year ago
If full 360 degrees becomes possible, we can use it in movies and we can see movies in all angles as the watcher desires. I can watch it anyway I want! Damn... that would be tight... if I watched Transformers or Iron man, etc... with it, it would be Jesus!
SourceOfInformation 1 year ago
Interesting! I wonder if small games studios could use these as motion capture for 3d model animation, rather than large, multimillion pound camera rooms?
WtFCBermudaCake 1 year ago
Cross talk would be a issue. its a major issue. It will cause ghost in the image.
thepoliticalstartrek 1 year ago
Maybe it would be theoretically possible to use multiple units by polarizing the infrared light the kinect units send out and using corresponding polar filters on their detection units ?
nexxus230 1 year ago
This is so awesome! After you moved the view around, at 0:27, you are actually not looking into your camera, right? But since the angle is moved in the computer, it still looks like you do?? That's very confusing =P.
Anto0044 1 year ago 2
@Anto0044 Good question, I don't recall. But judging by the direction of the "shadows," I am looking into a virtual camera that's maybe a foot off from the physical camera.
okreylos 1 year ago
this sux
trufiend138 1 year ago
Damn feels like an acid trip....only if i was on acid hmmm. Lets try it.
RTR9 1 year ago
if both sent out the signals at different times then maybe you could have 2 working together-
bumfie 1 year ago
What is the IR thing was sending out IR light in different "colors"?
Rudde47 1 year ago
how did the mirror thing went??
HELLPINAVENGER 1 year ago
What if you alternated the cameras really fast, with pulse width modulation? Much like they do when they make led scrolling signs.
DanielJDukat 1 year ago
YES IT IS POSSIBLE,
see video on youtube:
"Real-time People detection and tracking with multiple Kinect cameras"
alexutubeutube 1 year ago
@alexutubeutube Yes, we've known that for a few days now. Check "2 Kinects 1 Box", /watch?v=5-w7UXCAUJE
okreylos 1 year ago 11
Hey what if you use some sort of mirror and put it behind you while its filming the front of you
Blohhppy 1 year ago
You're getting 100FPS from a single unit, right? So if you have four Kinects, 90 degrees from each other, can you not strobe each Kinect individually at 25 or 30 FPS? This way each Kinect is timed so that only one is active and projecting its pattern and capturing data, thus no interference. Because the results would be interpolated it may give funky results with fast motion, but overall I think it would work well.
gge5 1 year ago 7
@gge5 The kinect sends depth and color images at 30 fps each. The higher frame rate displayed by the program is because the two types of images appear independently, so the program updates the display at least 60 times a second. And since the data arrives "bursty," and the software median-filters displayed frame rates, it shows up as higher.
okreylos 1 year ago
Comment removed
mdrejhon 1 year ago
@okreylos - Is the laser diode rotatable? From my observations of ifixit's teardown, it looks like it's just a tiny component that could probably be rotated 90 degrees. That way, with polarizers in front of the camera, you could have two Kinect's capturing 3D video of the same scene at the same time. The cameras are not polarization-sensitive, so it should work.
mdrejhon 1 year ago
nice degress
V3RM1LI0N 1 year ago
40 FPS =D
hpcompaqpresariov500 1 year ago
Maybe you could programm kinect to save the images in a queue and if they get this 'shadow' and the software has this 'shadowed' area saved, it loads it and display this image until it gets a new image of the 'shadowed' area not 'shadowed'.
Gosh, that was confusing lol
guilhermelhr 1 year ago
@guilhermelhr No, I understand. It's hard, but doable (to a certain extent).
okreylos 1 year ago
the less i understand the comments, the more i know there good
B0XofAWES0ME 1 year ago
thats looks cool, dunt get rid of the "nasty shadows"
komalhassan 1 year ago
that looks cool dude, i dunno why u wud get rid of the "nasty shadows"
komalhassan 1 year ago
Lol you could make a music video! Some of those look like this video! LOL
CarlosSpicyWeiner7 1 year ago
I'd bet we can change out the IR source and filters for each connect to run with it's own Infrared frequency so that two to three will not interfere with each other.
sokol 1 year ago
This has been flagged as spam show
I guess you proved yourself wrong, lol.
Great work.
mahchymk93 1 year ago
I guess you proved yourself wong, lol.
Great work.
mahchymk93 1 year ago
You know the composite shaped (like at mcdonalds and banks and such) security cameras that seea rounded image? if you could get them to record inwards and fully cercular and get into one (a very big one) you should be able to get both a 3D and holograpgic image (to be viewed fron different prospectives)
AvocadoCake 1 year ago
kinect catching pedos ohhh yeahhhh
zend0 1 year ago
Nerds Rule!
thearaban 1 year ago
you´re a Genius, its possible and ones retarded when say me "Microsoft copies Nintendo" but not´s true, i´ts something original when must made it, sorry 4 my bad english, over 9000 thumps up!
sergiocl91 1 year ago
Possible solution for multiple kinects: Just switch around, so that at the time only one infrared pattern is active, then switch to the next kinect etc.
abag2007 1 year ago
lol 3d messy room, the horror
YTBYlover 1 year ago
lol you look like the guy from eminems video "love the way you lie"........
rossneta 1 year ago
That is not true , ROS guys have connected two connects together looking at the same seen and it works well. There is still interference but very little
alphalegion 1 year ago 2
@alphalegion Check my latest video showing two Kinects in the same scene at the same time.
okreylos 1 year ago 4
@okreylos if there was 5 Kinect, would it be like totally 3d. Top, Left Right,Front, Back.
Chainer 1 year ago
you got a 3d glasses! check the free izd3 driver to convert anaglific games on your pc, and then try to cobine whit this, im very interesting in that technology too greetings from mexico! :D
getsdhc 1 year ago
@getsdhc Not necessary; the Vrui toolkit underlying all this already does many kinds of stereo, including anaglyphic and several modes compatible with 3D TVs.
okreylos 1 year ago
is it possible to interface the kinect with any pc games .. fps especially , has anyone done it yet?
Daisycutt3r 1 year ago
@Daisycutt3r Not yet, to my knowledge, but it's bound to happen soon.
okreylos 1 year ago
@Daisycutt3r Someone just did that with Minecraft rofl.
Christianmarquez 1 year ago
@Christianmarquez *punch blocks*
megamaa 1 year ago
Holy shit no dislikes? AMAZING
manacycler 1 year ago
@manacycler So... there's one =) You'r welcome
Syeal7 1 year ago
@manacycler nope 2 confused ps3 fanbois lol
Daisycutt3r 1 year ago
what if you triangulated the cameras out doors at a distance? would distance fix the rays from interfering or would it be harder to get a 3d reconstruction at a distance?
evilpimper 1 year ago
2 kinects? 300 euro?
go buy a f... ps3
TanoSoft 1 year ago
@TanoSoft And then... do what? Play games with it? How quaint!
okreylos 1 year ago 15
@TanoSoft yeah course the ps3 move works so well and is cheaper not
auzziesamurai 1 year ago
@TanoSoft lol your kinda missing the point there fanboi
Daisycutt3r 1 year ago
I'm not sure if someone already suggested that, but wouldn't it be possible to adjust the function of the Kinects to switch off and on in a quick sequence so that there's only one being on at one moment in time, kind of like how 3D TVs work?
harmacek 1 year ago
Very nice you were on English TV
You can fined your self at, BBC iplayer .com
On " click "
Godofhellfire321 1 year ago
Why not use differenet wavelength IR sources in each cam and differential bandpass filters to block out the wavelengths used by the partner cameras?
moses5407 1 year ago
@moses5407 You pretty much took the words out of my mouth. I think that should work also.
EaglesNestOne64 1 year ago
The Kinect coupled with image to sound translation could be an amazing aid for visually impaired people. It could be couple to patterns of reflective studs on items and bar or QR codes to identify objects @davidfcox
bedsdavec 1 year ago
a device could use modulated infra-red light to project a pattern and tuned receivers which would allow multiple devices to work at the same time. @davidfcox
bedsdavec 1 year ago
this guy is so cool, I want him to be the narrater of my life.
bogmire777 1 year ago
Maybe this has been said already but is it possible to move the camera around an object and capture images at different times to recreate that object and still allow it to move freely in the environment? I'm think capture multiple images of a person at different angles and the recreate that person as they move.
clownie126 1 year ago
Another idea: perhaps you can use some form of time-slicing, ie turn kinect projection on/off at fixed intervals, or even take sliced input from different kinects and combine
TinaToaster 1 year ago
I'm too lazy to check all of the comments, sorry.
Suggestion: how about using filters, or even a polarization filter?
TinaToaster 1 year ago
Could you not Cycle the kinects? Like Kinect 1 would take up a milisec of recording data than shut off, and kinect 2 would turn on and record and turn of and than Kinect 3 would turn on and record and shut off. and just repeat the cycle. The may be some delay, but I would be nifty to see if it could be done.
Lastknownuser 1 year ago 4
@Lastknownuser I was about to propose the same thing =)
tekhiun 1 year ago
Can each Kinect broadcast a slightly different wavelength of IR and only pickup that wavelength. This way, the other Kinect's wouldnt get confused.
syndicate5 1 year ago
this technology combined with the google earth street views cam used by a game company with great funding like rockstar could ammount to an actual real-sized 3d reconstruction of a major city for a next GTA game... wouldnt that be something?
Pasteldqueijo 1 year ago
Using Multiple Kinects:
If not polarising the light, what about alternating the kinects at a fast rate, ie, like a shutter system for both the lens and the IR (hardware or software)
sharantulsiani 1 year ago
About polarising the IR, it would definitely be worth trying. The only problem would be if the polarisation of the IR dot changed too much when it hit certain surfaces. Perhaps not a huge problem?
MWhybird 1 year ago
@MWhybird Most surfaces don't polarize reflections. One exception is liquid interfaces like reflections on a lake.
pchapman905 1 year ago
@okreylos, maybe if you had 2 kinects both aimed toward each other from opposite sides of the room, they would work fine and not see each other's IR dots?
I expect there would still be a gap around the sides of each object where neither the front or back kinect could see, but perhaps that could be interpolated?
Registration of the two kinects to know their relative positions would be interesting. You could point to a thin object in both or just manually specify the distance between them.
MWhybird 1 year ago
Nice accent.
But really, I was thinking that if you altered the spectrum input/output from infrared with a different Kinect, you should eliminate interference by making sure they're emitting/picking up different wavelengths. That would entail some fiddling around with the hardware though. (Warranty voiding!)
Hectorflix 1 year ago
Oh, so everybody thought about it. Cool. I knew it was the obvious thing. Keep trying. One of you will do it eventually. Good video.
limafilho27 1 year ago
You may want to consider mentioning the names (or at least usernames) of the people who give you these ideas (the mirror), simply because people get all crazy and sue-happy over this kind of stuff. Aside from that, it's just nice to give credit to the people with the ideas...I'm sure they would appreciate it.
Just a thought from a non-techy person who enjoys seeing what you're experimenting with. :)
lindseykei1 1 year ago
@lindseykei1 Interesting point. Of course, proper credit needs to be given. One of the nice side effects about having the discussion here is that all suggestions (good and otherwise) are listed in the comment thread, with the user name of the person who made the suggestion, and a (relatively accurate) time stamp.
I hope that's OK for people; otherwise I don't really know how to handle it.
okreylos 1 year ago 2
All these people talking about alternating the two kinects or modulating them are talking bobbins.
Use a polarising filter for each kinect, and make sure that the two kinects have polarising filters rotated through 90 degrees. That way you could have two running independently and capturing data at once.
MisstraD 1 year ago
microsoft should hire him pay him loads of money give him everything he wants and let him do the software
wurschti2009 1 year ago
Maybe it would be possible to use multiple Kinetics by modulating each one at "different frequencies"... Well, it is not modulation per se, but you get the idea: You could turn-on/off each knetic by a few microseconds so their infrared rays won't interfere with each other... then you can reconstruct the full image aftterwards... It may cause some jitter, and it may require a firmware change as I don't know if it is possible to have control over the infrared from "outside".. but it looks doable
loudenvier 1 year ago
Hey what about polorised filter on cameras and filters ? Like used in 3d films now a days
karandex 1 year ago
This would probably escalate the complexity of the programming a bit, but.. wouldn't it be a possibility to achieve full 3D projection with multiple Kinects by making them record the frames sequentially? ie. if Kinect A records frame 1,3,5,etc and Kinect B records 2,4,6,etc, or at a larger span of frames and both "strobing" on and off. That would eliminate the IR interference problem and produce two separate captures from different angles at minimal time difference.
TokiiiSwe 1 year ago
Here is an idea for running multiple Kinects simultaneously which may require mods of the Kinect software.
Put multiple Kinects around the room. Have Kinect 1 capture a 3D image of the room. Then shut down/block Kinect 1and take a frame with Kinect 2. Repeat with any other Kinects. Use your software to merge all the images.
This solves the problem of having the infrared light interfere with each Kinect without major hardware mods.
P.S. Do you now have the right to patent your ideas?
dominicanpapi82 1 year ago
Comment removed
Lamoni 1 year ago
Comment removed
Lamoni 1 year ago
This has been flagged as spam show
In theory, couldn't you attach a GPS unit to each Kinect? That way you'd have a real world reference point for each Kinect, and could parse the data correctly based on those distances between the Kinects? I think you could definitely build a 3D model of a room and not have problems as long as the Kinects had reference points of where the other Kinects were. (Obviously, an alternative would be to just hardcode it assuming Kinects were all exactly 10 feet from each other)
Lamoni 1 year ago
Comment removed
Lamoni 1 year ago
@okreylos, surely you could use multiple Kinects using polarising filters? If the two kinects used a polarising filter over the IR array, and a polarising filter in the same orientation over the sensor?
You could rotate a second polarising filter through 90 degrees for kinect number 2, and they shouldn't interfere with each other.
Much cheaper than hacking the hardware!
Love your video and the live 3d. Incredible.
MisstraD 1 year ago 14
@MisstraD Good idea. You can find Polarizing filters in every lcd display. It's possible that you need extra IR-leds but they are cheap. Good project.
HvdHaghen 1 year ago
@HvdHaghen When the camera uses a lazer, the light must be already Polarizeed. Than one of the cameras must be turned 90 degrees. Good chance there is a Polarizing filter on the camera. Easy to find out when you turn a Polarizing filter in front of a working camera.
HvdHaghen 1 year ago
@MisstraD umm........ What?
shadowblade145 1 year ago
@MisstraD Whatever this guy said!!! lol. Sounds like a good idea.
StarCili 1 year ago
Can you just process alternate frames from the Kinect when building up your depth map and image?
If so, why not have the lights on two Kinects set up so that they're only on when the other Kinect's lights are off? Do this with 4 of them and- while you'd lose out on update rate- you could have an all-round view in 3D (at about 25fps, which isn't particularly bad).
uberdeity 1 year ago
another idea I had was an Xbox360 accessory(like Kinect) that was placed above the gamer (in the same position as the light fixture that more than likely was already there when the home was manufactured)It could be hard-wired there replacing(or along with)the light.This unit could project IR downward(a constant beam of UV light spinning like a fan blade).This light(along with the Kinect looking directly at the gamer)could increase the 3D/4D image of the gamer....just another thought,lol
theasskicker 1 year ago
PARABOLIC MIRROR PARABOLOID CONCAVE CONCENTRATOR, lol...that's what I was thinking! :)
theasskicker 1 year ago
Just spit-balling some ideas. Please do not stop your journey of improving the Kinect, you may just revolutionize the gaming industry. Good luck sir!!
theasskicker 1 year ago
Keep in mind that you could also use "flex mirrors", they are a plastic type of mirror. Maybe these can be used to concave/convex the image? They make them in all sizes (including wall size). Also a "tint" could be applied, similar to the tint on car windows, but you could use a UV filtered type of tint. I think if two Kinects are looking right at each other they will blind themselves (like looking directly into a flashlight), but maybe if the IR were two different colors they might worktogether
theasskicker 1 year ago
This is just an idea coming out of a mind that has no experience with infrared light but could you possibly have each connect using a slightly varied frequency of light, only picking up that frequency so that each connect does not interfere with each other. Maybe have a lens that you put over the projector to change the frequency, then another lens for the receiving camera to return tot he original frequency to avoid any major hardware modifications.
kieve11 1 year ago
This has been flagged as spam show
How about a Cylon Centurion eye scanner? Similar concept?
ipededotinfo 1 year ago
How about a Cylon Centurion eye scanner? Similar concept?
ipededotinfo 1 year ago
Awesome work man, I'm totally impressed! Can you put opposite polarizing filters on the two Kinects to separate their IR signals? It would cut the light down by 50%, but that might still be okay for short ranges…
pchapman905 1 year ago
Comment removed
Oh686 1 year ago
Wikes and Wilson built a body scanner using mirrors and fringe imaging, so my guess is, it can be done with mirrors, I also think it can be done with filters.
Rapido3d 1 year ago
Can polarisation filters be used on infrared light? In this way you could filter out the raster you don't want to see.
In the same way 3D movies in the cinema's work now..
Jobjansweijer 1 year ago
Comment removed
Jobjansweijer 1 year ago
Comment removed
Ebannmobile 1 year ago
Comment removed
Ebannmobile 1 year ago
Comment removed
Ebannmobile 1 year ago
Comment removed
Ebannmobile 1 year ago
search for "Robots get down and dirty" on google. Using kinect to map the environment might be a cheep way to $750000
fsteel2k 1 year ago
What if you add a filter to the infrared so it will not be exact?
Fibr3Optix 1 year ago
@Fibr3Optix I agree, it should be achievable through filtering.
Rapido3d 1 year ago
Based on testing by Jaques from openkinect team he did not see a great deal of interference when kinect was 6 to 7 feet and 120 degrees apart. I did test mirror and you can pick up reflected dots but maybe difficult to calculate if they were a reflection or not. The scan room and build 3D mesh models seems like a great solution and the wearable kinect idea or the robot kinect that scans room is a good low cost option. ROS team as working on this and I believe have 6 kinects hear
NINKNOONAN 1 year ago
How about one kinect and multiple video cameras which are placed at a specific depth and location. So you know where the images are in respective to the kinect. Also why not block the IR projector of the other kinects and then use the IR from the first kinect to see for the rest. Although this could cause issues with objects blocking the IR. But I think using multiple cameras with one kinect should work.
awasteofaustinstime 1 year ago
@awasteofaustinstime multiple cams with one kinect would help fill in some of the shadows but would not supply depth information for a perfect 3d environment. Multiple kinects using the same ir beam would give distorted depth info and since the processing is done onboard i don't see a simple software solution either. I think.
blawyer 1 year ago
You could connect an arbitrary number of capture devices and go through them round-robin so they don't interfere with each other.
It might also work fairly well to make a model of the space, a model of the subject, by moving around the capture device and then use the single device in a fixed location to animate the models and patching in live-data for things like facial expressions.
Something as simple as a polarizing filter on light source and camera might also work up to a certain number.
IOIOOIIOIO 1 year ago
Actually, if you use a mirror, shouldn't it think the mirror has depth, and make your program make your room longer? Mirrors are not the same from all angles.
Beeh0 1 year ago 2
@Beeh0 Think of a mirror not as a glass plate, but a portal into a virtual world where left is right (actually, where front is back). So (without having tried), a Kinect would see my reflection in a mirror simply as another Oliver who is sitting behind the mirror. Now here's the trick: if I know where the mirror is, I can flip that virtual Oliver back into real space, and can make it match the missing half of the real one. Tricky math and programming, but could work.
okreylos 1 year ago 6
@okreylos
I do have 1 idea for you when it comes to using 2 kinects. I noticed the frame rate isn't perfect - which is a good thing. If you were to put 2 kinects on opposite ends of the room, you could make them strobe infrared light asynchronously and try having them only process the frames in which the light is on. So in other words, in 1 frame, one of the kinects will flash an infrared strobe. In the 2nd frame, the 2nd kinect will flash its strobe. Then it reverts back to the first kinect
schmidtbag 1 year ago
@okreylos The only problem with that mirror stuff is that if it is painting an IR grid on everything the grid would not extend into the mirror in a regular way.
WTFNoNamesLeft2 1 year ago
@okreylos Are you sure? It wont get any depth perception from the mirror, its a flat object, the image is virtual. It would give the camera a different perspective, but the IR detects a flat surface. The programming would have to mix the origional 3D image with a 2D image. Im guessing upscaling 2D to 3D doesnt work out easily, hence the need for IR stuff.
lfo98 1 year ago
@lfo98 Yes, as long as the mirror is IR-transparent (not all glass is), the Kinect wouldn't be able to distinguish virtual objects in the mirror from real objects. Think about it. For a camera, or anything that's based on visible light, a mirror does not appear to be a flat object, but a "portal" into a virtual world.
okreylos 1 year ago
Comment removed
lfo98 1 year ago
@okreylos Wouldn't the mirror reflect the IR light back into the room and interfere with the capture of the room itself, or would that reflected light only reach the surfaces not visible from the Kinect's angle?
mmortal03 1 year ago
Comment removed
Pankaj2308 1 year ago
This has been flagged as spam show
@Beeh0 This won't be possible. Kinect projects a structured pattern and then uses the deformation of the pattern to learn the depth of every pixel. There will not be any distortion of this pattern on the mirror. Our brain can perceive depth through the mirror but unfortunately Kinect won't be able to do that.
Pankaj2308 1 year ago
Part of my comment was removed so I'm having to add it here,
As long as you ensure the kinects are atleast at rightangles to each other, they wont be able to see each other's projector IR field. (even with something like a sphere in the middle of them, they wouldn't be able to see each others IR dots).
JonnyCasey 1 year ago
@JonnyCasey I think you're forgetting something; someone else please check this. Even at more than 90 degrees apart, the two could still see each other's'points. Imagine the two projectors forming a V, and the tip of the V is the point where their lines of sight intersect. Now place a flat surface at that point, which is parallel to the baseline of the V, i.e., like this: V __ Then according to mirror laws, the two Kinects will project right into each other's cameras.
okreylos 1 year ago
@okreylos Ah very true. Having two facing each other would not have this problem am I right? (if you also incorporate the blind spot technique I mentioned bellow)
JonnyCasey 1 year ago
@okreylos Ah very true. Having two facing each other would not have this problem am I right? (if you also incorporate the blind spot technique I mentioned bellow)
This would only give a back face though, It wouldn't help with the shadows would it. :(
JonnyCasey 1 year ago
@JonnyCasey I believe that is correct (both parts).
okreylos 1 year ago
If the project IR field of the kinect is less than 90° then you should be able to possition 3 or even 5 kinects so they do not overlap.
However, You can simply add a tiny blind spot to each of the kinect's IR projectors for the opposite kinect to reside in so as to prevent them from blinding each other. Then intorpolate the Z depth from the surrounding Z values around the blind spot (much like the human eye works). Use bluetak to add a blind spot to the IR emitters.
JonnyCasey 1 year ago
OK, I'm a bonehead. We're discussing the ins and outs of using multiple Kinects for full 360 degrees capture, and I forgot that I have an old video showing our previous 3D video system. I want all of you to watch the "Immersive 3D Video" video I posted as a video response. It shows what full 360 degree capture can look like. Don't be put off by the quality; it's an old renderer using only points and no triangles.
okreylos 1 year ago
@okreylos You can simply add a tiny blind spot to each of the kinect's IR projectors for the opposite kinect to reside in so as to prevent them from blinding each other. Then intorpolate the Z depth from the surrounding Z values around the blind spot (much like the human eye works). Use bluetak to add a blind spot to the IR emitters. As long as the projectors are more than 90° from each other you wont have a problem with them seeing each other's IR projections.
JonnyCasey 1 year ago
For use 2 or more Kinects , need switch on only one in same time , other switch off. If you make right frequency you can take info from every cam.
MAXHARDMANmodder 1 year ago
How about having two kinects interlave?
Both would have half the framerate and would not interfere with each other's work.
michalhantl 1 year ago
Oooooh. Right. :-) Thanks for explaining. You rock.
kwags1 1 year ago
and to think.. I thought this thing sounded cool before you explained it in more detail. Good videos man.
sorphin 1 year ago
I should add: the reason I'm so impressed with the Kinect is that I already have a bunch of other 3D cameras in my lab, so I know their limitations very well. In comparison, this thing is awesome.
okreylos 1 year ago
What about using just another standard Web cam, to feed more informaiton into the 3D scene? It would be cheaper than a second kinect too. Maybe you could have the web cam at a KNOWN position to the kinect, or use a cube of known size to help with the calibration. Best of luck. I hope I am helping this project, just a little bit. You are the true genious, doing the hard work. OR, polorized lens on the IR emitters, and sensors. 1 kin sees through an up down filter, the other see left-right filter.
Shakespeare1612 1 year ago
@Shakespeare1612 You might not believe me now, but that is one of the first things I thought when I found out the Kinect uses active sensing. There are pure stereo-based 3D methods using multiple regular cameras (I have two clusters of those in my lab), but they have serious problems. Using the Kinect as a baseline should give the regular cameras a better starting point, increasing quality.
okreylos 1 year ago
@okreylos I believe you, but it is flattering to know that I think like you.
Shakespeare1612 1 year ago
If a second kinect throws out an interfering IR pattern, what about using just another standard Web cam, to feed more informaiton into the 3D scene? It would be cheaper than a second kinect too. Maybe you could have the web cam at a KNOWN position to the kinect, or use a cube of known size to help with the calibration. Best of luck. I hope I am helping this project, just a little bit. You are the true genious, doing the hard work.