Pseudo-3D Video Conferencing with a Generic Webcam (Chris Harrison, Scott Hudson) - ISM '08

[smartinp]

is it the same effect developed for Nintendo 3ds?

[TheBigThoughful]

@dragonnutds @artifactingreality why are so many of these comments so unimaginative. this is with normal web cams, if there was a dynamic camera that could move, or was ride angle, or had multiple cameras would allow it to appear if the person was actually in the room rather than just being a talking 2D picture.

[ONKC123]

Program PLEASE!

[pantaneto93]

hi, good job! where can u send me this paper with the perspective transformation matrix to do homography? i didnt get the paper that explains the transformation matrix points. thnks

[kudgcn43]

Virtual shake cam. Just what the world always wanted...

[dragonnutds]

cool, but unnecessary, i see no practical use for this, other than to show off to friends.

[artifactingreality]

that's quite smart.

Completely pointless, but smart.

[sloopidoopi]

Great Example!

I love the Simple approach!

[SevenSixTwoNato]

It's an excellent simulation for true 3D, and certainly great considering its simplicity.

5 stars!

[ardnat2]

This is insane! Really nice work! Any example code?

[selfsilent]

I'm not sure I see the point of this. the only person I know of who moves around incessantly is Ozzy Osbourne and I doubt I will ever video conference with him.

[DaVince21]

Nice technique, but I can't imagine anyone swaying his head around that much when video conferencing. Also, with just the one webcam you'll never get a "true" 3D effect, limiting the effect to a flat layer of the person on a flat layer of the background.

Having two equally spaced cameras could give a more true 3D effect because you can get real depth/offset data, but then again that would be custom/more hardware.

[botoxication]

sounds really complex

but really basic techniques

[Donaldo]

The "immersion" offered by telepresence (short of connecting wires to your nervous system) has, to me, always seemed a little silly. Who wants that much immersion? The homesick! Potential applications: Holding up a sentimental trinket to the web cam and rotating it. On the other end, a 3D model of the locket/stuff animal/toy/etc. can be saved and viewed any time or used in Second Life-type games.

[becomepostal]

Excellent demo !!! When can I get this software ???

[death2allah]

useless

[TheLaughingImp]

Perhaps they're hoping that by wiggling about like that, no one will notice how bad their hair is.

[SpoofApp]

Everyone talking about the black background and why it should be removed by stretching the background...this is a demo video and you need that black background to demonstrate how it follows your heads movement. In a commercial application it likely the background would be stretched.

[jimmit]

I don't think the black background can be avoided without adding more cameras. At some point there is going to be information on the sides and above and below that is unavailable to the program. This is simply because the webcam on the remote users computer is not capturing the whole room, only a square frame. You are simulating changing the viewers perspective without actually changing the camera's perspective.

[spazmaster1000]

neat idea :D

[petemayo]

Sweet tech, good implementation of different projects.

I would say to the other person "quit movin your damn head already, wiggly bastard"

[Evi1M4chine]

Parallax is not REAL 3D!!!

It only LOOKS 3D in the camera.

It's as much 3D as a 3D game.

Not as in 3D glasses!

[jci10]

Actually wrong, a 3D game has twice the amount of information to represent a meta-3D environment, this is total BS. Go figure, I should have known better thant to trust a slashdot article. This is clearly NOT 3D..Geeez...

[Donaldo]

"Real 3D" (or stereoscopy as it's usually called) could be applied to this technique very easily.

[dcdomain]

Someone already mentioned cropping the edges, I had the same notion. But how about somehow extending the image beyond the viewing area so the black edges won't show when panning?

The black edges around the picture are pretty distracting when you pan around. Wonder if they considered taking a larger photo of the background so that no edges would show when you pan...

[jimmit]

The way I understand it, the background is a semi-static image. Really, there shouldn't be two video streams. They are simply filling in the space behind the remote user when you move your head around with some data they've collected previously. I'd like to see what it would look like if there was a clock immediately behind the remote user.

[xenoglot]

I wouldn't set my watch to it.

[Zarchimedes]

To get the clock 'proper', you'd have to remove the hands from it while taking the background picture.

Furthermore, you'd have to set the movement detection to 'extremely sensitive' in order to catch the hands' slow movement. (Marking them manually would probably be a better solution.)

I think it would look rather funny having the hands move about in the foreground, floating around in a ghostly manner, like creatures who have escaped from Disney's Beauty and the Beast...

[jimmit]

The hands would not be floating around in the foreground. They would still be on the clock. The only problem is the left side of the clock may have been captured at a different time than the right side of the clock. When these two images are merged you will get something funky. Probably two second hands, maybe two minute hands, and one hour hand.

[n45w73]

This is fun to see : -) I should be interesting to see how difficult this technic can be extended to multiple layers of 3d and masks, just think to apply this example a mall scene, im pretty sure you can track and mask easily few moving people and resulting 3d could be fun.

Hummm are software pieces available somewhere ? (for a bored software home user) :- )

[perfectionbox]

It's too bad it defeats the frame-to-frame coherency exploited by video compressors to make efficient use of bandwidth. Static backgrounds reallly help lower bandwidth use.

[meatmanek]

I'm guessing the actual data going over the net will be the two video streams, because the local computer should be doing the image manipulation.

Alternatively, the computer with 2 webcams could do the analysis, and then use some new compression method (compress person and background independently?) to send the data over the network.

[ddanopia]

The background, being semi-static, could be sent as a JPG beforehand and then only the foreground with the black background could be streamed. You could also have it stream as it normally would, then only have any processing occur on the viewer's side. This way you get lower latency.

[Flashguy45]

Yes, and that would be find for desktops, but with a laptop, you would have to take a new picture every time you move it. I wouldn't want to move out of the way every time I want to chat.

[carterson2]

the Stevie Wonder affect.

[webhorn]

What would make that even slicker would be if the webcam could capture and build up more background space either over time or from the user moving the camera in the beginning. Then as the counter-party moved their head, they could look "around the corner" and the "thru the window" effect would be preserved even better, like the Wiimote experiments! Very neat!

[grinick]

This is a pretty neat trick.

Crop the edges!

[Stevedroid]

It's very cool and all, but what's the practical benefit? As a business owner, and not a bored techie, why would I buy software to do this?

[mmlenz]

who said it would cost you anything? software doesn't always cost. In fact many would argue it should always be free.

[denzacar]

What are the practical benefits of a iPod - for a business owner that is?

[SirLudicrus]

The future of videoconferencing: everyone kinda sways around all the time.

[worldfamousbigb]

Very neat! Can I download this software from your website?

[paristo13]

And now just crop the background so it is without black edges. very cool feature for webcams.