Moveable Projected Displays using Projector Based Tracking

[sickman19]

wow!! men, excelent research and job!!!!! greattt!!!

greattings from Chile!

[RdneckEntertainment1]

I want one

[lTLrCl2uZ]

this guy is so smart ..im looking at this video and i dont understand shit in what hes talking about Lmao ..imagine this nerd(jk) fucking w black ops ..Lmao...

[seanme123]

9 People are plain jealous xD

[BelviGER]

I want the thing at 4:34 as a iphone/ipad game!

[nathaaaniel]

low fps. but when was this again? 2007 oh shoot its 2011 now! xD

[MrMrUm]

@nathaaaniel actually this was in 2004

[cappie2000]

Laying off the Ramen Noodles and going for a walk more often would render this guy infinitely attractive and successful..

[TheFXGuy]

And think, this was in 2007. Think of what he could do with this now!

[grandia09]

now its only a matter of time before every house equipments use his method of tracking.

[bleachlover124the2nd]

if all his inventions would be put into a single game it would be very costly.....

[The12ize]

cinese JIMMY NEUTRAUN INTEWuin NIIIIIIIIIICCE

[D1CROM1]

I tuned out at 1:03.... lol

[abcdefg9519]

omfg ... try to think about how big this could be if you find a nice sponsor ...

i mean this is the future ... the thinks this guy is "creating" is kind of this things we see as a fake in every movie :OOO

[Rakasha62]

Baciu to jest zajebiste

[Hafiz9596]

i love uuuuu

[IPIxGh0sTxIGI]

I realy realy hope you patend all of your work.

[skatardude10]

imagine if by some biotechnology or nanotechnology we could implement the projector side of things into our perceptory organs and have the process of reorienting the display surface calculated by our own brains, with inputs say your smartphone. So you could in effect imagine any surface you would like real world or imaginary and using your focus calibrate your smartphones display or any programmable software experience into your perception and interaction with the world around you.

[HGClark]

you are smarter than anybody i know of

[SilverGunZoO]

dude you are a fucking genius, I'm a caveman compared to you

[TheXRealXBrapp]

whoa whoa whoa wait. Those 4 squares arent visible in person?

[karadan100]

I love this guy!

[forallpurposesonly]

the next iPad!

[Demonairing]

man i read the new scientist every week and i dont recall ever seeing your name. WHY!! i dont know because the vids you have posted top alot of other research going on and these vids of yours are 3 years old !!!

[bobbied3]

To everyone going "GHETTO IPAD": Check the date. 2007.

[creac]

@bobbied3 And read the description! It says that he made the project in 2004, 6 years before the Ipad was released.

[mrchickenwing98]

Plastic laptops...

[ottomanbob]

8 people missed the like button

[Conellwuffie]

thats really fucking cool O_O

[sd9800]

y do ppl keep talking about that dumbass ipad

[dragonste]

why arent u in development????

[vampireboy1997]

looks like i pad

[sk8rguy4life1123]

DUDE...just...DUDE that was awesome

[sk8t3rd0de12345]

THE NEW IPAD!

[Mancy4]

@sk8t3rd0de12345 or old :P

[TheXRealXBrapp]

So your saying, humans can't actually see those grey squares??

[Feathertops]

He made a Ghetto ass IPad :()

[charlee567890]

you should be famous =) i'm asian and when i see a smart one i know and its you you can do it =)!

[TopGear6666]

Will someone please pick up on this guys talents he is a bloody genius and you are missing out big time.

[vadersdemise93]

Wow start your own company or something! You can make millions with your inventions!

[megaemad]

apple ipad stole your idea

[thepianoaddict]

just like an i-pad but with a projector :P

[steven9red]

I'm going to have to watch that again

[whosoever]

Once we have better batteries and smaller powerful projectors, we'll be set. There has never been a better time to invest in an HD-3D Projector. Optoma makes a really sweet one that you can get for around $650. Johnny has the software on his sight and extra sensor bars and remotes cost maybe 40 for the pair. I'm sure that Natal is going to be great but why wouldn't we want an open source competition. Google where are you?

[SolarCreed]

Ok you were doing this 3 years ago yesterday, where are you now?? Chillin out with your millions or busy working?!!!lol

[holyburritoimmexican]

imagine if he combined ALL of his inventions into a video game.

[93davve93]

that would be awesome :D haha

[cprich22]

@holyburritoimmexican u would get rea life lol

[TheLunarDream]

О_о

[hog242]

u made arows on 2 screens do u think it wud be possible to like make them pics,vid or pc desctops

[krypted]

You area freaking genius!

[luizinergia]

Hi there! I really admire your great ideas! you should try some cool stuff like that interfacing with opensource arduino or processing!!!!

[xxRukUSxx]

ALMOST real time. I love it. Keep on going Johnny! I expect my hover car by the end of the week!

lol

[gigaves]

And IMO that's how ,ladies and gentlemen, Microsoft Surface will work

[PacoBell]

Huh? That's nothing like how the Surface works. Maybe for Natal, tho.

[mataikan]

you are a engineering HERO!

[VINCENOIRgothFAIRY]

wow. marry me mr lee!

[lewisvz]

dang that's soo cool i will try to make this to!

[TheRealHITMAN666]

not that its not cool, but what will this do to improve computer efficiency

[PacoBell]

It allows you to divorce the display interface from the computer, much like the days of "dumb" terminals. You can manipulate high-bandwidth data virtually without physical restriction.

[TheRealHITMAN666]

Cool Thanks!

[AtomicDoorknob]

I wonder if he had anything to do with natal since he works for microsoft and has all these crazy experiments he runs?

[Covenantkilla25]

you are very smart you should work for some kinda gaming company like Microsoft or Sony.

[legomaster101]

gaming company?

he already works for a software company

[legomaster101]

scratch that, he works for microsoft

[The28thplatypus]

I want to go to this guys house and play with all this stuff.

[NakedCreep]

I'm sure you do o.o

[thecakeisnotalie95]

Youare a fucking genious.

[mtrue12]

???me no understand

[fgzf]

THIS IS THE FUTURE! i can see it that in the future there will be lots of stuff like this and holograms based on this technology too :D OMG!

[ctoon6]

this will soon be practicle when processors are faster and the projectors used can react faster

[Droqen]

And if you increase the speed of light it'll be even faster :D

[assalant777]

their is soooo many applications for this

[Klintoniusmaximus]

What if ALL sensors are lost?

[NikAmi]

If all sensors are lost then most likely there is no projection surface available so the question is moot. Now, if all sensors fail then obviously you would lose tracking.

[PacoBell]

And recalibration would be nearly instantaneous using his method.

[ChrisSketch]

I bet in a few years we can shrink down mini projectors to the size of those wireless headset cell phone thingies (sorry for that overly technical term).

If your projector is mounted on your head, you don't really have to worry about the display leaving the projection area. Or maybe instead of a projector, use clear display glasses and go all the way with augmented reality.

[PacoBell]

Yeah, projectors have their place, but mobile displays are ideal.

[SigmaStudios12]

thats hot

[chocolatechocochoco]

you are amazing..

great work

[mike081283]

i wonder if u can use the wii remote to make motion capturing posable

[PacoBell]

Well, it can already track multiple light sources (i.e. the wiimote bar), so yeah, it can.

[Procrastinatathor]

It quickly reminds me of the techniques they use in video game and CGI for motion picture, where some actors are covered with white dots, whose position and motion are then picked up by sensors and computed. Using this technique, game manufacturers for example make their 3D characters do real movements because the mouvements are created from real actors before, making them more realistic.

Such mouvement made enterly by hand would have been choppy, inexact or hard to make.

[ihateyourface]

This would have been great for commanders in the second world war lol. No need for models of planes, ships etc.

[madzy006]

Bring on WW3!! Jokes

[xXbrokenskaterXx]

It's essentially a moveable SmartBoard! Think how that could be used in schools!

You have my sub, as you are making my SciFi dreams come true.

[Procrastinatathor]

You ought to see his other videos, in fact go check out his presentation at TED! (here on utube) it's worth it, full of ideas and ... what?! his technology is already implemented in a EA game as of NOW?

:)

[adood101]

I want some source, johnny! :D

Great work.

[mavenGamer]

you defently got my subscribion

[diyamez]

Microsoft is now using this technology. Thanks to Johnny Lee who works there.

[chrisjarram]

lol, are you for real? So you think the angle the individual is looking from makes no difference?

[xXbrokenskaterXx]

No difference for what? And besides, it makes as much difference as anything else does.

[chrisjarram]

My statement is so obvious I'm not even going to dignify that with a response!! lol read it again my friend, and think about it for more than 2 seconds.

[xXbrokenskaterXx]

What, you mean making things invisible? Well, thank you captain obvious. Have a cookie. If you're trying to make something invisible or transparent from more than one angle, this is not the method to use. This video is highlghting use of this method for touchscreens that can change shape, size, etc. Not for making you invisible. There are better methods for that.

[chrisjarram]

There is this other great invention which makes this so much easier - its called a touch screen.

[ipod4]

omg dont be a smart azz this could end up being way better then just a touch screen -_-

[chrisjarram]

How? just curious...

[SunnyKattZekeGames]

This is for low cost, buddy. Way less money involved here.

[chrisjarram]

Way less money that what sorry?

[SunnyKattZekeGames]

An actual touch screen.

[chrisjarram]

Ahem, thats what I thought you meant. Because a high definition projector with tracking hardware is so much cheaper I suppose! ;)

Touch screens are EVERYWHERE now my friend, from the free phones given away by all service providers to large panels cheaply available for under $100. Where have you

been? With a real touchscreen you get infinitely better image definition for a cheaper price,and portability- I see no practical application for this whatsoever. Maybe Johnny could add his thoughts?

[LordVedli]

This is for moveable interactive displays for images displayed via projectors. The advantage this system has over a touch screen is that the touch screen has a fixed size and it can not be changed. This system allows the user to instantly resize the operating area of the screen without the need for numerous size touch screens while still retaining all the functionally of a touch screen.

[LordVedli]

In addition this is also much more portable than a large wall mounted touch screen that has to be fixed to the wall with screws. With this system a simple white board or large wall can be turned into a large touchscreen at no cost.

This was meant as a reply to Chrisjarram goddamn youtube.

[mrhonk123]

lol i thought on the tumbnail it was world of goo

but its johnny lee i like him

[warlord90210]

Very slow frame rate...

[maa105]

the technology is still young I bet it'll get better

[ScootechWayno]

Why hasn't someone given this guy a huge grant to build some of his ideas !!!

Please if you are RICH and watching this you must invest in him !!

[suckingcat11]

unfortunately that was waaayyy over my head... still badass

[fishfromchina]

really clever!

[Ticoguim]

hey man if you need a medical(with average-advanced technology skills) partner, count on me!!! I would love to work in a medical project with you!

You are the man, you are seeing (and doing)way far in the future than everybodyelse and thats awesome!!!!

[chrisjarram]

If you need a medic? lol - yeah, and if you need a flower picker who knows how to wire a plug 'I'm your man'. I would love to work on a gardening project with you.

[PacoBell]

LOL? I can see how he could use Johnny's expertise for medical imaging. Not so sure about your application, however. Landscape visualizations?

[BitzFit]

WOW! That was cool. Amazing.

[ParaHelix]

As far as I am aware the program which allows you to do this is built on the Wii-mote code library, so my question is does anyone know how to do this with a regular camera and perhaps a IR filter lens, thanks.

[NiallOCoinleain]

I just watched all your videos, you seriously are one of the most amazing human beings ever

[cskproduction]

holy crap ur smart ur like the next einstein!!!

but better hair...

[warlord90210]

Not really...

He's just using projection in many ways...

[Xreaper2070]

Johnny, I... I love you!!!

[Xreaper2070]

Johnny, I... I love you!

[Eljay8959]

you think you could lower the movement buffer for a updated vers. your awsome man i subscribed u

[Gershom123]

U rock man!!! u must have like a billion degrees :P

[Loki12451245]

your a f***king genius man!

[Codercise]

If you had 4 projectors you might be able to cover any location in a room. Also, fish-eye projection or another type of lens which had widescreen/telephoto aimed by teleoperated movable platforms would increase resolution at targeting areas. You could use an array of 40 projectors to create this in a space. As they cheapen and get smaller, this will become within our grasp.

[siipan]

Then there is a problem. How do you handle 4 or 40 projectors with only one system? It would be pretty hard to syncronize a couple of computers/systems to make them work in harmony.

(Excuse me for my bad English. I am from the country of Sweden).

[PacoBell]

Lots of parallel processing GPUs OR dedicated FFT ASICs.

[blah11blah]

So... I sure hope someone has hired him... this guys is epic

[Codercise]

What kind of optical sensors do you use? How do you wire it into the USB?

[histaf1]

genius

[dadoso12]

I hope noone military see thiz video,

if they have thez technology , for example map and the arrows for target and teammates,

like a frist person shooter...

I'm not jocking, plz dont sell this technology to any army...

[1individeo]

It is quite obvious that if it is on the net the army already have this, probably way before youtube existed itself. By the way a similar aplication was used in Justice's video "DANCE"

[themoose8]

You should use the pucks seen in the last frame to make some sort of virtual air hockey game!

[hunter500206]

so basic. a tracker and a projector that only show what need to be track.

[iamadonkey11]

this is amazing!!!!.... i wonder when we will get them in homes....

[Muvlonion]

can you use multiple prjectors for this to reach surfaces that are hidden to one projector?

[Cody1561561561]

Most likely

[nerdyhedgehog]

y must i copy and paste the word

"this" on ten vids?

how will that effect my moms lifespan?

[Mifiglio47]

JCL, would you bee so kind as to reccomend some resources dealing with Kalman filters being applied to computer vision/ dynamic servoing?

[samuelmichaud]

this is in 2004? holy shit, i thought this was still an embryo

[xfrancesco10x]

you are a genius man...

[samuelmichaud]

imagine the art you could do....two ppl dancing with these things on and a projector above them projecting cyclones of colored pixels around each one of them, then one big fucked up cyclone when they're close to one another, fighting or kissing, thats cool shit!

[xfrancesco10x]

XD lol

[ooSTKoo]

OM Gthis guy always blows me away

[dynpan]

It's amazing to see this project evolve with just the simple addition of a Wiimote :D

Every idea they had is now a hundred times easier due to it, and I think it's a very constructive way of thinking, I hope something very big comes out of this.

[fashnek]

You're completely mistaken. The Wiimote, which is an IR-sensitive camera that tracks 0-4 light sources, is a FAR stretch from what's going on here. This uses multiple sensitive sensors and rapid patterns to locate points. The projector provides a stimulus for location AND display, whereas with a Wiimote method, a display would be completely separate from the [Wiimote] camera and the [IR] light source(s).

[fashnek]

As I was saying, this method can be a LOT more precise, since different locating patterns can be used for each vertex, and precision is mostly based on projector resolution, which can be very high. You get to move sensors around, each being able to be precisely and independently located. With a Wiimote, on the other hand, you're stuck moving light sources, which (a) must be the (1-4) brightest IR sources in view; and (b) cannot be independently tracked without major assumptions by the software.

[PokeProd]

OMG, this is so great, i really love it, this is one of the best things in the digital field i ever saw.

[boy44412]

you should do the street map as the big one, and the satellite view as the small one. that would be unreal!

[emtep]

Oh dang

this will come so handle within the next 10 year so this could very well be the key to holograms

I see oh wow

[staticdisney]

o you guys should see his nintendo wii one on his website johnnylee[.net]

[metabog]

He has, watch the newest videos, it's real time now.

[metabog]

Sorry, the answer was meant for shadowslicer0.

[shadowslicer0]

can you increase the framrate on it?

[Nuuqis]

Thats like something from a movie where the newspapers images move. Truly amazing.

[zeldarooles]

This world needs more people like you.

[04mustangsvtcobra]

now i am just so very confused!! how is it being tracked when there is one projector at a fixed point and he is angling it and stuff??? please someone explain to me at 15yo talk!

[nitrox255]

it is distorting the imgae.

[04mustangsvtcobra]

yeah but how is it that the image is moving with the sensors on the object when the projector is in a fixed point, i dont get that. like if you move it 5 inches to the left, how does the projector know to move the image that way, theres no tilt or robot mechanism that moves the projector to follow the object? thats what im confused about. if he could make a vid on how thats done, and then watch this i would know how it works.

[Trogdor88]

It isn't that he's moving the projector, it's that he's telling the projector to only project inside of a certain square, or to somehow change the projection inside of that sqaure. Imagine it like the projector is projecting your monitor, but instead of your normal computer, it's the Windows screensaver that bounces a picture up and down. Normally it just bounces, but what his software does is tell the computer exactly where to project depending on what the sensors tell him.

[Trogdor88]

The projector knows where inside of its own projection it should show the object by the little gray "test patterns" that are on each sensor. They're displaying a sequence of patterns that tell the little sensor the EXACT pixel location of itself inside the projection, and allowing the projector to update it (probably using a technique called a "binary search" or some similar technique, but don't worry about that). As for how he tilts the screen, think about the optical illusions where you have

[Trogdor88]

a word that's been really, really stretched, so it just looks like lines when looking at it straight on. When you look at these puzzles from a very high angle, though, they seem to be clear as day. This is the same principle behind what the projector is projecting.

[Trogdor88]

If it were to project that image flat onto the table, it would look very, very distorted. Imagine it like the sun shinning on you when it's about to set, when you've got a very, very long shadow. Now, instead of it being that the sun puts out light, imagine that it was projecting a picture of you onto your body instead of the ground behind you (where your shadow is). If you were to move, the picture of you would distort to the exact same dimensions as your shadow, and would look really weird.

[04mustangsvtcobra]

okay now i get it, thanks for explaining it using real life examples, like with the sun and shadows, and the stretched lettering. that helped me a lot!

[hehehe6022]

Makes me want to go to Caregie Mellon. They'd probably kick me out :~(..... keep up the good work

[Intosia]

dude u rock!

[Santiagosony]

excellent work, nice1!!!!!

[Novatorm2k3]

Really cool! :D

But its kinda lagged :|

Gratz on your work

[Santiagosony]

agreed, but it is very cool.

[monkeysrock4ever179]

how did you build the sensor thing?

[macman13]

I WANT ONE!