Head Tracking using faceAPI & webcam only - no wiimote

[coasterpro]

It's astonishing to me, given the ubiquity of web cams on laptops and PCs, that we haven't seen the widespread adoption of face tracking to augment the 3D video game experience. Even if the technology is not used as a controller replacement (which I think is a bad idea anyway), natural unavoidable movements of the head, when tracked to affect the in-game camera, would evoke greater immersion on-the-cheap. No need for expensive glasses or 120Hz monitors. And it would do so quite convincingly.

[goatphilososphy]

so you dont have to the devil

[Free2101]

Johnny Lee's the man

[Chefkoch320]

THIS is going to be a big thing! Future starts here!

[Php48]

Does anybody remember the old arcade game asteroids, this teck would really make that game fun again, like you are the ship and you need to dodge the asteroids, if done right it could make for a very intense game. I can also see it as a handy way to increase desktop work space & combined with voice reco it could change the way we use computers, we could all become like dancing cobras lol, nar I actually have something more practical in mind like subtle off screen menus and zoom scroll.

[Vgeorge7]

guys i am new to this program and i want to know how to use this effect .. i saw lots of comments about using C++ or some programs like this and i dont even know what to do to to use this effect ... any tutorial / help would be greatly appreciated .. Pm me if you can

[steveski74]

This needs to be included in racing sims like Gran Turismo so you can shift your head to the side to look past the car in front of you like you can in the real thing

[Metzaltix]

its nice but only for one person

[Bangoustine]

this will be HUGE with 3D Vision !

[joshsora2000]

@Bangoustine Its 3D enough!

[Bangoustine]

@joshsora2000 no, this is 2D-3D effect, but 3D effect + real 3D can be heaven better !

[schoentje1997]

I am begging you! please put a download online of the jhonny-lee-3D with already the wiimote part changed to webcam part, so that people who aren´t handy with this kind of stuff (like me) can also see it.

[paine131]

Im haveing enough trouble as is with getting it too work with the wii mote. Could you just post the modified versions?

[Sdra81]

I downloaded faceAPI but I couldn't find any Desktop VR program in the package. I can not find Desktop VR on google. Can someone post the link to it please?

[firebomber39]

@Sdra81 johnnylee(X)netprojects/wii/

[Qkumber1]

@firebomber39 You need to wear stupid hat for that

[firebomber39]

@Qkumber1 my mom already made me wear the stupid hat!!!

[firebomber39]

@Sdra81 replace (X) with .net.

[firebomber39]

@Sdra81 replace (X) with .net

[Usulgurt]

I think Local Positioning System on lawn is better.

[ipellaers]

FaceAPI is an API, meaning it's only bits of code. These bits only track the head. It's in no way a product end users can install and just use ingame, it's for use for coders who for instance link the head movements to camera positions.

In short: You can NOT just install FaceAPI and use it in game.

[Raymint]

Keep up the fight, faceAPI. Stereoscope is an evil lie. That first attempt didn't work - try two webcams ...

[blenderpanzi]

Coooooool.

[blackcaptor2]

I have FaceAPI and I have the Desktop VR program.  Could anyone help me, you know, actually get them to work?

[I3L4NK]

@blackcaptor2

Agreed, I have both and would love to know how to get them to play nice

[ipellaers]

@blackcaptor2

Install Visual Studio if you haven't, load Johnny Lee's code, rip out the parts where it get's the head position from the WiiMote, and replace those parts with code getting head position from the camera via FaceAPI.

[DestructableMovie]

@ipellaers

Could you please give a short tutorial? I'm not familiar with this. It would help a lot!

[ipellaers]

@DestructableMovie

Can't really give you a short tutorial on it, you'd have to download Visual Studio 2008, find some basic C# tutorials, learn the basics of coding and C# and just have a go at it!

[blackrack2008]

@blackcaptor2 hey, if you found anything show me how you did it please

[metallsimon]

is there a description anywhere??

i have installed faceAPI.

i dont know how to use

[360kal]

i have instaled it but i dont know how to use it on my microsoft flight sim any idea anyone ? thx

[utkua]

it looks 3d because we see it on 2d screen, actually if you were there, illusion would be less convincing because brain uses 2 eyes to perceive 3d environment, flat screen is not enough for that.

[gevelegian]

@utkua Yes you would still need saaay 3d lcd shutter glasses or anaglyph glasses. And that will reduce framerate by 1/2.

[utkua]

yes and to be more precise each eye should be tracked individually, so it will remain convincing even if your head is not parallel to the ground.

[daedelusxxxhelios]

@utkua so close one of ur eyes :P

[xenoepist]

@utkua Now enter 3D glasses and 3D TVs

[littlelogic92]

single-player only.

[hafiz9977]

Anyone interested to create flash app that uses faceAPI data, drop me a msg. Search for "faceapi and flash" to view the demo video.

[hafiz9977]

Cool demo!

[rogervduarte]

Can I configure it as the right analog stick? For playing racing games in driver's perspective! =)

I guess it would be nicer than first person shooters, because using it on FPS may be a little confusing.

[6F5jedi]

can i rigg it to a game? like operation flashpoint 2? well my thought was to rigg it to make the character look around when i do! does it works?

[Ihaveapsp]

that looked almost 3D

[Virginityrocks]

Can this be done with a poor quality 1.3 megapixel or less camera?

[geogzm]

It tracks my eyes, nose and mouth, p. cool I guess. I want a room to move in though

[walter0bz]

I need this on my 6-screen Command-Centre :)

[mickeyshimitz]

WHERE IS THE TARGET SOFTWARE PLZZ ELP

[MrKennyBones]

I just downloaded this, but I can't find anything else than the webcam and head tracking-thingy. No 3d software provided so I can't really test the solution if it works well enough.. What's the point in advertising this software when all you get is some pointless headtracking points showing your own face! I want to try the real 3d software from this video!!

[loudornothin]

I think the were tlaking about jhonny lee and hsi wiimote vr thing. try downlaodign that and seeing hwo to connect it to that.

[MrMattWebb]

This is an API, meaning you have to be a programmer to use these tools.

[JahGuidanze]

theres a hl2 mod for that

[monkey192100]

PLEASE tell me how to use faceapi with the johnny lee program!!! PLEASE!!!!!!!!!

[CAngel222]

ya tell me two im stuck!!!!!!!!!!!!!!!!!

[warlockboyburns]

is this likely to work with existing shoot 'em ups one day?

[MrMattWebb]

Half Life 2 and all corresponding games on the engine supports it.

PC gaming ftw.

[NikoKun]

How? I can't find any download to enable such features.

I'd love to play hl2 or say, portal, with this!

[whatbrains]

Are they released as mods or do you have to script them together yourself?

[Mosessions]

Awsome... We need new '3d' video format allowing to create film/presentations build in 3d sceneries with masked (greenscreened) recordings put in world that despite regular camera movement will also render face movement corrections. Imagine that...

[Mosessions]

A good "combusting-like" program to montage 3d sceneries with recorded stuff and output it in standarized format. Face movements are somehow limited so even flat prerecorded scenes put in a 3d scenes would look extremly cool. Then a player with FaceAPI included to just playback and enjoy shows.

[Mosessions]

Maybe a blender addon based on blender game engine to put in recorded flat-keyed layers in 3d scene would do the work. Some efects in a player that will apply lighting to flat parts and maybe add light blur on edges.. Effect should be astonishing. It's time to make easy way of content preparation :)...

[Mosessions]

Few more notes. Camera handling in blender need little changes to suit face tracking while persisting possibility of flat motion layers. It'd probably need focus distance to determine influence of face movements on scene and maybe automaticly add some additional FX on flat layers distanced from focusing line. Also FaceAPI should influence position and zoom of camera frame not it's direction to keep flat scenes suitable.

[Mosessions]

Sorry.. Instead of blurring layers not focused it's better to render two seperate images and mix their visibilility so that they'll fit on focused distance... That would make illusion seem very real especially if given real eye distance values.

[Mosessions]

Oh.. to make it even more realistic - masked flat motion scenes can be projected not on flat, but spherical surfaces or i.e. on automatically generated 3d model from shape of mask depending on distance from borders. That however would considerably slow down rendering that should be able to react on face movement instantly.

[Mosessions]

Also trapezoid transformation on final motion should be applied considering position of eyes against center of the screen to reverse traditional trapezoid transformation of picture on the flat surface seen from different angle.

[moroho9]

How to install or start ???

[themenaceman1]

tutorial please

[EFFORTLESS7]

how do i do it

[BrianJensen]

that looks retarded, yet awesome at the same time...

[FIN358]

How did you manage to use desktop VR without wiimote?

[BitzFit]

Web Cam tracking... there is a Irrlicht cam lib using OpenCV if you would like to try it.

[SuperSZ]

this technology could help saving a lot of money. I mean, in spite of new TV's with 3D technology that will be expensive, they can put a webcam with this program (more developed of course) in the TV and it should work fine, isn't it?

[skyseeker901]

@SuperSZ

It should, unless you're watching TV with multiple people. TVs, unfortunately, are often used by multiple people. A better application for this technology is handheld mobile devices, which are increasingly equipped with cameras and are seldom meant to be seen by multiple people at once.

[yairbonastre]

amazing , but how to get the targets sofware..? I've got the faceapi sofware demo where I can see my face but i don't know how to see these targets. can anyone help me please?

[zaubertom]

got the same Problem :(

[MonsterbabyVashj]

Fake? I've seen it my self, tried it my self too. And it is a very simple program. Works for every computer, aslong as there is a webcam installed.

Some fellow students of mine at AAUE university made a project based on this system.

[PianozRULE]

I can't get it to show up what this video has :(

[MadHatter149]

Sooooooo Fake!!!

[CaptnCrotch]

idiot

[guywh0drums]

wow pretty cool!

[chad8641]

cool. where can i buy such a 3d monitor?

[phoolero]

That's not a 3d monitor. The webcam captures your head position, and recalculates all positions for the 3d elements on the screen. The new thing is the faceAPI headtracking system.

[goingwithzed]

actually they are 3d stickers you put onto the monitor.

[CaptnCrotch]

idiot

[MadHatter149]

I agree with Chad8641...cool 3d monitor but so fake!

[CaptnCrotch]

übertard

[skierplaterandy]

cool too bad there isnt a free version out :(.... Hope fr one soon :D id like this fr flight simulator because then i can see my panels better

[skierplaterandy]

cool too bad there isnt a free version out :(.... Hope fr one soon :D id like this fr flight simulator because then i can see my panels better

[eedvaarts]

there is a free version out, check the website.

[skierplaterandy]

there is now? cool.

[s0cks1985]

won't work in low light though. where as infra-red would.

[eedvaarts]

Just add an infra-red LED next to the camera.

[fashnek]

Unfortunately, I'm not so sure I'm impressed with the latency here.

[totallyonlyme]

Umm what latency dude? Looks pretty good in this video.

[fashnek]

I perhaps shouldn't have said "here" -- I was referring to latency of the general software and not specifically this video. I had watched a large handful of FaceAPI videos when I made this comment.

[21philip21]

nice...a great improvement on johnny lees idea

[XSportSeeker]

Great! I wonder if they'll open the code for it... as I see, it uses face recognition sofware to trace the position of viewer's head.

Some Logitech webcams already have the face tracking stuff.... but great stuff anyways!

Kinda reminds me of Augmented Reality...

[emgeko1983]

F A N T A S T I C !!!!

No Headset necessary, very cheap hardware needed (just a webcam) and greeeeaaaat effect!

This WILL change the world of 3D gaming!

[wildone106]

the future of 3d

[hubbardproductions]

interesting new take on Johnny Lee's stuff

[embeddedv]

This 3D looks so real

[eedvaarts]

far out!