Joystick Emulation using the DREAM Controller on the Microsoft Surface

Not satisfied with simply controlling 120 pound robots with a Microsoft Surface, one of the lead programmers from this video ( http://youtube.com/watch?v=48A8vdJ68lI and http://youtube.com/watch?v=HSOziHgQedA ) took a week off and hacked together an amazing on-screen joystick driver for Windows and the Microsoft Surface. Eric McCann's adaptation of the DREAM Controller is based on Mark Micire's PhD research http://www.cs.uml.edu/~mmicire and can provide all of the functionality of a physical joystick through multi-touch interaction. By simply placing five fingers on the table, the joystick is dynamically re-sized and oriented to the user's hands automatically on every hand placement. It can also be moved to any position of the screen by dragging the index and middle fingers. To demonstrate the functionality, this video shows the Microsoft Flight Simulator and Valve Corporation's Portal controlled exclusively through hand interaction on the tabletop.

Visit the Robotics Lab http://robotics.cs.uml.edu and be sure to subscribe to this channel http://www.youtube.com/user/umlrobotics . More fun to come... (Can anyone say steroids for Starcraft?) ;)

Category:

Science & Technology

Tags:

• microsoft
• surface
• game
• flight
• simulator
• valve
• portal
• DREAM
• controller
• joystick
• emulation
• hardware
• software
• multi-touch
• multitouch
• cool
• HCI
• human
• computer
• interaction
• airplanes
• computers
• aviation
• fps
• touchscreen
• touch
• screen

[Oldenheimer1913]

Yeah, FPS games would be a little iffy for something like this. But an RTS, hell, that'd be sweet with this touch screen thing. Just imagine that: StarCraft 2 (or whatever, I don't even have that game) with this giant touchscreen "controller." It'd be less like a game and more like you're actually sitting in some control room somewhere looking at a camera feed of the battlefield and controlling your forces with your bare hands. That'd be badass, right there.

Hell, even iPads could do it (sorta).

[UMLrobotics]

@Oldenheimer1913 See our latest video, please. :)

[WNxAtlas]

It looks extremley uncomfortable in the longrun.

[mmicire]

@TheRealBoxman Thanks for the supportive comments. This is all Eric McCann's genius and crazy hacking. It is just a proof of concept since everyone commented on our robot video that we should do it on a video game. I am disappointed (but not surprised) about all of the negative comments. It is no biggie. We are still having fun over here. ;)

You should google some of the 3M multitouch screens. We are evaluating some of the 22" displays that are around $1500 and can do the same thing.

[wintonwintonwinton]

lol, Apple is so far behind everything. Im sick of there shiny junk they make. They will be gone soon, especially because steve is dead and is like the only brains behind anything good that apple makes. Ms is the best.

[mileshoffman]

Dumb

[BLaCkEdOutSWIPZ]

Dear Santa..............

[reesealdrich]

the only time to touch a computer screen should be to block it from a child's view. we have gesture recognition technology, this is all just a hype, the DREAM controller is obsolete considering i could flip my hand and flip something on a computer given the right equipment

you know this is just a gimmick, Another desperate attempt for Microsoft to be relevant

[reesealdrich]

the only time to touch a computer screen should be to block it from a child's view. we have gesture recognition technology, this is all just a hype, the DREAM controller is obsolete considering i could flip my hand and flip something on a computer given the right equipment

you know this is just a gimmick, Another desperate attempt for Microsoft to be relevant

[TheChanclasVerdes]

mY FIngertips, they feel kinda toasty...

[sharp93]

any one think interactive porn.. or is it just me

[sygnus5]

Screw FPS games, I wanna see an RTS game use multitouch controls. Be like Minority Report, but with fewer double-crosses