Head Mounted Display Controls Humanoid Robot Head and Sees Through Its Eyes

Title says it all. This is the final component of my virtual reality control system for humanoid robotics.

The Vusix processes the head rotation by using a compass. So that means that if I rotate my body but not my head, the robot's head will move. So to compensate for this, I am using the Kinect to track the body's position (that's not so clear in the video).

Category:

Science & Technology

Tags:

• NAO
• HMD
• Vusix
• VR920
• virtual
• reality
• robot
• humanoid
• head
• mounted
• display

[kfuVids]

Hi Taylor, did you have any problem while integrating vr920 to -supposedly- ubuntu? I'm planning to buy one for a similar ros-based application, but according to their website they don't provide linux drivers.

Is there any API or something to retrieve 3DOF head orientation data that vr920 provides?

Thank..

[taylorveltrop]

@kfuVids You need to use an open source program called vrtrack. Search Google for vrtrack linux, it is the first result. The "driver" exists in user space. It's pretty good. I have a copy of vrtrack, and some example ROS code using it in my veltrop-ros-pkg repository. It is in the virtual_reality stack. There is also a handy script in there to launch it and to make sure the right network routing exists for the UDP multicast feature of the driver to work.

[taylorveltrop]

The framerate I saw was 5fps. My 2006 macbook couldn't handle the screen capture during this and so the framerate in the HMD view on this video appears at about 2.5 or worse.

Limited to 5fps to make sure that the motion commands are smooth (see the new cat brushing video) but I could have done much higher otherwise. Even 5fps is too conservative, but I am leaving room for the occasional drop in wifi bandwidth (microwave interference) and in the future I need to make room for streaming audio.

[notthere83]

i sure hope what you see in the display is mirrored, unlike the video here... otherwise it must be pretty annoying ;)

[taylorveltrop]

@notthere83 Actually the kinect view is what is mirrored. In the follow up video where I brush my cat I remembered to un-mirror the kinect video feed ;)

[ShadowZekromX]

If i could get it to work with the whole body, then i can put on a show, which will be pretty cool.

Possible?

[knospi]

Is the framerate of the video, streamed from the robot, really as slow as shown in the video?

[TheAmazel]

You can now turn yourself as a "T2 human". (no legs nor arms human) :)

[TheAmazel]

Oh yeah, that's so nice!

[smkrbr]

CooL!