Kinect controlled animatronic cyborg zombie

I can now control Mango with my body by using an Xbox Kinect and a Wiimote! He can now be controlled in real time (apart from a full second delay), to the point where I can do improv and interact with a live audience through him. Btw, the reason he's stumbling over the lines is because that's actually my voice being morphed real time through a microphone I'm wearing and I'm not a great actor.

I sincerely apologize for the miserable quality of this video. The lighting, the echo of the hall, the sound quality, there were half a dozen factors I couldn't phase out when filming this.

Anyway, the new system (which takes about four programs running simultaneously to work) allows me to control Mango's arm movement with my own. The Kinect is able to identify a human body and generate a virtual skeleton. I use the joint coordinates of that skeleton to calculate servo positions, which are fed into Mango so he mimics my every move (some servos need to be calibrated a bit more, working on that).
I modified the free OpenNI Kinect software to do this.

I use a Wiimote and Nunchuck to control his other movements: Jaw, head, torso pivot, wrists, fingers, claw, and lens color. This was done by using GlovePie to write button output data to a file, which Mango's master program references.

Off camera, I'm wearing a headset and running my voice through MorphVOX Pro to create Mango's voice. (Unlike GlovePie and OpenNI, this software was not free)

The end result is that by viewing the audience through the webcam built into Mango's right eye socket, I can position myself in an adjacent room hidden from an audience, and interact live with that audience through Mango.

Next time he is the greeter for a haunted house, he'll be able to chat with the people in line. (after fixing some minor calibration and sound quality problems)

Category:

Science & Technology

Tags:

• kinect
• animatronic
• cyborg
• zombie
• wii
• wiimote
• Mango
• hack
• robot

[LoadingCue]

That's okay, I didn't want to sleep tonight anyway.

[SeekerPsycho1]

@LoadingCue We take note of your concern and assure you that there is nothing to be afraid of. Our systems are equipped with numberous failsafes to prevent any of our machines from harming us. Your fears are unfounded.

-Your friends at Cyberdine Industries

[05Matz]

Truly awesome! The combination of motion capture and pre-programmed sequences makes it better than what I've seen so far. I don't go to haunted houses, but Mango almost makes it seem worth it!

Nice job with the mad scientist/snarky sidekick act too! You are a really good actor indeed! The hidden Nerf armament would make even annoying Mango funny--though I would stay far away from the scary left arm if I did!

The lag might make the gun inaccurate... I wonder if you can miniaturise a Nerf Vulcan.

[SeekerPsycho1]

@05Matz Thanks! I'm convinced that this system will give me the best of both worlds: the live control giving him a much more realistic feel and audience interaction, and the automated side allowing me to repeat sequences without wearing myself out.

I wouldn't be able to fire nerf darts at a paying audience, but I have been considering mounting a concealed air cannon. You probably haven't been hit by one, but from three feet away, those will knock you on your butt if you're not paying attention.

[catatonicprime]

What size servo's are you using in each part of the body?

[SeekerPsycho1]

@catatonicprime Since this is a frequently asked question, I have added a more in depth materials list in the comments of my Mango Halloween Testing video. Feel free to send me a message with any more spesific questions.

[013dirtdiver]

nice..you can see the evolution - Rc control, next Telemetry suit ,next Kinect control...maybe next is artificial intelligence..

[calcasla]

GUAU!

[fullagon]

i like it 30 thumbs up

[audiosport2]

this is the prototype of real steel ^_^

[GodMasterRuler]

OMFG The Birthday of Skynet! We're all doomed now! XD

Just J/K! :p

its a great job man!

Ur project can help paraplegics/tetraplegics in medicine also.

[GodMasterRuler]

OMFG The Bithday of Skynet! We're all doomed now! XD

Just J/K! :p

its a great job man!

Ur project can help paraplegics/tetraplegics in medicine also.

[05Matz]

@SeekerPsycho1 Air cannon? Meaning just a pulse that's that powerful with no projectile? Sounds good. Gesture just before and make his eyes flash and its just like magic!