Gesture Recognition using Depth Sensors (PrimeSense PS1080, Kinect, Xtion)

The Kinetic Space provides a tool which allows everybody to record and recognize customized gestures using depth images as provided by PrimeSense PS1080, the Kinect or the Xtion sensors. The software observes and comprehends the user interaction by processing the skeleton of the user. The unique analysis routines allow to not only detect simple gestures such as pushing, clicking, forming a circle or waving, but also to recognize more complicated gestures as, for instance, used in dance performances or sign language.

Three highlights of the software are that the gestures:

+ can be easily trained: the user can simply train the system by recording the movement/gesture to be detected without having to write a single line of code

+ are person independent: the system can be trained by one person and used by others

+ are speed independent: the system is able to recognize the gesture also if it is performed faster or slower compared to the training and is able to provide this information

The software has already been used by media artists, dancers and alike to connect and to control a wide range of third party applications/software such as Max/MSP, Pure Data, VVVV, Resolume, etc. via the OSC protocol. The software is written in Processing and based on SimpleOpenNI, OpenNI and NITE.

More information about the project and the source code can be found at http://kineticspace.googlecode.com.

Category:

Science & Technology

Tags:

• kinect
• hacks
• gesture
• recognition
• openni
• simple-openni
• osc
• tracking

[scottjanousek]

So if one wanted to record a gesture for 5 seconds, they would set the frames to 125 and then hit 0-9 to record the gesture? or, perform the gesture and then hit the 0-9? Not very clear in the PDF, but I will look more closely at the code. A video containing how to use the system might be helpful. I can make one and blog post about it if you wanted.

[TheKineticSpace]

@scottjanousek: Yes, 125 frames correspond to 5 seconds.You first have to perform the gesture and then hit the key. I want to add a simple editor in a new version to make it more convenient.

[scottjanousek]

Is this gesture pose or gesturial motion recognition?

[TheKineticSpace]

@scottjanousek: It is motion recognition (how could I otherwise detect the speed of the movement?). The images at the bottom are just a (non-animated) represent of the gesture.

[scottjanousek]

By default 25 frames are captured which equates to one second, correct?

Also, I noticed that in the current version botharms is actually "bothams" in the xml config.

[scottjanousek]

@TheKineticSpace Ah. I looked more closely at the video. A countdown clock before capture of the gesture would be a useful feature, plus the playback of the gesture in the bottom.

[glaucomardano]

Really nice, thanks. I'll use that on my project ;).

[osterstone04]

Looks promising ! Thnx for sharing, wil test it soon.