Kinect Gesture Recognition REALTIME! (now with source code)

Using a Kinect and Stanford motion tracking software, we are streaming motion capture data in realtime to the gesture recognizer.

Gestures currently supported: punch, kick (high kick and low kick), wave, clap, jump, throw (different from punch), and two flicks for navigation (flick left and flick right).

I have uploaded the source code at https://github.com/fyhuang/kinectgest

Category:

People & Blogs

Tags:

• kinect
• microsoft
• gesture
• recognize
• ai
• cs228
• motion
• tracking
• software

[kissmaniacFR]

HI, i have download your source code.

Where can i found gesture log file ?

I try to open your file "track_clap_00.log" with LogFileVisualizer but i have an exception during file reading, look likes is in bad format. Can you help me ?

Thx

[fyfhuang]

@kissmaniacFR what exception did you get?

[kissmaniac31]

@fyfhuang simple parsing error on gestule..log ( i am under windows) i replace all "." by "," and i am able to continue.

Next issue: where can i find "gestures/frames/ns_00.log" those file for train operation ?

What is the standard motion software ?

Do u plan to wrap ur app to kinect sdk or open ui ?

[fyfhuang]

@kissmaniac31 I have no plans to port this particular software to kinect SDK, but I am working on a newer version that uses kinect SDK. frames/ns_00.log is a "neutral pose", which you can extract yourself out of some of the gesture files. I think it should be relatively easy to convert some SDK-recorded gestures to this format (with some interpolation, etc.) if you are up to it.

[sibabi]

@fyfhuang can you provide or explain what format the ns_??.log files should be saved? Thanks :)



[fyfhuang]

@sibabi They are in the same format as the log files: I believe that each line in each ns_... file represents one training example of a neutral stance. I would suggest, though, that you think about implementing your own segmenter: the current one that uses neutral frames is not very robust...

[sibabi]

@fyfhuang I got it working, it's cool, the segmentation definitely is the weakness, providing clean models is very critical and testing it on different people is not a good idea (specially on kids since they are not as tall) I would like to spend some time and make the training live so on a series of low confidence detected gestures the average would become another model in the dictionary ! it would get so messy after a while but it would be cool to give it a try ;)

[sibabi]

btw, I'm planning on working on this, I would love to commit the changes I make back to GitHub if you guys like to !

[sibabi]

@fyfhuang Thanks, I got it working on RunRealtime , I can see the joints but not able to recognize the gestures I recorded ! what do you suggest?

[mayanktaneja05]

awesome is the word!! :)

[kissmaniacFR]

@fyfhuang HI , i found the reason.

simple error during parsing file track_clap_00.log. I replace '.' by ',' in file and all is "ok".

I can't find any information abour your Stanford motion tracking software

available ?

Do you plan to do same with kinect micro sdk ? openui ?