Markerless Human Motion Capture and Motor Action Recognition

Details in: http://luis-unzueta.zxq.net/
This video shows the results of the PhD thesis project of title "Markerless Full-Body Human Motion Capture and Combined Motor Action Recognition for Human-Computer Interaction" developed by Luis Unzueta at CEIT ( http://www.ceit.es/mechanics/index.htm ).

Category:

Science & Technology

Tags:

• computer
• vision
• tracking
• hci
• human
• 3D
• real-time
• markerless
• motion
• capture
• gesture
• interaction
• action
• recognition

[lilOzyH]

Hi,

How do you do the tracking process? I'm currently doing a similar project (much simpler, though) for upper body tracking. I'm currently using a mix of optical flow, border detections and a simple particle filter, but the results are not that good (yet) :/

[lunzueta78]

@lilOzyH Hi. The tracking of hands, face and feet is done by combining the LK optical flow of feature points, color probabilities for each of those features and spatial probabilities with respect to the median. A Kalman filter is also used for the "good" feature points selection (see the PhD doc for details). I also tried also other alternatives such as (a quite simple) particle filtering but the motion wasn't that smooth at all and had tracking problems between close end-effectors.

[NCJaye]

A quick question, does it work if the user facing the camera? how about swing the hand to the back?

[lunzueta78]

@NCJaye Hi again NCJaye. This video is just a demo of the approach proposed in the PhD for human full-body tracking and combined actions recognition, don't look at it as a product itself. You can change the viewpoints, including facing cameras (things would change significantly in this case), but the application should take into account what movements are feasible for each viewpoint. If you swing the hand to the back it is an occlusion. You can read in the PhD how these situations are handled.

[NCJaye]

May i know what type of camera you are using ? TOF? or Stereo Camera? Thank you.

[lunzueta78]

@NCJaye Hi. It uses a single "normal" camera at a resolution of 320x240 (concretely a PointGrey DragonFly2 camera, but the system would also work with a USB webcam). The depth variation of body limbs is solved through a mapping with respect to body limb pose databases (see the details in my PhD document that can be downloaded from my personal webpage)

[OrganicMotion]

This is interesting. You should check what Organic Motion is doing with 14 cameras.

[BlackBeanPie]

very poor

[NCJaye]

@lunzueta78 Thank you :) and it is very interesting and impressive tat you are using 1 normal camera. I am trying to do something like this as well but still on the stage of research...