Emgu CV Hand Gestures Recognition Advanced Tutorial

[KingcoolSam]

Thank you!

[karsun6218]

is emgu() function implementable in c++..???

[vapemylungs]

Thank you for this - it really helped me out. One thing though - its not really gesture recognition, because you can't train it to do specific features other than counting fingers. Have you had any luck with Emgu's ML dll classes or Accord's Vision dll?

[badrepent]

@vapemylungs

Thanks a lot for your valuable suggestions. Recognition in this video consist in recognize the number of fingers with a simple techiniques. ML techniques are certainly more powerful, my idea is only a proof of concept. Both emgu and expecally Accord has got ML classes that can help you in your project...

[TheArthurCarioca]

really great video!! may give me a little explanation about how works the face recognition? THX DUDE!!!

[RyanfaeScotland]

Thanks for this video and for sharing your code.

[KMKT89]

Hello again badrepent, I know that you need to find the defects to get the palm position, I now have the defects, but what do I do next? Can you please briefly tell me on what to do with the defects to get the center of the palm position?

[badrepent]

@KMKT89

You can compute center of mass to estimate hand's center using extracted hand contour....

[KMKT89]

@badrepent

But that is assuming that only the hand is detected and not the whole arm. I'm trying to make it so

that using the shallowest depth point found, which is at the wrist, i can derive the palm's position.

Any idea?

[badrepent]

@KMKT89

If you are using my code, it's based on skin detection so it extracts both hand and arm, you should

do some clever convexity processing to define hand location and it seems that you are doing it.

You should experiment with some custom ideas I have no reference for your scenario right now.

Good luck!!!

[KMKT89]

@badrepent

I see. Thank you for your help and reply, I'll keep going on.

[KMKT89]

Sorry to bother you so soon again, but I'm trying to detect the wrist position of my hand so that I can filter the arm away, and set a ROI around only the hand. Can you give me some pointers on how to do so?

[KMKT89]

hello badrepent, I am using C++ to code this and was wondering, you initialized the upper and lower boundary of the YcbCr colour range via YCC(). I know I can do the same using Cvscalar. But what exactly do I put in as value. It takes up to 4 values.

[badrepent]

@KMKT89

typedef struct CvScalar { double val[4]; } CvScalar;

The pixel values for each channel are in val[i]. For grayscale images, val[0] contains pixel brightness. The other three values are set to 0. For a three-channel, YCC image, Y=val[0], Cb=val[1], and Cr=val[2].

Hope this helps,

Luca

[KMKT89]

@badrepent

Absolutely, got it working, I didn't know that cvscalar is actually context sensitive, values representing different things depending on the image type.

Thank you so very much

[RyanfaeScotland]

@KMKT89 Hey KMKT89, would you mind pointing me in the direction of translating this project to C++. Do I need to re-write all the .cs files or can I use some of the .dlls as libraries and just write my own main file that calls on their functions?

[situkangsayur]

yes I already know about the existing highgui in opencv ...

I am confused is the use of the class on some object that is made like hsv_min, hsv_max, YCrCb, etc. ...

and also a bit difficult to translate extractContour method, and drawAndComputeFingerSum ... I still have not managed to translate it into C/C++ ..

before my very grateful for the explanation in the video ... very helpful :D

[situkangsayur]

i've tried use C/C++ with opencv to made a same hand gesture recognition program with yours, but I have problems with the existing objects in emgu ...

can you help me to translate your code to C/C++ language with opencv please?? especially for partial class Form1 :D

[badrepent]

@situkangsayur

If you use C/C++ opencv code, you should use highgui functions and windows as you don't have EmguCV display functionalities.

[kevincobain2000]

@badrepent

Thanks for the explanition and a nice app. and more over answering to people's question as well

A noob question. Instead of a video can it take a series of image (jpg) files as well? As at the momment i am converting the image files to a movie mpg file.

If its not a long code can please post it

[kevincobain2000]

@badrepent Thankyou for this nice app first of all and more over a nice explainition to it. How can I write the output of the fingernum to a file instead of displaying on the video screen???

Can please help me with that? if its a one~two liner code can please post it. Thanks in advance

[Nazirnaz123]

@badrepent ,Hello... badrepent i tried your code but it gives me an error, i was trying to make video streaming, i mean first my webcam starts streaming and then your code gets execute on my streaming video.. but all the time it gives me error, plz tell how can i use my webcam i place of Video File

[badrepent]

@Nazirnaz123

My code works with webcam, check that you are able to acquire webcam frames using

emgucv. You can try to run one of the samples that comes with Emgu.

Hope this helps,

Luca

[Nazirnaz123]

@badrepent I had made an camera capture program in c#. thats work f9. but in my case when i join that cameracapture code with your code, for video streaming , it gives me an error, "AccessViolationException Was unhandled" and it says.( Attempted to read or write protected memory. This is often an indication that other memory is corrupt ) :S. i would appreciate for your help .

[badrepent]

@Nazirnaz123

Is your camera capture program in C#, done using emgu?

If so post your question on emgu forum and the others and I will

reply to your errors...

[gebspsg]

thanks a lot for the great video. I am beginner to openCV and emguCV. I want to implement mouse click by the direction of hand motion.Can you please suggest to me where i could learn how to use emgu CV to do these functions...I am beginner and have never used emgu CV before so I do not know where to start ... I would really appreciate it if you could provide me some information on how i could learn it...thanks a lot...

[badrepent]

@gebspsg

If you are a beginner you should start with more simple task and then try to face more

difficult projects. The one you are referring to is a mid-level project, there are a lot

of ways to solve it, you could use optical flow as i've done in other video on my channel, you could adapt motion detection example that comes with EmguCV and so on...

[gebspsg]

@badrepent

Thank you so much for your suggestion. I will try to work from basics. I have one final question though. I do not know openCV. Should i learn how to use openCV first or does emguCV includes all openCV functionalities so i can learn how to use emguCV separately without learning openCV?

[badrepent]

@gebspsg

You can directly use Emgucv that nearly covers all Opencv functionalities. I can suggest you before start writing code to read "Learning Opencv" book and then try to implement using Emgu some of the exercises and samples...

[gebspsg]

@badrepent oh okay. i will do that. Thanks a lot again for you help.

[TheMechanify]

When taking input from a webcam, the contours that are drawn around the hand are not smooth. They are zig zag. What parameters should I modify to suit the conditions ?

[badrepent]

@TheMechanify

You can adjust the level of approximation done using ApproxPoly function.

[MadSaint100]

thanks friend..il try it soon

[MadSaint100]

friend can you create a tutorial where using hand gestures to click a button.. for example click play icon on a video using hand gestures

[badrepent]

@MadSaint100

I do not have the time in this moment to extend my project. Use my code as a baseline and then you can detect fingertips and simulate button click...

[ziadHak]

Please could you help me by sending the link of downloading the project?

Thanks in advance

[ziadHak]

Please could you help me by sending the link of  downloading the project?

[TheMechanify]

@badrepent Instead of giving a video file as input, i m taking inputs from a webcam. Its throwing me a NullPointerException at Line 173 of form1.cs. The line is

for (int i = 0; i < defects.Total; i++)

As per the exception, defects variable is not initialized before.

[badrepent]

@TheMechanify

It means that no convexityDefects has been found, try to adjust code parameters

for your needs.....

[TheMechanify]

@badrepent Hmm.. I will try that. Thnx a lot for your help !

[TheMechanify]

@badrepent

Thanx a lot for your help ! The code is working fine now ! ! :-)

[badrepent]

@TheMechanify

I'm happy about it :) i'll be glad to look at your result...

[TheMechanify]

@badrepent

Yes. Will try to post the video asap. I m trying to emulate mouse functions now with your proj as base.

[aminurwg]

I am failed to download your code and download

aminur

[badrepent]

@aminurwg

The download works fine for me, double check it!!!!!

[TheMechanify]

@badrepent from where can I download the project ?

[badrepent]

@TheMechanify

Download links are inside video description :)

[TheMechanify]

@badrepent ohh thank you !! I missed that. Sorry to bother you with this. :)

[shahyan30]

Thankyou very much bro :) AWESOME.

[tcboy88]

what are those dll opencv_calib3d211.dll,opencv_c­­ore211.dll btw?

becoz i seems cant find it in my emgucv folder, where are those dll come from?

wat is their difference with cv210.dll from opencv?

[tcboy88]

hi badrepent are u luca del tongo?

i have tested ur vs2010 version, and also vs2008 version, which u posted on andol website.

but the vs2008 version throw error when grabber = new Emgu.CV.Capture();

however i managed to fix it by copying all the dll from ur vs2010 version such as opencv_calib3d211.dll,opencv_c­ore211.dll to the vs2008 bin folder

Is this a problem?

anyway, i tried the 2008 version and it seems the finger detection has improved.

thanks for ur effort

hope to listen from u soon

[badrepent]

@tcboy88

I'm Luca Del Tongo and if i'm not wrong i introduce myself in the video :)

vs2010 and vs2008 version are based on exactly the same code...

i have provided both versions because a lot of you requested a vs2008 one.

Glad you like my videos, i'll hope to find time to post something new in the near future...

[tcboy88]

@badrepent

may i know wat are those opencv_calib3d211.dll,opencv_c­­ore211.dll etc included in yours vs2010 version?

becoz i seems cant found it in opencv or emgucv directory.

[tcboy88]

@tcboy88

i think i knew wat it is

it is just a new naming system in newer release of opencv

[badrepent]

@tcboy88

Yep you are right, sorry for replying you in late!!!

Opencv naming convention has been modified :)

[shelzmarie]

Hey Bro...may i know what are the other requirements for this. I already have c#, but 1. about openCV wrapper do you have specific version to use or can you use any version? and can you gave me a site where i can download this wrapper...

2. what kind of camera did you use for this project?

3. i read an article about openCV can be run on linux/Mac os...so does it have OS requirements to run this program??

THANKS FOR SHARING YOUR CODE MAN!!!

[badrepent]

@shelzmarie

There's only one requirement for this project:

1) EmguCV wrapper that you can download from emgucv website... i have attached the right dll version to make it works but you could replace them with the latest svn nighlty build if you know what you are doing

2) EmguCV is compatible with Mono, i have tested it personally so you can run my code both on windows and linux machines

3) My webcam is not supported on windows 7,so i use my sony handycam to record the video

Hope this helps

[shelzmarie]

@badrepent THANK YOU BRO...this will help alot but i think i will still be asking you some question on the way...is that okay with you?^_^

[akhlaqist1]

thanks alot man ;-)

[badrepent]

@asteris145

Il codice che ti ho scritto serve per la webcam, se al costruttore della classe Capture, non passi il percorso al file lui cerca di acquisire dalla webcam usando la webcam con indice pari a zero (di solito è quello il numero della webcam)

[badrepent]

@asteris145

Basta cambiare una riga di codice per farlo funzionare con la webcam, ovvero

Capture capture = new Capture();

[badrepent]

@asteris145

Vuol dire che nn trova il video, assicurati di aver scaricato la versione da 43 Mb.

Se poi avessi problemi copia il video dentro la cartella bin e metti il path cosi

".\M2U00253.MPG"