FAST corners and SURF tracking on iPhone 3G

Here's a demo of SURF descriptor matching(tracking) using FAST corner detection on the iPhone 3G.

At this stage, the framerate is quite low and calculating SURF descriptors take up about 66% of processing time, but hopefully everything can be optimized. Please leave a comment if you have any ideas/questions.

Category:

Science & Technology

Tags:

• Augmented Reality
• FAST corners
• iphone
• 3G

[darxide12345]

Good job!

IMHO down sampling the image could greatly improve speed, though I'm not sure about the impact on detection/tracking performance.Worth a try though.

Meanwhile, I don't think the CPU of IPhone have hardware support for floating point operations. If you are using such operations to calculate SURF discriptors, I think changing them to fixed point operations might help as well, although upflows and downflows might be a little tricky to handle.

[carelvwyk]

@darxide12345 Thanks for the comment! I'll check out your suggestions. Also have a look at: "OpenCV FAST corners and OpenGL on iPhone 3Gs" where I captured incoming video frames at a lower resolution and used OpenGL output for very fast FAST detection

[carelvwyk]

I don't think you need SURF for that.

You will need to google around a lot, but you can run OpenCV on the iPhone, you will use AVFoundation to access the camera video stream and it is not difficult, it will just takes some time.

[yxccumt]

Great job! What SURF code do you use? OpenSURF?

[carelvwyk]

@yxccumt Thank you! Yes, I used OpenSURF. However, I found that SURF is just too slow. I decided to use sparse optical flow for tracking (lucas kanade) instead of descriptors. What are you busy with?

[yxccumt]

@carelvwyk I saw Learning OpenCV on the corner. So you you use lucas kanade for tracking instead of SURF feature matchin? I'm working on some mobile phone head tracking stuff. I'm a newbie on phone programming ... Is running OpenCV difficult on iphone? How to access the camera video stream? In opencv it's just a function. Is it difficult on the iphone?

[carelvwyk]

Well youtube is being a dick and not sending through my reply.

[odedbendov]

Now there's a Lucas Kanade Optical Flow project as well to test.

checkout labs . success-ware . com

Cheers,

Oded

[TheFreddy4]

Nice job,

checkout this implementation of fast corners - search "fast corner iphone" and see the hatzlaha one (success in Hebrew)

it was hand tailored to offer the best performance I could reach, so it might offer some performance gain if you could incorporate it.

Cheers!

[yxccumt]

@carelvwyk Yes, for my job, SURF is not needed. But I'm generally interested in all vision techs. Thanks for your help!