Hatsune Miku sings 'Tell Me!! The Magical Lyrics' on the table

Updated version of the Augmented Reality program based on OpenCV. I've made many improvements since my previous post (watch?v=nJsIK9b7IpU). A postcard is used as an AR marker. ARToolKit is not used in this program. 3D polygon model display, animation and physics simulations are based on the source code from 'ARToolKit_MMD' software. Many thanks to the authors of music, dance, motion data, polygon model and ARToolKit_MMD.
(NicoNicoDouga : sm11517127)

[References]
Music: sm1323757 'Tell Me!! The Magical Lyrics'
Dance: sm6410906 Yumiko dances 'Tell Me!! The Magical Lyrics'
Motion: sm10133045 Trace of Yumiko's dance
Polygon model: sm11205201 'Lat-style Miku Ver 2.3'
Polygon model display software module: sm10027688 'ARToolKit_MMD'

Category:

Science & Technology

Tags:

• Augmented Reality
• OpenCV
• VOCALOID
• Hatsune Miku
• Tell Me!! The Magical Lyrics

[kitaru0kun]

The marker recognition program is something you programmed yourself? Impressive since it seem much smoother than ArtToolKit. What's the difference? Also, can the marker be recognized upsidedown?

[foxcage1]

@kitaru0kun Yes, I made this marker recognition program for my personal learning about AR. The difference between ARToolKit is that this program recognizes many feature points in the marker and has certain robustness against marker occlusion. However, it requires much CPU power.

The marker can be recognized upside down, so you can rotate the marker to see the back of the CG object.

[vedlith]

Nice job Sir, I see it can work even with 2 of the corners not visible by the camera. Did you use edge detection instead, some kind of estimators (I wonder if kalman+some rigid body physic can be used to stabilise rectangle detection even with some corners disapearing from view), or simply clipped the video and the rectangular marker was indeed visible by the software? xD

[foxcage1]

@vedlith Thanks for the comment. This program does not detect edges or corners of the marker. Instead, it detects many feature points inside the marker. So it can recognize the marker even if some corners are not visible.

[JAM007BLEACH]

genial ,donde puedo saber mas  informacion y el codigo fuente del programa base?

[foxcage1]

@JAM007BLEACH Lo sentimos, este programa está todavía en desarrollo y el código fuente no se distribuye por el momento.

[RoseBleueDuRosierSec]

Sería estupendo poder hacer esto con la 3ds...

[kaylieclarkson]

what did u use to do it?

[kaylieclarkson]

how did u do this?

[vedlith]

@vedlith baaka, or just continued processing with corners detected at the very bottom edge of image, I presume ^^. Good job anyway ;)

[x2s7]

you are great, kind sir, for using Lat miku..