Mixed reality - moving object, moving camera

[2RandomStudios]

This is a nice effort, but the frame rate is horrible -_-

[mikeyrt16]

Its blatently a real ball.

[PixelThink]

buenichimo amigacho

[Rollersteaam]

Wow what the fuck... am I tripping?!

[wightknight121]

obviose what your tracking

[RickRay9]

Wow, it looks like a game called Myst, you should try it out, it's a classic.

[alanpgoodwin]

Very nice, precise track. The lighting and resolution of the ball could use some work though. I'd use an angular map of the video footage itself as indirect lighting for realism. Making the ball bounce over the gap in the wall would give it more realism too.

[alexneskin]

really cool cam tracking

[SelfEmployedAssassin]

Am I mistaken or isn't this called 'Augmented Reality'?

[roomk]

Uhm, it would be "augmented" if the synthetic object was a representation of an otherwise invisible (albeit real) phenomenon; this is "mixed" because the ball doesnt'exist!

[anim8ordude]

Track? Perfect.

Render? Needs work. The lighting seemed unrealistic. Did you render it with a raytracing renderer? The lighting on the ball wasn't bright enough and the shadows weren't dark enough. Apart from that, fantastic job.

[roomk]

thanks for your detailed comment!

Render wasn't the object of my experiment, I was interested in a good tracking and in the simulation ("mimic" is more appropriate) of the rotation of the ball. This is why I chose a stripped texture and a simple raytrace for the render ;-)

[roomk]

...common logics?

what's that?

:-)

[EntanglementSatan]

You did much math and stuff but you forgot the common logics! I mean.. you see there is a gap in between the two portions of the wall. Ball ought to fall there or ought to get disbalanced!!

[Kortail]

Nice, good work. Try and make a photorelistic renderning next time that would be awesome !

[roomk]

Yeah, that would be the mandatory next step :-)

Thanks for commenting!

[acroduster]

only thing that dissapointed me was the lack of the ball dropping into the gap between concrete slabs. though I realize how much additional work that would entail!

[roomk]

yep.

I should have used real rigid body simulation, or (less elegantly) manual animation to fake it...

Next time!

[acroduster]

dont sweat it! truth is, sometimes its just crazy the extra hours needed for something that seems so trivial! I am working on a project right now that just does not have the budget to justify its existence, but we are trying very hard to take the dynamic simulations to the max that blender can handle (we have plenty of computer power at our disposal) and we are dying trying to fine tune all the details! its brutal! I love your work though so dont think my comment was an attack.

[roomk]

"[...]a project right now that just does not have the budget to justify its existence[...]".

That's exactly what art is for ;-)

Besides its technical implications, using Blender for dynamic simulations is always related to some form of art, I think (otherwise we should use more specialized software, in order to achieve "scientific" simulations).

Never thought your comment was an attack.

I appreciated it indeed. :)

[acroduster]

its interesting htat blender was the choice for one of the oceanagraphic departments at oregon state university for scientific evaluation of dynamic simulations, the physics engine and fluid sim features of blender are incredibly accurate to real work physics to the point that scientists are using blender for simulation evaluation. blender is not only great for art, but its parametric capabilities are very interesting for application other than art. but I know what you mean.)

[roomk]

Well, it depends on the purpose of the simulation, I think.

I'm pretty sure Blender can't (yet...) consider variables such as, for example, temperature variation and consequent local density variation and convective effects on fluid motion... This is very difficult also at small scale, and virtually impossible at large (that's why we can't predict weather in medium-long term).

[roomk]

I mean, models are semplified pictures of phenomena, and each model is a "view" on a subset of them (otherwise we should simulate everytime the whole world... and then take a snapshot of the part we're interested in).

So I'm sure that guys at Oregon are using Blender for simple models not asking for accuracy in thermal issues.

[acroduster]

no doubt! but thats why there are packages like pro e, kativ, solid works and inventor! with high end plug ins that can account for such variables. I doubt that hard core fluid mechanics will be included in blender or maya for that matter, any time soon. hel the plug in for full fluid dynamics in solid works is more than the core software right now. maybee in time we will get to play with open source engineering!)

[roomk]

>maybee in time we will get to play with open >source engineering!

yep, I hope so. ;)

If only... in the meantime... I could find... an open source and free 3D CAD...with a satisfactory GUI on Windows... :(

I'm in architecture...

[acroduster]

with a background in solid works and inventor, I would love to see open source solutions offered, but I think it will be some time, before we see the level of development currently in blender in any open source parametrically driven solid modeling. but I still hold hope.

[firasnashed]

Very nice job - look at my movie Bisso Cat.

[kusoheadexe]

is there any tutorial out there teach you how to do this?:)i mean with blender

[roomk]

I'm sorry, don't know.

These are quite complex tasks, and you've to read many tutorials on every single task involved (camera/object animation, motion tracking, compositing, and so on).

[kusoheadexe]

ermm...so what's a fiducial markers?

is it to do with a third party software?

[roomk]

Fiducial markers are used to mark and identify points whose position in the scene you know exactly. They are used to help your pc in data analisys. Every persistent point in your footage can be a fiducial marker, if you say to your motion tracking software: "ehi! this point in frame #1 is this point in frame 16! keep this in mind!".

It's better if you choose a lot of points (and every point far from the other), because a totally automatic recognition is a long process.

[kusoheadexe]

okay...that really sounds like a lot to learn :P thanks anyway, i ve done similar thing before, just that i dont know how you can get shadow project on a real object in reality, and also make your object follow the track despiting the moving of the camera

very cool indeed :)

[stokmarknes]

Could you explain to me(without blowing my brain out with your vocabulary) what formula you used to calculate the rotation of the 3,14dm ball?

[roomk]

:( is something wrong with my vocabulary?

Anyway:

1) make your ball rotate (you choose the speed) WITHOUT moving it.

2) a complete rotation takes a certain amount of time. let's suppose this time is 1 second.

3) make your ball move along chosen axis, every 1 second, for a distance exactly [3,14 multiplied by (ball diameter)], that is circumference.

In other words, every complete rotation, the ball runs a distance exactly equal to his circumference.

[stokmarknes]

no, nothing is wrong with your vocabulary, but it's to complicated for me when you start to explain in details:D

THANK YOU for the short tutorial:D

[roomk]

;-)

[ehrenk1]

what type of tracking are u using without fudicial markers. speed of the object looks constant which id imagine is very difficult? how does this type (moving object moving camera) tie back into the tracking.

[roomk]

I'm not sure I fully understood your question(s), but anyway:

1) About fiducial markers: some objects are always visible, for example the wall segment on the foreground; moreover, general geometry of the environment is simple, being essentially orthogonal; moreover, tiles on the groundplane are useful, every persistent joint is a fiducial marker... so I can reconstruct 3D data I need starting from incomplete tracking data.

[to be continued due to youtube text lenght limitations...]

[roomk]

[continued from previous comment]

2) About object (the ball) speed: once completed tracking process, I got in Blender a 3D cloud of points, some helpful, some only being "noise" (in computational terms). Then i modeled a simple plane matching the top of the main wall (this isn't a mechanical task, however); so I got a complete 3d environment for my animation: view angle is not important in this stage.

[to be continued due to youtube text lenght limitations...]

[roomk]

[continued from previous comment]

Then I simply made the ball move (and rotate) with constant speed on my simple plane. This done, I used tracked camera data to animate the 3d scene in a way that matched the scene foreground, that is the filmed footage.

Hope this notes are clear... Thanks for your interest!

[bluemorphomedia]

Very difficult to do! Great work!

[maideva]

Very nice!

[stone412]

That was quite a video you made, roomk! I see you're into 3D animation applied over real video! That must've taken you a while to do that, what with the precision angle matching, proper lighting and all! Job well done! Keep up the good work, K! :-)

[roomk]

This is my ideal comment :) I'm using free and/or open source tools to do my experiments with CG and real videos, so my happiness is doubled when people like them!

[stone412]

Well, multiply that happines by four, K! Because it's about time you're not only recognized for your work but appreciated for it, as well! :-)

[roomk]

:-)

[sweetlurch]

Deep Perception, dude,. It's like you found the different angle.

[abitofwhimsey]

I like this very much

[roomk]

Well, thanks! :)

[otislabcn]

Is your video processed in real time??

[roomk]

No. This is a 3D animation with transparent background, layered on top of original footage.

The most complex technical task is camera matching, that is, reproduce in CG animation the same camera parameters of the live shot; these parameters are basically focal lenght and movements of the used real camera.

When real and CG camera match, your 3D animation can be composited with original video as background.

Full process is a bit more complex, but this is what I essentially did.

[roomk]

Looking at your account I see you're very familiar with augmented reality, so forgive my rather simplistic explanation.

I'll leave it here for less specialized people ;)

[NintendoTom]

what prog do you use ?

[roomk]

All my cg videos here are made with Blender (www.blender.org), sometimes using Yafray (www.yafray.org) to render; both are free & opensource.

Camera tracking with Icarus, free for personal use, no more ufficially distributed.

[mastud]

I'll leave that up to you mate! If i had youre computer talent i would be one happy camper! Try having one of you're animations interact with you! Give it a shot!

[roomk]

Nice idea! Will insert it in my todo list :)

[mastud]

I have NO idea what you are talking about but all i CAN say is that you're video is bloomin class! The things you can do with ideas like that! think about it!

[roomk]

Oh, thanks! My video works here are basically technical tests, so I'm speaking about.. technical stuff. Like you on piano, I'm self taught on computer graphics. Any suggestions on things I could do?