@TheLegendaryJai: Which part in particular? Blender is avaliable from Blender.org (can't post the full address) and Monet is avaliable from the above link. As for video editing software, you'll have to find the solution that serves you best.
@Systemoffive Nope, Monet is not avaialble anymore, they removed it seems. Can you upload the version you own yourself for us to download? If you still have the install file I mean :)
@linkanparkfan It's generated by the python scripts inside blender based on the image data. The video needs to be edited before use, and overall the process is a bit longwinded, but it works. Download is free, and the manual should give you a decent overview!
@linkanparkfan The download link for the system is avaliable in the video description. The programme that it uses is the old version of Blender (sorry YouTube won't let me post links) but you'll need Blender 2.49 NOT the latest release (we're working on that). As for editing video: you'll need to export to GIF format - nearly anything can do that, and there's a simple chroma setup bundled in the zip. Good luck!
@ThatGuyWithHisHat I've had a couple of requests for that. Given that I'm busy developing for a contract at the moment I've had little time to work on Monet. Suffice to say, I've already written in the ability for the system to support custom point schemes. The present release does have a muscular solver for the cheek, but it's not the same ;)
@viwooshprogrammator Usted necesitará una copia de la licuadora, y una copia de la PIL, también podría enviar una copia del manual en formato RTF que se puede traducir. Enviar un correo electrónico.
The final version for Blender 2.49 is out and ready for use! Given the new tools with Blender 2.6 the next beta will be quite different to the present system, but it'll be all for the better. Meanwhile, I hope you all enjoy the new build. It's avalable from the usual place!
@FIBackRoom I want to have the final version for Blender 2.49 out within a month, and the beta for the 2.6 version released within the next two months. I also really need to get a video out that better represents the current build! Do read the manual before you start, because Monet isn't designed to be that user friendly.
Well I'm sick of waiting for the PIL to be released. I'm improving the base armature to have intuitive hand controls, and expanding this version of Monet so it can potentially record as many points as the user would like! How I'm going to keep it easy to use...now that's the confusing part!
Furthermore, I'm looking at getting the armature working with full-body motion capture. And am thinking about armature assisting scripts to make animation faster.
@TheGennaios The camera was an old water-damaged sony, it was edited in Adobe Premier 6.5 (both were purchased in '02 if I remember correctly). Pretty much any camera should do the trick - the main thing is getting a clean image with minimal noise, and exporting the sequence as a GIF so that it can be read by the script.
The new build of Monet for Blender 2.59 is progressing well, but is currently held back due to the Python Image Library not being compatible with Python 3.2.
I'll not be making any large changes to the programmes function, but will be "building in" a better core which will probably cause delays once the PIL has been released. The new Monet has a significantly improved rig compared with the present build.
@happymushroom100 As in a tracking point on the cheek itself or a cheek-deformation controller? I've been trying to push the build through into blender 2.5 reciently and have been having some difficulty with the script (and python image library in python 3.2 - that's a spanner in the works!). The Armature has had some attention and now includes bones for cheek deformation as well as better hand controls. I've been thinking about having the option of different dot setups. What are your thoughts?
@Systemoffive well I've done tracking in outer software before and I'm not a pro but i think things look more real when the cheeks are tracked. 1 problem is sometimes it can cause problems. tracking point on the cheek will work but sometimes you control it to stop some problems it depends on the video. a cheek-deformation controller that worked hand in hand with a tracked point would be handy.
@happymushroom100 I've considered putting some on the zygomatic arch (cheekbone) but it seemed redundant as the movement is partially reverse-engineerable using the locations of the mouth. With a good rig (unlike the one I've got) most of the zygomaticus muscles should be reversable - as for puffing the cheeks etc...we could do with a point to track that. I'll get onto it as soon as it's compatible with blender 2.5! In saying that - I expect some genus will come up with a better system soon!
@superherosrocktiger1 Youtube won't let me post a link for some weird reason (and it's to a google source page...annoying). The most recient links are contained in the video's description. Make sure you read the manual before jumping in! There are a few tricks for new players.
@Pabloko09 Nothing! Eye animation is added later - with the crappy camera I've got I couldn't really track eye position when I was testing it so quickly gave up (doesn't help that I'm part Chinese and I can't open my eyes much). The rig that's distributed with the system tracks eye movement to a goal, and any eye actions: blink etc. need to be added later. Again, I'm not a genus, I just made something to make my life easier, but: if you've any ideas/solutions - I'd love to hear them!
@mello1801 Good point! I hadn't really thought about extra movement there, but now that you've mentioned it I might overhaul the present release and tweak a few of the facial bones. In all seriousness this display is getting pretty dated. Shame you can't just replace videos on YouTube to keep it up to speed with the latest version.
@MurciellagoSV Hehehe yes I do understand, and call me horrible but I'm not going to release models from my portfolio for free! Although if someone happens to make a good model face that's suitable for animation, I'd be happy to rig it and release it for Monet in place of my rush job!
@MurciellagoSV I'm not precious about any software - it's just a tool. But you see, the thing is that a team of "industry professional" developers cannot compete with an expansive team of nerds from all around the world, with my addition Blender got a basic facial motion capture system and people add more and more features every hour. However, programmes like ZBrush will probably soon oust most 3D software - AutoDesk is already struggling...but due to support, blender will still survive.
@AniCator If only we could afford to hire people like that! Glad you liked it though. Figure I'll have to make a "how to" if we get enough hits on this :-)
@termi892 Yes - lighting is important. We've got a white longue which gives a good unamious lighting setup. Try to bounce light off the walls instead of using a directional light, and also play with your key colours a bit to try and find what works best. Experiment - you've often got to get inventive to get free things to work ;)
@termi892 Yeah - just writing up the documentation for the new release at the moment! There's been a nearly complete re-write in the control systems, and they're now freely editable so that different armature proportions can be used, and custom keys can be created and inserted - just trying to explain all of this in the manual is difficult. Don't worry there have just been delays for a few good ideas, it'll get released ;)
Suggestion: post this somewhere else, like Blenderartists. I think it would get more attention there. After all, this is a one of a kind script for Blender, I dont even know of a commercial mocap tool for Blender.
@termi892 That's not a bad idea! Again, as before, this script was really developed for my teams use, so I've not really been getting it out there like I should be. I'll have a shimmy on over with the new 0.55+ version in about a week. Thanks!
Quick, dumb question - whereabouts do I find the FaceRec.ini file? It wasn't in the zip, as far as I can tell. Also, I'm on a Mac, in case that makes a difference - I see from the manuals' screenshot that you're rocking a PC.
@spacemenspiffy That's something I hadn't thought of! FaceRec.ini is created when you do a quicksave while using PointAnimator.py. It's not essentual for function, and was designed to speed things up. You can change it's path by modifying this line (about 110):
inipath = "C:\FaceRec.ini"
You may need to give me a PM about the format requirements for the Mac as I've never programmed for it. Also you must have the Python Image Library compatible for your system.
@Systemoffive Also, I took your .blend file over to an old PC laptop I had lying around and tried it there (after installing PIL) and it worked fine. Well, not exactly "fine" - too much head movement in my reference video/GIF led to the face going all wonky and distorted. But I was able to take the .blend back to the Mac, and it works fine, so that workflow should work for me. But I'll keep checking into the PIL issue on my Mac.
@spacemenspiffy I've been trying to expand the usability of the entire system: this means keeping the ini file in the same directory, adding quick custom support for face shapes, adding in dynamic and editable menus etc. Problem is that the commands are windows-specific. While support for custom face shapes is due to be added in a few days (and the next release shortly after)...the system will be 100% windows after this. I'm not actually too happy about that.
@Systemoffive My next step - building a helmet-mounted reverse POV camera rig, so the head movement in the performance capture becomes a non-issue. I'll post up when I've made progress on that.
@spacemenspiffy If you want to cut the mesh up and convert it into something else - go for it. As a technical note: there's a heap of "helper" bones on bonelayers 2-5 for facial movement.
As for a head mounted camera - we actually bolted an old bike helmet to a chair, but it didn't take long for actors to learn how to preform without moving their heads. Try putting your hands on your head to limit movement, or lying on your back. I'll be keen to see what solutions you find!
@termi892 I'm also very short on cash - which was part of the reason I developed this.
I know that blenders composite node system can be used to pickup a green screen (or coloured matte) and it's similar to the method that I use to pickup the points. Although blenders system isn't as refined as that which you'll find in video editing suites, so takes a bit of tweaking. There are plenty of guides out there.
Good luck! And a shame it won't let me post links... |(
Theres a "select by color" tool in GIMP, will it do that? Ive tried it and different colored markers, but the selection always includes more than just the markers :(
@termi892 For ease of use and speed I think you'd be better with a proper video editing suite as they're much more effective than doing frames individually, blender does have the ability to pickup keys using the node editor.
(It won't let me post a link...? Google it, sorry!)
I use an old version of Adobe Premier, but you could use a node setup to return a sequence that the system can recognize. Will add process a tutorial with the next relese.
I think its possible to have markerless eye tracking somehow...
I think most image editors use the same name for the filters... so you could try posting the name of the filters you use in your program and people using other tools could still understand it.
@termi892 Page two of the manual that's bundled in the zip has a breif description on how to setup an output that can be used by the system.
In short: Use a Chroma or RGB key on the coloured markers and export the resultant alpha mask as a GIF sequence that can be read by the Python Image Library, so they're black dots against a white background on the exported mask.
As for eye tracking: figured eye whites would do the trick, but they're too unreliable. Cleanup outweighed manual animation.
@termi892 The easy answer: The eyes aren't tracked because I found there wasn't enough "expression" in the eyes with tracked points. Eye movement and additional cleanup of facial movements can be done using the FacialControl.py script in the setup.
You will, however, need a video editing suite to pick out the points. I've touched briefly on this in the manual but didn't go into it in depth as there's so many different suites.
Will make a "How To" with the next release to help people out!
Hi everyone! I'm preparing another release of this system and will hopefully have it done shortly. If there's anything on your wish-list please don't hesitate to tell me and I'll do my best to address it. The new system has much better point-correspondance and picks up more of the "little" movements around the face. The rest of my "To Do" list involves a cleanup script and some more tongue positions.
@funkboxing I'm looking at getting anothe release out in a few weeks - just something with a couple of tweaks and a few extended features. Would love to hear anything you could suggest! Best of luck - it's not the most forgiving system, I should write a "cheat guide" in the manual for common problems.
@PsychoCalamander I used an alpha mask from a chroma key setup in Premier Pro 6.5...the old version. Because the system uses output from this, I don't see why painted on dots wouldn't work. You may need to tweak the pointdia and totalpointdia to cover stretching of the points. As long as you get a good return from whatever key you use in your video editing suite, you should get a reasonable result from Monet. Would love to see how you go and hear any comments/improvements you have!
@Systemoffive i'm just a beginner :D First I have to wait 'till i get my room done (in june), then i have to buy a decent PC and then i have to buy a graphics tablet :) But yeah i wont give up my "dream" of becoming a 3D model artist / computer animator
@PsychoCalamander It's the start of a life-long obsession! My computer at home is only a single core 2ghz machine, 2Gb of ram and a whopping 500Gb hard drive (cough). Doesn't matter where you start as long as you're having fun! The Tablet, however, is probably the best purchase you'll make - I've just got a cheap Wacom Bamboo but it's brilliant. And all of this happens in the corner of a tiny 12x4 room. Wish you the best! And remember: The computer helps, but your imagination is key.
@lionsforlyons We've been working on an animated TV series for about a year now. Having apporached networks for funding they told us that they wouldn't fund and would only purchase a full series. So blender, being free, seemed like an obvious choice. The script was developed as a cost-cutting device (as facial animation takes time). So the machine had to assign points automatically also. Best of luck, and feel free to post feedback!
@98765a1 Gawd - just typical that I forgot to put that into the Manual! Basically all it is is felt. If you swing by your craft shop, usually you can get coloured felt with adhesive backing that's used in scrapbooking, kids stuff etc. And they're simply made with a holepunch. Read through the manual to get the roughs for how it works, and use the demonstrated mesh to rig your own - would love to see the results! Good luck!
@lionsforlyons They could easily be scripted in. Thing is I wanted a system that worked for my project - no extras ;). The other thing the system does is automatically sort points without the user having to assign them (unlike other programmes) so if more points were added there'd just be a bit more coding required - I've been considering having support for different "patterns" of dots on the face. Will gauge interest before developing it further for the community.
If the assets are the only problem, we could just make our own. You could even post in blenderartists for modellers help, I'm sure many would be interested to have their model as the star of this script.
Btw, I might get a kinect myself soon, could help on that (like I said I also know Python)
@98765a1 Well rented a Kinect, tried it, and frankly wasn't impressed with it. There's too much noise in the image, and cleanup costs time. Having been using the script for a few weeks now I can't really think of much more that needs to be changed. I can't find a way to track the eyes or tongue easily, and those would be the only other things I'd want to add. The tongueshape system could do with innovation, but that's about it.
@98765a1 Hey dude, I know you're just hanging out for this but there's a heap of hoops to go through with commercial property. I'm waiting on a reply from theFXGuy about webhosting, and want to add in support for custom bone names, some tweaks to the flow of the post-editing script, and am trying to get a Kinect working so I can see about hacking that into the animator also! (Tell me that's not an awesome idea?). The wheels are turning, and I just await a reply from theFXGuy.
As it turns out the whole lot is IP. However, as the creator I'm keen to get it out there and get this script developing. The key components related to the "screen presence" of the character are his physical look and form - so the mesh and armature will be stripped down, but everything else will remain the same. It'll be out within a week, once I've written up the documentation for everyone to follow!
I completely misread your post and had the impression that you couldn't release the script as you weren't the only developer and it was proprietary.
Don't worry about sharing the visuals, we don't need it and Im guessing the chances of that happening is low. Id be more than happy to rig another head for you.
So unless by the contract the script is considered not your property, then please share
IlI also be happy to contribute to the script to allow custom amount of markers and positions.
@lasolable First off: Great choice with blender. I'm learning Maya atm and it's just not the same. Again I've just got to wait for clearance from my team and production company. While the script has been refined, please don't expect a silver bullet, as it only works with the Armature that I've developed and would require extensive modification to work with anything else.
I'm busy writing up documentation for it atm so that the team can use it.
I don't understand. Is this a Python script using some proprietary algorithms from a commercial application? I got the impression that you were the sole developer.
I haven't created anything like this before, but from my other experiences, the script could take each frame from the camera and use for example contrast to get the marker positions, then you would make Empties in Blender and animate them based on the marker positions each frame and smooth the end results. then parent bone to empty
@98765a1 I wrote the entire script using nothing as a refrence (in other words I didn't look at anyone elses work), but it does use the Python Image Library which is avaliable under the GNU public liscence. I found mesh objects were more effective to track than empties, then the bone is positioned to the object. It's better to pickup points using a chroma key in your faviourate editing suite, and then export the mask as footage. These have been little teething things that I've found easier.
Hello, I have experience in both PIL and the Blender 2.49 API and have worked on file importing/exporting and have also used Python for game development. I'm no real programmer, but I'd love to see the code and help on this.
Though all my models use bones for facial animations right now, so I guess I'll need to switch to blendshapes, huh?
@98765a1 I've just put the finishing touches onto the system tonight. By chance was near an interent connection (sorry if I don't reply much), Against my best judgement, the system uses bones and bone positions to make the face shapes. I would have prefered to use shapekeys as they're more hardy, but had to come up with some smoke and mirrors instead.
As this is part of a production, the visual aspects are locked up in IP, but I will talk with the team and see how much I can release.
Subbing for this. I had an idea a lot like this except I was going to use the After Effects point tracker (because I don't know python) and use drivers/ shape keys to deform the mesh based on the positions of the points.
@TheFXGuy The system simply tracks points and applies offsets. There's nothing new, just that it seems to be the first time it's been done in Blender. I've had to develop a heap of smoke and mirrors to get it working well, and it's become very "rigid" in that it'll take some editing to work on a rig that's different from the one I've created.
I developed it for our time-starved TV series, and am hoping the script and armature can be released without any IP issues. I'll keep in touch!
@TheFXGuy I'd be dead keen to take you up on that! It's just a very busy time around the studio at the moment and our parent company has some seriously big stuff happening so I don't know if I can release it or if it's IP. There's a good chance anything I do release will have to be stripped down.
@RevJackStraw It's pretty raw at this stage, and not very flexiable, but I'd be dead keen to try and get a ball rolling amongst the community. I'll keep everyone posted and upload more videos as I improve and develop the system!
@Systemoffive I've subscribed and set up auto-alerts for when you post. I am into scripting myself and look forward to seeing how you coded it. Keep us updated
It´s amazing how the 3d characters can make us speak!!
Soon they will take over the world o_O
kyy656 1 week ago
Just noticed the caption link, I'll use that one :D
Luka1180 1 week ago
How do I download this sotware .
TheLegendaryJai 1 month ago
@TheLegendaryJai: Which part in particular? Blender is avaliable from Blender.org (can't post the full address) and Monet is avaliable from the above link. As for video editing software, you'll have to find the solution that serves you best.
Systemoffive 1 month ago
@Systemoffive Nope, Monet is not avaialble anymore, they removed it seems. Can you upload the version you own yourself for us to download? If you still have the install file I mean :)
Luka1180 1 week ago
what program was used to make motion capture data?
linkanparkfan 1 month ago
@linkanparkfan It's generated by the python scripts inside blender based on the image data. The video needs to be edited before use, and overall the process is a bit longwinded, but it works. Download is free, and the manual should give you a decent overview!
Systemoffive 1 month ago
@Systemoffive where can i dowload this script? and what kind of editing do i have to do to the video?
linkanparkfan 1 month ago
@linkanparkfan The download link for the system is avaliable in the video description. The programme that it uses is the old version of Blender (sorry YouTube won't let me post links) but you'll need Blender 2.49 NOT the latest release (we're working on that). As for editing video: you'll need to export to GIF format - nearly anything can do that, and there's a simple chroma setup bundled in the zip. Good luck!
Systemoffive 1 month ago
@linkanparkfan 2.6x there is a tutorial for mocap. Google it. Nice tut
Detroitfarm 4 days ago
you should add tracking for your cheek, and perhaps your nostrils. other that that i love it!
ThatGuyWithHisHat 2 months ago
@ThatGuyWithHisHat I've had a couple of requests for that. Given that I'm busy developing for a contract at the moment I've had little time to work on Monet. Suffice to say, I've already written in the ability for the system to support custom point schemes. The present release does have a muscular solver for the cheek, but it's not the same ;)
Systemoffive 2 months ago
@viwooshprogrammator Usted necesitará una copia de la licuadora, y una copia de la PIL, también podría enviar una copia del manual en formato RTF que se puede traducir. Enviar un correo electrónico.
Systemoffive 2 months ago
@viwooshprogrammator No está seguro de lo que quieres decir. ¿Te refieres a cómo utilizar el sistema?
Systemoffive 2 months ago
keep it up
termi892 2 months ago
Hi, When will the new version for Blender 2.6 be out?
johnwcaldwell 2 months ago
@johnwcaldwell Hopefully before Christmas. It's just a matter of finding the free time ;)
Systemoffive 2 months ago
love the eyes :D hehe
OnnomonnomonnO 3 months ago
@OnnomonnomonnO Yeah, it's gutting that I can make more realistic looking eyes with Blender than I can with zBrush or Maya! Thanks!
Systemoffive 2 months ago
The final version for Blender 2.49 is out and ready for use! Given the new tools with Blender 2.6 the next beta will be quite different to the present system, but it'll be all for the better. Meanwhile, I hope you all enjoy the new build. It's avalable from the usual place!
Systemoffive 3 months ago
thanks! I try
rotandroll666 3 months ago
Thank you! All tho my lip-syncing in my videos is simpler than most. This will save me Allot of time and effort. Now to get Learning!
FIBackRoom 4 months ago in playlist blender
@FIBackRoom I want to have the final version for Blender 2.49 out within a month, and the beta for the 2.6 version released within the next two months. I also really need to get a video out that better represents the current build! Do read the manual before you start, because Monet isn't designed to be that user friendly.
Good luck!
Systemoffive 4 months ago
@Systemoffive Cool, I will. It might take a while, however I'm willing to learn. And Thank you again.
FIBackRoom 4 months ago
Que negoço maluco.
Que progama é esse?
peumalu 4 months ago
@peumalu Sistema de captura de movimento para o rosto! E é grátis! :)
Systemoffive 4 months ago
Well I'm sick of waiting for the PIL to be released. I'm improving the base armature to have intuitive hand controls, and expanding this version of Monet so it can potentially record as many points as the user would like! How I'm going to keep it easy to use...now that's the confusing part!
Furthermore, I'm looking at getting the armature working with full-body motion capture. And am thinking about armature assisting scripts to make animation faster.
Systemoffive 5 months ago
what camera and software did you use?
TheGennaios 5 months ago
@TheGennaios The camera was an old water-damaged sony, it was edited in Adobe Premier 6.5 (both were purchased in '02 if I remember correctly). Pretty much any camera should do the trick - the main thing is getting a clean image with minimal noise, and exporting the sequence as a GIF so that it can be read by the script.
Systemoffive 5 months ago
The new build of Monet for Blender 2.59 is progressing well, but is currently held back due to the Python Image Library not being compatible with Python 3.2.
I'll not be making any large changes to the programmes function, but will be "building in" a better core which will probably cause delays once the PIL has been released. The new Monet has a significantly improved rig compared with the present build.
Thankyou everyone for being so supportive!
Systemoffive 5 months ago
need to track your cheek
happymushroom100 6 months ago
@happymushroom100 As in a tracking point on the cheek itself or a cheek-deformation controller? I've been trying to push the build through into blender 2.5 reciently and have been having some difficulty with the script (and python image library in python 3.2 - that's a spanner in the works!). The Armature has had some attention and now includes bones for cheek deformation as well as better hand controls. I've been thinking about having the option of different dot setups. What are your thoughts?
Systemoffive 6 months ago
@Systemoffive well I've done tracking in outer software before and I'm not a pro but i think things look more real when the cheeks are tracked. 1 problem is sometimes it can cause problems. tracking point on the cheek will work but sometimes you control it to stop some problems it depends on the video. a cheek-deformation controller that worked hand in hand with a tracked point would be handy.
happymushroom100 6 months ago
@happymushroom100 I've considered putting some on the zygomatic arch (cheekbone) but it seemed redundant as the movement is partially reverse-engineerable using the locations of the mouth. With a good rig (unlike the one I've got) most of the zygomaticus muscles should be reversable - as for puffing the cheeks etc...we could do with a point to track that. I'll get onto it as soon as it's compatible with blender 2.5! In saying that - I expect some genus will come up with a better system soon!
Systemoffive 6 months ago
@Systemoffive hey, can i get that script?
superherosrocktiger1 6 months ago
@superherosrocktiger1 Youtube won't let me post a link for some weird reason (and it's to a google source page...annoying). The most recient links are contained in the video's description. Make sure you read the manual before jumping in! There are a few tricks for new players.
Systemoffive 6 months ago
and what happens if you move your eyes around?
Pabloko09 6 months ago
@Pabloko09 Nothing! Eye animation is added later - with the crappy camera I've got I couldn't really track eye position when I was testing it so quickly gave up (doesn't help that I'm part Chinese and I can't open my eyes much). The rig that's distributed with the system tracks eye movement to a goal, and any eye actions: blink etc. need to be added later. Again, I'm not a genus, I just made something to make my life easier, but: if you've any ideas/solutions - I'd love to hear them!
Systemoffive 6 months ago
the cheeks need some movement their on the still side lol
mello1801 6 months ago
@mello1801 Good point! I hadn't really thought about extra movement there, but now that you've mentioned it I might overhaul the present release and tweak a few of the facial bones. In all seriousness this display is getting pretty dated. Shame you can't just replace videos on YouTube to keep it up to speed with the latest version.
Systemoffive 6 months ago
@Systemoffive I said that not because of the MoCap, but the poor modelation. To you understand better, as gamers says, "Shitty graphics". :)
MurciellagoSV 6 months ago
@MurciellagoSV Hehehe yes I do understand, and call me horrible but I'm not going to release models from my portfolio for free! Although if someone happens to make a good model face that's suitable for animation, I'd be happy to rig it and release it for Monet in place of my rush job!
Systemoffive 6 months ago
Blender Sucks!
MurciellagoSV 6 months ago
@MurciellagoSV I'm not precious about any software - it's just a tool. But you see, the thing is that a team of "industry professional" developers cannot compete with an expansive team of nerds from all around the world, with my addition Blender got a basic facial motion capture system and people add more and more features every hour. However, programmes like ZBrush will probably soon oust most 3D software - AutoDesk is already struggling...but due to support, blender will still survive.
Systemoffive 6 months ago
Comment removed
azmilarash96 7 months ago
He looks like the actor who played Cole Phelps in LA Noire
AniCator 7 months ago
@AniCator If only we could afford to hire people like that! Glad you liked it though. Figure I'll have to make a "how to" if we get enough hits on this :-)
Systemoffive 7 months ago
I have problems getting the dots to be recognised. What material were they again? Do you have light source in fron tof the actor?
termi892 8 months ago
@termi892 Yes - lighting is important. We've got a white longue which gives a good unamious lighting setup. Try to bounce light off the walls instead of using a directional light, and also play with your key colours a bit to try and find what works best. Experiment - you've often got to get inventive to get free things to work ;)
Systemoffive 8 months ago
Any news?
termi892 8 months ago
@termi892 Yeah - just writing up the documentation for the new release at the moment! There's been a nearly complete re-write in the control systems, and they're now freely editable so that different armature proportions can be used, and custom keys can be created and inserted - just trying to explain all of this in the manual is difficult. Don't worry there have just been delays for a few good ideas, it'll get released ;)
Systemoffive 8 months ago
Suggestion: post this somewhere else, like Blenderartists. I think it would get more attention there. After all, this is a one of a kind script for Blender, I dont even know of a commercial mocap tool for Blender.
termi892 8 months ago
@termi892 That's not a bad idea! Again, as before, this script was really developed for my teams use, so I've not really been getting it out there like I should be. I'll have a shimmy on over with the new 0.55+ version in about a week. Thanks!
Systemoffive 8 months ago
Quick, dumb question - whereabouts do I find the FaceRec.ini file? It wasn't in the zip, as far as I can tell. Also, I'm on a Mac, in case that makes a difference - I see from the manuals' screenshot that you're rocking a PC.
spacemenspiffy 8 months ago
@spacemenspiffy That's something I hadn't thought of! FaceRec.ini is created when you do a quicksave while using PointAnimator.py. It's not essentual for function, and was designed to speed things up. You can change it's path by modifying this line (about 110):
inipath = "C:\FaceRec.ini"
You may need to give me a PM about the format requirements for the Mac as I've never programmed for it. Also you must have the Python Image Library compatible for your system.
Systemoffive 8 months ago
Comment removed
spacemenspiffy 8 months ago
Comment removed
spacemenspiffy 8 months ago
Comment removed
spacemenspiffy 8 months ago
@Systemoffive Gotcha. I didn't realize the script created the FaceRec.ini file - that shouldn't be a problem with modifying that code to a Mac path.
I think my issue is actually with the PIL - it says it installed, but doesn't seem to be working. I'll do some more digging into that.
spacemenspiffy 8 months ago
@Systemoffive Also, I took your .blend file over to an old PC laptop I had lying around and tried it there (after installing PIL) and it worked fine. Well, not exactly "fine" - too much head movement in my reference video/GIF led to the face going all wonky and distorted. But I was able to take the .blend back to the Mac, and it works fine, so that workflow should work for me. But I'll keep checking into the PIL issue on my Mac.
spacemenspiffy 8 months ago
@spacemenspiffy I've been trying to expand the usability of the entire system: this means keeping the ini file in the same directory, adding quick custom support for face shapes, adding in dynamic and editable menus etc. Problem is that the commands are windows-specific. While support for custom face shapes is due to be added in a few days (and the next release shortly after)...the system will be 100% windows after this. I'm not actually too happy about that.
Systemoffive 8 months ago
@Systemoffive My next step - building a helmet-mounted reverse POV camera rig, so the head movement in the performance capture becomes a non-issue. I'll post up when I've made progress on that.
spacemenspiffy 8 months ago
@Systemoffive Last question - is the mesh included in your .blend free to use, if I were to just want, say, the mouth, rather than the entire face?
spacemenspiffy 8 months ago
@spacemenspiffy If you want to cut the mesh up and convert it into something else - go for it. As a technical note: there's a heap of "helper" bones on bonelayers 2-5 for facial movement.
As for a head mounted camera - we actually bolted an old bike helmet to a chair, but it didn't take long for actors to learn how to preform without moving their heads. Try putting your hands on your head to limit movement, or lying on your back. I'll be keen to see what solutions you find!
Systemoffive 8 months ago
the chroma key is very inaccurate. I get lots of other parts treated asgreen, although they arent.
termi892 8 months ago
Comment removed
spacemenspiffy 8 months ago
So you think it can be done with Blender's video sequence editor? im very short in cash.
termi892 9 months ago
@termi892 I'm also very short on cash - which was part of the reason I developed this.
I know that blenders composite node system can be used to pickup a green screen (or coloured matte) and it's similar to the method that I use to pickup the points. Although blenders system isn't as refined as that which you'll find in video editing suites, so takes a bit of tweaking. There are plenty of guides out there.
Good luck! And a shame it won't let me post links... |(
Systemoffive 8 months ago
Theres a "select by color" tool in GIMP, will it do that? Ive tried it and different colored markers, but the selection always includes more than just the markers :(
termi892 9 months ago
@termi892 For ease of use and speed I think you'd be better with a proper video editing suite as they're much more effective than doing frames individually, blender does have the ability to pickup keys using the node editor.
(It won't let me post a link...? Google it, sorry!)
I use an old version of Adobe Premier, but you could use a node setup to return a sequence that the system can recognize. Will add process a tutorial with the next relese.
Systemoffive 9 months ago
I think its possible to have markerless eye tracking somehow...
I think most image editors use the same name for the filters... so you could try posting the name of the filters you use in your program and people using other tools could still understand it.
termi892 9 months ago
@termi892 Page two of the manual that's bundled in the zip has a breif description on how to setup an output that can be used by the system.
In short: Use a Chroma or RGB key on the coloured markers and export the resultant alpha mask as a GIF sequence that can be read by the Python Image Library, so they're black dots against a white background on the exported mask.
As for eye tracking: figured eye whites would do the trick, but they're too unreliable. Cleanup outweighed manual animation.
Systemoffive 9 months ago
How do you make the video file to feed to it? I put markers on my face but have no idea how to process the video files. I have GIMP.
By the way, are the eyes tracked, or do did you add those later?
termi892 9 months ago
@termi892 The easy answer: The eyes aren't tracked because I found there wasn't enough "expression" in the eyes with tracked points. Eye movement and additional cleanup of facial movements can be done using the FacialControl.py script in the setup.
You will, however, need a video editing suite to pick out the points. I've touched briefly on this in the manual but didn't go into it in depth as there's so many different suites.
Will make a "How To" with the next release to help people out!
Systemoffive 9 months ago
Hi everyone! I'm preparing another release of this system and will hopefully have it done shortly. If there's anything on your wish-list please don't hesitate to tell me and I'll do my best to address it. The new system has much better point-correspondance and picks up more of the "little" movements around the face. The rest of my "To Do" list involves a cleanup script and some more tongue positions.
Systemoffive 9 months ago
Excellent, looking forward to testing this out.
funkboxing 9 months ago
@funkboxing I'm looking at getting anothe release out in a few weeks - just something with a couple of tweaks and a few extended features. Would love to hear anything you could suggest! Best of luck - it's not the most forgiving system, I should write a "cheat guide" in the manual for common problems.
Systemoffive 9 months ago
@Systemoffive is it possible to just paint the dots in your face with normal (green) paint without using paper dots?
PsychoCalamander 9 months ago
@PsychoCalamander I used an alpha mask from a chroma key setup in Premier Pro 6.5...the old version. Because the system uses output from this, I don't see why painted on dots wouldn't work. You may need to tweak the pointdia and totalpointdia to cover stretching of the points. As long as you get a good return from whatever key you use in your video editing suite, you should get a reasonable result from Monet. Would love to see how you go and hear any comments/improvements you have!
Systemoffive 9 months ago
@Systemoffive i'm just a beginner :D First I have to wait 'till i get my room done (in june), then i have to buy a decent PC and then i have to buy a graphics tablet :) But yeah i wont give up my "dream" of becoming a 3D model artist / computer animator
PsychoCalamander 9 months ago
@PsychoCalamander It's the start of a life-long obsession! My computer at home is only a single core 2ghz machine, 2Gb of ram and a whopping 500Gb hard drive (cough). Doesn't matter where you start as long as you're having fun! The Tablet, however, is probably the best purchase you'll make - I've just got a cheap Wacom Bamboo but it's brilliant. And all of this happens in the corner of a tiny 12x4 room. Wish you the best! And remember: The computer helps, but your imagination is key.
Systemoffive 9 months ago
@lionsforlyons We've been working on an animated TV series for about a year now. Having apporached networks for funding they told us that they wouldn't fund and would only purchase a full series. So blender, being free, seemed like an obvious choice. The script was developed as a cost-cutting device (as facial animation takes time). So the machine had to assign points automatically also. Best of luck, and feel free to post feedback!
Systemoffive 10 months ago
Okay, Im ready to test drive the script now.
This might be a stupid questions, but where can I get those green mocap points? don't tell me it's paper...
98765a1 10 months ago
@98765a1 Gawd - just typical that I forgot to put that into the Manual! Basically all it is is felt. If you swing by your craft shop, usually you can get coloured felt with adhesive backing that's used in scrapbooking, kids stuff etc. And they're simply made with a holepunch. Read through the manual to get the roughs for how it works, and use the demonstrated mesh to rig your own - would love to see the results! Good luck!
Systemoffive 10 months ago
@lionsforlyons They could easily be scripted in. Thing is I wanted a system that worked for my project - no extras ;). The other thing the system does is automatically sort points without the user having to assign them (unlike other programmes) so if more points were added there'd just be a bit more coding required - I've been considering having support for different "patterns" of dots on the face. Will gauge interest before developing it further for the community.
Systemoffive 10 months ago
I think he meant a webhost, probably wanted to have a webpage with link to the script, documentation, examples, contact info, etc.
These are also an option: googlecode, sourceforge, github, bitbucket. I'm sure there is some free hosting service for Blender users too.
98765a1 11 months ago
Oops, didnt notice your other post.
Very cool!
If the assets are the only problem, we could just make our own. You could even post in blenderartists for modellers help, I'm sure many would be interested to have their model as the star of this script.
Btw, I might get a kinect myself soon, could help on that (like I said I also know Python)
98765a1 11 months ago
@98765a1 Well rented a Kinect, tried it, and frankly wasn't impressed with it. There's too much noise in the image, and cleanup costs time. Having been using the script for a few weeks now I can't really think of much more that needs to be changed. I can't find a way to track the eyes or tongue easily, and those would be the only other things I'd want to add. The tongueshape system could do with innovation, but that's about it.
Systemoffive 10 months ago
Well you could post from time to time somewhere just to let us know you haven't left. Otherwise, how would we know?
And kinect support would be interesting, but we still haven't tested the current one...
98765a1 11 months ago
and he was never heard from again :(
98765a1 11 months ago
@98765a1 Hey dude, I know you're just hanging out for this but there's a heap of hoops to go through with commercial property. I'm waiting on a reply from theFXGuy about webhosting, and want to add in support for custom bone names, some tweaks to the flow of the post-editing script, and am trying to get a Kinect working so I can see about hacking that into the animator also! (Tell me that's not an awesome idea?). The wheels are turning, and I just await a reply from theFXGuy.
Systemoffive 11 months ago
@Systemoffive My webhost is more stable now, so just email me the file. I'm sending you a PM with my email and I'll give you a DL link.
TheFXGuy 11 months ago
As it turns out the whole lot is IP. However, as the creator I'm keen to get it out there and get this script developing. The key components related to the "screen presence" of the character are his physical look and form - so the mesh and armature will be stripped down, but everything else will remain the same. It'll be out within a week, once I've written up the documentation for everyone to follow!
Systemoffive 11 months ago
Im starting to loose hope
98765a1 11 months ago
Any update?
98765a1 11 months ago
I completely misread your post and had the impression that you couldn't release the script as you weren't the only developer and it was proprietary.
Don't worry about sharing the visuals, we don't need it and Im guessing the chances of that happening is low. Id be more than happy to rig another head for you.
So unless by the contract the script is considered not your property, then please share
IlI also be happy to contribute to the script to allow custom amount of markers and positions.
98765a1 11 months ago
hi! you have did a great job!
i wanted to ask you about source code for this capture, I`m a student and this semester we are working on Blender and me actually on facial mocap.
so i`ll be happy ti see your script =)
lasolable 11 months ago
@lasolable First off: Great choice with blender. I'm learning Maya atm and it's just not the same. Again I've just got to wait for clearance from my team and production company. While the script has been refined, please don't expect a silver bullet, as it only works with the Armature that I've developed and would require extensive modification to work with anything else.
I'm busy writing up documentation for it atm so that the team can use it.
Systemoffive 11 months ago
I don't understand. Is this a Python script using some proprietary algorithms from a commercial application? I got the impression that you were the sole developer.
I haven't created anything like this before, but from my other experiences, the script could take each frame from the camera and use for example contrast to get the marker positions, then you would make Empties in Blender and animate them based on the marker positions each frame and smooth the end results. then parent bone to empty
98765a1 11 months ago
@98765a1 I wrote the entire script using nothing as a refrence (in other words I didn't look at anyone elses work), but it does use the Python Image Library which is avaliable under the GNU public liscence. I found mesh objects were more effective to track than empties, then the bone is positioned to the object. It's better to pickup points using a chroma key in your faviourate editing suite, and then export the mask as footage. These have been little teething things that I've found easier.
Systemoffive 11 months ago
Hello, I have experience in both PIL and the Blender 2.49 API and have worked on file importing/exporting and have also used Python for game development. I'm no real programmer, but I'd love to see the code and help on this.
Though all my models use bones for facial animations right now, so I guess I'll need to switch to blendshapes, huh?
Anyway, waiting for your reply.
98765a1 11 months ago
@98765a1 I've just put the finishing touches onto the system tonight. By chance was near an interent connection (sorry if I don't reply much), Against my best judgement, the system uses bones and bone positions to make the face shapes. I would have prefered to use shapekeys as they're more hardy, but had to come up with some smoke and mirrors instead.
As this is part of a production, the visual aspects are locked up in IP, but I will talk with the team and see how much I can release.
Systemoffive 11 months ago
Subbing for this. I had an idea a lot like this except I was going to use the After Effects point tracker (because I don't know python) and use drivers/ shape keys to deform the mesh based on the positions of the points.
TheFXGuy 1 year ago
@TheFXGuy The system simply tracks points and applies offsets. There's nothing new, just that it seems to be the first time it's been done in Blender. I've had to develop a heap of smoke and mirrors to get it working well, and it's become very "rigid" in that it'll take some editing to work on a rig that's different from the one I've created.
I developed it for our time-starved TV series, and am hoping the script and armature can be released without any IP issues. I'll keep in touch!
Systemoffive 11 months ago
@Systemoffive If you need somewhere to upload it to, I'll host it on my website. PM me if you want it.
Btw, assuming its 2.49 cuz of python scripts? Or is it in the form of an add-on for 2.5?
TheFXGuy 11 months ago
@TheFXGuy I'd be dead keen to take you up on that! It's just a very busy time around the studio at the moment and our parent company has some seriously big stuff happening so I don't know if I can release it or if it's IP. There's a good chance anything I do release will have to be stripped down.
Systemoffive 11 months ago
@Systemoffive That sucks. Oh well, like I said, PM me if you want me to host it :)
TheFXGuy 11 months ago
WOW! Nice work. I'd love to try this out
RevJackStraw 1 year ago
@RevJackStraw It's pretty raw at this stage, and not very flexiable, but I'd be dead keen to try and get a ball rolling amongst the community. I'll keep everyone posted and upload more videos as I improve and develop the system!
Systemoffive 1 year ago
@Systemoffive I've subscribed and set up auto-alerts for when you post. I am into scripting myself and look forward to seeing how you coded it. Keep us updated
RevJackStraw 1 year ago