 They also say, oh no, you have a camera movement so you cannot use mid-painting. Or they say, oh no, this shot is in the stereo so you cannot use mid-painting. And these people are wrong. And what I will try to show you today is that with camera mapping and multiple projections, you can pretty much cover background elements, mid-ground elements, and even foreground elements easily and with photo realism. Other people say that our mid-painting is cheating. It's not real trading, so you are cheating. And these people is absolutely right. We cheat pretty much all the time. And you know what, visual effects, since the very beginning, 100 years ago, it's all about that. It's all about tricking the eyes of the audience. So mid-painters and the compositor guys, I don't know if Sebastian is here or Bartek is here. If there are any other compositor here in this room, there is. Yes, you guys. So we are the cheating guys of the production and we do this proudly because we just want, it's not personal, we just want to have the shot down. We just want to have time to have some, a couple of Belgian beers in the end of the day. So we cheat a lot proudly. So saying that, I will try to show you some difference between the two approach, the mid-painting approach and the full-trade approach, both are used a lot in production and they are not separated entities that they mix together. And then we've seen this difference, we can figure out together what are the pros and cons of each approach. Both use the same tools and have the same objective. That is to make elements photo-realistic and integrated them within a shot. And in full-trade, you start building full assets. So you model it, you shade it, you texture it and then you distribute these full assets in a scene and then you create an almost entire environment. In mid-painting, you create only what you see. You only take care of the elements that actually face your camera. So we create only what you see. In full-trade, it takes longer to reach a photo-real look. I'm pretty sure that there are some artists here in this room that can do it, that can reach a photo-real result in full-trade, in in-blender. It's not that you cannot do, it just takes longer and sometimes takes more people. We have the modeling guy, the texturing guy. And in mid-painting, as it is based on photos and photos are already photo-realistic, if you have a good eye for integrating them, you get faster to what makes a photo-real image. Things like values, relations between elements, photo-real textures, contrast, perspective, error perspective, you can tweak this faster. But in full-trade, you just point your CG camera anywhere and click render and you wait a lot because render takes longer or you submit your work to our friends on that render street. They are great guys, I don't know if they're here. Yeah, thank you very much. And this is great because what you create becomes independent. You can use it across multiple shots or across multiple sequences, so this is great. In the other hand, mid-painting is extremely shot-specific, shot-oriented. You see your shot, you plan what you need to do and you put effort on that specific area of that specific shot. Of course, you can use something that you painted for one shot in another shot, but this is not the idea of mid-painting. So, mid-painting changes and turnovers can be faster. Like, okay, change this mounting. You just grab another mounting and color correct it and integrate this and it's done. But if your camera movement changes drastically, if your director gets crazy, oh no, I want another camera, you get killed, simple like that. In full-treaded, the change in turnovers takes longer because you have to tweak everything, pretty much everything, but you hardly get killed. So, if you can compare the both methods, I could say that full-treaded is like a big gun, a big cannon, a bazooka, that it's harder but it's effective. And mid-painting is more like a small knife, like an opineo knife that with a carbon blade that you calculate and you see what needs to be done and you go there and put effort on that specific part and you get your shot down and you get your Belgian beers. So, let's cook some mid-painting. This is the basic recipe, the basic formula. If you follow this formula, everything will be all right. No, it's not like that. And like in every CG area, there's no formula. You have a lot of ways to do stuff and this is even true with mid-painting as it is shot specific. So, you have to take your shot, see it, planning, the first step, planning, planning, plus planning and see what needs to be done. And when I say planning, I say real planning. So, build an animatic. Build an animatic with simple primitive cubes and see if your idea is working because if your idea is working with simple cubes, it's obviously that if you put effort on top of it, at the end, it will still work and now beautifully. But if your animatic doesn't work or if you don't do an animatic, it doesn't matter if you have 15 artists from Industrial Light and Magic working for you, at the end, it will still not work. So, plan your shot and build an animatic. After you're building an animatic and do your planning, you will see that mid-painting is always a dance between building geometry and painting on top of it or having something already painted or a photo and creating simple geometry just to project on top of it. You're always doing, dancing with this, doing this to girls and the party. So, after you have modeled your geometry and you have your camera movement from your animatic, you define the hero frames of your shot. And what are the hero frames? Are the frames that are the moment in the camera movement that your camera covers most of your scene? So, you go to your camera movement and say, okay, I'm gonna paint this computer. So, this frame here is my hero frame. It's the frame that covers most of my scene. If you have a simple camera movement or if you have no movement in your camera, you will have only one hero frame. But once your movement starts to get complicated, just you decide on the side, you have more and more hero frames. So, with this hero frames, you duplicate your camera and you define your projection cameras. Okay, I'm gonna have a projection camera here and a projection camera here. So, you go through this projection camera and you render out a clay render and several passes that will help your painting process. And then, you go to an image editor, a game or Photoshop, and you start painting on top of it and you start dropping some photos, especially photos on top of it and you start cheating and you try to make this photo real. This is where the magic happens for Mac painters. So, you try to make an image that is photo realistic. So, once you're done with your painting, you project this painting back onto your geometry from your projection cameras and you fix issues that sometimes appear when you connect to projections or when your projection don't cover a part of your object. And then, you render out your animation with shadeless materials because now you have to be fast and you don't need more delight in passes and the physically accurate stuff because everything is bacon on your painting. And you render the separate layers for any compositor. Here, you can composite the shot and then you deliver the shot and go for your Belgian beers. So, the keyster that I will show you today is a documentary that I worked for for History Channel of Latin America. It's Latin America. It's a documentary about the construction of the Panama's channel. And I was responsible for do some matte paintings, some sky replacements, some set extensions. But there was one shot that I was responsible from the beginning to deliver it to a compositor. So, and this one is what I will show you. My briefing. In the highest part of the channel, there is a big lake and in this lake, there is a dam that controls the level of the lake. And the objective was to show that after a big earthquake, the demos is too solid and it's still holding the immensity of Gatton's lake. So, discussing with the director and with the supervisor and building an animatic, we find out that the best way to do this is was to do an upwards traveling shot that began showing the dam from below and showing the intensity of the lake there. So, a second step after planning and building an animatic was to get some reference images. For a mate painter or for any artist, never is too much to have a lot of reference images. If you can spend one day, two days, one week searching for reference and you Google translate the search words for another languages and you search in Portuguese, in Spanish, in English, in Russian and you get a bunch of reference images. Next step, modeling. I modeled based on Google Earth photo and also when I was searching for reference, I found this project. This is the actual project of the dam from 100 years ago. And it's important if you're gonna project stuff that you model it kind of based on the reality, like respecting the scale and the position of stuff because if you just do looking to your hero shot, for example, to your hero friend, for example, you can place things out of position and when your camera movement raw, it gets distorted. So I modeled it. This is my rough model of the terrain and the dam. And with this, we imported the camera from the animatic and we tweak it a little bit just to show things better and we could define the camera movement. So the upward traveling and we could define the duration of the shot that would be 800 frames, a lot for a shot, but okay. And we could define the hero frames that in this case, it's here, the first frame and the last frame. So we're gonna have two projections and we're gonna have probably some patches to fix, some issues that will appear. So first step, we go through our projection camera, see our first hero frame that we're seeing here and we render out the clay render with the passes. You use cycles for that because cycle is a physically render accurate render and it gives you great shadow passes, realistic shadows and realistic lighting. So you render this out without materials, just a clay render, but with good lighting. So you have your shadows and your lights nicely. So, and you also export this as an EXR and you export several passes that will help you to paint. So I have my clay passes here, some diffuse passes, some reflection passes for things that reflect, of course, for the water and the metal parts and the fence on top. I have some occlusion passes, shadow passes and also notice that things that overlap on the camera movement, objects that overlap, I export the separate like the fence and these blocks here and the walls and the grass because I need to paint them separately and then project them separately in order to have the appear right when they overlap on the camera. Next step, it's not the point of the presentation to see my painting process. So I did a quick time lapse of painting this. I use Photoshop. It's important to work non-destructively with your sources. I mean, if you're gonna erase something, use a mask, you don't erase if you want to. Oh, can you wait in the light please? Oh, it's just a quick time lapse. It's important to work non-destructively. If you want to erase something, you use a mask, you don't erase, if you want to color correct something, you use adjustment layers. You just don't change the color because if you want to come back, it's good to have it. And it's important to work in the maximum depth of color you have. So I'm working on painting on 60 bits. So, I'm working on painting on 60 bits. So, when I finish it, I come up with this image here. This is my final mate painting for the hero frame number one. This is my first projection. This is not the point of this presentation to judge the quality of this image. I recognize that a senior mate painter or a better mate painter will reach this point with a better image, but this is what I did within my deadline. So, I did this also for the hero frame number two, but in this case, I didn't need shadows and light here. I just need a reference to paint on top. So, I got an open GR render, super fast. And I paint on top of it the last of the landscape with the water and the forest, and the sky is the same on the first painting. And notice that I didn't paint that dam again because the dam will be projected on the first frame, so he will be here anyway. So, second next step, export your mate painting into projection layers. I mean, everything that's overlapped, everything that needs parallax will be exported in a separate layer. So, as part of the sky, this cabinet's here, the dam itself, it was just in one projection layer, the columns and the towers here, the fence, the antenna, the walls, the grass, the power plant, there is a big house, there is a left, and the water. And for the second position as well, the line of forest number two, line of forest number one, the water, and the sky is the same for the water. Ah, the water, this is nice. The water is actually footage that I shoot. I shoot real ocean near my house in Brazil, and then I stabilize it and I compose it on top of my painting, and then I export it with a great alpha, an animated texture, and Blender deals really well with animated texture, so I projected this image sequence where it's water. So, let's back to Blender and set our projections. First, you duplicate your scene, because you have a modeling scene, now you're gonna have a projection scene. Then you turn to Blender internal, Blender render, Blender internal render, because now everything is baked on your painting, so you want to be faster on Blender meters, so Blender internal. Then you apply all your modifiers on your geometry, and you can join your objects, objects that belong to the same projection layer, like the dam in this case, the columns, the bridge here, the wall, I imagine in the same logic, you can do it or not, but I do this because it gets more organized. And then you go through your projection camera in the hero frame, you see in your geometry, and you go to edit mode, super straight, press U, and then UV or wrap using project from view. And then your object will have the UVs prepared to receive a projection. You can use the UV projector, yes you can, especially if you want to tweak your geometry on the go, and you can use project from bounds, there are things this year, but when you use project from bounds, I think it scales the textures for the maximum UV space, I don't know how it works, but I get some issues. So I use project from view, it's straightforward, and if you want to change something, you just change it and go again, you project from view again, so it's projected from that camera. And then you create a shaderless material, you have everything baked, so it's shadeless, just check here and forget about any setting on the material, it's shadeless, and then you go to the image texture tab and you assign a new texture, what texture is this, we're talking about the dam, it's the dam texture. Remember to use alpha, because you need the alpha, and you set the mapping coordinates to the UV map you've just created, use a consistent name, not UV map, just like hero projection or something like that. And then you repeat this for all your projection layers, for all your objects on the scene, and then you will have set your first projection. And then you check your projections in your shot, ah, this is actually a print screen from my viewport from OpenGL, and it looks great, it can also have been used for a shot if you want to. So you check your projections and you see that in some areas there will be issues, like here where you have this area here that like a leak from the back, and you have some issues here, this I think is the 512, but you have some issues, and you have to fix these issues. Why these issues occur, because of the nature of the geometry, there are some hidden areas here, like you have your camera here and this creates a projection cone, and there are some areas of your geometry that don't catch projection. What they catch is this little thing, line of pixels. So this area here you have to fix. So how you fix it, or you're doing matte painting, you will fix it doing matte painting. So you keep going to your camera and you find a hero patch frame, I mean a frame that covers most of your issues. You can even push back your camera a little bit or you can even wide open the field of view just to cover more area, but be consistent, don't put these five kilometers away in paint because you're gonna have a resolution problem. So you find a hero patch frame and you render out a frame from this camera. And what you render will be something like that. You see the issue here and here and here, and then you go back to your image editor, create a new layer and paint on top of that. Correct this, and you can clone it or you can bring new textures or whatever and you fix it and once you fix it, you're gonna have something like this in a new layer and you export this as a new layer and you go back to your hero patch camera and project this image again on top of your geometry. You set a new projection on top of the same geometry and this is the trick. So to do this, you go again to your object, you go to this tab here, I don't know the name, it looks like a car face and you create a new UV map. And once you create a new UV map, you name it properly, okay this will be my patch frame and then you go again to your frame, to your camera, project from here again, add a new texture that will be your patch on the frame and select that UV map that you created and boom, you have two projections in the same object and your ratio is corrected. So what you have just done here, two projections, is a concept that once you understand that you can have like an object or an environment and you can cover it with several cameras and you can paint a matte painting or a photo and project it on top of it. I mean, having multiple cameras projecting different matte paintings or photos into different UV maps, you can cover pretty much background elements, mid-ground elements or photo elements using mid-painting. So we did the same for the second projection and finally, we render out our animation with a super optimized render. We just need the textures and good anti-allies. You can use motion blur here or you can pass to the compositor as a vector or you can do both. You can render a version with motion blur and another version without motion blur mix them, see what fits better and you'll render also separate by layers because the compositor need to tweak this after all and we come out with something like this if you can dim the light again. Oh, I can see. This is not the final shot but this is what I delivered for the compositor and this is pretty much an animation separated in layers in 16 bits. So the compositor could integrate this into the show. So that's it for this shot. Another quick example, this was a TV commercial for a brand for a Brazilian shoes brand and they did a collection with the sir here, Mr. Karl Lagerfeld and of course, Karl Lagerfeld didn't show up in Brazil to shot the commercial so they licensed this photo, oh, she can't do it again, sorry. That's a good note. We had this picture here and what I did was to create a simple cube here and a simple cube here and I did a quick sculpt, really quick sculpt with how to say, dynamic topology which is great and fast and your mesh don't get so heavy and you know there are some parts that are not correct, his chin is banded but you know he has chin banding and it works from my camera point of view so it's great and then I create a little synth setup and project my texture and you can see here there are some issues on the side but there's no problem because in my camera angle it works so here is the final shot and another quick example this was a personal project as well this is a painting from Mr. James Garnay this is in the book Dynotopia or Dynotopia, I don't know if you know this is my book and I wanted to turn this into a shot so I extended a little bit the concept here I start modeling some geometry for the gate I use it as sphere for the building behind and then I use some planes behind for the buildings I paint it on top of it with textures use it as food for G3 and put some bolts and stuff like that I've defined my hero frames hero frame number two the end of the shot hero frame number one on top and then I composite everything this ocean is blender ocean simulation looks really nice some lighting passes real people composite on top and this is the shot that's me so that's it thank you blender for being such a nice piece of software and more than this for all the philosophy you have behind you and I think we can change the world with blender so thank you very much if you have any questions please yes sometimes I just if it is easy or I don't know I project two textures and I just rotate my model and I can clone with blender texture and painting tools it's super easy yes I do this often yes definitely you can do it and once we have the ability to I think we have already the ability to use layers painting on layers and use color modes like multiplies yes and yes there's a lot of new stuff that I haven't already test I must confess that I cannot use blender in all of my work because there are many studios that just left about using blender and you cannot break their pipeline but I try to use it and I know that there are new features now that will help to improve the use of it one of the features I need I want to test is the FBX export because it's very important to export this geometry to Nuke or to other software and it must be straight okay export to FBX and you have to open this in Nuke and your axis cannot be twisted and stuff like that that happens today so I must test this and I hope this improves please on what sorry on your table yeah yeah no I make it I make it concrete textures or any material textures with photos and I try to take photos as well to provide the reference because it's better and also reference in 16 bit format and that you can have all the range and stuff like that but you can make it yeah and it's a problem if you are mixing a texture 8 bit with an image with 16 bit but normally you must take care of the sky and places that have lower values or higher values because this is what will give you trouble after I have a library and I think every blender should build your own library of stuff and sometimes you buy stuff you buy photos and sometimes you go to the internet and ask permission for using it yeah but I am building my own library I like to especially skies it's good to have some photos in HDR because you can mix clouds and mix stuff so it's good to have a good library of skies and forests or CDs stuff like that you can go and you can make a lot of photographs yeah and also know a lot of photography allows you to know about lenses about composition and stuff like that so it's important to be a good photographer so I think in my opinion you have an address? yeah this is most of my job you know, relighting stuff and it's a not complicated process you put an adjustment layer on top of everything and you build a mask on top of the shadow for example and then you color correct only that area so it matches with the unlit scene so that's why when you took reference photos you must choose a cloudy day with the unlit light so you don't have hard shadows and hard lights and for the sunny parts it's the same, you build a mask and you color correct just on that mask and this can take consume some time yeah yes, yes, yes yes, in Blender especially in Nuke you can work with a plane with four vertex but in Blender you have to super divide it especially in the corners you have some weird issues so you have to super divide it, yes but as I said you're always dancing between painting and modeling so sometimes if things are not fitting you push something or your vertex or geometry and you should divide only that part and you try to do, but yes in Blender you have to sub-divide a little bit your but this is good because the texture sticks pretty well on the vertex and you have ways to tweak things on the go please yeah, on my composition I just drop it on the top but this is problem for compositors but yes I have issues but it's easy to get rid of it you have to export your images with correct alpha and usually I export the teeth with 16 bits and they are exported on a dark background or a white background depending on what you need and the compositors must get rid of the permutiply yes definitely, if you see the edges of my dam there is like a bright and yes in the final shot it was so good yes I have tried Krita and I have tried Gimp I think they are both tools but Photoshop for me is like a glove for me the same I can say about Blender the interface makes my life so much easier I can model stuff really really fast and I saw my colleagues at work blaming Maya for the viewport that doesn't work for the extrude tool or anything but I don't see them blaming Photoshop and Nuke these two softwares are really solid and works very very well yeah, I can meet you, meet you sometimes I think everybody can... yes, that's what I said I think Nuke is solid and I use Blender Compositor as I said to deliver my shot just to see how things work quickly but once you start to get getting a lot of higher resolution 4K footage or 5K footage and a lot of nodes start to get slow and Nuke is super solid and also the studios have their own pipeline they have scripts running with F-Track and running together with Nuke running together with 3D and one thing has to feed another so it's very difficult for them not difficult but you know what everybody lives immersed into a commercial paradigm and with studios it's not different and it's very hard to make them see beyond but it's our job to change this especially the artist yeah well, thank you very much