 So show a hand to here makes money from somehow inside the 3d industry. They got a job at a studio. Yeah, almost everyone who doesn't Okay, what are you guys doing here? No? So I love this job, it's it's really it's fun and that that childlike wonder of like Yeah, bringing something from my head into life is still there. I just I really enjoy it But I want to continue doing this, you know, because it's it's a fun job but if you've paid any attention to the news and you've no doubt seen reports of Automation and AI the doomsday theories that we're all gonna be out of jobs all sitting on the street nothing to do and I always thought like yeah, but that doesn't apply to 3d because like what we're doing is art and computers could never replicate art But then you see a couple of stories like how machine learning was used to create high fidelity animations on the face of Thanos I'm like, okay. All right, and Then you see an algorithm that'll take the styles from famous paintings in history and apply it to a photo And then I'm really sitting up and paying attention because I'm like, okay that I didn't think a computer could do that And so I wonder like what what are we looking at here? Like it it kind of made me wonder if this is you know the old days of the Disney 2d animators being replaced by the 3d animators Are we gonna be replaced? by server racks running AI and machine learning to auto generate the movies So that is what I'm gonna discuss in this presentation. How AI and automation might change the 3d industry Quick side note. I'm gonna be using terms like AI and machine learning And some people are like no, that's not AI. You've got to call it this and For all intents and purposes doesn't really matter for what I'm talking about at the end. It's still the same result, right? It's a software doing something that an artist does so anyways There is one guy who's very good at making predictions for the future the Amazon guy Jeff Bezos Who's you know in hot water at the moment for his treatment of stuff wasn't very good at predicting that but For the last 20 years He has been basically ahead of the industry and so he said I frequently get asked What's gonna change in the next 10 years? I almost never get asked What's it not gonna change in the next 10 years and that's actually the more important of the two and he went on to explain that You know an Amazon customer of the future is never gonna say I wish the prices were higher or I love Amazon, but I wish it took longer for my packages to get here, right? So those two variables he knows for sure are going to be desirable in the future So actually what's not gonna change is what you should be focusing on not worrying about hypotheticals And so I thought that's that's great. It's a great exercise and applying that to 3d. I Think that any technology that makes things better faster or cheaper will eventually become standard It's inevitable things get rolled out once the studio execs realize that this is gonna save the money They put it into practice So what is costing money at the moment games are costing money? So Raph Costa Plotted some games from 1985 to today and he found a disturbing trend that every 10 years the cost of games go up 10 times So this is a log scale. It looks deceiving like it's not that big of an increase But every single one of those horizontal lines there represents a 10x increment on the one before it So basically every year it goes up 25% By 2020 the average triple-a game might cost 200 million dollars So they're already more expensive than feature films And it's getting unwieldy because they all have to spend more to try to outcompete the others and mobile gaming Yeah, it started out cheap But that's happening as well because it's now a saturated space So the costs are way too high and they need to come down a big portion of this costs is assets assets are unreasonably expensive So let's say for example, you're making a video game that involves the street You've got to have characters running down the street for whatever reason. So you got to model a building Okay, and it's a pretty detailed apartment. So you work pretty hard. You might be able to get it done in 12 hours Okay Then you got a texture it texturing often takes just as long as the modeling But let's say you get it done in 10 and then halfway through the the work week You've got it done in 22 hours Which by the way is unreasonable because that assumes you're productive 100% of the time It's usually more like 50% of the time, but let's say you got it 100% productive Then if you're in the studio, there's also a revision multiple of two to four times You know narrative changes. Maybe they want to make it take place in Paris instead of New York Maybe they won't have a tunnel running through the building for whatever reason so changes have to happen repeatedly And they have to go back and revisit old assets So at a cost of $60 per hour the average wage this building ends up being about $3,900 So when you look at a game like the division put price tags on everything And it's very easy to see why games are costing hundreds of millions of dollars It's and and this is kind of this is kind of dumb, right? Because like we're working virtually none of this stuff exists in the real world Like we should be able to approach this smart smarter Lee because I Think that a large portion of this cost Comes to the fact that this is a static workflow You're getting a one-for-one input to output ratio if you want to make another building you often have to repeat the work And sure you might be able to you know start from the base of the other But you often have to do so many details and topology change and then you got to re-texture it So you're still getting a pretty similar similar output so I Think that the first leap that's going to happen to the 3d industry in the next five to ten years is procedural workflows becoming Mainstream or the standard to say So approaching the problem of modeling a building to do it procedurally would look like this Instead of modeling the building yourself You would set the parameters for what a building should look like within certain ranges So a building should be this height to this height it might have between five to ten windows on it two to three doors It could have this many floors in range and then basically you let the software self-generate and This is how you bring those costs down So this was actually a course taught by Anastasia opera. I believe he's actually Dutch could be wrong But she made a procedural lake village using Houdini and she went on to Explain that she thought that proceduralism would really remove the the creative aspect of it But actually it forced her to understand like what makes something look good because you often don't know as an artist You're just like at doing things without thinking about it But when you had to put it into a program you had to explicitly Define what that looks like and so that was a good exercise And then also what you got out of the machine at the other end I was often ideas that you hadn't thought of before so it can actually be a huge advantage So this is procedural modeling obviously you've got to then texture it in this example. It's textured. I'm not sure how she did it but Texturing so I run polygon who here's her to polygon All right, my advertising worked So when we started polygon two years ago every single texture on the website was captured with a camera And we would find a floor like that and we said great We need another floor we would have to find another wooden floor capture it make it seamless process it So it was that one-to-one input to output then we discovered substance designer whereby you would Spend a lot longer Generating a node setup making something 100% procedural meaning that there was no camera was involved whatsoever You would make it digitally, but once you've made it once it's very easy to create variations from that and So immediately we realized the benefit of this not only could we capture things that are very difficult to capture Like marble would otherwise you got to go into people's homes and capture their floors But also it was a huge cost saving so basically that's all we do now We a lot of our materials are now made with substance designer save for photo scanning which are still best captured for for for grounds But yeah, we've doubled the size of our team and game studios have realized this as well Which is why you look at job listings all hiring substance designer artists Then of course you got to apply this to a model, okay? Model will have you know creases and crevices and all sorts of occlusions that you've got to take into account for So our algorithmic was very good by making a sister software substance painter Which enables you to take the thing it'll bake all the maps for you And then it'll apply the smart materials that you got from substance designer smart masks and things to add in grunge And whatnot So it's the number one texturing software in the world right now for this very reason it is saving studios Hundreds of thousands of dollars because it's procedural when you go back and you change something with the model if you've got the pipeline right It should auto update with the texture. So very very cool And then finally you've got level design. So putting the assets into it So I had the the start in New York City example, but let's say you're making a forest, okay? so Far cry 5 You know, they've made a lot of Far Cry's five of them in fact But Previously they went about, you know, like you would make a forest usually like you have particle systems and you place things by hand But the problem was was that as the story would change over the months the landscapes would move around as they had to have roads Go through different places every time they did that they had to update and like hand place and remove all of the trees This time they did it a different way They created an ecosystem So they set rules to define where certain trees and plants would live So big trees would grow towards the center of the forests small trees towards the edges underneath the trees There would be ecosystems of like smaller plants that would grow in the shadows if it was near a lake There was gonna be different plants there It would automatically update according to the altitude. So if it's higher up, there would be smaller trees I know we're able to build in handy tools into that as well to quickly build roads and and buildings and The whole talk is definitely worth watching by the way if you get a chance So this I think is gonna be the future this workflow Procedural modeling materials texturing and wall building already materials and texturing are pretty well embedded into the industry now It's pretty well standard, but I think modeling and wall building are are gonna be Taking the next spot At the moment Houdini seems to be really good at this, but I'm really hoping that that juxt wherever he is Yeah, there he is Hassle him about everything nodes Because I really hope that this can come into blender So that was leap number one was that Leap number two machine creep. Okay, so this is where it starts getting interesting So traditional software typically work like this You have an input See you've got a photograph and you bring it into Photoshop and then you want to apply a filter to it You apply the action to the photo and then you get one output. It's very predictable You know what you're gonna get and also leads to obviously a lot of manual work tweaking things as you need Machine learning is different. You start with an input. It might assess it It might apply an appropriate action and then most importantly compare it with others From its training data set then be able to judge whether or not it is good or bad Sometimes a human has to do that step if it's not try again and this by the way is a very gross Over simplification of machine learning. I'm not a computer scientist. I didn't go to university but this is what's helped me as an artist to understand this this process and Basically, the result that you get out of it is usually multitudes Multitudes better than traditional software So the key point here is that it has the ability to learn and to improve over time Now what it needs in order for that to happen is huge data sets and fast hardware So I think like five years ago. You started, you know Hearing the the train of news about machine learning taking all our jobs and everyone was hyped up and scared and then Nothing really happened And I think a large part of that is because it's not love not enough data and also not fast enough hardware I think now we're reaching the tipping point and actually you're starting to see some consumer software That is making use of machine learning already, which I'll show you in a second So one thing machine learning is very good at Denoising Okay, so everybody's familiar with noise you render something. It's grainy, right? And D noises typically will smooth it out a machine learning denoiser will do it really really well So this is one Which is owned by Nvidia. I assume it's the same one that is used for the RTXs Which is enabling them to do real-time ray tracing that is by the way how their RTX is able to do that It is rendering one sample every single frame of the game and then it is applying a denoiser on the same frame So it's doing this in real time And how on earth it's able to take that blue square That noisy blue square and read anything from it is a testament to how machine learning is is working Like that's just absolutely insane that it's doing that So blenders cycles denoiser as far as I know has nothing to do with machine learning or AI And so it obviously would very much fail in this situation Which is why I really think blender needs to get into the into machine learning and AI Disney and Pixar also own one and they're trying to solve the problem of the frame flicker Because that's a common problem with D noises, but they're also building artist tools to do it Nvidia own most of the papers regarding it And I think that's because they realized there's a huge amount of money in it because not only every renderer will need to use Their denoiser, but also every camera manufacturer because if you've got low light sensitivity and there's all this grain Well, they can solve it with denoises So that's one thing it's good at another thing is uprising So you've got a small image small little jpeg you find you're like, ah, but I need it to be double that size This is uprising. So as a test I took this image, which I'm making at the moment of a kitchen I rendered it at 50% and then using AI gigapixel from Topaz labs. I Uppressed it by 200% so that it is now as if it was rendered at 100% and then I compared it to an actual render at 100% And if you look at it, there's not a lot of difference there And obviously there's a little bit more detail in the one that was actually rendered 100% But not much and certainly not enough to qualify like a four times extra rendering So this was like really caught my eye I was like wow and this is consumer level that you can actually buy this Get the trial is 99 dollars or something like that to purchase the full thing This is already out there and this is a start. I think of of where the industry is going and There's also other uses like for example motion capture So I think we're gonna be doing away very soon with the mo cap suits and the expensive studios Because now you can capture just from raw video. So there's no sensors. This is not a special camera This is just simple video And the algorithm whatever it is I don't know what I'm talking about was able to figure out where the bodies are and most impressively It's able to guess the occluded parts like what's on the other side of that arm because it can't see that but it's guessing And it's doing an amazing job at it. So I think it's in the future. They're not gonna have mo cap suits I'll just film the actors doing whatever and that's gonna be it And then there's this other one which to be honest I don't know how it works But they filmed the dog in a mo cap suit for an hour doing running and jumping and then using neural networks Whatever that is They translated that into player motion in a game and what's most impressive is its transition from one behavior to the next going from running To walking to jumping is incredibly seamless. There's also very little foot sliding and There's also another example with a human like and it adapts to the terrain and it jumps over things I think this is inevitable. I think you're gonna start seeing it in games So that's machine-learning basically We're just gonna start seeing it in our software in the future You'll start, you know, you'll have an action in Photoshop and you'll be like, oh, that looks really good You find out it was machine-learning built into Photoshop premiere We most Autodesk products. I think it's really I think blender needs to really get into it I think it's the future and I think every every Silicon Valley company has realized it and yeah There's this quote by the guys who made the Thanos facial animation if you're not using machine-learning in your software You're doing it wrong. I think that's very true So elite number three is machine-assisted creativity This is really fun because this is something that I always thought that you know computers would never be able to do like Creativity like that's something that is very human and it's true like intent is very hard for a computer to do But you would be surprised at what it can actually do to help you, okay So as an example as I mentioned, I'm working on this kitchen scene Which I wished I could have got the tutorial app before I came to Amsterdam, but I ran out of time So it's gonna take another month But I'm working on this scene. I you know whenever I'm making something I reach a certain stage like this where it's okay, but I know it could be better if I tried more ideas So you just start throwing stuff at the scene and seeing if it looks better than what it did previously So you might change the lighting add in some little blinds and might be a marble on the left You know and then you try it with like adding food to the table No food adding some boxes no boxes a refrigerator instead And you just have to keep doing this and what I've realized that as my workflow This really eats up about 50 to 70 percent of the production time Because and there's a bunch that I haven't shown you like 20 other renders Where you just you've got to try this stuff and it takes so much time Sometimes you have to model the thing like was it gonna look good with a wine bottle I have to model a wine bottle or you purchase it or whatever Then you place it and then you have to render it and that is a really really long process So if there was software that could do this for you without you having to go through all the effort To add it in there in order for you to make the decision if this is good or not That would be incredibly powerful, right? So there is hey So this is a really impressive paper You give it an outline of an object the photo that the outline was based on and then it will generate a bunch of ideas for you and It works really really really well So it actually helps to see it as it's generating. So this is a building facade The only thing that was given was the top row and then this is the shoes And if you paused it at any point in this any one of these frames here could be a unique design So I think that this is kind of the future I think what we're gonna see is you'll make a character for example And then you'll put it into the software and you'll try out a bunch of ideas different clothing different types This is one used for an environment different sceneries And what's crazy is that the input that's given to it is so simple And yet it's just churning through ideas and spitting it out And I think this is absolutely gonna be part of a lot of creative meetings in the future And this is actually one you can try online if you go to that web address there It's based off a different paper technology, but very similar thing. So I drew this cat right very simple looking cat And then I hit a button It reminds me of those things where it's like, you know like the dad finishes the children's drawings And he's like a really good artist and he just makes it look high fidelity Obviously, it doesn't look very good. That's because my drawing is crap But you could obviously see like this is it's gonna be a finished product But this is definitely be a starting off point for a lot of concept artists So I think this is it's a sign of the times we're gonna start seeing this Speaking of which does anyone recognize these celebrities Does anyone know who they are? You want to throw a name at me any name you think it is They look like you they look familiar, right? You're like, is that like a footballer pretty sure I've seen her in something either she's a musician or an actor Well, I see they're imaginary. They don't exist. They were generated This is a separate paper whereby you feed it a bunch of images and it spits out a result Really really impressive. So obviously real estate examples You could get by you know if you've got like a game and you need to have 20 different NPCs And I don't know you just pull photos of people that match the region that you're creating the game for like I need 20 NPC Mongolians just download a data set of Mongolian faces. I mean you just got unique faces There's no like likeness Infringement or whatever you've just got uniqueness, but it also worked for other things I don't know how it did this because these photos have perspective in them But this was also generated these these bedrooms there, which is bananas That's exactly what I mean for my kitchen, right? If you could just put it in and then generate a bunch of things So I think that's what's gonna happen He's gonna have your mood board of all your ideas for your environments, and it's just gonna generate ideas or And this is where it gets really freaky real spooky for Halloween What if you didn't even have to provide it with images What if you just typed out your idea like this bird is red and brown in color with stubby beak That's real. That's not it's not pulling from a stock library by the way that was generated So Amazingly the way this works is that it was trained to recognize what the features are and what something is like red bird Okay, this is what a red bird looks like then when it notice when you type that out It creates the blob of whatever that shape should look like and then there's a second pass where it adds in the detail to that And it's crazy like this is like the closest thing to like sorcery I don't understand it. In fact like a lot of papers. I've seen I'm like very skeptical. I'm like, they're cheating There's something they're not saying because this can't be true But if it is if it's as good as this is and it's like today, I think this was actually from 2016 I think this is inevitable. You're gonna have a creative meeting with a director who's like, all right Let's let's create some ideas ancient city Road through the middle Canyon in the background spit out like 20 different ideas And then one of those ideas you give to your one concept artist To to flesh it out. I don't know. I think it's inevitable, right? And then another really interesting one is style transfer. Has anyone seen these by the way style transfer? Basically, you feed it a bunch of images that you like Like I want this style from Claude Monet, whatever and then you put an input in this is my photo I took on my holiday and then it spits it out Which is just crazy and It also works for video, which is really cool as well If you look at the paper they filmed like in I don't know somewhere in Germany people walking down the street And it's a painting but every frame is moving and it's so weird and What's most crazy of all is that it actually fooled 39% of art historians into thinking that they were looking at something That was actually painted not computer generated. I couldn't think of anyone more wanting the machine to fail than an art historian But it worked right you worked So my prediction is that artists will use machine learning to explore new ideas I think that's just gonna be the future. It's just gonna be part of our pipeline It'll first of all start going to studios and it's gonna be in the hands of consumers and we're all just gonna go like Yeah, how I made this scene first of all, I generated a bunch of ideas. I Think that's how it's gonna work. So these are the Expected changes that I predict will happen within the next five years. I think procedural workflows will become standard I mean texturing is definitely already there with substance designer and substance painter But I think it's gonna happen without level design as well as modeling Machine learning is just gonna slowly creep its way into all the software that we use and it's gonna do a lot of really tedious Technical things and I think we're gonna get creative assistance from machines as well So I can feel the mood in the room. It's a little like, oh, no What is this? Are we being replaced? so people had this thought actually in 1997 When Casparov was beaten by the IBM Deep Machine, that's what that's what it was called And it was the first time that a computer had ever beaten a human and people thought like that's it for chess And it might be the end of humanity too It was depressing and they thought like chess is dead and Then Casparov realized that like, you know, if the computer has access to all, you know Potential moves of all the previous of hundreds of thousands of games and it's able to work from that It makes it should be fair if the human also has access to that So he created something called advanced chess Whereby the human player has access to the same information the computer has and he will listen to the computer and occasionally Overwrite it just like you would as EPS navigation when you know something is wrong based on what the computer is giving you and Today the best player is A human to machine counterpart a purely AI chess will win in a tournament in 142 games Whereas a human and machine 152 So there is there is a purpose to your life. It's not just all machines And what's more so is that since then the number of grandmasters in the world has doubled So it wasn't the end of chess. It was good for chess So I think it's the exact same thing is going to apply to our industry. I don't think that these new technologies I don't think it's going to it's not going to make artists redundant because you know You give this to a monkey is not going to be able to make anything good out of it It's gonna be hit hit a few buttons and yeah It might generate some semblance of an image, but it's doing is the human to put intent into it so I Feel like the based on these these learnings the most at-risk jobs are those that are labor-intensive Narrow-skilled and repetitive the great jobs like mo cap cleanup rotoscoping retopo mesh cleanup make 100s of these jobs basically the kind of thing which is usually outsourced to a I don't know a floor of workers in India where they've been told to rotoscope these shots from the upcoming Hollywood film Where you just try to just throw manual labor at it. I think that's something which will definitely be replaced The safe jobs are those that involve critical thinking wide-skilled tasks where you know how to use Different softwares and different solutions in the right way and niche tasks which are too or too small to bother automating So our direction project management good generalists Program is obviously to oversee a lot of the scripts and the programs people are using and freelancers because they will always be a Market for people who don't want to use any software whatsoever and they want to pay you instead So really the the at-risk jobs are the undesirable grunt tasks It's the stuff that nobody wants to do but we have to because it's part of the job The actual art I think that's safe And failing that if that itself, you know wasn't enough to make you feel better The the forecasts for the 3d industry is insane. It is blowing up For almost every single one of these sectors you're expecting to see huge amounts of growth over the next five to ten years And that's not by the way even taking into account VR One report said that 3d rendering and visualization is expecting to see twenty five point five percent growth per year until 2025 compound annual growth or Basically double by 2022 and quadruple by 2025 So the number of people at this conference in four years from now could be double that by 2025 quadruple that I think the 3d industry is is poised to blow up So yeah, maybe each studio there would be a reduction of the grunt work The the juniors that they're hiring to do the stuff that nobody actually enjoys anyway But in its place there might be five studios around it so That is it that was that you climatic wasn't it thanks guys