 Okay, so I think we can start now Thank you everybody for coming to my talk. I'm gonna be talking about a project that I've been working on for about a year now Which is the grease pencil 3 project? I think we're gonna have some time at the end. So if if you have some questions that come up throughout the talk Then wait till the end and I can mean maybe answer them All right, so Going back one year to the beacon 2022 there was a meeting about grease pencil and the future of grease pencil and We were just like setting up the bi-weekly meetings, which we're still doing today and This was I think sort of for me personally where the idea that I Could be working on this like would solidify itself. So This was sort of for me the the beginning of grease pencil 3 back then Of course, I was still working at spa so At spa we were really trying to To push things to use grease pencil in the pipeline for layout for storyboarding and even for animation and We tried but we struggled and We had to modify blender To actually get something that the animators were satisfied with And and we presented that at talk last year at the conference with yarn and Yeah, so if you if you've seen that talk it was like, you know, we had these slides like complicated stuff going on and And it was basically this right it was like a waving our hands and and and You know talking about the struggles of trying to make things work And so I think this brings me to the the why why would we need a new grease pencil? and it's it's it's really about Some like bigger things like there's some major performance issues when you try to scale things up to a production scale you run into performance issues memory issues, etc and So that was the the first motivator to say, okay, let's let's start something new and Grease pencil sort of lives a bit in its own world inside of blender It has its own modifier stack. It has its own materials. It has its own render engine I mean, it's really in many cases separate from the rest of blender. So trying to integrate it more as like, you know Into the whole system of of blender is another motivator and then well, I mean, you know Development for grease pencil started like 15 years ago in like 2008 Back then of course it was still the annotation tool, but Throughout the years, you know that we sort of accumulated hacks upon hacks and and so that the technical depth just kept increasing so To summarize we wanted a new foundation for the future of grease pencil and to really Yeah Make it production, you know as a production scale Usable At the end of 2022 I came up with a proposal on how to do that the details don't really matter, but I sort of sketched up something and and and proposed some ideas to the developers on how we could do it and then 2023 started Spa was sort of thing of the past then unfortunately and we Began finalizing the the proposal. So then I was working for blender. I I came for one week for sort of mini workshop and We had multiple meetings on trying to figure out, okay What are sort of limitations that we see with the design and iterating on it and and solving all the issues? That we could foresee and so we came up with a proposal with with all the admins that everybody was happy with and that's When the project started so March 2023 and and we published a blog post about announcing it that this was gonna be something of course we Hey, it wasn't gonna come to 4.0. That's well, that's how things go So yeah, let's let's talk about three pencil three. So This is a technical talk. So I will talk about some technical things Starting with with the core data structure and how it changed from grease pencil to to grease pencil three I want to talk a bit about attributes and a blizzard sharing I know that was already a topic in a talk yesterday here, but I'm gonna look at it from sort of the grease pencil side of things I Want to talk about a drawing tool, which I guess is now sort of a tradition since I talked about already in the past talks And I have a I have a short video demo where we are at today with grease pencil three Of course, it's far from done, but you're gonna see that All right, so let's get started. Oh Yeah, so this is basically how the project is Being developed we started with the core data structure that was in March And then we sort of start building like a high-level API around the data structure so that you could do things like Add a layer, right? So this is like for the program is you know, give me a new layer or add a key frame Create an empty drawing things like that And then after that We moved on to High-level stuff Start to implement the operators report them implement tools like the drawing tool like the eraser Worked on modifiers just a few weeks ago. We added Geometry nodes support which is exciting and of course rendering Right, but let's let's start talking about the data structure so To remind ourselves and how this looked in the past let's start there. How does it look right now in increase pencil and We start with this linked list of layers. So this is your layer stack here It's going left to right, but really it's going like bottom to top and Then inside of each layer we have a bit of data like its name like its opacity things like that and Then every layer has this linked list of frames. So a frame is Is if you imagine your dope sheet, right? You have your key frames and basically every one of those key frames is a frame And so yeah, you just imagine them in chronological order and Then each frame contains also some data it knows about its frame number it knows about its key frame type, etc And then each frame contains a linked list of all the strokes So that is your drawing basically all of the strokes linked together Not visually linked together just in the day structure and then each stroke Contains Also some data like it knows about its material knows whether or not it's sick click or not And it has an area of points and each point Contains a position of radius opacity, etc And those if you imagine the points laid out a memory that's similar to also talk We saw earlier where each struct is sort of placed After one another in the in the memory So you have position radius, etc. And then again position radius, etc. All spread out Throughout the array So this is it. This is basically what grease pencil internally looks like right now So going to grease pencil 3 we still have this layer stack Going again from bottom to top I'm simplifying things a little bit here because we also now have groups where you can group layers together But really they don't change the way the the layer stack work They're just like a way to organize things in the UI essentially So yeah, we still have that and now each layer contains again some data like its name Has a selection, etc But it now has a sort of more specialized data structure to represent the frames where I'm representing it here is like a like a map. So you have your Scene frame and then those scene frames map to An index of a drawing. They're not actually containing any drawings. They're just mapping Into an array of drawings Using the index So this is what this would look like and the drawing array is at the object data level So it's at the same level as the layers They're not contained within each other Yeah, I'm gonna talk about some of the stuff that this implies afterwards, but what is a drawing so a drawing now is two spreadsheets and something an offset array So every Everything data related inside of the drawing is now stored in attributes and we have this these two what we call domains the point domain the curve domain and You can imagine a spreadsheet literally for each of them where every row is one of the elements So every row in the points domain is a point And every column is one of the attributes. So you have a column for the positions the radius, etc it also now means that we can have custom data on your drawing on the point domain on the curve domain, etc and I'm gonna talk about drawings more in the next slide, but we also have a layer domain now Which is which is interesting means that you can store attributes on layers And also custom attributes Yeah, I'm just having an example with the opacity and maybe a tint or you know, you can imagine plenty of things But let's talk about drawings a bit more and I want to look at So if the concrete example in the code the drawing is basically a curved geometry for any developer in the room So this is the same data structure that's actually used for hair curves now to sort of new Data structure to represent curves So let's look at an example This is a beautiful drawing With just two strokes and a couple of points and This is what the actual sort of spreadsheet of that Would look like so we have eight points in total right zero to seven With a bunch of positions. I didn't put the positions, but you could imagine there were you know positions And then the radii so if you look we have the first point here I Guess the second point is a bit, you know, it's a bit larger So it has a radius of 1.5 and this one is even bigger as a radius of 2.5 Just to illustrate like how these rows and columns correlate to what you see on screen So these are the points, but what about the curves so this is all you need to store for the curves So we have two curves zero and one One is not cyclic this one. The other one is the end is connected and The way we know basically which points correlate to which curves is using this offset array It's not an attribute because it has one more element which is the total number of the points But to illustrate what this means so the zero here means that the curve zero starts at point zero which is here and Curve one starts at point one at point four. Sorry, which is here And so then it goes five six seven and because it's cyclic Now if we were to change let's say the offset four two three and this is the drawing we'll get Right, so now first curve second curve starts at three And goes four five six seven and because it's cyclic back to three Right, so I hope that illustrates well how drawing is structured Going back to this so One more thing about these these mappings Because we're now mapping using indices. It means that we can do things like this Where suddenly another keyframe later in the timeline maps to the same drawing, right? So you can have sort of linked keyframes where You're editing one, but you're changing the other as well, right? So you imagine like making a walk cycle and just Duplicating the keyframes, but linking them so that you don't have to redraw the drawings that you're changing So that's one of the benefits of using this kind of data structure. Cool Right So I want to talk a bit about these attributes and Implicit sharing again something that we talked about yesterday, but What is implicit sharing and and why why is it so important? Basically the thing that Jan and I talked about last year the conference does you know trying to Minimize copies essentially right using this update cache thing. This is a similar Like it's a different solution to the same problem essentially, which is you want to avoid copying data So if you imagine You have original data, so you have an object like a mesh and We have this great concept in blender that you can do things procedural to an object, right? You can modify its data without actually destroying the original one So you work in an unconstructive way and we do this by making a copy So instead of acting on the original data, you do change just the copy of the data so that you don't destroy the original one You get it Now if you do this copy it can be quite as expensive right if you have billions of polygons points then This is gonna take a long time. So you want to avoid that as much as you can And the idea with implicit sharing is that well when you make a copy you then don't actually copy the data You just say oh, I'm I'm just Sharing the data. I'm just you know using the same data And then you might say well, okay now, but if I do my modify and all this evaluated object I'm gonna destroy the original object But the point is that you only do the copy when you actually modify so the modifier when it tries to Change the position. So example, it would then do the copy It so seems like at first we're sort of back to the original play we still made the copy. Where's the difference? But for one if you don't have a modifier, right, then we're here. So you didn't do the actual copy, which is great, you know, you sort of You can skip the copying and the other thing is that We're not actually like because here. I'm just you know, I just have this generalized data but in actual in actual facts You have these attributes, right? So here you have a drawing for example that you want to evaluate and Let's say you have some some sort of deform modifier and it it deforms the positions Well with implicit sharing we actually just need to copy the positions and then to form them in the copy And we don't actually need to copy any of the other attributes at all. We can just use the original ones So it saves a bunch of copying and yeah, that's that's basically the whole idea and Just to clarify this applies to the curves geometry, so it applies to grease pencil 3 now which we weren't able to do before Current grease pencil data structure All right, let's talk about drawing so Last year I talked about the drawing tool that I worked for Spa and We're starting sort of going back and forth with the animators and trying to find something that feels good, right? I used this term in the last talk as well Trying to make a drawing tool that feels good, which is really difficult. It turns out So when I started to work on a drawing tool for grease pencil 3 I sort of wanted to Do it right from the beginning and sort of learn from my mistakes from what I did at spa and Make it better. One of the things That I wasn't really satisfied with is that at spa the drawing tool The act of smoothing would smooth the whole curve while you draw when really you only need to smooth Sort of this this active window of points because at some point the points are not going to change anymore and so the the algorithm that I developed for the new drawing tool it Does a bunch of curve fittings so it tries to fit the points that you draw to like a bezier curve or B spline and It does that for every new sample that you add it does a new fitting and remembers those fittings and it then Let's the points basically converge and once they converge below a certain threshold like let's say point one of a pixel We then drop them out of this smoothing window, so then you get sort of this effect where Here the dotted line is like the the active smoothing window, but as the points converge They are no longer affected by the smoothing And I have a video here demonstrating this so what you're going to see is me just doing a bunch of scribbles and I'm going to visualize the active smoothing window using vertex colors, so it's going to be in red and It's very subtle like it's it's hard to convey this via video. It's really something that you feel but you see the The red window there sort of moving along as you draw. I hope you can see this on the video But yeah, so it's it's not a fixed length again It's using the convergence of points as you draw and the reason why you want to do this is that you want to basically minimize the amount of like Jumpiness so so one way of doing smoothing right is you draw the whole thing and then you let go and then you smooth the whole thing But it's sort of disturbing to the artist to see that their curve being like smooth afterwards So you want to do it while drawing, but then you also want to do it in a way that Like it doesn't feel like the curve the stroke is changing too much. So this is sort of the Probably the right approach. I mean many other programs also do it this way But yeah, this is the the state of the drawing too. I just wanted to talk a bit about the act of smoothing there Alright No, yes I Have a video demo for The state of grease pencil three. I tried to basically summarize Some of the new things that we can do is some of the things that were already possible with grease pencil before In one example, I hope I did a good job, but you'll you'll see I'm actually gonna pause to make sure that this is Yes, okay, so people might recognize this scene And I thought it would be fun to Put something on the blackboard and and actually animate something on it so I have my Sort of bouncy ball animation here One that I did It's not the best but a And So this is using grease pencil But as you can see there's sort of this shade on top of it, which is not something that you could do And we're actually actually gonna mesh grease pencil using geometry nodes But you can see it's it's you know your classic animation. You have your dope. She you have your keyframes you're doing your animation on on this blackboard and Here I'm having I have two geometry nodes modifiers just one to change the thickness so it's a bit easier to see what's going on And yeah, you can actually live draw with these modifiers something that isn't really possible right now with with other tools So it meshes The curves as you draw you can see the wire frame of the mesh and this is it rendered with Evie See I'm just showing off how how the drawing works with Sort of seeing the modifiers live and this is the geometry node setup Not gonna talk about it too much but It's it's basically just curved to mesh right this is sort of the important node And I'm just capturing a bunch of attributes before and storing them after so I can use them in the shader sort of a classic thing you do In this case, I'm building my own UVs So I can use them later on I think I'm gonna show the shader here to just a second as well But yeah, it's a fairly simple setup There's a lead geometry because I'm actually using a curved circle for the profile and deleting the back faces But again, it's not really important Those are the shader I mean, it's you know, it's just a simple Chalk chair it sort of looks like chalk maybe And I'm just reading a bunch of attributes so you can see the UVs for example. It's quite nice and then doing a bunch of math and using noise textures to Make it look like chalk So this is the output of the noise textures and then just using a transparent shader and diffuse shader mixed together Right No, that was not what I wanted and then another cool thing I'm showing here is the eraser So the eraser changed a bit We didn't just port it. This is not something that I worked on. This is something that I'm really worked on And it's now doing like a correct erasing in the sense that it's not only deleting points But it's also creating points at creating points at the intersection of the eraser So this is how this look like looks like it's also working live with the with the modifier, of course And yeah, it's I mean it's so much fun to play with So you can see that it it correctly it deletes the points And creates the intersections and then we can also with grease pencil, right? We have access to other attributes that you wouldn't for example have in the in the curve tool, right? So here we can use vertex colors to change the shading I don't know why I chose these bright colors because you can't really see any difference, but I think I I End up chill choosing like a purple. Yeah, there you go. That's much better. All right. This is this is it That was the short demo video of where we are at and so showing off what we can do right now, so I want to take a Quick moment to talk about the contributors because there's not only me working on this But also other people and look at a few numbers. So here are the like total commits and commits per author Yeah, guess who's full-time job this is But yeah, so Emily again, so Emily was in Amsterdam for almost three months working on the project So she did all of the dope sheet Integration so all the animation features are basically by her as well as the eraser Those are sort of the two big things Pratic is constantly Doing fixes and reporting things and creating tasks has been really great having him on board as well KC contributor who did for example the Transform operators in edit mode and to also worked on the lines overlay so you can see all selected curves and in the edit mode And so many more like Seats her who's also here today working on stuff Antonio's been contributing Yeah, thank you to all of you Some other fun stuff Commits per day Can see it's going up put the trend line in there. So yeah, and more people are contributing Since we now have you know a solid base line for actually developers to get in and examples to pick up you can see my holiday break and Coming back from holidays And again, I mean, I'm just I'm just bragging about like this is my job now, so Right, but there's a lot left to do I mean grease pencil 3 is not done by any means Here are some examples of the things that we're missing right now So we need your help if you're a developer and you're interested in joining the effort There's a task and scan the QR code over just go to the link Has a short description on how to get started and a list of tasks that you could possibly work on so Yeah, if you're fascinated by grease pencil and interest in 3d Here you go. Let's make it happen together. All right, so We do have time we have like 20 minutes left. So If anybody has any questions about grease pencil 3. Yes, it's a good question So the question was when we were talking about the drawing and how there were these two spreadsheets and this offset Array why that offset thing couldn't be like one of the columns basically And the reason is that we need this last element because it indicates how long the last curve is, right? So if you imagine there was the 404 and 8 this 8 basically Indicates the end of the last curve So it's it's a way of Instead of saying where something starts and then storing a size. We just store every start of the curve But this means that we have to have this one more element To make that work. So it's it's not an attributes on array. Okay, so two questions the first one I already forgot. No, it was Shadows, right so So there was an idea that grease pencil render engine could also cast shadows for fills for example and Now it's really easy actually to take geometry notes and say, okay I'm just gonna fill this curve and then you have a mesh and you can just render that with Evie and a cast shadows and Has reflections and everything so I think the way forward is not to add too many features to the grease pencil rendering But utilize the engine that we already have which is Evie And that already works really well. I had a demo for that, but didn't want to put it in just yet So yeah, and then the other question was about layers Yeah, our question was about sorry layers and What was the specific is you only want to want want on to work on one layer. So We we can have you know options like soloing a layer etc And actually the data structure now is is much it's much easier to add that functionality to say, okay I want to what I only want to work on like You know a soloed layer etc. So the API that is now in place for grease pencil 3 is is much easier to sort of add this sort of functionality and I think we're gonna have that yes right, so the question was about performance so what are the performances like now and also you know what are the Methods that we can utilize now like multi-threading parallelization, etc So I do have numbers in fact the before we actually started implementing grease pencil. I wrote a whole test of Comparing performances to make sure that we actually gained performance using this new method And yes, we did so I had some examples where things were like two three five times faster I think for me the more important thing is that we are Using a lot less memory. So files that like test files that I had Go from 300 megabytes down to like, you know, 16. So Crazy numbers when you think about them and of course that also means that the performance is much better right we when we do a copy we copy less data, etc and In terms of the methods that we utilize so we're basically relying on the fact that now at the whole code base is shared Right, so when there's an improvement like at the lower level for hair curves, it applies to grease pencil, right? We don't have separate worlds anymore. We're relying on the fact that these Data structures are similar and we can utilize Sort of the benefit of sharing cord the code there and I don't think I've I haven't looked at performance much yet. It's not sort of the First priority to get things, you know as optimized as we can Now we're just you know making sure that we have feature parody with grease pencil, too And so there haven't been much performance optimizations yet Right, so the question was since the data structures are the same, you know Can you like groom grease pencil now with the hair curved tools? And I think the answer is basically well you you would use Geometry nodes to convert run from the order for example, and then you have You know curves essentially and I Think like you know sharing modes is a bit more difficult and sharing the tools But we're going in that direction with no tools So I think the the goal will be that some of the tools You know some some grooming tools might be specialized for grooming They might not necessarily apply like a sphere's fall off for sculpting increase pencils less revelant Relevant than in grooming But you can totally imagine that those you know that that code would be shared for both because indeed they do use the same structure Yeah, it's true. So the question was With the new data structure like everything is in a race and that seems really nice for reading but what happens when you change today's structure like what happens when you remove a point, etc and Indeed it means that you need to do copies But for example for the eraser what we do is we essentially rebuild the whole drawing from scratch So you like we didn't we do need to do lots of copies, but In the end those copies are a lot cheaper Because the data structure is just optimized for copying right you just take a whole chunk of memory and Move it from there to there. So and The other thing that we can do is we net we can now work in parallel if you have like a whole Array of points you can say, okay I'm gonna allocate this chunk to one thread this chunk to another thread this chunk to another Etc. So we can do a whole bunch of work in parallel and in the end. We just get a lot of performance using that doing that Yes, so I guess the question is When I was talking about implicit sharing, I guess was the question with when I just need to copy the points Right so with implicit sharing implicit sharing is really only possible when your data is in one place Right before because data was spread out throughout memory It's not possible to implicitly share that because you have multiple pointers instead of just one to one chunk of memory So really implicit sharing only becomes possible when you have Like let's say the whole position is packed into one, right? Okay, so I guess the question is about Drawing on surfaces and and sort of re-sampling to move along the surface Yeah for now in the grease pencil 3 2 right now you can't draw on surfaces so it's not implemented yet and It's gonna be something that we're gonna look at in the future and Yep So it's going to be implemented. We just haven't gotten to it yet There's lots of things that we obviously want to implement But really the goal of grease pencil 3 is to build a foundation that we can actually add new features on top of That perform well and that I will integrate with the rest of them So the question was is the implicit share Sharing code using the same code as the depth graph because the depth graph has already this concept of copy on right I guess the copy on right in the depth graph is sort of a misnomer because it wasn't really doing the same thing So the implicit sharing is is new code by Jacques And it's it's not using any of the code from the depth graph the depth graph sort of I guess the copy on right from That's graph was more like the depth graph was hoping that the data structure itself would do a copy on right More than it was itself implementing copy on right. Okay, so the first question Was about what is the minimum copy we need to do per frame? I guess sort of So well, we just need to copy a pointer, right? It's Very easy to now copy like with implicit sharing. We just when we duplicate the object We just essentially copy a pointer for every attribute So I think in grease pencil we have a few attributes that are built in like the position like the radius But you just need to copy those pointers So, you know, it's it's negligible the copy I need to do that and the other question was about groups and what are the plans for groups and Can we expect for them like coloring etc? So the idea for groups is that you know an animator can just have a bunch of layers grouped them together And they were okay these layers belong together It's not something that is going to have like custom data. At least that's not planned It's not something that you you can do much with other than You know organizing your layers in a better way In this tree like structure, right because groups can contain groups There's gonna be color tags. That's something that is already there in the core structure It's not exposed to the UI yet, but yeah, you can have like color tags and color them again So it's it's really easy to see at a glance, you know, what your layer structures. Yes The age old question. Can we erase from fills? So the plan is of course that we can do that in the future. It was just something that We could have done that with grease pencil to the problem is that you start to really like stretch the limits of the data structure That you have right nowadays we can Represent for example holds in a fill much easier than we could have done before these were sort of limitations that We are now with the new foundation basically allowing so Erasing from fills is a project that is planned But it's not going to be there for this pencil three because the project with pencil three is Let's get to the feature set of grease pencil to and make sure that we can do all of that But much faster and much better and then let's look at new things like erasing from fills Yeah, so the question was about active smoothing and how you control. Can you control the amount of smoothing? Yes, of course, you can you can turn it off There's gonna be just one slider. I think at a higher level that allows you to adjust the amount of smoothing At a more technical level what that does is it changes the curve fitting threshold So if people know about curve fitting what that means is that it basically tries to fit the curve within like a certain boundary of You know could imagine a pixel amount So the slider would basically control how wide or Small that that boundary would be and it allows you to control the the active smoothing very precisely And there's gonna be a whole bunch of other options. I think in the current code the Algorithm has about 25 parameters that are now hard-coded because that's how you start But you could imagine that they would be exposed and would allow you to control the drawing in many ways it might not be Like necessary to expose that like in the tool setting But you know maybe in some setting in blender or something where you can adjust the drawing feeling basically Okay, no more questions. So that's it. Thank you