 Thank you for coming. So this is my talk about the new sculpting work from Rhym Blender. I'm Pablo de Barro. You probably know me for posting videos of Blender features on Twitter. But the important thing you need to know about me is that I love to make art and I hate computers and everything that has to do with them. But they let us do amazing things so we have to deal with it. I studied computer science and I ended up working as a assistant administrator and I ended up so psychologically traumatised that I started to sculpt cute K-pop girls and this is way much fulfilling for my life. So I started like designing better tools for doing this basically. My idea for this presentation was to, well, I spent working like crazy like this past two weeks to have 2.82 scale mouth ready to show you it here for the first time. But basically almost all prototypes have working for 2.82 and the most important patches are ready for review and I'm going to show you a little bit about that at the end of the presentation. But the important thing is that we don't have any 2.81 documentation about the scale mouth and basically right now the official documentation for Blender is the donut and we don't have proper videos on YouTube on how to sculpt a donut with the new sculpting tools. So this talk is primarily focused for artists and then I'm going to explain to you how you can use these tools and how they actually work at the code level. So this talk is going to have code, sorry. And it's going to have way more information that you will never need to use these tools. But it's important that you know how do they exactly work. So if you want to make tutorial videos or share this knowledge you are sharing true information and not confusing people even more on how this tool works. And yeah, it has not a lightning tech box, so sorry. Okay, this is the Boxel Remesher that is one of the main features of 2.81 that basically lets you break geometry as hard as you want with modifiers, with bugs in the scale brushes and it will fix everything just magic pressing a button. So this is the Boxel Remesher. It has 58 lines. So let's go line by line explaining how does it exactly work and I'm not joking. I'm going to go line by line of all the features first. The object and stop the operation if the box size is zero because computers don't like zero input fields with numbers because they crash. Then we store the geometry of the current object in the sculpting undo system. This is new in 2.81 and this will let us have sculpt operations which modify the mesh in a single step. Like trimming tools edit mesh brushes, insert brushes and all those kind of things. And after that, this is the whole Boxel Remesh basically. And if we click on that function and go inside that function we're going to find, started to see open bdb things. And basically the Boxel Remesher is just those two functions that is converting the mesh to a level set and converting the level set again to a mesh. If we keep clicking in those functions those are two open bdb functions without anything else. So let's go and explain what this function does with an extreme simplification that will make every open bdb developer cry but whatever. Let's suppose we have this line and we put a grid on top of the line. Then we put zeros in the cells that are closer to the line. Then we fill with one to one sign and one minus one to another side. Then we choose, then we treat and this is a level set. So then we are going to convert that grid of numbers back to a mesh. So we have this and we draw a line between the ones and the zeros. This line is something similar to the original line. Now you do this with 3D math and you have a pre-mesher. And this is what open bdb is doing. Then the next step the Boxel Remesher does is fixing the poles that is a really complex algorithm that you are going to see right now. Basically what this is doing like enter it enter edit mesh but you can think of it like enter edit mesh. And then we are going to do something across all the phases of the model. So we are going to check phase by phase. If there are two vertices that has three edges connected that are opposite in the quad. So this quad has four and five is not valid then it has four and three is not valid. The next one has five and five so it's valid and three and three so it's valid. This quad is valid. So we are going to remove it and create an edge between those vertices. And we keep doing this for the whole mesh. This quad now has five and three so it's not valid but before it was valid. Then we do this for all the meshes, for all the quads in the mesh and then collapse the edges. Basically you have your topology fixed. We also check for quads that have three connected edges in all of these vertices and that has, we remove it and we run the fill holes operator. And then we select everything and we smooth the mesh and you are done. This is what this option does. And this was an idea of Julian Caspar so I just implemented it. Then there is an option in the boxer remeshers that's called preserved volume and that literally applies a single out to the original mesh. And if you look at the code, you look that it's a single modifier and it's exact same code as a single modifier and those have their options of the single modifier. So if you look, if you go to a blender code and change these options you maybe could find better defaults that produce less artifacts because if you try the boxer remeshers it goes crazy and projects vertices to the other side of the object but it's basically tweaking these values. And then we have the project mask flag that runs this complex reprojection code that's searched for the nearest vertex in the original mesh and copies its mass value for the new mesh. And this is the final lines on the boxer remeshers and it's basically copying the data to the original data block and spooking the normals and store the mesh on the sculpting undo system to be able to return to the operation again without running the remeshers again. And that's it. That's the whole boxer remeshers. So now that we know how it works probably a user is going to download 2.81 and is going to run the boxer remeshers on the default cube and he's going to get this and the reaction is going to be this but the user is going to rotate the cube and he's going to get this and then the user is going to do this. But now we know that in the first case scenario it works because the faces of the cube are aligned to the internal grids we are evaluating the volume so it will look like this internally. So when you are recreating the mesh it will recreate quads perfectly aligned to the faces because the grid was aligned to the faces of the cube. And as soon as you rotate the cube you are going to get this and there's no possible fix for this. So this is the most important thing to know about the boxer remeshers. It is not a retropology tool and it won't ever be a retropology tool. So what is the boxer remeshers for? It is good for sculpting because it fixes self-intersections and geometry errors and all those kinds of things. It's so good for creating meshes for 3D printing for the same reasons. You can export a mesh out of the boxer remeshers and you can send it to a slicer and it is going to work just fine. And it is also fine for creating simulating proxies because it's a mainfoil mesh based off quads and usually the physics simulators don't have issues with this. What you should never do with the boxer remeshers is use it for retropology of any kind of objects because the edge flow is going to be completely random. And you should never, never, never subdivide a mesh from the boxer remeshers. I mean, if you look at this, there's no way this is going to look fine after subdividing it. So I've seen a lot of tutorials on YouTube explaining that subdividing this will improve the result and no. No. Next, quadriflu. Quadriflu basically does this. I won't do something. Okay, it creates quadtopology like the boxer remeshers but we are going to see that it's not the same. So quadriflu, we didn't implement the core algorithm it's from this github repository and from this paper and the implementation in blender is quite similar to the boxer remeshers. You just switch the two open bdb functions we just saw from the quadriflu and the rest is basically the same. And as you can see here, there's a bisect and mirror operations before and after and this is how we are faking symmetry in quadriflu. Really symmetry should come from quadriflu itself but this was the easy solution to get it working for 2.81 and if you are smart and that's me and you know how to go a decimate algorithm to put right there to avoid feeling quadriflu with a million vertices that will definitely improve performance a lot. So this is what I just said is the code from this checkbox in the mirror modifier that runs before quadriflu and then it's the same code from that checkbox in the mirror modifier. So quadriflu, if you read the paper that's based on, it's basically this you know that it mentions in several places that this is useful for creating measures for clouds to divisions. In that case, in this case when sculpting, especially stylized or hard surface characters you can have, you can sculpt with the boxer remesh and polish your surfaces with the scrapers but eventually you are going to have a topology like this and you can keep sculpting and you are never going to get a perfectly clean edges. So to have those kind of edges you need to have geometry aligned to the edges of the model. So you can polish like with the pinch brush and with the scrape and you are going to get this that looks perfectly fine. Okay, so for high poly sculpting edge flow is not important so basically any quad solution should work fine because you are going to add a lot of texture detail on top of it and it does not important. But for stylized and for hard surface edges must be perfectly aligned to the features of the mesh. And if there is one of those edge loops that is not correct, the result is not valid because it is going to have shading artifacts in your model. And there is no automatic solution that guarantees this in all cases. So you have two options or you do manual retopology that is basically what most of us do or we have quadriflow manual guides that we are planning to add at some point and with this you can tell quadriflow that an edge loop must go exactly in that place and that is the only important thing quadriflow needs to do in blender. So basically quadriflow is in blender just for creating base meshes for subdivision surfaces and it is not for remeshing the model while sculpting. This is very important but I'm going to come to talk about this later. And obviously it's not going to create final topology for animation ever and it's also not useful for mesh decimation because we have much faster algorithms than this to reduce the polygons of a mesh and it's not going to fix mesh artifacts and geometry artifacts for 3D printing but it's going to create more artifacts right now and it's not going to fix 3D and scans and generally it's not going to work that well with high polygons as an input. So this is really important and I think we also have this wrong in the official vendor documentation. Boxel remesher is not a faster less precise version of quadriflow. They are not the same and they are not designing for the same. You saw how the boxel remesher works and you saw how quadriflow works. What you are working is not a matter of choosing quadriflow or the boxel remesher depending on how much time do you want to spend between different remesher. Okay, this is what I just said. So comparing the output doesn't make any sense at all in any cases. They are totally different things and totally different tools even if they are in the same menu and in the same shortcut but they are also very important. The quadriflow workflow is not finished in 2.81 because basically the target for quadriflow is to have this operator. Is to have a button with one click that does these things. It needs to calculate a reasonable number of polygons for the batch mesh. Then it needs to generate a new batch mesh with this number of polygons with quadriflow then it needs to subdivide that mesh with multi-res to get a similar vertex count to the short mesh then it needs to re-project the details from the previous scale to the multi-res version and then it needs to reconstruct all the levels up to level zero in multi-res. This is how it should work. At some point it's going to work like this, hopefully. OK. So now let's talk about the SCAL mode tools in 2.81. You know that I was working on a branch called the SCAL mode features branch and I wrote this blog post in developer.blender.org and I was proposing three solutions in order to have those tools in 2.81 as soon as possible and there were those three and we end up doing this. I'm going to make an abstraction layer for the data, whatever. So I'm going to explain to you how exactly this works. Imagine this is your mesh created with vertex, faces and edges and you have to reference these points for an infamous SCAL tool. Before 2.81 you can reference these points in the code. You can tell the code I want to get the mask value or the coordinates of the normals or of that point. In 2.81 we added these indices to the mesh so we can write extractions in the tools like this and then the sculpting system will look up for that data in the cases how is the maximum data and you can get those values. But also what you can do with this is you can create another copy of that data and write another custom values that the tools are using and associate them with the index of that vertex and those values are what are making all the tools work. So okay and the other thing we did in 2.81 is to create a way to get from the index number of the vertex get the neighbor vertices, get the indices of the neighbor vertices. And this code works both for B mesh, for multi-resolutional, for regular meshes but we are going to talk about that later. And if you need to write code that runs in all the neighbors of that vertex you just write those three lines and it works. This is trivial to do with regular mesh sculpting because meshes are already unarrayed with indices so there's no problem there. And this is why all the features in the sculpting were only working with meshes. And within topo you can do this as long as you don't change the topology between inside the operation because you are going to mess up all the indices. So that's why the new operators don't work with the topo updating the topology because it's time to start a new operator it needs to rebuild all those indices in the mesh so the topo is going to be a little bit slower in 2.81. And this is not that easy to do with multi-res because of how multi-res work I'm not going to enter in that detail. Okay so what you need to know about the sculpting system is like we have that function that basically gets the coordinate of a vertex and given its index. And as you see it has three different implementations and these are three completely separate systems in which a scal mode runs. And you see here that one system runs with when you are sculpting with regular meshes with modifiers one system runs when you are sculpting with in topo and another one runs with you are sculpting with multi-res. And this happens all across a scal mode. This is the scal mode rendering updates you see it has three different functions. This is undo, it also has three different functions. And this is the brass deformation code that it also has three different functions. So this is very important. The topo is not just an option to check and it generates new geometry and that's fine. But it's going to make the sculpting system run in a totally different code in all areas. So it's like entering another mode instead of just a single option. So when you are using the in topo you are running the slowest possible mode a scal mode can run. It doesn't matter that the topo is not generating new geometry or it's not updating the mesh. It's just you are running a scal mode over remesh. I'm going to talk a lot about that later. So the good thing about the boxer remesh is that it keeps a scal mode inside the faces code path which is way faster than the in topo but you can keep sculpting with dynamic topology. So even if the in topo only adds detail what it needed it is never going to be faster than sculpting with a boxer remesh. So you should only use the in topo when you need it for creating some kind of effect with a tool like for the sniper to create that effect or something like that and disable it as soon as you finish. As I said you can always get to a higher level of detail with the boxer remesh because you are using the fastest no not the fastest but a faster scal mode version than the in topo one. The in topo mode representation is bimesh that's basically the mesh representation that is using in edit mode as it's an absolute overkill for a scal mode desillation that basically the only thing it needs to do is to keep triangles and split triangles and collapsed triangles. So we can still use this for new tool like creating edit mesh brushes because we have all the base code ready. So to show you this in the code how it actually looks we are going to see the code for the smooth brush that basically sees the checks the neck bone vertices of a vertex and average the coordinate and patch it in the middle vertex. This is how it looks for meshes. This is how it looks front in topo as you can see a simple algorithm as that is different from the two modes and this is how it looks for multirace. So if you are thinking of coding a tool like the post brush dealing with this it's never going to work and this is how it looks in 2.81 after writing the API. As you can see there's the neighbor iterator and there are two functions to get the coordinates of a vertex. So now that we know that this that is the base for all the new sculpting tools I'm going to show you how a simple operator works. This is massgrow operator. First you create a copy of the max data and then you check for that vertex what is the maximum value of the connected ones and you put that under there and you repeat the same for the rest of the vertices and the massgrow is as simple as that. Then another one more complex one which is maxspand you start with the selected vertex under the cursor and from that you put a 1 and in the connected vertices you put a 2 and in the tool you check the connected vertices again and you put a 3 and you continue doing this until you fit the whole mesh and that's why maxspand creates this kind of shapes if you try it and then if you want to expand the max from the vertex 0 to a 3 you only need to check the vertices that have a lower number equal number to 3 and this is how maxspand works so the thing you need to know about maxspand is that all those numbers all those interracional numbers are created as soon as you press shift A and that depends on the number of vertices the mesh has so now we go with the post brush I'm going to explain you how it works post brush is the so imagine this is the skull cursor and the red dot is the active vertex so like in maxspand first we are going to create like a sphere from that vertex with the radius of the brush and like in maxspand we are going to expand a data value from the vertices like that until here so there you see that there is a vertex that has one there's a line that has one vertex inside of the sphere and one vertex outside of the sphere so we are going to keep only those vertices that have one inside and one inside so we end up with this ring of vertices and then to determine the origin of the post brush and the rotation center you only need to average the position and you get the rotation of the post brush so the inverse kinematic post brush works similar to this but instead of starting again with the red dot it starts from another vertex in that ring it's something similar to that not exactly that but you have an idea how it works and to solve the inverse kinematic it is using this tutorial basically it's not anything super complex I basically learned everything I know about computer science from that channel so it's not like there are complex things in this code ok so let's talk about performance that is most important topic in skull mode so we know that basically all character modelling courses start with a quote that may look like this and if you read this you start to think that this does not make any sense but for digital sculpting we are used to hear this again and again and when we are talking about performance in digital sculpting usually we are talking about this the number of vertices that can be affected by the brush so when you say that there is a sculpting program that can handle 100 million vertices you need to think if that program can really support a stroke that modifies 100 million vertices in one operation but that tells you that it won't work so performance can also be measured like this this is my workspace and basically this is my the specifications of my computer that you see is not that powerful but it can move 12 viewports with differentiated modes with 2 4k monitors and 2 1080p monitors and textures in the sculpt and geometry with active modifiers so it's like this no other program can do this in a sculpt mode so we instead of trying to go to the like to try to say that we have the program that can manage and can manipulate meshes with a higher polygon can possible we need to focus on performance issues that are relevant for the sculpting workflow such as this the smooth brass in multi-resist and that's true in general the smooth brass is too slow for smoothing high poly meshes we also try can try to make entering and exit a sculpt mode a real time instead of rebuilding everything it's time which switch modes because that's our workflow issue that's not relating to brass performance and this is basically the same we don't need to rebuild internal data structures it's time which change modes so this is basically what I think about performance in a sculpt mode that a sculpt mode is just a modeling workflow and sculpting extremely high polygon characters and meshes is just a new case for that workflow and associating digital sculpting with mesh performance is just how we built a workflow around a particular tool that does this particularly well but it does not mean that we have to be a competitor in this area to that tool because it's going to make us lose an insane amount of time and we are going to go nowhere basically so for high poly sculpting before having an amazing performance and all of this we need to have this issue solved before that I were way more important than the performance especially the first one and about the multi res modifier I also want to say some things for 2.81 it is not finished and the new tools are not going to work properly with the current multi res modifier so for the new tools basically the yellow ones that are meant to deform the mesh those tools are way more performance demanding than the rest of the brushes and that's why they are designed to deform the lowest level of multi res modifier you don't need to pose a character in the 10 million vertex version of it so basically the issue with the multi res modifier we have is that we have this mesh and we sculpt some detail in the higher level of subdivision and then go back to the lower one to pose it with the pose brush if we return to the highest one we are going to have this and there's no way to fix this with and do with saving if you don't have a save file before you basically break all your work and I think we really need to fix this as soon as possible if possible before shipping 2.81 because a lot of users are used to this workflow are used are expecting the multi res modifier work and if we don't know this they are going to break all their sculpts this is what I'm talking about if we can fix this for 2.81 I think we should probably disable multi resolution sculpting in this release because it's going to be bad so let's talk about 2.82 maybe you've seen this video but this patch is already on review so probably next week you are going to have it in blender that's to probably relax this is like the relax tool that was in the Scala branch but it's a better algorithm that this can handle millions and millions of vertices and it should not create artifacts then we are going to have the new vertex paint mode that my goal is that you are going to be able to paint that the same resolution of a 4k texture with vertex paint with illustration level brushes not just a brush that paints the vertex touch under it so you will have hardness flow wet paint, smooths and all those kinds of effects that typical illustration program has this sphere has 7 million vertices and you can see that it works fine and this is just the first version we can still do a lot of optimizations then we are going to have a new face map visibility system where you can create groups of faces and hide them and you see to mask the tools and smooth different face maps which is way more advanced than the current tools we have which are limited to hide mask and hide box I don't know if this is going to be the final name for this feature but I have to write something in the code and they meet somehow so, as you saw all the tools that are in 2.81 are basically created just using these two things that are added in the base code and in 2.82 these are going to be the new base things that I am going to add to the sculpting system with vertex color data and face indices we are going to the new scale mode and the visibility system in order of topology connectivity we are going to get brushes that can create effects that flow across the topology like for example you are going to be able to extract parts of the mesh like dragging like this and things like that and with heodesic distances we are going to have things like topology follows and probably let the max expand work in a circular way instead of a square pattern and if I have times maybe I can try to code some based on a fixed timer event and this means brushes with physics simulation inside of them and this is basically it thank you so, if you have any questions yes, sorry there's also William ask if we are going to get the brush management system in 2.82 and that's Planet I personally want to do a lot of reorganisation in the code to support a better organization of brushes so if we can solve the asset management issue about how are we going to store the brushes we can probably have a better brush management system in 2.82