 Okay. So, hello everybody. I'm Paul Mellis. I work at CERSAR right here in Amsterdam. The work I'm about to show is by me and a colleague, Casper van Leeuwen, who could not be here, unfortunately. So, but it's... I should not do that, I guess. Something in the cable. Yeah. So, the image goes away. I moved the left open. Ah, so it's good. Okay. I won't touch it. So, I work at CERSAR, which is the Dutch National Supercomputing Center here in Amsterdam. We have a number of large compute systems that we offer to scientists, to all kinds of compute jobs or data analysis, stuff like that. So, basically the background of my work. I am the head of the small visualization group that we have there and we support all users of the systems with 3D visualizations, data analysis, stuff like that. So, just to manage your expectations, the talk will not show images like this. It's not blood on this level if you want those kinds of images. I think you have to go to the other talk, which is about surgery. The context of this talk is scientific research, scientific simulations, computational biomedicine. So, it's using computer simulations to do all kinds of useful things in terms of biology, surgery, stuff like that. It's actually, the science is the work of Garbus Zafotsky, Alvin Soekstra, and other guys from the University of Amsterdam. So, every part of the science that you see is not by us. It's by the UVA. And the context in this case is blood. One of the cases that they're working on is aneurysms in vessels. So, you have a blood vessel. There's a weak point in there, which is this bulb that you see over here. Your blood vessel expands and if it expands too much, the pressure becomes too high. It can rupture. And if that happens in your brain, it's obviously a really bad thing. So, they would like to be able to predict when that rupture and where that rupture might happen based on things like scans of your blood vessels and such. Using modern scan technology, like an angiograph, which is basically X-ray, you can extract the 3D surface of a vessel, discretize it, use a physical model, et cetera, and then use that as a basis for these simulations. Everybody knows blood. You've all seen it. You know that it's flowing through your body. There's a whole system with your lungs and your heart, et cetera. But what you might not realize is that blood is not really a simple fluid like water is. And when you look at blood under a normal optical microscope, you see all these little cells, red blood cells, platelets, white blood cells, which are shown over there. They actually make up quite a lot of the blood. So, if you put blood in a centrifuge, about half of it, almost half of it, is these cells. And the other is blood plasma, which is more like a fluid. But because there are a lot of cells in there, it acts really funny when you look at the physics. And it's hard to explain. I don't entirely understand it, but because it acts so funny, it's important to model it correctly when you want to do simulations of the body and of the blood. So, the way this is simulated is to have a 3D mesh of a single blood cell, almost 1,300 triangles in there, then come up with a computational model that, well, governs how these things move. So, this is very much like a class simulation in Blender, but this is for blood cells. And you can see there's all kinds of twisting motion going on. You can see on the top here that the cell seems to rotate while its shape and space maintains more or less the same shape, which is something that is known from reality. So, the computational model here is, of course, based on reality, I think, known from literature. It has to be a real blood cell. And then, once you have a single cell, you can do a whole bunch of them. So, I think this is 25,000 cells. These all interact with each other. They bounce into each other. There's fluid in there that makes it move. And, of course, you can't do this simulations on a simple laptop. You need to have a big machine for that. So, this is a distributed computation. Each colored piece that you see here is the work of a single CPU core. So, this is, I think, a thousand cores working at the same time. Some funny things that you can see here is that the geometry migrates during the simulation. So, a cell moves from one process to the next. So, it moves through the simulation. Cells that go out at that end come back at that end. So, you have an infinite length blood vessel, in a way, which is, of course, good because you don't want to have a wall here. And the other thing you might see is these colors here, which looks like sea fighting. Well, that's exactly what it is. There's a small overlap in the geometry between the processes. And that's a little bit tricky to handle when you want to visualize this stuff. So, this is, that's pretty non-standard when it comes to, oops, when it comes to geometry and blender, right? This is basically a new piece of geometry every frame. So, we have to handle that. There's also a lot of cells because they're very small. So, here you have two cubic millimeters of blood, two by one by one millimeter. There's about eight million red blood cells, half a million blood platelets in there. So, any kind of actual simulation of a blood vessel already has millions and millions of cells. And I'll come back to making this image in blender or actually cycles at the end because it was a bit of a challenge. And it's kind of interesting because simulations like this, you have to start with something. So, this is an initial condition. It's what it's called. It took one week to put all these cells into the right position without overlap. That's a 32 core machine. So, it's quite a big computation just to get started with these simulations. And then the simulation themselves, once you run them, you have quite a bit of data that they produce. So, this is fairly small, 16,000 cells, 500 time steps. And then you get, well, 830,000 files, half a terabyte of data for something like this. And it doesn't just contain geometry. There's also values on each of these cells. For example, the volume of a cell or the forces that act on the cells are there. So, it's a lot of data. And this is all saved in HDF-5. If you've worked with Alembic, you probably know HDF-5 that's the format that's underlying Alembic, or at least it used to be, I think. Used to be, they moved away, which is too bad. But they had good reasons probably. And then when the scientists start to work on these simulations, they use visualization as well, but they do it like this, usually in an application like ParaView that's shown here. So, this very functional representation, you have a color that shows like the velocity of the cells, they've got streamlines that show how the flow is, et cetera. And for them, that's fine. But we want to produce a nice movie out of this, because within the project that Aldersberg was done, CombineMed, we had an idea to produce a virtual human movie, short movie five minutes, show all kinds of these kinds of simulations within that movie, all kinds of nice animations, and have a voice over there in the story, et cetera. Barcelona Super Computing Center was in the lead here to produce it, to do the art direction, et cetera. And we produced these cell-based blood simulations, which I'm going to talk about more. So, this is work by Kasper who worked on the materials in shading, et cetera. He came up with some really nice shading to show cells with a bit of a soft look. We needed some x-ray shaders to show like a vessel with cells in it, so that you have some kind of context. There's also some metal parts, and I think he did a great job in producing these. One thing that I really like, although it's hard to see on this projector, even when you just add shadows versus no shadows, that already adds so much information about understanding the shape of what you're looking at. And for a scientist, they usually look at it like this, because it's very simple rendering in the tools that they use. But I think it has a lot of benefit to do it like this. So one of the challenges within this whole workflow of using Blender was, how do we process this data? It's a lot. We have several of these simulations, a couple of terabytes. We don't want to convert it all to, say, like a mesh format or a Blender file or whatever, because it's just too much data and we keep generating new files. So if possible, we would like to read directly from these simulation output files. And fortunately, we can, thanks to the Blender Python scripting, so we use the H5Py module, which can load these files, use a bit of NumPy and some 3++ to do all kinds of geometry processing. So for example, here you can see the individual parts. So each of these compute processes is writing out its part of the data. So you get, let's say, a thousand part files that you have to load in. There's the geometry overlap that we need to remove, perhaps merge the whole thing or even split out several cells. We managed to be able to script that all in Python. So it all binds it out to a simple Blender import script with a number of options. And we just handed the list of H5 files and say, we want one part, we want one cell in a Blender object, et cetera. And this works really nicely. This also was used to create this wall way of showing things by putting the cell side by side. It's also done in this Python script. So we're really happy that the scripting is there. I should not do that. Now the image is gone again. Let's see if it comes back. Did I miss the technical guy still here? Yeah, I touched the laptop and it's gone again. Unplugged? Let's see if it comes back. Okay. Remind me not to touch it. Yeah, thanks. Keep saying that every minute, okay? Don't talk, right? So one of the challenges when we started to do advanced shading is there's no UV coordinates on this geometry. It's scientific data. There's no goal to do 3D rendering in the beginning. So then you start to look at the geometry and think, how the heck are we going to do UV mapping? There's no way to wrap it in any way. We have a completely different geometry every frame, so we can't even match it from one frame to the next. So what do we do? Well, luckily within the files that are saved from the simulation, there are this vertex ID array which is the local index of a vertex within a cell. So each cell stays the same geometry, basically. It's only the positions that change. And we can use that to do the mapping. So the way that we did this was pick one cell, which is undeformed, unwrapped in the usual way. Just add seams unwrapped with the automatic unwrapping. Then save all the assigned UV coordinates to a file. So that's basically a prototype UV map that we do. Then during loading of all this geometry back in Blender, we apply it to each cell in simulation, apply the UVs again. And then, luckily, we can do it. And we even managed to add it to this Blender import script. So it becomes just another parameter for the file that contains the UV coordinates we want to use. And it was a really nice workflow because we could easily update the UV mapping and then see how it improved, etc. Still not perfect. Let's see some weird edges here. I'm not a very good 3D model or artist anyway, so I just did my best here and well, it's okay, I think. Emotion blur or something we looked at, but it's quite a challenge in this case. We're doing flipbook animation, new mesh every way, every frame. We might be able to use shape keys for this, something that I looked a little bit into. Have two shape keys, have to frame n on the first key, frame n plus one on the second key, and then hopefully the interpolation will work. One thing to handle here is that these cells cross all the way from here to the other side. We have to detect it and then actually put the cell a little bit over there to make it look as if it came from there. So that's something that the bookkeeping that we need to do. And we also need to make sure that, for example, a vertex in this cell matches up with the same vertex number in the other frame. And we're not quite there yet. So if anybody has any better ideas how to do this, please talk to me. I'm very interested in it. So we haven't been able to use machine blur as of yet. And then, well, being a supercomputing center, we render on a supercomputer, or at least the cluster. This is the Dutch national supercomputer, a photo shot in a colleague to show the size of the things. It's actually not oriented like this, more like supermarket aisles, all these racks. A lot of cores, a lot of memory, a lot of performance as well. It's a great machine to use, but it has its own peculiarities when they use it. So there's a job, best system running on your code slurm. You always allocate a full node. You request I want n nodes, and then you have to wait until those are available. If you're lucky, and you have a short job, then all those nodes are available right away. So you might be able to render a whole movie in one, the time for one frame, right? All frames are parallel. But there's not always room. To do the rendering, we created a render script. It just takes a single frame number, and then some options. For example, we could easily vary the camera to use or to have a preview material, preview shading, et cetera. It has to go through a job script, which is just a batch script for Linux, load in the software, pick the frame, call the renders by script, et cetera. And then submitting all these jobs was fairly easy. Using this command as batch from slurm, they say I want the range one to frame 150, or something larger or something less, and then add the options to use. And in this way, slurm will take care of all the allocation of nodes, making sure that the jobs run, et cetera. And it was quite easy way of working with it. And in most cases, I could render the whole animation in parallel, which is really nice. Now, Cartesius has both CPU and GPU nodes, although there's a lot more CPU nodes than GPU ones. Each CPU node contains two tests, like a 40s. And on a machine like this, for every compute hour that you spend, some budget is deducted from the budget that you have, because somebody has to pay for it. And the way this is done at our site is that the GPU nodes are three times more expensive than the CPU nodes per core hour. So we use something called an SBU, which is just a credit, basically. So that means that it only makes sense to do GPU rendering if it's at least twice as fast, because otherwise the CPU rendering would be cheaper. And I did a little test with this classroom scene from the Cycles benchmark. And you can see that on the GPU, it's more than two times as fast, but it's only 15% cheaper. So depending on the scene and the shading, et cetera, might have to look at what is the cheapest alternative to use. So then, after all the work and rendering, and letting the Barcelona guys turn it into a full movie, it was showtime at the London Science Museum. And they have these events called Delets, which is really nice. They open up the museum until 10 PM. You can go in for free. The whole museum is you can visit. And there was an IMAX theater there where they showed our movie with a program around it. So I started with the movie. Then there were short talks by the different researchers that worked on all these simulations, and then questions from the audience. We went there. It was really cool to go there. Unfortunately, I don't have the movie yet, so I can't show it. I will show some of the clips that we made without any voice over or music. No, I think it's about 20 times slower. Yeah. I think the simulation works on the level of the nanosecond even. And they write it out in my microseconds or something like that. So a frame is about a step off a microsecond, I think. So I promised to come back to this image. So that was the image that the researcher, Garbor, showed me when he said I have an 8 million cell. That is it. Anybody want to take a guess what the renderer was used here? Not blender, not cycles. No, it's Povray. Some of you might have known that. I think it's just as old as it. And he did it himself, so he wrote a script to generate the scene, and then had Povray do all of that. That's cool, but I think we can do better if we can manage to do within blender and cycles. But then it's an 8 million cell data set, so it's about 10 billion triangles. And because this is a scientific data set, all the cells are there, even on the inside. There's no tricks like hiding that automatically, because we don't know where those cells exactly are. And we might even want to cut away a part and show what's inside. So we really have to use all the data. But luckily, all the cells in this case, because it's the initial conditions of the simulation, are the same geometry. There's no deformation yet. So we can use instancing to do the trick. And then being completely stupid, I figured well, I'll just make a little script and set up the scene like this in blender. I have a single mesh, instantiate for each of these 8 million cells a new object as reference, that same mesh, and then just add it to the scene and we'll be done. Well, no. I think I did some searches. Anybody that tries to do this in blender and after a couple of thousand objects that you add to the scene, the time is just explode. It's exponential when you start to plot it, and this is even 100,000 objects only. You see it already takes like 12, 13 minutes, I think. So there is something in the way the scene is built up in blender that is not linear. So then I thought, well, maybe even if I can get blender to load the scene, I'm not sure that Cycles is able to render it. So maybe I should focus on that. So that's what I tried. And Cycles has a standalone repository for the source code. You can get the C++ code for just Cycles, and it's really cool. It has an XML file format so you can create a scene and then render it in Cycles, and that was a good basis. So what I did was I added an XML tag, let's say an element of the scene called cells, which took a binary file containing all the cell positions, et cetera. You extract, for example, minus 1 meets all of the cells. And then this is the geometry of a single cell, which is undeformed. And then I had to add surprisingly little C++ code to get us all done. So it's just a loop over all the cells. We create the mesh once, add an object for each of the cells, and then basically done. And we render it in Cycles, as you would normally do. And surprisingly little time would work perfectly, actually. So this is 10 million triangles. There's insane amounts of detail in here. And I think it's a really, really cool image. Cycles is absolutely beautiful in that respect. So this is instance. All of the geometry is the same object. And then I figured, well, if I might as well try none in this geometry. So here I create unique meshes for each of these objects. I managed to get up to half a million, because after that the node went out of RAM. I had 256 gigs. And I could get, well, 710 million triangles. Still perfect. It's amazing. So Cycles, for me, great, great job. And that's about it. Thanks for your attention.