 great. Hello, welcome. My name is Martina Fröschl and I will tell you something about Microscopic 3D data visualization in Blender. But first of all, I want to thank the whole Blender conference organization team because we are also doing a conference in Vienna and I know how much effort it is to actually run such a conference. So big applause to the organization team. Yeah, and I also dressed a little up. I have no chickens due to something but yeah, I have this funny mites t-shirt. I made it like four years ago for another presentation because I thought, huh, mites, most of the people think they are very creepy and so on so I wanted to do some funny t-shirt for the presentation in front of some biologists. So yeah, I will talk about Microscopic 3D data visualization and I will first of all talk a little bit about my background and then I will step into the workflow and at the end I will give like a short work in progress example. Yeah, this works. So my background, I started the studies. I started studying at the University for Applied Sciences in Austria back then and then I started my career at the visual effects company at a small that did a lot of documentaries. So it was like the very first step to go into the topic that I'm talking right now. So the most influential documentaries for my work were like the Bionic or Nature Deck series. You can see I put it here, yeah, right. These are like three documentaries and one of them even won the Emmy Award, I don't know, like in 2008 or 2007. Yeah, when I was just working as an apprentice at this project, but yeah, it's amazing work back then what my colleagues did there, yeah. And another influential work back at this small studio was Uplanet. Uplanet is, we did like 20 minutes on animation for the documentary which is a real lot of animation for a documentary and we did like scans of skin, like a lot of scanning electron microscope projection things and animation of creatures on the skin. So Uplanet is basically dealing with the human body as an ecosystem, as a planet, as the title says here. And yeah, it's just beautiful documentary. Go watch it. Yeah, that was like this one, but I think you already saw it. So yeah, the next one, the next image was also very influential for my path on like going to this work with the microscopic imagery. This is still, it's my favorite still from the Incredible Water Bear. It is a pitch, let's say a short film that was a pitch for documentary, full CG documentary. Yeah, sadly, we cannot, maybe, can we for a moment just put the light down. It's very, it's very park, the image, but yeah, that's much better. You can see the volumetric light now. And yeah, I loved that work very much and sadly, it never got funded because for like a full CG documentary, you need someone like IMAX producers and so on and they didn't really wanted to, they were a little afraid of that whole volume because it would have used a lot of money to just produce it. But we were close there and yeah, that was also a very important step because there was this creature and this creature was the very first creature animation that I saw in Blender somehow. It was done by a colleague of mine by Anya back then in our studio and that was the step when I saw somehow, huh, you can do that with Blender. You can do like such cool and cute animals and animations with Blender. So and then there were some changes and so on and so I left for university and I just decided to work from that step on with Blender. And yeah, what I also wanted to say, you can see the whole animation, it's like three minutes or so in the internet, just Google the incredible water bear. It's a very beautiful animation. It's done in Riri and 3ds Max mostly, but as I said the main character is in Blender and yeah, just watch it, it's gorgeous. I still love it. And the last image up there, it's from my work now. It's a very abstract, but also very philosophical story about a meabre eating baromethium and it's all actual real scans. So we have the organelles and everything. So the organs of the amoeba also scanned in and a lot of simulations and so I cannot show the whole movie right now, but I think next year it would be out. And we have also this abstract animations going through the amoeba. I love it a lot. So yeah, I hope you all look forward to it. Yeah, and now, yeah, now I'm with my background at the present time. I'm working at the University for Applied Arts in Vienna and I'm like in a research project, but we basically build up like a 3D model collection. And with this model collection that is very detailed, we are going to do different projects like we want that the students work with the very accurate models, but we also do like, we just started a project with the UCLA California to do interactive presentation of some environmental issues about a plankton and noise and so on. And in two weeks I will go there and we will just work together with the students there to make this interactive installation work with my actual plankton. Yeah, and I also wrote the University of Vienna there because there are sitting all my brave biologists, so I have no biology background. There are a lot of scientists involved and they make the amazing imagery, so the microscopic images and so on. And they also are very involved in the feedback process when we repair the models for the actual database like the collection of the 3D models. So my next slide is the inputs that I have. These make my work somehow special because we do not only work with like references and model the animals, but their actual scans like they're mostly nowadays like confocal stacks and we convert the data to 3D models. So this one is like a SEM image, Rastael electron microscope or scanning electron microscope and it is like the mouth shows of a mite, but the SEM images we add them additional to the 3D models that we get out of the conversion from stacks to real models. So yeah, and yeah, we have also a lot of additional data that we just add and that makes somehow this workflow special. I then show a video of my workflow. I mean it's always a little, it's always a special workflow. It depends on the animals and so on and there's always some more to repair, less to repair, but yeah. I have the case example in the end. Yeah, then I have here a cryo microscopic image of a Salmonella. We plan to do something with that and we were also like talking a lot with the biosenter in Vienna that is an international research cluster also about visualizing because visualizing is also very important for them to just communicate their work also to their funding and so on. I just split in like a slide also of the confocal scan of the Domoptaris, that's the light microscopic image, that's the animal I'm just working on right now. So you will see it in the end of the presentation again. Yeah, and what do I really get out of my image stacks? So this is how the image stacks look when I get them. It's like depends on the detail of the models but 600 to like 1000 or 1500 images and these images you have to get out of the basically volumetric data. So the images are stacked up on and interpolated in between and then you have the 3D data and you have to get, that's the main thing about my work, you have to transfer these images to real 3D data and the first step might be a volumetric render. That's Blender internal. You can do it pretty easy and very fast and I always do it with every animal because it just, it looks cool. You have all the details and real the hair and every organ and everything. Yeah, but of course you cannot really animate it. It's too much details and it's volumetric data, it's no polygons. But the last image, it's not from me but I also did like this kind of, it's already a polygon mesh but yeah, it's not to handle somehow it has over 100 million polygons somehow. But that was a really special case because it is very detailed and I got from the MITE workshop, the MITE presentation, the cooperation with biologists that work on MITE. I got this stack that was very detailed and they that were biologists from the Technical University in Dübingen and they had like a slot at the Synchron Facility in Grenoble and they just put in this animal and they are actually also researching on it but I got the images from them for experiments. So this is a very special case. I wanted to put it here because it's so amazingly detailed and yeah, that's also like when you cut it, you see, I mean, it's only, I have no turnaround or so here but there's so amazing details in there. So it's just incredible, yeah. And back then that was like four years ago, I had to convert the stacks myself and I did a lot of experiments so I just put some software that you can just convert stacks if you want to try them. I mean, you find someone has some stacks in the internet so if you want to try it yourself, I just put some of the software with them as I experimented with the software back then and yeah, I came out nowadays when I convert stacks to 3D models then I use usually today ImageJ with 3D view blogging. It worked out for me the best way but you can do it also with the software I just wrote down here. They're cool for experiments, all kind of, I mean, I'm a digital artist and I was alive and I have to experiment. So yeah, then do the actual workflow. I always have to ask myself when I start with a new project what I want to show and what are the demands and which imaging can I get from the biologists because it makes a huge difference if I just show the whole animal or I want to show the digestion system or something or like processes going on in the body and so on. So we have to be very aware of what we want to show and if there should be animation, what should be animated and also which details do I need if I want to see the details of the skin then I have to do like scanning electron microscope and put it on like with textures on that animal and yeah, there are a lot of questions I have to ask myself and there are also always very different and custom challenges and custom solutions because as I said before I started with soil organisms and they have like a very hard skin so they are easier to prepare. You see like a sample here and the biologists, they had easier time with their hard surface animals and then with the with the plankton I'm just working right now so my case example will be like this Domotaris plankton and they have like really thin skin and they shrink easily when they are prepared for the confocal microscopy or for the light microscopy or what else but so it is harder to work with them and also for me the sculpting, the adjusting sculpting is much more changes in the scanned model but we try to keep as much data so it's always like the feedback process is a longer and so on so yeah that's as I said the case example animal Domotaris in the Petri dish and yeah I also wanted to put this image in the presentation because it stands for the demands for the like special demands we have sometimes this is actually a motion capturing setup for microscopic animals and we did only like to custom Nikon cameras and I wanted to do some motion capturing back then when I started my PhD about the microscopic 3d data processing and also I wanted to do the movements very accurate very correct and we did some motion tracking yeah but it was basically basically done like scientific rotoscoping and some shape recognition because you cannot of course put like motion trackers on that yeah right the next images yeah it's like the case example workflow that I just will show in the video afterwards we will go through like the raw data the the actual raw data then the the high res mesh and then the whole back and forth processing of the sculpting and the adjustment of the geometry because yeah the as you can imagine the animals the scans the actual scans we use they are like cramped and stay they die of course when when we do the samples and so it is always a long process to have like this animal that looks like life in as a living again yeah and then you see the the shading the actually process also the rigging and some of the staging I did like a sample stage somehow just to show a little bit but it's all work in progress then yeah and I also put here the whole collection as it is right now we have we have up there the the term of terrace so the latest plankton animal and noctilucca they are both like shining in the dark so the real special animals so my supervising professor wanted also some animals that are very special and they are just also giving good images um and um we have the okra plora um lava here and the um amoeba and then the archicocetis we saw before the mite, the cyanobacter, some centipede about a microscopic centipede so it's a real small animal then it is also from the incredible water bear short film um then uh some uh the parametrium then that's also a very interesting thing you can all try it out these data are all free available in the internet so these are like um uh they are in the in the protein database um you can just google it it's like a database where you find a lot of uh molecule um species yeah most of them I built them uh these are like phages they are infecting bacteria so they are very small and yeah but maybe you have seen them in an in a nature program or something and yeah they are freely available and uh you can just experiment with them I did like a volume rendering and uh a shaded view and yeah then some more mites down there the psiloscope you and uh spring uh a spring dale and another mite there so we did at the beginning we did a lot of mites because we wanted uh we wanted to make a movie about the solar organisms so yeah the next huh will be the case example I hope it will work so it's just um some screenshots about the workflow um I did over the time and the samples and the light microscopic references and uh then like test animation but yeah the whole the the largest process yeah that's the volumetric rendering but the the um most time of the process was of course the the adjustment process of the model just see yeah I also lose uh I also use a 3d code because I love volumetric um sculpting for that process is because I have to break sometimes the animal and just put it back again but uh I always try to just do it with without um destroying the actual shape of that animal so that's always a little and some yeah shading tests a lot of tests all the time I do it also very like as a creative process somehow and textures a lot of yeah these textures were actually also of the of the light microscopic images so I projected them with uh texture projection yeah and then we have like the very first blocking test animation well huh but we have to watch that again because we we've lights down I put I put so much effort and uh much effort but but like two days in the in the background and there were these cool particles and everything yeah I think yeah um I think we can see the hole again it's only like two and a half minutes yeah so yeah thank you oh and it's even large now great I also used the Autodesk tool for mesh repairing sometimes it's also for free um it's like for 3d printing very handy sometimes I like to use some tools that just help also huh it's the background yeah okay thank you yeah I I'm nearly at the end but it's like only I want to show some links that I think give you some extra some extra information about about what we do at the science visualization group at the university I mean I'm the only free the the only 3d artist at the moment there in this group um but but but there are a lot of biologists and yeah I hope that there will also some uh students that will be like interested to work with me then under the project or yeah we have to force them no um so yeah so there's my website there's not so much on it but I'm I'm planning to just add some more then there is like the website of the digital art class there you can some find some background in infos about the department I'm working in then this is the website of my producing and supporting professor he has under services a pretty good description about the science visualization group and they're all also all the contacts of the people who just um were involved in in the whole projects and I mean there were some projects up to now um and then I also added like um the conference um I'm also in at the organization board of this conference it's pixel Vienna and we have like uh the 12th time of the conference next year and it's also a pretty cool community event all organized by volunteers um so if you ever come around to Vienna it's a pretty cool event and a worth absolutely worth visiting and the last link is very important if you're interested in this work um in the whole like 3d microscopic 3d data visualization it's a paper we did like four years ago about the work at the uh about the mites yeah so it's like a workflow and also the the biologists put some of the um of the data in like how they did the preparation and everything but also I I wrote like um some some chapters about the 3d um workflows yeah so you can find it for free uh at the sankenberg um website yeah that was pretty much my talk so