 Good morning. Good afternoon. That's a lot of animals in this room right now. That's pretty crazy. So I'm here to talk to you about the digital life project. And as you can see, this is about the scanning animation and sharing of animals, living animals specifically. So myself, I am a werewolf. I used to be a werewolf. I was at one point in time. My name is Jeremy Bot. A lot of people, my friends call me Jer. So I've been using computers for a while. I'm sure like a lot of us. Initially it was to play video games. I bugged my dad and said, hey, you know, I need a computer. I need it for school. I need to learn all this stuff. The reality is I want to play games. But you know, the joke was on me because in order to get those games to work on the 486, I had to spend a lot of time tweaking. And then in that, I fell in love with the technology. I fell in love with the logistics and the power of it. Eventually I started playing around with some animation tools. This newspaper clipping is from a while ago. A high school radio station. I was responsible for moving us from the old reel to reel to a digital editing workstation in audio. Wrong button. So I guess this really starts. I mean, I like to think of fundamentals. And the fundamentals for me is starting with my mother who was a respiratory therapist. Rather studying to be a respiratory therapist, she had to stop because she had to take care of my sister and I. And she had these textbooks, medical textbooks all over the place. And since she wasn't in school, I don't think she cared too much if her kids were rifling through these expensive textbooks. And that's where I fell in love with anatomy physiology. I couldn't read any of the Latin, so that was kind of a benefit to me. But then growing up, I realized I wanted to be a doctor. The concept of helping people, but then understanding these biomechanics and seeing the infinite in that was fascinating to me. But then she told me, she's like, well, you know, just so you know, it's kind of expensive and then also there's going to be a lot of school. And for me, that terrified me. It was like going to jail. So instead of that, I became a Microsoft certified systems engineer, which later became the solutions expert, which I think is definitely more accurate, because it was basically a product specialist. And you know, that meant I had to kind of keep recertifying every time a new product came out, so eventually I just let it go. So then, reboot of my life, study the animation, basically give everything away that I had, and got a job in Vancouver, BC. I worked on some animated feature films. I worked on many animated TV series. I worked on some monster movies. I worked on a video game. And after working on the video game, I was like, you know, there's got to be something more than just entertainment for animation. It's such a powerful tool. It's one of the most powerful communication tools I've ever seen. I remember the first time I saw, it was like a BBC special on human body and they're kind of traveling through ear canals and stuff like that. And that one image was more powerful than all these textbooks I had seen over the years. And I was like, gosh, like the things you can communicate through this is incredible. So I eventually got into some medical documentary work, specifically on the lungs and blood and the eyes. So then that's kind of one branch. And then there's another branch that's been around a lot longer than I've been around. Matter of fact, it's been around for 30,000 years, at least, at least. These are the shovel caves, I'm pronouncing that right, in southern France. And they discovered these in, I believe, 1994. And it's, I mean, it's absolutely incredible. I don't know. When I picture like 30,000 years ago, I don't picture cave paintings like this. I mean, these are absolutely gorgeous. And then there's these funky drawings, another example of trying to record life. And it's funny, because I'd never seen this before. I didn't know that this scientist did this. But he's recording the movements of plants. He developed a really interesting mechanism for doing this. And it's funny, because it's Charles Darwin. And of course, what he learned was that these plants were greatly affected by their environment. So we can't separate the model from the environment. It's a very difficult thing. We just, like, we can't separate the model from its behavior. So in essence, a 3D model, unless it's moving, it's kind of taking away from what that really is. And of course, I think a lot of us recognize this. If you take these sequences of images that were shot, eventually you have motion. Pretty much, I think, the first accurate recording of a movement of an animal. And then, 1957, we have the first digitally recorded animal. Walden. Good old Walden had a dad who was a developer. He created this drum scanner. So he took a photo of Walden and fed it to the scanner. And he got this whopping resolution of 176 by 176 pixels. Let's go to 2017. This is kind of where my story comes into this branch. And in 2017, I saw this online. And I was able to spin and tumble this in my web browser. You know, this open source portal to information. And I can see it in 3D better than I can see it in my 3D application that cost a couple thousand dollars. I was like, what is going on here? So I contacted Duncan Erchick at the Digital Life Project at the University of Massachusetts. And I was like, how can I help? And he's like, well, we'd love to have you. So that's kind of where I come into this. So now we're going to be digitizing all this stuff, right? Why? It's a really big question. I mean, why? You know, entertainment was easy. I was like, well, we're entertained by it. It's like, oh, you can stick stuff in there. Now we're digitizing animals. We're getting serious about stuff. Well, it's easy to have a mission. And that was the thing that Duncan ultimately said. He's like, well, Jeremy, here's our mission. Number one, open access science. Open access meaning that it's not behind a paywall. It's not in some bookshelf somewhere that you have to get a library card, a special university to access. Like, you can actually, like, just, it's yours. Like, it's digital as well. So it's easy access. Number two, hands-on immersive education. Meaning that, you know, I can look at those books, and as I was sharing before, you know, as I'm looking through these books, it's like giving me all this information, but there's nothing like seeing it move. So then take it one step further. What if I was able to, like, take those lungs and then move them around and dive through them, which is what I got to do eventually. And it was mind-blowing what that does. And the last piece is something that I'm still having this challenge of really discussing, because it's like, well, how does this conservation come in? This is not conservation, like, we're trying to build a matrix, like, let the animals die, and we'll scan them all, and we just live in this virtual reality, because that's one part of it. But conservation in terms of making us aware of what's out there, making us aware of what a rhino really is, making us aware of the different species of rhino. There's 20,000 of these left. There's only, you know, 60 of these, and there's maybe 80 of these left. Sumatra and rhinos. And we start to empathize. It contains its story. It contains its scars. It contains all this stuff. We empathize with it. So when there's a call to action, we're losing a friend, potentially, in danger. So we share them. We get them out there. Digitize them, put them out there, make them free, make it non-commercial so that someone doesn't take that shark and immediately make another Jaws movie where it's just tearing people apart, but they use it for a non-commercial purpose. And what we discovered is people are using this stuff in ways that are communicating very powerful messages. How do we do it? Process. We're going to get a little bit technical here. So process is kind of funny, because when you're dealing with reality, there's infinite variables, so why? Process. Number one, you plan. So that's usually through a relationship. A relationship with a local biologist that's in the area to where these species exist, having a relationship with them and then setting up a time to visit. And then when we visit, we set up cameras. This is Duncan basically setting up cameras in a polar bear. Inclosure. And then also setting up cameras for the rhino at the Perth Zoo in Australia. Next step is record. Record. Duncan's taken pictures manually on the ground, but you can see that there's cameras set up around him. So the camera's around him up in the air for the photogrammetry scanning. Man, that's a hard word. And then on the right, we have a shark that's being scanned. The shark was brought aboard, the pump was put into its mouth, seawater's flushed through its gills. It's on the boat for about a minute, and then it's put back into the ocean. And the other way we scan is in the lab. Usually smaller animals. You can see that there is the beast cam starting on left. And that one's handheld, but of course when it's handheld, you end up with blurrier photos. You don't end up with much detail. Then we have the beast cam as it stands, the more tried and true one in the center. And that one was used to capture the Tokyo gecko that you saw earlier. And then on the right, you have a way of kind of dealing with the problem with lighting, because if you have shadow, that shadow will bake into the final model. So reconstruction. So if anyone's done any kind of 3D modeling, I think they've seen something like this. You grab image planes, you line up the image planes, and you just kind of model. And it works well for cars and stuff, but when you start modeling organic creatures, plus you're dealing with distortion. When looking at anything from one perspective, it's easy to distort that view. So then, how do we handle this illusion, this distortion? More cameras. More angles. More perspectives. And how many people are familiar with photogrammetry? Alright, I'll see you later. Alright, then I'll make this quick. Where's the eye? Where's the right front? Okay, so how we know where that point is. Boom, we have a 3D model. It looks great. Well, first you have to mesh it, and you can project that texture, but you already know that. It doesn't always turn out that well the first time. Sometimes we have to tweak the lens that's coming in. We have to be information from the camera. Sometimes we have to tweak the colors themselves. Sometimes the environment's constantly changing, so we have to set the cameras to automatically regulate the color. We tweak those and we get something a little bit better. It's like, yes, this is totally going to work. And then you try over and over and over and over. And if you've ever done photogrammetry processing, especially with more flexible software, it's very time-consuming and very painful. So eventually we do get a model that looks like, okay, this is going to work. This is something that's worth meshing. And it works pretty good, but of course you're going to have areas that are occluded. And those areas we have to somehow clean up. Fortunately, there are other people out there that are bleivers in sharing information. For example, this rhino head, you can go and grab from Sketchfab and you can see the skull and how far away it is from the mouth. And you can see how articulate that mouth is, kind of like a horse, right? It's like you literally shake hands with that mouth. That's another story, by the way. Then you have turtles. So there's another issue. So you have something that's very flat. It's laying on the ground. Then you make two scans. There was a little bed of towel set up and then we literally take one shot. We do everything in our power to make sure if it doesn't work this time we have another animal. So we take the shot and make sure it's comfortable and put it back. This is Scallywag, by the way, and it was released in February back in the ocean after it was at the sanctuary for a couple months. So a big part of this, too, is to maintain the accuracy as well as find tools to stitch this back together. So actually this is a different turtle. It's about two pounds. And how do I put this together? Because it's like, okay, I'll just grab these points and move them together, but then I'm distorting the information. This is for science. This needs to be accurate. This is not purely just to get it done. So I actually rig it. So this is where a process starts to break down. We think like, first your model, then you texture, then you rig, then you animate. Actually, before I even have a finished model to animate, I'm already rigging. I'm going to go down, make sure the pivot points are correct, pose in a position. I have a full model. We need to apologize on top of that and maintain all its proportions. So when a scientist looks at it, and the best part is, too, of course, is if you have one rig and another rig and then you pose it, all the points line up, you have a true. So then you have a third rig, which you know is accurate. But you still have occluded areas. So you have a scientist in the little Glaria, and you're like, you're missing these parts. And they're like, okay, no problem. They send you photos. Do you know what that is? Anyone know what that is? What part of an elephant? There's actually a rhino. It's the neck and chest of a rhino. The top part is underneath here. So next we have the verify. And this is another piece. So the big thing with dealing with these animals is that we have these proportions, right? And that's part of like rigging it a couple of times. And then the next is the model. So it's important to verify the data because we can kind of, it's easy to kind of just go off on this tangent and like, oh, it looks great. It looks great. But yeah, but did you line it up? Did you see how well it lined up? Is it true? Do we maintain the story of this animal? Do we maintain the top, the texture? So we have the suntan spots, the parts where it got caught in a barbed wire and it doesn't maintain its story. It's another way. So this is what Blender 2.8 was able to give me. And then, you know, it upped the game a little bit. Rigging, here we're kind of out of order now already, right? So now I'm rigging. But this is the rig rig, the real rig, legit rig. So here's a turtle that's rigged and you guys all know if you know photogrammetry then you know how rigging works. So we have the big red. The big red is pretty much the influence of the shell. We have the sling-tailed agama. Lizard, it's about that big. And we animate. So animation, we got to call the experts in for this one. We got to call the professional actors. So we have Animal Mocap. This is Savas in Cyprus. He's doing some amazing stuff over there. It's a pleasure working with him. And then we can give it to the machines and let them do their guesswork and maybe they'll eventually hit it right. Or we can do the go old-fashioned way. It's funny to call this old-fashioned. But manually, so we have video reference of the animal. We animate it. Then we work with those zoologists, biologists to make sure that we're savvy. And a lot of it has to do with timing, of course. Like timing and weight, timing and weight. And of course if your anatomy is correct then it makes the other stuff much easier. And I do want to make one kind of backup point which is like a lot of these processes that most artists that work in the industry they'll say like, you're crazy, why would you want to rig it? It's like I put the pivots in there and I hit a button that says skin. There's no options, no settings. I just hit skin and it's skin. So why not? Last piece to share. So right now I don't have any animations for you in this presentation but you can right now on your laptop or your phone you can go and see 50 plus animals at the super high resolutions and you can download them if you want. You can pop them into Blender and you can make your own interpretations of what you see in this. The other part of sharing, pretty critical piece in this is the software. And having used commercial software having spent tens of thousands of dollars on software myself it's a terrifying thing. New version comes out, am I compatible? Hey I want to work with you, I want to work with you. How did you do that? Can we work together? I'm using 2017. I'm using 2011. Open source software not a problem. It's like I can pull out a USB stick and give you all my project files especially when you have open source animals and the software and everything you need even your operating system, your entire operating system is free and accessible and you can give it to other people without having to have a second guess on it. What that does for freedom of speech is just like I put these presentations together it's like okay I'm going to use this image okay who made that? How do I get in touch with them? How do I do this? It's so burying but when you start to bury yourself in open source, open concepts creative commons it frees you to say what you need to say without having to worry about somebody suing you for something that am I really taking away your money? Particularly I want to point out wire.com if any of you are sick of Skype open source piece of software it's encrypted, it's stable it runs on any platform it'll run on your phone via installed app it'll run via a browser if you want screen sharing 30 frames a second I'll load up a video on YouTube and stream the video I'm watching from my desktop with my peers that are in Germany or at University of Massachusetts and of course I think you recognize a lot of these other pieces I'll go over to the Blender add-ons particularly when it comes to photogrammetry but first and foremost for me is interface, how we interface with this stuff for me is important I don't want to click on another list this linear thinking stuff doesn't work for me I want to gesture like a wizard if I'm in 3D I remember being a kid and make something and pull this and bash that and move this together I think gestery, Iron Man exactly, this is what I want and radio menus is huge and the ability to customize radio menus is for me it's priceless it takes away the thinking, it becomes literally muscle memory so then we have point cloud visualizer which if anyone's interested come grab me and I'll sit down with my laptop I'll show you what this thing does it's absolutely incredible the fact that it runs on my little laptop that doesn't have an AMD or an NVIDIA card is incredible to me then we have the last one to add on photogrammetry importer basically that's how I'm able to get my cameras into blender, that's how I'm able to do all that double checking work and I think it was made by a student you're going to be a blend con, he's like I got class, this is mind blowing so the future of the project where are we going here well we're going to keep on making animals for what, but then there's these certain areas that we want to pay more attention to because we get kind of when you're dealing with the accuracy maintaining the model, other things start to slip and we want to figure out ways where we can do this but when it comes to a color checker it's a little bit tricky because the animal's not going to stand next to the color checker while you take your shots so we have to figure out ways to do that we have to figure out accurate ways of capturing translucency or subsurface scattering and then of course procedural textures, which is a whole other level we're looking at this animal, now we're moving to the surface how are we going to capture the cells, the wrinkles the scales, it's like a lot of that stuff you're talking fibonacci and filiotaxis patterns all over the place so if you go crazy with that rubber stamp it's like, yeah, the model looks great from here but the moment you get here you're ruining the truth of it I look at the Agama lizard it's like these spirals that just come out of the center it's the most gorgeous thing and you have to keep that story it's important because through an animal we can see ourselves a little bit in a way that we're not aware of when we see them here another one is putting them in their environment capturing the environment and putting them back to where they were so we can see them because again, without their movement, without their environment what are they? a lot of times you can see the environment you can almost picture how this animal would behave in its environment how it would need to behave to survive in its environment last piece for now is fur is in particularly important at this moment because everyone wants to put it on everything but the other reason is because the next animal that I'm going to be working on is a polar bear and this thing is gorgeous and it makes my heart melt love it so excuse me, thinking about these concepts of open source open ideas and it's like how would this project really truly exist in a way that feels real if we didn't have open source and I believe it wouldn't I went through a lot of times in the industry where I was like how can I keep doing this what can I carry on I'm a Microsoft certified system engineer again I'm a product specialist again so what happens when they fold or someone buys them out and shuts them down, I'm like this information is lost it's trapped behind a pay gate again how can I share it how does this open science how do these open ideas how does this open exploration exist if you need a minimum $2,000 ticket to entry per year another part of it is I often forget it's like it's gone, it's gone my eyes foveate and tighten and my head hurts and I realize I have something pretty amazing in this we all do we have something that ultimately every one of us really has got a computer so what is that best reference what is that thing when I look at an animal that I can kind of test some ideas out to have a better understanding of its articulation it's important piece, I just want to stick it in there it's easy to forget as I walk, I work in a studio half my time and I work with the digital light project my other half of the time and some other little bits on the outside but it's like when I walk through rows of animators who are plugged in and they're making life like this where's the reference, oh it's just on the second monitor right they just when you have this easy thing to forget so that's all I have to say about this besides a big thank you to Blender to Blender Community to anyone that has any kind of contribution in an open source project even if you find a bug I'm sorry, 2 minutes and 35 seconds to do a quick note of that bug maybe plop it somewhere else, hey, I found a bug even small things like that are huge and I'd like to thank you on behalf of Duncan Johnson and Robert and if you want to get in touch with me you can reach me direct through verbal007.com there's a little form or Twitter or go to the SketchFab page check out the animals have fun, play with them and let us know what you learn and if you make something cool please share that as well thanks