 Our next presentation is Paul Clarkin talking to AI for Data Division Equatic Species Identification. Welcome again Paul. Hi everybody, my name is Paul Clarkin and I'm a PhD student at the Virginia Institute of Marine Science. I did my master's in shark taxonomy and I've actually run some workshops for FAO on shark identification. So today I wanna talk about why I think AI could be the future for shark identification taxonomy and fisheries management. Sharks are poorly managed and that's mostly because they are so poorly known. Half of all shark species are data deficient and this is because some of these species are very rare and also because they're hard to tell apart so it's hard to get a lot of information if we can't figure out what species they are. Sharks are currently being discovered at a rate that exceeds any previous time in history even faster than the beginning of the Linnaeus naming system and as a result taxonomy and being able to identify sharks is more important than ever before. However, the way that we identify sharks is mostly based on traditional taxonomy, the same thing that Linnaeus was using and this requires an expert in the field that there is some literature that people can read on how these sharks look to identify them but it's very dense. The training guides are often a little bit verbose and hard for people to use in the field especially if they're at sea and because of this there's a lot of misidentification and this can actually be very harmful to life history estimates because if you have different species all mixed up and you have different estimates of how old they get and how much offspring they have this can really mess up our understanding of their life histories and make policymakers make the incorrect decisions to protect these species. So I'm gonna do a quick overview of kind of an example of a group of sharks, the ghost sharks and how complicated the taxonomy can be. Carl Linnaeus actually named the first ghost shark in 1758 and his description was highly qualitative. He mostly talked about the conical shaped snout, the large spine, the large eye, the tapering body and this worked for his description because it was the only species known for that group. But this was kind of like describing the first bat. A bat is pretty different from a mouse, it's pretty different from a turtle but they're not different from a lot of other bats. So in the mid 1700s it worked and then about almost 100 years later another one was described but you can see that a lot more species were described and now we have over 50 and they're pretty hard to tell apart. Color is not really a reliable character for this group and so a lot of the time you're trying to describe the relative shapes of the fins or the body. Dominique Didier in 2002 published what has been kind of the standard way for measuring these ghost sharks with a series of measurements and I actually was lucky enough to work with her and I was surprised that she said a lot of these measurements don't do anything. They take them, they're what's being used but they're not very descriptive and the way that she tells them apart are largely on qualitative things like stockiness of the body or how skinny the tail looks. So I actually worked with her to come up with a series of measurements that would better describe ghost sharks and be able to discern between species. So we came up with nearly 100 measurements and I took them for about a little bit over 100 specimens and you can see that they actually could tell the species apart pretty well. We could look at the individual measurements and how well they define the species so you can see there is some overlap but more or less they fall out separately and then what we wound up doing is we ran an analysis to see how much each measurement contributed to a species being dissimilar to other species and we were able to rank them for the most important characters to the least important characters. So I think this was a great study. It was a first attempt to quantify the efficacy of measurements for defining species and it's useful but it's mostly useful in the lab. It's hard to take a bunch of measurements at sea on a moving boat and it also kind of requires an expert. You have to be familiar with what the measurements are and how to take them. Plus there's still a kind of the aspect of just having an eye for it. There's certain qualitative things like the curve of the spine, the waves of the fin and that would require actually taking measurements of the angles and that's just not happening at sea. So while I think this is a good step I think artificial intelligence could be the next step for being better able at defining these species. So I think a lot of us are familiar with Google's on supervised training of artificial intelligence to identify cats. What they did was they just took a whole bunch of images of cats. They dumped that in the training program and they were able to create a neural network that could identify cats or not cats. In this case, a cat or a dog. And I think we should start trying to do that with sharks. So we could do a cat shark or not cat shark like this dogfish. So the reason why this could be a little bit hard is you need a very large data set. You need lots and lots of pictures but we are working with the fishers and we have access to a large volume of pictures that they've been taking for a while and they're volunteering to continue taking for us. We'll have the ability to identify these sharks. I've been working in this area for a long time and we have the advantage of not only having the photographs but having taxonomic description and we're gonna have a genetic barcode for each one of these. So we're gonna have tissue that we're gonna run the genetics for and keep a physical specimen in a museum collection. And with this it will help us better identify these sharks. I've worked for the UN in identification workshops and there's a lot of information at once. Some of the catalogs and identification books are a little bit dense. And so I'm hoping with this we can back generate some better distinguishing characters and make it easier for people to use in the field. I think it will complement workshops and guides and help people better able to identify these sharks. We can do it from a photograph being sent in from the field and hopefully down the line we're gonna make it so you could actually use it in the field. And I think this better identification will lead to better data coming from non-experts and that will lead to better life history information, a more reliable estimate of how old they get, how big they get and what kind of human fishing pressure they can withstand and that will lead to better management. Right now we are in the very beginning steps of this. We are still working with the fishers. We're acquiring the photographs and I am very happy to talk to you if you have any questions or you're interested in collaborating. Thank you very much, Paul. My question to you is a little bit coming from the experience we've had over the last few days and working from those keys and books towards systems which might be a bit more intuitive. You probably heard of eye shark fin which again try to triangulate across different sections of a shark to understand it's fin. And I'm wondering if there is an opportunity to not just take the pictures that you're looking for and get the DNA barcode but also take macro shots of different parts of the shark. So we've been hearing today about how adding tail and head shape to a recognition software is allowing sharks to be more quickly called down, sorry, tuners, more quickly called down to which species they are even though the video recognizes that it's a tuner very quickly. And I just wonder how much experience or what type of crossover you're getting from other people's experience which might allow you to, for example just taking pictures of an eye or just taking pictures of a tail along with the other types of pictures you take of the whole shark. And see how potentially these allow you to get to where we wanna get to more quickly. Thanks. Yeah, that's a good point. And there are parts of sharks that are important for telling a part of the species but that's often like in the groups. It's kind of when you identify sharks you kind of go through kind of like the key if it has this fin or that spine you separated. And in some groups towards the back of the tail there's a mark that is helpful or in some groups it's the angle of the arch of the mouth. And so it would be probably useful to have in combination with the classic full lateral some other images. And that would probably mostly be group specific whether it's a guided by an app or if you just take a suite of photographs and we're able to use those. But there are definitely some weird things like that. I've noticed when the sharks are still in the net you can see through the mesh and I'll know what species it is by looking at a piece of the snout. But I'm like that's really not very descriptive but I think being exposed to a lot of them you wind up noticing that kind of thing which AI could definitely pick up on. Yeah, I'm just seeing this from the perspective of deep water sharks because we just don't get the numbers on deck so maybe we need multiple shots to help us to core down. Max, can you give us some any questions to add to this? Yeah, I'm really fascinated by this progression of species identification from traditional measurements through to AI looking at a biometric data there. And it's kind of interesting because AI can kind of discover traits which might not be which a taxonomist might not use. It might discover some characteristics. I was wondering if you'd explored looking at point cloud data from multiple images because the actual geometric curves around the surface between key points for example might be good indicators which might provide more information and how do you consider using point cloud data for example from the photographic view? I haven't but that's a fantastic idea. I've thought about trying to do like a kind of cheap 3D scan with a bunch of pictures that you stitch together but point clouds would be interesting especially if you can label them as points that can be easily re-identified at the beginning of the spine or something like that and then have all the measurements to identify what actually contributes to them being discernible. That's a good idea. I haven't thought about that but something to consider. Got another quick question from Anton. Yes, a really quick one. So how do you deal with gravity? Because a fish in the water looks very different from a fish on a taxonomist table, no? Oh, yeah, yeah. So I mean, I think it would mostly be a tool to be used on the deck, by an observer for the fish that are coming up. So that, I mean, I think there's a lot of sharks that are getting caught out there and we're not getting the data back because they can't identify it and even if they take data, sex length, maturity location, that data is useless if you don't know what species it is and it's even less useful if it's a mix of different species. So if we can start using it as a tool for factory workers or observers to be able to tell what species it is and collect those data points, that's fantastic. The idea of identifying it in the water would be, I mean, would be fantastic and think about how many deep sea surveys we could accomplish with much less funding. But yeah, that's probably another hurdle to overcome a fish swimming away into the darkness as opposed to on the taxonomous table. Matt, that's your camera. Just a quick question from Anthony. He had his hand up, so I'll just check them into the mix. Thanks, I appreciate that. Just a couple of comments and a question. So given that you have a bunch of data and images and you have labels for the species in those images, you can just try the Yami. So you can, it works directly for that problem. So you can see how well the existing deep learning classification methods in the Yami work for your sharks. People have used Yami to distinguish now between hundreds of different fish species. And it's easier and harder, as you know, just from your own taxonomy work. Some species are really hard to tell apart for a human and some are really easy. And that's true for the algorithms. So you might just try it and see which species are hard to distinguish from others and which are not. Deep learning, as Matt mentioned, will automatically learn the features that it's using to distinguish them. There's nothing in the Yami right now which will tell you whether the algorithm is paying attention to an eye or a fin or some part of the animal. But we're adding those features. That's called explainable AI. And it would actually tell you which part of the animal is used in the classification decision. Those capabilities exist in general, but not yet in the Yami. So something to look forward to. And the last part about having only a few instances, a few images for a given species, that's a really hard problem. So when you have very few, like just five or six images of this one species and being able to recognize that one, that probably won't work well in the Yami. That requires methods for these few labels problem, which is still more of a research problem. So those are always going to be harder still. Okay, we'll say, I wrote down, I'm very excited to learn more about the Yami. I have a little note over here. But yeah, that's very exciting. Thank you very much for that presentation Paul. And thank you for throwing explainable AI into the mix, which takes away the black box story and allows us really to start exploring.