 And our next presentation is from Noah's Christian Kahn, talking to the Development of Operational Systems for the Use of AI in Photo Identification and Very High Resolution Satellites, more focused on marine mammals than fish. Welcome again, Kristen. Hello. I'd like to update you on our work applying artificial intelligence to right whales. My name is Kristen Kahn and I'm with NOAA Fisheries, but this work has been a monumental undertaking with researchers and data scientists around the world. And I will touch on several different projects underway. I fly with the Northeast Fisheries Science Center aerial survey team to monitor the abundance and distribution of right whales. The photographs we take on these surveys are matched to the North Atlantic right whale catalog and form the basis for mark recapture estimates of abundance along with many other important research efforts. The use of AI for photo identification of right whales began with the Kegel competition in 2015 to automate the classification of individuals from aerial photographs. We collaborated with the winning team, DeepSense AI, to publish the results of their deep learning approach in 2018. And then in 2019, just before our last right whale consortium meeting, the winning deep learning algorithms from DeepSense AI were retrained and deployed on Flukebook for North Atlantic right whale aerial photographs. You can now access the automated matching system directly from the North Atlantic right whale catalog website. And the system was expanded to southern right whales in April of 2020 with catalogs from South Africa, New Zealand, Australia, Argentina, Brazil, and Chile. As expected, accuracy for southern right whales was considerably lower, a 26% top one accuracy compared to 61.3% for the North Atlantic. Future research is anticipated to make the algorithms more generalizable so that the southern right whale model can more closely approach the North Atlantic model in accuracy. Machine learning is underway to expand the capacity of the system to recognize whales from lateral vessel-based photographs. Wildnee has achieved some impressive results on these challenging images with background subtraction and a state-of-the-art machine learning algorithm named PI for pose invariant embeddings. This novel approach is achieving 55% top one accuracy and 80% top five accuracy on the database of around 3,600 left side head images. This work, to be able to match whales based on multiple features and from multiple different perspectives, is really pushing the boundary on what is possible in automated wildlife identification. Thanks for sharing that, Kristen. It reminds me of a paper I just read recently where they're following individual fish of humans and gillatis with the kind of facial features. These kinds of opportunities are going to hopefully start moving us really up the value chain from not just looking at what people are taking but also their surrounding environment. I just wondered if you've got any, you've got a great story to tell about how you've managed to bring on different competitions and different community to your story. And I just wondered if you could give us some of an idea about the evolution of your discussion and how you've managed to push that forward. You know, just through the time from when it was in someone's head in a pub to where we're sitting today where you've actually got people around the world starting to look at your system for understanding their whales and understanding your world better. Yeah, I'd be happy to chat more about it. Thank you for having me. This definitely has been a long journey, as someone said earlier, and it really stemmed from an early vision of just having one of those days where it was a little bit frustrating trying to match individual right whales and spending hours and hours struggling with the photographs of who's who took a break on Facebook was offered. Hey, would you like to tag yourself in this photo? This was kind of in the early days where they first started doing that. And I was like, no, I did not need help recognizing my own face, but please help me with this right well. And at the time I knew nothing about artificial intelligence or deep learning and didn't really even know it was possible. But I started to think, you know, clearly this is possible in human facial recognition, clearly the technology is there. It's just a matter of me finding a way to harness that technology for my particular scientific application. And so really just a lot of communication and networking eventually led me to another scientist at a conference who suggested this data science competition route as a way to get a lot of different talent applied to my particular problem, which was fantastic because I had data scientists from around the world. You know, as many of them as there are right whales trying all different kinds of approaches to see what would work. And at the time, you know, we knew that we could find a way to identify here's the whale and the photograph but to really match to the specific unique individual whale was, you know, a challenge and in some ways I think excited the data science community they were happy to work on it and so we got a lot of a lot of interest and a lot of people really devoted a lot of time. And then this winning team was able to match North Atlantic right wells to the correct individual with 87% accuracy. And you know quickly became apparent that all the top winning solutions involved deep learning methods. And to be able to package that winning algorithm into a user friendly interface that biologists could actually use was really a significant challenge to. But again with, you know, enough communication and networking we landed on the partnership with Wild Me and their fluke book platform, which had really done a lot of the background infrastructure that I needed to plug these algorithms in. And so that's been a fantastic partnership so that folks are actually able to use it with no need to understand anything about machine learning. And then, as was mentioned in the introduction that the sort of next phase of trying to apply machine learning to conservation applications for me is trying to detect whales from satellite imagery. Very, very high resolution satellites that certainly will not be able to tell me the individual well but in this case I'm just looking for species identification. And there have been various PhD projects, most notably out of Peter Fretwell's lab at the British Antarctic Survey, where they have demonstrated that it's possible to identify whales and satellite imagery. But these early attempts have largely involved a lot of manual processing manual annotation someone really spending months on one image to find the whales. So what we're trying to do now as part of a large collaboration with no fisheries, the Bureau of Ocean Energy Management, the Naval Research Lab, and Microsoft AI for Earth is to take these early attempts at finding whales and satellite images and see can we actually build a machine learning pipeline to actually churn through satellite images and high volume. And there's a lot of challenges associated with that goal, but we are making some some major progress on pursuing that we've been collecting imagery around high density hotspots of North Atlantic right wells and the Cook Inlet Beluga for about a year now. And we're starting a crowd sourced annotation campaign with the Maxar Geo Hive platform to really be able to generate that large annotated data set that's needed to start to do machine learning so I don't know so there's still a long road full of bumps ahead of us on that projects but we're hopeful that we might be able to do something broadly. It is interesting that you've managed to you know bring in competitions I know in Western Australia we were bringing in the community to watch Penguin videos for us and give us numbers and if we had three or four people saying the same number that would allow that to work and we were getting the same number there's lots of different opportunities and it's interesting to hear your story Matt have you got anything to share. Yeah, I'm. It's a fantastic body of work, Kristen and obviously an extremely long journey that you've taken. It's a kind of a fully developed body of work really. What do you think have been the main outcomes in terms of management and conservation from from all the work that you've done. That's a good question I think for the right well photo identification it's primarily been, you know and it's still I feel like it is still on the cusp of it is fully operational now in a user friendly meaningful way. But, you know getting marine biologists around the world to all change their workflows and adopt to these new methods has had a lot of challenges to. So, you know people are sometimes a little slow to adopt so the getting it to really streamline and increase the efficiencies I think is still that's still sort of a work in progress like I think. I think we need a little bit more broader adoption maybe to plug in some of the photo identification to the existing workflows where people have their, their systems and their spreadsheets and their way of submitting data to various repositories that can be fairly time consuming. So I think we still have a little bit of room to grow on that. But ultimately the goal really is just going to be to speed up time and increase efficiency on matching these individual whales. And it may seem, you know and important to know the identity of an individual whale but to back up a little on that, you know, once we know who whale is that really allows for very nuanced understanding of the population in terms of population size. Distribution reproductive parameters lifespan health. So we can really learn a lot from that key data and this artificial intelligence approach gets us there faster. And then this new satellite initiative really has tremendous potential. You know with the increased availability of satellites over the ocean the this new max our Legion program is going to launch shortly with 14 satellites around the world daily over the world's oceans. And so we really have the potential to vastly expand our understanding of marine mammal abundance and distribution around the world. Really, these detailed vessel and aircraft surveys are only done in very small parts of the world, you know around the coast of the US and Europe and Australia, but in the middle of the ocean or around, you know, other parts of the world that don't have the same resources, we know almost nothing about what's there. And so if we can really get some machine learning pipelines to work in connection with a fire hose of satellite imagery really has some incredible potential I think for conservation. Well, that's a great story and I think it teaches us all the ability just that AI is not just allowing the biologists to work but I think it's changing the way people see the world. So people talk now more about an individual whale and they might do about 1000 going to 800 when you lose one that you know the story changes the political importance changes and I think this is the ability for fisheries to get ahead of the game if you're catching one or two charismatic species in a troll net and you can get a camera in there and alert people to what's happening in your troll net and you can let that one go that opens up a whole another social license to operate for the fishing industry. That's great.