 And our next presentation today is from Thomas Galan of Sattling R&D, he's the Sattling R&D Manager. We'll be talking to machine learning integration in electronic monitoring for fishing fleet management. Thank you for inviting me to this FAO Artificial Intelligence for Digital Blue Planet Conference. We're very happy to be here. I'm Thomas Galan. I'm working as a R&D manager in Sattling Company. And well, I'm going to speak about challenges and goals of machine learning techniques to identify species on board fishing vessels. So I'm going to share my presentation now. So as I was speaking, this is the title of my presentation. So hope it helps. For those who don't know Sattling, Sattling is a Spanish company. It's a technological company founded in 1992. Specifically for this conference where I'm going to speak about electronic monitoring business, that we have more than seven years of experience and more than 250 systems deployed around the world. About Sattling, just to explain a little bit more about what we do. We have different solutions for governments and regulators, solutions specifically for the fishing industry, and solutions about satellite devices that we can install differently, even if it is on a fishing boat or even in the earth. Our experience in EM, we have more than 250 onboard installations and more than 100 train of service and nine data review regional sensors with all the infrastructure deployed on the place. So what is C-Tube and what is the solution that we have? I'm going to be very quick on this one because this could be more than 30 minutes of slide, but on the main focus that I want to do is that the Sattling C-Tube, it is a piece of hardware that you can install on Vodabessel and it's recording that or its data that is recorded into the C-Tube, it is encrypted. All this information could come from an IP camera or a sensor unit or satellite devices or it could be even transmitted through satellite and even with a GSM 3G, 4G or even 5G in the future for transmitting information into a server, a land server. So the key question here is that the analysis part could be a very expensive part because you need to go specifically to analyze the moments that you want to check on those videos. So we've developed machine learning algorithms to try to find out what is the specific moment or the interesting moment depending on the analysis purposes. So these algorithms at the end will reduce costs during EM analysis. So we have a set of algorithms. We've divided into two big categories. One is for long-line fishing and the other one is for person fishing. For long-line fishing we have two algorithms. Those are the setting, hauling detector algorithm in that it will detect when and where the setting on hauling operations took place. Let's say the start and the end moments and those are based on a specific neural network based on GPS and other parameters that we include in the C-Tube part to extract data to be easier afterwards to detect these moments. And the other long-line fishing algorithms would be the settling capture algorithm. I'm going to show a video in the next slide. This is for identifying target and incidental species just to identify the capture event, not to describe it. And the other algorithm that we have, it is a person fishing brain detector algorithm just to know exactly when a braille has moved with fish. So then all the fishing effort would be easily detected. So as I was saying, this is the capture long-line algorithm. You can see that the algorithm has detected a fish on board a vessel. Even more, you can see that the fish, of course, it is moving. There are people hiding the fish and the neural network is still detecting that fish. But even more, if there's more than one fish on deck, the algorithm will detect it as you can see in this specific moment. But this is detecting fish. It's not classifying it. So the challenge for this is that if we ask them, okay, what's next, then it would be the classification. So we know when, let's say, we know when a fish is on board the deck, but we want to know what or who, let's say, this species. So this is the settling species algorithm that we are just finishing. And we have all these results here. We have a big data set, more than 200,000-leveled images that we've labeled specifically saying which species are and everything. We have 13 different classes to classify the fish detected. For example, tuna, skipjack, jellofin. Of course, skipjack and jellofin are tuna-like, well, are tuna families of fishes. But we have a specific category for tuna if it is not really clear between skipjack, jellofin, big eye, or whatever. Of course, we have other categories as catch or bycatch events if they are not target species. Our success rate, it is depending on each class. Tuna-like fishes between 70 and 93%, as you can see in the slide. That's, for example, rays or rays between 81% and even 99.9% of classification with the previous algorithm run. The success rate differentiation target versus incidental species, it is the 90%. And this could be the most interesting part of this algorithm because then you can go exactly for the moment for the incidental species target. So let's see how it works on this video. As you can see here, the video is showing two species. First, it's shown very bad values. But now the value is this big eye and it is saying that the second species could be yellowfin. So the algorithm, it is working fine because it is detected that this is a big eye as it is. But of course, depending on the angle of each image that the algorithm is analyzing, it could detect different species. So what are the challenges of developing this algorithm? These challenges, for us, we've did a very big investment while training the network. We need to pay a lot of money to people to identify exactly what the species are. It's cropped fish on the image. And of course, we need to create a very big and debug data sets. One of the problems that we had, and this is something that everyone that starts with this, they can face a lack of images on a specific incidental species. For example, in our algorithm, turtles, we identified that we have very few images of turtles. So then the neural network is not very well trained on specific species because we don't have all that images. And of course, we need to have a balanced data set. So the challenge would be to have as many as incidental species as target species to train better the algorithm. This is a sentence I wanted to write that it is that if a human cannot classify it, a computer will not be able to do it. This could sound funny, but at the end, it is what happened. We cannot imagine neural network as a magical thing. It is trained with our knowledge, so if we cannot classify it, the computer will not be able to do it. So I hope this small chat worked for you and I hope I can clarify any other questions that you have. So thank you very much. Thank you very much, Thomas. My question regards some of the challenges of shrinking down your data set considering you've got cameras on boats, many boats collecting over time. Are the algorithms working on board the vessels so that they can already snip out the footage that you want to keep, or are they collecting everything and then you post-process to find the events once you get the imagery back to shore? Hi, everyone. All these algorithms require a lot of processing power. This is usually done using GPUs. These kind of devices, as I said, they consume a lot of power. It is very difficult to integrate them into a small-size equipment that it is going on board a vessel at some point with some limited space and limited power that the vessel can give to the unit. So we are working on two different ways on integrating all this machine learning and the running of the algorithm. First, it would be extracting the video and then analyzing it into a land server that could have better power and better space to have it with all this ventilation and temperature checks and everything because all these devices got very hit. The second approach that we are testing right now because the first one is something that we have already in production, it would be to install some equipment that has less power than a GPU on board a vessel and then try to analyze and identify when is sufficient activity being done or whatever and then extract those videos using 4G or VSAT connection using a satellite and then analyze it on land. So it would be a device that could be installed on board a vessel then it will consume less power and the output it will be less precise. This is something that it's called edge computing. So that's something that we are doing some research right now but at the end, if we want to... the results you show in the presentations are done using a GPU in a land server. Thank you very much. Matt, can I hand over to you for any question please? Yeah, sure. You said about the species with the limited images. Did you have any ideas about solutions to tackle species with limited data that they're claiming? Sorry, Matt, but your audio for me was not clear. Sorry, apologies. Just asking, do you have any ideas of how to tackle the species with limited data tech of the training images? Yeah, well, this is a problem that we are facing a lot because as I said in the presentation the idea would be to have a balanced dataset. So in the presentation I was referring specifically to turtles that we have the proportion between tuna-like species in our field and turtles fortunately is not the same. That's because they are not catching turtles for fishing as a target species. So the idea would be to extract from a video clip thousands of images because right now we are people having and stunning and labeling and identifying inside a video. Okay, here is a tuna or yellow thing or whatever and then we take that specific image and then we label it. With that mechanism we've reached near 200,000 images. But what we can do is to extract some seconds of that video and extract each of all the frames that compose maybe a 10 second video or whatever and then we can multiply the dataset to increase the quality of it. So that's one of the ideas but of course that's something that we must invest on to improve the algorithm and learn that unbalanced classes. Yeah, it feels very interesting. Another option, I was training a model with John Polo and I work in some 3D animation packages typically used for film production where there's photorealistic rendering, texturing and we can make things wet and dry, dark or light. And I actually rendered up some video scenarios of fish where I knew that it could challenge the algorithm and Delibriton presented them as difficult scenarios faking the actual, you know, the fish didn't exist and it was a really good way of training them. So even like we've got a presentation this afternoon on cryptic shark species and some of them, you know, we've only got a few photos but 3D rendering actually does offer where you can take a background image and obviously what you see when you see Jurassic Park or, you know, a film with an animal in it. In fact, I worked on rendering animated prehistoric fish on a production about 25 years ago. But you can simulate these images and present difficult scenarios deliberately to your model and challenge it like that which is kind of an interesting approach. Yeah, well, our idea is not to generate a big lab on board our fishing vessel, okay? What we want is just to have some, you know, EM equipment installed and not to modify how they fish or how the operation is made. So that's why we want to keep it like that because we don't want to interfere with the fishing. But even with that we're facing that some images that are extracted that they allow us to use for the development of these algorithms. The camera eyes, you know, with wider drops and it is blurry and everything because we don't want them to be every day, you know, taking care of the EM. What they do is to fish and EM is something apart. So that's the image quality with wider drops and everything. It could give bad results on the neural network, right? Because it could explode everything, you know? Sorry, Kim, can I, very quick, maybe very stupid question to Thomas. I like your question. I'm way not as advanced as Matt with his image rendering, but why don't you use like a plastic turtle to test your algorithm? Well, it's anybody I've heard try to do that. Well, it would be in our case, of course, we have several hundred images of turtles. But the answer for this is that maybe we can find out some false positive detections. This is because, at the end, the neural network, they try to learn what is a turtle by example, okay? So let's imagine this plastic turtle gives some brights because of the light or whatever and the plastic, you know, texture. Then if a real turtle doesn't have those brights because of this material, then it would be, we will be contaminating the network presumably with this plastic turtle. So it is a good question. It wasn't a stupid one. And this is something that we've spoke a lot in the company. Try to, okay, let's do a mock-up turtle or tuna or whatever with a toy or whatever. But it's not, they say it's not working as good as it could be, the idea initially. I'll just answer you as well, Anton. In a rendering package, you don't have a cost of going on to the boat and throwing a plastic turtle around. And also there are controls such as subsurface scattering, which are very fine controls of how surfaces respond. And you can get extremely close at a macro scale with a simulated texture and contemporary rendering packages. The simulation, the physics simulation of photons is extremely comparable in comparison to a plastic turtle. It's cheap as well. Before we go off down the plastic toy making or rendering avenue, let's carry on this presentation. So thank you very much for yours and Sat Link's work talking to that. Thank you.