 A presentation that benefits by access to big data. This finesses well into our next presentation from Australia Townsville, where I did my PhD. Michael Bradley is working there and talking to the critical issue in our times of big data, which is data integration for actionable knowledge using machine learning. Hi, my name is Michael Bradley. I'm a fish ecologist. I'm here to talk to you about our group, Marine Data Tech, JCU. I'll give you an overview of who we are and what we do and what we'd like to work on in the future. So we're a group that focuses on marine problems, usually biology and ecology problems in the marine environment. We use advances in data science and advances in technology. And we're a hub because we're a central node within our College of Science and Engineering at James Cook University. We bring together a diverse group of people. And we're really all about creating actionable knowledge. So not just using advances in technology to produce new information, but producing information that's robust, that's useful to take actions in the real world such as managing marine ecosystems. We bring together the combined expertise of a range of ecologists, biologists, fisheries scientists, biostatisticians. The group is nominally led by these individuals. And we connect and work with a range of information technology, internet of things, data science and engineering experts. So all up a large group of both professorial and senior academic staff. And we bring this expertise to bear on applied problems in the marine realm. So our role is really about innovation, making sure that innovations in the IT space and spaces like IT move across and help ecologists, biologists and fisheries scientists do their jobs even better. So this is a Google Earth overview of the kind of sampling that we do. So each of those little arrows is an underwater video. So we put out thousands of underwater video samples across this landscape. So going from islands to coastal areas to estuaries and rivers in the rainforests and mountains of tropical North Queensland. So this is an example of the kind of projects that I carry out as a fishery ecologist, fisheries scientist. And this data helps us understand the habitats and resources that fish need to complete their life cycles. And also allows us to monitor fish and fisheries and monitor their health over time and look for any declines in populations. And this is what one of those samples looks like. This is the underwater footage we get. So this technique is great, but there's still a few major impediments to using this technique broadly across the fishery science and fish monitoring space. One of the main impediments is the sheer volume of footage that you end up with. While we like to take thousands of samples, produce really robust data, we end up with a huge amount of sample processing time. So that's one of the first issues we tackled as a group was trying to remove this sample processing time burden. And so we've developed an AI that's capable of removing fish, removing frames from the video that don't have fish in them, which is empty data for us. So here you're watching the 10 second highlight reel of a 30 minute video sample. So we've been able to process that video footage much faster. The AI we developed for this is very broad. It's capable of recognizing a broad range of fish species. And it simply recognizes that there's a fish in the frame and then presents that footage to us for analysis. The great thing with AI is you can peel back the curtain and actually see what that AI is picking up on, and how it's recognizing fish. We can go as far as training AIs that recognize specific life stages of specific species. So we've developed an AI for recognizing the juveniles of this adult mangrove jack. And this is another major issue that AI can solve in using this technique. So we end up with consistent species identification rather than the observer bias that comes in when using human viewers, and especially when using a range of different human viewers to process footage that might have different biases. So there's always going to be bias, even when using AI to process footage. The difference is you can look at that bias, understand it, quantify it, and then account for it. Here's an example of our AI picking up a juvenile mangrove jack in this footage. So we're able to process large volumes of footage looking for this one species, and it's usually able to perform even better than a human viewer. As you can see, this species can be quite cryptic and hard to see. And there are a range of other things you can do to advance the use of video for fish monitoring, such as counting, automatically counting fish. What we've done is we've taken the footage we use to make these networks and made them publicly available. So that is now a publicly available image library known as Deep Fish, and it's a realistic image library. So it's footage that is from these difficult environments to work in the marine environment, a diverse set of conditions, a diverse set of fish species, and lots of movement in the water. So this is publicly available on GitHub, this image library, and we're hoping that it can be used by others to train and test models for how they will actually operate when used in the field for fish monitoring. It encompasses a range of environments from mangroves to reefs, seagrass, kelp beds, those typical environments of near shore coastal waters. Our key needs for progress in this space are sort of two pronged. We would like AIs capable of recognizing the whole suite of diversity of fish species that occur in the environments that we work in, and that can be up to 2,500 different species. And on the other hand, we need simple AI that can operate on low-powered devices. My colleague, Kurt, will take you through one of our projects using AI on mobile devices, which at the moment have to be, those images have to be processed in the cloud, whereas ideally we can create powerful AIs that are housed within the mobile device itself. Another application for simple AI is the enabling of AI within the monitoring device, the actual device we put under the order. So at the moment we go from a camera that takes video to stored video, and we process that stored video with an AI on a GPU in the office. We've been developing cameras that take the footage straight to an AI, that AI sorts the footage and then only stores the important bits of video, those highlight reels that I was showing you before. And this drastically increases the capacity of these devices, so instead of only being able to put these monitoring devices down for an hour, we can now put them down for something like 16 hours. And this is a really exciting space to be working and actually improving the technology and improving our capacity to monitor these environments for fish. So thank you all very much. I'll be keen to take any further questions. Thanks very much, Michael. And as anyone's deal with big data before knows, it's not easy. I mean, even to run this event, getting all the videos and making sure that they have been moved between drives and ready for Ricardo to use was a task. And that's just 30 or 40 videos. So it's great to see people working on the bigger questions of big data, such as what Michael's team is doing, where they're taking us away from place-based data and starting to develop systems-based data, where they're following from range to reef and using information trying to integrate that. So you can imagine the challenges that come. And I think that that's where my question comes to you. What were your major challenges in dealing with your big data? And then secondly, give us a little bit of an indication where when we started to, for example, do the shark beta app, which went onto a phone, the design of the app, which you would imagine would be something that you could sort of do a picture board, story board, and then get it written up in code, was actually quite complex, much more complex than you can imagine, because it has to be intuitive to the user, but it also has to enable some of the technology behind it. So whether they're going to capture images and work on it later when they have access, or as you pointed out, some simplified mechanism. Just give us some ideas, Michael, of the backroom conversations you've had going from a place-based video to a systems-based video and how to move such big data and how you have struggled or what kind of avenues you've taken for thinking about how we develop these user-based apps that people are going to hold in their hands. Thanks. Thanks, Tim. That's a great question. I'm very excited to be part of this conversation. I think my experience has been over the past few years and moving from a solo researcher who would carry out my particular project to working with a large team, and in doing that, finding that those connections and collaborations are really vital to solving all these challenges. So I'm an ecologist. I don't have the expertise to handle a lot of the different problems that face us. And so in working with a range of different experts, I think that's been really, really critical. So working with engineers, working with IT professionals, this has been the key thing that our group has done is connecting these people together to have these conversations. So I guess in answer to your question, yeah, we've moved from moving things around on large hard drives and sending, you know, hard drives in the post across countries to trying to thin down what needs to be sent and then shifting it up to the cloud. So at the moment we're working with a phone-based application for monitoring market-based species across the Pacific and something that's been really critical in those, and in that project is allowing data to move from the mobile device back to our servers in tiny, tiny packets. And I'm not a software engineer, so I haven't been involved in doing that. The nitty-gritty of that, but it's been a key innovation is to allow this data to trickle through in small packets under low connectivity circumstances rather than waiting to put it all on a big drive and then wait months for it to end up in the right place in the right country to do that processing. Thank you. Matt, do you have a question for Michael? Yeah, fantastic work, Michael, and a really broad spectrum that you're covering there with your presentation from mobile apps to IoT underwater. And I think it's important to highlight, isn't it the low cost of a lot of that IoT technology and a simple Arduino board or Raspberry Pi perhaps with a camera attached to it? And how accessible that is for anybody with a bit of technical knowledge in fact, so downloads and code. Do you have any plans on developing that also to be accessible? And it'd be nice if there could be like a how-to somewhere online, you know, build something like that if people are interested. Yeah, certainly. So that particular example has been published as a sort of a how-to guide. So that's how to build that camera with the Raspberry Pi setup. It wasn't work that I led myself, so I'll be able to share that paper with anyone who's interested. Yeah, that's really our vision to bring these technologies and then turn them into methods, you know, peer reviewed, understood methods in ecology and in fishery science. So to make that translation. The really interesting presentation this afternoon with Dr. Gianpaolo Coro from CNR. It's worth watching. He covers an IoT devices underwater project that I worked on with him, so I would say it was interesting, wouldn't I? But we're now covering 360 degrees and sizing fish on the fly with prototypes that can last weeks and considering the data that we're gathering, the cost is pretty reasonable, to be honest. That's fantastic. Yeah, it's incredible what you can do. And I think, yeah, just to reiterate that I think we really need, you know, we need the right people in the right places in this big collaboration between groups to make sure that the IT is being performed in a way that fulfills the needs of the ecologists and their sort of needs for rigor and particular scientific method. So I think that conversation is just a really, really exciting back and forth to hone in on these cheap, useful methods that can be used across the world, yeah.