 Bingo! One o'clock rock on a Monday, every Monday. Research in Minoa. Today, we have Dr. Mark Ronsted. He's a specialist in HMRG and a design engineer. Design engineering, and that stands for... Wait, I'm going to tell you. Hawaii Mapping Research Group. HMRG. Very important because we're going to talk about that. And that is at the Hawaii Institute of Geophysics and Planetology in Sowest at the University of Hawaii at Minoa. Mark, he joins us today to tell us about a special competition that is happening here in December. Mark, thank you for joining us on the show. Oh, thanks for having me. How'd you get into this? This is really interesting. I mean, it's a vehicle. It's an autonomous vehicle. We're going to see pictures in a minute. Why didn't you bond up to this particular technology? Well, I've been working on marine technology for a long time, mostly with sea floor mapping sonar. But I've also been helping students with robotics competitions, like the first robotics, the mate ROV competition. And so when we heard that the Office of Naval Research and AUVSI, which is Association for Unmanned Vehicle System International, I never remember the title, but they're the sponsors of this competition. International, yeah. The Association for Unmanned Vehicles International. Yeah. They are the sponsors for this competition, and there's a number of corporations who are also helping to bring the competition to Hawaii. Okay. And UH is going to put one in. Yes. You have one ready. Your students, your group, your team, ready to go here in a few days. And so people can understand what we're talking about. Let's take a look at some footage we have of what this looks like, this vehicle. You go, wow, that's an impressive piece of gear. Why do I want one? Who's great, right? By default, I want one, but why do I want one? Well, more and more different vehicles are getting autonomous capability. The ability to have a machine do something that might ordinarily be done by a person that would be dirty or dangerous or boring. And machines, if they're set up properly, if they have the right sensors, they can do a lot of things very quickly and safely. And this is intended to get students interested in working on these technologies. And so the competition is all autonomous. The vessel is a bunch of sensors. It's programmed in advance and just set loose in the course and needs to carry out a number of tasks. And we can kind of go through the individual tasks. Why would this be, this has got to be a connection, not at the engineering school, but at HIGP. Oh no, this is at the engineering school. So this is the engineering school. You happen to be at HIGP. That's right. But you're just crisscrossing your disciplines, that's all. Yeah, I'm basically helping to mentor the team. A fellow who's a professor in mechanical engineering named Zach Taylor is kind of the faculty lead. He's at the College of Engineering. Yeah, he's in the College of Mechanical Engineering. And there are about 20 students, undergraduate students who are on the team, engineering in mechanical, electrical, different fields, computer science. And then there's also a graduate student that helps out with running things. So I'd like to talk about the device for a minute. This is something that you guys created, the College of Engineering and you as the leader of this. When did you start working on it and how did you create it? I mean, how did you do the essential design of this vehicle? I mean, you've got to have it designed to make it move through the water and not fall over or sink. So where did you get that from? How did you design that? Well, for this competition, all of the teams have to use the same vehicle. And actually there's a picture on the monitor right now. So they're all using this. Who makes this? It's made by Marine Advanced Research Incorporated. They have a range of vessels that are this sort of design. It's a catamaran with a suspension for a load platform. So it's a load platform that makes it... On the load platform, it's going to be the stuff that your team is putting together. Yeah, the sensors, the control system. And then there are propulsion motors for the university team attached to the back of the vessel and also to the front. Okay, so you got the basic structure, we can see it. That comes from somebody else. But the platform includes all the electronics, all the smarts. It also includes a propulsion system and includes all the navigational connection between the computer and the smarts and the propulsion system. Right, right. And so in the competition, there are... Gosh, I think there's about seven different tasks that the vessel needs to do. Like what? Well, I guess can we go to... Yeah, this is the first task, which is just kind of proving that you have a functional boat. It needs to be able to pass in between those two buoys, the red and green, and navigate for 10 to 30 meters and then exit between a pair of buoys. Now, this is all done autonomously. There's no one remotely operating the boat. So the boats typically have one or more cameras so they can see these buoys. Also very common are LIDAR systems and the UH boat will have a LIDAR system. It has a LIDAR to measure distance to buoys. And color. Yes. They have to tell the red versus green color. Right, right. They need to... That's pretty sophisticated. But why... You know, maybe I would cheat on this by saying, well, just point the boat directly through the buoys. Put it on a course. Put it on a course of exactly so many degrees and put it straight. Make sure it goes straight. Use whatever inertial guidance or satellite control and make sure it just goes straight. They don't get you... I mean, you could do that, although the position of those buoys will not be the same from day to day and possibly not even on the same day. Oh, so it has to read off the buoys. Right, right. So it does need to be able to recognize those things. So, okay, so these sensors, the camera is going to look at the shape, find the buoy, and then it's going to look at the color and figure out what's the buoy on the left or the right. And then I have trouble conceptualizing this. And then it's going to send signals to the navigation propulsion system to stay on course. These are the buoys it has already put in its memory. Yes. Actually, could we go back one picture? Yeah. This is sort of the first main task. The team gets assigned a sequence of colors. And they need to have their vessel, their boat, go with circles around these different colors in the right order and are in the right direction. And there's also a bunch of black buoys, spherical buoys, and you have to avoid them. If you hit one of those, then you get points subtracted. So this is kind of a big task for recognizing the different colors and shapes, avoiding the round black balls and going around the colored ones in the right sequence. Well, so, I mean, let's hold on that chart for a minute. The one with the colors, yeah. So as a programmer, I can program, I would write a program for this. And I would say, well, you head toward that first buoy over there, but you don't head directly on. You head slightly to the right. And then when you get close enough, then you make a loop around it. And here's how to make a loop around it. When you finish with the loop, then you look for the blue buoy. And then when you get there, you don't head dead on to the blue buoy. You make a clockwise turn around the blue buoy. And when you've done that, then look for the yellow buoy. I can see lines of code here telling you what to do. Is that what happens? Yeah, yeah, that's what happens. The sensors, the cameras on the boat are used to look for the different colors and shapes and figure out where the buoys are. And the LiDAR helps to determine exact ranges. Sure, you have to have a range, so the cameras can't help you with that. Well, I mean, if you had multiple cameras, you could do binocular vision. But the LiDAR is, you know, it gives you an exact number and range. So you have to coordinate the cameras with the LiDAR. Right, you've got to take all that information in. And there's a process known as simultaneously mapping and localization, which is actually used in a lot of applications. When you come to an area you don't know anything about it, you start mapping it using your sensors to build up an image of what's there. And then as you go into the area that you're mapping, you can extend the map and also use what you've determined from the map to figure out where to go. And in the map somewhere will be the blue or the green or the yellow. Exactly. Of Pui, and then that'll help you to determine a course. Yes, yeah. And then, okay, so the machine, I love this stuff. So the machine will have a course then. It will actually achieve a course based on the mapping. And it will be able to tell the propulsion system, go left, go right, and this is how long you go, and it'll be verified, verifying that you're not passing too far ahead of benchmarks, right? Right. Going toward a given point. But when you reach a certain distance with the lidar, then you do this and do that. Yeah, yeah. But it could fail, right? Well, it can. What kind of failure would you expect? I mean, what is possible? I mean, one of the things that the team has had problems with, and I should say that it's the students who are doing all of this programming. Yeah, what fun. Yeah. But we found that these colored buoys, on an overcast day, they show up really well. But if it's a bright sunny day, and especially if it's later in the afternoon when the sun is low in the sky, you get a lot of backlighting, and it's really hard to tell what color the buoy is. Well, then you're out of luck, huh? Yeah. Well, what they're talking about is putting colored filters and having multiple cameras. So one would be sensitive to green, one would be sensitive to red, and using that to try and subtract some backgrounds. It's very creative, isn't it? Yeah, yeah. Yes, with that, yeah. So then you put all those in there, and then you get a composite picture, and then you really know where the yellow and the red and the green are. And then you take that and feed it into your course plotting program. Right, right. And so you have, okay, so you have cameras, they're acting as sensors feeding into the mapping, so to speak. You have a lidar, and the lidar is what, the lidar is also another piece of data that's coming in. Yeah, another input to making the map. Okay, and then you have the central processing, I imagine, which takes all this in, and it has its instructions, as I was outlining before, to the course that you want after you get this external data through the sensors. Right, right. And then it builds a navigational course which has time, speed, direction, all that thing, all that navigational course is supposed to have to get you there. Right, right. Wow, that's really sexy. You must love this stuff, too. Yeah, I guess if you want to go to the next picture. We're going to go to the next picture right after this break. Oh, okay. Because I think by now whoever is watching is going to get a little excited anyway about what we're doing here, and they want to be on the team. You might have 10,000 people on your team. That's Mark Ronstadt. He's a specialist in HMRG design engineering at the HIGP, University of Hawaii. We're talking about this competition at HMRG coming up this month. Very soon, we'll be right back. Hi, I'm Stacey Hayashi with the Think Tech Hawaii Show, Stacey to the Rescue, highlighting some of Hawaii's issues. You can catch it at Think Tech Hawaii on Mondays at 11 a.m. Aloha, see you then. I'm Ethan Allen, host of likeable science here on Think Tech Hawaii. Every Friday afternoon at 2 p.m., you'll have a chance to come and listen and learn from scientists around the world. Scientists who talk about their work in meaningful, easy to understand ways. They'll come to appreciate science as a wonderful way of thinking, way of knowing about the world. You'll learn interesting facts, interesting ideas. You'll be stimulated to think more. Please come join us every Friday afternoon at 2 p.m. here on Think Tech Hawaii for likeable science with me, your host, Ethan Allen. Back, we're live with Dr. Mark Ronstadt of the Hawaii Institute of Geophysics and Technology. So you would give me the universe of all these organizations and how they work and who gets together in what organizational chart, so to speak, for the competition. Tell us about that. Okay, well, the AUVSI is sort of the main company or organization, really. And the funding comes from the Office of Naval Research. I believe Northrop Grumman is a major sponsor. And then here in Hawaii, Navatech is a big sponsor. They've been helping out a lot with the competition. This is their cup of tea. And then there are a bunch of organizations like the University of Hawaii System, Honolulu Community College, that are contributing at least in some way or another. The UH team has gotten some financial support from the local chapter of the Marine Technical Society and the local IEEE chapter. So there's a lot of people involved. But it's basically to put together this competition that will start a week from today. First time for UH. It's the first time for UH. It's actually only the second time the competition has ever been run. Let's take a look at last year. We have some footage of how it was last year. Yeah, this is actually two years ago. The competition is every two years. So these are the people who were involved last time. This is not in Honolulu. This is somewhere else. This is in Singapore. Singapore, okay, you mentioned. It's raining, of course. Yeah, you can see some of the buoys out in the water. These are teams working on their robots. And this is for the press, I guess, this was made. It's the same kind of boat structure. Yeah, very similar. And you can see all the gear piled up on the top now. We can appreciate that. Yeah, there was one boat... 2014 recap that I saw that actually had a radar mounted on it. You can use anything you want. Is that the idea? Yeah. So if you can find additional sensors that would help in your programming, you can be on top. Yeah, this is another of the tasks that will be in this year. The vessel has to dock autonomously based on signs at the head of each dock. It's like a parking technology for these new cars. Yeah, yeah. Oh, there's one of the buoys. Yeah, yeah. That's another aspect of the competition where the buoy flashes different colors and you have to recognize the sequence of flashes and that determines what you do in the next stage of the competition. They're going pretty slow. Could they go faster or is this not important to go faster? Well, it's not that important to go fast. You want to go precise more than fast. You know, I think some go faster than others, but you get points taken away for running into things. I'm sure. And so most teams try to go at a reasonable rate, but not so fast. Not when they can correct. Yes. And I'm sure they got correction program also. If they find they're off course, you want to be able to correct. Yeah, exactly. So now what is it testing? It's looking. It's thinking. Yeah, and sometimes running into things that we're supposed to avoid. But as you can see, there were teams from five different countries. Singapore, Australia, Japan, South Korea and the U.S. who were in the competition two years ago and all of those countries are represented in the competition that will be here in Hawaii starting next week. Okay, so yeah, I guess we're about finished with it. Yeah. I just want to pursue something you started with as it's coming up soon. It's coming up like in a few days. Tell us how the competition runs and where and whether we can go and watch. Yeah, the competition will begin next week on the 11th. Although the first five days are basically for the teams to get some experience operating their boat in the actual company venue. Practice. Right. They're not necessarily being graded for that. They're just sort of getting organized. Yeah, they're practicing. The real sort of nuts of the competition will happen on the weekend of the 17th and 18th. And that's when the teams will be going for scores and by the end of the day on the 18th the winner will be announced. Okay, and who are the judges who will determine that? You would judge? No, no, I'm not a judge. I'm involved with the U.H. team. That would be a conflict. Yeah, so we're kind of kept at arms length from the people who are running the actual competition just so no one can say, oh, the U.H. team has an unfair advantage. I mean, we kind of do in that we don't have to ship our boat to a venue far away. It could get broken or who knows what of the shipment. But as much as possible we're trying to be just like any other team. So I actually don't know exactly who the judges will be. I'm sure some will be from AUVSI and from Office of Naval Research. They rate each task, whether this team was able to handily successfully perform this task or that task and give a point score and then it must be averaging and tallying up the point score and then the moment the highest point score is the one that wins. So I just wonder how... This is an order for as far as Office of Naval Research is concerned in order to achieve better navigation, better perception of the environment, better way for this device, this boat to know where it is and how it's going to do this and that in a completely autonomous way. That's right. Would you say that we are already, we meaning all of us are already at a point where we know a lot about this or are we just starting to learn it now? Well, there's a lot of work that has been done on different aspects of this. Things like computer vision, being able to recognize objects in a field of view, using different sensors to come up with a map of an area. This competition is pretty unusual in that it's putting all of that together and the work is being done by undergraduate students. Well, you know undergraduate students, they're unfettered, right? They'd be very creative, especially about coding things. So maybe it's a good thing. That's why ONR wants it that way. I mean, I think the underlying motivation is to get students interested in this kind of technology, get them skilled in this kind of technology and have people who can do this kind of work in the future. It's an irresistible thought to think that this kind of technology would help enable vessel, might even help a weapon to get where it had to go and do what it had to do. And I mentioned before the show there was some similar exhibit that was shown a few years ago. It was by Mark Margo... Right, and Brian Bingham. Right. Same people. Yeah. Mark Margo Edwards and Ryan Bingham at Snugg Harbor, the University of Estonia in Sand Island, to... it was a smaller, much smaller vehicle and it was using sensors to sense chemicals. That's right. And other things that might be antigens, you know, in the water, might be terror, terror weapons and all that. And it was not... it was not autonomous though. It was controlled by radio. Well, I mean it was semi-autonomous in that you could give it a sequence of waypoints. Yeah. It had a GPS receiver. Yes. And it would go and follow that sequence of waypoints. So you could set it up to map an area. Yes. You're recording sonar data or chemical data. Yeah. But if you gave it a waypoint that wasn't in the water, it would try to go to it and run into whatever got in the way. Yeah. So we do have some experience in this. Some, yeah. You know, we know how to put these components together. But I guess it depends. I mean, if I gave you a million dollars to do this competition, you could get more components, right? Oh, yeah. Yeah. I mean, cost is a big part. You know, the UH team doesn't have an infinite budget, unfortunately. Yeah. And so finding things like a LiDAR that was inexpensive enough that the team could afford it meant that you couldn't have the features that fancy your LiDAR systems have. Yeah. At the same time, though, I mean, I don't know how much latitude there is vis-a-vis the CPU, the computer that's bringing all this together and making the navigational plan. Is that something which is standardized in this competition? Or you can go out and get a big one or a little one or whatever you... You can use whatever you want. Yeah. The UH team is using a Intel Nook. It's the next unit of computing. It's a very small computer. It's about this big. Mm-hmm. But, you know, as powerful as a desktop machine. And you program it with a desktop machine, I guess. Oh, yeah. Yeah. Or, I mean, you can use it as a desktop machine. Yeah. So you get a keyboard, monitor. Yeah. And you can look into it, and you can use software to program. Right. And then put it on the boat and have it run. Mm-hmm. Actually, there's a wireless connection between that and the shore, because one of the requirements for the competition is that you have to be able to shut your boat down. If it starts going out of the area, then you need to have a remote shutdown, as well as a button on the side of the boat that someone can go up to and hit, and that will disable it. And can you set it up so you can call it home? Yes. Yes. And then flip it in between autonomous and remote control so that at the beginning of a competition, the students would actually drive it out to the starting point and then switch to autonomous. And then from that point on, it's on its own. Yeah. And then, if necessary, you can switch it back and bring it back. Right, right. And then bringing it back is not necessarily a point against you or anything. Or is it? Well, if you have to sort of take over in the middle of a task because the boat isn't doing yet, then that is a loss of point. One other thing, we don't have a lot of time left, but one other thing is when everybody gets together and all these teams are out there readying their boats and practicing and connecting things up, what have you, improving them, improving them through the last minute, do they talk to each other? Do they compare notes? Can I borrow some duct tape from you? Can you show me how your thing works or is it a little too tense for that? Well, I've never been to this competition. It's only happened once before. But in other robotics competitions that I've been involved with, like First Robotics and the Mate underwater robot, there's a lot of that. A big idea behind those competitions is what they call gracious professionalism. If a team needs something and you've got it, then you share it with them. Wonderful. I'd like to go down and see it, maybe take some footage. So where and when should I go? What's the best time and place? Well, actually, if we could go to the last slide, that's an aerial view. The competition will be in Cahee Lagoon. That's Sand Island in the lower part of the picture. And you can just see the edge of the Sand Island bridge. So the competition will be in that protected water between Sand Island and that little reef island. And to get there, you go over the bridge and make your first right. And then follow the road around. There's the Honolulu Community College Marine Education and Training Center. It's the white square building, almost all the way to the right. And then there's a large parking lot and a boat ramp that will be used to deploy and recover the actual competition boats. Well, good luck, Mark. Good luck to your team. And we are all going to learn a lot by this. And good for Hawaii for participating in the competition. Thank you so much. Hopefully in years to come.