 Friday morning, 11 o'clock in downtown Honolulu, folks. Ted Ralston here on Think Tech Hawaii with our show, Where the Drone Leads. I'd like to welcome on board today Dr. Phil McGillivary. Phil, long time you've been here in Hawaii, long time you've escaped being on this show, and we got you. I'm here now. You're here now, and once you're here, you come back a lot. That's how it works. Anyway, just let our audience know that a lot of the people in the world set their clocks to four o'clock on Friday afternoon, assuming this show is going to be a four o'clock Friday afternoon. So folks, you got a five-hour reset on today, so make sure that you don't think it's four o'clock, it's only 11. And next week we're going to switch this show to 11 o'clock on Thursday, so kind of keep you on your toes. Anyway, Dr. McGillivary, Phil, is here for a conference this week and for some other activities, and Phil always brings with him incredible insights and background and almost impossible to state your total litany of history, where you came from, where you're going, and the things you've been able to do, Phil, in your life here with the Coast Guard. So, yes, I am the science liaison for the Coast Guard Pacific Area, which for more than 120 years has been based out of Alameda, California and San Francisco Bay, and we have responsibility for the Arctic Ocean, the Southern Ocean around Antarctica, as well as the Pacific and Indian Ocean was added a few years ago when we realized we needed to start doing some stuff over there as well. So one of my principal job responsibilities is to manage the science on the United States icebreakers, which the Coast Guard runs, and so we have one medium icebreaker that will break through about 10 feet of ice or so, and one heavy icebreaker, which is principally used in Antarctica, that will break up to 21 feet of ice. And the mission in Antarctica is to resupply our main base at McMurdo in the Ross Sea, which was set up back around in the late 1950s, and they maybe put it a little bit too far south because the ice is quite thick there, and that's why we need the heavy icebreaker to make sure we can get food and fuel into our base in Antarctica. So we have a number of other issues. I guess if we could get the first slide, that would show some of what the Coast Guard's fundamental missions are. So these are some of our challenges right now. The Coast Guard's responsibilities are for search and rescue, of course, for illegal fishing, monitoring, safety of life at sea, responding to natural disasters, and international and law enforcement in general, as well as I mentioned, ice breaking operations. So one of the issues that we have is in order to combat things like illegal fishing, we're trying to stay focused on persistent surveillance. Now, the Coast Guard has C-130 aircrafts. Those are fairly expensive to operate, and so we've been trying to see how we can supplement those with unmanned aircraft. That's a great thing to say that we're a drone on this show because it's about drones, so I was hoping we're going to get to that point. Okay. Well, drones, unmanned aircrafts, either I have to face the fact that's what the world calls them, so that's what they most need. I'm fine with that. Okay. So we started, I started working with unmanned aircraft in 1998 with Burt Rutan, and the aircraft weighed about 350 pounds, and the ship captain thought that was a little bit too much in case it crashed into the ship. So we've been working with much smaller aircraft since. We started in 2011 with a RavenEye, which is a four and a half pound aircraft that was developed for the military and used principally in Afghanistan and Iraq. And it did a very good surveillance. It's been worked on quite a lot, and so it's hand-launched, and you literally can hand retrieve it as well. And our first use of it was off the ice breakers in the Arctic to look for ridging of ice. We have a saying in the ice breaker world that ice isn't what stops ships. It's when ice ridges up and down. That's when you get really heavy ice. So you want to find those ice ridges from onboard without actually putting a foot patrol forward? Yes. Well, we used to have helicopters, but the budget cuts meant that we got rid of the helicopters, and so that's why we were really moving. So when the budget cuts really get UAS going, and the budget cuts happen there, what are we going to use? Paper airplanes. Well, I think the unmanned aircraft, the drones are inexpensive enough that as we move towards them, we should be able to use them. So the initial mapping of ice ridging was done with structure from motion sequential frame video. And when we started in 2011, it took about six or eight hours to actually turn the data around, and the captains of the ships wanted that once twice a day in morning. Well, now we can turn it around a lot faster. And so structure from motion works pretty well if you are over land and trying to map stuff. It does not work, however, that well over ice at sea in certain circumstances, because at sea the ice moves. And when you're doing structure from motion and the substrate is moving, the accuracy falls off very quickly. Is there enough accuracy, enough resolution in the constant color ice to generate a good ice ridge picture? Yes, and you can film in multispectral. And interestingly enough, the ice ridges are in places both warmer and colder than the ambient ice. And for example, if you look in infrared, the ice ridges stick out right away because they are a much different temperature than the regular ice. I know this wasn't part of our thought process, but I just wonder if I can take this problem down the street to some guys at Oceanette after later on when we're done or something. I think maybe some of their video extraction technology might be useful in this issue of moving the moving ice in... Well, it's possible, but where we're going with that, and I think that we're getting ready, there's a number of groups that are working with me to try and get this going. The actual solution is inexpensive LiDAR laser mapping, which gives you much greater accuracy. And two years ago or so, that was fairly expensive, 30,000 or more. And the processing of it is complicated as well. Well, it does have very high bandwidth data. These are fan beam LiDARs that are 50 megabits per second. So it's more data than HD video, but you get very good accuracy. And we don't necessarily need super high accuracy for ice ridges, but if you're doing something like beach erosion or inundation, a centimeter... This is about centimeter scale accuracy is what you're going for. So the price of those systems has come down a great deal now, and the systems were available, but the integration into the aircraft was not easily done. And now that's also commercially available. You can just plop it in your drone and actually get centimeter scale resolution with very precise GPS mapping. And with some reasonable, simple round processing or off-board processing? Yes, and the processing can be done on the plane, in fact. So that's incredible. A couple of years, how much that has advanced. Yes, the advancement has literally been over the past 18 months or so we've gotten to that level. So that brings up an interesting question. Do you think... What we're talking about is almost experimental approach towards things, put things together, see if it works. Is there a way that that could be turned into a requirements generating process where the necessary requirements can be established in advance? Well, we definitely have established. So there are a number of interagency federal working groups. There's one for the National Science Foundation and the Navy for unmanned aircraft. They've been discussing requirements and training requirements and operational requirements for a whole variety of different things. There is a federal interagency group, which is NOAA, NASA, the Coast Guard, Department of Energy and U.S. Geological Survey. And so it depends on whether you're looking at erosion of river banks where you maybe need five centimeters or so if it's a fairly large signal. Or if you're looking at things like inundation and high tide maps, those accuracies are a centimeter. And for earthquake stuff with U.S. Geological Survey, they will actually try and go lower than that when possible. So people are actually using the end state that they want, the end delivered product, working backwards from there in terms of the resolution required. And that will then define what the sensors and cameras look like. The other aspect of that is, so you have this spatial dimension of it, what accuracy do you need spatially, how large an area do you need to cover as the other components of that, and how frequently. And the weather you have to operate in and the temperature and the winds and all of that goes. One of the reasons that I've been pushing the limits is because our operations are in the Arctic and the Antarctic. And it's always extremely windy in the Antarctic. We almost never have a day where the wind is less than 30 miles an hour. And since we work for the Coast Guard for search and rescue, we have to be able to operate in all kinds of conditions. Cloudy weather, foggy weather, sleet, snow, ice. So we have sort of the worst set of possible conditions. You're a great place to get requirements from. If you can meet the Coast Guard requirements, you can probably survive elsewhere. So we've developed a carbon nanotube anti-icing coating that you put on the wings. And it takes just a very little bit of power. And it will melt the ice off the wings, which is a big problem for us in the polar regions. There's also a silver nanoparticulate paint that's self-organizing into a network. And that will go over the lenses of the cameras. And so while you're filming, if your camera ices up, you could hit that a little bit. It'll melt the water off the camera. And you can continue the mission. So there are a number of issues for the polar regions, in particular, that are among the most demanding that exist. But you also have the traffic regions. I mean, there must be a similar set of requirements coming forth, dealing with sea level rise. Well, I've been supervising a number of graduate students who've been working on coral reef mapping. We should mention Stanford, right? Yes, I am one of the mentors of the Stanford Unmanned Aircraft Club, SWAB, which would SUAVE, if you look online, which was founded by one of the graduate students at Stanford that I work with. Can we pull up that video? Do you think right now? Yeah, well, let me introduce it a little bit. So part of the issue was many of the unmanned aircraft simply could not handle very strong winds, such as we had at sea. And some of them land in nets. We had a number of experiments that we did with NOAA off of our ship with some aircraft that landed in the water, which was a problem because then you had to put a small boat in the water to go get them, which is a whole ordeal. It involves the crew of the ship, and nobody really wants to spend time doing that. And then some of them landed in the nets. But in high latitudes, the GPS fixes aren't that good. Sometimes the compass gets screwed up. We operate near the North Magnetic Pole, the South Magnetic Pole. And so they don't always land in the net. And so we went towards working with Stanford graduate students to design basically an aircraft that's like the Osprey in the military, where it goes up like a helicopter, and then the rotors will flip horizontally. So we do have a video, I think, that shows part of that aircraft, which is now commercially available. Dang, this was very much of a prototype at this point. Yes, this was a prototype. This is a slightly, it looks about the same now. But you can see that there are two rotors in the front that are takeoff vertically, and then they flip over horizontally. And so this is basically useful for ships. But one of the other key things, this is in Honolulu Harbor. And with a quadcopter, you get 10 to 20 minutes, 15 to 20 minutes of endurance. But with this aircraft, because the engines flip over and fly like an aircraft, you have much better endurance. So we're getting more than two hours endurance on the same batteries that you would get 20 minutes. The ratio is about five to one in terms of rotor-borne versus wing-borne endurance. So this video shows us mapping the coral reefs on Lanai, which is one of the least studied areas of coral reefs in Hawaii. We also mapped some of the traditional, there are five traditional fish ponds on Lanai, one of which is being restored. And so we wanted to get some sort of at least before pictures as the restoration process proceeds. So this configuration being really evolved to satisfy a coast guard requirement has applications everywhere, not just limiting the coast guard. Yes, no, no. I mean, I just needed something that would land and take off vertically and work in high winds. But the interesting thing about that particular plane is that Trent Lucasic, who is the designer and owner of the company with his colleagues, he designed it with what's called blended controls. So every single second, it's checking to see if it's a quadcopter or a plane. And what we did, actually, is in 45-mile-an-hour gusts, we flipped the plane over. And if you do that with most unmanned aircraft, they pretty much hit the ground or go in the drink. This one, it flipped over. And since it was checking once a second to see what kind of a plane it was, whether it was a plane or a quadcopter, it instantaneously righted itself with no human interaction. It only dropped about six feet and then continued the mission. Adaptive control, so to speak. It was adaptive control without any human interaction. Let's get back to the whole subject of requirements and such and keying off from that very observation when we get back in our first break. OK. Aloha and ha'olimakahikihou, which is happy new year. And I hope it's a happy and prosperous new year for you. I'm Kelea Akina with the Grass Root Institute. Every week we partner with ThinkTech Hawaii and produce a program called Ehana Kako. Let's work together. We bring together movers and shakers who are making a difference here in Hawaii, making a better Hawaii for everyone. If you're interested in improving the economy, the government and society, join us every week on Mondays at 2 o'clock p.m. for Ehana Kako on the ThinkTech Hawaii broadcast network. Until you see me then, aloha. And we are back live, folks. Ted Ralston here, hosting our show where the drone leads at ThinkTech Hawaii. And Dr. Phil McGilvery. Bill, thanks again for stopping by. Sure, Ted. Being part of the show today and that really exciting first part of the show here. We were talking about requirements kind of as we got towards the break. And I was intrigued by the fact that you've told us that there are high level thinking going on at the National Science Foundation level or equivalent in terms of requirements that UABs and drones are gonna have to think about addressing as the future comes at us. Well, NOAA is very interested as are some other federal agencies in also using drones for meteorological research, including Airsea Heat Flux, Hurricane Generation, Tropical Storm Studies. And one of the issues that has been persistently problematic for them is what we call the bottom 100 feet above the ocean. Usually- Energy transfer and everything going on there. Wind speed, all of those things. You're not gonna be flying manned planes generally below about 100 feet. We have some planes that drop things sort of an attached drone, but really they're very interested in getting that lower level because that actually gives them the information they need to know what the path of the hurricanes would be. Really, that 100 feet layer. There's information. Yeah, that bottom layer is the absolutely key thing. They've done about everything they can with the models now and that's the one missing component that they're very interested in using the drones for. So information from that area could drive the algorithms and the models that predicts path. It predicts the path. That's the key thing they're trying to do is right now, they don't know where the storms are gonna come ashore. There's a lot of uncertainty in that. And so the one thing that they think they can do to improve that is get that bottom 100 feet. The other example of what we're trying to do is- I think if you have a better place than Hawaii, you go find that out. Like in Hawaii. Well, Kauai and the Caribbean, we have plenty of storms. The other thing that they're trying to do, which has been done a little bit but not very much right now is to use multiple drones to get the three-dimensional wind field. As you know, you can go out and measure the wind at one point, but it's gonna be different every place else. And so what they really want for some of the meteorological models is the turbulent spatial scale. So if you have three or four more drones up, you can plot a path for the drone and then the deviation from that path will give you the wind direction and speed at that point. And what we've been doing with colleagues of mine is sharing the data among the drones and sending it back for, which allows you to get a four-dimensional wind field time. So three dimensions and over time. And there's a lot of interest in things like if you have an oceanographic front, the wind changes because the temperature on the sides of the front is different. And so there's quite a bit of data that you can get back at that. There's the air sea breeze that happens in the morning and the evening. There's also an ice breeze that happens over the edge of ice and the ocean as well. And so this helps a little bit with improving the meteorological models as well. But one of the other things we've been doing is using the drones for illegal fishing monitoring and for monitoring endangered species. The obvious thing that's been done with Alaska and the Alaska Pan-Pacific Unmanned Aircraft Center, which as you know Hawaii is part of, is they've been using them to look at whales. We've done quite a bit of work with whales with seal colonies. We've used them for seabird colonies as well. And you can actually size the animals. But we took it I would say one step further with some of my colleagues and we've used the drone. We started with mola-molas, which are these very big slow moving ocean fish. And we allowed the software to actually recognize the fish automatically and follow them without the animals being tagged. Software on board the drone. Software on the drone. Automatic control on the drone. Yes, so it recognizes a mola-mola, which are brown, so they're quite recognizable. And then it will follow them around. The other thing that we did was to have the drone with infrared camera, actually recognize an ocean thermal front. Because many of the fish and turtles and marine mammals will aggregate at the fronts. So it can recognize the fronts and automatically start mapping the fronts and then recognize the animals at the front and track and follow those as well. And so that's, we've been doing that because there were some illegal fishing that was actually taking in a whole bunch of juvenile sunfish and sharks. So we moved from sunfish to sharks, which move a lot faster. But we've been able to automatically do oceanography and marine protected species. And we're now trying to move towards and have done some flights on automated recognition of whales. And the reason for that is that we have had a number of whale ship collision incidents. We had seven blue whales killed in San Francisco Harbor in one week. And 30 killed in LA Long Beach in a three week period. So we would like to be able to have patrols of unmanned aircraft that would tell the ship, hey, there's a whale directly in front of you, slow down. What's interesting to me again is we're talking about requirements. And we've talked about operational requirements. You talked about science, very deep science in terms of that 100 foot layer and figuring out what's going on energy transfer wise. Question I have is how do we bundle all those requirements or find where they're generated and feed them into our pan Pacific unmanned test range complex? I would say that they're, you know, as soon as I leave here, I'll be going back to another federal interagency unmanned aircraft committee. And we've only been having these meetings and trying to get the requirements really done for about the last at most three years. So I mean, we started talking about it three years ago. It's only been the last two years that the requirements have been drafted and they're being circulated. So I think you're going to see that once that comes out and people agree on it, that it'll actually have a pretty big impact. We had a guess though, because we had a large meeting about that in January, February. I can't imagine that much of what's out there today is going to survive those kind of requirements. Certainly the things you're talking about and the temperatures and bearing ice. I have the harshest requirements. So there is one other requirement that I wanted to show you some pictures of because I thought people might be interested. And yeah, so we can pull up the other photographs and I think let's see if we've got the right ones. This is how to operate your aircraft in GPS challenged or GPS denied environments. And so this is a cartoon of that. And what you do is on the underside of your aircraft, you have a video camera and then you have a parabolic mirror that's looking at the whole surrounding. So let's go to the next image there. And that actually shows the video camera pointed at the parabolic mirror. And then in the next image, this is the sort of picture that you get. And this has been called BI technology because these bees navigate by the flow field. The closer you get to ground, the faster the trees are coming at you and the more their height appears to increase. So this allows automated for takeoff and landing and also airfield, airspace maneuvering. It tells you and automatically identifies where the horizon is and what the angle of the aircraft is relative to the horizon. Based on sweep rate and scene compression, scene expansion, that's how bees sink. Yes, this is designed specifically to mimic bees. Okay, and bees hit the flower every time. Doesn't matter what the wind's doing, does it? They hit the flower. If they didn't, they wouldn't be here. And so this is an interesting alternative. This was developed under Department of Defense funding because there are GPS-challenged environments, but many places, I was just going to say, many places have GPS. If you go up in some of the Polly areas and some of the valleys here, the mountains, the Polly's will block the satellites and you will be in a GPS-challenged environment. Environment canyons as well. Yes, and urban canyons as well. So this is one way that I thought would be interesting for people to see that you can address that issue. And it has been worked out, it actually works extremely well. What's incredible, Phil, you're the science master between the operational requirements and the deep technology. And few people can handle that transition. Well, and so I thank you for... We have to have... How do we do more? Adaptive use of this. Well, I think that one of the things that we're trying to work with you and others to do, and others include a number of NGOs. Right now, there are a number of NGOs that have been using unmanned aircraft specifically for doing things like patrolling the turtle-nesting beaches in Costa Rica, patrolling the areas in the northern Gulf of Mexico, where the vaquita, the world's smallest and most endangered porpoise, still lives and is still at risk from illegal fishing, and also to try and monitor fishing around Palau, illegal fishing around Palau, and at the same time coral reef bleaching. So many uses, and particularly as the sea level rise comes up as warming comes up, we even have issues of managing the land side because populations are going to get this place and such, and so there's no end of use for this functionality as it comes forward. Well, the Coast Guard also is one of the first responders in natural disasters, and so one of the key things that we need to know is where do we need to... Where can we safely put the Coast Guard people to begin assisting? Where are the people most in danger? Where is the damaged infrastructure that we need to look at first? And I think what we'd like to know, mostly as the pan-Pacific unmanned air systems test range, is what are those total requirements? And in any form we can get them, like we talked the other day with the folks from Primo, and use them to start driving the direction of technology in our research here at the university and our grad student programs and such, and also in investment from private companies and such, and get a holistic process going that's not just hit and miss, but is taking real requirements well thought through such that you're presenting and turning them into real meaningful student projects. So much that we don't have, so much of this in the drone world, frankly, is a model airplane experimentation. Well, the first thing... We have to get past that. The first thing that we did was to use... And we're coming to the end of our time here, so... We did a secure communications protocol. Secure communications... So data formatting and data protocols and data security. You don't want everybody to see necessarily everything and getting the data format so they can be shared. In a very practical way, so making a movie. Important starts. You don't want to have a movie and using a drone to record the imagery for your movie. You don't want to have somebody else pulling that data down and putting it on the internet tonight. So data formatting and data security are important, but the other key thing I would say... Right now, most of the data streams have been done with wireless communications with delay and disruption, tolerant networking, and that works pretty well, but the data streams we're getting now are much higher bandwidth. It's not unusual to have an unmanned aircraft with a bunch of sensors so that your data streams are in the 100 to 150 megabytes. Most interesting. So we're going optical communications or the way to solve the high bandwidth. Two weeks ago on this show, we had a gentleman from a spectrum management company and we were under a lot of interest from O&R and thinks working in terms of disconnected, intermittent, and limited or low bandwidth operation. So we have this emerging contrast between higher bandwidth requirements performing a mission and potentially lower capability in terms of reliability of that supporting spectrum in order to perform a mission. So we're talking about a lot more stuff. I want both, Ted. I want both. I want reliability and security and high bandwidth. And we want you back on the show next time you're in town and we absolutely want to dial into that pile of requirements that are forming up because that is so essential to get those requirements and push them into our educational program and into our test range here. Well, the educational component is a big aspect of things as well. And so the materials for that are still being developed, but they're coming. Okay, well, Dr. Phil McGilvery, thanks so much for coming on the show. Thanks very much, Ted. Of course, guys. You're going to have to make your habit to get on this show every time you're here in town. Well, I'll be happy to do that. Okay, thanks very much. Thanks again. We'll see you everybody next Thursday at 11 o'clock.