 This is Mitch Yuan, your host for Hawaii, the state of clean energy today. And this is hosted by the Hawaii Energy Policy Forum and by the Hawaii Natural Energy Group provides the funding for this. So what we try to do is trusting stories or show and today is no different from that. We're going to be looking at wind energy. And I'm very happy to have Patrick Karjela. I think I got it right. Karjela. Yeah. So he's from Team Kilo Hoku. So Patrick. Hoku. Thank you. So welcome. I've only been here 20 years. I still have a problem with Hawaiian. Sorry. Well, welcome to the show, Patrick. And very pleased to have you. And so first of all, so the name of the title of our show is Navigating Virtual Oceans. So what does that mean? So our application Kilo Hoku VR is actually a virtual reality application that is meant to help teach both people learning about the Hoku Lea as well as people who are teaching about the Hoku Lea. And on top of that, the people who are learning and teaching about how they do non-instrument open ocean navigation. And so the virtual reality application allows people to experience what it's like to navigate on the open ocean using the techniques that they teach their own navigators. In this day and age. So let's have the second slide up. So tell us about the Stoen canoe. Yeah, sure. So the Hoku Lea was originally kind of idealized and constructed in 1974 by the Polynesian Voyaging Society. And it was built with the purpose of essentially proving that native Hawaiians were able to sail across multiple thousands of miles of open ocean with actual purpose. And up until that point, the kind of common knowledge of the day and age was there was no way they could have done this. They just kind of drifted around the ocean eventually somehow ended up in Hawaii. So it's just sheer and lax. Proposterous. So what they did is they decided we're going to prove that, you know, they were actually able to do this. And so they constructed this double-hulled sailing canoe also known as a Vaakalua. And in 1975, she launched and in 1976, they were actually able to successfully navigate from Hawaii to Tahiti. And I think it took them about a month, maybe a little bit longer than that. And unfortunately at the time, the knowledge of how to do this sort of open ocean non-instrument navigation had been lost in Hawaii. So they found a gentleman from Satawal. His name is Malpiliag. And he was able to help them do this initial journey and essentially taught the beginnings of this type of navigation to Nainua Thompson, who then developed his own system of open ocean navigation that is still used here in Hawaii today. And from there, essentially they decided, okay, we were able to do this. And they decided we're going to continue to do these types of journeys so that we can show that one, you know, this is really possible and it's culturally important to Hawaii. And two, so that we can show that we can want to continue to have this sort of knowledge down through further generations. Great. So let's shoot to the next slide because there's a lot involved in this. There is. And what we see here on this slide is the actual canoe in the center of this sort of mental construct. This is called the Star Compass. And again, this is developed by Nainua Thompson at the Polynesian Voyaging Society. And it sort of divides the horizon into these little sections that they call houses. And you can see the names of the houses around the screen there. Hema, which is the one you can see all the way at the bottom itself. And then they kind of go clockwise, Haka, Naleo, Nalani, Manu, Noio, Aina, La up to west and then around to north and so forth. And each one of these houses divides the horizon into about 11 and a quarter degrees. So you get these equal divisions, no matter where you are, you can look out to the horizon and say, okay, if a star rises or sets in this part of the horizon, then I know that that is the direction that I'm facing. So if a star rises all the way on the east, right in that house, then I know that's due east. And of course they, over many years of training, memorize hundreds of different stars that are rising in setting times and are able to then figure out what direction they might want to sail in or what direction they're facing based on the positions of these stars. So I have a question. So how could they memorize all that? Did they write things down in those days? Did they have a way of documenting these things? Or was it all just by word of mouth, by memory? My understanding is the way that Malpelioq taught it in Slantavall is that you started learning when you were very young. And basically it was rote memorization. You were taught how the system worked and you spent every single day, you would practice memorizing all of these different things and facts. And you would go out on the water and actually practice them on a regular basis as well. Like I said a little earlier, unfortunately the practice no longer existed in Hawaii. So what Nainua Thompson had to develop from scratch was basically built off of the system that Mal had developed. And from there, he made his own system called the Hawaiian Star Compass, which is what we showed on the slide there. Okay, so presumably in this day and age he has this all documented? Yes, unfortunately we are able to write all of these things down. And speaking to some of the people who do the active practice of wayfinding is what they call it, or the non-instrument open ocean navigation. They do write things down and they do spend the time while they're studying it to take notes and all of these things. But they're also heavily encouraged to kind of put that aside and basically ingrain it into their own sort of memory and consciousness to get the feel of, this is just how it works. I don't need to keep referring to a book. It's really much more natural and ingrained in that manner if they're able to do it without having to continually refer to the source material. So let's flip to the next slide, which headlines the next section we want to talk about. So what are the difficulties? So one of the major problems is that there is only one Hokulea. There aren't a whole fleet of sailing canoes that you can necessarily go out on. And there are a couple around Hawaii on the various islands. We have Hiki on Aliyah here on Oahu and then some on the outer islands as well. But the opportunity to be a crew on one of these va'a is very, it's not very frequently available. They don't go on sales terribly often and when they do they're very long journeys. So the training opportunities to get out on an actual va'a and then practice in context is rather hard to get. Another problem is the weather doesn't always cooperate. You can't practice celestial navigation or guiding yourself by the stars if it's cloudy outside or if it's rainy or you can't see the stars because you're too close to land and there's light bleed all over the place, which is a problem that we have here on Oahu. So that's another opportunity that this... One of the problems that they run into essentially when they're trying to learn the material. The other problem is that much of the material is taught on flat paper. So you're looking at this flat sort of star chart saying, okay, here's what the horizon is going to look like and here's where the stars are going to appear on that horizon. But it doesn't really easily directly translate into that's what it actually looks like when you're out in the environment and looking at the sky right then and there. So those are some of the obstacles that they run into when they're teaching this material. But there are other clues beside the stars. Is this strictly using the stars? Or I think look at the wind and they look at the current and the wave directions of the waves and things like that. So there are a number of things in the environment that they look at. Like you said, there are long... Or what they call long swells or basically ocean swells that travel over many hundreds of piles. But you can feel them when you're on the canoe and you get used to the sense of, okay, that swell happens every five minutes and it's coming from east to west or something like that. They also look for signs in the sky. There are certain cloud patterns that form around islands. And the entire point of this navigation is to get to an island. So you look for these cloud signs in addition to other things like birds. When you get close to land and the sun is starting to set, the birds will be flying from the ocean back to land. So if you follow the birds, they will naturally lead you towards land. Our application specifically right now only focuses on the celestial navigation portion. It's really meant to help with that mental model specifically. We would like to eventually add in other things like long form swells and that is something we've discussed with some of the navigators that we've worked with on this project. So when you actually, have you actually been to sea on the voyaging canoe yourself? I have had an opportunity to sail offshore of Oahu on the Hikianalia, but I have not yet done any deep sea voyages on either the Hokulea or the Hikianalia at this time. It's something that I hope to be able to do and was actually supposed to do this past April, but then we had COVID-19 hit and all the voyages essentially got canceled because of that. So we are currently preparing at the Polynesian Voyaging Society for upcoming voyages in 2021. So another quick question is how do you steer? Like, you know, the stars move around the sky and all this stuff. You're the helmsman. What's the, or are you going to tell us that in your app? How do you actually steer the boat and make sure you're going on a straight course? Because like I said, stars move around. For sure. So really the only time that the stars are used as a marker of direction is when they're about 10 degrees, about within 10 degrees above the horizon. As soon as they start to get higher than that, they're not as reliable as an indicator. So the navigator will set a portion of the sky and say, okay, that's the direction I want you to sail. Use that marker star for this amount of time and steer towards it. And so the helmsperson will grab the steering sweep or the hoi luli and they'll make sure that they, you know, mark on forward straight towards that specific star or guidance point in the sky or environment that they're supposed to be sailing towards. And the navigator will continually update what direction they should be sailing in and give those directions to either the captain or the person who's currently at the steering sweep to maintain the direction that they want to go in. But it's like you said, you can't just use that one star because it's going to arc and move through the sky. There are some stars that remain relatively stationary such as the North Star and that can be used as a reliable point of navigation and say, keep the North Star this many houses off to the port side of the vessel and make sure that you're sailing at that angle so that we maintain a constant direction. And that's essentially how they figure out what direction they're going to be sailing in as the stars move through the sky. So I guess without, I guess they weren't worried about speed over the ground so they would like do what they're called dead reckoning. You just you just keep on going or you do or are you concerned about speed? No, when you mentioned dead reckoning that's actually the exact method that they use is they try to keep track of how far have they traveled in a given direction over the course of the day in order to determine where they're currently positioned. So they will say, okay, the canoe has been moving at this speed throughout the day so we average that we've moved so many nautical miles across the ocean throughout this day of sailing. And then they'll say the currents drifted us off course in this direction because we were sailing in a particular direction and the winds weren't favorable. So we're probably need to change our course in this direction to offset or to make up for that offset. And they'll basically do this on a daily basis as they'll say we probably sailed about this far today and we'll probably need to turn in this direction and this is about how where we are right now based on with those numbers. So how do they measure their speed then? I mean in the old days they used to throw what they call a log over the side and had like knots that's why we talk about how many knots you're going. And you know you have a an hourglass and you time it and the number of knots that went through your hand they can tell how fast you're going. So how do the Polynesians do that? So the Polynesian Voyaging Society are the people that are voyaging specifically on the Hokulea and the Hikianalia have basically a method because they know how long their canoe is. So they can say if we drop something in the water at the front of the canoe and we time how long it takes it to get from the front to the rear of the canoe we know how fast it moved. And doing that essentially they know what their current speed is. They'll even use if there's enough froth or bubbles on the ocean surface they'll try and time it via bubbles moving past the canoe in similar methods. So in that way they can say okay we're currently moving at this many knots and we keep track that we did that for maybe an hour and a half so we know about how far we moved during that time period. Wow, that's pretty good. So how many people on the canoe would actually know how to do this? I mean is there only one guy that keeps track of all this or is there enough knowledge on board that the guy's got to sleep some time. So So the navigator typically there is only one navigator and they basically don't sleep I've been told. They just kind of enter this state of constantly being aware of everything around them and they'll occasionally nap or other things like that. When they're training new navigators of course they will go through shifts and allow them to sleep at times and then come on okay you're gonna navigate for this portion and I'll help you out. But everybody on the canoe basically tries to help out and make sure okay I need you to check how fast we're going and then tell me that information or I need you to make sure that we're steering in this excuse me in this correct direction. So basically everybody's in touch with the environment all the time. Absolutely it's something where once you're on that ocean you are going to be involved in making sure that your island or your little canoe is going to be headed towards the correct destination because that's all you have is your little island your little canoe that you're on. Well let's flip over to the next slide. So what we have here yeah so what we have here is this is Kila Hoku VR and what you're looking at on the left side is we have this sort of starting area. Hang on, hang on, hang on what's VR mean? Oh sorry virtual reality I apologize I should have said that slide. So Kila Hoku VR or virtual reality is basically you start off on the dock side standing next to the Hoku Lea and you're able to explore or walk around using teleportation methods so you can point to a direction and then it will move you to where that pointed place is. And then you can point at the different parts of the canoe and it will identify them for you in both English and Hawaiian so that people who are learning about what is this part of the canoe and why would I need to know about it can start there and identifying it. So again the opportunities to actually be on or around the Hoku Lea are very far and few between. So this makes the Hoku Lea available right there accessible to anybody who's in the simulation. Very good. Does it actually show you actually how to operate some of the some of the equipment like you know how do you hoist a sail and how does the rudder work and all that kind of stuff. So the only part of the canoe that you can actually interact with at this time is the steering sweep. We intend to eventually add some more things where you can raise and lower the sails using line on the canoe and other things where you can for instance change the oh actually this is a feature that's in the simulation is you can set up these markers on the railings and these markers are actually physically present on the canoe and they're sort of these glow tape so we've kind of enhanced it to make it so that there's this physical marker that appears in the simulation. And it physically marks off for you that 11 and a quarter degrees so you don't have to know exactly how far apart 11 and a quarter degrees is because it's already marked out on the canoe for you as long as you're standing in a particular place on the canoe and that's called the Kila Hoku itself or the navigator's seat. So who did all the artwork and what program did you use? The application was developed using Unity 3D. The model of the Hokulea itself was developed by a gentleman named Michael Pie who works at NOAA the National Oceanic and Atmospheric Administration and he was kind enough to let us use the model that he had developed of the Hokulea in the simulation and we're very thankful to him that we were able to do that. The rest of the things that you see in the simulation were developed by our team and put together so that we would have an accurate representation of the sky and all those running parts and if we want to jump on to the next slide I can go ahead and show you guys the next part of the simulation. So this is the actual physical equipment that we use for running the simulation. It is called an HTC Vive and this is specifically the HTC Vive Pro and it is a virtual reality headset. The one that we have pictured here is higher resolution so it gives a really crisp display when you're wearing the VR headset so you can really accurately see things in the simulation. We can go ahead and jump to the next slide. So what we have here is a quick comparison of the actual Hokulea on one side and that was taken during the homecoming when they came back from the worldwide voyage in 2017. I took that photo myself when I was down at Magic Island when they were coming in to dock and the model that we have of the Hokulea on the other side that you actually see in the simulation and you can see that they're pretty accurate in terms of how they appear compared to each other and we tried to make this as accurate as possible because that's the whole point of the simulation is to try and get accuracy in there so that when you're using it it's going to be actually use actually relevant when you have an opportunity to go on the canoe and practice any of these navigation techniques and we can go ahead and jump to the next slide. So this here is kind of the meat of the application. What we've got here is a VR pointer and it's pointing up into the sky and it has highlighted the constellation or some minor and the constellation will actually pop up in the simulation as a highlighted constellation whenever you point at it using this tool we have called the star pointer and you can see the words there on the right hand side of the controller where it's actually saying what each of the buttons do. So you can in the simulation when you're on the canoe and you're in the stargazing scene you can actually point at various stars or constellations you can steer the canoe so that it lines up facing any one of these constellations you can then speed up the rotation of the sky such that when you are you know actually out in the environment if you're sailing over a number of hours you'd be able you would want to be able to know when does this star rise and when does this star set all around the horizon so that I know what direction I'm facing essentially and then near the bottom of that picture there we've got the Hawaiian star compass the words of it all around the horizon so you can see each of the houses and know where is this star rising and where is this star setting relative to the particular house that I want to be able to navigate towards or to be able to know what direction I'm facing in and we actually have a feature in the simulation that allows a teacher who is sitting at the computer to turn certain implements in the simulation on and off so they could turn on the star compass and say okay using the star compass please go ahead and steer the va or the canoe so that it's facing to east and then the teacher could turn off the star compass and say okay now we know you're facing east using the stars in the sky I want you to go ahead and face north or south or any other given direction and they would be able to use the clues that are inside of the simulation based off of use of either the star pointer or just if they know where a particular star is and what time of night it is they would be able to then practice orienting the canoe based on the direction that they want to sail in the simulation also allows you to set any latitude and longitude that you want well really any latitude because the longitude you can't really changing that only changes the time but it allows you to change the time of day so you can set it to any time of night that you want and you can set it to any north or south latitude that you want and you can basically practice I want to sail in this direction from say Tahiti and this is what the sky would look at on this time and date in Tahiti or this time and date in Hawaii or this time and date in practically any part of the Pacific Ocean that you might want to be in so in that way it allows them to specifically practice or train on a particular place that they might be voyaging to and we actually had one of the navigators who tried out our simulation for the first time he told us after trying it the first time my goodness I wish I had had this available to me when I was actually training for one of our voyages because I would have just sat in this application for hours on end practicing different parts of the sky and knowing what it would look like so to us we hope and I'm not going to say it does but we hope that this means that it directly translates in a meaningful manner to how they would actually be navigating out of the open ocean and that's part of a study that we are aiming to start doing in the near future with the simulation so how long did it take you to develop this program it looks like it was a monumental task to do that there's a lot of software there we started working on this as part of a class assignment actually for a virtual reality an augmented reality graduate level class at the University of Hawaii at Manoa that was taught by Jason Lee and we started developing it in 2016 and we had our first release at the end of that year and we continued to develop and work on it over the years we had a few publications about the simulation in IEEE no sorry not IEEE I apologize in MIT's presence journal which is their virtual reality publication and we were so we got so much great feedback from the community we were like we have to keep working on this so we were able to over the number of years add new features really kind of make the application more robust and eventually came out with what we now have available to the public today and we want to continue adding features to it based off of feedback that we've gotten both from the community and from educators who would like to use it as a teaching tool so what kind of feedback are you getting what flavor does that take so wow this is great or there's more depth in that of course there's always the wow factor with virtual reality especially for people who have never tried it before of wow this is amazing just putting on the VR headset and being able to see all this it's like you're really there and then we have people who have actual knowledge of this and they start saying I'd love to be able to have those ocean swells like you mentioned earlier as something that we can then try to train on in the simulation how can I tell what direction those ocean swells are coming in we were asked if we could add moon phases to the simulation which is something that's mostly in there at this point we just have to tweak a few more things and then it should be working we were also asked to add sunrise and sunset because the direction or location of the rising and setting sun is actually a huge marker during that time of day because the sun rises and sets in a very reliable location every single day except for as it slowly moves throughout the year but you know over about a month period it's not going to move all that much so it's a pretty reliable indicator on top of that they've also asked if we could start adding other canoes into the simulation so if we wanted to add Hikianalia which is sort of the sister vessel of Hokulea here in Hawaii to the simulation so that you can try training on different sized va or canoes in addition we've really been asked oh this is a great slide to be showing this is actually Nainoa Thompson trying out our simulation at a demonstration that we had at the Polynesian Voyaging Society and he really was enthused about it and wanted to make it available for his high school teaching essentially you can teach mathematics using the simulation you know what angle do you need to turn to and how do you determine for how long you'd need to travel and you know the math involved in that so we've had some educators come to us and say we really want to be able to use this to be able to teach in a variety of different ways we check this out at the Bishop Museum yes one of our early demonstrations of this was at the Bishop Museum there was a specific day that they had dedicated to Polynesian Voyaging and we were able to be there for a whole weekend and we got tons of great feedback from that as well people were really enthused about it so at what age level is the interest I mean are the young young kiddos like the Kiki like five six seven-year-olds do they get into this or like this a little bit too difficult for them at what age is appropriate age to start into this the best age that's recommended is really based off of the virtual reality hardware itself it's recommended that people are about 13 years or older because when you're younger the eyes are still developing and the headset's too heavy essentially it's not very comfortable to wear for long periods of time and it essentially falls down off of their face so their heads are too small it doesn't work all that well but about 13 years and older is the intended age group okay so how is this funded funding for this was provided through the ACM at the University of Hawaii at Manoa they were very kind to give us the original funds that we got in order to develop the application sorry the the Academy of Creative Media there we go there are two ACMs so I tend to get them mixed up sometimes so the Academy of Creative Media got all the equipment for that VR class at this time we basically are making the application available for free the website is on the page right now so you can go there and download it if you have the VR equipment which is also listed on the website we are seeking additional funding and donations at this time so that we can develop some of the additional features that we listed and we're starting off with a really you know small starting point of about $5,000 and that will let us port the application to additional virtual reality headsets so that you don't have to buy this specific one that we originally developed it on and then you can try it out on whatever VR headset you might have okay so where do you see this going our hope is that this will be used at the University of Hawaii to help teach the Hawaiian Studies 281 and 282 classes which deal with Hawaiian astronomy and celestial navigation and we hope that it will become a widespread tool for people who want to learn about or even just explore celestial navigation from the comfort of their own homes or for actual use in future voyaging okay Patrick believe it or not we're almost at the end of our time seconds to go so I always like to give my guests a chance to just make one more comment of something that we might not have touched on that you feel is important I think that it's really amazing how virtual reality allows you to develop these applications in this day and age that allow you to get access to things that you just simply wouldn't be able to do without it I mean people outside of Hawaii would never be able to experience this unless they somehow flew here and were able to sit down on the canoe itself so this really gets the experience out there and acts as a bridge towards Hawaiian culture and learning so the Hawaii Tourism Association we could use this not only here in Hawaii itself but in further marking on the world to attract people to come to Hawaii the culture the Hawaiian culture absolutely and I would love to see them starting to do something like that okay well Patrick thank you so much this is extremely interested as a former Navy navigator myself I found this fascinating I told you my navigating stories on a submarine crash surfacing grabbing a sex tent getting wet and crashing down below so this is really great we didn't have virtual reality we didn't have anything in those days so thank you very much appreciate it and this is Mitchell and signing off from Hawaii the state of clean energy we'll be back next week with I hope an equally interesting guest Aloha