 Hello, my name is Satish, Satish Chetty. I'm actually, hold this in pocket. We got a few more minutes to start, but I think I'll start anyway. Gives me a few minutes to answer your questions if you have any. My name is Satish Chetty. I'm actually pretty surprised and happy that you guys are all here. There's another interesting talk going on and I thought I wouldn't have so much coverage. Anyway, so this is a strange topic and I wanted to give some background before we go further into it. That's me standing in front of the Arctic Ocean that's all frozen and all that stuff, so I wanted to give a spectacular view kind of thing before we start. Give a little background of me and my introduction and what this all means, how Linux and Arctic Buies and how that affects and that kind of stuff. My primary job actually, I'm a VP of Software Engineering at a startup in San Jose. We do Earth Imaging Satellites and I do the OBCS, the computer onboard computer as well as the software software systems on it. My background's pretty much been on embedded systems for the last couple of decades or more. Most of it has been on the firmware level but more on the systems integration stuff. I do work a little bit of hardware but it's more on a high level stuff. I don't design circuits or circuit boards kind of thing but I do work closely enough so that I can affect the software on it. But the last 15 years or so I've been volunteering both at Stanford University as well as with some climate change groups and this talk is actually some of the work that I did with them. And so I've been to the Arctic and the Antarctic and developed some sensor-based systems and this is one of them. And my background's been on special instrumentation. I've also done work on smart meters that are outside on the cold and the heat, the instruments that fail and the software that affects it and so on. I've worked on some commercial and industrial products as well, both the hardware side and the software side so I kind of, the system engineer, product manager kind of things on those areas. By the way, feel free to slap or ask any questions if you want and keep it as informal as possible. I wanted to give a little bit of the background on this. Why, where with Linux, embedded Linux is going with this stuff? So the goal of this project is to build a sensor platform that measures various parameters like incoming sunlight and outgoing sunlight and sonar and ice thickness and so on. And the location for the initial deployment is actually Barrow Alaska, Barrow location-wise. It's about 1,300 miles or about 2,000 kilometers from the North Pole. It's 72 degrees north. It's the northernmost US town and it's got nice facilities and all that stuff. It's still a little lower but it has a little bit of brutal weather on it. You can only fly there. You can't drive there on no roads to this town. You can only fly but there are daily flights. Alaska Airlines actually serves them. And it's cold and dry and there are no wind barriers. The Arctic Ocean's actually on all three sides. During the summer, I mean, during the winter months, it's 65 days of darkness. So what do we build has to manage and survive that. And it's on the Arctic coast. Lots of wildlife, so polar bears, Arctic foxes. And actually all the stuff affects our software too. So I'll talk about this too. And there's good infrastructure there. So the local community there provides a lot of research work. So there's a lot of research work that happens all over the world. And scientists from all over the world come there. China, India, it's the US. There's a lot of infrastructure, good communication infrastructure as well. When I say good, good for that kind of location. Okay, sorry, sorry about that. Is it better? Okay. So this is a quick video. This is in July. We were landing at Barrow and I took this out of the window just to give you an idea as to what kind of terrain this is. That's still the Arctic Ocean. It's still frozen and all that stuff. But you can see it's starting to melt. And this is when I took this video before we landed before we had to do some servicing on the equipment. And this is, I wanted to show, actually I filmed this for my daughter but it's a nice interesting video. I wanted to show what happens at minus 40 when you throw a cup of boiling water inside. You would have seen this. There's a lot of videos on YouTube about it but I wanted to try this. By the way, I'm at that research facility. Outside is minus 40, inside is nice and toasty. And the dangerous part was actually carrying the hot water out. It instantly turns into snow. So the challenge with this stuff is, I mean it goes on for a while. When I showed this video to my bear guard, by the way, when we go out into the ice and actually have to set up this equipment, we need to hire a polar bear that can attack us so he comes with a gun and all. So when I showed it to him, he said, please don't waste water. Anyway, so the big challenges are, just to give a little bit of the background. So we've been testing some of these sensors and sensor instruments in Tahoe, in parts of Canada, in Minnesota, wherever we could do. I mean some of the weather is similar but not the same especially with the salt water and ocean conditions. So some of the challenges are the size. Whatever we build has to be nice and small. The smaller it is, the easier it is for us to pack, ship, and get customs clearance. It has to be lightweight. Accessing this location, even though we fly to Barrow but once we land there, we wanna make sure that it's easily transportable. Sometimes we have to carry it, sometimes we have to sled it so we have to make sure it's nice and lightweight. That includes not just the electronics part and the sensors, it's also the structure, the buoy and things like that. Power, most of the time, power is the biggest challenge for us so some of the kernel compilations and power management stuff that we had to do was actually based on power. So the big power hogs, I'll talk about them a little later, and the performance. So you have to compromise the swap thingy, right? So you have size, weight, power, performance, and of course the cost. And performance wise, yes, you can throw in a big computer with a lot of GPU and computing power and all that stuff but it doesn't serve much purpose in that kind of sensor management stuff. Of course, the next question is, why can't you use something like an Arduino or a microcontroller to do that? And that's where the performance part comes in. Several of these sensors that we use are actually low cost sensors. And these sensors also have USB drivers or software that actually works with them. So instead of us trying to get the hardware, build our own custom PCB, solder it, do firmware level testing, it's easier if we use a Linux system and just plug it in. So we need something that is usable but we still have control over how we configure it, how we compile it. And I'll talk a little bit about those experiences as well. And the cost. Cost is important because yes, we can spend some of the other, the scientific government sponsored buoys cost anywhere between $30,000 to $50,000. So of course this group is a nonprofit, it's a privately funded group and the cost was a significant aspect of it. So yes, the sensors are good when you pay a lot of money but you also lose them and I'll show some pictures about them. And you have to always plan as if you're never gonna retrieve these buoys, including your data. So you wanna make sure the cost is low so if you lose it or if you damage it, you know, it's not a big loss. Secondly, keeping the cost down allows us to build multiple buoys and we're not putting everything into one system. Communication and this is the challenge also. It's so remote. And in fact, geostationary satellites have to be pointed down like this instead of up because the equator is somewhere around there. So all these antennas are always pointing down which means the bandwidth is always restricted. You don't have a lot of usage. In many parts of the world, you can use unlimited 2G, 3G data kind of thing. It's all metered so you have to pay for it. So communication is a challenge and you can use satellite systems like Iridium which is fantastic because worldwide globally it works but it also costs an armor like it costs about $1.50 per minute of dial-up and it's not so much the cost that's the problem. It drains the batteries like crazy. So you wanna make sure that that's something and there's local data storage that you have to manage in case you can't communicate, you can't transmit the data, you wanna have it locally so even if your buoys found several miles away you can actually pick it up and retrieve the data. Transport to size, this is actually part of those first two items, size and weight so you wanna make sure it's easily transportable. Give you an example, when we shipped everything we had to do, it came in big containers or suitcases sometimes and would land at the airport and we would drive it to this research site and from that research site we have to use snowmobiles or ATVs to go to that site. So you have to make sure that it handles all the vibration, the bouncy stuff and so you don't wanna have something that is too small that is hard to handle. In assembly, some assembly on the ice happens and operation it's autonomous, semi-autonomous and manual. You wanna keep it autonomous as possible but sometimes things do go crazy and you have to reboot or unplug some sensors and put them back and do some field level testing and it's usually not engineers who built them. It's usually the support staff or someone who's not familiar with this equipment at all they have to do this and sometimes you have to do it manually meaning you tell them plug in this cable run this on the phone kind of thing to help them out. The other part is just as assembly is important disassembly is also important. So most of this research and study actually happens when the ice melts. The goal is to find out how the ice melts during that melt season but we still wanna collect how the ice behaves during the freeze cycle. So we put it usually around this time, November timeframe and then study it throughout the winter and then when the actual melt happens it happens so fast meaning it happens within six days, eight days kind of thing and pretty much all the ice is now liquid water. So you wanna make sure you can disassemble it properly or you're gonna lose your equipment or you might topple over or you might drift off or you will have other challenges. We've had that one time. The buoy was not anchored properly so it flipped into the water and the nice thing the box was waterproof so it was taking pictures of all the fish inside. So removal from the site. You know, it's sometimes easier to take it to the site and it's harder to bring it back because it's got other stuff on it and so that's another challenge and transport back to the lab or disposal. Sometimes parts of the buoy have to be disposed because they are broken or can't be used anymore weather and environment and other challenges, animals as such. Okay, so I still haven't come to the Linux part of it. I promise to do it. So here's the deployment. There's the buoy, you know, we have a sonar which actually is measuring the depth here. There's a Albedo sensor that's measuring how much radiation is being bounced off the ice. So you have four different quadrants so they're measuring four different areas. That's the battery pack. There's a camera system here. This is kind of where we'll be talking more and you can see the other one, the sensors pointing up so it knows how much. So the essence of it is how much sunlight is coming in and how much sunlight is going off, how much is actually getting converted into ice and so on and so forth. So I'm not so much on the science part, more on the instrumentation and the work part. So the computer, this is the challenges. I mean, we needed a computer that does this. It has to be fanless. Of course, if you have fan, you're wasting power. And first of all, wasting power because you're heating up parts that you don't have to heat up and then you're running the fan itself. So anything with the fan is a bad idea. So the slower the clock speed is, the more power efficient it is. And this is one of the good ones. So nowadays you get arms for arm processors, one gigahertz, you know, low-powered ones. But the other part is interesting for me at least. So it's configurable to operate at a lower speed so you can slow down the clock and actually use less power. But of course it's much slower. And there's another good thing about this computer board, I'll show a picture about that, is it's a peripheral on and off you can do through the software, which means you can turn on and turn off a particular bit and the USB goes off, right? So you can manage power like that. It can run at half a watt when it's fully running and when it's sleeping, it uses very little power. Can operate at very different operating voltages. So you can run it at eight volts, nine volts, 14 volts, 24 volts and it'll still run. And this is good because the battery doesn't maintain charge all the time and depending on the time of day and other aspects, it goes, the voltages vary. It's got built-in sleep timer, which means I can shut it off, meaning I can programmatically say after you've done doing this, this and this, go to sleep for four hours and then get up, right? So we're not burning power. Even half a watt of power is quite a bit of power. And that causes clock drifts at minus 40, I'll talk about that. And so what I did is we did, I mean, I worked with another engineer as well on this. We used Debian kernel mainly because we used Logitech webcams. And these are, you'd see them a lot of places. This is a particular model number that works at minus 40 and plus 85, it's about $75. It's actually a little bit more expensive than our computer itself, but it works great. Meaning, you know, and it's easy enough, you can just order it off Amazon or something. And the drivers work on your desktop, but not for these embedded ones. And that's where we, that's one of the work I did, meaning I had to connect them and actually compile the right UBC driver to make sure that you can turn it on and off, you can do all the video controls, you can take still imagery, but I still wanted to do video, I'll explain that in a bit. The video is actually, maybe I'll explain it now, but video is important because it helps the camera focus. So the camera's inside a box, a plastic box, actually a fishing bait box, and it survives very well. That's the reason we tried different boxes, you know, several hundred dollars worth of boxes, and finally that one worked out very well. And it doesn't let the ice and rimes stick to it. So when you have these cameras, and you say turn it on, take away, you can use let's say FS webcam or, you know, your standard Linux tools and grab an image, it'll grab most likely the image right in front of it, which will be the reflection of the box itself rather than the image outside it. So having the video controls was important because about what we eventually found out that running this video for let's say three to five seconds lets the camera focus on something outside and eventually it gets a good enough image of it. Once that is done, we discard all the image that was captured and then grab one image out of that stream. So compiling that at the kernel level was actually important. And again, the question is, yeah, there are cameras that will work at minus 40 security cameras outside buildings and all, why can't you use that? Challenges, power, weight, cost, performance, and so on. And if any of these things, I mean, the camera doesn't work, we can always replace it very easily. We actually had local Wi-Fi. We learned it a bad way that we needed with local Wi-Fi. The reason is a lot of times the, we put in a small SD card to store data in and sometimes the modems will fail or they won't communicate and we want to take the SD card out. And like I said, you have to do it before the melt happens which is slushy, dangerous water when you're walking around it. And at minus 30, minus 40, well it's not that cold when it melt happens, you are wearing gloves and you want to make sure that you pull the small SD card out of it, balancing on a boat. So having Wi-Fi helps us saying, I can configure this as an access point when the computer boots up, it actually comes up and says, here's my access point. So if I'm close by, I can use my cell phone or an iPad or something like that and download the data much faster. So compiling that was a challenge again because we found out that certain drivers require certain other network drivers, some bit to be said before you can compile them. So every time you add another piece of software that needs to run, you're adding up more complexity, the boot time increases, which means it's burning more electrical power. So another one was removing non-essential packages, making sure only the right ones come up and reduce the boot time and shut down time as well. Boot up as quickly as possible, do your stuff, shut down, otherwise you're wasting power. And testing, testing, a lot of testing, we did that with about freezers and actually tested this. And the interesting part is, we usually do this work in mid-summer and we are running freezers and we're wearing jackets in 30 degree heat. So the basic operation is start up, check the disks for any issues. There's onboard EMC as well as an SD card stored for local data. Make sure there's any issues. If there are issues on the disk, just move on, don't try to fix it because if you try to fix it, you're spending power on it. Start the weather data logger. It's actually using a software-defined radio modem card that we had to compile as well. So many of these accurate and weather stations that you can get off the market, they actually work fantastic even in that cold. But they transmit at 433 megahertz, the unlicensed band, and so you can pick it up in your home. So what we did is actually built a, I mean, we got a software-defined radio USB dongle, plugged it into it, compiled the driver, ran it, and so we can read off the weather station remotely. So that's another work we did. There's a one-wire temperature logger. The one that you didn't see on the picture was there's actually a long cable, about four meters long, that goes deep into the ice with sensors at every level. So you get a profile of what the water and ice thickness is. Interesting fact is the place where we were testing, the melt actually happens from bottom up and not top down. That's because all the melt water that flows into the area actually starts to heat up and melt the ice from bottom up, not from top down so much. So log system parameters, so this is important for me. I want to make sure what battery it is, how much disk space, what the CPU temperature is, what the IP address is, if I'm getting an IP address. And try to send this sensor and data log through local 3G network. I said 3G because it's not good and at that location, we actually had to sell repeaters to get that working. If none of that is possible, we capture images, we capture all this data, I'm trying to transmit it, none of that is happening. Just one try. No three tries, no four tries. Move it to an archive, meaning move it to a directory and then initiate a shutdown. Because the more we try, the more power we waste. So we try that and do that. And initiate a shutdown and then do a disk flush and sleep for X seconds. X seconds, usually the researcher says, I want data every four hours or two hours every day, then they define those seconds. That's the computer. It's about year size. And there's a small battery that keeps the clock alive. Nothing else on it. Very similar to the Raspberry Pis and the Beagle Bones. They actually work very well too, but this one actually has a real-time clock and all the software-based peripheries and everything built into it. It's not terribly expensive too. So compared to the time you spend on this system, it's made by a company called Technologic Systems. It's about $150 or so. But you do get different models. It can support external hardware and all. And this is, after a long time, the hardware you're seeing, you might be surprised, these actually work very well at minus 40. And this is actually a Belkin thing. Of all the USB hubs we've seen, Belkin works great. It actually has an external power supply, so which means I can, using this computer, trigger it on, meaning after the boot sequence is done, then I trigger it on, which means once this one's powered on, the rest of the devices start to show up on the slash dev directory and then I can start managing them. This is a Huawei modem. Expensive for the rest of the stuff. It's about $50 or so, but it actually works up to minus 40, minus 20. Anything lower than that, it connects, but it drops packets or doesn't do handshakes. Of course, this is such a low volume that the manufacturers wouldn't spend time on trying to discuss why that happens. But anyway, this is kind of the lab kind of a scenario. Again, challenges, right? So that sensor that I showed you, we had a pipe and a fox chewed this up, right? And why this is important is when we're trying to read the sensor and we're getting some weird values, we wanna make sure that we're not trying to read again and waste power, so I wanna move on. So knowing this was actually helpful, there was an Arctic fox and apparently they chew a lot of cables, so next thing we had to actually put a cap on it and make sure that they don't get to the wires for some reason they actually come in and chew. This, as you can see, there are three arms, the fourth one's actually standing here, we believe Polar Bear stepped on it and broke the sensors. And so when we started getting data, missing data coming up, or when your numbers start to show as minus two, minus three and all of a sudden your temperature data is 3,000, 32,600 and something, something, you know something weird is happening. So we had to make sure that we catch all that data as well, you can see there's a small Wi-Fi antenna here. And this is another scenario where we saw, earlier I showed you a sonar, it actually tells you how much ice depth is. Well, after one blizzard, there was so much ice inside which we were getting all incorrect readings. So even though the data was showing up on our server side and we were doing parsing and storing it into our databases trying to plot graphs when it's like this, because suddenly you have all this data. So finally we had to actually send someone to chip out all that ice, but after a while we found out that the problem was so repeatedly happening that they decided to change the sensor. It's meant for industrial automation, so it's not meant to, so even though it's temperature rated, it's not meant to handle this kind of situation. Minus 40 other things happen, cables that are normally flexible become like sticks so you can actually tap them and break them. So those kind of challenges happen and any time you have a sensor failure, a minus 40, it's hard for you to open the box up and try to fix these things. Even trying to get it out of the box is a challenge. We tried and we kind of ruined the entire experiment one time because we pulled this, tried to pull the cable, the cable snapped, part of the cable was still sticking inside, but it yanked the boards that the board pins broke and all that stuff. So, and it's also not possible for us to dismantle the whole thing, take it to the lab, fix it and bring it back because now the sensors are all embedded into the ice and it's fairly ruggedized. There are physical design changes that are happening so the next version was don't run the cables inside but put some good connectors outside so you can dismantle the connectors and leave the box as it is. But you still have sensors running into the ice so you still have to drill another hole and start to do other parts of it. You know, that's another one. You know, all this rhyme is not really good and inside the temperature though, inside the box though, it's very interesting because when I get the system log and see it, when it starts up, it'll show up like minus 20 because by the time the CPU has heated up and it'll show minus 20, by the time it's shut off, it's plus 40 degrees C inside the box but it's minus 40 outside. It's not always minus 40, it happens fairly regularly. And deployment constraints, I think I told you about this, we have to ship by air, there's no ground transportation, there's a local SD card. We even started using Bluetooth but there are some operational challenges if multiple researchers want to see this. Wi-Fi is better, vibration I think we spoke, cabling, cellular and satellite modem use Iridium, so we start using Iridium only for text data, you don't dial up, once you start dial up, it drains your battery like crazy. And we've started putting in solar panels as well and also started to use a cellular network. We tried to set up Wi-Fi repeaters into the ocean but managing each of those repeaters themselves becomes a project. I think I spoke enough about this, stop me if I'm rushing too fast. The buoy structure is one, so we use ring buoys, these commercial grade buoys. The batteries are lithium batteries and this is another challenge. The bigger the battery is, the harder it is for you to get clearance to take it up on an airplane. So it has to meet certain specifications. The cameras are the larger tech ones, we started using small five watt solar panels, question is why can't you use bigger ones? Because what happens is that that angle, the sun doesn't just, if that's east and this is west, the sun doesn't just come up like this, kinda goes around you because you're on kinda that part. So it kinda rises up and goes around you, so which means you have to put the panels almost 90 degrees. So the bigger the panel is, the more wind resistance it has. So you wanna have something that is small enough that doesn't act as a sail, but something on all four sides because you're not just pointing on one angle, you're now catching the sun on all four sides. So we started putting smaller five watt, 10 watt solar panels. We tried, we thought we were clever enough by putting flexible panels, but flexible panels tend to flap a lot and that actually causes more problems. So tightening things is much easier. And so the lithium battery part I spoke about, there are charge controllers that we can use small 12-volt, five-volt charge controllers that control this. And the nice thing is several of them are serial devices. So they actually show up as slash dev, TTY, SO, S1, and so on. So you can actually read off and say how much battery voltage do I have and what's the state of my system and so on. Drivers, they provide, but this was mostly a hack because that particular charge controller was designed for a Windows system and they gave their own drivers. So we had to recompile a whole bunch of those things. Lot of open source tools. And sleep timers. I said I'll talk about this stuff. At minus 40, the clocks tend to go slow. And at higher temperatures, they move faster. So when you ask the clock, say, wake me up in two hours, it actually wakes up much later because now the clock frequency has gone down and it actually, there's a drift happening. And over time, this accumulates. So instead of reporting in and saying every two hours here is your data, it'll report in after two hours, five minutes, two hours, 10 minutes, and so on. And because you don't have, and there's nothing you could do about it because even the battery thinks that's what the time is. I mean, there's nothing you can do unless you can do a connect to an NTP server and time sync it. And those are expensive operations, which means you have to keep a participative connection long enough to do all this time synchronization. We do that occasionally. Once you start to see that occasionally, once every three days or four days, especially during the polar winter, it actually comes up and does a time synchronization. The board also has a particular quirkiness because once the time server thing fails, it resets its date back to 1970s. So sometimes you start to see things like, okay, it's all 2017, all of a sudden, and now you're getting data from 1970 and the clock goes crazy. So we have to work around those challenges as well. The most power intensive system are the transmitters. And the Wi-Fi, the satellite modem, the Bluetooth. So keeping them as minimally used as possible is the best operational method for us. The second biggest power hog is actually the other cameras. So the less cameras, the better it is. Sorry, but I think I've rushed through most of the things. I'll be more than happy to answer any questions. We don't, actually. The problem is, there are advantages too. I mean, we deployed in November and pick it up in July. So we can always do software updates. In the past, we have done updates, but it's not been software updates, it's been configuration updates. So usually you have a CSV file or an XML file and say instead of doing sleep every two hours, I would say sleep every six hours and it's a configuration change, right? Similarly, if I want to start monitoring some parameters, I can do that too. Occasionally, and one of the reasons I put an IP address as one of my standard parameters is, occasionally when from the research site onto the Buoy site, we do have Wi-Fi connectivity. It lasted for a while, meaning then the blizzard came up and took out the Wi-Fi transmitter. But when we had that connection, we could actually SSH from California into the box and actually do our changes and monitor things. That was very useful because we could even play around with it, set timers, run diagnostic on it, do the NTP server, time updates and so on. Very risky to do, not advisable at all. Because if something goes wrong, you have to ask your bear guard who's more interested in handling your gun to go to the site to say, okay, I'm gonna reboot something rather than actually doing that. So yeah, short answer is no, we don't do updates on the field, but when we bring it back, we release the next version and so on. So the question is how do you test it? And we have a big ice cream freezer. It goes up to minus 29, not minus 40, but we believe that anything that we cannot find out at minus 29, we won't find out at minus 40. So usually things like, we find leaks inside that allows cold air. And then when the system warms up because the CPU is running, that condensation shots the circuitry and all. And so we gotta do a lot of these testing. Some of these equipment don't work the way their spec sheet says. So in fact, not some, almost all of them. Sometimes to the betterment. I mean, we've tested modems that are rated for minus 40, but they work at minus 55, not that we needed to test them at minus 55. Minus 40, by the way, is the industrial temperature range for most products. And that's what we went with. In one of my other projects, we tried to, if you go to try to do anything lower than minus 40, there are a lot of export control restrictions on them because they become dual use technology. So minus 40, so freezer is one of the tests. And we don't do anything like liquid nitrogen and things like that is just too much work for a software engineer to handle. But it is testing interesting. And most of the development, once it's done, I would say 80 to 85% of the time is actually testing, testing, making sure why something works, something does not work. And like the camera focus issue was another problem. The reason I said that, the first season everything went fine. The second season when we tried it, there were small icicles sticking on the box and the camera's always focusing on that instead of this imagery outside. So having to have the video option compiled and capable on the camera was actually important for us. So a lot of testing has to be done. When I say testing, not just physical testing, but actually running the full software suite test, start cycle, stop cycle, do it many times to actually get the data out. Both actually. On site is because every site is unique. Some of the sites, some of the tests we've done in Chicago and Minnesota actually face more brutal conditions than there. The temperature is much horrible. But there is also a big engineering community that actually works on the technology part of it, not the science part of it. If you could look up PolarTechnologyConference.org, that's like a conference that happens every year where all the engineers get together and say this works and what doesn't work and what's the cost of it and they share their knowledge kind of thing. So they do share a lot. In fact, the camera aspect was something that I found out from them. They were the ones who said, oh, don't use C910, use C920, that works at minus 40. And of course, we test 10 cameras and find out that nine of them work, one doesn't work. It'll work in your desktop environment, it just doesn't work. Meaning the images would all be pink at minus, anytime it goes to lower temperatures. Actually, that's a, yeah. Yes, that's a very good question and I'll tell you why I was happy that you asked that is because if you just keep the memory alive and keep the process of alive, then your boot up time can be fast, right? Which means you save a lot of time. That's good like in an automobile kind of an environment where you have to shut down and start fast. But keeping that memory alive also consumes power and even though I will like that feature, that computer board does not support it. In fact, the power off sequence is actually by another microcontroller that you can talk through the Linux driver and say, when you say sleep 40 seconds, it'll just cut power immediately, right? So which means disk corruption, disk flush, you better be done with all that stuff. So in fact, what I did was we actually put all the sequence of disk setup, process scale, everything and the last one in the slash initd slash k11 reboot file was just to cut the power off a day. And that worked very well rather than trying to do our own shutdown sequence because the shutdown sequence is handled by Linux and the last statement is cut power to. Well, nothing I would love to, mainly time is the constraint. There is, I do share my findings in fact with the board guys. I mean, the board guys were interested to find out how I did certain pieces of work and I published that data. The Polo Technology Conference is another one that I talk about and publish work as well. Not a lot of Linux guys there but there are a few of them who do some hardcore work as well. They've done some even more serious Antarctic buoys and robots that operate outside volcanoes, inside Antarctica, that kind of thing. So they do some real good work. And so some of their work they use on their real jobs too. So yeah, there is contact me if you're interested. I'll be more than happy to just do part list or even software. Most of it is hack. Just get it working quickly enough so that we can roll it out. Sometimes it's a well thought out processing. Yes, we got to do this. Oh, that's a, okay, let me give you, I've been working with this particular group since 2009. The first time we did it, we had four massive buoys. Each one can handle 60 kilos and the whole thing had to be assembled on ice with metal brackets and all. And so we had like a big electrical box that was waterproof and a big computer and that did a whole bunch of these things. So the reason I tell you the backstory is we found out that when the ice melts and that happens, you cannot even on a big strongboard how powerful you are try to pull something like that into your board because there's a $4,000 cable stuck to a piece of ice that's pulling you and your board down, right? So you want to make sure that you retrieve the data. The very least, you want to retrieve the equipment as such. So we went through multiple iterations and we went through multiple boards like the board that I used was a 72, 40 or some version number that worked well. There were other boards which did an equally good performance. So every time we changed boards, we had to redo, recompile, make sure the ports were assigned properly and all. So to get something like this working tested properly took me about three full-time months. Eight hours a day, three months kind of a thing, right? So of course, this was my volunteer time so I had to do it nights and weekends with a lot of help and all. So on the average, that's kind of the time. But once you get it afterwards, modifying it becomes easy, then you know which library has what and you don't have to find a particular library. So one of the electrical engineers actually built a small probe which you can plug into the board and find out what your power level is. And it has an A2D converter on board so we just read the A2D converter periodically and found out when we start the battery, this is what happens and we plot graphs and actually visually take a look at it. We actually, that was good when we were doing the freezer test. We could, on my desktop, 25 degrees centigrade, everything would be fine, works fine, you can heat it up, cool. Once you start putting it in the freezer, you can start to see the battery go down and then it starts to pick up and so on. So we actually had to build a special probe for it. The newer charge controllers handle it much better because they're meant to do industrial stuff but some of them are heavier and some of them are also more expensive. So one of the charge controllers was a $300 piece of equipment, so more expensive than the computer itself. Speaking of the batteries, batteries, you use light bulbs? Yes. And these are normally not very working very well in their lower temperatures. How did you deal with that? Okay, so the question is that they're, okay, I know we're running out of time but I'll try to answer that. I won't give you a short answer. So when we initially started, one of the reasons why we used that big 60 kilogram buoys was we had a lot of lead acid batteries. The reason for these lead acid batteries are they're cheap and you get the sealed ones, the ones you use in wheelchair, which means you can transport them on a plane. And of course there are some restrictions, you can handle them, fill out the paperwork, you can do that. And so if your battery dies in the middle of the Arctic, it's easier for you to get a snowmobile battery as a replacement and plug it in. So we started with that but then we realized that these batteries, even though they're cheap, they're very heavy, heavy means you have a lot of other logistical problems. So we needed a battery that actually works at minus 40 properly. And the radios, the transmitters, they actually pull quite a bit of power when they do this burst. When I say quite a bit of power, I'm not talking about megawatts, I mean, at five volts it would jump periodically to two amps for like maybe a second or so, right? And then it would drop back. So your other batteries, alkaline batteries and all wouldn't do that. But they wouldn't cut off power either. So what happens is you would see that the motor wouldn't try to transmit and transmission failed is what you would get. Then you would be looking so much into your heart, your code and your software and you would blame your modem for not doing it and then you realize it's actually the battery that's not good enough. And after a while, actually in one of these conferences, the lithium batteries are very good in doing this. They can provide a whole bunch of power. But the problem with these lithium batteries is you cannot charge them at a low temperature. Or when you charge them, you need a very specific charge controller. A lot of the lithium fires that you see are because you try to charge them at the wrong thing and then it leads to a tunnel runaway. So for a lot of times we use just generally AA lithium batteries that work great because you can buy these lithium batteries and put it on. But at this location you can't buy them and since we are flying them, they wouldn't fly them, they wouldn't ship them over. So we started to resort to using battery banks that we use for charging our cell phones because now you could ask one of the bear guards to say unplug this battery, go back to your research station, plug it into the wall, plug, charge it back and come back again and just connect this USB connector and it works. But there are different challenges with these battery technologies and how you connect them to your computer because you wanna have something that's reliable, compact. In fact, we can shrink this a lot, a lot smaller. I mean the box is about DA size. But you can shrink this to like your cell phone size. But the biggest challenge there is your battery pack. The battery packs are big and they're the ones that consume a lot of power. So right now we haven't integrated any power management system into these battery packs because these are detached, low cost systems that you just plug in and use. Any other questions? Thank you, thank you very much. And I'll be here if you want another question.