 The next talk will be presented by Break the System and the talk is entitled The Mars Rover onboard computer. The stage is yours. This took entirely too long. Here we go. Over the last 50 years a lot of missions have tried to reach Mars. Unfortunately Mars is hard and landing on Mars is even harder. So a lot of these failed, especially the landers. Funniest failure by the way is Mars climate orbiter which launched in 1999 and had one component that used the imperial system and one component that used the metric system for measurements and when they tried to exchange numbers the thing came way too close to Mars and burned up in the atmosphere. Sojourner was the first cute little rover about this big about the size of a skateboard and deployed in 1997 to the Martian surface. Spirit and opportunity the Mars exploration rovers or MIR landed in 2004. They're about this large the size of a golf cart I want to say and they're wildly successful. Spirit was active from 2004 to 2010 whereas opportunity was active for a full 14 years until just this summer when her solar panels got covered by a dust storm and she went to sleep. She traveled for a distance of 45.16 kilometers completing a full marathon before the end of her mission. Also funniest landing method ever with a series of airbags just like bouncing across the Martian landscape. Today I want to introduce you to Mars Science Laboratory also known as Curiosity. Curiosity is a completely autonomous vehicle and she has to be because radio signals from Earth to Mars travel about between 10 and say 40 minutes depending on the positions of Earth and Mars. She has an atomic power source the MMRTG so she's completely independent of solar power and Curiosity's top speed is 100 meters per hour that's not a type of one of her coolest instruments is the freaking laser which she uses to zap stones from up to seven meters away that's the other end of the stage here and uses a spectrometer on the resulting gases to analyze them. The rover has spent over 2,200 souls on Mars a soul by the way. As a Martian day it takes about 24 hours and 40 minutes. During that time she drove almost 20 kilometers and she's still going. She exceeded her original mission requirement by a huge margin. So say hi Curiosity. Curiosity has been in different configurations the cruise stage in interplanetary stage navigating with star maps and making course correction maneuvers to plan the perfect arrival on Mars. The entry descent landing configuration with its combination of guidance thrusters, heat shields, supersonic parachutes, retro rockets, fricking sky crane that's a whole other talk right here like that's not this talk sadly. And finally the rover configuration we see now. All of these configurations were controlled by a single on-board computer using sophisticated and very modular software. So let's have a closer look at that today. Quick introduction. Hi I'm Daniel. I'm a software engineer. I'm an amateur space nerd and I learned a lot about space hardware and software when trying to build satellites for my startup. I work at Keep Safe creating privacy-protecting apps. Thanks for paying my hotel room by the way. And I want to look at the Curiosity mission today through the lens of a person who writes software and uses hardware. A quick disclaimer by the way. I've used work of these people here, Emily Laktawala, Dr. Katherine Weiss and Dr. Mark Mahmone extensively in this talk. And I admire them greatly, but I didn't have the time to contact them. So if you find any and all mistakes in my presentation, they are mine and mine alone and not these people's mistakes. Let's talk about Curiosity's hardware. This is the BAE Red 750. A radiation hardened version of the IBM PowerPC 750. Created by BAE's BAE systems, it can be clocked up to 200 megahertz and it's used in space a lot. On Curiosity, this chip is clocked at 133 megahertz. This is the chip and this is the chip within its board. You see two different boards here, one is, or board sizes here. As far as I know Curiosity uses the board size on the right. On the board, we have 256 kilobytes of EEPROM. We have 256 megabytes of RAM. And we have a whole lot of two gigs of flash storage. And we have all of that twice. In case something fails, the rover can switch from the A side computer to the B side computer. They are both completely independent. They also have a few components that belong only to one side of the computer. For example, and this is going to be important later on, the navigation cameras. Of course, all of this is radiation hardened. And there are a lot of techniques to radiation hardened a chip and board. And this is another talk that I would like to hold someday, but today is also not this talk. So I'm going to leave you this slide and tell you that the last space shuttle used a 386 equivalent processor. Hubble has been upgraded to a 486. And the international space station uses, or the command computers, are using 386s. This is because radiation hardening is hard and takes a lot of time until the chips come out. So space stuff is always very, very slow. What kind of software can we run on this hardware? Turns out, we're using VxWorks 6.7, a simple real-time operating system. This screenshot is seven, but it looks cool. VxWorks is also used in other NASA spacecraft, various SpaceX spacecraft, the Boeing 787, the KUKA industrial robots of my hometown, Augsburg, and also the Toshiba eBridge range of photocopiers. And before you ask, yes, it can run Java, but in this case, it doesn't. The software that runs on Curiosity has a long lineage. There has been a continuous code base that started in the 90s when JPL wrote very code for their prototype rovers. And then part of that was used in Sojourner. And then that got rewritten and whatever and was used in the Mars exploration rovers. And then it was ported from C to C++ and refactored into a component-based architecture for MSL, aka Curiosity. And yes, we are using C++. Not full C++ as you'd use on a desktop term. One thing that's missing is processes, for example. VxWorks gives you this concept of tasks, but they're not as independent as processes. They all use one big chunk of memory. And it's up to you to make sure that they don't, like, stomp all over each other. We also don't have exceptions, templates, IO stream, multiple inheritance. And the only operator we're overloading are the new and delete operators that are used to allocate new memory for objects or for things and to deallocate that memory. And they are overwritten by the custom memory allocator that JPL wrote for VxWorks. This is a pretty cool piece of software, because it guarantees graceful access to these defined memory pools. And it has that well-defined behavior for out-of-memory conditions, which is very important, as you can imagine. It supports multiple pools in different areas of the RAM, so that's how you can separate your not processes. It provides diagnostics. There's a display that shows you each new and delete operation, and it can also give you a map of the RAM on request. The rover can even downlink a map of its RAM, if needed. The same allocator is also used for development on JPL's Unix machines, which is, of course, very helpful. And there's no garbage collection here, which would be too expensive. Instead, you have to be very, very careful not to have any memory leaks, and these are usually enforced by unit tests. If I add a title of something, by the way, you should be able to Google it and watch it on YouTube. Those talks are super interesting. One of the philosophies behind the flight software is its component-based architecture. Functions are grouped into components, and a component exports an interface and identifies the interfaces that it needs to function. That means that you can replace any and all components of the system as long as they export the same interface. It also means you can work on your own component without having to touch all the other parts of the system. This is very helpful for development and also for testing and extensibility. You'll also also notice that the components here are organized in layers. The lower-level components deal with the hardware directly. They turn switches. They move actuators. They prepare data for being sent back to Earth. They also abstract away redundancy components. There's a lot of redundant components where just a thing is just twice in the rover in case one of them fails, and the lower-levels will automatically fail over to a backup component without the upper levels having to know the intricate details of what exactly went wrong. So they will get a notification that says something went wrong and I'm using the backup, but they don't have to know how the backup is activated or how it's being used. Further up, we have components that coordinate these very atomic functions into actions. And finally, we have components that group these actions into whole activities like land on the surface or take and analyze a surface sample. The land on the surface module, by the way, is not on here because it was deleted right after the spacecraft actually landed on the surface. The most complex module in the whole code base is the surface navigation module, which was in some form also used for the Mars exploration robots already. On MIR, it was about 21% of the full code base for MSL or Curiosity. It's about 10% of the whole code base is just the surface navigation module, path-finding, moving, that sort of kind of stuff. There's about 100 individual or actually over 100 individual source code modules, each with its own owner. Another philosophy is fault protection. Mobility and science modules should not have to worry about the spacecraft safety. Instead, the infrastructure in addition to providing the platform to do mobility and science must keep the system safe and away from the dreaded safe mode. Unsafe actions should not be allowed and should be blocked. The pattern here is fault should be detected and corrected as low in the layers as possible. Finally, we have just loads and loads of testing. We have unit tests, we have static analysis, and we have validation and verification where we run code on the various test beds like this complete duplicate of curiosity down on the ground. And then there's test as you fly. Once the software is uploaded, behaviors will be tried out and tested on Mars during the mission. So this is really cool. Curiosity's flight software can be remotely updated from Earth. So engineers on the ground can change the software and introduce new functions and behaviors or tweak the landing code. The mobility module, by the way, the thing that allows the rover to move her arm and drive around, was actually developed while the spacecraft was already in cruising stage towards Mars. It was not part of the launch load at all. It was uploaded once the rover was safely on the ground. It's kind of like the console you got for Christmas. You unpack it, you plug it in, and then it just has to update for a week in this case. How does the software update mechanism work? First, the software update is uploaded into the active computer's RAM and run there. And if the system behaves erroneously, it can reboot and load the old version from its flash storage. Then the rover is going through an extensive series of tests to see if everything works as expected. If that passes, the software gets uploaded into the flash storage and booted from. After another series of tests, that half of the computer is updated, and the process then gets repeated the same way for the other side, because we want to have both computers ready for anything in case we need to fail over to the backup computer. Rover days start around 9 or 10 in the morning local martian time when curiosity boots up from her dream mode. While in dream mode, various FPGAs already flip on the heating system and prepare everything so that the rover is warm once the main computer is booting up. And then she gets directions from the ground and sets off for her day. She drives, she collects data, she runs experiments, she drills, she zaps. It's pretty fun, but it's very exhausting. The day ends shortly before sundown when the last satellite pass occurs. And that is used to upload the last bits of data for the day before the rover goes back to sleep. Curiosity does have various antennas, and some of them could talk to us directly, but that would be at the speed of about a 14.4K modem. Using the relay satellites, the data rate goes up to about 500 megabits per day. That's about 62 megabytes per day. So the data with the highest priority gets sent pretty quickly, and the rest can just follow at some point. And then she needs to go to sleep. The MMRTG the atomic power source produces about 110 watts of energy. The rover needs 45 to 70 watts even while sleeping, 150 watts while she's awake, and up to 500 watts while driving. So she spends most of her life asleep and recharging her batteries, being awake for about six hours each day. Lucky her. Next I want to talk about how the driving part actually works. Rover drivers use a system called RSVP, the Rover Sequencing and Visualization Program to plan out drives. This has been used and updated over the years, and has been used for previous rover missions as well. In RSVP, the terrain data that curiosity sends back is being reconstructed in 3D, and the human rover drivers can look at it, like fly through it, do local simulations of activities, some sanity checking, does this really make sense in this context, and then send orders to the rover. These orders could for example be blind driving. The easiest driving mode is blind driving. Just point the rover to any directions and go for 10 meters just blindly. This is dangerous in cumbersome, however, because you can only drive as far as you can see, and even then, like the further away things you can see, like maybe there's things hidden by perspective, and then you have to wait for more orders from the ground. Even then you might run into obstacles. The thing though is that this is way easier computationally and way more predictable, so it was used a lot, especially in the beginning of the mission. And we have visual odometry. Odometry is finding out how far you've come by counting wheel revolutions or other things. The thing is, odometers in wheels don't work here because curiosity can't detect slippage in real time and the metal wheels, they slip a lot on rock and sand. Instead, what she does is she takes before and after pictures with her stereoscopic cameras and compares them. She will auto-select various features in the terrain that she finds interesting and follow them along as she drives, so she will go and then see like a super interesting rock, just like look at the rock for a while and see how far she's come. This is all done on the rover. This is not being sent back to Earth or something, so this is a completely separate system and it works very, very well. And then there's AutoNerve, where the rover drivers set a target or a destination location and the rover tries to find her way towards the target completely autonomously while avoiding the red no-go areas that are designated by ground control. Curiosity has stereocameras and takes pictures with them. Features in the image pair are correlated and triangulated to get the distance of these features. The range data must satisfy a series of tests before being seen as correct, like unclear and misleading parts of the image like pixels that are only in one of the images or parts of the rover are being ignored. Using these stereoscopic images, Curiosity will then create a geometric model of the surrounding terrain and subdivide it into grid cells of the resolution of a rover wheel. She will then rate each of these cells according to its traversability, how much she'd like to drive there. Taking into account slopes, steps, roughness, excessive tilt and other environmental dangers like things that just stick out too much from the ground and would scratch her belly pan. These obstacles are expanded by the radius of the rover so that her centre will not touch these things and she will stay far enough away from them. This gridded traversability map is used for each step of the navigation. So she will then project a number of paths onto the map and evaluate them. Is this safe? Will this get me closer to my destination? She then chooses the safest path that gets her closer to the goal and drives a short distance on that path. And after each step, the navigation process starts again. Until the goal is reached or no safe path is found or the rover is commanded to stop. To find her way, Curiosity uses a version of a star search optimized for driving tiny increments at a time between map updates. Because of this optimization, the algorithm is much faster and more efficient than regular star search and therefore better to use on the rover. Autonav is still too slow to use all the time. But as the mission went on, the rover did more and more autonomous driving because people learned how to trust the rover and give it the correct directions and also the software was improved over and over. This is Curiosity looking back at her very first drive. She surprised the engineers here because she avoided a very, very tiny rock. You can see it at the top where this little corner. So she went straight and then there was this rock and she was like, oh, this is very dangerous. I'm going to go out. And you've seen this before. This is basically exactly that drive being replayed in higher speed in RSVP. And look at the head moving around and also the other cameras. You can see them, but they also take pictures. And this is visual odometry and Autonav working. All right. What other features do we have? We have the Mali camera. The rover has an arm. And at the end of that arm, there's a camera. Turns out it can actually take pictures of itself. For example, of the belly pan or of the wheels directly after landing to see if they survive the landing correctly. Using that arm, she can also take larger panoramas by taking multiple pictures and therefore cover her entire body. This is what the result looks like. This is Curiosity's very first selfie. If you stitch them together, they look pretty cool. Like they are amazing pictures and I love to look at them. The arm is usually photoshopped out because you will only see half the arm all the time because it doesn't fit into the panorama. Let me tell you about what happened on Soul 200. Suddenly, the rover couldn't save data anymore. Transmitting images live back to the ground worked fine, but saving them for later upload would fail. The rover also rebooted multiple times without apparent reasoning. In the end, the engineering team decided to switch to the B site computer. From there, they could assess the damage and it turns out that the A site computer has a fault in its flash. You can't really tell what it is, but they have now disabled half of the flash in the A site's computer and this seems to work fine. Curiosity was on the B site computer from then on. This is more work than you can imagine because, as I told you earlier, there are various cameras, especially the navigation cameras that are directly connected to each of the boards. So if you switch from the A site to the B site, turns out that the cameras, they're slightly offset and this gives you different angles and these have to be programmed into the software, of course, or it will give you faulty data. And then it also turned out that during the heat in the Martian summer, these cameras warp slightly and this warp like warps the images a little bit and that makes the triangulation of the stereoscopic images very hard. So they also had to write code while on earth just by looking at the pictures, how to calculate out this swapping. It's a whole thing. Exactly one year after landing on Mars, Curiosity celebrated her first birthday. This is SAM, an instrument that includes a very exactly controlled sieve that can sieve with these perfectly programmable vibrations. At the first birthday after landing, it was programmed to play the following vibrations. Imagine the tiny rover Curiosity sitting in the Martian desert just like singing a birthday song for herself. This only happened once, by the way. You hear sometimes on the internet the rumor that this happens every year. It happens exactly once in the first birthday. It has been six birthdays, five birthdays since. So and those were all just regular working days. After some time on Mars, engineers noted that the wheels were getting torn up way quicker than expected. The wheels, they are only about a millimeter thick and they're made from machined aluminium. What happened was that Curiosity ended up in terrain that wasn't experienced by Opio, Soduna. Normally these small rocks, as you see on this picture, for example, when the rover drives over them, they get just pushed to the side or pushed into the ground. But this clearly is not happening here. The terrain is ripping deep gashes and holes into the wheels. So JPL engineers on the ground, like Amanda Steffi here, constructed various ground testbeds that simulated different new ground conditions and tested the wheels to failure, as they say, so that they know exactly which types of terrain are the most dangerous to drive on. And it turns out that if you have rocks that are cemented into the ground and they have very sharp edges at the top and they don't really move aside because they're completely fixed in place, then the back wheels will force the front wheels that can't really move because of the sharp rocks onto those rocks with the huge torque that the rover has. And this leads to these punctures. So the driving software was changed to allow the wheels to move at different turn rates. So to avoid this pushing onto the sharp rocks thing. Also, the rover has been very carefully steered out of the rocky areas and is now back on the more sandy areas that she prefers. On Sol 2172, that is September 15 of this year, another computer problem was detected. Curiosity was completely healthy, but could not access parts of her memory where she stores data for later uploads. This is clearly a serious problem, but it doesn't endanger the rover's safety. You'll notice no reboots this time. For now, the rover was switched back to the A side and is merrily doing science again while engineers in the background are trying to diagnose the problem. And you remember the A side is safe until as long as it doesn't address the faulty part of the flash. What does the future hold for Curiosity? So Curiosity's primary mission was to launch from 2012 to 2014. And she's still going strong in 2018. So that's pretty cool. Turns out a few months ago at a conference in Toulouse, NASA and JPL engineers said that there were about 10 kilometers left in the wheels if conditions are about the same. We can also expect the MMRTG power source to last for another 10 years or so. So if nothing happens, she will live on for a while. She has more or less already reached her goal. This is Mount Sharp aka Aeolus Mons, the central peak in Gale Crater. And Gale Crater was once filled with water. And because there's lots of different sediments at the bottom of Gale Crater, there's a boatload of interesting science to be done here. And after driving for 20 kilometers, Curiosity is now in the foothills or as JPA calls them the butts of Mount Sharp. So there's lots of cool stuff to look at. In about two years, we should see the arrival of Mars 2020. Curiosity's sister rover. It has the same chassis and the same body, same software more or less, slightly updated wheels and a completely new set of science instruments. It will also be the very first rover to include a microphone so that we can hear what the surface of Mars sounds like. And maybe, just maybe, if we listen close enough, we can hear a tiny, far away birthday song. I guess we have lots of questions. There are microphones in the in here in the aisle exactly. And we start with the microphone too. Hi, thanks for your talk. Two small questions. First, why you call the rover Xi? And the second one, can you say something more about error handling? Does the software have different kinds of errors which are treated differently? Right. Good question. So the first question was why is it a Xi? And it turns out that JPN at NASA call all their rovers by female pronouns. So I just did the same because just seems nice. And the second question was about error handling. There is a whole pattern that is being discussed by Catherine Rice, for example, one of the software engineers and architects of the software where there's like a whole error detection patterns and they do it all in the same way. They have like three status codes, green, yellow and red, where green is just everything is nominal. Yellow is something happened but the lower level component was able to fix it by, I don't know, like routing something around or switching to a backup component or whatever. And then there's red where the component just says like, okay, something higher up in the chain has to deal with this. So this way they can like bubble up these error status codes. Okay, we have already more questions than we can take, I am afraid, but we continue with number five, please. Okay, thanks for your nice talk and I want to know how does the image recognition work for the past finding? Does the rover compare the images or does it have a kind of LiDAR sensor or how does it work? It is all just regular images so there's no LiDAR. I'm looking at my backup slides just now because I have a lot on this. So basically I'll find it later. Basically it's just regular image recognition. There's an algorithm in there called Rockster which can recognize rocks in the sand like the ones you see here actually by their contours and also because you have stereoscopic images you can calculate like a 3D map of the terrain. Number four please. Hello, I've heard that some components or this echo are controlled by Ross Robot Operating System. Is that right? Robot something rating system? Ross Robot Operating System. It might be true and I have to be honest I haven't heard of that term before. So that's the edge of my knowledge apparently. Number eight I guess. So when the autonomous driving software was designed would be probably around 2005. Today we have autonomous cars being designed for pure cameras. Is NASA on a dead end a different path? Will they integrate what is being designed today for cars here or is this going to be completely in a different frame which is not compatible? Are they going to augment or rewrite from scratch? So I don't have any affiliation with NASA but if you want to hire me I'm here. I don't have any affiliation with NASA so I can't speak for them. To me it seems like it's two very different use scenarios where the one where like autonomous cars like they use all kinds of like detection specific to roads whereas the rover has to navigate a rocky terrain but I can imagine like because there's a lot of research going on that and this research is public that they might take the best bits of that research and integrate them like these people are incredibly smart. Okay we take one question from the number six. Is there a specific reason why there's no radar or leader sensor included because I think that would make getting 3d images more easy? I do not know of a reason it might be because it's just like the whole design is basically from the early 2000s maybe Linder wasn't wasn't good enough back then. What I can recommend to you by the way is this incredibly good book the How Curiosity Does Its Job by Emily Laktawala from the Planetary Society like this will definitely answer this question I imagine but I don't know. Number one please. What about security? Are updates signed? Is communication encrypted? I know there's parity bits and hashes going on well that's your answer. Number four please. Hours per day now I'm not sure how many hours in Mars per day but how it's organized herself to work in different times per day or it's in the same time every day. The rover always works in its own or in her own time zone so if I say 10 o'clock I mean 10 o'clock in the morning on Mars so about a few hours after the sun has risen. A Martian day a soul has about 24 hours and 39 something minutes so it's very close to an Earth Day but there's some movement between Earth Days and Mars Days so there's some offset that's like worse and worse so for the first year or so or maybe even two years the JPL team that controlled the Mars Rovers actually worked in shifts that corresponded to Martian days and that led of course to night shifts, weekend shifts, all kinds of stuff and they have since created this whole system where they can actually work just during the Earth Day and it results in about these 30 days stretches where the rover still gets new updates and orders every day and then in between there's always like a few days where just the rover just sits there doesn't have anything to do because the people need to sleep and during that time it will just upload all the data that it has collected because of the slow uplink speed. Okay two more number five and then number two and then we close the session. Hi great talk. On the two incidents when they had to switch the computer sides do you know if they found out what the cause of that was? Was it like cosmic rays trying something or the thermal expansion working? They did not like this is the thing like there's also other incidents that I didn't talk about like for example the drill broke at some point and in all of these cases you can't really find out what's going on because to do that you would have to go there open the lid and just look in and see the things so like all they know is what their software can tell them so their software can tell them it's crashing when it accesses this part of the memory so don't hold it that way. Thanks. Okay and finally number two. Hi those two computers named A and B are they of the same hardware design or are they different? Okay they are both both the chip that I that I told you about earlier the BAE 750. I'll be in the bar just outside the exit the overflow bar if you have any more questions find me there. Okay with that let's thank the speaker again.