 Okay. Hi, everyone. I'm Rachel, thanks. So a little bit about myself before I get into my talk. I have been in data visualization for a while. I got my start at Stamen Design, and then I was doing freelancing for a while, and then up until recently, that brings us to JPL. So JPL is one of the NASA centers. They're responsible for most of the past-earth spacecrafts, so that sort of thing. So basically, you could say that really what my job was was data-vis for space robots, which is how this is pitched to me. Of course, I was like, I have to get to that. So what I was visualizing was telemetry, which is the generic term for data that's transmitted. So over the course of my time there, I worked with four different missions. There's SMAP, which is an Earth Orbiter, dealing with climate data. Cassini, which is around Saturn. Dawn, which is in the asteroid belt. And Curiosity, which is the rover on Mars. So basically, when I was being asked to come and work there, the pitch was basically that Elasticsearch had just come to JPL, and it was this new exciting thing. And then suddenly, search was fast, and so they could return data fast. And so once they had that in place, they needed a front-end. They needed something with D3. And it was basically like, what will you come up with? And I was really excited, like, okay, yeah, I'm going to go to NASA. I'm going to make cool visualizations. Everyone lives NASA, right? So I was really stoked. And it's funny, over the course of time that I was working there, some friends of mine in the visualization world, they're like, oh man, what are you working on at NASA? And my answer was like, well, I'm doing time series data. I'm making line charts. Yeah. And their face would kind of fall. They're like, oh, you're doing line charts. And yeah, it's time series data. That's what you need to look at it. But I'm ready, here we go. So the thing is that you can see that there's just like this D3, SVD line kind of thing being called, but there's actually all these other things that are going on in the chart that of course I can't really show you because it's all ITAR controlled. But all of these features that we built into this glorious line chart, but unfortunately to you, it just kind of looks like this. And so when I was putting together this talk, I was kind of thinking like what to include, what to talk about. And this sort of gets into the same sort of debate about like UI versus UX, but it's the debate of kind of like what is important enough to be called like the visualization, what do we include? So I figured I might show you the charts in context. So this is what it looks like. And I bet that most of you sort of zoomed in and focused on that chart area. I would do the same thing. And you're kind of looking at it, you're like, eh, she's got some little bars in there, but it's mostly a line chart, got some massive G-axes, like we're good. Now the thing that you probably didn't look at is this. And what you can't possibly know is that this took several weeks to develop in both research and testing, prototypes and putting it in front of users and saying does this actually, actively convey your needs. We've got time bookmarks and this varied by mission. Different missions wanted to look at different chunks of time. They wanted to look either from most recent time or from the end time. We've got your start and your end text boxes that has UTC day of year time formatting. And then below you've got this little time slider that you can be dragging to zoom in on the graph. And what you couldn't possibly see also is that that little calendar icon opens up this date picker. You'll notice that there's both the date and the day of year in this date picker as well as the week number on the side. Now this seems like a small and minor detail, but when we were showing this to the missions, it's actually opening this that really got people's attention. They're like, oh, oh my God, I should use this. Because right now their current process is to print out a calendar for each year, pin it up to the side of their cubicle, and then every time they need to convert between date and day of year, they have to go look that up. And so even just having this, that they could click on something and quickly go to the date time that they actually wanted, was like saving them precious seconds. And so people got really excited about that. And so I wanted to address for a bit before I get into the meat of my talk, kind of what we talk about when we talk about visualization, like what gets to count. And to get into this, I want to talk about the sort of design and art spectrum. And there are many different definitions for this, and I think we all have our own. One of my favorites is that design solves problems while art creates them. This is sort of cute and pithy, and like everything pithy, it sort of generalizes. But sort of a more general definition, at least that I'd like to propose for this talk, is that art is sort of about taking your own experience or your own thoughts and kind of projecting it out into the world and saying, hey, you should think about this too. Design, in contrast, is about taking the perspectives of many other people and synthesizing them, saying, okay, how can I build something that many people understand, that many people are going to know how to use, and that will be clearly communicating my data. I think that we like to have data visualization sort of situated halfway between these and say, ah, it's this happy medium where I get to take data, this really cold sterile thing, and I get to apply some creativity to it, and voila, data visualization. I'd argue, however, that it's much more like this, that data visualization is really a subset of design and should be treated as such. Now, this isn't to say that art is meaningless or something and if you're sad about that, there's lucky for you this nice little term, data art. And this is not to be little, any sort of art or side projects or anything that you might do with data. I do this in my spare time, but for the purposes of data visualization in what we are being hired to do as data visualizers, I would argue that it's basically design. Data visualization is not your creative outlet. It's not your free expression. It's not your time to just put stuff on a page. Data visualization is about making data understandable. It's about making some sort of interface that people can actually use and get value out of. So, at this point, you might be thinking, Rachel, what if you just didn't try hard enough? Like, what if you're just sad that you went to JPL and you thought you were gonna do all this fancy stuff and then you just had to do line charts? And that's a valid point. Maybe I'm just bitter. But sort of the answer that I have to that is that when we were building all of these tools, we were doing it with user-centered design. And I bring this up because it was the first time in my data visualization career that I was really able to take advantage of this, both in that the entire user base of the tool was at the same campus as me. And then also I was working within a human interfaces design group. Now, in contrast, this is sort of my process for data visualization, you know, which was sort of like, okay, yeah, I get the data, put it on the page, find something cool, build the thing to show off the cool thing that you found in the data. It's done. And I bring this up to say that I've spent actually most of my career kind of in this haphazard methodology. And I still remember one of the first talks that I gave, I was showing some Twitter visualization and I was showing off all the animation and all that. And one of the questions afterwards is like, oh, how do you know that the visualization that you've built is understandable? And I was like, oh. And I think at the time I was like, oh, well there's no possible way I could have done research because it was two weeks. And kind of like wrote it off, but afterwards I'm like, oh my God, what have I been building? Terrible visualizations that no one actually knows. And I just think they look cool and I've kind of left it at that. So one of the wonderful things about working at JPL was being able to have, taking advantage of these more traditional design processes, which is something that I think that data visualization as a field could stand to take a page out of the design book. We're able to do research, we're able to come up with these interfaces, draw them on paper, put them in front of people, test them, see like, does this actually solve your needs? We are doing job shadowing. How do you actually use your data? We know what sort of things could we help you with? Like what could we build? So for the rest of this talk, I'd like to take you through two features that we built. I figure I'll put my money where my mouth is and actually show you what we made. But before then, I do have to teach you about how spacecrafts work. So hope you don't mind. We're gonna do a little telemetry 101. So there are three main types of data on spacecrafts. There is EHAs, EVRs and data products. EHAs are basically a stream of numbers. So you might have the temperature of a component. So it's sampled at about every 30 seconds for some spacecrafts. And there's also a way to encode states of information. So you might have like on or off as zero and one. And then that gets transmitted over radio waves and then decoded on the ground. Secondly, there are EVRs, which are event records, which are basically a print statement. So for EVRs, you have some logic in how the spacecrafts operating and when certain triggers happen, then you get this console log basically coming out with your print statement. The third is data products, which is this sort of cop out everything else. But this might involve pictures or more highly sampled rates of data. But for the purposes of today, we're gonna focus on EHAs, which are basically numbers and EVRs, which are words. And that's it, see? You've learned everything you need to know about spacecrafts. Okay, so to put this in context, Cassini has a couple thousand channels, a couple thousand EHA channels. And they actually are too old to have EVRs. Which is kind of a funny thing that Cassini, which launched in 97, is too old to have print statements. So they have been doing all of this only with these channels and basically comparing on the different states of the spacecraft, which they decode, and then they line up anything that goes wrong with a temperature going out of bounds. And that's how they operate the whole thing, which is insane to me. Voyager, which launched in 1977, has about 100 channels. And most of them have not changed a value in the last three years. Don has, I don't know, a couple thousand channels and they get about 100 EVRs a week. So for Don, every single EVR is a pretty important thing that they need to know about. It's like the start of a maneuver, the completion of a maneuver. So they take those, there's a lot of weight in those. Don also gets a downlink, which is when the data comes back once a week. So once a week, everyone gets all the data and then looks at it and then if anything goes wrong, they start working out a plan and then send up the data in the uplink. SMAP, however, which is an Earth Orbiter, is a near constant communication with the Earth. So SMAP has about 25,000 channels and they get about 25,000 EVRs a day. So it's just this stream of text. And I've watched people sit there, mostly when they're really stressed out in crisis mode and just watch these EVRs go down the page. And I don't know, it's insane to watch just that volume of data coming through. The Curiosity River has 25,000 channels and about 30,000 EVRs. Curiosity does a downlink about every seven hours. So there's a lot of data coming back from these spacecrafts and there's also a different cadence on how often people are getting data, how often they need to look at it and how much time they have to create the plan for the next day. Curiosity currently has a 20 hour sort of planning cycle where they get the data, they analyze it, if anything's wrong, they go and fix it. If not, they make the plan for the next day. Plan, for example, in this case, would be like, okay, we wanna drive over to that rock to go drill, so we're gonna make a little route according to what rocks we see in the middle and then we're gonna try to drive and if we complete our drive for the day, then we do the next little leg and if not, then we're gonna reroute. It was very complicated and there's hundreds of people around the world working on that one specific rover. Okay, so case study number one. So there was this concept called the Gumball that was there when it came to JPL and this was sort of pioneered on SMAP and Curiosity which are both the more active missions that are getting downlink daily several times a day. So what is situational awareness? It's basically this, you're driving and then you hear a little ping and you have to look over but you're still driving and you kinda need to get the data but you also need to be keeping your eyes on the road. So it's hard, you wanna bubble up information but not overwhelm people by how much you're bubbling up. So the Gumball is basically this renamed stoplight idea that there's green for everything's good and then there's yellow alarm and then there's red alarms and your data gets situated among those. Most of these Gumballs were rollups for different subsystems. So you might have the telecommunications subsystem would pick a couple different channels, maybe 10 or so and they would set thresholds for what the value should be and then basically you sum all those up together and if they're all good then you get green and as soon as one fails you get yellow and red. There's a yellow and red alarm thresholds for every channel. So you might be thinking, wow, that doesn't seem like that would be that useful. It's just this rollup that seems insufficient. So when we came in we're like, okay, we're gonna make the coolest stoplight ever, we're gonna redesign this Gumball we're gonna have so much more information. And so we did, we went out and did all this user research, we said, hey, do you like this? People are like, no, I don't. We're like, oh, okay, well then why don't you like it? They're like, well, it's very simplistic. If everything's okay, that's good for right now but is it trending towards something more dangerous? If something went wrong, how do I quickly find out what went wrong? How do I diagnose this? One of them called it the idiot lights in your car or just, you know that something's wrong and suddenly you're like, oh, I feel disempowered because I just know something's wrong but I don't know what it is. So this is the proposed design that we came up with. This is for SMAP. This is the nine different subsystems and each subsystem is this, you know, sort of rectangle and then they've chosen their channels and you have your channels that are like a value range, like temperature, you know, has this bar slider showing where in the range it is and then the ones that are boolean that should be like a certain value or a certain limited set of values, those are the solid colors. So green, everything's all good and you see some yellow, mostly the booleans that aren't working but there's also some of the ranges that have been exceeded. That super dense one is thermal because they keep track of lots of things. And so we showed this to people and we're like, hey, look at this, now you know where all the values are within each range, isn't this so much better? And people kept saying, oh yeah, I guess that's better but really what we had built was more like this. We're suddenly, we're just overwhelming people with information being like, okay, there's all this stuff going on and you don't actually know if that's the information that you wanna be looking at at that particular moment. And so at the end of the day, we kind of realized that this is all people want. They want to just have something quickly saying, hey, everything's fine. Now, once it's not fine, oh yeah, then they want all that stuff, top of the information on the screen, they wanna know exactly which channels are in alarm, they wanna be able to get to graphs quickly but until that point, they just want this. Case study number two would be these event records. So I mentioned the EVRs, those console log statements and I was sort of picking this background image to be funny but this is actually what most people were doing with these event records is they were just grepping through some amount of time and then trying to find the event records that corresponded to their subsystem that they needed to look at, which is very functional but it's hard to know what's there if you don't know what you're looking for. So just to show you, this is a table that we built. This is basically duplicating the existing tools that they had right now. So you've got your different level. There's seven different levels. There's three warnings, warning low, warning high, fatal and then four like activity high, activity low, diagnostic command. So you can see how there's sort of this information, it's very text based and there's also a time. Skept is the time component. Fun fact, spacecrafts keep track of three different times. There's spacecraft emitted time, earth received time and spacecraft clock time. Spacecraft clock is an incrementing number and the funny thing is that the deep space network that it processes all this data chops off the millisecond from Skept time. So if something happens in the same second, you have to look at these clock time, spacecraft clock to actually correctly order your event records. Time, it's really hard. So basically what people are doing right now is they had this, like a text box to be filtering down and then when they found something that they wanna look at, that they wanted to examine, they're like, okay, this is an event record that I need to look into further. This is what they would do. They would write it down. They would write the timestamp down on a piece of paper and then they would go to their graph program, type in that timestamp and pull it up, which, wow, that's so inefficient and so that's something that we wanted to improve on. Now I know that this just looks like a simple histogram to you and it is a simple histogram but this is also the piece de resistance of what we were building in so far as Elastic Search gave us these nice rollups. They gave us buckets of the types of EVRs that were coming down. And so we saw that, it was like, oh, we can put it in the bar chart. Pulled that up on the screen and showed people, like, hey, is this interesting? And people were like, oh my God, because they had never seen their data like this before. If all you've ever seen is a table, actually seeing this, like, oh, there's a cadence and frequency to my event records. What? Which, I mean, it shouldn't be mind blowing but it is because you're limited by the tools that you have access to at that time and so suddenly you can see that like, oh, daily there's, when we upload all the commands and diagnostics and it starts the new cycle, like yeah, there's a spike and people didn't know that before. So we built this and I'll show you what it looks like in context. And the other part of that histogram is that we use that as sort of a filtering method so you could deselect types you didn't wanna see. It was also had a nice brush so you could zoom in on a period of time that was interesting to you and filter down quickly in your table to the event records that you wanted to see. And so a lot of what we wound up doing in this project is sort of cross-pollinating these different data types that had previously been very siloed that people had to look, grep through event records and then go to another graphing program that literally took minutes to create a graph with the query that it was doing and then suddenly we had this nice webinar face that these were returning in seconds. And that was sort of, the best part of working on this project was getting people to think of event records as points in time and so we built ways to have event records show up on the graph and so you could quickly jump between these different data types and do these analyses faster. So some lessons learned as we were doing this. So the primary thing in building these data visualizations is to know that user. This is a photo of the SMAP team in the mock. One of the things that's important to note about these visualization pieces is that all of these missions had been running for several years, if not decades, by the time that we came in and so there really wasn't a lot of kind of innovation that we could do of like, hey, instead of you doing your job that you've been doing for several years, how about like I make a graph and then you don't use your job? And like you have to remember that like if someone screws up, it's on them and you can't really fix it. You know, like this thing is hurtling through space. Like you can't get the spacecraft back and so you have to be very, very careful in the analyses that you're doing. And there's honestly comes down to a bit of trust of people saying, you know what, I have my own process and that's very nice that you've made a little web mine graph thing but like I'm gonna stick to my process because I know that it keeps the spacecraft running. And so what we wound up being used for was more anomaly times, you know, when something did go wrong and previously people basically had to do a best guest of what data to look at because the queries took so long and they had limited time before having to make the new plan. And one of the things that our tool was able to accomplish is people were able to look at more data more quickly, cross reference, you know, actually investigate like, oh okay, have I seen this before? When have I seen it before? What is causing this? What do I need to do to change? So we were, you know, not an everyday tool for them but that's okay because they had their process and we were coming in when they already had that process. The second thing is that visualization literacy takes time and you know, we could have come in and just send like, okay, you're gonna get steam graph and you're gonna like it. But like, you know, for a lot of these people they're in this sort of high pressure time sensitive role and they don't have the time to sort of like play around with something and figure out like, oh, am I learning something new from this? And what's interesting is that when I came in there was actually an engineer on the MSL team who had sort of self taught JavaScript or taught himself JavaScript and then taught himself D3 and was like super stoked that he was able to make visualizations and this kid was like the poster child for this where he would just like add all this stuff and I'd be like, Matt, why are you adding this? He's like, well, cause I can, like it only took me a couple of hours. He's like, now I have a stinky diagram. I'm like, is this useful? Like, does anyone get anything out of this? He's like, I don't know. And so that's kind of what we were trying to not do with our sort of user-centered design approach. We were trying to make sure that people understood what we were putting on the page that you know, it was actually useful and not just kind of gratuitous. And so in closing, I'd like to kind of reiterate that I think that intuitive visualization will always trump shiny and you know, maybe it's a bit harder to show at a visualization conference but I think it's important to be building useful tools. Thank you. Sure, let's do questions. At this job or elsewhere? That is such a good question. I don't think I have an answer to it. I mean, I guess the answer is that the process is that I, I don't know, kill your darlings in so far as like, you know, make lots of sketches, try lots of things out and then don't get too hung up on an idea that you have to let go of it down the line, you know, like try to be, you know, iterating and testing frequently so that you don't, you know, go super far down and have created this beautiful thing and people are like, actually this doesn't work. Unfortunately, I have made things that I think are kind of difficult to understand when I was at Stamen but a lot of them were marketing visualizations, you know, it was like the point was to get people looking at it rather than to really necessarily understand the exact percentages of Twitter people talking about Beyonce versus Kanye. So, you know, like, it's kind of like candy but I would try not to go forward in my career and still be pulling in those types of visualizations. So, unfortunately for that question, we basically are not trying to tell a story and that was a very deliberate thing that the engineers that we're working with know how to use the data way better than us and that it's more our place to be building tools for them to be doing explorations. So, I guess my answer is that we tried to build very kind of open and malleable tools, you know, like all of those sort of chart functions on there, like we just tried to build it a lot of features but not force people to have to use them and build it in such a way that if people needed to, you know, that was available for them but it was really up to the engineers to be going through that flow. Excellent question. So, we both met with our customers about every two weeks to do demos, collect feedback, do feature prioritization. We also had like an email link on the site that just said, you know, feedback, hey, and it would email the entire team so that we could, you know, respond to it as, you know, who was the best person to respond. How to avoid people being backseat drivers. I think that really just comes down to kind of establishing your authority, you know, that like you were the designer, you were the person building it, yes, they are the client and yes, they're the person who understands the data better but like, you know, you also have a design background that they don't and that was a thing that came up, a lot of JPL because I think especially like flight software engineers are some of the smartest people and a lot of them are self-taught and they kind of, you know, like, they've done so much in their life so they sometimes think that they know everything and so yeah, this would come up a lot of people being like, well, actually, I just think you should do this and it's like, well, okay, but you're like a mechanical engineer and I'm a designer so I'm going to say that actually it's better to be doing this design work and the benefits of doing testing is that, you know, you can back that up with data and who can argue with data, you know? So if you do enough user testing and you're like, actually, you know, like the way we built it is way more understandable and then this other thing that you were suggesting and so that's why we're gonna move forward with our design. Yeah, so it sort of depended on mission because some of them have larger crews as they're more active, but I would say that there's probably a couple hundred people using the tool. Insofar as it becoming a primary tool, I think it would really have to be in at the ground floor when people were designing the spacecraft for launch. So something like M2020, I think there's a lot of time right now to be designing, you know, the processes for people looking at their data, but it would really have to be from the get-go because once there's a process in place that people are comfortable with, they're really not gonna wanna let that go. Thank you.