 Yeah. So I pitched a topic for this conference that was pretty big. Obviously, I learned everything about how humans perceive graphics. But I found it really hard to fit it all into 30 minutes. So obviously, I know everything. So why don't we just call it 37 studies on human perception in 30 minutes? So maybe just take some pressure off of you guys to hear all that information. As Irene said, I work at The Washington Post. I'm a graphics editor. And that means I'm part developer, part journalist. And usually, people like me rely on common wisdom and experiential knowledge to inform our decisions about making graphics. And over the last few years, I've really been wondering what kind of studies are out there. My world really doesn't get involved with the academic world very much, at least I don't. And I don't know. It just seems like that's something I should really know. And so I've been collecting little bits and pieces here and there. But when I really started researching this topic, I became fully aware of how huge research is on visualization. And it's very overwhelming. And I know I am not including everything here. And I'm giving you very synthesized versions of the things that I read. But I gave it my best shot. So OK. In Colin Weir's book, Information Visualization, he wonders, is visualization a science or a language? It's perhaps a science because it must represent data accurately and methodically and without flourish so that we can see trends and patterns. And because of this, selecting a data visualization could be prescriptive based on what we want to show. However, many argue that it's a language because it uses diagrams to convey meaning. And data is encoded into symbology and semiology. The syntax and conventions of these diagrams must be learned and are not inherent. So throughout this presentation, just think about what you think it is. Is it a science or a language? In 1984, William Cleveland and Robert McGill published a study that can only be described as the archetypical seminal study for information visualization. It's referenced by many of the studies I'm about to talk about. And in fact, when that happens, I put William Cleveland's little head next to the study. I couldn't find a picture of McGill. It gives us our first ranking of the so-called elementary perceptual tasks, which are the most basic visual tasks we perform in our perception of graphics. At the top of the ranking, it's easiest to perceive position along a common scale. So think about a scatter plot. Scatter plots are anchored on two common axes, which is x and y. And so when you're comparing circles, you're actually comparing them across two common scales. Bar charts are kind of the same. They're usually compared along a common x axis. But Cleveland and McGill say that length and area could also be a factor in perceiving those as well. But this study was done over three decades ago. And how relevant is it really today? Fortunately, we have some clue. Heron Boestock revisited some of Cleveland and McGill's old experiments in a study. They did as more of a proof of concept for using mechanical torque for this kind of experiment. The results were very similar to those of Cleveland and McGill's, at least for the perceptual tasks that they tested for. So that's kind of reassuring. Three studies show that we have inherent biases related to the types of graphs we see and the objects we see in them. Three biases made these biases may distort the information we retrieve from a graph. We know that from Stephen's law, when an object is seen in context of other larger objects, it appears larger itself. When it is seen in context of smaller objects, it appears smaller. Jordan and Skiano found that spatial separation of lines could produce either effect. If lines were close enough, a line's length was more similar to the length of the line around it. If the lines were further apart, long lines appear longer and short lines appear shorter. And two other studies, Skiano and Tversky, found that charts were remembered as being more symmetrical than they actually were. They gave participants charts that look like these and told them that they were charts or maps. They found that when participants encountered the charts, they remembered them closer to the imaginary 45 degree line. Further, when the same line was presented to them as a map, no distortion occurred. When text appeared next to the chart, calling attention to its symmetry, participants recalled it being more symmetrical, even if the chart was actually not. This leads me to believe that annotations may be pretty powerful. In a separate study, they confirmed a systematic bias toward an imaginary 45 degree line and line charts. When a diagonal line was presented to participants, they continually remembered this line as being closer to 45 degrees than it actually was, meaning they underestimated the larger angles and overestimated the smaller angles. Thus, the 45 degree line is an imaginary reference point for line charts, but not in other contexts. Their work suggests that many different visual systems promote different reference frames. Croxons found over eight decades ago that bars were more effective in communicating comparative values than either circles, squares, or cubes. Circles and squares were about as effective as the other, but cubes were undoubtedly the worst, but there's more on 3D later. Much is said about the relative merits of bars and circles for showing proportions. All five of these studies legitimize the use of pie charts when conveying proportions and even show their superiority over bar charts. I did not encounter any studies that said that we should not use pie charts for showing proportions. Yolstin was among the first to publish a paper on this topic in 1926, and it's time, much like they are today, pie charts were ridiculed for their assumed perceptual inadequacies. For example, we're told the human eye cannot judge arcs and goals or chords very efficiently. You've all probably heard that. He wanted to know more about how circles were processed as well. So we handed out worksheets to psychology class and asked them to estimate the proportions in these pie and bar charts. Not only did he find that pie charts can be read as easily, quickly, and accurately as bar charts, but that as the number of components in the chart increases, bar charts become less efficient at encoding the data. The opposite is true for pie charts. He found that 50% of the people use the outer arc to make proportion judgments, while 25% use the area, and the other 25% use the inner-arger angle. Furthermore, 71 people in the class preferred the pies, and only 25% preferred the bars. He concluded that we ought to use pie charts, not just for their appeal, but for their scientific accuracy. He also concluded that men were superior to women in estimating these proportions. So, hats off to the men in the room. We've done it again. A follow-up study in response to Ile's work a year later did not find that pie charts were so conclusively better than bar charts, but they did pull ahead for some of the cases. At the very least, they were as accurate as bars. Six decades later, and in three more experiments, pie charts were hailed for their strength in conveying proportional data. Simkin and Hastie had participants make proportional judgments, and segment-to-segment judgments, like these little dots in the bars there. And they found that for segment-to-segment judgments, simple bar charts worked best, followed by divided bar charts and then in the pie charts. For proportional judgments, pie charts and divided charts were tied with simple bar charts at the worst. Spence and Lewandowski have found that comparisons among multiple segments take longer and with lower accuracy, and when multiple segments must be compared, pie charts are best. Hollens and Spence found that as the number of components in bar charts increase, their effectiveness at communicating proportions decreases. In fact, for bar charts, with each new component, the reader needs 1.7 additional seconds to process. Tables are found to be inferior to everything, except for communicating absolute values, which is against what Tufti advises. In two studies, researchers found that when participants were shown bar graphs and asked to describe the data, they continually referenced contrast between the variables and bars. For example, A is greater in X quantity than B, whereas in line charts, they describe the trends. For example, as X increases, Y increases. Even when a graph showed a third variable of data, the line chart descriptions remain focused on X by relationships, whereas the bars branch out a little bit more to include this new variable. These studies show that people have a hard time seeing messages in line charts beyond trends. Hollens and Spence evaluated the performance of a graph, that evaluated if the performance of a graph depends on the type of judgment that needs to be done. They felt that lines were superior to other graphs because they have their integrated systems, meaning that you can just tell the change just by the slick with a line alone. They tested participants' perception of change in proportion among bar, pi, and line charts. Pi charts obviously failed at communicating change efficiently, but they found that bar charts had similar success to line charts, so they hypothesized that it was because that people would draw imaginary lines across the bars to create that slope. And so he created a new terrible graph called a tier bar chart that would break that up and tested them again, and they found that yes, any chart allowing the reader to see a real or imaginary trend line was the best at communicating change. For proportions, if charts had no scales, pi charts were best. Line shape can be loaded with context that fascinates us, but also distorts our perception of data. As we know, the independent variable, the cause, is usually plotted on the x-axis, and the dependent variable, the effect, is on the y-axis. But we also tend to perceive slope as being a metaphor for quickness, height, or amount, and these two conventions can be in conflict with each other. Goddess and Holyoke designed an experiment where the slope could indicate height or altitude, but that also meant that independent and dependent variables were on the wrong axes. They presented both the right and the wrong chart to participants and asked if the dotted line would indicate a quicker or slower rate. When the altitude was on the y-axis, corresponding with the visual metaphor, participants were more accurate. In other words, we tend to see slope as representative for quickness, height, amount, or rate above anything else. They concluded that there are certain pictorial properties of slopes that facilitate reasoning above all others. They also revealed a universal association of more or better with the upward direction. Carswell and co-authors found that when line graphs showed trend reversals, people studied them longer. They found that this was not the case when they varied the number of data points, symmetry, or linearity. So here are three studies that suggest that we evaluate 3D objects a little more accurately than we commonly think. Two of these studies individually reject Tufti's popular high data-intra ratio mantra. Sea Grace finds that among bar charts, 2D is not superior to 3D, but 3D charts take longer to process. With pie charts, 2D is better, and the perspective angle really makes a difference when you're perceiving the individual slices, probably just because it obscures the data. Levy and co-authors acknowledged that 3D graphics, while glitzy and sexy, do not convey any additional knowledge and force the reader to deal with redundant and extraneous cues. Participants were given the option to select among 2D and 3D charts. When they were told to select a chart to present to other people, they tended to choose 3D charts. They also selected 3D charts when they were told the data had to be remembered. They selected 2D bar graphs more when they were told that they needed to convey specific details, and selected line charts when the message had to be communicated quickly. The authors conclude that 3D charts might be useful in some cases. The final two experiments by Spence deal with Stephen's law, which again says that an object size appears larger when it's presented with larger objects and smaller when it's presented with smaller objects. Spence found that contrary to popular physics, this distortion does not happen when comparing two shapes of the same dimensionality. Only when you vary the dimensionality among the shapes does this distortion happen. Cleveland and co-authors found that people come to conclusions about the correlation in scatterplots partly based on the size of the point cloud. When the same correlation is represented in graphs, but in one graph the scale is blown out so the point cloud becomes very small, people perceive it as having a higher correlation. Experimenting with simple type in scatterplots, Lewandowski and Spence find that altering color is most discernible to that eye. When varying color is not an option, varying fill or shape or even non-confusable lettering has no great loss in accuracy. He suggests that using letters has one clear advantage, providing a semi-label for the data, for example, in for males, F for females, but I personally think this can be achieved with annotation without the risk. In a crowdsourced experiment, three researchers reordered the native Tableau color and symbol palette so that it'd be ordered to be most discernible to the eye. Ziemkiewicz and Kosara found that directing participants to navigate tree maps with metaphors to complete certain tasks made them more accurate. For example, directing participants to find a data point inside a container-like tree map and telling them to look below a cascading tree map work best. Kong here in Argoala found that people discern values in tree maps best when the components are rectangles with diverse aspect ratios. Some might counter-intuitively, squares are not easily compared to each other. Extreme ratios in rectangles are also ineffective. They additionally found that surprisingly to them, small multiples of VAR charts were better than tree maps at representing data sets with fewer than 1,000 data points. In 2001, Barlow and Neville asked participants to compare four different hierarchal plots, this is 2001. They found that participants did not like the tree map and preferred the icicle plot and the org chart. In 2007, Cawthon and Moore showed that aesthetics can be linked to an individual's engagement with a particular visualization. They found that the sunburst vis was the most popular and that participants found the 3D-looking beam tree to be the most hideous. Engagement was best with the sunburst icicle and star tree. Performance was worse with the beam tree and the tree map. They concluded that the sunburst exemplifies that beauty can be usable. Harrison and co-authors ranked the effectiveness of several visualization types for depicting correlation. They found that scatter plots and parallel coordinates were best at this. Among the stacked chart variants, the stacked bar significantly outperformed both the stacked area and stacked line. And here's the ranking. A year later, Haire re-evaluated the data and split the visualizations into four groups but the top ranking remained the same. However, Haire, Hong and Arguwala explored how dense graphics can be and still be sufficient for readers. They mirrored the negative values in a time series chart and overlaid the extreme values into two, three, and four color bands, effectively reducing the chart size by 75%. They also manipulated the height of the chart so that some appeared to be only six pixels high for participants. They found that the more color bands presented in the graphs, the more people made mistakes, suggesting that sometimes not all visual markers are helpful to viewers. Regular line charts performed worst at small sizes, showing that the mirroring effect doesn't provide a great loss in comprehension. Haraz, Kasara, and Frank Connery experimented with using pictographs instead of generic shapes to represent data in simple charts. They found that using discrete shapes, whether they were generic circles or pictographs, helped people remember the data better than a single bar. Using pictographs as a replacement for text on the axes led to more errors. They also found that replacing generic shapes for pictographs was not a detriment to perceiving or remembering the data presented in the stacked graphs. People were also more inclined to investigate visualizations that use pictographs rather than generic shapes. In one of my favorite studies, Colin co-authors investigated if participants could understand computer renderings of line drawings as well as a human artist. They found that many algorithms today are indeed comparable to human renderings, but some do not come close. Even when viewers interpreted the shapes inaccurately, however, their interpretations were similar to each other and were concentrated in hot spots on the shapes. Some of the shapes were kind of funky though. I mean, no one really knows what the thing on the right is. And they got that one really wrong. A team of researchers evaluated how personality traits affect performance with list-style visualizations and container-style visualizations. Those with high internal LOC, those who believe that they can control external events weren't as good with the container-like visualization. Those who had high external LOC, who believe that they are merely controlled by external events, perform faster and more accurately overall. Similarly, more neurotic participants were best with a more structured container-like visualization while the other participants display the opposite trend. Introverts were more accurate overall than extroverts. Another team of researchers observed novice users when they encountered unfamiliar visualizations and tried to make sense of them. Participants had difficulty in moving away from their initial frameworks, even if they were incorrect. Thus, first impressions are quite important. Harrison and co-authors provided participants with a New York Times article aimed at affecting their emotion, either positively or negatively, and tested how well they performed at simple perceptual tasks. They found that negatively primed participants made more errors, and positive priming tended to improve performance. But only one in five were successfully primed, however proving that that is quite hard to achieve. Holman, Adir and Shah replicated hair in both stacks earlier study with Mechanical Turk, but with one major change. They showed a social histogram of the last 50 responses as the participant answered questions. And sometimes the histogram was skewed upwards or downwards from the real values. And those who were shown the incorrect histogram obviously got more wrong. Liu and Hare found that latency of a half-second with interactive graphics had profound effects on the way a viewer engages with the graphic. They moved the mouse less, shifted the types of interactions they did, and lessened some of their interactions altogether. They were also more likely to comment verbally on the interface. This delay also affected subsequent sessions for that particular participant. Here she was less likely to engage in a graphic during a session that followed. Originally the researchers wanted to include a one-second delay option, but in pilot studies users found this unusable. So think about that as you vote your next graphic. In 2007, Wigdur and co-authors found that completing elementary perceptual tasks like detecting position or angle direction was a lot harder if the screen was flat on a table. Therefore the orientation of a screen can distort the perception of a graph. In 2008, researchers gave participants a large data set presented three different ways in an animated fashion, aesthetic but traced version, and a small multiples version, and asked them questions about the data. Even if the animation version, even though the animation version went time and time again in a preferences survey, for healthfulness ease of use, enjoyment and excitement. Researchers said that the small multiples version was more effective for larger data sets, was faster and led to fewer errors. I wasn't gonna do anything in color because Rob Simon a couple years ago did a great talk on color that I highly recommend, but I really like the study a lot. A group of researchers created an algorithm to identify symmetrically resonant colors, which means if I wanna talk about oceans, I use a color blue, if I talk about love, I want to use pink or red. The algorithm searched Google images and assigned a particular color to a keyword. And then they crowdsourced how well this algorithm did with mechanical Turk participants, and some of it kind of was humorous, like the right hand side here, the blue cheese dressing, the algorithm wanted to call that orange, but the participants assigned that to blue. In a later experiment, they had the Tableau symbol artist who created this one at the lower left here, design a human-specific palette for each symbol that they are gonna use, and tested their algorithm against that and a non-semantic color scheme. They hypothesized that the human-picked palette would be the most effective, and so they bet against their own algorithm, but it turned out that it did just as well as the human-generated palette with a subset of the data, at least. So what can we conclude from these studies? What is still left to be uncovered? And how do these findings change as visualization becomes more commonplace and people become more visually literate? Most importantly, what do you think about what Colin Weir said about whether data visualization is a science or a language? I don't think data visualization or data storytelling is prescriptive. I think that there are many options and an infinite number of possible visualizations, and many of those we haven't even discovered yet, but I think it's not purely a language either after seeing all these studies. Obviously, we have biases and distortions with graphing frameworks and how we perceive objects, and some of that's learned and some of that seems inherent as well, and that's all I have.