 All right, thanks. Thanks, Irene. So throughout history, people have been recording sequences of events so that they can reconstruct and tell stories about the past. Now today, I'm going to talk about visual representations of these sequences of events, about timelines, and the different types of stories that we can tell with them. So I'm going to show you a lot of design choices and examples today, but what I also promise to give is a structured way to think about this design space. And then finally, I'll introduce a new tool that my colleagues and I at Microsoft Research have to tell expressive stories with this type of data. Now in order to talk about timelines, we first need to start by thinking about time itself, and specifically about how people use spatial metaphors when thinking about time. Because I'm speaking to a room of visualization people, I think we're all pretty good when it comes to mapping the dimension of time to coordinates on a page or a screen. So let me put some questions to you. I'd like you to take a moment and ask yourself the questions that you see appearing here. The question is, where are these moments in space occurring relative to you at this moment right now? Not when. So if you were to draw this, where would you put the future? Where would you put the past? And as your visual metaphor for time, does it change depending on the distance to the event and the scale of that event, its duration? So I put these questions to you because the ability to visualize time and space, it's an ability that people have to learn and refine over time. It differs from person to person. It differs as we age, as across cultures and across application contexts. So how many of you thought of a straight line as the line of time? So probably most people, right? As an arrow from the past to the future. But which direction did your arrow point? So depending on your cultural background and your first language, how it's written, maybe your arrow pointed to the left. Maybe it points downwards. Maybe you're on Twitter right now and your arrow is pointing to the most recent tweet. So how many of you thought of a different visual representation for time when any of those events, those questions I had on screen? I think so a few people maybe. So even if you didn't, I'm gonna talk about some alternative representations for time today, hopefully give you a sense of when to use each of these different representations. And I'll begin with this most familiar, this time as an arrow metaphor. So this representation for time, it's dominated the design of timelines now for several centuries. It was first made popular by Joseph Priestley, is a chart of biography here, which she's mapped the dimension of time running from left to right. And he's drawn the life spans of several notable historical figures along this timeline. And he's offset them vertically just to avoid overdrawing. Otherwise, there's no real meaning here to the y-axis. There's just a faceting that's going on here between statesmen or politicians or leaders at the bottom. And men of learning are philosophers at the top. So this representation, this type of design is still very popular today. So here's another example, it's much more recent. This is a timeline of Guantanamo detainees. This was by Jonathan Corm for the New York Times a few years ago. And here, it's the same idea. Each horizontal line that you see is a time that an individual was detained between 2002 and 2011. It's a very large graphic. In these cardinal directions, they work great for print and static images, especially when giving a bird's eye or an omniscient view of time. But another common mental model that people have for time is one that's egocentric. It puts you along the line of time. And that arrow is like taking a trip down a museum hallway and examining its contents over time, walking into the past or walking into the future. And we can see this egocentric view of time in examples like this one. Here's just a depicting the history of the Tower of London. And it's a bit like standing on an airport sidewalk, moving sidewalk on the airport and the time actually flying past you. Another really common way that we represent time, it's the circle. And especially when we're representing natural cycles, events that repeat, biological life cycles, seasons of the year, some of our daily routines, like when we sleep, when we eat, when we relax, when Beyonce performs. What if your data doesn't contain cycles or periodic repetition? Is it still okay to use a circular or radial representation for this type of data? So this example, this is Georgia Lepus' circular depiction of her life history. This was done as part of the Dear Data Project a few years ago. And in this example, her birth is at 12 o'clock. Her current age is at around 11 o'clock. At that point, that timeline here, it veers upwards and it trails off into uncertainty with these dotted lines, not back towards her birth. So it indicates this discontinuity here and still yet uses a radial representation. But here's another example. So this radial timeline, it depicts the Arab Spring beginning in late 2010 and the different events that entailed different countries. So this chart was featured on the information is beautiful site a few years ago. A lot of people really liked it. There's some ways that we can think of improving it though. Like this discontinuity between the end and the beginning of the timeline, maybe a bit misleading. And also the timeline spans 13 months and not the annual convention of 12. So this prompted Alberto Cairo to design this as a linear timeline. Certainly, it's more efficient for comparing the beginning and ends of events across these different countries. But you might argue, and a lot of people did when he published this that it's not as aesthetically pleasing, appealing as the original. So I say wanna have it both ways. So here I've taken the same data and I've tried to, I'll draw the viewer in with an attractive radial representation. Albeit one that shows the discontinuity between the start and the end, that gap in the top left. And then I'll use an animated transition to transition to a linear representation that allows the viewer to make more accurate comparisons across these countries. Or I could fast at these data into a radial small multiples and that maintains the same amount of screen space devoted to each of these individual countries. Perhaps you're more inclined to see time as both linear and stick, as linear and radial, as a coil or a spring or something that repeats, but yet it spirals upwards or forwards like this spiral staircase. And this spiral metaphor, it's particularly effective when we're depicting processes like growth, like a vine wrapping around a column. So this example here, this is a bunch of spiraling timelines that Moebio Labs created for the Ross Institute and it emphasizes this interconnectedness, this repetition growth that occurs in K to 12 education. Now spirals, at least to me, they evoke the slinky toy. And the slinky toy, it adds a new dimension to how we think about visually manipulating time. That's the ability to bend it, to compress it, to expand it into new larger shapes. So if that's too complicated, we could think of the linear analog to that and that's the accordion of time. And this is already a metaphor that we already use when laying out these collapsible elements and user interfaces. So can we do the same thing for time? So you can similarly imagine collapsing representations of time and to do this I'll use this example from several centuries ago. This is Gerardus Mercator, so of map projection fame. He also doubled in the art of chronology. And this is a page from his chronology from the late 16th century. Now his contribution, it was to indicate a more precise passing of time in the margins next to these written descriptions of events. So he was using fewer tick marks for sparse periods of history where there was nothing recorded and more tick marks when a precise date was known for an event. So looking at this now you could imagine collapsing these sparse periods of history with some interaction or animation but still retain some indication that an event has occurred, that time has elapsed. So I'm gonna illustrate this idea just by drawing here a timeline of 50 years of category five hurricanes from 1960 to 2010, beginning with dawn on the 60s and Katrina's 2000s. So I can collapse this chronological timeline but still retain the sequential order between them. And then I can show the time elapsed between these hurricanes using these bars. This shows something different from that chronological timeline before. It indicates that there's been some fluctuation in the periods between these hurricanes and that they've been getting shorter in the past couple of decades in this dataset. And another approach to collapsing time is just to use a different representation for different granularities of time. So this is a timeline that Catholic missionaries were using in the mid 19th century and they were using this to teach Native Americans about the history of Christianity. And they were indicating individual years along the central axis as dots and centuries here as these horizontal lines just to show different granularities of time because understandably they had to show a lot of data here. This is a very large graphic, it was a big tapestry. And they're using this approach to abridge the time between biblical creation and Christ and the founding of Christianity and the 19th century. So calendars also encode granularities at different, using different representations. Using a nested structure of days as cells, roses as weeks and months as these polygon shapes. And like radial representations, calendars are good at showing repeating patterns and deviations from these patterns, particularly when these patterns correspond to culturally redic bond conventions of weekdays and weekends, months and quarters. So in a way, calendars like a bookshelf where each shelf you might have a week with seven books on it. But this shelf calendar metaphor, it works really well also for larger granularities of time, like decades or centuries and millennia. And this is the one instantiation of this shelf metaphor. This is Elizabeth Peabody's Polish American system of chronology, it was from the mid 19th century. And she was using this as a teaching aid to learn about history. And she was using this 10 by 10 grid depicting a whole century. Each decade is a row here and each cell is a year. And then the cell was colored in according to what type of event and what category of event occurred in that particular year. And if you're interested in this particular example, I do suggest checking out a revival, an interactive revival of this teaching method by Lauren Klein and her colleagues at the Digital Humanities Lab at Georgia Tech, shape of history.net. Now the last representation I wanna talk about is much less restrained by any particular shape. So what you're seeing here are these very playful curve timelines and these were drawn by Mark Twain over a century ago and he published an article in Harper's Memphis Magazine about using these shapes as a way to remember dates. So I was wondering maybe if I use this approach I can finally remember the succession of Canadian prime ministers, which as a Canadian is something I should know, if they resemble a shape that kinda looks like a maple leaf. Maybe. So I just took you in a brief but certainly incomplete tour of different spatial metaphors for time. And along the way I showed several different examples of timelines but I'm gonna move on now to talk about other aspects of this timeline design space. You also need to consider what a timeline is for and how different combinations of visual representation of visual metaphor and different time scales tell different stories. So what is a timeline for? So one way to go about answering this question is just to try to distinguish it from other types of charts that answer different questions involving time-oriented data. So I would say my simple answer to what's a timeline for is it communicates what happened when. But meanwhile if you have other chart types you have linear area charts or kinda scatter plots they tell you how much over time. Trajectory map or a flow map will tell you aspects of where, what, and when. You have dynamic graphs, family trees, the storyline charts that were made popular by XKCD a few years ago that tell you how entities are related over time. Finally you have this question of what if and that's flow charts and branching. So of course there's a lot of instances out there where you'll be combining a timeline with some other chart type or you might be developing a custom chart type that combines these questions in some novel way. I think of Menard's map of Napoleon's march as one example of that. But if we focus on this deceptively simple question of what happened when and try to take it apart we arrive at some more questions. Like in what sequence do these events occur? For how long did these events occur? And if you take any two events event A, event B, did they co-occur? How did they overlap? What was the nature of that overlap? And when did they occur according to some common baseline event? Like event C here. So as an example to take parts in these questions let's look at these timelines that were drawn by Accurat a few years ago and these depict the lives of notable 20th century painters. And here you have every lifespan of these painters, all their individual artistic periods where they traveled, their romantic conquests, all that. And that's shown along this common chronological timeline. So here I'm gonna redraw part of this timeline, remix it a bit. Here I have Salvador Dali's biography based on that original Accurat graphic and you can see his artistic periods which are shown in brown here like realism and data and some of his travel to places like Paris. And now here is Matisse shown alongside him and I'm using a common chronological scale so I'm gonna shift over Dali so I can accommodate his earlier birth and you can see their respective cubist and abstract periods and perhaps when they might have influenced one another. But now when I change the scale to one that's relative to their birth dates you can now more easily compare the age at which they started their art careers, how long they lived and point out some similarities like they both traveled to Paris when they were in their 30s. And finally I can filter this data and remove everything else and I can remove all the chronological duration of all these events and just focus on one type of event, their artistic periods and then retaining just that sequential order of all the different periods they went through. And this allows me to compare the variety of styles in their careers. So ultimately each of these transformations that this went through they resulted in a very different visual timeline and it tells a different story each time. So there's just a recap now there's several design choices that go into telling stories of timelines. There's a course representation, there's different scales like using an absolute chronological scale, you could use log time scales, sequential scales. Now a third consideration that I'll refer to as layout is just how you lay these things out on a page or on a printed screen. You can facet these timelines into multiple timelines like I did between Dalí and Matisse or you can wrap them into meaningful segments like by century or decade. Now in case you're worried about the huge combinatorial space of these design choices, don't worry. My colleagues and I have written a journal paper about this timeline design space. We based this on a survey of over 250 timelines and timeline tools and then we identified 20 good combinations of design choices that correspond to feasible, perceptual tasks, meaningful narrative goals or different intents. So what would you wanna communicate with the timeline? And we also comment on aesthetic trade-offs and comments about the aspect ratios of these timelines given most data sets. So forgive the very dense figure but you can browse both of this survey in all of these galleries of examples of these timelines at timelinesrevisited.github.io. So now that we're equipped with these design choices, I'm arriving now at the question of how do you tell stories with these different points in the timeline design space? How can you tell, combine different, different interactive elements, animation, annotation. Tell these stories incrementally to reveal parts of a narrative piece by piece. You make different types of comparisons. And you've already seen this a little bit in my little example of comparing Dalit to Matisse. At the one end of this storytelling spectrum, you can present an interactive, I would say a choose your own adventure timeline. There's quite a few tools out there that allow you to do just this. This is just one example, it's called Time Glider. So these are some open-ended, I would call them more exploratory tools and that I would say that they don't guarantee that if you have a story that they aren't necessarily communicated to the viewer. They're certainly fine for open-ended data exploration by clicking on these events and finding out what they're about. There's certainly a lot of data analysis tools out there that do just this, especially in the domain of electronic health records where you're doing an open-ended analysis task. But if you have a story, if you have a narrative, these choose your own adventure tools don't really guide the attention of the viewer along a narrative. At the other end of the storytelling spectrum, there's a tool and a style about revealing a timeline as a slideshow, one event at a time and I left to right chronological order in which the events occurred. This is the approach taken by this tool which is Timeline.js from the Northwestern University Night Lab. But there must be some middle ground between open-ended exploration and one event at a time in chronological order. There must be some more flexible options out there. And if we consider how stories unfold in written language, they don't always stick to a strict linear chronological ordering of events. I could have walked through Dali's life story one event at a time, but the way I did it was I focused on his artistic periods, then I compared him to Matisse and then I showed some commonalities in their past. So it wasn't a strictly chronological unfolding of those events. And this characteristic of written stories, it can be seen when I extract temporal references to events in a written text. So I'm gonna show this tool called Timeline Curator. This is an open-source tool developed by Johanna Fulda while we're both members of the Infobis Group at the University of British Columbia. So this tool, it creates a linear chronological timeline after you load in some text. It uses natural language to extract references to events in a document. In this case, it's a document with the history of pop music in Iceland. And in this interface, it becomes possible to simultaneously highlight within the text an event that occurs as well as on the chronological timeline that you see above. And here it becomes possible now to show that the event in which it appears in the written story can differ from when it appears chronologically on the timeline. And this characteristic is not unique to this story about pop music in Iceland. It's true of a lot of written stories. They don't always unfold with the earliest event at the start. So if we are to tell compelling stories with timelines, our tools need to be more flexible and expressive. For some stories, a chronological introduction of this happened, then that happened, certainly makes sense. But for other types of stories, like in my example of comparing the life of Deletion Matisse, it may not make as much sense. And similarly, if we bring in what I talked about at the beginning of this talk, all the possible ways we visually represent time and scale time, our storytelling tools shouldn't be constrained to linear representations and chronological time scales. And this goal of expressive storytelling, it's why we built a tool called Timeline Storyteller. And this is an open-source presentation tool that my colleagues and I released a few months ago. And I've already used this tool in this talk to show several of the examples. So this is a scene-based presentation tool, runs the web browser. Each scene is, in the story, the author can choose to hide or reveal or emphasize or annotate parts of their timeline and choose a new representation or timescale to better suit that narrative intent. So let's look at a little bit of how it works. When I open up Timeline Storyteller, I have a load dialog, I can load in tabular data, JSON, CSV data, Google Sheet data. In this example, I'll just recreate that Deli versus Matisse story I showed earlier. So in each row in this dataset describes an individual event, I'm faceting it by painter, so I have a painter column or the name of the painter. I have categorical data, I have a start date and end date, some of the end date is optional and some descriptions for the events that will surface as text annotations. Now, when I load this data, I first get a summary of what's in the data, it's chronological range, the categories of events, different facets, and then when it's initially drawn, I get a single linear chronological timeline. It's not faceted yet by painter, so I choose a faceted layout. And next, I want to filter, just to show Deli and Matisse, I don't want all the other painters, so I use this filtering control to hide the ones they don't want. And then I can click on these events to annotate them and I can position these annotations wherever I want and when I'm satisfied, I can record this scene that I'm building as a scene that appears down in my little scene tray down here and that's retained for the first scene in my story. Next, I can then change the scale as I did before to one that's relative to the beginning of the first event in their facets. This is according to their birth and I can repeat that process again. This allows me now to compare Deli and Matisse according to their birth. I can annotate again and I can record this as yet another scene in my story that gets added on. And then I can change the representation using this interface too. I can annotate, I can change it for a radio representation and I can also bring in other forms of annotation. I've used text annotation already but I can also bring in images. I can put in Deli into his, respect to his timeline here, same with Matisse. And then when I'm satisfied with my new scene, I can save this as a scene in my story but I can also export this at any time. I can export this as ping images, as SVG images. I can export the whole story as a sequence as an animated gift sequence or I can toggle a presentation mode just to present this in a live presentation like I would be now. So I just want to finish by telling a bit more of a complete story with timeline story to learn it's a remix of a few existing projects that appeared a few years ago. So this was motivated by two timeline graphics. They were in turn inspired by the book Mason Currie's Daily Rituals, How Artists Work. So on the left, what you see here is a set of minimal, near chronological timelines depicting 24 hours in the day of a famous creative person. And the mouse over here, it reveals what particular activities someone was performing at that time of day. And on the right, you have a much more heavily annotated group of static radial timelines by R.J. Andrews of InfoWeTrust. Now, Podia's design on the left is perhaps better suited for comparing co-occurring activities, synchronicities across the day, what someone was doing at the same time. And Andrew's design on the right is probably better suited for telling individual stories and for emphasizing this notion of a repeating routine. Something that happens over and over again. So what I want to do now is tell a story that combines aspects of both of these existing examples, but also highlights some additional interesting aspects of this data. So let's begin by considering the daily routine of French writer Victor Hugo. And his routine, well, he devoted a lot of time in the morning and the evening to periods of creative work in his day. So he would write several hours in the morning, resume again in the evening. But the rest of his day, filled with a number of other eccentric activities like bathing on the roof of his building, visiting his barber, visiting his mistress, strenuous exercise, et cetera. So how typical is his routine? One might wonder, are all creative people this eccentric and different? So let's bring in someone else to compare him to. So I bring in fellow writer Franz Kafka here, who very, very different from Hugo, seemed to struggle to sleep at night. He had a day job at an insurance company and he would catch up on sleep in the late afternoon. So is it the case that all creative people in this data set or in the book that it was based on have a very individual and different routine? To do that, let's bring in some more people in this data set. So I have 25 people in this example now. But looking at all at once and scrolling like this could be a bit overwhelming. Where are the patterns if there are any? So we need to direct our attention a little bit. So I'll direct the attention just to the periods of creative work during the day. We have some people that work in very short bursts like Immanuel Kant and Charles Darwin. And we have people that work in very long durations like Murakami or Voltaire. We can make another type of comparison. Direct your attention again to types of sleep patterns of these people. Is there a relationship between sleep and creative working? We have some people that are nappers, but most people tend to sleep at night. We have some unusual exceptions. We have already saw Kafka. Balzac goes to sleep at six o'clock apparently and Voltaire only sleeps for four hours. Now another question we might ask, it doesn't pertain to something that has a chronological duration but it's about variety in a routine. So to do this, to answer this question how varied are these routines? Let's remove the aspect of chronological duration altogether and just retain the sequence of the events in this data. And this allows us to see how many unique activities appeared in a day. And using a linear representation as opposed to a radial one, it allows us to compare these sequence lengths more accurately from a perceptual standpoint. And immediately we see that someone like Charles Darwin had the most varied day relative to someone like Murakami. And finally we can restore that chronological duration aspect and effectively reproduce that podiographic I showed earlier. And finally this allows us to spot these synchronicities between events and people just by scanning vertically and seeing when are people sleeping, when are people doing creative work or having meals. And at this point I'd close the story by asking the viewer, well how does your daily routine compare to these people here? What story does your timeline tell? So I just wanna finish by reiterating that despite the apparent simplicity of the question of what happened when, I've hopefully communicated this richness of a design space here. In terms of different visual representations for time, different time scales you can use. This design space, it grows even richer when we introduce dynamic storytelling elements like the incremental reveal, reflective annotation, highlighting and animated transitions between points in this design space. So the projects I spoke about today they wouldn't have been possible without the help of my colleagues at Microsoft Research and the University of British Columbia. Once again all of the research that went into informing these design choices that I made in Timeline Storyteller, it's available at timelinesrevisited.gethub.io. And if you're interested in the tool that I showed briefly about scraping text documents for events and creating timelines that way, that's at timelinecurator.org. And now you can go to timelinestoryteller.com right now and create your own timeline stories except JSON, CSV data. And as researchers at Microsoft, we're interested in studying what stories you build using this tool or using these techniques as well as collaborating with you to create some stories together. And finally I'll just close by saying if you're interested in joining a discussion about this type of data, about open challenges with timelines and standards for timelines data. If you're a journalist, if you're an educator, if you're working with an educational institution or a publisher, I'd like to invite you to join a timeline consortium. This is sponsored by the Sloan Foundation and it's together with colleagues at HarvardX. So with that, thank you very much for having me and I look forward to seeing what stories you create and chatting over lunch. Thanks.