 Hi, so we're kind of in the doldrums of the afternoon. We just had lunch two full talks ago. So if you'd like to stand up and stretch for a moment, please do. I'll just prattle for a little bit while you shake it all out. OK, sit down. So I've been wanting to come to OpenVisComp for quite a while. This is the first time that I've been here. And I finally figured out how to do it, which is to have Irene pay for it. So thank you for having me. And I was talking to Jim yesterday, and I understand that from talking to Jim that the art talk is kind of an uncommon thing here. So that either means the bar is very low, or it's very high. And I'm not exactly sure, but we'll find out. So I wanted to talk about this concept of using data as a creative constraint to an artistic or creative process. And I think that as creative people here in the room, we're all familiar with the concept of constraints in a creative process or an artistic process. Charles Eames famously said the design depends largely on constraints. Constraints give us a wall to lean against. They limit the number of variables that we have to balance and give us a sturdy framework upon which we can experiment. So to bring data as a constraint into your process does not mean that the data will dictate the form or the output, but it kind of offers us a raw material that we can use to chisel down into a new artistic expression. So that's this concept of bringing data into your art. And that was the sort of founding concept that I was going to base this talk on. But I realized that the way that I do work does some of that. And then it also actually goes the other direction, which is to bring the artistic perspective to your visualization. Cute kid, huh? So data visualization fundamentally is communication. Communication is a fluid medium, takes on the characteristics of the communicator, of the content that you're working with, and often of the audience itself. So McLuhan's concept of the medium being the message, I think, is very true for data visualization as well. The techniques that you use to visualize your data have an outsized impact on the perception of the data. So if you bring artistic thinking or presentation into your data visualization practice, then you're offering new avenues for communication about your data. So we talked about bringing data into our art, and I think that that generally results in what we refer to as data art, which is sort of first and foremost, it's art, right? The intention is to generate an artistic work, and then the data are used to augment the art in some way or to constrain the art. And we can go back the other direction, which is to bring an artistic process back into our data visualization work. And this, I think, results in what Marie Stefano recently referred to as artful data vis in applied context or bespoke visualization. And these concepts, this way of moving back and forth between data and art is a way that you can think about how you produce your work, but it's also a research technique. So if you expose yourself to both processes sort of in this cycle, I think it will add richness to the work that you present. Even if the work that you present is not necessarily visually complex, or if it's more sort of succinct and legible is your focus, then this can still, I think, apply in a way of thinking about your work. So in terms of art in general, I think the area of art that is the most applicable to data visualization, in my perspective, is generative art. Generative art means a lot of things to a lot of people. I'm not gonna try to define it exactly, but I'll try to sort of describe it. It also has a lot of aliases. So it's known as algorithmic art. Creative coding is a newer term. Procedural art and even just computer art, whatever that means, I guess it means generative art. And there are a number of characteristics of generative art that I think helps describe it. So the first is that, of course, it's generative. You generate something through a process, and that process usually employs a set of rules and a system within which the rules sort of bound the transformation of your input into some sort of output. And in generative art works generally, or not generally, actually, sometimes you have emergent behavior. So when you feed something into this set of rules, you don't necessarily know what's gonna come out the other end. It's not a requirement of generative art to have emergent behavior, but I found that some of the most compelling works of generative art do exhibit emergent behavior. I think in the context of data visualization, emergence is not necessarily something we want because if we don't know how our end result is going to look, then how do we know how we're gonna convey and communicate to people? So there's pros and cons depending on how you're looking at it. Sorry, I have another bullet point slide and then one more after this. We'll get through it. There's a number of techniques that I think are applicable from generative art to data visualization. You can think of algorithms, randomness and probability, particle systems, recursion and repetition, motion and color. So I'm gonna dive into each of these at varying levels of depth and look at how they're used in generative art and then also how they can be applied to visualization. We'll start with algorithms. So algorithms can be used to illustrate behaviors of complex systems. You can create an algorithm that sort of defines the way elements sort of appear in your work. You can use them to generate textures, to indicate some quality of the data or even to act as a backdrop to the data in which case the data become apparent as sort of negative space against this algorithmic texture. And you can use algorithms to generate empathy. If you use algorithms to add character or personality to the elements within your visualization or your artwork, then you have the opportunity to introduce an epithetic link between the data and your audience. So this piece in the background here and this one are both Saul Lewitt pieces. Saul Lewitt is a generative artist that I think a lot of us are familiar with, his work primarily in the 20th century was concerned with drawing things on walls and galleries. And the way that he would go about this is to write a set of rules and present this written set of rules to an artist or a group of artists who would then execute these rules on the wall often with pencil or some other form of marking. So the rule set for this piece is lines while drawing 69. Lines not long, not straight, not touching, drawn at random using four colors, uniformly dispersed with maximum density, covering the entire surface of the wall. So this is one of many examples of how this set of rules could have played out in the gallery context. The point is that it's all about this sort of process that emerges, sort of this piece of art that emerges from the rule set that he's created. Cacerius is a, oh no, internet. Sorry, oh internet, no internet. Do we have a wired connection? I'm sorry, I totally forgot about that detail. Bear with me. Oh, have you guys seen this? Can I do it? It doesn't work in embed. When you don't have internet and you load up a page that shows little dinosaur because in Chrome, you can, if you start messing with your arrow keys and the dinosaur comes to life and you can start playing this little game where the dinosaur runs around and hops around. You can't do it in the embed player so I can't entertain you while we get the internet working. What's that? I don't want to do that. Okay, let's try refreshing. Oh, I'm sorry. Totally forgot about this detail and I can't see where system preferences is. Is this it? Yes. Network. Where's network? Connected. Sorry. Oh, I shouldn't have to use wifi, right? I should be able to, it looks like I'm there. Let's try one more time. Oh, go back. All right, we got it. Thank you. Okay, network. Thank you, sorry. Where's my mouse? I need it. Okay, so this is a piece by Casey Rias. He also follows a very similar pattern to Sol LeWitt. Before I get into that, I'll say that he might be familiar with some of you as the co-creator of processing with Ben Fry at Fathom here in Boston. And Casey's practice is very similar to Sol LeWitt's in that he will also set up a set of rules and then instead of having artists execute his rules, he has software execute his rules. And he'll often even display his rule set alongside his works on the wall in the gallery. So the rule set here, sorry, is a rectangular surface filled with instances of element three, which is another rule set. Each with a different size and color. Draw a tiny transparent circle at the midpoint of each element. Increase the circle's size and opacity while its element is touching another element and decrease it while it's not. So we can see similar to Sol LeWitt that this pattern emerges, this artwork emerges based on this set of rules. Here's an example that is more directly tied to data visualization. This is a piece by Nervist System, also in Boston, in which they have an algorithm that generates what they call a highly controllable enisotropic macroscopic foam structure. Sorry, Amanda. It's both lightweight and strong. It's a cellular structure. And what they do is they adapt the density of that foam. They adapt the density of the foam per the foot strike of the runners. They have the sample individual runner's foot strike. So we'll see here in a moment the areas of highest force from the foot strike are the areas of the highest density in that foam that it's generated with an algorithm. So the presence of the data actually manipulates the algorithmic texture here. This is a piece that I did for Stamen for a news organization that shows the stories that are getting the most views at the center of a cluster that's determined by all of the other stories within that section. So this algorithm is a simple clustering algorithm that lumps together everything that's in the same section of the paper. But by using an algorithm applied to a particle system, I can then sort of remove that algorithm and allow the particle system to fall back into a more standard layout. Randomness and probability is another technique that is used to great effect in generative art, I think less frequently in data visualization, but we can use it to indicate natural variation that's inherent in the data for which the quantity is undefined or perhaps not as useful to understanding the patterns within the data. We can use it to represent fuzziness in data. So data are observations, fundamentally, and they're not always precise. Like analog instruments, for example, have a lot of inherent fuzziness. And then, or noise. Using probability to represent probability itself is something that we often have to deal with as data visualizers. This is a very early example of a generative artwork in which randomness is very critical. It's called Musikkallisches Wurfelspiel, apologies to any native German speakers now. And basically what you do is you roll the dice and you hear things, if the video plays. Whoa! All right. No! Play. Doesn't sound particularly random because it's Mozart. And it's a bunch of individual pieces that are stitched together in the order determined by your die rolls. Jumping ahead a bit in time, this is, and moving from sound to visuals, this is the simple random walk algorithm which is used quite often in generative art. It's a way of using randomness to move through space. And essentially the algorithm says move forward or backward or left or right a random amount every step or every cycle. Looking at how that can be applied in an artistic context, we have this piece by Guido Correo called Random Walk Triangles in which there's a number of individual colored triangles that just randomly move around the screen and leave a little trail behind them. So this one is maybe not, yeah, anyway. And then Plainton Populations is a piece that I worked on at the Exploratorium in which we show the populations of different types of plankton in the world's oceans as colors on the map like a biome map and you move this lens around on top of the table and when you look into the lens you see individual plankton swimming around and the likelihood of individual plankton appearing in that lens is determined by the density of the population of that type of plankton at that point in the ocean. So essentially what I'm doing is probabilistically generating the little critters under the lens depending on the overall population at that point in the ocean based on a simulation that models plankton. And then, sorry, come on, next. Hi, Jen. This one, Amanda already showed. So I don't really need to say too much about it but I think it's a great example. I personally do, I know not everybody does, portraying the uncertainty inherent in this case in polling data. One of the nice points that I read that Gregor Ayshe mentioned is that it's not completely random. It's bounded by the 25th and 75th percentile of the simulated outcomes so it shows us uncertainty without being totally crazy and wild about it. And if you wanna know more about how you might use randomness in your own work especially if you're interested in generative art or data art, Anders Hof has a series of write-ups on inconvergent.net. Sorry, I should put that on here. That show you how you can sort of shepherd and bound randomness and sort of shape it into different forms. So particle systems, I think are something that we're all very familiar with here. So I'll run through it fairly quickly. We can use particle systems to represent components of a system, representing the behavior of a system via the interaction between particles. We can also show relationships between elements. So for example, a force-directed network graph in which the distance is inversely proportional to the strength of the relationship between the nodes. And to show data elements. So data elements don't have to be dots on a screen. They can show complexity on their own. They could be glyphs, they could be roast charts, curves, whatever. Something that has sort of inherent representation within it and then has sort of more of a bit of meta information about the entire system. This is an early example of a particle system from 1986 by Craig Reynolds called Boyds. I think a lot of you are probably familiar with this one. The simple rule set here is that each particle attempts to stay as close as, well, approximately the same distance from all of its adjacent particles. So what ends up happening then is that this particle system naturally exhibits flocking behavior just from that simple rule set. And this is a more modern version of the same algorithm by Robert Hodgin, Flight 404, in which he has tens of thousands of particles doing the same thing. It looks very much like a Starling Memorization. This piece by Memo Acten in Quaiola is a data art piece that visualizes the movement of the Olympic athletes from 2012 Olympics. And what they've done is run video analysis on footage of the Olympic athletes and then use that to determine sort of salient points of their bodies as they move through space and then apply that to particle systems. Particle systems are triangles, lines, bars. They use different forms as the piece goes on. I would love to sit here and watch the whole thing, but I can't. This piece by Zach Watson at Stamen, Facebook Flowers, shows the spread of a George Cicke post on Facebook as it moves through the entire social network of Facebook. Well, not the entire number. And each one of these pedals is an individual share. So when we see these bursts, these tendrils, that's sort of a viral moment where it gets shared sequentially through a lot of people. I know that we all know this piece by Fernando Villegas and Barton Wattenberg in which they set up a flow field. Well, I don't even need to show it because you all know it, right? They set up a flow field from NOAA wind data and then introduce particles into it. When I put together a talk full of streaming videos, I thought there's no way that this is gonna happen, right? Sorry? Yeah, hint.fm slash wind. It would be great to not sell them short here, though. I'm gonna move on to the next one of my apologies to Fernando and Barton. Great, so my workloads, and theirs doesn't. So this is a piece that I did at the Exploratorium that was sort of inspired by their work, hint.fm slash wind. You should go check it out if you haven't seen it. Instead of using individual dots for particles, I'm using low alpha white squares. And instead of using only wind data, it's also using a model from NOAA that forecasts liquid water content in the air. So what we call in the Bay Area fog. This is essentially a fog forecast that is then projected onto a topographical model at the Exploratorium. And you can see the fog actually sort of follow the patterns that we're familiar with, where it rolls over the mountains. This is 280 running right here for anyone in the reservoir, familiar with the Bay Area. And it kind of goes between the peaks of the mountains. So this is where things get a little strange. Recursion and repetition I think are tough to find a use case for in data visualization, but I think that exists. Just trying to rack my brain to figure out any context in which it would happen. And the one that I found is context or domains in which recursion and repetition are really critical. So machine learning, genetics, and evolution are kind of the only places that I've found recursion and repetition to be useful for data visualization. But if you have other ideas, I'd love to see that. This is one of my favorites from back in the heady days of Flash from Yugo Nakamura's UgoP.com, which is no longer online, unfortunately. But this is a simple algorithm in which a clay ball splits apart into two clay balls, and then that splits apart into two clay balls. And whenever there are four clay balls adjacent to one another, they crumple back together into another clay ball. And the space is bounded at a grid of 16 by 16, which means that when we fill up the screen, we end up with this punch line. We end up with this singularity, this moment of recursion in which we start over completely. This just goes on ad infinitum. This is a piece by Alvin Lucier from, I can't remember the date, 1970 maybe, which is an audio piece that might play. There we go. In which the... Any semblance of my speech with perhaps the exception of the rhythm is destroyed. What you will hear then are the natural resonant frequencies of the room. So he actually speaks into this microphone and then records that recording back in the same, so he plays the recording back in the same room, which is then recorded by the microphone and then plays that back into the microphone in the same room. Does that over and over again until you end up with the resonant frequencies of the room itself. Fascinating piece conceptually and really difficult to sit and watch a performance of. So a more modern example of this is Patrick Lydell's version by doing the same thing with YouTube. He records himself talking into his camera, uploads it to YouTube, it gets compressed, he pulls it back down and then repeats the process over and over again and you end up with... Scary, right? That was about a thousand iterations right there. So this is an example from the illustration world. John Fransen called each line one breath in which he takes a single breath and as he lets it out, he draws a single line on the left edge of the paper. He draws another line with the next breath as close as possible to the first line and he repeats that over and over and over again. And what happens is that through this process of repetition recursions, kind of muddy, errors propagate through the entire work from left to right and you end up with this texture that feels very sort of like fabric or cloth. I was in a really boring meeting recently so I took the liberty of doing one myself. It's pretty fun but I hope none of you are doing that right now. And then here's an example and data viz. Try to bring it back. This is a great piece by Tony Chu and Stephanie Yee, also known as R2-D3 called Visual Introduction to Machine Learning in which they demonstrate how a decision tree is calculated by finding all the split points through recursion and then they feed the data down through the tree. So another technique is motion. Motion is great for bestowing personality. This is what animators do for a living, right? They take static models and they bring them to life by adding motion. They show, motion can be used to represent relationships between things in your system. They can be used to illustrate change. It can be used to draw attention both to individual elements and depending on your context, the piece itself. That was important for me when I was working in the Exploratorium and there's just a lot of distractions. Just to create another distraction. This piece by Design.io, it's Theo Watson. Emily Govel, there over in Cambridge is an early augmented reality experiment, early 2009, in which you hold up something in front of the camera, maybe 2011. And the camera sees this marker and sort of gives you this experience of moving up and down through space and elements in the screen move around and they sort of come to life through this motion that you can't see right now. And this experience of being immersed in this world has all of this character and this really sort of, I don't know, quality of wonder that comes from the illustrations but especially from the motion of the illustrations. And another work by, with Balloons in it, with John Harris and Cep Camvar, as I want you to want me. This is a piece that they created by mining profiles from online dating sites in which they represent each person on the online dating site as an individual balloon and each balloon sort of has its own characteristics that they determine through natural language processing. And especially this movement called Matchmaker in which they algorithmically pair up two profiles, I think uses motion to great effect, but these balloons are just sort of intertwined and they're sort of expressing their love for one another. This is a piece that I, oh, next. This is a piece that I did with Zana Armstrong at Stamen called The Atlas of Emotions in which we took Dr. Paul Ekman's data about the five universal human motions which you might be familiar with from the movie Inside Out by Pixar. And we represent them the intensity of these emotions as aerographs. So the way that aerograph kind of came into life is meant to indicate the emotion of anger. So it kind of flares up and this one is showing fear which kind of flicks out its talons. The next one is disgust which has this kind of heaving quality to it and sadness takes a deep breath and a sigh. And then we have enjoyment which is a bullion and bouncy and happy. So we tried to actually represent the emotion themselves using motion. And of course, the canonical example of how we can use motion to indicate a change in state is Hans Rosling's 200 Countries, 200 Years, Four Minutes. By enormous disparities today we have seen 200 years of remarkable progress. That huge historical gap between the West and the West is now closed. In which we see the world's countries lifting themselves out of poverty, increasing their lifespans and their per capita net income. So color. You obviously all know why color is important to data visualization so I'm not going to attempt to drill into that too deeply but it's worth mentioning it's good for encoding, for highlighting, for legibility and then also of course for visual appeal. So rather than talking about how we can use color I wanna talk a little bit about how generative artists source color. Of course we can use color ruler but there are other techniques as well. This is a piece by, that was not meant as a stab. Color Brewer is awesome, you should all use it. Eric Natsky was a flash artist back in the 2000s primarily and what he would do is he would take photos and then as he would use it sort of as a backdrop for his digital brush strokes. So as the brush hits the screen it samples from an area around the brush from that source photo and then his brush strokes represent that they sort of repaint the photo. Jared Tarbell did something similar with this box fitting image in which he used a box fitting algorithm to fill out space and then color with the dominant color underneath each box. And then Mario Klingman also known as Quasimodo took that one step further into the realm of data visualization which he did a pie packing visualization where he shows the distribution of the dominant colors underneath each circle. So those are about sort of sampling color from a point. This is an example from Vegas and Wattenberg. I get to show one of their pieces, that's good. Called Flicker Flow in which they take a bunch of Flicker photos from Boston Commons and look and sample them over the course of a year. So the year moves around in this circle and the thickness of the line in the stream graph represents the frequency of that color in the photos that they found from Flicker. So in the upper left-hand corner we see summertime. Down towards the bottom we see wintertime. I hear it snows a lot here. So this is a way to look at colors sampled sort of across time instead of from a point in space. And Brendan Dawes, that's something similar with Cinema Redux. This is Alfred Hitchcock's Vertigo broken down into individual frames. So every, I believe it's one frame per second reduced to an eight by six pixel image and then a matrix made of that. So what you end up with is a reduction down to sort of the primary colors of each frame. You can see the stripe about two thirds of the way down where the protagonist goes through this total hallucinogenic trip and goes crazy and if you've seen the movie then you know what I'm talking about and it's as colorful as it looks here. And then Kevin Ferguson has a piece called Western Roundup in which he watched a bunch of Westerns and averaged all of the frames together. So this is sort of both across time and then across space where he's sourcing his colors. Great, so I showed you a bunch of pretty pictures and I know this is OpenVizConf not IO so there's probably more skeptics in the audience than I might find at IO. So I just wanna say that while this is a celebration of complexity and artful techniques I'm not advocating for this in all cases. You need to use these techniques responsibly if you're aiming for clarity. If you're aiming for the ability of people to sort of understand the point immediately when they look at your visualizations then a lot of these techniques are probably not going to work for you. So just be conscientious about how you apply your techniques depending on the context, the audience and the dataset. And this is a conversation that's been bouncing around the visualization community lately is how we make these decisions depending on what role we have, depending on what kind of products we're making. This is a diagram that Elijah Meeks recently put out which shows sort of on the product side the analyst data scientist, engineers over on the left side doing more precise numeric clean work and then folks over on the right hand side, data journalists and especially consultants and artists doing more design and engaging and complex work and what he's arguing for actually is not that this is the way it is and should be but that we can both sort of move towards the center and learn from each other. So and this is especially relevant to me personally because I just transitioned from Stamen Design to Uber to Nicos team. So I moved from the right side over to the left side but I want to land somewhere in the middle. And this conversation is something that we've tried to capture on a medium publication visualizing the field and we would love to hear from all of you. If you have thoughts about it then please tweet me and I'd love to tell you more. The next few sides I have are another side project and I was happy to hear that there are other people here in this room that are considering this as well. Lately I've been feeling like I'm not very well represented by my government and I've been trying to think about what I can do about that and why that is and I think that this is a large part of it. This is about the third time, maybe the fourth, that Jerry Mandring's come up in this conference which is awesome and this I think is a fundamental reason why I don't feel well represented. Our system of representation is broken. So some folks have come together to address the problem. We have a D3 team, some people from Stamen, current and former, some people from Cardo from MAPSEN and some other folks. And we've been starting to do some experiments. This is one by Mike McGursky where he used the efficiency gap metric to generate a bunch of different what ifs for Wisconsin. So this is six different ways in which the district map could be drawn fairly according to the efficiency gap metric for Wisconsin. And it all started when Waldo Jakewith from US Open Data contacted Stamen to ask us to do a piece in which we bring the redistricting process out of the smoky back rooms and out into the public. It essentially uses GitHub as a back end for displaying, redistricting proposals, allowing the display of revisions to that and then for people to comment and potentially even to suggest their own alternative as if they're so inclined. So another call to action, if you're interested in trying to fix Jerry Mandring then tweet me and I'll get you on the slack and we can go from there. Thank you.