 Thanks Irene. Let's see. My talk is on the blocks page. It's D3 and Canvas here. I'm going to go fast. So I don't know if you'll be able to read everything on this slide. My talk is a little bit of a split personality. It's a little bit of tips and tricks and something about learning D3 and programming in general. We're talking here about pixels and subpixels which are, I don't know if you've ever seen one of these since the iPhone 4 came out because they are so small. I think a child nowadays may not even be aware that displays are made of pixels. Nowadays there are probably billions of pixels in this room. Think about all the computational power that goes into driving those pixels. This is a picture of this display here. It's a macro picture. Let me zoom in just a little more. This is a real photo of this computer this morning. This is what pixels look like. They are red, green, and blue usually. They can come in a couple different layouts. This is IPS. This is a kind of standard layout. You can control each of these red, green, and blue values individually. Once you have enough of them and they're small enough, you don't see red, green, and blue anymore. You get color mixing effects. It happens on the surface of your eye on your retina. This is an image if you just pull it out my eye and slice it in half. Sorry, this way. Slice it in half. The retina are these cones and rods. Cones are the ones. They're really a sort of peak in the center of the eye. They're sensitive to color and shapes. Then we have this layer of processing bipolar cells, ganglion cells that actually do a lot of visual processing right there on the eye before the information gets passed through the optic nerve, through the optic chiasm, and the visual area of the thalamus, visual cortex. Our eyes are directly connected to our brains. The images that go into our eyes, they just reverberate through your whole brain. We want to control these pixels and draw things on the screen that give us ideas. In order to do that, we're going to have to deal with computers and programming languages and all this stuff that we've created. It's really the photons and this reaction that's happening in the eye and the ideas in our brains. That's what we care about. We have to pick something. This saga is about Canvas. It's a JavaScript API developed by Apple in 2004. There's a specification created by the WhatWG a couple years later. It's basically in all browsers nowadays. It's been that way for about a decade at least. It works on mobile phones, pretty much every device. I think it could drive most of the billions of the pixels in this room, except maybe like your camera viewfinder or something. In order to program on Canvas, we're going to need to use JavaScript, which means we're going to use number strings, Booleans, arrays, which are a great data structure for if you're interested in visualization. They have a scary name, array. Functions also have a scary name. Arrays, functions. Don't worry about it. They're cool. Then methods on arrays, like map filtering for each. There's a talk given last year at OpenVis.com by Marco Casaca. It's a wonderful talk on Canvas, especially the image data object, which I will not really discuss. Basically, the image data object is the bitmap that is actually the Canvas. It's the RGB red, green, blue, and actually there's an alpha value at every pixel on the Canvas. There's just a flat array, and that's what the Canvas data is. Marco says she really, really likes arrays. She likes mapping them. She reduces them, but she does it in the physical world. These are the metaphors that are useful when you're learning a programming language, especially for the first time, because something real is happening in the computer. The words don't necessarily correspond to exactly what is happening in there. Sometimes it's so complex what's happening here. It's overwhelming to think about. Having ideas about mapping, which is basically transforming a set into something else or reducing, which is taking a set and turning it into something smaller, generally, it's useful. We need some boilerplate. Boilerplate sucks, especially for beginners, because you can't do anything until you have it. The nice thing about Canvas is it's not too bad. You need one Canvas element and then one script element to put your JavaScript in, and then you need to do this scary stuff, some DOM selection, and then you need to get the Canvas, and we'll render on the 2D context. Almost everything I talk about for the rest of the talk in terms of rendering is going to happen on this 2D context. Here's some basic drawing with Canvas, just filling recs. The Canvas context has a bunch of properties, like fill style, stroke style, line width. They're all camel cased. You can put, in all of the styles, they basically take colors. You can put any HTML color in there, RGB, HSL, dark orchid. Fill rect, just immediately, you pass it in x, y with an height, pass some arguments, and it immediately renders on to the Canvas. It's an immediate mode rendering. It doesn't keep any memory of this rect. This is how you do circles. Circles are a little different. There is no Canvas method for a circle. You have to begin a path, and then you have to draw an arc, which takes an x, a y. This is actually radius, and then a start angle and end angle. You don't have to draw a circle. You can use this for pie charts, generate the arcs of a pie chart, but this is a circle. Code like this, this is imperative code, and I think imperative code is actually great for beginners because it's line by line, something is happening, and that's kind of the way, it's good to learn to read code that way and see things happening. Eventually, we want to sort of create words which are more expressive ways of talking about things. Here is a word for sort of a metaphor for a circle. What we'll do is it will pass an x, a y, a radius, and a color, and it will immediately fill the circle on the screen. We can create the exact same image I showed a moment ago with our new circle function, and it looks a lot smaller. So don't be afraid to define functions and play with functions. Also, once you are making a visualization, you may find that this particular circle, as it's defined here, doesn't really fit what you need to do. You may be annoyed that it sets the fill style because after I've drawn this red steel blue circle, anything else you fill in the canvas unless you update this fill style again will render a steel blue, because canvas, once you set something, it kind of stays that way unless you go back and change it. I have a sort of sequence here exploring the lab color space. It's a perceptual color space. It's available with D3. This is the first D3 method we're going to look at. It takes three arguments, L, A, and B, so I've defined them here as variables, and we're just filling a rect that fills the screen. This is just picking a single color in lab space. So you can just take the return value of D3 lab and set it to a fill style. We can make a 1D gradient out of this by using D3 range to create an array. It creates an array the same width as the canvas, and then we iterate through this array using four each. So we're doing something on every pixel of this canvas, and this here, this little kind of scary mathy bit, what it ends up doing is it ends up going from negative to 100, which are the valid values for this A parameter in lab space. So for each one of these, it draws a slice full height all the way across the canvas. We can take a 2D section of the lab space by nesting yet another loop and doing the same thing on the B parameter, and then we can trip out with D3 timer here. Oh, no. I changed the order of my slides. I made this code not run on this slide. All right. Let's get that slide if I have time at the end. It was cool. It was cool. Here's another example of D3 timer. This is a GIF. It's not a live example on the page. Turned out to be a mistake. So D3 timer, what D3 timer does is it takes a function, and it just will keep running that function. Internally, it uses something called request animation frame. Request animation frame sounds scary. What it is is basically request animation frame will go to the browser and say, hey, are you doing anything right now? And the browser will say, why no, I'm not. Request animation frame will say, run this function. And then the browser is like, okay. And then request animation frame, like 16 milliseconds later, will be like, hey, are you doing anything now? And the browser is like, why do you just mean that? And he's like, no, I'm not. And then animation frame is like, we'll run that same function again. At least that's how request animation frame is used in D3 timer. It just keeps running this function over and over again. And the function that it runs, it passes in the elapsed milliseconds since you started. So I'm using it here to, with satellite JS, to just plot satellite. So the way this code works is it creates a date, gets the current time, adds on 150 times the, it adopts times. It's basically running time at 151 x speed. And then this data object, I'm not going to talk about loading data. This is more a talk about rendering. So I'm loading data, they're actually TLEs from Celestrack, which is the orbital parameters of a satellite. And I'm running it through my plot satellite function, passing in the data for that satellite in time. Inside the plot satellite function, I use satellite JS. And this is kind of pseudo code here. The structure is right, but some of the methods and arguments are a little different. It propagates out that satellite to a time in the future. So it's running it through its orbit, takes the position of the satellite at that time. And now it's a position in space. Because, remember, the earth is also rotating underneath the satellite. So we also need to pass time and the position in, and you can pass it straight into satellite JS to get the geodetic coordinates, which is sort of the, where the satellite is directly above, and then draws it on the ground. Obviously, there's also a map underneath. I'll get to maps a little bit later. This is a talk about Canvas. This is another astronomy data set. It's the exoplanets. This is parallel coordinates. What I just wanted to show you is the only demo I'll load externally. My first open-viz talk is about this. This uses Canvas move to and line to to render these lines across the chart. There is a double loop. So there's a loop iterating through every data, every exoplanet. And then there's a second loop, which loops over all of these dimensions. The discovery method, the planet letter, number of planets in the system. Each of these dimensions has their own scale. And so line to, as it's rendering across the chart, just goes and gets that scale and it knows where on this axis it needs to hit. Also, when any interaction happens on this Canvas, I do kind of a trick. I have an utility called renderQ. It uses D3 timer as well to basically render the data slowly. Why would you want to render the data slowly? Well, the reason is because there's 3,500 exoplanets almost. If you try to render every single exoplanet across all these dimensions every frame, it takes a couple seconds, maybe a second or two, which means any interaction that you attempt to do in that time, scrolling, brushing, almost anything else on your computer, your CPU will just be totally maxed out for that moment while it's just trying to get everything on the display. So I use renderQ to render things slowly so that the interactions are quick, even if it's rendering. You can interrupt these interactions. The code for that is a little tricky, so I'm not going to go through it. The thing to know is that once you draw something on the Canvas, there's no cost to keeping it around. You can just keep rendering under the Canvas. So if you have a way to stream in data and put it on the Canvas, I mean you can render gigabytes of data onto a Canvas. Another trick is particle trails. You can create this sort of whimsical effect when Canvas very easily. Every frame in the Canvas, usually you clear the whole thing every frame and render it anew, because it's not that expensive to render things in Canvas. And if you want a particle trail, instead of clearing everything, you just kind of put a transparent rect over it, so fade it out a little bit. And every frame, you just do that and you get these trails almost for free. So I also want to talk about learning in Canvas and SVG. I found this developer roadmap thing, which kind of bugs me because this is like the path to become a friend and developer you learn HTML, CSS, CSS6, JavaScript. You play with TypeScript, Webpack, Angular, React, CSS, you learn Flexbox gradients. At some point down at the bottom here you may play with SVG and then, oh, finally you get to learn D3. You know who will never do this is like 99.8% of the population. I mean this is a, it's an incredibly complex like the ecosystem right now. SVG itself, I think, has some challenges to learning. This looks like a boring slide. It's very, trust me, it's very inflammatory in the D3 community. This slide. There's a lot of complexity involved in managing a DOM hierarchy. And in the relationship between SVG and HTML and CSS and adders and styles, path strings have their own language so you have to do string concatenation. If you've ever tried to learn D3, you've probably struggled with exactly what is happening between these methods, select all data, enter append and merge and transition. And then it gets slow with a large number of elements, about 1,000 elements. But there's tons of tutorials out there. We've invested, I mean as a community, maybe tens of thousands of hours of developing beginners materials using SVG. I think a lot of people who learn D3, they struggle and we get frustrated. And I've even, I mean I've known a lot of people who've learned D3. I think even some people struggle with like a minor, mild depression at some point trying to learn it. And these people laugh because they know it's true. I'm not, I'm not being hyperbolic here. I think it's because it's different than other technical tools you may learn because they think you learn D3 because you want to express something. And you have a vision of what you want to express. And you see the amazing work by the New York Times. And you want to do this too. You want to learn this technology because you have a story you want to tell. And you have a unique perspective. And all this technical stuff gets in the way and then all of a sudden, you know, you're stuck somewhere on some bug, you have no idea why this thing is not happening or this line is being drawn this way or, you know, something. It happens to a lot of us when we're learning. I think almost everyone has this experience. Even people who program and come into it. Pay no attention to this slide. Canvas is perfect. One thing I do want to mention is that the really, one of the big things you lose, I'll show you how to get some of these things back. But I don't have a solution here for an accessibility fallback. With SVG, you get for free or with a little bit of effort, support for screen readers and other accessibility tools. You do not get that from Canvas. People, if they can't see, they can't experience your work. This is a New York Times graphic. How epidemic of drug orders ripples across America. It's about sort of the rise of opioids. And it's kind of related to these other sort of diseases of despair. These are choropleth maps. Data comes from, I don't know if that particular data comes from CDC wonder, but if you're interested in doing work like this with mortality, CDC wonder is a great place to start. It's got data by county. You can break out. You can get breakdowns by year of death, by age group, race, gender, cause of death. Where they died, if they died in hospice facility. Choropleths in D3 with Canvas are actually pretty easy because GeoPath actually supports just taking a Canvas context. And so whenever you pass in GeoJSON into the path, and I know now that if you haven't learned D3, you're going to get lost in terms of the technical examples for the rest of this talk. I apologize about that. And all you need to do, you do all the same Canvas stuff, and when you're ready to render a path, you just pass in the GeoJSON into the path element, path function, and it will just run it. And you get to pick your projection. You can do everything, almost everything you do in D3 Geo, almost everything you can do in D3 you can do with Canvas. You can get retina support with this little thing. The device pixel ratio, basically like this display has twice as many pixels in both directions as what it sort of reveals to the Canvas. It's sort of basically by default things look fuzzy because it's a retina high density display. A lot of displays are like this now. So that means for every pixel that it looks like it has when you say font size 13 pixels, there's actually two pixels in both directions, so four pixels for every one. So if you want to render, get that level of detail on every single actual pixel on this display. You need to put this little chunk of line of code. Here's another advanced trick. If you're a beginner, I mean, this is hitting Canvas lookups. Basically, a common complaint with Canvas is that you can't select objects on the Canvas because there's nothing there. It's just a bitmap. This is a trick that I just want to put it out there because it works with a lot of visualizations. You can create a second Canvas which you use as a lookup. Basically, you put a unique color for every object that you've rendered. And then you save that mapping from the colors to the original data into a JSON file. And then whenever a Canvas mouse event happens on the screen, you use getImageData to get the color off your hidden Canvas, and then you can look up your original data file. And then you can add tool tips or highlighting or things like that. Yannick from Boku actually has a great blog post on how to implement this, but I just want to give you a visual that it is possible to do it. A couple other ways to do interactions. You could use D3 Quad Tree. This is especially useful. A lot of people use Voronoi diagrams. They'll add a Voronoi layer of SVG which sort of maximizes the area that you can interact with a point on a visualization. If you use a Quad Tree and just search for the nearest point, you get the exact same interaction service. So in this sort of example running in the background, it's actually a bunch of Voronoi is being drawn on a Canvas. And I use Quad Tree to figure out which Voronoi is being drawn and highlighted. And as you can see, they are the same. It creates the exact same interaction service if you use it that way. Force Simulation has a new method in version four called simulation.find. So it's pretty easy actually to do. There's a lot of examples of the force layout in Canvas with interactions. And the force layout is actually pretty fast. If you've only used the force layout with SVG, the biggest bottleneck is SVG. The force layout really can support tens of thousands of nodes. And up to about 10,000, almost no problem. Sorry, tens of thousands of objects. So I'm talking about nodes and edges. So here I think there's about 6,000 if you add up all the nodes and edges. D3Path in version four, I think it's a new utility. I forget actually. But it makes available these Canvas methods. So these are all methods that are available on the Canvas. And if you use them in D3Path instead, it can still render it to a Canvas and basically do that. If you're only using Canvas, there's not a big reason maybe to use it. At least I can't think of one off the top of my head. But the cool thing is actually you can generate SVG path strings. And it's actually used for all of the V4 shape generators. So arcs, pies, lines, areas, curves, symbol stacks, all these things are now built on D3Path, which means they too can render to Canvas. So all of these shape generators now can do both in version four of D3. We have a huge problem, I think, in terms of educating new people on how to use D3. I think the learning curve is extremely steep. I think a lot of people get discouraged because it may be their first time programming. And I think it's a critical moment when someone says, I'm ready to take this active relationship with the computer and actually learn how to program the computer to make images because for whatever reason that they decide to do that. And if they hit this wall of complexity and fail to really get to a point where they feel comfortable, they're less likely to try it again. And if they do try it again and they fail again, they're not able to get that place where they feel comfortable and they feel like they can actually express themselves, they probably won't try it third time. And I think if we have this complexity in the learning process sitting there, we're going to lose a lot of people, just a sort of attrition of people who have worthwhile stories to tell. I've never met someone who, well, sorry, I've met a lot of people who are wanting to learn D3 and are struggling to learn D3. And they'll tell me why they're trying to learn D3. And it's just a worthy story that sometimes they're trying to tell. And often a complex story and a story that I've never heard before. I had the privilege to work for several years at Stamen Design. One of my favorite habits at Stamen is that when we are making prototypes and creating stuff sometimes, we just have an idea. And people will put their laptops on the floor like this. We kind of do it just, I guess it's the height of the couches, but it's also because it sort of minimizes the area that the laptops take. And we're, at that point, you know, an idea is consuming us. And we're walking around the room and we're, you know, thinking and talking and brainstorming. And the data and the ideas and the computer have leapt off the page and are now in the room and we're thinking about them and we're brainstorming. And I think that's really the goal of visualization. I mean, I'm talking about pixels and things, but pixels, you know, it's just a mechanism to get these ideas into our minds to be able to think about it and have this relationship with data, especially, you know, there's no way to have a relationship with a gigabyte of data, unless you're actually using computers somehow, some tool to internalize it. I was inspired to learn data visualization by this wonderful talk by Hans Rosling. At some point he explains that what you've just seen is something like 40,000, 50,000 points of data. And I was like, wow, I feel like, you know, at no point was I overwhelmed. I mean, I know I could go back and look at all the details of this visualization. We have this incredible capacity to consume a large quantity of data, especially when we have the right person to sort of guide us through the data and tell that story. And so I think we need more people in our community who are that person, who have that vision of the story that they want to tell. So I hope you will rise to the challenge of thinking about this problem, of how to make learning D3 and programming more accessible to a broader audience, because it would greatly benefit us and the stories we're capable of telling, to tell by having that diversity. That's all I have. Thank you very much.