 Welcome to a universe of procedural random. Well, the building's geometry, the roads, the water lines comes from an open street map. Carefully curated and chopped into tiles in our servers at Mapsyn, then desalated and extrude on Tangram JS, our 2D 3D map engine. At the rest, the grids, the numbers, the patterns, the pulsing tiles, the movement of the camera. All that is constructed in real time, directly in the graphic card of this computer. There's no video. The language to program directly on the graphic card is called shaders. My name is Patrico Gonzalez-Vivo. I'm an artist interested in landscape. My previous year, almost like two years ago, I had the privilege of being working with all these great engineers and designers. I have been learning so much. So I feel very privileged to be working there. And especially because of everything that we do. Oh, since when that happened? Thank you for pointing that. So all the things that we do are open source. So please send it to our website. We would like to listen to your needs. We have a part where it says you can get an API key for free. So there I have been working in some of these experiments. Working from the back end to the front end. And lately, I have been working on the engine, not just doing implementations on it, but pushing the limits of what cartography is. And because I'm not a cartographer, I kind of do that. So I am having a lot of fun doing these demos and stylings for the maps that we do. And lately, I have been trying to add more data to that. So I have been finding ways to bring animation and data to our maps. And most of the examples that we saw here use what is called GLSL shaders, which is particularly the thing that I love the most and what I really enjoy about my work. So what are shaders? Let's do a test. Who knows what a shader is? Great, this is awesome. So I'm going to go through a small introduction to it. I have been working in my free time in this book called The Book of Shaders. And I'm going to bring some of the contents of it into the presentation. So if you feel lost or you want to dig deeper, I encourage you to take a look to it. So this is a shader code. And we can start thinking if we take a look to it, you will notice that it looks like C. Yes, it's a blend of C. So you see your main function there. And then it's very small. And then you will discover there are specific variables and things happening there. So I will try to do a small interpretation of what is happening here. And it was great that Mariko percent that, because all she was explaining about the double loop to do pixel-by-pixel calculations, actually everything that she puts inside that double loop goes inside this part. So I'm going to go back. And I'm going to start figuring in metaphors. So I like to think in shaders as the good and per-press. So shaders are for graphic engineers, like me, what the good and per-press was for authors of books. They let us free for the one single thread that Mariko, I guess, thank you for explaining that. And let us put our work in designing a page. So making an algorithm that we'll, once we have it, print a whole page once in a time. So there is no need of creating a letter after a letter, after a letter, to make a word, and then put all words together in order to have a line. And then zero lines makes a paragraph, and zero paragraphs makes a page. Here is all the letters, all the words, everything in a single page happens at once. So I have a second metaphor to explain that. I like to use this particularly with my students. So imagine this single thread as a pipe, right? It's designed as robots, your computers. This century, this year, we have very robust computers. So they are designed to do very complicated tasks. And probably, I see a lot of new computers. So you have around four of these big pipes for course. But for small things, for example, computing pixel by pixel, that slows down. So she was speaking about how when you make pixel by pixel operations, this slowed down your computer. Well, why is that? Because imagine that deciding a color of a pixel is a very tiny little task. So if we have these big boxes that are like complicated task, deciding the color of something, something very small. So we can picture it as like a ping-pong ball, right? So in an old computer, in an old screen, that's almost half a million of pixels to compute. And in a screen running at 60 frames per second, it's almost 26 million calculations per second. Imagine now with one of these new retina displays. It's just very about five million pixels. Put that at 60 mHz per second, it's around 250 million calculations per second. It's insane. So if we try to do that with a CPU, we get a sad CPU. And so imagine that we have these industrial pipes, and you were trying to put all that stream of pixels of ping-pong balls of small calculations. And they will stuck right in the entrance. So hopefully, and thanks for everybody here with a new computer, engineers have solved this problem by using parallel computation. This has been out for a while, like 15 years ago. And the idea of it is like to sub-device, to have like little tiny little pipes, very small processors running in parallel. So going back to our metaphor of the big industrial pipe where things pass, these are like more like, I don't know, toilet paper tubes. So the idea is that this constant flow of ping-pong balls that are like our computers all the time generating will pass through them like water in a strainer. Each one will go for a different pipe, right? And to illuminate this metaphor that I'm like insisting on, imagine each one of these tube as something that is less powerful of an Arduino. I don't know if some people has like the chance of programming on an Arduino, but it's very simple because an Arduino, what you give it is a C code. So imagine that you have a table of Arduinos and you have the, and each one of them is controlling one line, one light, one like LID light, LID, RGB light, to be more precise. And you have the opportunity to flash them all at once and at the same time. When I say flash them all, I mean like upload a firmware all at once. And also imagine that these all this stack of Arduinos that here are not enough, we have one, imagine that you have one per pixel. They share some resources. And one of the resources they share is a memory and this specific memory is what we call the memory of your graphic card. Probably you have heard that when you go and to buy a computer, it's a completely separate memory that is designed to allocate images and geometry mostly. So this is my introduction to what a graphic GPU unit versus a CPU is. So let's jump into the coding part of this. So be patient with me, I want to make this bigger to everybody can see, right? So this is an online editor that I made for the book. This is where I spend most of my time coding examples. And I want to point out to some important aspect of this code in order to explain what shaders are. So this is the main function where everything happens. This first variable over there is the position of the pixel that is running the computation. So each one of these pixels on this viewport is running the same function. Here it receives the information that where it's located. And here is the variable that you will write what is the color of that. So if you are paying attention, you will know that these colors are, as Mariko also noticed, it has four values, right? But here, instead of going from zero to one, to 255, it goes from zero to one. So we will say that all the values are normalized for the colors. So we pass here our color. In this case, we are passing the position of the pixel normalized set. And in the blue channel, we are putting that sign or side wave of the various of it over time. So let's do some, let's start making something more fun. So I'm going to take this line out, whoop. And in order to illuminate what I just said, I'm going to put the coordinates in X. So you will see that now the pixels in the top, in the this side, in this side are zero. And when they achieve, when they get closer to the other side, they get one, right? And this make this nice gradient. If I change the X by a Y, the same happen from bottom to top. So now, if we put two things together, we know that the top, the zero, zero position is the top, the right, the corner. So I will do a small demo of how to make a shape of this. So I'm going to get the dot product of ST, which is the Corning system. And that will give me this like half sphere here. I want a circle. So I'm going to move the whole Corning system, the variable that holds the position. I'm going to move it on a side. So I'm going to pick this. So well, this is, we can say that this is 55. And then, what I love the most about Shaders is that the API, the language is very small and it's mostly mathematical. And the code looks very atomic. And also, the language comes with some useful functions. And one of them is essentially a threshold, which is called step. So we're going to say that that's not nice gradient. We want to cut it in 0.5. So we're like, in the previous step, we imagine that that was like a mountain, like from black to white. And here we're like making a sling cut in one specific point. Let's invert this. And now we have a red, white dot over black. And this is where the moment when I was like, oh, okay, I'm sell, I buy this technology. Because this is calculated in each one of the pixels on the screen, it doesn't, there's nothing like a circle. So if you have drawn circles before, you know that there is a moment where the computer is like getting angry and sad that you're making so many circles. But here, because everything is calculated per pixel is the same if you put one or a million. And then let me show you that. So I'm going to grab the screen space, I'm going to fract it. That means get only the fractional part. And I'm going to multiply this by one first. And then I'm going to multiply it by two, three, four, five, 10, 100. And the computer, and the frame rate is constant because I'm not in any place saying draw one circle. And each one of these pixels is giving the coordinates to draw a circle. And they are checking if they should be drawing something black or white. I hope that makes sense. And I'm afraid to look to the screen too. So one more, another example of that. I'm going to make ST. And I'm going to add ST to SY. And I'm going to use these two, oh, sorry. I'm drawing over black. So basically I'm making a sine wave using the combined position of X and Y. So that makes a sine wave in the side. So let's multiply this with a big number. So one more big, two big. And now let's do the step thing again. And we have a straight pattern. So this technology is not rare that this is used a lot in game industry because it's very cheap to make textures, generative textures dynamically. And that's what I love about this. And this is what I do all day. So that brings us to the second theme. How to apply this to a map? Well, the data comes from OpenStreetMap, right? And that's vector tiles. We chop the information from OpenStreetMap into vectors. So basically that's like SVGs, right? Positions on the screen. And then what we do is we extrude them. Some information in OpenStreetMap, for example, the buildings. Some buildings have height. And so we generate extra geometry in order to make that extrusion. And this is basically the canvas that I work with. Once we have this, I can apply using the same techniques that I showed before, any pattern on that. I have an example here. So this is basically the same simple patterns of dots and lines, but apply into lower Manhattan. And all this is running directly rendering in real time. Or with a little more effort, you can go in the other direction. Something more realistic. So here I'm using pseudo random to generate some, to light some squares in some parts and not another to generate the illusion that these are windows. New York doesn't look like that really. It's more like a Gotham. So I have the privilege of starting this project called Tangram Play. I really like making tools for the things that I do because we're coders and we're lazy. And sometimes when something takes you too much to do something, you start thinking, how can I make this go faster? So I was editing using Sublime and I wanted to start making online tool to edit the scene files that we use for Tangram. So I hacked something together and then Lou hung that he joined the team of MAPSEN. He's a UI engineer and so he took Tangram Play to the next level. You always refer to it like we have this baby called Tangram Play. So here we have Tangram Play which I encourage everybody to take a look. You have a lot of examples that you can look but I don't want to overwhelm you with a lot of things. So I want to explain a little bit what is happening here. So probably most of you have worked with MAPS before and this is very similar to a Carto CSS or 10 milis. Basically something that decides how you take the data, how processor and how to display it. So here we defined our source. Our source is MAPSEN vector tiles. Then those vectors that have different layers inside them and I'm deciding which colors goes where. So for example we can change the, what are two blue that's very, sorry, you know today. And in this case for example in the building, which is out of screen because if everything was zoomed in the buildings I'm defining a custom style called buildings. And here's where the magic starts and when I mix my job so satisfying is that Tangram gives you some space to access the pipeline on the WebGL pipeline and specifically to, you can inject shader code into the pipeline. So here you can see how the buildings style set a new line inside the shader which basically adds this nice gradient between the bottom of the building to the top. So if I comment this out you will see that the sad buildings. Let's make happy buildings. I'm feeling a little bit Bob Ross today. So I will, how we do this? I will mix two colors. I'm going to go for red to blue again in a crazy act of creativity. And now we see that the gradient goes from blue to red, right? And we can add some animation to that. So let's say, let's turn the animate thing on and then use time to move this up. So I will add time and because time goes indifferently I'm going to make a scene, wait, no. Let's make it right. I had to close the parentheses. So now we have a happy, flashy buildings. I leave the rest for the data visualization people. Hope you're getting excited as I am. So now comes the data equation on the maps. Well, maps have data, but what to put in a map. So this is something that I have been working on, some work that I have been doing. But first I want to speak about efficiency. So when you have an engine and it's a WebGL engine and you go on to draw stuff from the engine part of you you want, the thing that you want to avoid is to hit the server too much, the driver too much. So imagine that we have the CPU, we describe that and I present this idea of the GPU like in a bottle. You don't want things to be sending constantly. It's like making ships in a bottle. You want to put all your stuff in the bottle and then just move things around in it because otherwise it will not go into work. So, and also I make reference how the graphic card is really have some resources inside it. So it has the ability to store the geometry which is going to be the map, the same tiles that I already show you and images. This is very used in graph and videos, right? You put up everything, the walls has a texture and all that. So for this project that I was working at the end of last year called Line of Sight, I want to visualize in real time all the satellites that were present. This is only a subset, right, that I made. And yes, it's a lot because I want to get in trouble if I do this, this takes longer to load or yes. So now we have all the satellites available and imagine this, what I do this. Each one of them you have to predict where it's going to be, right? This is a prediction of where it's going to be that satellite in the next hour. So this is around 15 southern satellites that have to be visualized with precision. So that's a lot of points. And a lot of things to be doing to be passing information from one point to another one. So I was thinking a lot about how to solve this problem. And I came back, I came with a technique that actually is very well known in the game industry which is encode the data when you pass it. So this is, so you need 24 frames per second, right, to make something smooth. I have 15 hundred of satellites. That's around 36 calls per second if we want to animate all these satellites which again is a lot and this has to run in a browser so it's kind of like asking too much for a browser. So the solution was to put the information to use the memory that the graphic card has designed to hold images to store all that data. So if I packed all the data and imagine that this image is like a piano roll, right? If we put all the information of all the satellites in an image and we pass it inside the bottle and all the information is right there. So this is how the image looks and it's very interesting. So each satellite will look in one specific row and then it will know that the latitude is at the beginning and then the longitude is at the beginning of the line if and from the second half of the X axis is going to find the latitude. So basically each satellite knows which line to read and then it's reading from a moment and like moving the header away. I don't know if that makes sense? Yes, good. And that I was very happy also to discover that that saves a lot of memory. So here the same JSON file that holds the same information is around 20 megabytes while the image of that is only two megabytes. So yeah, 10% efficiency. How the satellites knows where line to see? Well, I'm using color again to encode an information and I want to put some little bits of info. So this is some JavaScript that you can also put in the YAML file for Tangram. And here I'm using the red channel and the green channel to encode the idea of that satellite and I'm lifting the blue channel to see if that satellite is hovered or not. It's selected or not. Then in the shader part, I have these nice functions that according to the color of the satellite, it will search for a specific place. So it's a way to trick the graphic card to pass information through. And what I like about this is it's kind of the same way. It's poetic in this way that it's how we know about the composition of the stars. Like astrophysics have been done this before which is analyzing the color of something, the light coming from something and analyzing the color and getting all this extra information about this thing. So it's a nice analogy between a way of like reconciling these two universe of the CPU and the GPU. So I have one more example which is, I kind of put it together for this talk. So I, who here has a weather map done? Because Fernanda and Martin has a very famous one and I know that, yeah, yeah, thank you. So I said, okay, this is an OpenVis conference. I had to do better. I had to make something that they will enjoy. So don't judge me. This is my first data decision with the data. So I set up my Raspberry Pi in my desktop to be hitting the NOAA servers every day and collecting the last 24 hours of data in wind and temperature data with a Python script. So that's around 70, 100 stations. You have been doing this the last 40 days. That's around 960 hours. That's almost half a million samples, right? And I'm going to, I'm storing the temperature of each station in the red channel and in the green, the wind speed and in the blue channel, the wind direction. And in the server, this image is building up. So I actually, let's get out of the presentation mode. I will pass all the information about this. But if you come here, you can see the image. Actually, you have access to the image itself. And the Raspberry makes the pull request. So this has been working on weekends. So this image is building up. And now, I made a small map. What it does is visualize it. I know what you're thinking. I'm using rainbow gradient. I'm looking, everybody in the eyes, don't judge me. So what is happening here is we have all these weather stations. All that information is passed directly to the graphic card. The graphic card have all the information for all these weather stations. So the animation can go very smoothly. And I only need, if I want to add some interaction to this, I only need to pass the offset value from the moment. So you can let me, I think from the distance, you can see better patterns. I add some feature, if somebody's curious. If you click one of the stations, it will give you the data and how the data looks in the image. So here, you can see the spin direction, the wind direction, the speed of the wind, the temperature changing over time and being interpolated in real time. So this is, I think this is a compelling argument of how something, how combine all techniques from other fields could be very beneficial. And also it's very, I find very interesting that this image by himself, I have the chance to travel the last weeks and we were in Virginia with my wife. I think in this here, no, in this red spot here, which means there was more red. There was kind of nicer days, more warm days. You can see here the blueish days, blueish, greenish days and how the last days has been better than better. Also it's very interesting to see that there are some places that never gets cold. I guess this Alaska is always cold. So here's the link to this. Feel free to play with it. This is a link to this presentation. Any question that you want to answer, I will be glad to reply it. I will going to be around. I want to thanks so much to everybody for going through this. Thank you for dealing with my accent. And thank you, Baku, for inviting me. Bye-bye.