 Can you guys hear me? Yes. Thanks, I read. So yeah, hello, everyone. My name is Nicolas Palmonte. I run the visualization team here at Uber. And today I'm going to be talking about a framework we've developed that's called DECGL. It's in version 4. And I am announcing it right now, so probably should provide some background. Hey, what happened? You don't release something in v4. So I joined Uber about two years ago and started a visualization team there. So people there act as a functional team, like design or data science. They go and embed into different other teams to solve visualization problems. So within data products, you can imagine experimentation or machine learning visualization or even partnering with a self-driving car team to have these 3D immersive visualizations that I cannot show, which are really cool. Anyways, we started using DECGL for that. And so it's been stretched and used in many different use cases around geospatial visualization but also for abstract data visualization use cases. And we've decided to open source this end of last year. So I'm gonna talk to you a little bit more about this framework. So what is DECGL, if I were to summarize it, is a large scale WebGL powered data visualization library. Large scale means you may have data dense visualizations, that doesn't mean that you need to render a million points on a screen. You can use WebGL today to do also data processing and data manipulation and aggregation and filtering and so on. So this framework also attempts to basically help you with that. But also if you were to render 1,000 elements or 10,000 or 100 or a million, this can be done with it as well. We decided to go with WebGL. There's a new major version has been released like a few weeks ago, WebGL 2. It allows for a lot of interesting general purpose like GPU programming type of stuff. And we're planning to kind of conquer that space with DECGL as well. So again, high performance GPU computing, we've used it, Uber manages billions of GPS points every day, so we've used this with maps primarily. It does something very specific very well and then it plays really well with other libraries. So you can hook it with React or you can hook it here with MapboxGL which provides perspective mode. So what we do here, for example, is we use MapboxGL as the base layer and then on top of that we overlay different layers built with DECGL. And so you see how the camera systems are synchronized and stuff like that. The other interesting thing is it follows this like instancing and layering paradigm. So the instancing paradigm, you might know it from D3 like this data binding. Like you're creating a scatterplot, you have an array of points, of elements and then you map them one to one to circles. And so what you do with that selection is it's instancing a bunch of circles and then you can tweak them by switching or changing a few attributes. Coincidentally, like WebGL 2 is extremely efficient at doing that. It's called instancing and you could have for example a cone or a sphere or whatever primitive like a circle and you can instance many of those with just only one call. And so you'll get basically very efficient way to do this sort of data binding. So we follow this pattern, which is both interesting from the InfoVis perspective but also interesting as well from the WebGL like performance type of perspective. And then the layering paradigm comes from GIS, right? Like you have a map and then you have multiple layers. Like a layer could be land and water. Another one could be like labels, roads, et cetera. And you could now have layers on top of that that display data. In this case we have a scatter pod layer on the top left, a hexagonal layer on the top right, grid layer on the bottom left and an arc layer on the bottom right. And so it's a good way to reason about your data. It's a good way to plug into the framework if you wanted to develop your own layers. But we also provide like a dozen different layers that do these things for you. As I mentioned before, the layers are not purely visual stuff. It's not your only rendering stuff but you can also perform different things like aggregation and filtering and so massaging the data in different ways that all happen within the layer as well, right? And so you could imagine this as well as combining for example D3 with DECGL as a Voronoi layer. You pass in a bunch of sites and then the layer uses D3 Voronoi to compute the polygons and then it renders it with the polygon layer inside, right? So you have this sort of like way of adapting code and doing some data processing in the process. What we're looking at here is like a hexagonal layer. We're changing the granularity for the grouping and we're changing a few of the visual attributes and we're also like filtering by percentile. So as you slide the upper percentile, you basically remove the top percentile of the distribution. We've used it for abstract data visualization. So I mentioned we do some of the machine learning visualization stuff as well. Here we're visualizing the output of an embedding using T-SNE. This is clustering, spheres representing stuff that I cannot tell you what it is but it's very insightful and the purple stuff is especially interesting if you know what it is. Finally, there's something that we're really proud of that we've worked on here with this framework which is WebGL has a limitation quote unquote which is as you're coding your shading language, it only supports like 32 bit precision floats. So basically there's only so much you can get from it. And if you were using GPS like global positioning systems and you wanna get that to like the centimeter level for example, well, WebGL would not enable you to support that. And so there's a lot of things that maps do like using web workers or changing like coordinate systems and stuff like that to be able to support that. We took a fully different approach. We basically it's a brute force maybe approach. Like we decided to emulate 64 bit computations, operations by basically doing this technique where you pack two 32 bit floats into a vector two and then we had to basically redefine all math operations linear and non-linear in the shader so that you could support 64 bit. Anyways, what that means is that when I show this very trippy image on the left you have the 32 bit precision it tells you how much you can go and then on the right one you get to see that you can go exponentially further, right? And so all that is emulated in the GPU. Cool, so now that you know what it is and where you could be using it, how does it work? So if you open up the black box, this is how it looks like. It starts, I would say probably if we start from the left side, you have this react component it has some data, you wanna render it using deck GL. Let's imagine that you wanna render it on top of a map. Then what we do here is the high level components that we provide are the family of frameworks that we have deck GL, react map GL which is a react friendly wrapper around Mapbox GL and Lumet GL which is a lower level library to the WebGL rendering which is internal mostly so you would not need to play with it that much. And so you're passing the data through the react component. The interesting thing with react map GL is that the camera system and the viewport that deck GL has are combined. So as you pan, zoom, tilt the map, like both the layers and the map are synchronized. And then as you pass your data, let's imagine you have a layer manager. In this case, this is a rather complex example because you have a layer that has three sub-layers. So you could imagine this, let's imagine you wanna render DOJson, right? So you're passing this DOJson to a layer, then the layer says, well, these feature is a point so I delegate that to my scatterplot layer. This feature is a polygon so I'll delegate this to my polygon layer. This feature is a line, I'll delegate this to my line layer. So you could kind of have composite layers in that way. But if you wanted to simplify your example here, we could just have one layer, that's the scatterplot layer. And Lumet GL takes care of the rendering for that layer. Hopefully that makes it clear. Anyways, so what's react map GL? So react map GL as I mentioned before, it's a react-friendly wrapper around Mapbox GLJS. And if you don't know Mapbox GLJS is a WebGL version of Mapbox and it's amazing and you should totally use it. And then react map GL, what it does is it provides a simpler react-friendly interface on top of that. What you can put inside of your map can be anything, it can be a div, it can be a canvas. So this is where really DECGL comes in. It provides you a way of managing all these layers to create a WebGL context or reuse that context and to make it really performant. And then Lumet GL, we decided to go with this because sometimes there are differences around vision and direction for certain open source frameworks. So for example, some of them wanna invest more in VR and AR stuff which is totally fine and fun. But then things like adding WebGL to support get the prioritized. So we really wanna have control on that and so we developed Lumet GL for it. So the cool thing about it, it's a general purpose like with WebGL framework. It is written using ES6 code base, using classes to wrap main WebGL one and two entities. So it's actually like, if you wanna get familiar with the lower level WebGL stuff, reading the source code of Lumet GL is a good starting point. And then it's WebGL to ready. So you can use floating point textures, instancing, transform feedback and so on. All right, so that's the overall architecture. How would you use this? Where are the main use cases? So there are three approaches. I guess they get more complex as we go down on that list. But on the right you get to see some of the core layers. This is an actual screenshot of the layers. So we have more than a dozen core layers and some of them can be quickly adapted to your needs. So one of those use cases could be using existing layers which is pretty straightforward. Another one which is pretty straightforward as well is creating adapter layers which we'll visit. And then finally, if you wanted to go more and tweak some shader code, et cetera, you could extend these layers to have them customized. So on the first point, using existing layers, if you go to deck.gl, there's a gallery, you'll find a lot of examples. Each one of these examples are standalone examples. You can just clone them, copy them, tweak them, learn from them. You can view the code for this example. There's a fat arrow there. You click there, it'll take you to the GitHub page. And then as you look at the code, you'll see a bunch of props. You can use the documentation, API docs, all interactive. And you could kind of figure out like which one of these props, like what do they do basically. In this case we're instancing a new scatterplot layer and then that's the code inside of the larger piece of code. We're wrapping that around a deck.gl overlay which instances deck.gl as part of its return statement and adds that layer in. And then finally what we do with that deck.gl element is we add that deck.gl element inside of the map.gl map. So in this case we have a map with the layer inside of it. On the extending layers, as I mentioned before, you might want, for example, to write your own shaders for a layer. You might want to add uniforms when drawing a layer so that you want to add extra functionality. In this case, for example, base layer, arc layer, arc brushing layer, you want to add a mouse over functionality or filtering or something like that. You can do that with the arc brushing layer. And then finally, if you want to go more wild and have your own custom picking mechanism or how full flexibility into what WebGL can do without this framework bothering you, you can just do that by extending the layers as well. An example of this is, I don't know how useful this example is, but it's a good illustrative example. You have a scatterplot layer, right? It renders a bunch of circles. You wanted those to be rounded rectangles. So in this case, what we do is we extend the scatterplot layer and then we override the draw function. So what we do in the draw function is we just call the same draw function as before, but we pass in a new uniform. You can think of a uniform as a prop. And it's a corner radius, right? And then on the shaders, instead of passing in the same shaders as before, now, since we're adding a new functionality to these shaders, we'll pass in our own custom fragment shader. But we'll reuse the same vertex shader as before. And then finally, we set up the default props for the corner radius there. Basically, this is a fragment shader. I'm not gonna go much into detail for that, but in order to create, so imagine it renders rectangles. In order to create a circle, it discards a bunch of pixels that are being rendered on the screen, so you can control that basically from the corner radius prop uniform. And then finally, composite layers. So as I mentioned in the architectural diagram, you can manage multiple primitives into one layer. GeoJSON is a good, clear example of that. I think that the cool thing about it is that it breaks down a complex rendering problem into smaller testable classes that you can reuse. But you could also use composite layers to do some data processing or aggregation before you do the rendering. We talked about the hexagonal layer, that video, like there's some aggregation before the rendering. The Voronoi layer that I just exemplified, like you pass in a bunch of sites, you compute the Voronoi using D3. Once you have those polygons, you can render them with a polygon layer. That's a way of adapting a layer, changing their interface so it suits better your specific domain and problem. Yeah, so those are pretty much kind of the use cases where we see people kind of plugging into the framework. So for that we audited a lot of the APIs, made sure that they were consistent across multiple layers and also provided good documentation for these specific three use cases. So the first year at Uber I stopped coding. So for a year and a half I didn't code and I went back to coding for this example and I realized why I stopped coding. But it's an interesting use case. It's a way of creating custom layers to visualize wind. And in this case the data for the wind is not correct. We do some data processing from sparse data coming from different kind of wind stations to have a more uniform pattern so we can render this. But I wanted to showcase that because I think it's an interesting also way of using WebGL to do just data processing and not the rendering itself. So the story for this example, two years ago as Irene mentioned, I was here and I showed a wind map as well. And last year, Patricio, who used to work at maps and then took the same data set and also created a visualization of wind using Tangram which is an awesome tool. And so I wanted to do a re-remix of this. The data is already cleaned up, it's easier. So something that really bothered me though about this data set was the fact that the data set is like sparse. You have all these points. They encode wind velocity, direction, temperature and elevation. But the discrete points, they're not interpolated. If you pick a random point on a map, you won't get the data for it. So how can we linearly interpolate all this data in a sense that no matter where you are in the US you can get some sort of approximation of what the wind information is. You do triangulation. So basically what we do is we can encode wind speed into a color channel. Let's imagine like red and angle to green and speed or I don't know what temperature on blue. And so you can render those dots as colors into an image and every color will encode this three dimensional or three dimensional data. So what do you do with the intermediate points where if you have that as a triangle and you render it through WebGL, the GPU is really fast at linearly interpolate all these values in between. So you could use that. You could take advantage of that to do that for all your data sets. So the only thing that we need to do first is the deloni triangulation for the stations in the US. And I think this might be the first time I use the deloni triangulation for something that's useful and not only pretty. And so once we render that, these are a bunch of triangles, right? So you will get all that interpolated and it will return an image where you can sample any pixel in that image and you'll get that interpolated wind value. So the step after that was getting a vector field to show up. So for that we created a layer, use the instancing paradigm, right? So we have a bunch of lines that can clone multiple times and that way we can basically have an overlay of how the wind looks like at every point in the US. And this is the first kind of approximation of it. So we used the deloni triangulation also to encode elevation. We use it as yet another layer. We have a layer for the stations, a layer for the deloni triangulation and a layer for the vector field. So I used that, I added a little bit of lighting and stuff like that during the process. And this is the first stage. So on the left you have a slider. So these are the last 72 hours of wind. So you can see that vector field kind of interpolate. On the top right you get to see like the outlier of wind like Mount Washington where wind is like exponentially stronger than anywhere else. So don't go there. And then at the bottom you get to see some of the stations. So every yellow dot there is a station but we don't care about them anymore because we're interpolating all of the data. So the next thing that I decided to use was transform feedback. And this is a new technique, well a new feature that WebGL2 has. And I'm not gonna go into detail with the rendering pipeline but previously it starts from the left to the right. So you're passing a bunch of points, imagine like 3D coordinates as an array. It goes through the vertex shader which is programmable. You can type in GLSL code and you can perform like affine transformations to those points so you can translate them, rotate them, scale them, do stuff like that. And then there are multiple other processes, like from the points you create triangles. From the triangles you have a 3D concept, you need to put it on the screen so you rasterize it. And then it goes through the fragment shader which after rasterization it decides like which color to add to each pixel on the screen. But what we care about though is that vertex shader there. Like that's pretty interesting. You're passing a bunch of points and you can do a lot of like computations in the GPU in parallel and sometimes you don't really need to render the stuff in the end. Like you could just do random math with it, right? And so WebGL2 enables this thing called transform feedback where we can basically use the output of a vertex shader as a buffer and use that as data. So you could imagine yourself just, you know, defining a sine function in the vertex shader and passing in a random array of points and then getting back that sine function back. There is a little bit of overhead of like creating the buffer and sending it in, et cetera but it's overall super fast and made parallel. So this is how it looks like. There are some voids in Florida. So we throw particles about a million like randomly in the US and at every point we sample the place in the texture where those particles are moving around and we kind of update the velocity and the position of that particle. And then as you change the time, you'll just like keep sending over particles. Every particle has a time to leave. So, you know, they'll move around for, I don't know, a few seconds, then I'll take them back, throw them at some other random place. So, of course, what's interesting is, well, you know, there are interesting patterns. Like you have finer granularity on top of your data and the way it's being visualized, I think it can be reused in multiple ways. Actually, we'll add these layers or as core layers in some like later version of DECGL. But it also shows you the flexibility of the framework, right? You can just plug in anything, build your own layers and create an app with it. So, I'm taking all the credit, but this is a DECGL team. As I mentioned before, we do visualization within business insights and advanced analytics and a lot of core visualization work more around centered, like around computer graphics. So this team has been working thoroughly on that. It started as an internship project by Yang Wang there and now it's pretty core to like a dozen different internal labs at Uber. So that was pretty powerful. Thanks.