 Let me try to get a power cable here. So yeah, I'm going to be talking about Regal, which is a functional abstraction over WebGL to make it easier to write modular, reusable 3D engines and visualization tools. But first, I'll say a little bit about myself. So just to get out of the way, I do live in Hawaii. I live off-grid. And for the last five months, I've basically been living on an active volcanic lava flow trying to survive. So if you've seen my commit history drop off a little bit, that's why. But it's starting to get under control now. So I'm kind of getting back into the flow of things. So anyway, let me say a little bit about Regal. So these are some examples that people have made. So initially, I put Regal out maybe about a year ago. And since then, a lot of people have actually adopted it and started using it in real systems here. So I'm going to show, first of all, a couple of things that people have made with it. So oh yeah, this is our cooperative website, so I should point that out first. So this is a little creative project that's basically using Regal. You can see this here. I don't want to go too much into it, but all the effects are in WebGL. This is CAD company that does finite element analysis, and it runs on your browser, more or less. And this also uses Regal for the visualization. Ricky Roycer has written a bunch of different demos using Regal. Just show a couple of these really quick. So this is like a real-time GPU simulation of erosion on some terrain. Here's another one that he did, which is some kind of n-body simulation with like a million particles, and they're all kind of forming together and making galaxies and stuff. So it's sort of like a little creative sketch. Here's another one that he has, which is a bunch of particles flowing around in 3D. This is one by Greg Tatum. I don't know exactly what these are. They're kind of terrifying, but this is also all in WebGL, and they're really cool-looking, so I'll give them a lot of credit for that. This is a terrain demo by Ry Tyrol, also using Regal. This is a convolutional neural network, so this is also running on the GPU using Regal. It's basically like you draw a little digit using the MNIST data set. So it works, it's a comnet, and it's in WebGL, so that's kind of cool. This is a visualization from 538 showing gun deaths in America, and it's also using the same framework under the hood. And then also there's a ton of examples in this gallery here, so just pages of this stuff, so you can click on any of these, check it out, and then figure out how it works or look at more of the details there. So okay, with that all out of the way, I can now see that it's a real thing, people are actually using it. So what's it about, why did I create it, and what is it good for? So, at a high level, if you want to do computer graphics like 3D graphics or 2D graphics, there's two big approaches that people consider. So you can either grab some 3D engine out of the box and just configure it with different options and then use that to render your scene or your game. So for example, you could just grab Unreal or Unity or 3JS, and then you just set it up and then maybe you got something that works out of the box and you're all set. Or the other option is you can roll your own and do it yourself, and on the web this makes a lot of sense because you have a limited amount of bandwidth to download stuff so you don't want to burn megabytes of JavaScript loading features you're never going to use, and also you have more control over all of the details. So this can be good if your project has to be maintained over a longer period of time and you need to customize it and add special features to it, but the difficulty with this is that you're then going to be re-implementing a bunch of stuff. So you either have something where you have more control, which might be better for a one-off project, or it might be better for a longer term project that has to live for many years, or you have something where you can basically get something up and running right away, but then maybe down the road you have problems. So this is kind of annoying, right? It would be great if we could do both, right? What we want is something where everyone has the full power to do whatever they want with the 3D engine that they're using. So we want people to be able to get into the shaders in their 3D engine and write their own code with that, should be able to dig into whatever kind of geometry assets are inside the 3D engine and you can then mess with them or just play with that data doing something novel rather than just having some opaque asset format that you can't introspect into. We want reusable components so you can take just tiny pieces of different rendering engines and put them back together, and things should be more explicit rather than just some implicit thing that has a bunch of weird configuration options that you have to know about before you can do that. And then finally it should also be open so that anyone can go in and start adding pieces to it and build exactly the 3D engine that they need for the problem they're trying to solve. So I made some earlier attempts at this goal and the first idea, or the first big approach was this StackGL project. So I worked on this with a variety of different people in the NPM community. And it was marginally successful. So there are sort of three big parts to StackGL. There is a collection of WebGL wrappers which basically take the core WebGL API and then break it into different classes that abstract things like shaders or buffers or different parts of the system. There is GLS-LFI which was a module system for writing shaders and a set of mathematics and geometry library for doing geometry processing. So stuff like surface triangulation, working with higher dimensional arrays and so on. So StackGL was sort of a mixed success. I would actually say that the mathematics and shader logic that came out of StackGL was incredibly successful and it worked very well. The WebGL wrappers had a lot to be desired and I don't think they never fully realized this broader goal of creating a modular system for writing 3D engines. And the reason is that each of these little WebGL wrappers had a ton of coupling because they were just trying to take out little parts of the WebGL API piecemeal and never were able to take some broader, more holistic approach to this thing that could actually solve it. And so there's a lot of coupling between these modules and also because they were fairly complicated and the interfaces were very staple it was hard to test them and there were unclear specifications for how a lot of things needed to work. So it didn't really work that well but I mean at least GLS-LFI worked well and the math stuff is still very useful and lives on in a variety of other projects today. So Regal is an attempt to go back and revisit this and fix some of these mistakes. So the goal is to basically kill all of the shared state that exists in WebGL in the same way that React kills all the shared state that exists in the document object model. And also while doing this to go back and do something that's well documented, well specified, carefully tested and also very low overhead. So reducing the total size of all of the code assets and also reducing any runtime overhead in the system. So just make something that's small, fast and kills all shared state in WebGL. And that's what Regal is about. So what do I mean by reactive or no shared state? Well, we've already seen a couple of talks today about this. The basic idea in a sort of more traditional rendering pipeline, something like 3JS or the document object model is you have this large external graph of objects, right? Usually in some hierarchical structure. And you have your data, which is what you actually care about. And then you have this data binding thing, right? Where basically you wanna update something in your data, then you have to go back and update its representation in that graph and vice versa. So you have this thing where you're constantly trying to keep these two copies of the same data in sync all the time. So it's this sort of dual right problem in a database where if you think you have an index here and you have the actual data in the database, then when you wanna update it, you have to update the stuff in index. And so you're always fighting and making all of this complex stuff happen in your code to try to keep these two copies of what is really the same thing in sync. So that's basically what most things do. And the reason things tend to look like this is that if you're just working in writing a 3D engine, it would be nice if you could only think about the scene graph and not worry about this. But when you're writing an application holistically, you have to consider the entire system. And so this data binding is problematic, right? It creates a lot of friction. So there's another way to do it, right? The other way is now as popularized by React, but going back even further to things like immediate mode, GUIs from Molly Rocket or a variety of other older ideas in a sort of functional rendering system. You just have a single copy of your state and you have a function that transforms this into pixels on the screen. So you don't have this data binding stuff going on. You just have one state. This thing gets updated by whatever means you choose. And then you have some code that you write that turns that state into rendered visualized stuff. So this would be how React and Redux work languages like Elm also do this. There's a variety of examples and it's a very popular and very successful paradigm for creating interactive GUI applications. So this is now where we get to Regal. So Regal is basically this replacement for all the WebGL stuff that was in StackGL with a new functional set of libraries. And I could go on longer about how all this works in more detail. But I'm going to actually do a little live coded demo so then you can actually see for yourself what this is like. And hopefully I don't blow anything up in the process and maybe this will all work. Or it might not, I don't know, whatever. So yeah, so let me kind of move some windows around here because the projector is a different size. So, okay. So can everyone read the text on the screen here? Should I make that bigger? All right, we're all good. Okay, so what I'm going to do here first is just create like a little 3D engine from scratch using this thing. So I'm going to start out by clearing the screen to black. So this Regal module, when I load it, what it does is it basically gives me a constructor. Hopefully I did that right. I may have to go restart the server here because, okay, there we go. So, right. So what this is doing is this require here loads up the constructor for Regal. And then when I call it with no arguments, I just get a single full screen Regal context which I can then basically call different functions on and use that to draw stuff. So by default it comes with this thing called Regal.frame which hooks a callback that gets rendered each frame. And then I can draw different colors over here. So when I do this clear, it clears the draw buffer to some different values. So there, I set it to red, can set it back to black. I can set it to like green or blue, whatever. Or I can even make it flash. So slide up the lepsi warning here. Can do something like that. But I'm not going to do that anymore because that's like kind of annoying. So, right, so that's the basic idea. So now I can draw stuff within this. So in WebGL, you basically have an API for drawing triangles. So I'm going to show you how to draw a triangle. And this will at first seem kind of complicated because drawing a triangle, there's a lot of different pieces to it. But once you can draw one triangle, it's actually not that much more work to draw millions of triangles and some crazy configuration. So what I'm going to write here first is a fragment shader. So what this does is this tells WebGL what color to make each pixel on the triangle. And I'm just going to start it by making them all white. I can change that later. Similarly, I have to give it a vertex shader. And this tells WebGL where to put each of the vertices of the triangle on the screen. And this needs to take in some data. So I'm going to go take in the position of each vertex in 2D right now just to get started. And then I'll just go pass through this to the output position variable, that. And now that'll just put it wherever the input comes from. So I now have a vertex program that's going to run on the GPU in a fragment program. And now I need to give it some data to run in the vertex shader. So I have to give it that position attribute. And then I will just give it these two coordinates. So in WebGL, the coordinate system works where the lower left corner of the screen is negative 1 plus 1. And then the upper left corner is negative 1, negative 1, and the lower right corner is positive 1, positive 1. So this will draw a triangle. I have to tell it how many vertices. And now I can call this function, draw a triangle. And I got a triangle. There we go, right? Great. Let's turn off the blue. Now we can make it move around. So we can do something like this. So in WebGL, you have these things called uniform variables, which you can set to different values. And then they basically get broadcast into the shader. So these are like a global variable that you set outside WebGL, and then they go into WebGL. And so I have this new variable, translate, which is a 2D vector. I'm just going to add that to there. And at the moment, it's 0, so it does nothing. If I make this value 0.5, it shifts it over there. Negative 0.5 shifts it over there. And I can even do something like this. I can actually read in the current tick. And I can make this a function. And I can just have this do, say, like math.cosign of tick, and now it will oscillate back and forth. But of course, the ticks go really fast. So I need to make it a little bit slower than that. It's still kind of fast. We'll make it like that. Now it's OK, like maybe a more moderate thing. Now, this is cool if I just want to have it go into some fixed loop back and forth, but I can also take in data from other sources, like, for example, the mouse. So I have a little module here called mouse change that just listens for mouse changes. It just gives you some state info about that. And then now what I can do is I can call this function here with the data from the mouse. So I'm going to actually give it this vector. We'll call it translate here. And we'll take in the x component of the mouse and the y component of the mouse. Of course, these numbers are going to be very large, because they're in pixel coordinates. So I'm just going to eyeball and divide by 1,000. It could do something more precise. I could show that in a moment. But now here's what I'll do. So rather than reading a function there, I'm going to read in this prop that I'm passing in, which I call translate. Oops. And now when I move my mouse around, it should translate this thing, roughly, although the y is flipped. So I'll go switch that around. Now moving my mouse kind of drives the triangle around. And so what's happening here is basically it's reading the state of the mouse from this module. I get these variables in. This is just some arbitrary blob of state somewhere. And then I'm passing this into the WebGL context through this uniform variable called translate. And then that's popping out on the screen and giving me this moving triangle. So OK, so that's kind of cool. So I'm going to skip a few steps in the interest of time. And we're going to do something in 3D now. So I'm going to need a camera. So this is basically a thing that tells WebGL. This is basically a module that handles all of the details around creating a 3D viewport for basically doing the matrix math and everything around that. So I have this camera. And I'm going to go read in a test object, which is going to be a model of a bunny. And instead of calling this thing drawTriangle, I'm going to switch it to call it drawMesh. So we're going to draw a mesh. And actually, let's just call this bunny thing. Let's just call this mesh. That'll be maybe a little more descriptive. So the attribute data, I now want to draw the stuff from the mesh. So I'm going to do mesh.Positions. I'm not going to use that anymore. And then I need to draw all the faces. So the elements here is a special keyword in regal that basically reads the cells. So let me go back up a little bit. So what is this mesh that I just loaded? So I'm going to pop open a little node window here. Make this really big. And I'll just say, mesh equals requireBunny. I mean, all this stuff also works in node, by the way. There's nothing special about JavaScript. So if I look at the mesh.Positions, this is basically an array of tuples of points. And if I look at the mesh.Cells, this is basically the triangles of the mesh. So this is an array of indexes of those things. So this is just like some data. That's it. There's nothing fancy going on there. I mean, it's just plain old data. So now what I'm going to do is I'm going to use that camera module. I'm going to call drawMesh. At the moment, it's just going to do nothing interesting because I'm still using this uniform variable here. But I will now say, UniformMetForProjection in view. And these are basically the state of the camera. And I will set the position value to basically be the projection matrix times the view matrix times the position variable from the mesh, or the position attribute. So if I do this, and assuming I didn't make a mistake, which I guess I did. So I have a syntax error here. Oops. Wrong file. But line 22. Wait. Hold on. Here we go. All right, let's load this. Wait, where did I do the typo? Attribute position. Vec3, Vec4, where did I put that? What? Where, right here? Vec4? No, it should be a Vec3 position, not Vec4. Oops. Which line did I do this wrong on? All right, let me. Hold on, let me comment this out. Oh, I know why. All right, OK, it should be this. Let's try that. There we go. Phew, OK, that was kind of weird. The reason it gave me a bad error is because I was running this babelify thing. OK, quick recovery on that one. I'm still jet lagged. It's like six hours time delay or whatever, so it's kind of a long time. So OK, I got this opaque looking thing. It's not that interesting. So let's color it. So I'm going to take this module here called angle normals. And we're going to go compute the normals, the mesh.cells, mesh.positions. And if we do that, we now have this normal value, which isn't doing anything. So we have to actually pass it through to the fragment shader. So we do that using varying variable. So we have this varying variable, which we'll call color. And we'll set the color equal to the normal vector. So we'll do 0.5 times 1 plus the normal. And now we have this varying color. So this is just a three tuple for the RGB components of the color vector. We put that out. Assuming it didn't typo anything. Yeah, now we got a little sort of shaded bouncy bunny thing here or whatever. And we can do some stuff, too. So we got this normal value. We can make the bunny get fatter. We can do 0.1 times the normal. Blah. Now it's like a fatter rabbit. We can make it skinnier. We can subtract normal from this thing. It could even skinnier than that. Let's make it. Now it's really kind of messed up looking and deformed. And we could even do something like this. We could take a uniform float t. And we'll just multiply t times the normal. And we'll set t to be, I don't know, say, math.cosine times tick, 0.1 times tick. Let's see what happens with that. Whoa. It's now kind of oscillating, doing some crazy stuff. And we could even do something like this. We can do, say, cosine of position dot y plus t. And this will now do something even weirder. Make it kind of go in some strange oscillatory pattern. I don't even know what's going on anymore. Whatever. OK, so you get the idea, right? So this is just the basic stuff. So now I got a bunny over there. But all right, that's fine and all well and good. Let's do something with some real data, right? So OK, I got some stuff here. And I got some time. So I think this should be fine. So we're going to create this. We're going to abstract this a little bit. We're going to create a function called process mesh. And instead of taking the bunny as input, we're just going to take an arbitrary mesh. And we're going to basically read this mesh as some data. And then we're going to return the output of that. So this is just kind of like refactoring the code a little bit here. So now the mesh comes as an input here. And then we basically compute all this stuff. And then we return some resulting function, which we can then call as a command. So we're going to say this process mesh bunny. And this should still work, right? So I didn't really change anything. All I did was refactor it. So let's make sure I then break something there. Oh, two S's. All right, forward. Oh, bunny is not defined. I called it mesh. All right, there we go. All right, so I'm still doing the same thing as before. Nothing has changed. So let's now actually load something. So I'm going to use this module ND array, which is basically this module for working with multi-dimensional arrays in JavaScript. And I'm now going to load some data. So what I'll do is I'm going to use this other module here called resl, which is short for resource loader. This sort of works well with Regal. And I have to give it some resources. So this is going to basically do an XML HTTP request to go pull in some data. So I only have five minutes. I'm just going to do this really fast. Binary. So this is basically the data I'm going to work with here. This is some neuron data from a mouse brain. This was scanned at the Howard Hughes Medical Institute using a microscope that shoots photons in them or something. I don't know. The mice have these little things that go on top of their brains and you can do it while they're alive. I saw it. It was twisted. All right. And I think the size of this thing. So I'm basically going to load in a 3D volume set of data. So I'm going to make a 3D volume viewer. They charge $15,000 for this stuff. And I'm just going to do it right here, the same thing more or less. So now I've got this neuron data loaded. I'm going to go copy that thing and put it in this onDone loop. And now what I've got to do is I need to extract an isosurface from this. I'll just use another module here. So this is a module I wrote called Surface Nets. It basically extracts a level set from an nd array. This works in 2D and 3D. And it also works in 4D. I haven't tested it in anything higher than 5, though. So I don't know if it works in those dimensions. I wouldn't recommend that. So I'm going to call process mesh. Well, first I need to get the mesh, right? So I'm going to go basically say the mesh is going to be Surface Nets. And I'll take neurons. And I'm going to use the level set 200. I think that one looks pretty good. I could adjust it later. And then what I'm going to do is basically update the positions of the mesh. So I'm going to do positions.4 each. I'm just going to mutate this in place, because whatever. So I'll need another module here. 3 equals require glvec3. All right. And then we'll do vec3 divide pvp neurons dot shape. This is just like in a sci-pi where you have a shape variable. And then we're going to pass that mesh in. And so assuming everything is good, mesh, there we go. All right. And then let's see. OK, yeah. Well, wait. I still have this weird oscillating thing going on. I've got to turn that off. All right. Let's turn. Let's just draw the thing normally. We're not going to do any of this other weird stuff. OK, cool. Yeah, so we got some neurons. But whoa, something's weird. I got to do one other thing here. Because these neurons are actually very large, we have to use 32-bit indices. So I have to do this. This is basically a little feature in WebGL that allows you to use 32-bit indexes for triangles instead of just 16-bit, which is what you get by default. But yeah, now here we go. It worked. So cool. We got a little chunk of neuron data. It's flipped around. That's fine. We can fix that. Here's where we load the data. So we're just going to transpose the first two axes. And now it should do its thing. Yeah, OK. Now there we go. And you can see the different neurons and stuff in there. I don't know. I think those are neurons. I'm pretty sure this is from a mouse. So yeah, and we can do other stuff. We could modify the level set and do some other things. But I've already kind of gone on long enough with this little demo. So I'm just going to go wrap this up really quick. So OK. So right. So that's Regal. The whole point of this thing is I just did this live coding demo here to show you that you can do things very quickly with it. But really, this is meant for longer-lived projects. So you start out with a lot of low-level control. You can build things up from primitives like shaders and vertex buffers and whatever geometry format you want to use. And it has clear, well-specified APIs. And I should also point out that even though I have been living in a lava flow for the last couple of months, like six or so, and there have been a lot of people hammering on this thing and using it in production, commercial applications, in this entire time there has not been a single substantial bug report. There have been patches about the docs and typos over there. But no one has actually found a substantial bug in the code itself, which is good. So I consider this to be sort of a success. So that's great. There's also tools like HeadlessGL, which have been developed in partnership with things like Uber and Mapbox, they've supported some of this. But this allows you to actually run unit tests for Regal and code that you write in Regal in a CI environment, which is great. And also there's a ton of improvements in performance. So I didn't get into the details of this, but the way Regal works under the hood is that when you do have one of these commands that you create, it basically just in time compiles a little blob of code. So if I set a breakpoint right here and I step into this, you'll see, hold on one moment, we get to this. This code here, and this code is actually generated at runtime. And what this does is it actually applies a very minimal diff of this sort of virtual WebGL state from the state that you constructed right there. And so the actual operation that you get with this is very close to zero overhead. And this has been benchmarked. So it's basically on the order of microseconds, the difference between using Regal and writing hand tuned WebGL code. So it's fast. There's also code for profiling, and we do a bunch of different benchmarks, so it's pretty stable. There's a Gitter chat room, which is pretty active. Then there's also a website with a bunch of demos. And I'll conclude by saying that here are some people that were very instrumental in getting this project off the ground. Jeremy Freeman, who definitely helped with financial and moral support and guidance. And also Irkeman, who just came in from nowhere in Sweden, and then did a bunch of work getting a lot of demos together. Thanks to Boku for putting this event together. And then also Ricky Roycer, Greg Tatum, Realizdat, Substack, MK30, all my other friends in the cooperative. And with that, I think I'm basically done.