 Okay, I'm going to talk about a Geography problem that I started thinking about recently and the problem is this Given any particular spot on the ground like this yellow dot if it rains there. Where does the water go? so In particular, what is downstream from that spot and then also conversely? What's upstream from that spot? So all the places where if it rains there the water is going to go flow past that yellow dot So I started thinking about this recently and I'm not a Geologist or a hydrologist actually a mathematician But I looked into it and found out that You can get a data set from the US Geological Survey that describes this and so I wrote kind of a fun graphics application to visualize the results so let me Want to switch real quick and give a demo of the application and then talk a little bit about what went into creating it So let's see Yeah, so here's the here's the program. This is online at this address. I'm running this locally at the moment But you can check it out if you want so what's happening is as I move the cursor around The program is drawing in blue everything that's downstream from where the mouse is And It's drawing in red everything that's upstream And the maps interactive so you can pan and zoom around like in Any web map that you'd expect and it's it's interesting to look at some particular areas here like let's zoom in on New Orleans and highlight the Mississippi River near downtown and then zoom out and you can see all the area that Drains to New Orleans and Another area that's cool to focus on is Chicago If you zoom in look how close the Mississippi watershed gets to downtown Chicago. In fact It actually overlap. I mean how close it gets to like Michigan. It actually overlaps a little bit there But that didn't used to be the case Some canals were put in in the 1800s to connect the Des Plaines River, which is this river right here with Lake Michigan and that's what allowed ships to transfer between the Mississippi River and the Great Lakes and That gave rise to Chicago as an important transportation hub in the 1800s Another area that's kind of fun to to focus on is You go out here to Montana and So this is the starting with the Mississippi way over here and then coming up on the Missouri River It's to see how far you can get over to the west before jumping across the Continental Divide To get to the Columbia River watershed to go to the West Coast and that's in fact exactly what Lewis and Clark were trying to do in their famous expedition 200 years ago or so Another area that's fun to focus on is Nevada and part of Utah Look at this. It has the property that in a lot of cases the downstream flow never makes it to an ocean This is what's called a great basin and it's known for being particularly dry and and having this hydrologic property There are a few streams and creeks there But they just peter out all the rain that falls either evaporates or soaks in the ground or flows into lakes They have no outlet like Salt Lake and I think that's why it's salt so anyway, so let me say a little bit about What I learned about surface water hydrology in order to create this program Let's see So like I said, the data comes from the US Geological Survey There's a database called the watershed boundary data set and it works like this It divides the country up into regions. It's a nested hierarchy of regions and I'm going to zoom in to the smallest level the one that's the greatest level of detail and Look at this one particular example So I'm going to focus on that particular region and then switch temporarily to a 3d view that I made with Google Earth So that we can see a little bit more about the shape of these regions So as the camera moves around here, you can see that The boundaries of these regions more or less follow local ridge lines These regions are about five or ten miles across at this level And the bound their boundaries are very carefully chosen so that all the water that leaves each of those regions does so at one point and In just a second that polygons doing a turn red. Yeah, so the red polygon You might be able to make out there's a stream in the middle of that that flows up This one flows north and enters the yellow polygon at that one point So back in the map There's that red polygon and it flows into the yellow one at that one point and This other red one also flows into the yellow one And the yellow one flows into that blue one and that blue one flows into that blue one and then so on This gives a relationship between these polygons. It's a mini to one relationship So every polygon has one that's downstream from it and possibly several that it is downstream from And so that means that you can think of these polygons is acting like nodes in a graph And there's edges directed edges between them that describes the flow And so that turns the whole map basically into a gigantic directed acyclic graph Which means that the original problem of figuring out what's upstream or downstream just translates into graph traversal Which is kind of a well-known problem and a lot of fun to code So once I realized that I said yay, I know how to do this, but it turns out they were There were some challenges with the data first of all the data set you download this from the USGS It's two gigabytes. It's way too big to do anything with interactively and I wanted to make kind of a fun web app Like I showed you and so I had to figure out something about that and part of the problem You can see if I zoom in on just one polygon Part of the problem is that there's so many vertices in the boundary thousands or hundreds at least and so that can be remedied because This is a common thing in GIS and there's a lot of good open-source GIS software that will simplify a polygon and Replace it with another one by dropping vertices in a way that retains most of the original shape So you can cut down the size of the data set But remember there's a there's a bunch of these polygons There's about a hundred thousand and they're all right next to each other they share borders And if you simplify them all separately You end up with regions like this where there's little gaps and overlaps and So the reason for that is that in the original data set each polygon is stored separately And so what I ended up doing is to translate the data into a format That the stores the shared border once rather than separately with each polygon And there's a format called topo JSON that was invented by the author of D3JS that does exactly that and so by converting the data to topo JSON and then simplifying I got a good clean set of polygons that fit together well and Got the size of the data down to about 40 megabytes And and then I decided to use leaflet JS mapping library to make the map Leaflet's a great library. I like it a lot But there's an issue in that the way it draws polygons by default is it uses SVG which creates a node in the DOM for every polygon and This is the Mississippi watershed here and it's got about 30,000 polygons in it And so the first time I ran this the browser just crashed and So I had to figure out a different way to do that So the DOM manipulation is slow that didn't work Turns out that you can use leaflet with HTML5 canvas rendering also though and so by switching to canvas I actually got it to work and it worked reasonably well But it was still a little bit slow and so I made one more optimization and that was to notice that When the program is drawing these big red upstream areas, they actually consist of a bunch of little polygons But I didn't really care about drawing the individual polygons all I really cared about was drawing the one red blob and So by doing a little bit of computation in advance to merge All the polygons in each each of these red blobs into one and just store the perimeter It cut down the rendering time from being linear in the number of polygons to square root which for Big regions made a huge difference and so that's what allows the program to be as snappy as it is and made me really happy So I also made a mobile version just for fun that uses some slightly different optimizations But most of the same ideas and so it's online and if you want to check it out and talk to me about it I'd be thrilled There's my contact info. So that's it. Thanks