 Is that good enough? So it's our pleasure, our big pleasure to introduce David Gotch as to one of these numerical wizards who's gonna tell you a little bit about process languages on the Wharf Hydro-National Water Model. Great, thanks, Jay. Appreciate the offer to come and talk about the Wharf Hydro System and its implementation as the National Water Model. Some folks who've been involved in sort of more of the forecasting community as opposed to just sort of the basic process research community may have heard about this National Water Model. So I'm gonna spend a little bit time just basically framing why the National Water Model was built. It is an operational surface water prediction system that the National Weather Service runs right now and you can get forecast 24-7, 365 with it. But from a scientific perspective, there's still a lot of holes to poke at. And we're gonna poke at a few of those in this talk as a means of sort of stimulating some interaction and hopefully some collaboration because this is a brand new effort that just went into operations last summer. And the idea, did they get that? The idea with it was that it's not just sort of a NOAA thing. It is sort of built on a community modeling set of infrastructure, which is this Wharf Hydro System where Hydros had interactions with CSDMS and the NASA land information system and a bunch of other tools as well. And so really trying to find some pathways to engage the broader community and CSDMS does a really good job at that. So the motivation for the National Water Models is really NOAA trying to address a host of environmental prediction problems. You could probably drop a list of these on your own of what are some of the critical needs for water information, operational water information across the country now. What's listed here is sort of a long-term vision where NOAA wants to move its water forecasting capabilities. So it starts with this basic things that they do a lot of or even we're doing before the National Water Model came online, sort of flood predictions for the protection of lives and property. But they're gonna be moving forward with other partner agencies in doing more environmental predictions to things like pollution, drought and long-term water resources, forecasting and then aspects leading into ecosystem sustainability and biodiversity. And of course, all of these topics cross a number of different spatial scales and temporal scales and they're multidisciplinary in and of themselves. And where NOAA was, and say before the National Water Model concept was started to be incubated about three or four years ago, four or five years ago, their tools weren't really aimed at providing information like this. They were sort of a more traditional set of engineering forecasting tools designed and heavily calibrated to produce a stream flow forecast at a point, but not necessarily a more holistic representation of hydrology or of surface water hydrology anyway. And so the water model is an attempt to move in that direction, kind of embody more of an earth systems modeling approach. So embedded within that, of course, is these ideas. I'm not sure why that keeps cutting it out. Of moving across scales. So the National Weather Service does weather prediction. They do climate prediction. They're constantly moving information across scales, particularly in the atmosphere. They run global numerical weather prediction models. Those go through different sets of either dynamical or statistical downscaling tools to try to provide impact-based forecasts, which ultimately reach down to the street level. At least that's the vision or that's the intent. And increasingly, instead of doing that with what I would say are some ad hoc methods, they're trying to use more physics-based methods. Sometimes in fully two-way coupled physics-based modeling systems, sometimes in sort of a one-way cascade of models and model outputs. But you move down to regional scales. You move down to watershed scales and ultimately down to street scales. And that's where the water model is gonna be trying to head over the next several years of development. So as I mentioned, we went into operations with NOAA with the Weather Service last August. And these were a number of the different guiding principles or goals of that. It wasn't to tackle the whole, we're gonna go do biodiversity prediction in year one. That's sort of the long-term set of goals. These were the kinds of things that they were looking at. So trying to provide operational streamflow guidance in areas that previously had no forecasts. So there's about 4,000 or so forecast points around the country that the river forecast centers were making forecasts for, largely at or below, say reservoirs or river locks, levee systems, things like that. And now they're trying to move way beyond that to reach up into the entire channel network for the nation. Also trying to provide hydrologic information on a lot of the other surface water states that we encounter, snow packs, soil moisture, evapotranspiration, and of course, flood inundation is really one of the biggest high impact forecast variables that they were after. They wanted to seamlessly interface these new water forecast products with what they call geospatial intelligence, or basically doing the intersection between water information and critical infrastructure. So bringing things together into sort of a GIS framework for enhanced decision making. Their older models that just gave you a point forecast didn't really allow them to do that. You got the forecast at that point, you didn't have a lot of other spatially distributed information. And then finally, that last bullet is, moving towards an earth system modeling capability. So this was the goals of version one. The schematics you see here, just sort of some of the boilerplate resources that are now available. There's a webpage that Noah hosts for the national water model. There's a variety of different outputs that I'm gonna highlight in a little bit that come out of the national water model in addition to just stream flow. As I mentioned, we went into operations in August of 2016. And since then we've done two version upgrades. We've got three versions of the model which happened within a year, which is pretty tiring. But we're sort of getting over the hump of that. And along with that is a lot of verification work, which I won't have time to go into today, but we're slowly starting to push that verification work out into some online resources and then some webinars that have been put together. So technical specs on the water model itself, it ingests about four and a half terabytes of data a day, which is largely coming from the national radar mosaic and numerical weather prediction models. We output about three terabytes of data a day. We utilize the USGS NHD plus channel hydrography or stream flow network. So it's got a representation of about 2.7 million channel reaches for the nation. And in the first versions, we had about 1,260 reservoir objects that were identified within that. There is no active management of those. So it's pretty simplistic at this point in time. There's around 360 million computational elements, about 75,000 lines of code. And it uses in operations with the various forecast cycles about 100,000 CPU hours a day. So from a hydrologic prediction perspective, this is a pretty big computational problem, particularly compared to the prior generations of operational forecast systems. But in terms of our system models, climate models, weather models, this is not too big. And even that level of sort of parallelism and CPU usage and data throughput is not really fully taxing the capabilities that we have. And so if any of you sat in on Reed Maxwell's talk, you'll see, you know, there's another generation of hydrologic models, which are really sort of pushing the envelope in computational hydrology. The modeling workflow looks largely like this. There is in the top left-hand corner there, there's a whole component, which stands independent of the physics of the model, which takes and pre-processes meteorological data. We call it a meteorological force engine. It does downscaling. It does bias correction and re-gritting units conversions to get all the meteorological data onto a common grid and into a common format for ingest into the model. And then the key model components that we'll focus on here are these other ones. And this is where sort of this multi-scale aspect comes in, which is sort of the title of the talk. One is that we drive from that weather information a single column model, basically a distributed column of land surface physics, which handles the exchange of energy and moisture to and from the atmosphere. It's a full energy balance model, mass energy conserving. And that is set up on a one kilometer grid. And so that's just vertical process representation there. That's where the snowpack, the canopy processes, the vertical soil, water flow is. And then there's two way coupling of that with overland flow and saturated subsurface flow routing components, which happen on a much finer grid. Now in this particular instance for the water model, that happens on a 250 meter grid. I know many folks in this room would say, wow, that's really coarse actually. We're doing the whole country at 250 meters. So it's a big challenge in that sense, but we're still sacrificing a lot of resolution, particularly in some of these very complicated areas where inundation dynamics can have a very big impact over very small vertical ranges. So it's just a shortcoming of the current version of the model. There's plans to keep marching down in resolution, but that's where we started with this version. So that two way coupling exists between the column model and the routing physics. And then we actually had a requirement from the weather service, which is to do all of the channel routing on this NHD plus network, this vector network exactly. So we have to go from a grid to a basically a river vector transformation. And so we utilize this set of catchments which are associated with those river vector network elements to handle basically the remapping of certain hydrologic states and fluxes from the routing components into the channel components. And then once we're in the channels, we can route down the channel network and go through reservoir objects. So there's a couple of different spatial transformations here. I'm gonna highlight a little bit more as we move forward. One last thing I'll say about just the sort of the implementation of the water model itself and operations is that there's four different configurations that are sort of cycling all the time. There is an analysis cycle which is generating a set of initial conditions for every forecast cycle. And that analysis cycle runs from about three hours in the past up to the present and is assimilating in real-time streamflow data from the USGS real-time streamflow network. We drive each of these configurations with a number of different weather models. I'm not gonna have the time to go into those today, but I will say that there is a short range forecast model which uses sort of the latest greatest very high resolution convective weather forecasting model that the weather service runs. That goes out to 18 hours. There's a new forecast out to 18 hours. It's updated every hour. There's a medium range forecast which cycles only four times a day and that runs out to 10 days. So that kind of helps us with sort of larger river system forecasting. Big flood events like what just happened in the Midwest a few weeks ago. We were really looking at the medium range forecast all the atmospheric river events that happened in California this winter. We looked a lot at these medium range forecasts. In the summertime, we look mostly at those short range forecasts looking for flash flood events. And then there's also kind of a stripped down and simplified long range configuration where we don't do the high resolution terrain routing. We're doing sort of a macro scale hydrologic modeling approach here. And that's a true ensemble model which has 16 members a day and runs out to 30 days. And that's more for water supply forecast. So here's a closer look at the physics in a sort of a schematic sense. We've got the column land surface model which is called this NOAA MP, NOAA Multiphysics Land Surface Model that comes from the atmospheric science land surface modeling community. That interfaces with an overland flow, a gridded overland flow methodology which uses sort of a steepest descent of diffusive wave equations associated with it. There's a lateral saturated subsurface flow module that's hooked in on that high resolution grid as well to do basically shallow VATOS zone saturated transport. This is not a true groundwater model. We don't really have aquifer process representation. It's really just in a soil column in the VATOS zone. Because we don't have a groundwater model we need a conceptualization for base flow processes sort of the long term memory processes. So in this current generation there is a fairly conceptual base flow scheme that is hooked into that. And then those overland flow lateral saturated subsurface flow and the space flow scheme provide the inflows into the channel network which happened on this NHDPlus network. And there you can see, this is for the lower Mississippi River Valley in that diagram. You can see the little dots in there like the orange and the pink dots. Those are the traditional forecast points from the weather service. That's basically the resolution of forecast you would get spatially just at those isolated points. And then the blue line network associated in there is what the new water model is giving you. Just in terms of coverage of forecast service. And then of course there's these basic reservoir objects that exist in the model. So focusing in on the spatial transformations and this is where I think CSDMS has also had to address a set of certain set of issues or a similar set of issues. Right now in the current generation we use this straight rectilinear regridding process. So it's pretty rigid and it's pretty, you have to define everything pretty rigidly in advance in order to do this. And it's just basically a straight decomposition of some of the runoff fluxes from this column land surface model to pass them over to the routing components. We've been working with the ESMF versus the modeling framework folks who've developed a set of generalized or generic regridding tools so that we can move across different spatial elements here. One of the things that many of you folks would know... Sorry about that, I'm not sure why. We waste a lot of computational time having all these fine grids everywhere. And so moving to some unstructured grid frameworks is something we're working with now to try and gain some computational efficiency. But that requires even more generalized regridding frameworks. It looks something like this. If you map it out, this is essentially the grids overlaying on Springfield, Missouri. And then the blue line network cuts through that so you can see the various resolutions of the different grid elements as they map onto this blue line flow network in this area. Now, in the NHDPlus dataset, there is a unique catchment that's identified with each one of those blue line elements. And that's how we get the mapping from those grids into one of those channel elements. And those catchments and those river channel elements, the river reaches have a common, unique identifier associated with them. And that then provides the backbone of this information service for which all the screen flow information now starts to be served on. The only other thing that's in there is that we do have these reservoir objects. This is Hetch Hetchy Reservoir, just a snapshot from this morning's set of forecasts. The gridded field we see over lying there is the one kilometer analysis from the National Water Model of snow water equivalent that's left in there. And so you can see at this scale, things start to look a little bit crude compared to what they would look like in reality. But you get an idea of what a reservoir object is. We have to topologically link that in with this channel network and then have the ways to map those land surface model states and fluxes into this catchment framework to pass fluxes into the channel network. So that's the bare essentials of what the spatial transformations are. Didn't throw up any equations just because of time, but all of that stuff has to be worked out in advance as inputs into the model to go through this dynamic spatial transformation or re-gridding process on the fly. So here's a summary then of sort of what's in operation right now. You can see the full model domain exists up there. We do the land surface model grid and the routing grid over that entire domain. We see the sort of rainbow catchments highlighted here is where this NHDPlus dataset is actually defined, the channel network is defined. And so those are the areas where we're doing stream flow. And those are basically the areas that are tributary to all of the river systems within the US. And the version that we'll go into operations next year, we've added the Great Lakes tributary basin on the Canadian side as well. So we'll be doing Great Lakes tributary flows starting in 2018. Some of the other datasets that are used, we use the USGS national land cover dataset in the 2011 version. The one kilometer status goes soils data, which I'm sure in this group has caused some people some nausea, but it is unified for the conus. And that's one of the things that we had to rely on right now, but certainly open to a lot of experimentation improvement associated with soils classification there. There are a couple of different vegetation datasets that we use, time evolving climatological datasets. We'll be moving to some more dynamic updating datasets as new versions come online. Most of these other things I've mentioned already in the past. The only other thing I'll say is that the channel routing as we're doing it right now is fairly simple. It is a hydrologic-based methodology, which uses this Musconium-Cons-based methodology, quite old. But for the first implementation of the model, we need something that was fast and stable. And so this was amenable. It had been sort of demonstrated to sort of be very scalable across the country. A couple, we'll use about 512 processors in operation for each model job. So computationally, it was efficient to use and again, very numerically stable. Obviously, there's a lot of room for improvement with more modern hydraulic-based methodologies for channel routing. Okay, so what do some of these products look like that come out of the model? This is an animation now of one of our retrospective simulations that was done in preparation for the original version of the model. And you can really start to see a lot of the sort of interesting hydrologic and hydrometeorological behaviors expressing across the country. This is actually 2015. We're getting into the end of May here. It's a very wet period here in southeast Texas. And in Louisiana, Houston floods happened right over the Memorial Day weekend of that year. So you'll see these very fast-moving patterns of rainfall moving across the country. They kind of light up the small river channels as they respond proportionately to the local rains. And then behind those, you see sort of the lagging of these flood waves moving down the bigger river systems. The Colorado River there you see sort of draining out from the spring snow melt signal, same with parts of the upper Columbia as well. So this was one of the first times of sort of seeing this national hydrography in sort of a dynamical mode responding to real weather events. And, you know, this is actually what's running in operation now. From any of those reaches across the country, you can go in, you can pull out forecasts at all those different time scales that I mentioned. There's a short range and medium range and a long range forecast. Start to overlay forecasts together and try and get some probabilistic information on-screen flow from these. A lot of work going on now related to validating these flows. You see a lot of variants in all of these forecasts. These are coming from weather models which have a lot of variants from the precipitation drivers in and of themselves. Down in the lower left-hand corner, that's just an example of what's produced from the long range forecast. It's more of a water supply forecast. We're not really deterministically saying what's gonna happen on day 27 of a forecast. We're saying this type of application is useful when we look at the accumulated flow processes over this large period of time, over like a snow melt season or potentially over a drought period and looking at what the water resource might be from that. I'll just show a few more examples here and then start to wrap up. This is just an example of a forecast that was made during this past winter, January 3rd through 7th. It was an atmospheric river event in California. So we were looking at this area near Merced, near Fresno, California on the Merced River. And this forecast started to pick up this event about five to six days in advance. You see the red and blue dotted lines on the bottom were about the seven and eight day in advance forecast and they weren't really tagging this event. But then the forecast model sort of came into consensus and everything started grouping around what ultimately became the observed flow, which is the solid yellow line in there, within about five to six days in advance of the event. But there's still a lot of uncertainty associated with the peak flow amount that came in with this and the timing. The model does tend to be a little fast in terms of propagating a lot of these events downstream and that's likely meeting some more calibration of the Muskegum-Cunn's method. But this is an example then of the kinds of forecast products that could come out of this. We're starting to track snowmelt throughout the Rocky Mountains. This is from about a week ago. This is a medium range forecast and we start to see things like these concatenated diurnal cycles of snowmelt patterns that are occurring. This is the Animas River at Durango and we have a high bias in this particular case at this particular forecast period. The model in the version that it was running then tended to be a little bit early on at snowmelt which is contributing to this bias. So this is something we've been working on and tracking moving forward. This is just another case study event for some of this recent Midwest flooding. This is just showing an example nine days before this event here in northeast Oklahoma. The dotted line is the observed. There was one forecast cycle that was starting to pick up this event. Now I'm just gonna step through and look at a few more. This is three days before the event. We're starting to get good forecast consensus around what ultimately became the observed event. There was one forecast cycle which still didn't validate well. So you get again a lot of this variance of forecast to forecast. And then the day of the event, there were a number of different events that forecast cycles which didn't produce a ton of rainfall although some of them did. And again provided a level of probabilistic guidance at this particular site. So again, you could see there's some dotted lines here. Those are the National Weather Service defined flood stages. Regardless of the forecast cycle that was used here, pretty much all of these ended up being in a major flood category. And of course this was an area that saw a lot of inundation. There's a number of other products that I'm gonna zip through really well really quickly here to wrap up. A lot of folks who run hydrologic and river routing models have their own operations like the river operations coated in. They may not be that interested in the river flows that come out of the model but they might be interested in the channel inflows coming off the landscape. How much snow melt driven runoff or other types of runoff are being generated as different kinds of scenarios they could use. And so one of the outputs from the water model is basically just what water is flowing into this channel network all along those 2.7 million river reaches. And so this is just a snapshot at one point in time of where water was flowing into the network and some of our pre-operational runs. Another thing that would potentially be interesting to this community, which is it's kind of interesting to look at, we don't have much confidence in it at this point in time. This is an aerial shot of the Grand Canyon and if you add the channel elements in there you see some interesting features. Some of those map up really well with the dry river beds in that area and some of them don't map up well at all. They kind of head straight up a canyon wall in certain cases. So that NHDP plus data set is far from perfect. Is one thing that's noticed on this but the other thing is actually plotted here in terms of color is the river velocity. So as we go through the channel routing calculation we can back out a velocity term from that. So we go ahead and serve this. We don't have much of any faith in this in the absolute sense, but in a relative sense it's actually kind of interesting to look at in terms of how velocity would change across different stretches of rivers and really as it's different events propagate through the system and where river velocity start to pick up very quickly. I'm sure this community can think of a host of applications to things like river velocity, habitat and erosion and even recreational applications but I just wanted to mention that that was one of the interesting outputs from the model and it certainly, I don't know, there's gonna be some interesting times looking and validating some of that stuff. A lot of other variables. There's precipitation products, there is soil moisture products which are coming out of this. There's sort of this depth to saturation and a ponded water value. How much water is starting to pond up on the surface in response to the overland flow calculations as well. These are all snapshots from that Midwestern flooding that happened a few weeks ago with flash flood watches and flash flood warning polygons overlaying on that. So this information is just now starting to make it into the forecast office for this. Are we about done, Jim? One minute? Okay. All right, so snowpack evapotranspiration. That's just another high level map of the shallow water table depth. You can zoom in and look at a number of different pieces. One more thing that I wanted to show at the very end and this is sort of a new capability and it's where the weather service is looking at sort of an on-demand computing to put in nests at very high resolution for specific events. And it's called this hyper resolution nest development. And this is just an example of one of those that we set up for Hurricane Matthew of last year. This is the model implemented just over this limited domain at 30 meters, turning the channel routing off and basically resolving the flow processes here as just an overland flow, coupled surface and overland, couple surface, subsurface, overland flow process for this. You can see some interesting features and the time evolution of inundation and some of the barriers to flow like some of the roadways that exist and actually present barriers in the DMs. So with that, I'm done and I'd be happy to take a question. Thanks. Can you just pop back a slide to the R package that you went through? I'm just curious what that was. Okay. I didn't get a chance to do that. So one of the things we had to do is there really wasn't, when we started this, at least the weather service didn't have one. We didn't have one. We didn't know of one at that time. This is two, three years ago, sort of a comprehensive package for doing hydrologic model evaluation. So we built this package. It's called our work hydro because it was originally designed to link up with this work hydro system. But it's actually pretty generic. I mean, screen flow time series or screen flow time series. You get a whole different model and put this together. So right now with the water model, we're doing evaluations with this R package for screen flow, snowpack, soil, moisture, ET and inundation as well. So we're trying to build out this package. And there's a lot of other people who are starting to use this for different applications, but it's available online and GitHub and trying like everybody, improve documentation. Is there another question? Is the gridded model input available? The gridded, like the foreseeing data? Yeah. All of the data that goes into and out of the model is available on NSAP, the National Center for Environmental Prediction, their NOMAD data server. The National Water Model web pages has the HTTPS and FPP link to that data. So all of the foreseeing and all of the outputs that are coming out of the model are there. The operational outputs are stripped down a bit compared to what the model could output, right? And we don't output every model state, stem temperature and every component of it. So that's the main hydrologic variable, but the eight meteorological forcing terms are there. Are you able to say what it would take to put in, let's say your diffusive wave into a kinematic wave in your hydrological model? Are you missing channel dimensions? I mean, what's your holdup? Yeah, into the channel routing scheme, that's a great point. Probably the biggest source of uncertainty we have in terms of going to a more hydraulic based methodology is bank holdup. And this community does produce estimates of that. So it'd be really interesting to talk to people about coming up with spatially distributed estimates of those that are scalable to the whole country, right? And all the different variety of channel structures that we take. That and roughness is also a good one. Channel geometry, sometimes we can get at in other ways, but yeah, all of those are core parameters that would really help us move beyond this more complicated method. Yeah, and but slot flies in 2021? Yeah. Let's give David a big thanks. Let's see. Okay, so my name is Brian Walsh.