 I will open the floor then for a conversation about model uncertainty. So I'd love to hear from folks out there questions for the three speakers or comments in general. Thanks very much to the panelists for providing some really provocative and interesting comments early this morning. I actually had a question, I think, for Lauren. And that's related to the models. I wonder if you integrated modeling you talked about. And I wonder if you could talk a little bit about the sweet spot for the time scales of applicability of the modeling that you're doing. Because it becomes challenging when looking at different systems we talked about yesterday, the fact that the surface, what we understand about surface water and the time scale for water flowing through a surface system is different than a shallow groundwater system is different from a deep groundwater system. And if one is combining those models, for example, with some of the climate modeling, we know that, say, seasonal to sub-seasonal modeling is very challenging right now. So you have a better view for the longer term and some of the climate models. So when you're combining these models together, do you see a sweet spot for the applicability of the time scale for the results of that model or where there are challenges in terms of bringing the modeling results into time scales that have applicability when we're trying to use them for decision-making? Sure, so I think that there's a couple of important time scales to consider. So I think that they're already showing, like with the modeling that they're doing in Europe, that if you're thinking about weather time scale forecasting, that having the anti-season moisture condition correct is really important. And that's where they do get benefits from having this deeper subsurface included. I think that's probably the easiest one to achieve in a lot of ways because we can do a lot of data assimilation with that. I think that incorporating better storage terms into the global models for decadal scale forecast is really important, and that's where there's been a lot of work recently. I showed the example of Bridget's paper, but there's been a lot of other papers mostly using grace, showing that these longer-term storage trends that we're missing in our global models can be really important for controlling whether we have water energy limited systems. So I think that's where we can get the biggest gain from incorporating the larger storage term with respect to whether we do that with integrated modeling as opposed to some other more simplified approach that we incorporate into our land surface models. I think that probably we'll end up somewhere kind of in the middle because we maybe don't need to be doing all of the complexity of a really high resolution integrated model for our decadal projections when we have so much uncertainty in what we're going to do with pumping and all of those things anyway. But I think that we can use the integrated models to do a better job of understanding how we want those more simplified storage terms to behave. I mean, right now, if you look at what we're doing with that paper I showed from Bridget, we're all over the place in how we're incorporating storage and what that's actually meaning for those trends. So I think that's a big important area. Matt, Rayelle, and Asa got her. So since Bridget's papers come up a couple of times, I feel like I have to say a little bit about it. So Bridget and I are good friends. And she's not here, so I can say this. But when she was writing that paper, I had some serious misgivings about it because she's basically testing some models that were never meant to simulate trends on how well they could simulate trends. It's kind of like, I'm going to test these right around lawn mowers and which one's the best commuter vehicle. And of course, they're all terrible because that's not what they're built to be used for. So I think when we're looking at those models I mean, of course, if we want to be better at simulating trends, then we need to focus on that. But there's no expectation of what the ones that are out there right now should be any good at that. And it doesn't mean that we don't know what the trends are. I mean, we have Grace telling us what the trends are. And that gives us a target. But I just want to make that point that just because this set of models that Bridget examined was all over the place in terms of trends that is really no surprise because that's not what they were designed to do. Yeah, so just a quick response to that. Yeah, I totally agree with you. And it's not my intention to throw those models under the bus. And they weren't built for that. And that's OK if they're not necessarily matching that at this point. But a lot of people want to start doing that with models. And that's what people want to start applying these models for. So I think Bridget's paper is really important to still highlight the fact that we're not really ready for that. And I agree with you that we have the Grace data to compare to. So we do know what the right answer is over large scales for the period that we have the Grace observations for. But I don't think that the period of record that we have Grace observations for necessarily represents the variability of the system. And so I'm not sure that just using that period of record to test whether we're getting low frequency variability and storage dynamics correct, that we could be really confident in our models just if we did a good job from 2002 to present. And I'm curious to hear your reaction to that. Yeah, I agree with you. I mean, it's a short period. So obviously we don't have a good understanding of the longer period variability because we just don't have the observations at the global scale. And if that's something we want to be better understanding, then we need to develop models whose purpose is to understand long term variability. And starting with a lot of the models in that paper didn't even have a representation of groundwater. So why even bother if you can't represent groundwater, of course you're not going to get the long period variability right. And as you well know, there are newer models that do represent groundwater. And those are the ones that we really should be focusing on. But it also comes back to the inputs. So we can develop a model that's really good at representing groundwater or trash or water storage variability. But if we don't have good precipitation data for the last half century or whatever, then you're still not going to get the right answer. Yeah. No, I totally agree. I mean, I think it's a challenge. And I think that as we're trying to add storage to our long term projections, that's like a big area of uncertainty. And that having the grace data for the period of record that we can evaluate our models that are really actually, because we do have, I mean, that was kind of the beginning of what I was showing, is we do have large scale models that are actually incorporating groundwater now. But as we move forward to figure out whether we're doing the right job when we're projecting out 100 years as we have a climate that's changing and recharge regimes that are changing and different recharge mechanisms, I think that's like a big uncertainty and an open challenge. I mean, when we go back to a more classical groundwater modeling type framework, like the use of mud flow type models for groundwater management, historically, there's been an approach to parameter estimation inverse modeling, whatever, with in-situ data that people had some sense of what the data really meant. Where is the state of the art in terms of, in data simulation, clearly, you'd like to have a model prediction with prediction uncertainty, a measurement with less uncertainty, that then hopes that you get closer to what may be a right answer. With remote sensing based observations that are hard to understand, a lot of surrogates that they're measuring, and then you're processing that down to come up with an inferred quantity, where is the state of the art in terms of using large scale remotely sensed observations in a data simulation or inverse modeling approach for the classical groundwater modeling? Or are there any examples of that that? Sure. So I'm not an expert in data assimilation. So maybe someone else here will have a better answer than me. Oh, I have an answer. Oh, great. Terrific. Go for it, Jackie. So we just had a, so this is underway right now. So it's just getting started. But we just had a project funded by NASA's applied sciences program, a water resources program with Brad Doar, and he's the program manager there, to work in California with the USGS and the California Department of Water Resources, CWR. You know, California has this, I'm sure you guys have talked about it over the past few days, but this sigma sustainable groundwater management act coming up, where a lot of these critical groundwater management regions need to demonstrate compliance with sustainable levels of groundwater withdrawals by 2022, I believe it is, and timeline varies for different reasons, which include mitigation of subsidence. The subsidence is happening and is associated with unsustainable groundwater withdrawals. And so what this project does is we're going to be taking the Central Valley hydrological model, which is run by Claudia Fons at USGS in San Diego. And that is essentially a version of mod flow, as I understand it, and not my expertise, but as I understand it, that's a version of mod flow, where they built actually a routine in the model to simulate subsidence. And I don't know exactly yet how they do that. I think it just has to do with what's, I can't remember the phrase, is the relationship between the two activities. You're thinking of the coral elasticity and assuming 1D vertical deformation, they can make a calculation with just fluid pressures alone. Yes, exactly. So the goal for the project is literally that is to take great observations across scale. Of course, they already incorporate well observations when they have them, and in far observations of subsidence. And to, the model's already kind of parameterized. So just to kind of calibrate the model for different regions and get something, especially with subsidence, that is kind of spun up for different groundwater management regions, and it's calibrated so that it can work properly. With sigma, each of these regions is responsible to do their own groundwater modeling. So even though the USGS and DWR provide models, each groundwater management region is responsible to come up with their own plan and some projections of how they're going to reduce, make pumping sustainable and mitigate. So that's one project that's just been started, but I'm not aware of many others. So this is a follow-on to what JT was talking about, not as much an answer to your data assimilation question. But I do think that there's a big disconnect in the data sets and the methods that are used for our really large scale models and what we do for regional and smaller scale models. And that's true not just with respect to inversion and what we're doing with geostatistics, but also with respect to what we do to water management, because we have a lot of really great groundwater models that are built on watershed scales that are built in collaboration with the stakeholders of those areas that are using the water, and some of them are using litigation, or management, or all these things. So I actually think there's a big opportunity to have a better communication between all of the models we've already built at small scales and what we do at large scales where we're limited to just much worse data sets, but we know in places like the Central Valley or the High Plains, or we have a bunch of really heavily studied regions where we could do a better job. Yeah, just to add, that's something that the surface water community, the continental scale and global scale surface water community is also struggling with is these kind of different modeling communities, land surface modeling community, catchment modeling communities, they're all hurtling towards having scales at which could be applied to management, to answer management questions and the struggle then is are there common areas that can be tackled by all communities to try to make efficiencies in that? Things like data assimilation is certainly one of them, but that is something the surface water community is gonna be facing as well. So why don't the, I'm sorry, go ahead. Tonya Garberson, NGA, I guess the question I have for you then is the ability to do that data assimilation and that is that the lack of the data itself or is the lack of maybe metadata so that way people understand the data that is available and whether it can or can't work in their models? I don't know that I wanna answer for the whole, I mean, I know that we already, so like the group in Europe, they're doing data assimilation for soil moisture and I don't know the details of exactly how they're doing that. So I think, I'm not sure that there's like a huge barrier to entry just like with respect to data not being available or they're not being methods. I mean, there's great data assimilation techniques. I think it's probably the limiting factor with respect to recharge and groundwater flow is that we just don't have a lot of datasets to assimilate. So we have grace, we have soil moisture, but in terms of like deep groundwater, it's like what is the data that you're going to assimilate? So one of the questions I always wondered was depth to water table, very simple question, is how easily is it available and can we figure this out not just in U.S. but globally? So, Shreya, go for if you want. Well I just, USGS collects this data, we have it even at a 15 minute resolution and groundwater wells across the United States but I think there is, I would argue in the USGS, we haven't had a lot of effort to try to understand how to extrapolate that across large continental scales. So I would say that the data is there but the point you had made earlier, Venkat, about taking the data and then knowing what we're collecting it for and then understanding how it would assimilate into models I think is well taken in that respect. So I think there has to be more work in those areas done. We also have areas where we do have groundwater, surface water, stream gauging and groundwater set up next to each other so we know exactly what's happening between those two data series and there hasn't been much analysis of that. I think that's one of the criticisms we have internally at the USGS is we collect a lot of data but we tend to not always be the best analyzers of our data so I think there's a lot of opportunity there. Yeah and I would also point out there is a global, Yingfan did a global groundwater map. That's like, I would say, that's like kind of the gold standard I guess in terms of collecting all of the water table depth observations globally and she has put them all together. She has a map showing the point observations and then created a water table depth surface with just a simple model. So that's like what we have and then all of the other large scale models that get developed get compared to this which is good and bad. I mean those are the observations we have but also we know that like in the US if you collect all of the water table depth observations those are gonna be biased to the places that we've drilled a lot of wells which are the places where we wanted to use water so we drilled a lot of wells. So I don't know if that really answers your question or not but. I think that brings up another point that I had in my written comments which is really a lot of this comes back to like understanding data gaps and where there are funding or there is potential opportunity to put that next well or put that next stream gauge. That is an area where I've always I don't think has felt there's a lot of attention paid to that is really understanding the data sets that we have not just using them but understanding where the gaps are and then trying to look for opportunities to fill those gaps and particularly in places where you could where there are challenging hydrology or hydrogeology or hydroclimactic regions that can then be used for extrapolation to other areas where we also don't have that data. So I think that's a big opportunity in general in our field is really how do we understand gaps in the network and then fill those gaps. And the USGS does have the, I can't remember the name of it, but the set of groundwater observations that should be used for climate that are. They do, yes. Yeah, yeah. I can't remember the name of it right now. Right, yes, right. They do have a set of wells that they've kind of filtered out that right only have effects of climate in them. But there aren't that many. Great, exactly. There are not, yeah, there are not, correct. So Laura and Stacy, do you think that if you were to be able to pull together just for the US to begin with, all the well data, not just the USGS data, but the state-permitted wells that people are actually operating and had the strategy associated with them, our flow or more flow models that are run at the continental scale might actually show some different results? Like is it worth doing to go through the effort of really pulling together all the vertical information that's in there? Yeah, I think that we would get different results because I think that what we're limited to, I would say more with respect to the hydrostatigraphy than the wells, because the wells would really be, we have to know a lot more about management in conjunction with those wells if you're gonna use the well information meaningfully. But I think the hydrostatigraphy is really a limiting factor in terms of mapping where we have alluvial aquifers, understanding the depth to confining units, that's really important for getting groundwater configuration right over large scales. So if I could get one thing that that would be it, that would actually change what the model results look like. Should we put that on the engineer? Yeah, great, thanks. Actually, I'd like to step in and throw Burke under the bus for a second, which is that actually there's a really good example of this from Nebraska, right? Where the USGS had, I don't know how many wells, but a ton of wells and tried to calibrate a mod flow model, I believe, and still couldn't get those pieces right, would you? Yeah, it's sort of, I think one of the challenges is in missing sort of the non-linear features in the subsurface, a little window in a confining layer that promotes flow through, it might be missed with even a thousand wells. So that was the value in providing high-resolution geophysics was to identify the geometry of the aquifer system so that you can reduce uncertainties in making predictions in, say, for example, a flow through a lens in a sand aquifer or something like that. That example was pretty compelling in terms of the difference in the model performance after introducing those geophysical data, at least. And I guess just to counter my own argument a little bit, I think a lot of these questions, and we talked about this in one of the breakouts yesterday, is that all of this depends on the question they're asking, right? In some cases, maybe the hydrostatic if it doesn't matter for a specific question. And so we try to produce models that can do everything, and that's nearly impossible because the world is too complicated to do everything. So we have to define these uncertainties, I feel like, about surrounding specific management questions, and how do you do that effectively? Sure, yeah, and just to follow up on that, it really depends what scale you're at, too, because we talked about preferential flow paths. Those are really important, depending on the questions you're trying to answer. But also, if we're modeling at one kilometer resolution, like, what does that even mean, and what is the hydraulic conductivity, like, what does that even mean physically, like? I think that if you can at least get a better job of getting the structure, like the geometry of it right at large scales, that that will matter. But then at smaller scales, then we have like a whole other mess of complication, too. Yeah, so that's exactly the issue that we're in, what I showed in the Mississippi alluvial plane yesterday, is that we have one kilometer grid cells in detail at much finer scales that shows there are features that are important. And so how do you bring in those uncertainties into a groundwater model? So is it gaps in physical processes that you can't represent, or scales that you can't represent? What are the biggest challenges? I'm just wondering to, oh, sorry, I didn't know if someone else had their hand. But I'm wondering yesterday, since I wasn't here, was there a discussion about the role of airborne geophysics or geophysics in general, and the, okay, so yeah, because I feel like we spent a lot of the morning talking about different types of data, but that seems to be one I know, particularly for this Mississippi alluvial plane, airborne geophysics has been really helpful in resolving a lot of the discrepancies in the hydrogeology, so. Manu, I think you get the closing, closing video. This is a very simple one. Much of the discussion yesterday and also today didn't actually get into the types of aquifers, because traditionally we have also thought about the fact that you have sedimentary versus hard rock versus cars, and most of the comments seem to generally apply to sedimentary systems. So now I'm wondering that in many areas that are critical around the world, it's hard rock or cars. Do you guys have any thoughts on whether you have a decent representation of issues with those, or this needs to be a special attention to those? So that's a great point. I think that a lot of the integrated modeling work is really focused on shallow groundwater because the focus is on understanding what that means for our land surface connections and connecting from the atmosphere down and working our way down, but there's a whole other mess of questions for our deep confined aquifers where probably we don't need to have the land surface coupled for our deep confined aquifers for the time scales we care about. Carst systems, that's like another giant can of worms. I think also, I mean, we can represent like mountain block recharge and mountain front recharge in our integrated models, but I think that's also more of a frontier area of understanding what, how important those really deep long flow paths are and the relative importance of mountain front versus mountain block recharge. So that's a really good point. And I think that the tools and the modeling for those other systems are probably a little different. And I don't wanna imply that that's also not being done. Like there's groundwater hydrology and groundwater hydrologists have worked on car systems and deep confined systems, but it's like kind of a different group, I would say. Now my main reason to highlight that again is Tony's question from yesterday. It really becomes critical, from the human perspective, that the areas which are important and, you know. Yeah. Yeah, there's a bias. In the interest of time, I will unfortunately have to cut this conversation off, but thank you guys very much for your participation. And we are going to head towards our breakout sessions. And I think we all know, hopefully, where we're going. So take a few minutes, take a break and we'll see you in those rooms. Thank you.