 for the presenters as well as the committee member they can use the raise hand function to ask the questions. And also we welcome your participation after the session as well. So if you would like to share perspectives and insights on today's guiding questions, the four questions that I listed before, share your thoughts with the committee. You can click submit responses link which is there is a link on the meeting webpage and you can get to today's meeting website through a link on the main project website listed here shown over here. And then there are also similar links to contribute to other previous information gathering meetings where we ask different sets of questions and you can get to there and also submit responses. So with that, without further ado, I think we can get started with today's session and I welcome our first speaker Christoph Scheer from VTH Zurich. Chris, please go ahead and share your screen. Okay, I hope you are able to see this slide. Yes. So yeah, it's a pleasure to be here and to contribute to this session. It's a very interesting topic and very interesting topic from a scientific point of view. But it's also a very important topic from a societal point of view. And just as a motivation, I show you a few events that we had experienced. I show you here, the first event which happened in walking distance to my office, the location where this happens there at one point in time, there was a small creek and this creek is now in a pipe and if the runoff exceeds the capacity of the pipe, that's the kind of thing one gets. So this was just property damage, but there are other events like there was this major flood in the R valley two years ago in Germany, which caused a terrific, terrible destruction and a large number of casualties. There are also events in the tropics that hit societies that are not really equipped to react to these kind of conditions. Now to begin with, I would just like to motivate that looking into what high resolution climate models are able to do. I show you here an animation as a motivation, what you see here in a whitish color are liquid clouds. So these are water clouds, liquid water clouds. These blueish clouds are ice clouds, so they are high up in the troposphere and the colors that you see here in those clouds, actually they represent heavy precipitation events, so it's a color scale. So this is a very high resolution domain. So it has actually 340 million grid points and so actually it has more grid points than we have pixels on the screen. So we might not see everything, but I should try to start this. It runs now. So you see the animation. It's actually one month in 2006. So initially maybe you should watch the squall lines over Africa. So you see the tropical thunderstorms moving from east to west, moving out into the Atlantic. You see here parts of the intertropical conversion zone and you see vortices that spin up to hurricanes as the one that is active right now. You see also in the southern part of the domain, complex cloud patterns, not everything is observed, but overall the structure and scaling of this cloud pattern is quite accurate. Now you see a low pressure system in the southern hemisphere down here and here again elements of hurricanes and tropical depression that form. Now one of the really fascinating aspects of these simulations is that these models, they have never been taught anything about tropical meteorology. They really start off from the governing set of equations and then intuitively generate the appropriate mesoscale structure without ever having been taught about. And I show you here, oops, I think you are here. You see actually the equations, essentially the equations that are in these systems and these are based on the conservation of mass, conservation of energy, conservation of momentum and conservation of water species. So we have a very physical approach to the system. We don't rely in such simulations on artificial intelligence. Now of course what I showed you here was impressive to see but the question is really quantitative validation, does kilometer resolution increase the credibility of the simulations? I show you a few aspects of that. Here first I show you the results of the first ensemble of kilometer resolution climate simulations that is available over the alpine region. It has been coordinated experiments where many groups have contributed simulations and so we have a more than 20 10-year long simulations that cover this area at both at 12 and 2 to 3 kilometer resolution. What we are looking at is the intensity of our long heavy precipitation. So we look at the 99.9 all our percentiles and this is for the fall season. On the left hand side you see observations. Here we have the two kilometer simulations and here the 12 kilometer simulations using parametrized convection. Now you can see that actually the two to three kilometer simulations they have a decent chance in representing the observed spatial distribution of these heavy events. So the events we are seeing are very well known in fall period when the mediterranean is still warm and warm moist air is affected from the mediterranean sea into the continent. There is a coastal structures, a post-set ariatic sea over here and the tyrannian sea. You see also signals that are associated with the presence of the alpine topography or topography in France. So in general we can see here that two to three kilometer simulations really can capture these events by the 12 kilometer simulations do not. This one example and some other evidence that you see here is the validation of the diurnal cycle. This is now for our own model also from a 10-year long simulation. First showing the observations the mean diurnal cycle as a function of the UTC which is almost local time in our spot. You see mean precipitation that our frequency and occurrence of heavy events also expressed in person tiles and you can see that actually at 12 kilometer simulations really has dramatic biases. It suffers from this problem that the experts among you the diurnal the daily drizzle so that almost every day the models generate a little bit of precipitation but not with the appropriate intensity. Now if one adds the two kilometer simulations one can really see how much improvement is in these simulations really one of the best improvements that I have seen in my lifetime despite increasing the horizontal resolution of models for 30 years or something like that. The next slide I show you that this can also be used to for sub hourly precipitation. Previously I showed you results for hourly precipitation and in the study I refer to now we actually plot here or validate data based on stations in Switzerland so each dot here is a station and what you see here on this axis is the observed person tiles in millimeter per one hour and here the simulated person tiles for a range of different person tiles and you can see for the stations where we have long-term hourly records we see a fairly decent agreement. These now for the one hour person tiles but the other panels now show the 10-year simulations the validation for 30-minute accumulation and 10-minute accumulation and you can see that even for these durations these models generate decent output it's not perfect but overall they capture the really strong geographical variations that occur in these indices. There is some more potential to it very quickly only one can add diagnostics for hail the top panels or diagnostics for lightning bottom panels and this here only shows now two case studies but in the paper we discussed a much larger number and what is plotted here is the simulation against corresponding observations now the lightning we validate against the linear data set which is based on the electromagnetic magnetic pulse of lightning you can see that there is a reasonably or a decent correspondence between observations and simulations you can also see these for hails and here the validation data is radar data it's based it's only possible within the area that you see. Now how to use these models for climate projections there are essentially three ways how one can use these models the first one I just referred to as the conventional downscaling all these simulations they be information from a transient GCM simulations so this could basically be a simipsic simulation and the color scale shows the transition from cool to warmer climate in this century and in these simulations one typically conducts at least a decade long time slice experiments representing the control climate of the recent past and future scenario climate. The second approach uses the pseudo global warming approach and here actually one uses a reanalysis we usually use the era 5 reanalysis to drive the control simulation or to control climate and then we have a modified era 5 reanalysis which represents future condition and this modified reanalysis corresponds to the reanalysis but shifted by the GCM climate change signal so if you express this in an equations you could argue that the lateral boundary condition from the pjw run are the reanalysis run plus the delta the change signal by the GCM. There is a third approach which we have started to use as well which uses bioscorrected downscaling so in some way it's closer to the conventional downscaling as control and scenario GCM time slices are directly used to drive the high resolution model but actually in this case one tries to correct to bias correct the GCM fields we find this is particularly important in the topics where the GCM have really substantial biases some of you may know the double ITZ problem and of course if you use that information to drive a high resolution model then our results show then also the high resolution model will have a double ITZ problem but the problem can actually be mitigated by using this bias correct downscaling. Show you here a little bit of the results so what you show here what we show here is again for now different return periods the scaling for the whole of the alpine region for the summer season June July August and so on the horizontal axis is the inverse return period so it's this basically the number of events per year and so these are hourly events so this goes from 1000 hourly events to about 10 to the mine hourly events per year so this would be the 10 year return period and on the vertical axis we don't show the return levels but actually we show the scaling meaning on for the average we show how much the return value increases scaled by the warming so we scale these results by by the warming actually on the the regional warming at the 700 hectopascal level which we think is representative for these cloud systems and so we what we observe actually is that the lower that there is an intensity of the events increases at all scales for all return for all accumulation periods but actually the signal is more pronounced for the heavy events so for the heavy and rare events we have larger increases scaled increases than for the smaller events interesting aspect is that actually it appears that this is limited by the Clausius Clapeyro increase of about six to seven percent per Kelvin and this actually could be a very helpful result because in the past I personally thought who it's a very difficult task to say something about extremes because we even have a problem with mean amounts to say a good to have a good statement about mean amounts and this diagram signals that it might actually be easier to project changes in heavy precipitation extremes than mean amounts because we have this increase with the intense obvious the rareness of the events limited by the Clausius Clapeyro scaling well I should mention this is a was actually old wolf Clausius that were brought up some of that and I have the pleasure to pass at this street Clausius Street almost every morning because Clausius actually spent some time at ETH now I have two more slides what are some simple ideas how to adjust current estimates and I think the first question is how to keep things simple and so I refer here for a report which has some inspiration or gave me some inspiration it's really this is a community is a very poor community that is using these kind of estimates and if we change everything dramatically and make it very complicated it will be very difficult to make this work actually used and so you see here what is written in this in this report to take account of climate change historically derived rainfall rates should be multiplied by a clear climate change allowance which they refer to as the CCA and they suggest that for the design of new drain and sewer systems a CCA of 1.4 should be used that means actually a factor the return values are multiplied by a factor 1.4 and if you look at the warming that we expect maybe 5, 6 centigrade multiply it with the 6 per cent per Kelvin then you really end up with a CCA in that category so there are some more details in the report so there is some under some instances one may use smaller CCAs than this 1.4 for recommended above should maybe also show or mention that in Switzerland we don't use we don't use PMPs we use really return level as a function of return period these are estimates for one station like Logano Monti in southern Switzerland and you can see why we are interested in this topic so on the left hand side you see daily precipitation amount the blue is the best estimate and you see that the 100 100 and 200 year return levels they amount to 300 to 340 and for 380 millimeter per day so these are huge events actually this would mean that you get three or four monthly total in one day on the right hand side you see the same analysis for hourly precipitation you also see that there is an estimate which is actually a Bayesian estimate of the uncertainty which also depends on the availability of the data record that of course depends on the station considered now you see what happens if you would allow for a climate change allowance of 1.4 then actually the 10 year return levels go above these uncertainty range the 100 year return levels are staged slightly below these levels this there is a we have a serious challenge in Switzerland because 15 years ago they only used the blue curve there was no uncertainty assessment going with it now they are the communities introducing this uncertainty estimate meaning is beefing up their estimate from the blue curve to the upper green curve and now of course we are discussing already the next race in return levels sorry to interrupt you have two minutes left thank you okay okay so that means we have some internal discussion you also see that why we generally don't use PMPs because all the estimates we have and we have a number of stations maybe 50 stations that are more than 150 years old long we see increases of the return levels as a function of the return period maybe these levels off at some point above but we don't really know where this happens now then I'm already coming to the last slide so there is really broad evidence that climate change implies increases in heavy precipitation events that means that observations are losing their value as a guide to the future in Switzerland there are studies that show that already today we can using the 100 year records demonstrate that there are trends in heavy precipitation roughly consistent with the Clausius Clapperon scaling during the last 100 years I tried to highlight that kilometer resolution models they have a better representation of moist convection and yield an improved validation for heavy short-term precipitation events I have shown this for the diurnal cycle spatial distribution of extremes statistics for heavy events I intended initially also to show you that we also improve tropical circulations quite dramatically with this high resolution I just mentioned these theoretical ideas that have some modeling support namely that this increase in intensity will be limited or at least the model suggests so is limited by the Clausius Clapperon rate associated with the increased capacity of air to hold water vapor this amounts to 6 to 7 percent per degree warming I've shown that some countries start to using a climate change allowance factor of about 1.4 the same procedures could in principle be applied to levels to return levels of precipitation but also to PMPs because PMP is an intensity so you can scale it with this rate then last point I didn't have time to highlight that there is a big effort and some of you are involved with that as well there is a big effort to increase the resolution of global models and likely within 10 years we will have global models that can be run climate time scales at kilometer resolution so there is something at the horizon that will help us actually to make better estimates of heavy precipitation events in a future climate thanks a lot all right thank you very much grace wonderful presentation and perfect timing so now I would like to open up for questions a committee member you can just raise hand and I will call on you and if I don't see any hand yet I certainly have one question for you okay I'll go ahead with my short question first so Chris do you think that your results about the upper limit of the Clausius-Clapeyron could be dependent on the region or dependent on the model because I think there have been studies that suggest there could be super Clausius-Clapeyron changes so I haven't really seen any convincing evidence maybe we see some of that later I think there are some really important technical issues one of these is that for this analysis one needs to use all our person tiles and some of the studies that found super adiabatic scaling meaning faster than Clausius-Clapeyron they used only wet our person tiles and so there is some conflicting results from that and from a physical point of view we really have to go to to absolute frequency that means one has to use all our person tiles to in in this kind of analysis we have some I think your question about the region is certainly valid I could imagine that things change if we look at the tropical system systems because there is some likelihood that latent heat release as it happens in these convective cells may invigorate the cells but I haven't really seen much evidence for that so far but certainly not to be excluded important is that we have to use also temperature estimates for for the scaling and we always use not the surface temperature because depending where you are surface temperature might not really represent the lower top of the field great thank you very much Chris Effie Hi Chris very nice did I notice correctly that in your person tile plots that you had the model person tiles and observed the 10 minute amounts consistently overestimated the model consistently overestimated where when you want to one hour it consistently underestimated and where is that yeah absolutely we don't fully understand but it's certainly agree with your observations I think these of course the details of these models there is a lot of tunable parameters in these models hundreds so I think one could try to tune a model to be a bit better for the 10 minutes and another version to be a bit better for an hour but I think that that would not be very convincing so I think we need to to have a procedure where we can live with some model uncertainties and that's why also why I think that it's good to have a methodology that uses observations to estimate the return periods or return levels from current data or past data and we use climate models to assess the changes that will happen to these data in the next decades and centuries yeah so thank you as you say it's it's not good to tune probably the model was tuned too much to produce extreme convection so thank you yeah I mean this model I mean these are very expensive simulations actually some of these models I mean it's essentially the NWP version that is used here no extra tuning over climatological time scales is taken place if you decide to run such a simulation tenure will take you three months we take three months on a big computer so there is very little opportunity to tune thank you so we there is less tuning in these low resolution models a high resolution models than there is in the low resolution yeah okay thank you um John yeah thanks um thank you for presentation Christa I noticed yeah in the same plots that Effie was asking about the basically there was a the biases on the order of somewhere around 30 to 50 percent in terms of estimating the magnitudes of those the high percentiles do you have a feel for whether it's at all possible to estimate the the bias in climate change trend is it is it at all tied to the extent to which you can estimate the numbers themselves or is it something different do you have a feel for that well not a very not the one that I could attach numbers to it let's put it this way it's I mean there are really large a large sequence of uncertainties we also know that the GCM stone represent the changes in large-scale circulation very well certainly all GCMs do different things so there are uncertainties in large-scale circulations which of course would also affect these kind of processes so what we see in the present models is essentially the thermodynamic response and but the thermodynamic response is of course the most important part of that response and so there is some hope that we we can do that but much more work will be needed to address the question that you just raised so I think we are here at the beginning of these kind of studies and much more will be needed okay thank you all right uh Jim Chris very nice I wanted to get a Swiss perspective on US PMP one of our big problems is sub-daily rainfall extremes and mountainous terrain some of the largest flood peaks and rainfall magnitudes short durations are extreme convection and mountainous terrain and the and looking at these there's a paleo flood geomorphology picture that there are hot spots of extreme rainfall and they kind of match with our extreme rainfall events the the the question is are the convection permitting climate models are they able to resolve these types of hot spot cold spot features of PMP magnitude rainfall and complex drain yes so I mean I can say maybe something about the ELPs so in the ELPs we have these and there was one diagram that showed it maybe I should show this again there which really highlights how we have a chance to see that let me show this again sorry for but I think the question really asks for it so you should now see this light again and what you see here is really the observed distribution of some of these hot spots and so these are hot spots associated with the seven mountains and the massive central these are the ELPs and you see that the ELPs doesn't have a band in the south following the Elbein border but it's really this kind of two hot spots one is the Lagomachore area and the other is the Veneto area these are well known from observations and they are reasonably well captured by these models we can also see some other hot spots along the Adriatic coast well we don't have the data there that's one of the challenges and we of course see the coastal bands in this area here also interesting actually if you look at Corsica you see and Sardinia you see heavy precipitation events to the eastern side of the island and not to the western side so it says something about the weather situation that occurs so just a quick so these are 10 year if you look at say something closer in the hundred to thousand year rainfall would you expect to see the same kinds of capabilities yeah that's well if you go to 100 years I think we don't have a study that really enables to do that so far I'm not aware of it and certainly this is work that has to be done finding out whether these kind of I think these paleo-flats are often combinations of very special dynamical circulations very moist hair for the Mediterranean so I think there is a potential because it's really the mechanism that is represented in these models and so if the models have a chance to represent the circulation plus the mechanisms that happened in the case of upslope precipitation where we have kind of atmospheric rivers running up across the Alps then there is a chance to do that but I agree we don't have a proper idea. Chris can I can I really quickly follow up on this question are the are these maximums that we are seeing produced mostly by cold season storms or summertime convective storms yeah so we have that post so this here we're looking at fall in this paper from Pigelli et al there is also diagram for summer so we have it in summer and in summer much of it or part of it let's put it this way is heavy convection generated in over mountainous terrain which has a strong very strong diurnal cycle while in fall now this is really associated with moist humid flow from the Mediterranean towards the Alps usually this happens when a cold cold front runs over Europe and steers this moist air towards the Alps or when we have actually lee cyclogenesis formation in the Gulf of Genoa so it's a combination of these events we also have some heavy events to the north of the Alps that occur when a cyclone moves from the Mediterranean sea to the north over the Alps and then steers actually air from the Mediterranean sea in a long loop across and then towards the Alps particularly Austria and Switzerland are affected by such events great so we go to our last question from Obi and then we move to the next speaker Obi yeah Chris thank you very clear presentation we just have a simple question have a simple question on your frequency curve you went only up to about 200 year return period most curious was there any reason you stopped at 200 year or there was no need to go beyond 200 or whatever there might have been other factors that didn't give you confidence beyond that yeah good question so I think the tradition of these curves has been of course observationally observations and I think going and the uncertainty estimates as was evident actually increases with the return period of course quite dramatically I think if one does really plain statistical analysis it actually uses extreme value theory and some special treatment of uncertainty estimates if one does this kind of analysis with data of a duration of let's say 150 years for more than 200 years then the engineers in Switzerland I think they wouldn't trust us anymore it's probably the main reason so they would rather say okay then let's add another 20 percent they would rather do it themselves than let us do it extrapolate into an area where we don't have data thank you great thank you very much yeah thank you Chris for the wonderful presentation so now we move on to Kevin and he's going to yeah he's from Stony Brook and addressing similar questions that we have asked him yeah go ahead yeah great yeah thank you Ruby and thank you for for letting me speak a little bit today and what I actually framed my presentation round was actually kind of bridging the first two questions so I'm going to talk a little bit about some work that I've been involved with through a variety of collaborations both on evaluating the simulation of precipitation and climate models by focusing on storm type but also using different approaches to look at how precipitation within these different types of storms might be changing in the future and so the real motivation for some of the work that that I'm going to present on here is really going back to the fact that if you look at the most impactful disasters in the United States and you look of course at the eastern half of the United States a lot of these different types of events that produce a lot of damage are often resulting in you know too much rainfall over short periods of time and so a perfect example of this of course is hurricanes and tropical cyclones in the southeast and east coast and Gulf of Mexico but you can see that the variety of way in which we experience extreme precipitation in the United States comes from a different a variety of different storm types however when we actually look at a conventional climate simulation so we're going to back up a whole order or two magnitude from Christoph was just showing and focus on CMIP scale models that typically have grid spacings on the order of 100 kilometer by 100 kilometer that the realistic representation of precipitation in these operational climate models is remains a significant challenge so it depends you know no matter what metric you look at whether it's going all the way out to the 99th percentile or even less extreme cases or you know the most extreme day in a five-year period or or sorry your average rx five day or all these different types of metrics that there's a variety of skill within kind of conventional climate models and so one of the questions we've kind of had is you know is this coming from the fact that climate models struggle with the precipitation distributions themselves or is it that they struggle with simulating the specific events those circulations those storm types that lead to the extreme precipitation that we're interested in and so first I'm going to show some work in which we started to use storm typing as a way to kind of evaluate different approaches to to climate simulations and then I'm going to finish talking a little bit about how we start to use storyline approaches with climate models in order to kind of bridge this gap to look at how rainfall by a specific storm type might change in the future and so just I'm going to be really quick on this so we're going to analyze a variety of climate models and the way we're going to do this is using the Tempest extremes algorithm which is a publicly available software package developed by a variety of people but mostly Paul Ulrich and and what we've developed a variety of methodologies which I won't go into detail here in which we can track individual tropical cyclones for example but all a variety of different storm types we can determine the scale of the individual storm type so in this case we can determine the scale of a tropical cyclone using its dynamical wind field and then we can use that dynamical scale of the storm to extract precipitation that's only from that specific storm type so we've developed ways to do this for for hurricanes but we also do this now for MCSs and and extra tropical cyclones and conventionally what we've done is we've realized that if you start to focus on individual storm types and start to look at rainfall changes that way that not only do in some cases models depending upon the type of model can they compare a little bit better to observations and the biases in the storm type precipitation might be less than the general biases but they can also be used for looking at changes in rainfall by storm type in the future and I just wanted to mention that that we've done this for example with hurricanes in the united states in which we look at right you can look at the left column which is a frequency of the basically the hourly frequency of storms per year in a present simulation into the future and then you can look at the rainfall that comes just from those type of storms in this case tropical cyclones and then you can start to look at some of the details of the rainfall field by looking at things like what how much rainfall do you get per hour of impact because changes in the rainfall in the future are going to be driven not only by changes in intensity of the rain but also by changes in the frequency of those type of events but I'm going to gloss over this because I want to jump real quick to this paper kind of opened our eyes to the fact that maybe we should start to actually evaluate the climate models we use not just by their mean or extreme precipitation indices but maybe that we should start to do this by looking at the the different storm types that produce the extreme precipitation so I'm going highlight an example of some work that I've been involved with through the Department of Energy in which we've we've used the storm typing of of different types of storms so we're going to focus on tropical cyclones extra tropical cyclones and mesoscale convective systems and and the motivation for this study was in part because of of course not getting at the high resolutions that Kristoff just talked about but looking at new approaches to climate modeling for the coming decade to do century scale projections and the question is you know going to a higher resolution so again not all the way to those sub 10 kilometer resolutions but a high resolution for climate models on the order of 25 kilometers does that offer an improvement in precipitation particularly in in storm by storm type compared to other approaches to high resolution climate modeling which could be the multi-scale modeling framework or as many others might recognize it as the super parameterization approach to climate modeling in which you embed these kind of CRMs that were just talked about from the global sense but you embed them as into the the climate model itself and so this was kind of our motivation but I'm going to kind of show us a few ways in which we started to do this to look at precipitation extremes within these different approaches and so in all cases I'm going to compare it to to the low resolution what we call a roller low resolution which is a conventional climate model on the scale of 100 kilometer grid spacing. So the way in which we started to do this and this is the next few slides are for some work that was recently published and any day now should actually be on the the GRL website but what we've done is we've developed the capability now this is for observation so this is using the 20 years of the NASA eMERGE dataset in which we we run a variety of packages using tempest extremes in which we're able to track in this case we're defining extreme precipitation as a really just a heavy precipitation of the of the 95th percentile so we track individual 95th percentile blobs of precipitation and then we can using the dynamical aspects of storm different storm types those ETCs in the middle of the tropical cyclones and on the right the mesoscale convective systems we can associate specific extreme precipitation contiguous regions of extreme precipitation with specific storm types and so what this does is of course it first you know validates a lot of ongoing decades of work to suggest that you know in the United States the a large amount of precipitation comes from extra travel cyclones particularly in the north the northeast and an upper midwest and that of course in other parts of the country like the southeast you get a lot of contribution from tropical cyclones and then of course during certain seasons you get a lot of your contribution from mesoscale convective systems so this in itself doesn't tell us anything new that we didn't already know but what it does allow us to do is start to compare okay how well do climate models do at representing the actual pathways to extreme precipitation in in in reality and so here's an example of a conventional right this is e3sm but i would say it's a conventional climate model that are typically used in e3sm exercises and so what you can see which is maybe what we at all expect that the model is actually able to capture some of the heavy precipitation that's associated with extra tropical cyclones in part because we these processes are well reasonably well resolved in the model at these scales but if you look at the other two types of precipitation that provide a lot of extreme precipitation over conus in tropical cyclones and mesoscale convective systems you can see that conventional models are a virtually create zero precipitation from these type of events which means that if you're using this model to project future climate change and to try to understand what that means for precipitation extremes you have to be a little bit cautious because it doesn't actually include multiple pathways to the extreme precipitation producing events and then we've done the same results where we look at these two different approaches to high resolution climate modeling and you can see that when we start to go to the the MMF approach that we're able to start to capture some of these precipitation by type from important things like mesoscale convective systems as well as tropical cyclones but if you look at just the high resolution approach while it does start to do a better job of actually simulating some aspects of the extra tropical cyclone heavy rainfall it does still struggle with certain types and and so i'm just i'm going to go over this really quick but we did this for annual accumulated accumulation but we can do this for the amount of rainfall per event type and so here's just an example where you can see that mesoscale convective systems produce a lot of rainfall per you know per instance and and that even for extra tropical cyclones a lot of their extreme precipitation comes in the the frontal aspects of them and so you can see that that's also captured but when you look at again conventional models or even the high resolution approaches that there's varying skill and the ability to capture the extreme precipitation amounts per event and and so what we've done with this approach is we started to say okay well can we start to understand these biases in the model some and so this is for these three different types of climate simulations in which we look for conus and we can see that you know we can look by storm type and we can start to explore you know is the error in the amount of rainfall or extreme rainfall dependent upon the metric you want to use in the climate model does it come from a lack of intensity in the model or does it come from an error in the number in simulating the number of events of the storm type you're interested in right so a model might have low rainfall in part because it doesn't simulate tropical cyclones or mcs's which we know contribute to to precipitation and we can also start to do this at national or nca regions and i'm just mentioning this because it's in the paper as well you can actually start to explore the actual tails of the amount distributions as well to try to explore you know how why is it that some of the simulations are producing more extreme rates than others but and so the summary here is that this is kind of just new work in which we've been using to try to explore the different ways in which new approaches to climate modeling at least you know in the near future what the impacts that it might have on precipitation and extreme precipitation by looking at storm type and this this paper has a few main takeaways but what i wanted to do is just finish the last few minutes with talking about another approach to looking at precipitation changes in the future by focusing on storm types and so this is a storyline approach that we've developed both within the community atmosphere model developed at ncar but also within e3sm at the department of energy in which we start to run a series we treat the climate model like a weather model and so we run a series of seven day forecasts in this case because we focus on a specific type of event in this case a hurricane we often initialize the model at a specific times in advance of a hurricane landfall or throughout a hurricane season using the operational initial conditions that go into the operational global forecast system and then we run a counterfactual and this is which we so very similar to the pseudo global warming that Kristoff just mentioned in which we update the large-scale thermodynamic fields for temperature specific humidity and sea surface temperature and we can look at we can either do it where we look at 1850 and we look at what the the conditions would have been there or we can look at the future and so this is just another way in which we we've done this so we've done this now for all of the 2020 hurricane season in which we've calculated the climate change signal and we've removed this from our initial conditions and we run a series of forecasts now in this case we're running forecasts every through days through an entire hurricane season so essentially we're creating a database of a bunch of plausible tracks so of course there's a range in the different storms at the various initialization times but we end up with this actual forecast of the world that was and this counterfactual forecast in this case of the world that that could have been without climate change and then we're able to start to look at changes in the precipitation field so this just shows the the accumulated precipitation from out the season but what we can start to do is look at distributions of three hourly rain rates or we've also done three-day accumulations and we can start to dive into the differences between these and just to highlight one of the main examples that we've seen is that if you just look at the 2020 hurricane season as a whole we've actually seen about a change in a 10 percent in the most extreme rain rates so the 99th percentile three hourly rainfall amounts within the the counterfactual compared to the actual and so that suggests that there's been around a 10 plus or minus five change in those extreme rain rates but this of course will be dependent upon as Christoph was mentioning the metric you might use for extreme precipitation and so this is just my final slide is that this I was just highlighting how we've been using climate models now to look for attribution like studies but we've also started to do this to look at how does specific events so you can take an event that was particularly damaging or produced a particularly large amount of rainfall in the historical past and then we run these events under various future degrees of warming such as a two-degree warming a three-degree or a four-degree global warming and then we can look at characteristics in the change of the precipitation field so this is this is we have a study that we've just submitted that's focused on looking at hurricane Irma and looking okay how much more rainfall would fall in the future just interrupting two minutes left thank you and you can see that we see substantial increases in rainfall in a four-degree warmer world compared to the present warming and and so typically right when we do these type of storyline simulations you focus on the models that are well or the sorry the simulations that are well produced the observations so you often say okay this model well produces the location and the track of the storm and so this is a good model to use to look at the potential impact of climate change but one of the other things that we started to do is that actually by using storyline approaches you can actually maybe look at these other types of tracks so you can see in this example over here these these orange and red ones these are plausible tracks that were simulated at at further lead times and actually were compared to what the operational center was saying at the time and that actually you could look at these models under these you know maybe not well simulated tracks but maybe worst-case tracks for example for the miami region and you can start to say can we start to explore how rainfall would change in those in the future so that we can start to build up kind of these plausible land falling storms to help broaden the sampling that could be used for projecting kind of precipitation extremes throughout the 21st century so um so there's just some a high level summary and I'll actually stop and and hopefully take some questions thank you all right thank you very much Kevin wonderful talk and now good timing as well so now I'm also going to maybe you can move back to your last slide so we can see some of your summary point while we open up for questions any questions for Kevin so so Kevin maybe maybe maybe can you talk a little bit about your last point like doing the storyline where you actually allow the track to deviate so so in that case it really depends on the initial condition I suppose and probably the model formulation and many many different things so your idea is that by doing by allowing the track to deviate from what has been observed and then you would be able to quantify some uncertainty related to future projection in terms of like how widely spread out that possible storm might actually be producing extreme precipitation yeah so so I'll use this as an example if you kind of just look at the top plot here this shows the tracks both in the actual and the counterfactual and these are the ones that are included in our analysis and in typically in these type of storyline approaches that are focused on looking at the change in the intensity or the characteristic of a specific event you often want to focus on right the simulations that that well reproduce the observed storm and that's the common approach but but there are you know dozens if not in this case hundreds of ensemble or hundreds of simulation members that we didn't actually analyze in this study because either they were initialized early in the storm so because of the initial conditions there was a little bit more uncertainty of where the storm would go or because they just for whatever reason interacted right with the dynamic large scale circulation in a way in which took them to have slightly different tracks which makes it difficult to do that you know the apples to apples comparison for trying to understand changes in the storm itself but if you actually start to look at those storms you actually can increase the number of plausible right tracks and therefore rainfall fields associated with these storms and the interest or the reason why that could be useful is that right variations in where a storm makes landfall or the speed the translation of this of the storm can have huge implications for the actual rainfall field in the rainfall extremes associated with it and so the idea is that if we start focusing maybe on on these cases over here these poorly simulated storms that we will actually produce you know a broader set of what the actual range of precipitation might be for a Irma like event in the future that maybe actually has this earlier forecast time come true so I don't know if that kind of answers the question a little bit but the idea is it's a synthetic way of producing more and more storms and it's worth noting I'll just mention real quick that this is actually a common aspect within the hurricane community that we use all the time we use different types of statistical or statistical dynamical downscaling techniques to to come up with a range of potential tracks both in the current climate in the future and that's one of the ways in which they assess hurricane risk and so the idea is that maybe you could start to do that with the more dynamical approaches and kind of you know get a better sampling of the potential landfalling storms yeah wonderful thank you and in fact I see every hand up and I was also wondering like whether this might be something to do with transposition as well but in any case Effie please go ahead yeah no very nice talk I was intrigued by your plot that you brought the you broke down the bias in error in the number versus error in amount and I was wondering if someone had enough events that have both the error in the number of events and then error in amount what implications would that have for the frequency or accidents probability of design events you see I might have many few events that will affect my frequency of that event versus the amount you see is not straightforward in my mind but if I had enough of these events that I have broken down their bias in number versus amount the consequences for the design event accidents probability um would be something to study does you see yeah so I think I understand what you're saying and and yeah and I think that that's one of the right one of the ways to think about it right is that um if you know okay let me think for a second right so if if your model can well simulate the specific event type so it can produce the right amount of rainfall when you have the event but the bias in the model is that it just doesn't have enough of the events exactly that you could right so that's what I think you're articulating that that means that the model if you knew why how it was biased in terms of the event that it could still be really useful right if you if you understand the implications of that and I think that the answer to that is yes um but one of the things I'll just mention though is you have to be careful because not all of these right are are not producing the model or the the rainfall appropriately just because they have not the right number of events so they do fall on this kind of one to one line to some extent but it's not perfect and so what we have looked is when we start looking at the different distributions we can see that some of them not only are they not producing the right number of events but their their intensity of rainfall is also off mm-hmm okay thank you yep all right thank you very much uh Russ hey Kevin uh yeah this is this is all really good stuff um going back I mean I guess we're sort of here back to the to this the earlier part I'll look forward to to reading this paper um but I guess I'm curious whether you you know in this analysis or maybe just more generally like is the analysis uh capable of doing this right like for the purpose of this committee the 95th percentile is not actually of much interest it's something much much much higher than that and so I wonder if you if you have looked at that with your analysis method or or like what you see as the the potential for for insights at these even more extreme scales yeah so that that's a great question Russ and so if you know full disclosure for these simulations that I've shown we've selected the 95th percentile in part because their development like simulations right so they are they're only under the order of 10 years and that's that's not because I mean in a there are a little bit more expensive than a conventionally run but we that was just the design of the project and so I think if you had longer term simulations or if we wanted to like we eventually do right wanted to apply this to the broader c-mip archive um then you could certainly focus on a more extreme metric um and you know but what what that is that it was still I mean the way we do this methodology is that we have we look for contiguous regions of extreme precipitation which we define to be the 95th percentile for each grid point and so you could um you know come up with the 99th or the 99.9th and so that's very doable um but in terms of looking you know if you wanted to to go anything much more extreme than that might be a little bit challenging depending upon the ensemble side but this is actually now I'm saying that out loud and I'm just realizing right well this is could be a great use actually of a lot of the large ensembles that are out there at conventional climate simulations would actually need to look at that across those you know multiple modeling centers that have these large 30 plus member ensembles so actually there might be a way to look at a much more extreme event if you had the right dataset okay thanks yeah thank you very much thank you Kevin uh so now we move to our second session um so our speaker is Andrea Prine and he is going to more particularly talk about storm types even though we have also already been hearing a bit about that yeah Andrea go ahead yeah thank you very much for allowing me to talk today and I think I go back a little bit more to Christoph's talk but now shifting the focus really over the US so I use mostly telomere scale models to look at extreme events and um I will focus on those three topics mostly um that we were tasked to and I wanted to start with the middle one can high resolution models provide actionable accurate representations of sub daily PMP for the US in the current climate um when I looked at this immediately I thought now let's change that and say extreme precipitation because it's really hard to evaluate if the PMP is correctly simulated because often there are large uncertainties but extreme events are a little bit more feasible but I try to reach or breach to a PMP towards the end so what I'm showing you here is um a simulation that we finished a couple of months ago this is a four kilometer wolf simulation for for the recent 40 years downskilling era five and its spectral nudged over over North America and what I'm looking at here is the hourly extreme rainfall event like the highest hourly rate every year over the last 40 years and this is what we get from the co-op station so our station record in the US and this is basically what the model is showing us and this is the difference so generally you see that there's a lot of blue which I think is not too bad because we compare a grid spacing of four times four kilometers to a point observation so you would expect a little bit low but what I really like in this in this um simulation is actually it's very consistently blue there are some pockets maybe in the middle Atlantic region which is um less blue but it's consistently underestimating which basically means no matter which storm type we're simulating we're doing it fairly consistently well so it means like down here there are tropical cyclones like Kevin alluded to that mcs's or smaller scale thunderstorms in the mountains they all seem to be simulated fairly well this also holds if we go to more more rare events like this is the 10 year event now here um and you see a similar picture I also wanted to point out there's some really blue dots in here and this is mostly because I think we still have some issues with the observational record so I come back to that observations are not perfect either later on in the talk a different way to look at this is to look at pds we looked at those already so this is showing you now for different climate regions in the u.s. this is the northeast uh comparing three data sets the first one is the co-op station data sets and we're looking only at the grid cells of the co-op stations here then this is the 40 year simulation that we did with with wolf which we call conus 404 is the red one and stage four is the blue one and you can see all of those fall fairly well along the observation like I would say co-op is really the best observation that we have and this is all the rain rates over the last 40 years so you can see this is really well simulated by the model and this is showing you this for the same well for different climate regions and actually in many of those regions the the red line is closer to co-ops than the stage four stage four just to mention this this is a radar based data set that's corrected to gauges so this is one of the best hourly rainfall data sets that we have over north america and globally I would say however as you can see for example in some regions this data sets have some major issues so you cannot trust the data set blindly for example in the great basin where especially in the mountains where we have radar blocking it's it's not perfect so you might argue that we tuned this model to work really well in north america and this why my model tuning can be a problem most of our future climate projections so I also wanted to show you basically the same modeling system how it works over south america so we did really use the same wolf model setting in south america and we're comparing here to this set of stations where we have hourly rainfall this is only one year but I think it's fairly representative what you see here again is a pdf and immediately like the observations that we had here was gpm i merged as a satellite data set and you can see it's way lower frequencies and the intense events era five which is the best three analysis that we have is down here so it seems like wolf the red line the high resolution wolf simulation that we did is heavily overestimating rainfall however if you look at the stations that we have there this is by far the best data set that we have so it's exceeding the quality of the the radar data by far and even like way better than era five which has a lot of data assimilated so going back to the us one of the concerns and we talked about this already if you look at this this is 40 years of data of course we are really interested in extreme extreme events so this is for example for the northeast region if you look at the region approximately a 10,000 year event or a thousand year event will be around 90 millimeters per hour plus minus so the question now is really can the Kona simulation keep up to produce really well high resolute high rainfall intensities even for very rare events or is there a bias emerging once we go to very very rare events it's very hard to to evaluate I think we talked about this during Christoph's talk what we did here this is now really looking at weather forecast that we did at NCAR and we picked out some of the most intense events that we had in the forecasting period this is for example hurricane harvey I think one of those is a different day of hurricane harvey there's the west Virginia floods and very very extreme events and we evaluate the model forecast against stage four again which is basically if there's no no bias it's close to zero here and you see like the rare of the event the further right it is of course and you see like there is some dependency of the model bias to the rarity of the event but overall it's fairly well confined like there's a large uncertainty also how this line should be but overall you don't see that the models are drifting off in any direction if you go to very rare events which is very encouraging so I wanted to show you one example of that the west Virginia flooding 2016 was I think something on the one in the 500 year event really massive flash flooding in this region very complex was a complex metrological situation up up up slope flow here for example the appellations are here in this region as well and what you see here are different daily accumulations during this event and only one of those is coming from observations and the others are coming from weather forecasts with three kilometer wolf and the point is really it's really hard to tell which one is which so the model is doing a pretty good job in representing the peak accumulation during this event so this is actually the observations up here these are different versions of forecasts and this is very similar to what Kevin said before you can see for example this is I think that this was the best forecast according to the peak accumulation we have few forecasts that are very weak but a few that are very intense and as far as we can tell the model like these events like the real event might have turned out like this event so the point that I wanted to make here is really that we can use this ensemble forecasting framework to look at at potential storms that might have happened if the metrological situation would have been a little bit different so I think this is very much in line with Kevin said before so I think there's also a big opportunity to learn from the radical weather forecasting when it comes to using these models and forecasting because they have a lot of experience in how these models are simulating heavy events and in the climate community like these kind of approaches are now really emerging under the name of ensemble boosting so I think this is a really interesting way to use an extreme event that we observed but really put some envelope around what this event might have looked like and some of these possibilities might be even worse than the event that occurred since we are already in the Appalachian region I also wanted to show you this result this is again using very intense events in the Appalachian region so the top 20 events over the last couple of years and looking at how the vertical gradient in extreme rainfall was captured again zero would mean no bias this is stage four and what you can see here clearly the three and one kilometer simulations have a very nice vertical gradient they put the rainfall in the right elevation band if you only use station-based observations like prism or grid map, Livner these data sets of course like most of the stations are in the valley so you have to extrapolate somehow and in this case we put too few precipitation in the valley and way too much on the higher peaks which of course is a really big concern if you think about spatial representation of extreme rainfall so I really wanted to stress out we talked about this during Christoph's talk already but in especially in our graphic regions I think this can be a really big benefit to use these kind of models because our observational systems have really big problems and the station density is often very low in these regions. Coming to observational uncertainties I just wanted to touch on this as well a little bit because of course we have evaluated our models against observations and then we often say that there's a bias and the model doesn't do really well we really have to keep in mind that the observations are not perfect either so what I'm showing you here is the daily accumulation of a tropical storm bill when it made landfall in June 2015 I will highlight these four different observational data sets stage four is the one data set that I already mentioned it's a radar-based data set merged with station observations corrected and you can see very high precipitation accumulation so this was record breaking rainfall in southern Oklahoma and northern Texas in this case if you use a different data set so this is the multirata multi-sensor data set this is probably the gold standard that we have in the US when it comes to gridded high resolution on precipitation estimates you can see like this pocket of heavy rainfall looks very very different even though like we use a similar like same radar data they assimilate additional information into their systems but you can see it's it's definitely not the same this is what prism looks like the prism data set prism uses stage four in their gridding you can see like this this spot here looks similar but along the coast you miss a lot of these heavy pockets and then this is what a grid station only based data set would look like so if you wouldn't have any radar information you get something like that so this is really just to highlight the station record that we have is not really capable of observing this peak hourly high resolution extreme rainfall rates that we see for example in radar data like radar observations and also in the models so moving on to this actually this first question how well can climate models simulate different storm types so i know kevin talked about this and this is i will have to confess this is a little bit hand wavy and this is really open for discussions i would be happy to hear other people's opinion on that but i think we can put this on a diagram that's a little bit like this like the large the large scale forcing so synoptic scale forcing perioplinicity low from low to high into spatial scale of the storm from small scale to large scale and i would argue that storms that are um up here in this corner have large scale forcing and they're fairly large storms this can be atmospheric rivers, extropical cyclones, frontally forced swallines models do pretty well with those that this is um this is these are storms which are really well simulated then we have this intermediate class and then down here it's really where it's getting a little bit tricky where you have this cloud bursts of very local storms supercells can be in this category as well this is one example not for the u.s but for actually for japan where they had the convergence of sea breezes and you can see that the scale of these this accumulation is very very small but it's going up to 600 millimeters this this reminds me a little bit of the fort lauderdale flooding that we had a couple of weeks ago and you can see they had to really go down to actually sub-kilometer scale grid spacing to simulate these storms this is just something to highlight we have to be aware like we we often use focal armor models and these very small scale storms might not be as well simulated we have to be cautious of that and really make sure that we don't miss these the contribution of these kind of storms to p and p when we look at current and future climates so moving to the last piece the climate change projections so i think i put this this way what we would like to have i think is global large ensemble transient kilometer scale climate simulations from an ensemble of models like this would be our wish and christof mentioned these kind of models and they might come online in 10 years but what we currently have often is really deterministic runs a single simulation of or sometimes you have small ensembles some of them have very small regions that they're covering or focusing on individual cases so it's really a patchwork of things that we have at high resolution but i really like and kevin mentioned this before this large ensemble framework i think it's extremely valuable can be extremely valuable for p and p estimates so what this is is basically you start a lot less two minutes left yeah okay i will speed up i won't go into the details larger songs i hope you you know what the systems are it really allows you to to estimate the internal variability and get really a large amount of data for a specific climate you can use them in two ways the storyline approach this is what we already heard so you basically use something like a pseudo global warming or storyline you perturb one event and you look at how it changes in the future um michael weyner and patricola they they used this approach for tropical cyclones maybe michael is mentioning this later on this has a lot of of benefits here low computational because it's fairly bias free you can evaluate the model against the observed storm and it really allows you to get insights into process changes um the the downside is really that you stuck with with the storms that you have observed so it's it really doesn't allow you to look at extremes that are not in the observational record and this is really where the other approach comes in you you can screen through all these large ensemble members and pick out events that are simulated and downscale them to very high resolution so this is a study that did this very successfully and this is really allowing you to add events that we haven't observed so far so this would be something like black swan events and this is really the events that we have to be worried about but there are also a couple of downsides if you if you use these kind of of approaches so coming to my summary um i wanted to give you concrete answers to these questions how well can climate models simulate different storm types i think larger storms that are large scale force are really well simulated with smaller scale weekly uh forced storms i think we have to be a little bit more cautious even though like the models sometimes can really capture those as well um next can high resolution models be used for extreme precipitation of pmp estimates i will definitely say yes um especially if you have a well calibrated model you can really get to quality data quality that's within the observational certainty and what are the key challenges i think it's really the sampling of the large uncertainty space that we have in the future we have to find ways to deal with the gcm biases if we look at events uh that we directly downscale from gcm and how we account for black swan events is is really not that trivial so how do you find those in the gcm's are they realistic and can you bias correct those if they are not um realistic so last slide just a couple of things additional considerations um i think kilometer scale models are really interesting because they can help you to boost the observational records you can simulate events that you haven't seen before i think this is really what we need because these black swan events like sometimes we see events that occur and we think we never thought that this is possible but exactly these kind of events are the events that we should be interested in if we think about pmp and then the last point is where the collaboration is key i think learning from the forecasting community we use forecasting models like christoph is using a weather forecasting model i use both which is often used in weather forecasting uh of course pay their climate study um bringing p pay their climate people is important statisticians that we have to like how to estimate pmp's from these return value events is not trivial and then storm transportation methods and the last one is really we need more coordinated kilometer scale modeling activities in the u.s like what christoph showed they had activities in europe where they really jointly ran some simulations this is not happening in the u.s and i think we really have to change this to to make these simulations more comparable and more useful for really doing things that you are tasked to do with the pmp estimates and with this i want to thank you for your attention thank you so much andreas wonderful talk um so now we're open for um questions before i see any hands up i would like to um first maybe ask one question so so andreas i think it is quite clear from your presentation as well as from our previous two two speakers that we really need ensemble modeling right so but but of course ensemble modeling would be very expensive so can you maybe comment a little bit like with limited or even with a lot of computing resources that we have what are your priorities in terms of creating the this ensemble because you can perturb initial condition you can perturb you can use different models you can even select different types of storms so there's many ways of doing that and what might be your like hey these are the things i really would like to to have yeah i think like to be if you really want to be computational efficient it's of course the best way would be to focus on extreme events like if you run fancy and simulations 99.9 of the time you don't have an extreme so if you can't focus on extremes you already save a lot of resources so this is easy if you look at historic extremes that we that we observed and i think this would be a low-hanging group to run ensembles of perturbed forecasts or modeling simulations for really this high impact storms in different regions and then i think it would be really interesting to like searching for extreme weather patterns in global large ensembles and downskating those patterns specifically the danger is really that these patterns will be often associated with large scale forcing so you might miss things like this for lauderdale lot of this very small extreme events that have not really large-scale fingerprints so but i think there's definitely some some work that we can do to improve that right i i yeah i also like very much this kind of approach where you perturb a set of historical storms but but you might potentially miss the black swan because a storm historically that wasn't the best that wasn't the worst could become the worst in the future or something like that every true yeah right okay uh jim andre is very helpful um question um on um weak forcing and uh appellation setting um the upper ohio valley along the western margin uh of the appellations it's uh has some of the largest rainfall accumulations in the world at timescales less than six hours uh and dominates probable maximum precipitation and it's mainly weak uh forcing events um you've shown that uh in simulating the climatology of mesoscale convective systems there's a low bias uh for those mid-summer weak um uh forcing events what what are the prospects for doing a better job uh with uh with that climatology yeah thanks for bringing this up um so we actually improved the model quite a bit concerning the slow bias it was in in wolf system mainly related to land atmosphere coupling you basically drifted off into a very arid climate in the central yestering summertime and doing a better job with land surface processes really helped us to increase the numbers of mcs's in the late summer season so the simulations that i showed you here don't have the severe bias anymore um but yeah very true like we we should really look at i like your question about this pockets of heavy rainfall in the alps and i think this is something we haven't looked at at this simulation so this would be very important and helpful to evaluate if we can simulate those pockets of hot spots similar to what christof showed in in the appellations but also in in the rockies thanks um okay thank you yeah uh john yeah uh comment which i'll let your comment on and then the question um so i think the idea of you know running ensembles on situations that are uh maybe set up to produce very high precipitation amounts it is a good one and the obvious thing to look at is high precipitation water amounts for example but um i don't think i share your pessimism regarding capturing things like for lauderdale because you can for lauderdale was unusual in the the projected storm motion was almost stationary so if you're looking for those sorts of things you can catch that aspect of the extremes as well um and then my question is um i'm i'm old enough to remember when we couldn't really forecast uh extra tropical cyclone the intense extra tropical cyclones with the american walls very well and then then finally we could and then we were over forecasting them uh we made it to the good part of the extreme are you concerned at all that as we get further along we'll discover that now we have to deal with flaws that in within the micro physics and so forth that are making the processes an accurate yeah i fully agree like i think this is the big benefit of using convection permitting models like you get rid of a lot of these prioritized processes that have a lot of errors like especially deep convection is a big big problem and then yeah now the micro physics play a much bigger role and so i think like i also tried to highlight this at my conclusions i think a well calibrated model is really key like you can with war for example very easily change the the physics settings and you can see that you get quite a large spread independent which physics you use um so you really and again like this is really helpful to talk to forecasting community because they do do this parameter testing and physics testing quite a bit and have large experience and how to set up the model well thank you right are there any other questions okay so if not i think we can take a break right now for about five minutes and then we come back and we'll have our panel discussion starting with three speakers who would particularly talk about uncertainty and then we welcome other speakers to also join the panel and then have a discussion about uncertainty so so with that so we'll come back in about five minutes which is roughly let's say 12 35 pacific time all right thank you see you all again ruby there is a oh yes uh there was a question for andres from oh uh sorry oh from i'm sorry from who again i didn't see you what should we ask it um let me just i can i can read that question it's from okay it says what uh for andres what type of modeling experience experiments would you design to explore forcing mechanisms that produce pmp type storms that have not been observed in historical record meaning what if i what if i look for an alternative to a pgw or store storyline approach i want to explore alternative or new mechanisms for pmp type events that's a very good question i think an active area of research like the large assemble approaches again i think a very good one for example in our new large assemble which has a hundred members there is a tropical cyclone hitting california and we know like historically this can happen but we don't have one in the in our like observational record so you could at least for if you have some kind of large scale proxy fingerprint of extreme events you can search for those fingerprints and course resolution models and hope that this models can reproduce those fingerprints and then don't scale those situations but again you have to rely on these fingerprints of finding these large scale situations which is not trivial so there's definitely i think urgent need to do some research in this area all right thank you very much so now we have exactly five minutes for our for our break so please come back in five minutes uh 12 35 pacific time we'll see you again bye this is michael i'd just like to test my audio can anybody hear me yep uh-huh yes i can hear you uh-huh good i realized i forgot to do it earlier no thanks thank you so are we starting uh live streaming soon i'm just curious i think so um yes i'm ready whenever you are yeah i i am uh-huh thanks okay so um i'm going to hit and get started with the countdown so starting from five four three two one all right welcome back everyone um so this is our last session of the day and it will be a panel discussion on uncertainty in simulating extreme precipitation so we will ask each panelists to first speak for about five minutes and then we will go into questions for the whole panel including um the previous speakers as well so our first uh panelist is paul orrick from lucy davis paul can everybody see that okay uh-huh yep okay sounds good all right thank you very much for inviting me to talk i'm going to try to kind of bridge uh our previous discussion on um different modeling techniques and discussing some aspects related to uncertainty particularly i'm going to focus very briefly on um the usage of storyline simulations and give some of the examples that we use uh in the hyper facets project in order to assess future changes in precipitation extremes um so we've heard a little bit about storyline simulations already but just to emphasize these um these storylines are basically based on events or periods of significant impact from history and so basically historical examples where we've seen uh extreme precipitation events as they have occurred and of course there's some debate as to how close we get to the pmp uh level of extreme precipitation over the historical record but i would say that it makes sense that if you consider basically worldwide uh extreme precipitation events as they have occurred um then we have some pretty good idea of what this relationship is between the intensity of or the total accumulation of precipitation and the duration of those precipitation events and so we can leverage those historical events in order to develop in order to basically understand future change in those uh events as well and part of that work involves of course this assessment whether or not models are actually doing a good job of assessing it and we've heard already a lot of discussion on um the quality of our modeling systems and how they've improved dramatically over the past several decades for simulating some of these very important extremes in the hyper facets project we're assessing a number of storylines targeting uh important high impact events across the conus uh including everything from drought to uh wind storms uh in the east uh the two i'm going to be talking about today are atmospheric rivers and uh a duress show events in order to get that basically discrepancy and scales from the large scale uh precipitation event driven by an atmospheric river and a finer scale convective precipitation event uh and each of these are going to be simulated in the regionally refined e3sm uh model so the first event that i want to talk about is the 1997 new year's flood event which was a major atmospheric river that caused widespread flooding across california and caused many gauge stations to basically hit their overall record this is both for stream flow and for overall precipitation event uh precipitation amount and caused subsequent very uh strong impacts on uh very high infrastructural damage as well related to this event in order to simulate this event um we used again the regionally refined e3sm which is this global climate modeling system with embedded high resolution placed over california and with with uh this higher resolution then we were better able to capture topographic features and as andreas kind of alluded to earlier um we developed basically an ensemble of uh forecasts from this event driven by different initial conditions and then subsequently simulated this at different warming levels to see what the sensitivity was of the precipitation to the upstream uh to the warming um if you look at this figure now in the bottom middle here this actually shows a comparison of model performance versus precipitation gauges and other observational data sets and i would say that kind of what andreas also alluded to earlier our biggest issue is having observational data that is actually able to capture these uh extreme events so while we have confidence that the models are actually doing a decent job it's very difficult in order to make conclusive statements on that because we don't have the benchmarking data sets necessary to uh or at least the uncertainties in those bench benchmarking data sets are too large for us to really state very clearly that this is a good representation of the event as it occurred historically nonetheless if we look at the precipitation gauge record over california during this event and compare that to this 3.5 kilometer regionally refined simulation you'll see that there's a pretty good match particularly at the peak of the the sum total precipitation amount here so that does indicate that as we go to finer and finer resolution and as we kind of enter into this single digit kilometer scale regime that we're actually capturing total precipitation amount at the levels that we would need to in order to estimate pmp successfully and we can also see basically a natural response here of um the overall total precipitation amount to warming and so what we're seeing is basically a five percent increase in mean precipitation which again accords very well with uh christoff's comment earlier about how extreme precipitation is changing into the future and it allows us to then make more confidence statements about that multiplicative factor on how much extreme precipit how much pmp is basically changing in response to climate change the second event that i wanted to talk about was the 2012 north american duresho which began as basically a small storm cluster centered over wisconsin minnesota area on the 29th of june in 2012 and it basically passed over much of the eastern us although dureshos are perhaps most widely known for extreme winds they are also a form of mass scale convective system that delivers with them extreme precipitation as well and so here we wanted to make sure that even this climate modeling system was able to capture the most extreme precipitation from these duresho events and what we generally see is that yes the model actually does a pretty decent job again the issue that we have in order to make confident statements about the model quality is related to the quality of our observational products so here what we see is that if you look at the mean error basically at 6.5 kilometer and 3.25 kilometer simulation in the regionally refined region we underestimate the precipitation amount relative to stage four at 1.625 we actually overestimate the precipitation amount here but we are getting closer to see more of an eye merge as comparison precipitation products given that there's a large spread in observational products it's very difficult to say whether or not the model is again producing the correct amount of precipitation consistent with reality but we can say that we seem to be doing a much better job of capturing these high precipitation amounts and kind of this one to three kilometer regime is where we generally want to be with models even to capture these convective storm systems so getting back to some of the questions that we're addressing in this session how well do models capture PMP magnitude events across different storm types again these are my personal opinion on the matter for most of the examples considered I would say that models generally fall within the observational spread and I think that they're pretty robust in terms of overall predictions of probable maximum precipitation independent of storm type once you have sufficient resolution within the model and once you of course deal with some of these issues with atmosphere land coupling how well do they capture rainfall extremes and complex terrain it's a very difficult question to answer because at fine resolutions we can simulate kind of these short duration summertime convective storm events in the models but we don't have confidence in the observational data in order to benchmark those precipitation amounts as Andreas kind of referred to or mentioned earlier you have issues with like radar blocking that prevents us from getting very good estimates of total precipitation amounts associated with these storms and so it's difficult to benchmark them exactly how well do they capture rainfall extremes in the absence of topographic forcing I would say that we're doing a pretty good job honestly I am impressed at some of the performance that the models are producing particularly for these mesoscale convective system events in terms of recommendations how does spatial and temporal scale affect how well models simulate PMP magnitude rainfall I would say that historically researchers have generally reduced the resolution in their models to finer and finer resolution in order until things look reasonable and then generally stop because of computational limitations I think there hasn't been a lot of work done on pushing the models to even finer scales I think this was alluded to earlier as well and so we need to be confident that there is convergence in the models as they get to find scales when it comes to PMP how can large ensemble simulations be used to inform sources and magnitudes of uncertainties and so again getting to this uncertainty question I think that there's a desperate need for long duration cloud resolving simulations over the U.S. this has already been mentioned once presently available large ensembles are too coarse in order to say anything about these black swan high intensity extreme precipitation events and so we need to have better large ensembles to answer these questions and how can we estimate PMP in a future climate in this PGW framework there's a large ensemble of the PGW simulations basically allow us to constrain that multiplicative factor associated with PMP and I think that we're going to be leveraging that primarily in order to draw out the statistics of PMP over the whole length of possible durations we do need these large ensembles in order to identify possible hazards but again as I alluded to at the beginning worldwide we've seen so many extreme precipitation events that they tell a pretty good story about the relationship between event duration and event intensity so this figure was distributed as part of the the notes associated with this session I think that it's really important to be able to reproduce this figure in our models one of the shortfalls that I'm that that make this very difficult in the modeling community is that we don't output high frequency precipitation intensities that is sub hourly it's even hard sometimes to find sub daily precipitation amounts and so pushing the community towards outputting very fine frequency precipitation amounts will allow us to truly assess how well our models are at capturing these short duration extreme precipitation episodes and whether or not they're getting those corresponding processes correct and as I mentioned earlier if we draw upon the global sample of extreme events as they have occurred over the historical record we may be able to derive the statistics or interpolate the statistics of these extreme black swan events horizontally through through this distribution in order to understand more about what PMP might look like for black swan events great thank you very much Paul so now we move to Kristen from Colorado State University okay can you hear me and see my slide yep okay well thank you for the invitation and this is a really great discussion today I'd like to add some additional thoughts on what has been talked about before okay so this is this was just actually shown by Paul I want to go back to the ingredients based flash flooding forecasting framework produced by Dozwell et al in 1996 where precipitation is the product of the average rainfall rate and the duration basically the heaviest rainfall is falling where these high rainfall rates are located for the longest time and if we think about this in this intensity and duration framework similar to what was just shown there was actually a wide spectrum of events that can produce heavy rainfall and extreme rainfall here's an example from the precip field campaign that happened in Taiwan last summer where there's a wide range across you know strong vertical forcing events and also long horizontal strong horizontal forcing events that produce extreme rainfall from really deep convective systems that are local in scale all the way out through tropical cyclones and these bigger broader kind of stratiform precipitation systems that may last a really really long time and so considering the modes in the in these different types of spectrums that we can see is actually quite important now if we look and we gain information on this spectrum from the information from the very first space borne precipitation radar aboard the trim satellite that orbited for 16 years across the entire tropics and subtropics we can look and see where these storm modes occur kind of in an observational framework and you can see there's a wide variety of where we see the biggest deep convective storms here in the middle panel and the organized systems more representative of mesoscale convective systems on the bottom so if we think about the research at the intersection of weather and climate in the context of understanding extreme precipitation I would argue that to understand the spectrum of storms we really need to consider models across scales and their environments as well and so in one way that some of my work has been aimed at this question is to really think about convection from any regional climate simulations or we're actually trying to represent the the spectrum of systems in their natural environments in a current and a future climate so the pseudo global warming technique was mentioned by Christophe Char earlier and also Andreas I'll use this example simulation set 13 years in the current climate over the contiguous united states shown on the right for a 13-year period in the control simulation and a pseudo global warming simulation for that same 13-year period with that climate delta change signal from a 30-year mean in the future and a 30-year mean in the current climate we apply that change and then drive the simulation again so one of the things that were noticed from in analysis of this simulation was that we're getting really great estimates of precipitation Andreas gave a nice talk on that we also see resolving some of these topographic features at this four kilometer grid spacing or graphic precipitation is much better represented as represented by some snow tell snow observations compared to the simulations we're also getting diurnal and seasonal cycles of organized systems we're getting propagating mesoscale convective systems that maximize in the nocturnal hours you see examples here on the right and it we're allowing convection to naturally evolve in its environment and so we're getting a better answer with these particular storm modes we also looked at how these particular systems change from weak systems these are radar reflectivity differences from very weak systems on the top right all the way through really intense radar reflectivities on the bottom right showing that we're seeing actually broad decreases in the weak to moderate convective population but then as we scope to more and more intense reflectivity ranges we see increases in in these ranges and so this is an interesting shift in the convective population we've also looked at storm modes specifically in these cone simulations this is an example from the 2011 super outbreak of tornadoes across some of the midwest of the united states and we've compared some of these specific storm modes to gpm this is the follow-on precipitation satellite in space these same storm modes and the gpm observations on the top row and the cone is control simulation on the bottom we're finding broad agreements we're seeing these these different types of storm modes from deep systems through wide systems in their intersection across regions across in the united states now i will note that we've done similar analysis in the south america simulation and we do see some challenges in representing storm modes in tropical rainforests like the amazon so there's some information there in terms of uncertainty okay we look at how these storm modes change basically taking that difference in from the current and future climate and we see actually really really large increases in the number of these specific storm modes as we go in a current future climate so if we map this back on to that intensity duration framework we can think about what this means in terms of changes in those extreme precipitation values speaking of extremes i want to focus quickly here on looking at specific extremes in these simulations and my previous phd student erendocrity did some great analysis looking specifically at 584 flash flood events across the united states in that 13-year period and what we find is if we look at maximum uh rain rates within just the storms that produce that flash flood rainfall uh we do see that those rates are actually uh shifting to the right the the pgw uh distribution is here in the red and i and we've mapped this into the percent change per kelvin and you actually see that when we specifically highlight very extreme flash flood producing systems we actually do get as um scaling that that goes beyond the classiest clapper on scaling leading to the important role in considering the important role of thermodynamics and dynamic changes in the systems in a future climate we also looked at area average rainfall we see that these systems are overall producing more average rainfall in a future climate and so i go back to this uh that you know it's really important to consider the spectrum of storms producing extreme rainfall there's a wide variety of systems that do so but uh one of the things that we've had challenges with uh is it's very hard to assess uncertainty due to computational constraints in this convection permitting framework we've heard lots of discussions on ensembles and new approaches to do this i'm really excited about these opportunities and these types of of of new tools in the future to try to assess uncertainty in these convection permitting frameworks okay so we're considering uh it's very important to consider strengths and also weaknesses of this extreme precipitation estimation and models across scales i'll leave you with this framework here that if we're thinking about extreme precipitation estimation in general we think about satellite observations we have really oops looks like my animation is broken here well then let me scope through here sorry about that um that these satellite observations provide broad coverage we look at three-dimensional structures but we only see snapshots in time when we scope down to these regional climate models we have really good seasonal dinor cycle precipitation we do see storm modes that are reproduced relative to current observations we we do see good representation representations of orographic precipitation but often we only have one estimation of these future climate simulations especially if we have really long-term 13 you know and 20 year types of simulations global climate models as has been discussed today too they actually really well represent the thermodynamic environment supporting extreme storms there's large ensemble types simulations that are available they do represent changes in storm tracks through the climate dynamic framework but they do have challenges in representing these storm modes so i think all of this is important considering how we estimate precipitation extreme precipitation from models across scales as well in there thank you for your kind wonderful thank you very much christine so now we go to our last panelist michael weyna from ron spergley national lab thank you ruby let's bring up my presentation and you can see that i hope uh yes uh huh yep all right so as much as i'd like to talk about uh superclosies claperon and event attribution and storyline i'm not i'm going to talk about statistical models and the question was asked can climate models be used to calculate pmp in current and future climate i'd like to ask that about statistical models uh here's my standard disclaimer i'm only speaking for myself so let's for those of you who don't know about extreme value statistics this is a just a brief pictorial of it there's the equation for the generalized extreme value distribution and here's what it looks like the probability density for various um uh values of what's called the shape parameter and so extreme value theory is a very well established theory for the tit describing the tails of distributions and um it's a three parameter distribution and the so-called shape parameter xc determines the the the upper tail of this distribution of the tail of ordinary distributions and so there's really three kinds the if that shape parameter is zero it's called the gumbel distribution and it goes out to infinity if if the shape parameter is positive it is unbounded at infinity and finite infinity if it's large enough and if it's less than zero then it has a sharp bound and uh you can see that one uh taken from wikipedia for a relatively relatively large uh negative shape parameter the green one of minus one half when when we we we can use this to calculate return values and return times and you see that for the three different shape three different categories of the shape parameter and um uh as return value um uh as return time goes up the the the how rare the event is if the param if the shape parameter is negative and it's bounded then these um plots of return time would a return value rather would also be bounded and one could calculate that um and that would be one way of estimating the problem-assent maximum precipitation so um this is a problem that's bothered me for about 20 years and and it's nice to have this opportunity to think about it again and I went back to a paper that I wrote some time ago that hasn't been published where we had an experiment of 2,364 simulations of just 1990 and so the beauty of this was although it was rather coarse it is completely stationary and uh iid independent and identically distributed and we divided this up into uh four regions of the western united states we got a coastal region a desert region a great plains uh region and an upper midwest and um they were defined based on their mean and very variability characteristics and when we calculate this this use these um the data from these experiments to calculate return value uh compare as a function of return time with these different regions only the pacific coast is a bounded distribution you can see this it's not large 2000 years is not large enough to to estimate um from this picture where it's going to saturate but at least it's turned over and indeed this is the only one of these four regions that had a negative shape parameter so the other three are unbounded with 2,364 years now this bothered me for a long time but I think I finally understand now that that the g e v the generalized extreme value distribution is indeed fit for purpose to describe a long tail but bounded distribution which which precipitation must be um if the shape parameter is is negative but small and so um to summarize these four regions the pacific coast region was bounded that had that had a high max mean maximum a high variability and a high return value whereas say the southwest had a very low mean uh maximum maximum mean temperature mean precipitation but a very high uh long period of return value and then the others had different variability characteristics um this is a picture just describing sort of what what i mean by this thing is fit for purpose this this is a picture that mark research has made yesterday showing um this distribution with small shape parameters and this one of point zero five point zero five is actually what i got for the pacific coast region and so you can see it's got a really long tail where it's really close to zero but not zero out in this case the to 20 in this normalized unit but then beyond that is defined as v zero so when we fit observed precipitation or even model precipitation you know more often than not is unbounded and that's unphysical because the the problem maximum precipitation can't be infinite it just can't be but if it is indeed bounded like it is in this case we can use Bayesian methods to estimate the upper bound to the upper bound you know so let's say like the the five percent significant level you know what is that upper bound and li kung-jong has been doing this with temperatures of those those methodologies would transfer directly um and so i think that in order to get this from climate model days we'd have to have thousands of years of thousands of simulated years depending on the region and these very very different variability characteristics and i and i think i believe now and i'd like to test this more with the large ensembles the large ensemble one of them has 6 000 years it's non-stationary that throws a wrench in their works but i think it's still good but i think the true shape parameter is probably negative but very small you know somewhere between you know just a little over zero and and 0.5 as far as best practices because these data sets the large ensemble data sets are going to be non-stationary we need to have multiple covariates mark brister has written about this recently we can get away with a lot of covariates to account for different forms of non-stationarity both human and natural and also i think these the analyses have to be done on seasonal maximums rather than annual or else the data is not independent and identically distributed especially identical because as we've seen winter storms are different than summer storms and in fact i think it would be better as we've seen today in these other talks that we really should should footprint these storms in these simulations and consider the different storm types separately because there's no reason to believe that the statistics of these storms are the same or that a mechanism to change would be and so you know this this is sort of some cold water on on the idea of using ensembles because the number of years of observations will never be large enough to get bounded distributions and the number of simulated years in convective and permitting models is not going to be 10 000 for a long time but there is this one opportunity that that i think is worth investigating and it's a borrow some advanced statistical techniques that have been used to estimate a maximum earthquake magnitude or the maximum human lifespan you know you can't live forever and earthquakes can't be infinitely large and so there is a there is a technology there that has been applied rather successfully at least in human lifespan i think in earthquakes as well but that's going to require statisticians who are much much better than i am and so thank you all right thank you very much so i want to thank all three panelists for the wonderful presentation and i would like to open up for questions before i perhaps ask a few more general questions that all of the speakers and the panelists can contribute to the discussion so so for now i'm opening up to see if there are any questions for any of our panelists so i haven't seen a hand yet so so there might be some specific questions that come up later but i i'd like to maybe throw out a few questions for the panelists as well as our previous speaker a bit more general but i think it would help us the committee to think about how we recommend the path going forward well maybe jim yeah okay go ahead jim yeah go ahead ruby okay all right uh so i think there are a few questions that we have heard right so so many of you have talked about lower resolution model but also very very high resolution model so my first question is are there ways that we can combine the advantages and disadvantages of this low and high resolution model to really give us a better estimate of extreme precipitation as or or pmp more specifically so so the first question is how do we take advantage of like both low and high resolution model my second question is related to what michael just talked about in terms of statistics it seems to me that we ought to be able to also take advantage of both modeling and statistical theory to help us so so i would like to hear comments about how these two different ways of thinking the modeling and statistics to help us better bound uncertainty and better get estimate of extreme precipitation and then the third question which i i've also previously asked andrea a bit before is can you help us think about how to build the ensemble because it seems like there are many ways to build it and it's very computationally expensive and and i would like to hear from from many of you um your comments about how to build this ensemble um and then i think we also have an important question about black swan how do we get at the black swan or the gray swan type of problem and then lastly i would like to come back to a question related to the super clausius claperon i think it's a very important question because if we do know that the maximum is really the cc i think i think that gives us a very strong constraint but if that is not the case then i think our our task would be quite a bit more difficult because i we don't know like how much higher you know is it like doubling the cc rate or or what right so so i'd like to hear some thoughts in that regard and how we might narrow the uncertainty related to this clausius claperon to better understand what might be a possible maximum uh or the limit of what what that might be so these are some of the questions that i'm throwing at all of you and i would like to see if any of you like to answer any any one of those questions yeah so i see two hands up so jim please go ahead first uh no i was i had another question so for all those who were answered i have another question great okay uh michael is your hand related to yeah it is and it's it's about super clausius claperon and um and perhaps this is some way we could use high and low resolution models you know i i think the only thing low resolution models are going to tell us is changes in large-scale meteorology i i don't really believe it informs us directly about extreme precipitation partly because it doesn't as as kevin showed it doesn't even make the kinds of storms that produce extreme precipitation particularly well um some of them not at all um so that high resolution models are the only way we're going to get at that um but i i think you know what we've been doing with event attribution and and and kevin and i and others have written a lot on this is trying to understand what the processes are you know so we certainly know that saturates fully saturated atmospheres increase at humidity at um specific humidity at clausius claperon and then the question is are storms becoming more efficient at at raining that out and um and that's where i think the modeling really can help us understand if there are physical things happening you know which are probably local dynamics so not large-scale dynamics but local storm dynamics um in the case of um of tropical cyclones it's my feeling that the winds are a little bit stronger so it's a little bit more intense storm we haven't detected that because that's a noisy field and probably we're not looking at it right but that's that's a plausible physical mechanism for why event attribution studies a tropical of extreme intense tropical cyclones more often than not if not all the times show a super clausius behavior clausius claperon behavior of two to three times that that that reading for the most extreme precipitation in the most extreme storms now other storms would have different mechanisms um you know there's been some some positive for for atmospheric rivers by you um and and and for mcs is by by by any i think and so that's where i think we need to use these models to gain our understanding and then develop theories about how we can scale with temperature great thank you michael and i see two other hands off for now christoph and then christian christ yeah well thanks a lot for these three presentations this was a very interesting and inspiring i um like the question you raised ruby about the importance to find out whether the super cc scaling exists or whether it does not and it appears to me that we do a lot of modeling the comparisons and it might be a good location actually to do one more in the comparison and there are different ways one one could do that one could have one model running different pjw versions and one could have but one could also have different models running the same pjw version i think this would help us to understand why our results differ to some extent and whether even speaking we have different beliefs regarding the role of the cc scaling i i definitely think that this could be a comparatively small in the comparison if a couple of groups or one group does these simulations a few years of simulations would be sufficient to find out whether these differences are really due to the whether these differences or the super cc scaling does exist or whether there are differences due to different treatments of pjw we just like to raise one concern i i know of a number of papers including those in nature climate change a number of papers none none of them written by any of you which which use pjw with a uniform temperature in in the vertical and that's just unrealistic because in climate change we will see changes in stratification due to the fact of the moisture robotic changes in moisture robotic lapse rate and so if we just take a temperature and humidity changes and not account for the associated stratification changes we really run the risk of overestimating or underestimating certain things there has been another issue raised i think by paul that about the convergence that's another area where we do far too little work where one could do a set of experiments at different resolutions and trying to find out what is the the the role of resolution it appears to me that the from from my own experience it appears to me that kind of kilometer resolution one two three kilometer or maybe four as andreas is here i should say two four kilometers that that somewhere in this area it's something like a sweet spot i mean not much happens if you if you are at higher resolution the improvements are not so significant as if you reach this kind of four kilometers or so but we don't really know what happens if we go to even higher resolution maybe things continue to change so yeah maybe that these were suggestions for future studies that one could also andreas said one could do that the u.s scientists should collaborate more closely i think this would be a potential really to have a collaboration across continents yeah yeah very well said but um before i called on christine i know your your hands has been up for a while i just want to follow up on uh chris discussion a little bit so so we have heard a bit about resolution but i'm a little surprised not to hear much about physics in terms of the type of uncertainty that might be introduced i i just wonder if any of you might have a comment about that because we know for example cloud microphysics could be really important in simulating convective storms um so so yeah anytime if anyone has has something to chime in on that so so that's also kind of related to my question about how to build the the ensemble yeah all right so i'll go to christine yeah thank you i mean i had some similar comments to both christoff and you know just thinking about that super scaling perspective just one additional we did separate our flash flood cases by intensity of the system so basically by strength of vertical motion and we did see that the scaling increased as we went to stronger and stronger vertical motion system so i think the role of local storm dynamics in these systems is actually really critical particularly in these pgw simulations where we're actually doing a good job of representing thermodynamic changes in climate but not getting storm shock shifts and large-scale climate dynamic shifts so just a comment there um actually on ruby on your comments about parameterizations there's been some really heavy testing of these you know long 13 and 20 and 40 year simulations to make sure these parameterizations and the coupling kind of interact well there are some challenges there we've seen some local challenges with you know tropical versus subtrapical and mid-latitude behaviors with these parameterizations so i think that's a really important and ongoing discussion but it's a really important thing to bring up and discuss great thank you very much yep are there any other okay paul yeah just very briefly on that point i think that there's an opportunity here to use large eddy simulations combined with machine learning models and are kind of bypassed parameterizations and see if we can perhaps validate those parameterizations using the machine learning approaches great uh mike back to michael and michael you are muted yeah all right paul you said the magic words about machine learning um and i'd like to make a comment about that um some of you may be aware of this new nvidia code called forecast net which is a trained version of er a5 and there is a lot of noise about making some hyper large ensembles you know 10 000 member ensembles with this thing because it could be done and i think the there's a lot of unknown questions here you know will you know typically when you train something um uh you you you you you can interpolate a lot but you're not train you're not going to extrapolate outside of the training space in general um and so they're 44 years of er a5 you'd expect that on average the biggest precipitation would be a 44 year return value um but they don't train on precipitation they print they train on winds and um and geopotential height and temperature and humidity comes into play through the initial conditions and so it's actually really unclear to me whether or not we could populate this distribution so we could really get you know black swan events um or will the dynamics that the large-scale dynamics that it's trained on be the limiting factor and that it will just be 10 000 instances of large-scale meteorological patterns that we've already seen i don't think we know the answer to that but it is an exciting new area um that that that bears bears investigation yeah so yeah thank you Michael i think this is a very important point to bring up about machine learning because there are more and more uses of machine learning and i think some some might feel that the type of estimate that we are talking a lot about that pmp estimate a lot is writing on it because some people might be making decisions based on that and so the trustworthiness of machine learning is really an important question right i think these are also um things that we need to consider so i think i saw Paul's hand up uh no it's actually Kristen's first before Paul okay i am actually changing the topic so Paul if you had something on that topic go for it oh i just wanted to comment very briefly on um super clausius claperon as well um so i have to admit that i don't have like a full understanding of the complete literature on super clausius claperon but i think in many instances you get super clausius claperon behavior because you sharpen your individual precipitation systems or in the case of patrick walla and wainer your tropical cyclones basically contract and as a concept and that's related to the fact that basically you have more upward motion and basically the eyewall of the storm so i think further studies on specific storm types in order to assess um under what conditions do you get more sharpening of the system and i know ruby of course you did that work with uh xiao dong chen as well on sharpening of atmospheric rivers i think it'd be very interesting to see based on the scale of the system to what degree we can anticipate sharpening in the future in response to increases in like land surface warming or the vertical profile of temperature i would add i'd love to see that with mcs andy yes the audience that's also to they'll allow us to to do this investigation now yep great thank you paul and then so next is christin and then next is andrea so yeah i wanted to go back to your questions ruby particularly number one the low and the high resolution models and so one of the things that we've been seeing is that um although the you know the global climate models are not representing some of these things like mcs's and things like that what they do well is they actually do get the thermodynamic changes so things like convective available potential energy convective inhibition things like sheet you know they're actually getting some of those convective environments that we know are important in driving the distribution of storm modes across the globe and so uh you know linking in in kind of the thermodynamic space in some of these convective environments i think is one way to bridge that gap i also do think a broader conversation going back to some of christoff's comments bringing the weather and climate community together and in particular coordinating our efforts kind of you know to to do these types of simulations and thinking about what types of ensembles we wanted to do that may be driven by climate models or driven by other types of forcing data sets this is a i think a a really big opportunity that we should actually really start to self organize on so yeah just some comments linking those and going back to number one i'm down and also know you there thank you very much christen uh huh and so next is andreas and then next is dan yes so i just wanted to comment on michael's comment we see this actually in our simulations that the mcs is like the heavy rainfall area is more concentrated in the center which is really concerning if you think about flash flooding specifically but i also want to mention like christen brought this up in her talk translation speed is extremely important as well like anything that happens with translation speed if the storms would slow down on the future conditions and we have some examples of some papers that suggest that that would be had would have a big impact on on pmp yeah indeed important question right because if we are looking at a local location and the storm moves slowly then it would done a lot more precipitation and and so so andreas in that sense do you feel that we have some theoretical understanding about why the translation speed might be changing so this is like in our simulations the translation speed at least of mcs didn't change significantly i know that there's some literature on tropical cyclones that seems to prefer to stall especially when they make landfall right we know like the the shear environment is changing in the central uras in the future it should be less shear which could have impacts on the translation speed and potentially slows things down but they can also have impacts on the organization of storms so these dynamical changes are really important as well like i think we have a much better handle on on the other thermodynamic changes than we have on the dynamical changes yeah i'll just add to we've looked at south america right the subtropical south america hot spot and the zero six kilometer shear there in the large ensemble actually increases in contrast to the us so these things vary globally and and also are very these duration and intensity frameworks yes so we have more challenges great thank you okay so next is stan and then michael ruby my question really takes uh things in a different direction and so i don't want to cut this discussion off prematurely so so michael are you responding to some of the early comments i had a short comment about storm translation speeds jim kosten has demonstrated in observations that tropical cyclones over the united states what's after the big landfall have slowed down and there's weak to moderate evidence supported in the 25 kilometer scale models that this is happening not at every latitude but unfortunately at the mid latitudes of where we live probably not so much in the tropics but i don't think there's actually any any reason to suggest that the kinds of stalls like we saw in hurricane harvey are are more or less likely only because we just don't have enough data to say they're just they're only two stalled storms in the whole atlantic hurricane record and and it doesn't seem to happen very often in simulations either so i see also hands up from paul and kevin are you both also commenting on the translation speed or yeah i just want to comment very briefly that uh translation speed is one of those areas where the pgw simulations don't really help a whole lot so that's where we definitely need to rely upon potentially a high resolution ensemble but as michael alluded to it happens so rarely that you're going to need 20 years of simulations in order to do it so it's going to be an outstanding question for a while i think yeah and kevin are you also commenting well i was just i was actually going to say exactly what paul just said that that's one of the downsides of the kind of storyline approach but i was going to also add to the discussion around really quick the the clausius clapperon and and there has been work that has suggested right um and actually some of it is by one of christin's postdocs right now a former phd student in my group which which there are different ways in which different scientists are trying to break down changes both from the thermodynamic component from warming in things like tropical cyclones as well as trying to look at changes in intensity or changes in spatial scale of the storm and so there has been efforts to start to quantify if you do see a super clausius clapperon change in some set of simulations or over some set of data set to try to quantify what percentage of that is coming from a thermodynamic change in the sst what what percentage of that is coming from you know the change in intensity that you see if you hold everything else constant and so there is some work to suggest that when we do see the these kind of super clausius clapperon changes particularly in tropical cyclones that it is coming in part because there's this additive effect of a change in the the circulation but that is mostly true in i'll just mention right in model simulations of the tropical cyclones and i think there's less evidence for it thus far when we've tried to look at it in the dynamical sense or sorry in the in the dynamical sense in in observations in part because i think as michael alluded to earlier i mean you don't have full representations of the full 3d circulation for an observed storm you and you just have these you know opportunities when i'm talking about hurricanes now but when you when you have the hurricane hunters that go through at specific levels and you have snapshots of what it looks like but kevin i would like to follow up on your comment that when you see super clausius clapperon in tropical cyclone you said is more related to circulation i think there are two aspects of circulation one is really changes in the larger scale circulation that might have changed the tracks of the tc and things like that the other part of the circulation is actually the vertical velocity which itself is actually driven by thermodynamics right so so which part are you talking about so when uh elissa standsfield has done this work she's focused on um actually using the dynamical measure of intensity of a storm so whether that's like azimuthally averaged horizontal rinse um but but looking at you know under the same when you have storms that are you know at the same ssts but in different climates or whatever um you know can you start to capture how much of the change in rainfall is due to the fact that the storms are actually stronger so this link to the vertical so it's more on the getting to the vertical yeah so it's convergence and then you have your your vertical after us great thank you very much i think we should get to then now then yeah um so as i said this is taking it in a very different direction um last week we were dealing with stakeholders these were people who had to tell uh dam owners how how big to build their dams and i think they would look at this discussion as as largely academic and we heard some pushback last week to using climate models at all um there was there were comments of that the uncertainties were too too too large to be useful there were different projections there were differences between the models um and i think that uh they would probably say if we if we need to integrate climate change into pmp keep it very simple keep it use clausius claperon or use the scaling factor that that christoph talked about something something very simple um so i'd love to hear some pushback from the climate modelers themselves of what value does using the models bring that a simple approach couldn't use uh couldn't couldn't uh what value to do the models for it very good question so first of all i want to just mention so jim i i know that you have a different question but but since we are also kind of branching out on different questions i would come back to to you so that you have your chance to ask your question but so michael are you responding to then uh question yes and i would agree we can't use models alone we need to use our brains and and what i mean by that is we have to understand what's going on we have to have we have to have not only model results but even more importantly we have to have good theoretical foundations about what's going on you know kevin just mentioned some of these things you know changes in in vertical motion you know causing a mechanism for super clausius claperon or changes in translation speech or whatever it happens to be and it's obviously going to vary from storm types and we'll get some of that information from models but most of it we're going to get from each other and and and and hashing this out so it's it's it's not the models it's the modelers and the scientists and i think that's why that would be my pushback to this community that that particular community is you know we don't believe the models anymore than you do you know but we use them as one set of tools right okay so uh paul um i i fully agree with this um stakeholder statement honestly uh and i think that um christoph was very correct in saying that when it comes down to actually using the data having a multiplicative factor is going to be the most easily communicated means allowing folks to adjust to the future and i think that the use of models is to allow us to actually have confidence in whatever multiplicative factor we actually come up with like when we say that 1.4 and we say it's derived from clausius claperon do we have a response when somebody comes along and says well how are you so confident that the change in precipitation will be constrained by clausius claperon and so we can use the models to explore the space of possibilities and see if there are instances where that 1.4 factor may not hold and that deeper understanding of the the underlying processes will then give us greater confidence in its employee in uh impacts thank you uh andrea i also fully agree with the statements i think we really need multiple lines of evidence like whatever tool and this could be machine learning statistic statistical methods observations and often those tools have um different strength and weaknesses so really using all of the information that we have um is is essential and i also want to defend the models a little bit a little bit here because i think it depends which type of model you're talking about if if stakeholders only work with c-mip type models i fully agree like but this this high resolution models are different type of models if they believe in weather forecasts like if they get a warning take care like in a week your dam might spill over because the high resolution model puts a lot of rainfall there they should also believe in this kind of of modeling that we are using because it's the same type model um so it really depends here as well and and one more thing i think your task not only for changing the future but also looking at current climate estimates isn't it so i think modeling has a has a role to play here as well so taking into account in the current climate what the pmp is in addition like paul made this comment as well like observations have large uncertainties and there are really big regions in the us that are very badly observed um and modeling can definitely help to fill this gaps thank you uh christian and then uh christoph yeah i'll keep my short i was about to say almost the exact same thing andreas just mentioned but i will say that it matters what stakeholders were talking about if you're trying to build one dam in one particular area maybe it's complex terrain right the way that you would look at a gcm versus a convection permitting model estimation of the future is quite different than if you're trying to engineer like the you know the levy system in the mississippi river basin right these these things are 30 years out of date we need to think about these broader scale things so i think the what the stakeholders are looking for and what the strengths and weaknesses across these tools across skills i think all of this should be considered and i also think we need to do a better job of communicating our uncertainties in these different modeling frameworks as well uh to better bridge with those stakeholders very well said thank you uh christoph can i just comment on that okay hi not just not just convey our uncertainties but also convey our confidence yes thank you thank you yeah uh chris yes i i i think i wanted to say something about combining models and scaling and i think that's something we have to look into there is some evidence in europe for instance where we have the mediterranean amplification and dramatic reduction in precipitation frequency forecasted by the gcms and rcm at any resolution in the medical of the mediterranean during summer in particular and so it might be there might be ways maybe to combine changes in frequency taking these the kind of information from the models which will depend also on changes in circulation which will not be included in the scaling and and changing scaling which can which can really be based on a physical argument based on the thermodynamic processes so there might be ways that should be explored whether one can combine these kind of different pieces of information great thank you very much so now let's go to jim sorry jim you have been holding for a while yeah well um my question kind of ties back to the long history of probable maximum precipitation and its heritage uh was a quantity that meteorologists would develop first principle assessments of maximum precipitation and my overly broad question is whether there are ways of of thinking about that problem thinking about bounds physical bounds and i think it would would serve a variety of roles but one would be providing a physical process context for studies so any thoughts on on studies that directly address upper limits to precipitation 1800 millimeters per hour in one minute how do you do that and can you do more yep any any response very tough question yeah um i've always been enamored with this study by martinus villalobos and neland from a couple years back which is entitled wide precipitation intensities fall tend to follow gamma distributions where they use a very simple physically driven model physically justified model in order to derive probability distributions associated with precipitation intensities and i think that line of thinking could be taken further in order to um in order to get better statistical distributions of precipitation intensities and basically what they find in that analysis at least for somewhat longer term precipitation events on the order of a day or so is that you have a regime where you have a gamma distributed precipitation distribution and then you have a fall off region um and so i think one of the difficulties in the statistical analysis of the precipitation distributions is getting the point at which that fall off region really begins all right thank you um michael i'm going to defer to christin okay christin yeah so just a couple comments it's a really good question um and in thinking about kind of the dozeball ingredient space framework for flash flood forecasting which is intensity and duration pieces of that equation include things like precipitation efficiency and vertical moisture flux and so things you know thinking about how those particular parameters uh you know may shift in a future climate thinking about you know what is the actual maximum amount of rain that we can get out of the atmosphere some theoretical kind of studies on that may help some of that i know there has been some work looking at precipitation efficiency i think we need to do more on that uh but i i think it's a really hard question i think we should all think about how to answer that question with the tools that we have available to us yeah and i'm also curious whether the answer to this could also depend on the storm types that that we have been discussing all along yeah if you're in the tropics or the sub tropics or the mid latitudes you know these things would would you know these things would change so yeah great thank you any other comments i think this is indeed a very important question right so so so the maximum the absolute maximum in the present day as well as in the future michael yeah i wanted to wait until i heard what chris had to say but it is an important question and and i think jimmy you raise it you you pose it in a different way than we've been talking about you know about instead of looking at precipitation look at the drivers you know like you know that we're that that originally went into these calculations a long time ago and and you know perhaps these kilometer scale models have the fidelity to tell us something about that and that might be a more tractable question you know by framing it based in the old way and looking at the inputs to that calculation and say how do we think those are changing you know we certainly know the moisture is going up by clausius clavron but how are these other factors changing and and that would be an interesting thing to do and we probably have enough model output from christoph's calculations and andrea's calculation to to make some headway great back to christa i'll just make one quick comment too that you know i show a table in my meteorology class on you know the historic rainfalls right the the rainfall records of the of the global um you know distribution and a lot of the rainfall records come from cheripunji india in like the 1800s so the question is going back to how well do we trust those observations right we saw some really great today about challenges and observational networks even today so you know some so this brings back to the question you know how how trustworthy are some of those 1800 you know metrics of rainfall records that's something also to consider in this in this question yes thank you uh obi uh yes let me let me explain my my issue i think uh i think michael mentioned you know number of models that are needed to do the estimation statistically and you know the upper bound and you know the shape parameter is very poorly estimated in some methods and you showed you had to have a very large number of models now given that i think from christry heard that you know observation based estimates just don't go to very rare extremes because we don't have enough data so my question is are these you know two questions is there any other alternative but to use a combination of data and models and the other question is do these models the high resolution models have the right up right type of physics to sort of so they can be driven to generate pmp like you know estimate i think it's ruby's question like do we have the right physics in there and that may be my lack of knowledge on the model so great so obi i think you're asking two question number one is how can we combine data with model to better take advantage of both oh is there any other is that the only way because given the non-stationarity and climate change right uh huh uh huh yeah and and that goes back to also tie a little bit to to one of the questions that i gave uh the panelists is how can we take advantage of both models and statistics to help us maybe get that that upper bound or constraint right and then your second question is related to the physics yeah so i would like to see if anyone okay so i think paul's hand was up first yeah i just want to address that question about the physics if i was to rephrase that i would ask are there any vertical distributions of relevant meteorological fields like aerosols and water vapor content and cloud water that give rise to anomalous precipitation regimes and so an anomalous precipitation regime here is one which is like far outside the norm of what we would expect from precipitation uh and so in order to make an argument that the physical parameterizations don't capture pmp type events we would have to make the argument that pmp naturally emerges from an anomalous precipitation regime and so i'm not sure what the literature necessarily says on that subject but i would suspect that you could we could make an argument that the parameterizations probably capture the the relevant physics correctly oh there's gonna thank you andrew yeah i wanted to comment on exactly on this as well like if we look at our simulations also the weather forecasting simulations that we run at ncar like this very like we had a range of very intense rainfall events over the last couple of years like in the order of 500 to 1000 year return values and those are very fairly well simulated off like it depends on the large situation how predictable those are but harvey for example the west virginia flood that i showed you the kentucky flood um tennessee that we had last year two years ago like this were all pretty well captured and like especially if you look at the ensemble the mean of the ensemble intensity was pretty close to the observations which gives me quite some confidence so but i think also what pristoff showed like we really have to look at how if we really are interested in the in the physics we have to look at fairly short rainfall records like 15 minutes or really maybe even higher resolution this gets challenging with observations as well because if you look at daily accumulations you can get various ways to get to higher accumulations by raining over a long time a little bit or having a really big system that drops like so i think if you're really curious about physics we have to really look really closely at the process level and i think there are some compensating errors that we have in the models which work in our favor but i think overall they do really good all right thank you any other comments related to um both obis question as well as if we want to go back to some of the previous questions like from jim and some of the earlier ones or particularly i don't think we have heard too much about combining data or statistics with model yet now michael yeah i wasn't going to comment on that specifically but rather on something that pristoff said about um inter comparison of different models i think it's something that we really should do and you know we have we know we know how to do inter comparison um we should should think about what are some interesting experiments you know either short term forecasting which means we could do it relatively easily or or else or otherwise i think that's an excellent idea and and use the e3sm the screen the the the icon morph whatever have you and and and not just one but maybe multiple uh uh uh inter multiple situations to inter compare so it so this can be in the context of model in the comparison as well as in the context of generating ensemble right yeah and you know the more things we do the more we learn i mean the office great so i think the next yeah the next hand up is andreas and then paul yeah fully fully agree with this point and i think that even idealized simulations like if we can break down everything to get to the core could be very interesting to run with this different modeling system as long as we can do this i think one really important point and pristoff brought this up as well like we really have to make sure to use the same statistics to look at the the quantities you're interested in right if you look at the literature you can almost get anything you like by using like dry days wet days use some local viewpoint or higher level temperature like we really have to work together to make these studies comparable because otherwise like it's confusing it's confusing even for us that work in this field but we mentioned how confusing it must be for like stakeholders that want to use the data yeah and sometimes these details are even lost when people read the paper or maybe not even reported yeah good point paul yeah i was just going to say as part of the hyperfastest project we have a big component of the project being the development of testbeds which are effectively individual events where we simulated with this with several models and try to combine try to compile a consistent set of observations that folks can then use to evaluate their own modeling systems and so we have a recent paper out on the 2012 North American Duresho that you kind of saw as one of the slides in my presentation but the idea is to expand that to six or seven different historical events of different types and have that that data set of consistent observations available to the broader community in case they want to run essentially the same simulation and same uh evaluation right thank you any other comments or we could even ask new questions yeah if anyone has new questions to ask between the panelists or yeah ruby i guess one of the questions you posed that we haven't talked about is that black swan right what about the events that you know we don't currently see in the current climate but may happen in the future climate just bringing it up in my list yep uh huh right yes uh huh i think it's a particularly hard problem you know the suitable for example right we take existing events and apply thermodynamic shifts right to see how those change but when you think about a climate model right you know while the thermodynamic environments are well represented and we have storm shifts and storm tracks how do i identify those those black swan events that then we would downscale i think that's a really tough problem but it's a really interesting one in representing potential extreme precipitation that we don't the spectrum of modes that we see in the current climate so right yeah so this is also partly related to an earlier question that i asked andre is it's like if we select a set of storms to do the storyline type of simulations i mean how do we know that those are the storms that will produce the maximum amount of precipitation in the future potentially that even the storm types could change right the the types that produce the the maximum amount now may not be the same storm type in that location that would produce the maximum amount uh michael i found it amusing you raised that because i talked about that today in the other lecture i gave oh we had a project that at berkeley lab led by provot for using machine learning um pattern recognition uh technologies i call it the facebook app where we could identify storms um by training um so a supervised machine learning technique what we never got to though was the unsupervised machine learning pattern detection and and that might be the way to find these things and let's say you have you know multiple decades of um of high of kilometer scale or two kilometer scale simulations that's a a lot of data you're not going to be able to watch movies to find these things that i've tried that um you know so having some you know exploiting these pattern recognition algorithms in some unsupervised way could find the things that we don't you know the unknown unknowns as it were yeah i like that a lot michael and you know some of the flash flood history that i showed in my talk comes from the you know the big challenge of predicting heavy rainfall it's really hard and there's so many different storm modes and synoptic modes that produce flash flood rainfall i see russ on russ schumacher on the call here his group has done some really exciting work with kind of short-term heavy rainfall predictability using machine learning um and i think that you know applying some of these techniques to these longer time scales may help us identify some of these situations that fall outside of what we currently observe and so i think there's a lot of merit to what you just mentioned michael and it comes into the context of you know how forecasters actually have built these pattern recognition types of frameworks because of the challenges of heavy rainfall across the spectrum wonderful any any other comments related to a similar topic on black swan or going back i or going back to the one about statistics combining statistics and and modeling i think these are the two questions that perhaps we um could hear a little bit if there are more suggestions of how we move forward that's a really hard poll okay yeah city poll has an answer well i mean we just need a meta analysis right basically you have statistics and models are two different ways of trying to tackle the same problem and so we just do the sure what at all thing and collect all the evidence and see if it allows us to constrain the uncertainty bounds the the the issue is that you know if you were just to calculate return value on 50 years of data with all the covariates you have when you go to say the thousand-year return value the uncertainty bounds are pretty large the i actually i didn't mention this in the pictures i showed there were two sets of error bars and the the wide set was just an empirical estimate of of the return values and the the the narrow or blue bars were that from the extreme value analysis and so extreme value theory if nothing else it didn't really change the the the mean estimate of your return value but it tightened the error bounds on it and so you know that part's good but you know you know in the u.s. we do have some we have quite a bit of data that is over a hundred years that is that that that would pass most quality control as christin says in the 19th century you know it's a little wonky but not everywhere i mean you know maybe some places great thank you uh andres uh like what came to me was really to to try to trade space for time like we do this with the stochastic storm transposition like the the pmp field depending on where you are if you're in the mountains it's way more complex but if you're more in flat terrain it should be fairly smooth i would say so you could try to to get better estimates by really doing the statistical transpositions in a smart way and i know that there are people out there to do that and i saw some fairly encouraging group on that in indeed andres um my my esteemed colleague mark riser has some new papers on that over the united states um where we we we use multiple covariates um uh seven different ones including modes of natural variability as well as greenhouse gases and aerosols um and spatial statistics um uh there's a concept called um uh tail dependence that um we've also been exploiting that our other colleague li huong jang um i can send you some of those papers if you're interested it is powerful and and and and um we haven't looked at it they're really extreme really getting more in a detection and attribution sense but that that actually is something i'll bring up with mark next week when i see him all right any other comments or or new questions okay i'll just wait one minute to see if yeah if there are new questions that we have an answer i would have expected some questions from dan uh cooly or statistical um specs andres this is more a question for you what are the next steps that you're taking now like how do you incorporate this information into what you're doing yeah yeah yeah indeed in my closing i will okay very briefly talk about that okay yeah huh all right any other questions okay so with that i really want to thank everyone the speakers the panelists for really lively discussion i i learned a lot and there's so much so many new points and so many new insights that we have heard i think this is really wonderful and for for all the time preparing uh the presentations definitely those are really high quality and very um useful presentations so so with that i want to thank everyone again i also want to thank the committee member for joining and participating in questions and and and a lot of great discussion um so first of all i'd like to say that there is a recording of this session which um anyone can go back to uh to to view and and in fact even the previous two information gathering session you can you can go to our committee website and and review those um and then also the event website has a link to a form where you can contribute perspectives as well as insights on the session's guiding questions we really welcome uh those who um perhaps haven't participated but would be able to provide some comments and perspective and if you visit the project website you can also find the links to the event pages for past event including recordings of those events etc so what we would be doing next for the committee would be to review all of these information that we have gathered from the information gathering sessions three of them and then we would have more meetings convening um among the committee member to review what we have learned and ultimately um we would be producing a report uh based on that uh based on all of these information as well as uh deliberation by the committee um i want to maybe also call on Jim to see if Jim has any um perhaps a comment or two related to the next steps since Andreas asked about that um ruby just one comment um the committee is tasked with addressing these four specific tasks um not to address other issues um but to fully address each of the issues and provide recommendations uh to know and that's what we plan on doing mm-hmm great yep and you all have been helping us a lot i think we have heard a lot of important points that that and and also new insights that are that are very useful so with that i think we would be ending um thank you very much all for joining in especially also for Kristoff and Michael who are joining from Europe um and it's very late over there um thank you all again and have a good day bye