 Okay, Brad, do you want me to get started? Go ahead, Marshall. Okay, so Brad and I have tagged team on moderating this session. We want to thank you all for hanging in there and this very productive two days or day and a half. And I know that many of us are looking for sort of a synthesis discussion. And so we've invited select members of the week to participate in this round table. Julie Demuth from National Science Foundation's supported National Center for Atmospheric Research will be a part of the round table. Brought on from HAAS Alert. Gina Esco from the NOAA Weather Program Office. Sherman Gilliams Jr. from FEMA and Rebecca Morris from the National Science Foundation as well. And I believe some of these participants are virtual and some are in the room. And so we just wanted to sort of think about the broader sort of conference in totality. And so I'll cede the discussion with a question and then my colleague Brad will manage the discussion once we get going. But I guess I'll throw this out and anyone can tackle it. What surprised you most from the workshop? I'll be happy to start. You know, I think that one of the things that I find so fascinating is again not to harken back to my English degree. But so many of the challenges and communications always come down to message medium and audience. And I think that the more sophisticated our understanding becomes of the content that we are seeking to communicate. The more complicated those questions become. It's one thing if, you know, you're just trying to tell people there's a storm coming. But the more that we know about the storm, the more that we can communicate and the more that those, you know, the audience that you're communicating to and how you're communicating it and how they're going to receive it and what the implications of that communication means. There's all these sort of follow on, you know, secondary tertiary impacts around it. And the more opportunities and methods we have available to us to communicate. The more complex those decisions become. Right. So if you're just limited to a radio broadcast for everyone, and you have a like a single language in that community. In many ways, your job is a lot easier, but the more tools you have available to you now you can communicate over television and over social media and you've got multiple ways of delivering that message to different audiences and you can communicate at different times as you approach this particular hazard. And so I think that this, this is only going to become more complex and our tool sets are only going to need to become more robust. And I think that especially for the experts in this room and the, and the people who manage emergency communication. I think that it's an illusion to think that, you know, there's one message that we need to get it out there and we got to keep it simple. I think it's only going to become more and more sophisticated as we go. Anyone online or go ahead. Really go ahead. Yeah. I'll keep this pretty quick, but I think actually what surprised me maybe builds really nice and what you were just saying. I really think it's interesting and Sherman your comments were so powerful in this respect as we've been talking so much about personalization and hyper localization how important that is. But to me, it seems like there's a tension between doing that and everything we're discussing about communicating uncertainty, especially at longer lead times because we can't give that hyper localized hyper personalized information. I think there's more and more tools to be able to do that. I think also more data is our fingertips to be able to do that. And so I'm not quite as articulate everything you were just saying Brock, but I think I think this is a future and it is a direction that we're really emphasizing but how do we do this at the same time that we don't necessarily have the skill to be able to do this at the individual level, whether it's spatially precise temporally precise for a certain kind of hazard for a certain kind of need. I think this is a bad thing maybe I think Gina said this beautifully a challenge is an opportunity but this is a challenge and I think recognizing the potential tension of this challenge with some of the, the, some of the issues and the difficulties of being able to do this is really important. I'll just jump in to say it's interesting on the one hand. I'm not sure anything shocked me, but yet I was so eager to learn more right and hear more from everyone that was in the room and virtually online. I think my one message that I'm sort of coming away with and this this comes from working on a project where we put simple in the title it was called hazard simplification. We're going to quote simplify the National Weather Service watch warning advisory system. The one thing that I will say the one takeaway from that is simple isn't easy. So even though simplicity is desired. What I heard today was a lot of complexity to get to that quote simple message and so I don't want us to conflate simplicity with ease or efficiency or easy right because I think that achieving that parsimmonious simple message is actually quite challenging one trying to understand all of these unique audiences understanding the forecast and having the agility of that forecast to meet all of those unique needs. I am excited about that. And again I guess I'll reiterate my point I don't think any one of us can do it alone and so I think what what doesn't surprise me about today but excites me about today is how do we take all of this and move forward. How do we how do we empower each other to learn from one another and work together moving forward right. I think actually Jen Henderson of China look at all my notes from the last couple days. She made points in her talk earlier yesterday gosh I feel like it was two weeks ago now just because there's so much information about our organization set up to study this. Are we to siloed and what can we do to work even more collaboratively together to achieve these shared goals. I think Gina really powerful. Rebecca or Sherman. I'll chime in here I think like Gina I'm not sure anything surprised me but I think one thing that really struck me was the diversity of perspectives here which I knew was going to happen but also how everyone's working in the same direction, and also the depth and each of those perspectives it was just, you know, one meteorologist and one social scientist working at no one one person from a, you know, communication company and one researcher. We really had a huge amount of depth in all of these perspectives, and that really, sets up well for the future as Gina said because each of these perspectives is so complex into these contributions to addressing these issues is so complicated that you need multiple people from each kind of area at the table and so. Yeah, I mean the kind of depth as long with the diversity I think was really striking and then even if people didn't agree on the specifics I think everyone was really working in the same direction. As far as communicating information better helping save lives helping reduce other impacts all those kinds of things. Thank you. Well, I just got here an hour ago so I don't think I should be remarking on what surprised me but I will say from the little time I was able to observe. I hope the resonates is that we can't afford to look at reaction as a way forward preparedness is the first response and the way you deal with hyper localized decision making is through. Having that thought process happen before there's a disaster what are my options if I live in this area. If you know, and that way I you know that that paradox between safety and certainty. With disabilities we live in an unsafe world all the time we're used to it. There's a lot to scare somebody who is broken their neck or has to navigate with no sight and all those things it's the safety part that they value more than the certainty I mean the certainty more than the safety but but having these conversations before. And it's hard because it's like talking about life insurance there's no real great time to talk about it but there is a worse time. When you're on your deathbed right and when you're talking about communication with these communities on the eve of a disaster or right before landfall that's the worst time. So how we hold local you know emergency management entities responsible for having these conversations how do you get people interested. You know we spent last year trying to get people in Chicago to think about heat in a different way. We had a day at the beach is great but there are people are going to die in this heat, unless we begin to talk about it in a way that resonates so having those discussions prior to is probably the only time. You're going to be able to contend with what will happen during that decision making matrix that will run through when the crisis is upon them. Okay, thank you. I'm a little prerogative as moderator. What I'm going to do Sherman is turn right back around to you I think we all found this afternoon session on accessibility, pretty compelling and interesting. So I'd like all the panel to give some thought and share your thoughts on what is a single most important thing that we can do to make future risk communication more inclusive. And start with you Sherman. And start with the relationship. I went to Tennessee and I was, you know, I was looking at all the response and all the different entities and I didn't see a lot of volunteer organizations, the big veteran community so I thought well you know you should at least call the you know the local VSOs and things like that. But they didn't have a relationship and it turned out that they were invited to the table. And the stakeholders simply didn't take them up on the offer and I don't know if that was because they didn't trust them. Some of his resources people don't have time to do anything for free. Right. And you're not going to exact you know you're not going to hire a bunch of people come in and do all this stuff but when we see exercises happening and there's no participation. We got to be the what's the barrier there. Do they do they feel like their, their input will be valued is this process really to prove that everything's great, or are we really looking for gaps. And are we really going to listen to people who are not going to have great things to sell but will learn. I think with this community as a whole it's hard to it's a hard little swallow when people tell you you're inadequate but if you listen to stakeholders long enough they'll tell you what's going wrong and bring it up to the table it's tough because it is trust booming, especially in areas where there have been failures in the past. But I would say a great amount of investment in trust building and constantly having them invited to the table. It's got to be non stop. And it may go on forever that way but but that I think that's what starts. That means we know you're up next so. Um, this is a great question I think. I think probably being able to take a broader view about what are all the barriers that people are facing I think Jeanette just emphasize this when with her comments here about understanding how people are making decisions about there's some uncertain hazard that's going to hit me but also building on Sherman's comments but I'm certain that if I go here if I go to that the the I can't remember the name of this the San Diego football stadium but that that is going to not be good for you right we were talking and at the table about the Paduca tornado and how people who are in the candle factory, how they didn't take shelter but they were told that if they did take shelter they would lose their job so they're facing kind of this trade off of an uncertain tornado hitting them, versus an uncertainty, the certainty of losing their job, or if people might, you know, evacuate from a hurricane they might lose their jobs or they might not have income, maybe they don't even lose their job they aren't they're going to lose some income. I think being able to understand what those barriers really are so that we can address them we have some other work. Also in the tornado context where people are really aware about the tornado that's bearing down on them but they would say I have no safe place. I actually don't know where to go. Right, and when they're having to make these decisions I completely agree about preparedness is the best response and you need to help them think about that ahead of time, but also in the moment, if they don't know what direction the tornado is coming from maybe it's at night maybe they only have a couple of minutes Rebecca talked about this beautifully earlier about, you know, in many cases people are making decisions on very very short time scales it would be ideal if they don't. But that is the reality and so what are some of those barriers and can we figure out how to speak to all of those. I think we'll make things more inclusive. Excuse me I want to be mindful of Sherman's I think excellent guidance earlier in in our session around the sort of mission creep around the word accessibility here so I want to first think about the disabled community and, you know, handicap community and the broader sort of spectrum of people who might lack access to resources to capabilities that a lot of people take for granted. And I think that the, the way that you solve for accessibility is inclusiveness as you mentioned here, I think that there's a degree of understanding that everyone is a permanent part of local communities and that is Sherman also mentioned you can solve for this ahead of time by bringing those stakeholders to the table and including them and the sort of plans you have around sort of communication and methodologies of getting people to understand risk. In Haas Alert, we had an eye opening experience in the early process of designing digital alerting, where we learned from the deaf and hard of hearing community just how valuable what we were doing was and frankly we took for granted the fact that there's a lot of drivers who never hear sirens and giving them a conspicuous visual understanding that there's a hazard ahead of them and time for them to anticipate it is such a life saving tool for them in ways that that we didn't even factor in and now they are permanently at the table with everything that we do moving forward because we don't want to take that for granted. And so, you know, one of the benefits of the time that we live in today I think is that we can see these various groups more they're more visible to us than ever, and, and we can be intentional about including them I think. And, and we get to a point where, you know, we check boxes and we think that's the solution. Even within a disability community it's very diverse I sell time. You've got 61 million people in the US with disabilities. That's 61 million ways of living with the disability. It's different for everybody and what I learned even as a person in a wheelchair is, you know, for example, for somebody who's deaf in a disaster at night, the importance of light reading mouse isn't, you know, forget about the flashing lights and everything's coming having a flashlight reading mouse having first responders understand I'm talking to somebody and have to have a light on my face because they read lips. You know, children with autism are attracted to water bodies of water. That's the highest rate of drowning the cause of drowning for children with autism because they see water is really abstract and don't sense the danger so having even a nuanced understanding within a community somebody like me won't even have all the answers so Checking the box because I'm at the table is not going to do it. It's got to be so inculterated that you even start to see differences within a disability community to a point where disability wasn't a point. It's the differences in experiences. That was the whole point of having those different voices at the table. Gina or back at Gina. Yep. Sure. Thank you. I'm going to put my program manager hat on to you know how to we how do we make risk communication inclusive. You prioritize it from from a NOAA perspective. It goes back to the question that Castle post in his talk how do we know when we know enough from research to influence a change or to recommend a change to our operational partners. And it's not good enough for us to just quote trust the samples we need to know who who were in those samples right where all diverse audiences included did we check different vulnerabilities did we check different diversity right. And so from from our perspective it means having gaps analysis of our R&D. Where have we studied what geographies what people what populations what unique abilities right and what haven't we studied right and make sure that we tried to put all of those types of audiences and make them a priority right. It's not good enough to just suggest it. We can make sure that our research really includes them and I'm happy to say with our fiscal year 23 we really did focus on diversifying in our academic community really answered the charge. And we've really diversified our portfolio projects not necessarily all on tropical cyclones, but it is broadly on risk communication deaf and hard of hearing migrant populations pregnancy and heat. A variety of different unique situations. And so I'm, I'm proud to say with, you know, within the R&D portfolio we have, we are making it a priority and I think that's one way to include is you include the act of actually doing it. And I think, as a community, we've done a nice job not to say that we don't have more to do. There are always more opportunities but I think we're, we're beginning to answer the charge. Yeah, I'll just chime in quickly to say a building on what the recent speakers have said, especially Gina and Sherman. Really the importance of knowing who you're trying to include and doing the work of including them, including them in the conversation. And including the diversity of perspectives and then really listening to the complexity of their situation and not just kind of assuming it's just one thing and one solution will solve it or just a simple thing that's going to address a bunch of issues. And then the thing, other piece is to keep on listening because things change people change or technologies change or situations change. And so what might be the most valuable thing for a group. Now, there might be some nuances that are important to keep in mind that, you know, come up later or once you address one big thing, then maybe the next things behind that can come like putting the flashlight on your face. And for that that's like the first level but there are probably a lot of other things and so to kind of keep listening and keep including people in the conversation. This is more methodological but I feel like it's important to say, I think everything that folks are saying is functionally saying this but I want to be really explicit with it. That this qualitative work with people in the communities is so essential I stood up here yesterday and I talked about the longitudinal survey work that we've done. But that was all built on a lot of preliminary qualitative research that we did Jason showed some really powerful quotes yesterday from some of the qualitative work he did. Jeff I think it was yesterday was talking about what functionally are people's mental models when he was talking about people who heard about the rain, the total amounts of rain over a few days from Hurricane Harvey and people said 35 inches over five or six or seven days I'm going to divide that. How could we possibly know that that's what people are thinking, unless they're you're in the field of course that was after the fact to understand that that's how people are processing information if we don't know those things if we don't know really richly and in the complex ways Rebecca and Gina and others are talking about this, how people are processing kind of what some of those issues are that they're that they're facing. We can't design some of the bigger like n larger number kind of quantitative work that we're doing to measure that so just really want to put in a strong plug for all the important rich qualitative work that is done in this community to great. Thank you. Marshall I'll take one more and then I'll pass it off to you want to get ready for another part of this session that half second half is implications for the future. So what I'd like to do now is working with panel we'll start with our virtual participants first is what what is that big takeaway from this week what what are you excited about moving ahead with where are we going from this workshop what what is your big takeaway. Rebecca do you want to start and then Gina. I was hoping Gina would jump in that's a tough question I mean there's there's so many options but I think one big thing is. I think we heard people talking about how the hurricane hazard or other kinds of hazards kind of fit into the world that they are in that they deal with every day. And then we heard from the people who are the meteorologists how the other things that technology is social science communication fit into their world. And I think that kind of the biggest takeaway for me is the importance of bringing those perspectives together to have everyone in the same room is really important. And we've made huge amount of progress in terms of each group knowing each other's perspectives but to make sure that the work that each group is doing is contextually relevant to what everyone else is doing. And that is hugely important as well as tacking back and forth between kind of the simplified context like work where you show someone a message see how they respond in a simplified context and learn from that but then how do you bring that back up to the real world. And then how do you take the questions from the real world and and ask do research or understanding or you know build systems to address them in a more focused way so that kind of tacking back and forth in the full complexity and the different perspectives or the simpler context. That was very general broad I guess but I think that we're finally at that point where where there's enough people in the room, you know to be able to do that. Great, thank you. Gina. Yeah, I have to go back to my notes here so I think I have two major takeaways and you know looking into the future. One is is bridging research and practice practice to research. Again, I think there's so much value I think sometimes we study our partners, but we're not necessarily really working alongside them. And I think that's a challenge like I want to challenge myself, I want to work alongside an emergency manager. I don't want to get in the way, but I feel like I need to be in the field I need to know what you're hearing feeling and seeing right because I think it's really hard. I think that the research community is always trying to emulate we hear a lot of the challenges from our partners, be it operational meteorologists or emergency managers, for example. And we take those challenges we turn them into hypotheses and we try to test them as best we can and replicate that that real world environment, but it's hard and so there's always some gaps between research and practice practice and research right. I think we need to continue to bridge together, but also that knowledge transfer, if we have learned things from research, you know, if we're not, if they're not being utilized, because of either trust in the research or believability or confidence. Right. Well, then, in my opinion, research that isn't used isn't particularly useful. Right. And so it doesn't really have a lot of value and so I want to make sure we're increasing the value. One of the ways we do that is through these unique partnerships and and working even more closely together. The second takeaway I have. As I heard a lot about everyone has a different perspective. You know, a county emergency manager state regional what's happening at my home. We all want that type of personalized information. And so the major question and the innovation, sort of for our whole field is, how do we create an agile forecast to meet the needs of every user, while still maintaining an official forecast. Because I think that's still something when we talk about consistency of message within tropical psychones, we hear a lot about we still need that official forecast. Well, that official forecast, undoubtedly is only going to work for a subset of people. It won't be agile. And so how do we do this? How do we have that official forecast and still have agility with our messages and I am excited to see this group of people 10 years from now and hopefully we will have made a great progress toward this goal. Thank you. Great. I'm going to continue to piggyback on the excellent comments of my co-panelists here, particularly Gina where the concept that simple isn't easy. I think that we are trying to get to a world where on an individual level people have very simple, clear understanding of what they ought to do. And to Gina's point, that is going to vary dramatically on an individual basis, depending upon a person's location, a person's family situation, a person's resources, and then the scope of what's coming at them. I'm also going to harken back to Julie's earlier point from the technology side, from Hossler's side. I absolutely think that hyper personalization and localization is the entire future that we are going towards. And just to sort of provide a practical example, right, if Hossler begins to offer emergency response coordinators the opportunity for custom messaging to people in vehicles, right, like that's level one of complexity. Well, now we can deliver specific messages to people in cars and like it might be get off the road or stay on the road or go that way or whatever it might be, right? But then the next layer of that is, okay, well, what about people in vehicles in this location versus people in that location? What about people that are already on the evacuation route or headed away from the hazard or towards the hazard? And so any sort of second or third level iteration of that, it goes deeper and deeper towards this level of localization, personalization, a person identifying, okay, give this to me in this language, give this to me at this level of repetition. And so a corollary to that is that if we design those systems around the hyper personalization and localization to create feedback loops of data, then that also gives us a really, really powerful tool for understanding how we can continuously improve those systems to be optimized and to be effective. And to continue to expand their accessibility, right, we can identify which groups are not getting these feedback loops, which groups are being excluded. So I think that when you take that and inject it with all the other things that are happening around AI and machine learning and all of these other tools, there's a really exciting frontier of what's possible here. But ideally we build it from the top down with best practices around what should be a national approach to this, I think. Be sure you don't want to jump in. Okay, you get the final word on this, which will be really powerful, I'm sure. I think in some ways, this builds a little bit on what Brock was just saying. This might seem like a counterintuitive sort of like take home but I think we have some important prediction challenges in front of us. And I mentioned yesterday that I was going to take the risk calm frame and expand it a little bit more, but I think this is important because so much I even wrote down this idea last night. I feel like so much of what we're talking about regarding risk communication assumes that we have knowledge, and we have skillful predictions of the hazard and the impacts or really the risks that can be communicated but I think there's a lot we still don't know. And two kind of components to this or threads the way I think about it is sort of like meteorologically the things we don't know Alex talked yesterday about rain rates. And I brought this up yesterday and I'll bring it up again this isn't important like it's a challenging predictability and prediction problem right as is rapid intensification. I think Robbie mentioned yesterday we're getting better with that but there's still a lot we don't know Otis was an example of that. The co occurring compounding hazards like our ability to predict those things is still limited. So that's sort of like when we're thinking about this from like a hazard perspective but I just want to put a point on this those of you who have worked with me will know I'm going to say this predicting impacts. And I'm going to again quote something Alex said yesterday because I thought it was really powerful. And you were talking about the tails of the extremes but it's hard to know how the meteorology will map to the impacts and that's true right. But this is really a risk communication challenge in general when we're thinking about being able to predict the impacts because we don't know right we understand kind of we observe and we understand the processes that go on kind of in the atmosphere but when that atmosphere intersects with the land surface, or with social systems, or with the built infrastructure any and all those things we don't hold that knowledge I think it was. He said this really nicely yesterday was referring to that winter storm when he was basically saying, we didn't know what the power impacts were going to be because we didn't understand how the power grid worked so we didn't you know put that kind of information out there and think he was framing that as what is the role of the meteorologist or where does, where does the meteorologist role stop. This is again a systems and earth systems like what the atmosphere is doing and how it intersects with society in the built environment. These are, these are predictability and prediction problems and we don't really have data, necessarily, although increasingly I think we do have data to be able to predict the impacts and I think I agree with what Brock just said I think this is an important potential role for AI to help us think about using new techniques to integrate these really desperate data sets in ways we haven't thought about doing. But my takeaway is we have a lot of prediction challenges in front of us, I think in order to be able to have something to communicate the risk about. And I guess I'll just offer a kind of a hasty observation I haven't spent a lot of time with you but in the short time I think it's important to remember that at the end of the day. We're still dealing with human beings which all of us happen to be and a lot of what happens in the lives of people in disasters we can process some of that in our own experience one of things we did about three months ago as we tested the natural alert system with the FCC. And we had all these ideas about doing surveys afterwards and all this stuff and I said no I'm going to go I'm going to go to a restaurant and I'm just going to sit there. And I'm going to watch how this this little sample I have reacts. I wasn't surprised by what I saw I was a bit surprised though that that we're surprised that people don't really react or absorb messaging the way we expect them to. I thought in that moment it was important for me to take a more emic perspective because I'm one of them I have to understand why they sort of, you know, cast off the danger or. We talked about a lot of prediction what would you do in your life what would your family unit, you know we're not all that different, where we're all under crisis. This is sort of, you know, this impulse toward this homogenous will to survive or instinct to try to be safe. We all share it's just, we all have different ways of realizing that like what's different is going to dictate what we decide but we all want to live generally. We all are the same in that way and I think if you, you know, if I think Gina want to touch on a little bit, immerse yourself in experiences of people where there are problems. But by first understanding, like your place in that dynamic and you as a human being, you're surrounded by the problem, you're surrounded by people and I spent a lot of time talking to people I was at my son's school. I'd land me the other day, doing a tornado display we took a, you know, hot, the ice dry ice and added water and all that stuff but. But we were trying to take the mystery out of it but also wanted to see how people generally react to danger when they're not under crisis and what they think of it how they process it. So hearing a bunch of second third and fourth graders talk about it. It gave me this idea that if we talk to people in casual casual walk, you'll find out you'll find out why people don't need these disasters, these threats and warnings. Whether you can generalize is another question you know you've got your ways of doing that but but you all are people and deep inside hopefully none of you face a disaster but you'll, it'll be real to you in your own walks. So don't underestimate the importance of having your own sort of emotion and your own experiences in form. Some of these some of the ways that we think about these questions. Well, thank thank you I think that really as Brad and others predicted and I believe, Julie I think very profound and useful way of thinking about the summary of that particular section of the panel I'd like to sort of. So bring my next line of questioning and feel free for any of you to tackle this all of you are subset of you. But after listening to a day and a half two days of discussions. Are there any glaring gaps or significant opportunities that you feel could emerge from the discussion. Gina looks like you're wanting to say something. Well, I mean I'm going back to my presentation and we did leave you know six major questions just sort of sitting there so the simple answer is yes, there are gaps. But simple isn't easy so it's the complexity of it. You know I think on the one hand. I'm so in awe of what everyone can bring to the table here in the last two days right. The advancements of some of the technological developments AI, the amazing social science research that's happening right, the predictability the, the, how rapid the hurricane center is making product changes. You know I think 10 years ago, we were probably at a slower pace right so much progress has been made so on the one hand I'm really in awe of what we've done. I think I will reiterate what is in our in the presentation I made today. We really don't have a good way to evaluate the entire system. I think Julie was right to say this is this is a system, it's a system of systems. Right, it's not just the forecast, it's not just impacts but it's the intersection of the two. And I think while we have so many individual components happening at any one given time, it's incredibly hard to evaluate the success of any component of that system, or how the system is working all together. On the one hand, I'd say we're all doing pretty well right on the other hand I think what we're hearing is that you know one size fits all messages. We don't work, we need to localize, we need to personalize, but as Julie so well noted, the predictability isn't always there to localize and personalize that information. I think that some of the tools that we have available to us right now are based on sort of an older way of thinking right it is based on the one size fits all. And so you know some of our gaps are literally building forecast systems that provide that agility that provide that that a mix and match of forecast components with individual needs. What are all of those individual needs. Right, I think we had a lot of those perspectives here in this room but I certainly with not at 100% confidence level could I say I know each unique need of the people that we're communicating to and so I think. And even if we were to take a census of that today, I think we heard it'll be different tomorrow and a year from now and five years from now. So I think one of the biggest challenges is how do we keep up with that. You know how do we keep localizing and personalizing and understanding all of these audiences within the limitations of resources right when we all have limitations and so I again I'm in awe of the last two days but I do think that there are gaps. And ways to approach this. I think I think Julie said it I think I'm AI has a lot of promise here. AI is a tool. I always think things as a tool not a solution right. And so once again sort of how do we all work together in some kind of continuous fashion in a knowledge transfer hard how do we continue to have this over the next two days, but almost do it more often. And that's a hard thing to answer. Feel free to jump in but I did want to say and I think I'm capturing a note that Brad just shared with me I think in some of these questions about key takeaways surprises and opportunities. We after we hear from the panel I think we're certainly open to any thoughts from those of you in the room and virtually as well so go go right ahead. Absolutely. Once again going to quote Sherman here. I don't know if I would define this as as a gap in our discussion here but I think it's something that we take for granted which is the the human element at the local leadership level. The best example I can give as a as a resident of Florida is it is fascinating to hear so many experts in this room like the level of rigor and thought and effort that goes into building these systems and products that give us with granularity the ability to say you know there's an X percent chance of Y risk happening in Z area and then a county supervisor can decide I'm not going to tell my people to evacuate. And then and then at the last minute they can say oops we should have told them to evacuate even though we received this guidance even though we had all this probability you know it's like three dudes with an opinion. So I don't know what the answer is to that because I don't think that I think that that is just a built in like it's an inherent component of our system. But I do think that it it sort of gives us the opportunity to think more holistically about when we are designing tool sets and products that are available to the public, especially if they are enhanced with hyper personalization and localization that oftentimes you might actually be filling the gap that is created by a local supervisor or local county official or something that just doesn't think it's a problem. So I think that it's just frankly just a permanent a permanent fixture of addressing these challenges. Actually I'll offer this thought and then if. Well if others don't have anything to say maybe this will open things up to folks online. This is. I don't believe Daphne touched on this yesterday forgive me if she did but one thing I don't think we've touched on here is a gap is critical incident stress of the providers of the information of the forecasters of the emergency managers of the broadcasters and I think you know some folks in our community Jen Henderson has touched on this a little bit. I think him cloco has a little bit, but for those of us who have shadowed some of the providers of information whether forecasters are gone in after some of these big events and done interviews with them. The stress that they experience. I mean, you know whether or not we we start talking about moral injury, I think, because they have only so much capacity to be able to protect the lives and livelihoods of the people who they are there to serve is a risk. And it is in many cases kind of a broader frame to how we're thinking about risk and risk communication keeps saying I'm broadening the frame more and more. But I just wanted to offer this I think it is a gap of something that we've missed in this conversation. Yes all for kind of a, I guess a type of gap and it's the assumption that we have to look at all of these through deficit view, what didn't work. There's a lot of what you're doing that's working. And I think we should be just as focused on what is work I can tell you before Hurricane Fiona. There were an awful lot of people on Keaton and dialysis who prepared three days before that. Fiona hit. Because they knew what to do, and they had enough information to act and a lot of the medical professionals in Puerto Rico or instrumental and having them prepare for extensive blackouts and people lived longer. Assume that what you're doing is working to an extent we want 100% you know and zero defect but it's not possible people have too much autonomy. But I hope that one gap that we that we sort of don't don't underestimate is how successful your work is in helping people who have to make decisions in a split second people in those tornadoes who instinctively go to the middle of their home, or you're getting a bathtub and when you ask them. Oh, this is what I heard I don't know why I heard it but this is what I heard or I saw something those college kids at the university in Tennessee. You're reaching people so I hope you take that away from this and it's not really a gap but I but it should be said here that thank you because what you're doing is saving lives. opportunities for risk communication I think people have excellent ideas about the gaps and things moving forward I think that what I'm really excited to see is each of us who are in this room go back and take what we've learned and bring it into our everyday work, and then come back again together, of course with, you know some changes with different people and see what we've learned but I think the power of that at each of us taking what we do best in our strengths and bringing in what we've learned into that is is really amazing. I also think as far as risk communication itself on the near term I think seeing the information providers, the communicators bring in some of what has been talked about today into how they communicate, especially know it will be amazing I think as someone who's been watching this for a long time I feel like Noah the Weather Service Hurricane Center Office of Atmospheric Free so everyone is teed up to now take in all of what is being learned and really change how they communicate to improve the communication as well as kind of leverage their strength and their strengths and their kind of role as the official forecast along with agility so I think in today's world Noah and the Weather Service still set the stage for how hurricane risk is communicated and we've done research on that and others have it. It matters a lot, even if you can't always see it every place and so the opportunity for really the groups that are setting the stage for the communication to raise the bar and take in this knowledge and improve how they communicate can really help everyone. Okay, thank you. I think I'm going to jump in with a comment really quick. Brad, before you make a comment, could I jump in? Okay. I just need to leave. I just wanted to share that I thought the session was ending at 4.15. My son's got a basketball game. I've got to get him to, but you're in good hands with Brad, so thank you all. Thank you, Marshall. Appreciate it. And really your comments about dealing with uncertainty and especially with increasing your projection time brought back big memories of my own when we were doing the gridded forecast in the Weather Service, the NDFD National Digital Forecast Database. And we wanted, this was 25 years ago or something, and we wanted, you know, I think it was five, but then we went to two and a half kilometer pixelation and, well, how can we go out seven days at two and a half kilometers pixelation? And it was a struggle because we can't forecast it two and a half kilometers at seven days. We can't do that tomorrow, right? So, how do you rationalize getting, but what was driving it was sort of the topography, the geoclimatic signals, knowing where the lakes are and the forecasters and especially now with AI. There's a lot of detail that isn't time dependent. That's more, you know, highs and lows, right? You know, it's going to be a diurnal. So, we started pulling and we have these things that there isn't a lot of increase in uncertainty with over time and then you with the messy stuff later. So, when I sort of step back and look at what we've talked about, there's a messy uncertainty issue with weather information. But there is a lot that we know about localization and personalization about the built structure, I mean the infrastructure. We know about DEMs, they don't change. So, we know if you move a little bit more into the physical environment, we certainly know tidal cycles. So, even at day seven or whatever, if you're risking some kind of flood, even the fact that communicating that your most likely period of flooding that day is at your high tide and tell them the high tide. I mean, there are things that you can pull out. So, I'm not saying it's easy. Thank you, Gina. But there's stuff that you can keep, that you can keep personalized and localized all the way out through that projection period. And then, so don't let the uncertainty piece be the driver of the personalization necessarily. I think that there is a separate discussion around both. So, maybe we do open up to questions from the floor, or I don't know, Hugh, do we have some Slido questions? So, there's not a ton in the Slido right now, but I'd love to take some from the room. I think I'm the only local emergency manager in the room and online right now, but I just got off for a perspective. And it comes down to that. The comments about decision making when it comes to evacuations and things that take place in our communities. I just want to hurricanes coming our way, but it could happen in a moment when a fire chief is standing on the corner, and he or she has to make a decision to do something to save lives. I would encourage you all to consider looking into that element of decision making at the local level to provide you a little bit better understanding of how. It needs to be done, but how deliberately it is done when you need to do it in my community where we can go on our website. We have a emergency decision making video. It's about five minutes long. We had to develop it because our community came under fire when we made decisions during COVID to close our community and not let people into it. We had a lot of questions as who's doing that? Why is it getting done? We laid out the authorities that we have, where they come from, why we have to use it, the measures we can and can't use. And to the point of, yes, there's an elected official who has the authority to put his signature on the paper that makes that order. He or she is not doing it without the sage advice of everybody an emergency manager can bring it through a room to give that person the best information possible at the right time. And in decision making when it comes to evacuation orders, and you have a little bit of lead time, it's easy. But if you're that police chief, that fire chief, that EMS chief standing on a corner, you got it around you and you're relying on it. So I would just encourage you to research that a little bit and think about that, because there's a lot of demystification that could take place if you talk to the local emergency managers about processes and things that need to get done. Thank you for allowing me to share that comment with you. It wasn't a question, comment just a thought. Actually, I'd love to add on to this one of the things that has shocked me the most and hozzled and working with first responders is how under resourced first responders are at a local level. You know, one of the things that we do is we lobby as part of Hill Day with the Congressional Fire Services Institute around AFG grants. And I don't know whether or not I'm sure actually most of this room probably knows this but the funding, the public funding that local and volunteer firefighters draw from and especially, you know, AFG grants in particular as a federal source for resources just to get protective safety equipment has declined over the last 10 years and it hasn't been re-upped. It hasn't been reauthorized at higher levels. And so we are expecting our local responders to do more and more and take on more and more of this risk with less resources than they've had available to them at any point in the last 20 years. And so your points well taken and to beyond that, you know, to have the weight of those decisions when you've got very limited resources and you're trying to figure out how can you best deliver safety for everyone in your community is in many ways an impossible task. So I absolutely agree. And they don't necessarily need to be questions. They can also be lessons learned and takeaways and like as please in the back. Hi, I'm Bob Hershey. I'm a consultant. Has there been any approach to putting a mathematical expectation on these events of the dollar value, which has often been come up with for past events of the property damage, plus injuries plus deaths and putting a dollar amount on it, and then multiplying that by the probability. Has there been any attempt to do this and presented to the public in real time? Are you talking about doing this or just ahead of time? Obviously, it will fluctuate as the storm gets closer that you'll get a better probability of it. Or you might find what the path of damage is likely to be, which would increase or decrease the dollar amount. But having a mathematical expectation, I think will make things clearer to the public and to everybody involved. The danger in that is, you know, if you've got a 95% probability of bad that 5% is what people may hope, you know, hang on to. I don't know. I know that there are calculations made flood areas, things like that. I'd have to think about that because there are people that are going to. So you're saying there's a 5% chance that that, you know, so. But that'd be interesting if we could do that would be a 5% chance or a 1% chance or a 10th of a percent or a 50% every event is going to be different. Yeah. But having that in advance, I don't know, people might make the decision. Well, the further in advance, it is obviously the bigger the spread of the probabilities. Yeah. And uncertainty piece. And Thank you all. There's no way I'm going to be able to summarize what you guys have said in my next few words. So I have a question for all of you. There's been a lot of discussion here about partnerships and the importance of working together. And I'm just thinking about, I mean, I know they're in Washington state they've been consolidating like fire departments across communities because they're such so low resource and you know that means they can't respond as quickly and so there are all kinds of issues like that going on. I also know that within agencies like when I worked at NSF for a little while and partnering with other agencies was a problem because they expected you to add it to your current job without any extra pay or time or anything. So how I'm thinking about all the discussions of partnerships here. How do you feel that there is adequate resource to partner with all these partners and how do you partner with under resourced local responders, given that they are already taxed. Do you have strategies for thinking about how to do this constructively. Well, part of our job is is to, you know, assess what you know what what the local needs are a lot of it is the state and the federal government or the municipality or the tribal working in hand with the federal government to ascertain what it is and depending on the level of loss and profound nature of the of the of the damage. You know, there could be significant resources that's one thing that is there the resources are there the problem is, you get what you get when you show up and if that community wasn't prepared. And that's not I'm not going to any local community a lot of them are dealing with, you know, tough things but you got human made disasters, you know, even a lot of it is reacting to what you have and then trying to figure out what what you have and you can't do anything. So, you know, every disaster is different, there's there's no one way to look at them so it's it's really going to be understand as quickly as you can, you know what the threat level is what you can mitigate. And at the same time, how do you stop the damage from getting worse, how do you stop things a lot of the, you know, he he does a problem after hurricane, and it kills a lot of people after so I won't say a lot of people in some areas that that becomes the other other threat so I would say the resources are there from a FEMA perspective a lot of it is just what do we have when we show up what's there what was already there. How much of the state wants to own it Tennessee, you know they Mississippi they were great they really run their disasters and there are some places they're a little more, you know, come on in and help us out. So it's different every time but I don't think it's a resource problem at least from the federal government side. I've been probably spoiled a little bit at hozzler because we have, we literally build a partnership model into every operation we have we don't believe that there's anything we can do in isolation and so working with the ecosystem is a permanent part of our scope and it's a permanent part of our approach. I'll say that I've been shocked at how much ingenuity, it is required of us to accomplish our goals by literally creating new legislative vehicles for funding I mean we lobbied and got a $15 million in the bipartisan infrastructure law to equip local elites with digital alerting, because we identified that there was such a small pool of funds available to local responders for basic protective equipment, and those grant programs are competitive. And so we didn't want to inject more competition into an already limited resource pool and we had to, you know, partner with our customers to go to the federal government and demonstrate need. So that from the private sector perspective, you know, the whole point of innovation is to is to create new opportunities and it's always amazing to me what you can accomplish when you go into a circumstance and you take for just an assumption like, you know, this local community is doing everything they can to meet their needs what are things that we can bring to bear to help accomplish these objectives. So in a sense that most of the people in this particular universe partnership is just a standard part of what of what you do right everything that you nothing you do is in a vacuum. So I do think though that we are going through a process right now as a country of in many ways sort of training ourselves from a from a policy perspective to move as quickly as the market can move. And that is a difficult thing to accomplish but I'd like to say that it seems like we're getting better, but it takes constant work for sure. I'll jump in to an answer this from a, you know, a no a perspective from an R&D perspective. I think one of the good things that came out of the pandemic was an increase in virtual tools. And so that has enabled more participation from people that perhaps couldn't travel. But now there's more inclusivity right they have more options to participate you don't need to travel you know you just need a laptop and hopefully you know there's access there where there's not well then we've got to work on that right. But I think that has opened up a lot in increase we have not seen in the last five years of our program we have not seen a decrease in emergency management participation and research I'd say we've seen an uptick in that participation. I also think there have been a lot of opportunities to provide feedback. I agree with that I'm sorry I didn't recall your name but the local emergency manager who's in the room. Thank you for those earlier comments and I take them to heart. It's one thing, you know, a lot of our researchers meet with emergency managers after the fact and this is what I mean by sometimes it would be really nice we'll sign an NDA to sit during an event and literally watch what you do and the reason why I say that is not because we're trying to make you a subject of research, not because of that, but because being there live real time is very different than hearing about an event after the fact and having increased empathy for the pressures on you and the demands on you that's hard to understand in an interview afterwards as compared to actually witnessing it during an event that that is a perspective I do not have right, but I'd like to have I want to make sure I have that understanding and empathy for the hard work that you do. In addition to that, so again participation grants can include and I say this publicly for those who applied. If you want to put in travel for emergency managers as a subset you may if something were to require in person travel or of course the academics can travel to the emergency managers. I think what our program has found is that if we want to do knowledge transfer events, we have to go to them we should not expect them to travel to a conference that is not one of their normal conferences there's not a lot of travel money out there. We need to go local to them and so you know to the extent that we have resources to do that we will try but again, I would say one of the nice things about the pandemic is is utilizing those virtual resources for more open communication. Okay, thank you Gina. And we're going to move on. So I want to join me in thanking our brilliant roundtable members. And now I'm going to turn over to and for wrap up. Well, I will try to keep this short because I certainly certainly can't top anything that everybody has said here. It's been a really fantastic two days I hope you all agree with me that it's been extremely extremely rich conversation that hopefully the community will draw on for years to come and and we will take will take strong recognition and benefit from all the things that have been said here today. I'd like to piggyback on a couple of comments that were made in this last section and earlier. Sherman reminded us that this broader tropical cyclone risk communication community has achieved a lot of successes. So I'd like to emphasize that again we heard a lot about advances and successes here that for me personally we're exciting and I think for all of us were enlightening and the various things that we have learned about preparedness is the best and first response and partnerships and strong relationships and community and communications efforts and in research efforts can facilitate cultural competence hyper localization and personalization of preparedness. So really big opportunities here that we see arising from many of the things that we heard about here today. The topic of the workshop is advancing risk communication with decision makers for tropical cyclones and learning from unprecedented and extreme weather events. And that as Julie reminded us requires an earth system science approach across all the sciences. This challenges all of us to figure out how to evaluate the success of the system and its components as Gina reminded us. So that we can design systems with dynamic population representative and inclusive feedback loops we heard also in this last panel panel to understand how well the system is working using a national approach. But that is a challenge that we have yet to conquer so something to focus on. And finally meteorological social computational and other sciences working together with partners on the ground and in exposed communities can help each other advance. So there are lots of opportunities here to to build on what we've done. I'd like to thank you all for a really amazing workshop, especially the committee that has worked so hard to put this together. And we have identified a few gaps and lots of opportunities and successes to date. I've heard a lot of conversations between people who haven't met before. I'm really pleased about the relationships that I see developing here. I'd like to thank the board on atmospheric sciences and climate climate for generating the idea for this workshop, and the sponsors NASA, Noah and the, or should I spell all that out in the National Science Foundation and the planning committee as they said, has been really diligent has met more times than you would care to think and has been really heroic and committing to all of the work that it that it took to put this together. The recording is going to be shared online. Thank you. National academies and feel free to share that with anyone who couldn't participate who missed some key part that you thought they should have listened to and the proceedings document will be produced. We were told, not as soon as we would like but pretty soon. So, so later in the spring or in the early summer of this year to summarize the event. Thank you all. You've been a really excellent group of participants. I don't think anybody in this room has been quiet the entire time. And it's been, it's made their workshop and incredibly rich experience. The meeting is adjourned.