 All right. Welcome, everyone. We'll get started in just another minute or so. We'll give a few people as last precious seconds to find where was that Zoom link? Where did I put it? And get a cup of coffee. All right. So why don't we go ahead and get started? I want to welcome everybody to this latest CSTMS spring webinar. Today, we're fortunate to have Michael Barton from Arizona State University speaking to us. Michael is a geoarcheologist and an anthropologist who's in the School of Human Evolution and Social Change at ASU. And he's also a leader of Comcess, the Network for Computational Modeling in Social and Ecological Sciences, as well as being the visionary behind the Open Modeling Foundation, which is what you're going to hear about this morning. So now, further ado, I'll turn it over to Michael with just a note that, unless you have some really burning question during the talk, we'll generally hold questions to the end. At the end, we'll invite you to either raise your hand and avoid your question aloud or to post it in the chat. And I'll try and keep an eye on the chat. Over to you, Michael. Thanks, Greg. I'm really happy to be here, even if it's only virtually, I'd rather be in Boulder, but it's nice to see you all here. Thanks for inviting me. I mentioned a new thing. I'm also now, my point was, I'm in the new School of Complex Adaptive Systems, which is in ASU's new College of Global Futures, which is an interesting visionary and ambitious enterprise we're doing. And so it's going to fun to be creating a new academic and research unit at a university, something that hasn't I've never been involved with before. So that's kind of neat. As Greg said, I'm going to talk to you about the Open Modeling Foundation. This is an initiative that systems has played an important role. It's a really critical role in helping get started and continuing on. We're partnering with ConsistNet and others to try and make this reality. So anyway, what I want to do here is start with pointing out to all of you that the Earth we live in is a coupled natural and human system. It's not, it's neither purely a geophysical system, a biological system, or a social system. It's all of the above. I also want to note that as opposed to kind of a weird meme that's circulating in the right wing mediaverse right now, I have the proper substances coming out of the proper ends of the cow. And if you're interested in that, I can try to find that meme for you if the flatulent cow. Anyway, the Earth is indeed a coupled natural and human system in which people and biota and geophysical processes interact in really complicated ways. And in addition to the complication of atmospheric dynamics interacting with oceans and changing albedo and evapotranspiration of plants on land and all that kind of stuff rivers and streams and coastlines. We also live in a really unique social world today. And with as a historical scientist is one place I can get away with saying, in a few short millennia, and you understand how short that is. We as humans have changed from being normal, a normal terrestrial animal mammal a very successful one that spread all over the world. We obviously have unique social and cognitive capabilities use of technology, but we've been generally normal up until a fairly short time ago, when we've become a really unique global phenomenon. Now there's over close to 8 billion of us. That's a lot for a large animal. Over half of us live in urban hives of millions of people. I even something the size of Boulder is unprecedented in most of the mammalian world. And things the size of Denver or Phoenix or New York or Mexico City only exist. Among some social insects ants and termites. We are really weird in that sense. We've this is but this has made this coupled naturally in one world, even more complicated and complex than it was before. This unique social world that we live in. Now, even more so we've got digital media, rapid transportation, this connects all of us economically, socially, culturally and as of course we've seen recently, technologically in this planetary networks where it, you know, you can be in one part of the world, but you have consequences throughout the world are the footprint of all of us, especially in first world countries is global. And so we live in this network of telecoupled, cross cutting ties that are socially economic ecological and geophysical. And so this comp social natural system is highly complex to a level it's unprecedented for any organism in your history. And I, and I, you know, I understand that we're mammals were affected by evolution we're part of the earth just like everything else we're natural. We're an organism that is doing things. That's a first. No other organism has been connected in terms of information processing and their, and their consequences, like people have right. Even a few short centuries ago, like, most people lived, didn't live like this. We lived in small communities, and anyone living this communities if you were living in them, you could actually observe you could see the, the, the important social phenomena you'd know the people you lived with and, and, and interacted with. You could see the natural phenomena that affected your life. And, and you have a pretty good idea that if you did certain things socially or ecologically, you would understand what the consequences would be of these things. But that's not the case. That's not the case. We interact in various ways with people we never see, I mean just like right now. We interact with were affected by socially culturally by by humans all over the globe in ways that it's hard to imagine our consequences. If we didn't have the kind of tools we have and we'll talk about in a minute. There's no idea through direct observation that our individual actions affect global climate, certification of oceans, loss of soil erosion dead zones in the Gulf of Mexico on and on. So these are things we can no longer observe these things. And so, in this kind of a world we live in, that's dominated by these telecoupled processes and that's changing really really rapidly. We need to expand our vision, right from the small scale communities where we could actually observe what's going on as managing our little local gardens of of serials and and legumes and things like that. We need to change our vision of how we think about the world and what we do with it to try and manage the dynamics of a planetary system. A sociological technological system one that we have created in part. And that's that's a big, that's a big ask that's something that nobody has ever done before. All right, so if we if we're if you think we're having a hard time doing it. And we're making mistakes. I mean there is an excuse that we're faced with a nearly impossible challenge that humanity as a species, and us as individuals have never been asked to do before. We need worse, we have to think about the impact of what we do on the atmosphere in in the other half of the world on oceans on land, things like that so as well as other people. So I would say that really, we don't have the kind of capacity in our brains to think through all these things. The interactions are way too complicated. And that's not due to our own failing. There's a certain biological limits that we have even our collective knowledge is not sufficient to face this challenge. And so we need to leverage our capacity. Or as one of my colleagues put it, we need the prosthetic of information technology. So we need to. We need to add to our existing abilities and replace it with information technology to meet this challenge and of course humans have been using technology to meet challenges socially logical challenges for a couple million years so this is nothing new for humans to do this. Modeling tools, mathematical and computational ones that began to be developed in the middle of the 20th century. And help us understand the dynamics of especially biogeochemical processes in the atmosphere and the land and the oceans. And think about this in ways that we could not imagine have been successful in trying to leverage our intuition. And they have been important giving us, you know, really critical insights to not just the processes that affect the earth system, but also how we interact with that and and potential futures of the earth system. This is something new we've never had the ability to look ahead, like we can now. But these, these systems these models, modeling systems developed 50 60 years ago initially and and continuously improved on and built on are starting to face growing limitations. Many of the tools, which are really important for understanding the dynamics of human systems and natural systems right are operate independently of each other, and do not account for the complex interactions between human systems and natural systems. You know, they operate in different worlds. And to some extent that's that's illustrated by, I mean, we're in partnership but by the origin of systems as a network and organization focusing on modeling, geophysical processes and Conscious Net, which is a network and archive and things from to get to bring together people modeling socially ecological processes like but in reality, these things all interact. But that's not the way that that modeling is today. A lot of what I would call first generation models that have been so important. There's a there is a big focus in funding and activity on modeling natural processes over societal processes. There's some feeling that societal processes in humans can't be modeled even though we can do it in fish and wildebeest and ants and everything else. Somehow, we're we're too special for that. And so people have looked at natural processes. And as I said, when people do try to model human systems, they do it independently of an often ignoring the connection to geophysical and biological systems that we are connected with. Many of the big modeling systems for for understanding large scale global processes started in the 60s, and people have built on and modified and built on these to a point now where they've accumulated so much code that they're difficult to use they're difficult to modify. If you want to run them, you have to take highly specialized knowledge, and often you need supercomputers to do it there are enormous piles of Fortran and other kinds of code, powerful. But now it's good they're getting very difficult to manage, and they're not accessible outside of first world countries and even in first world countries. Often these kinds of modeling systems are only available to a few well funded national laboratories and private private firms and people supported by the World Bank. So they're very limited accessibility please. And because of the size and and and complication and accessibility and often proprietary black boxes. Often they don't really, they lack flexibility they were designed to look at certain kinds of questions and it's hard to shift them, like a giant ocean liner to look at other kinds of questions at different scales. I would say a grand challenge for our time and hopefully others in the scientific community are beginning to think that this kind of infrastructure is a grand challenge is to try and grow scientific modeling infrastructure in new ways to understand these interactions between humans, societal dynamics, biophysical components of the Earth's system, and at multiple scales. And to that end, a group of large group of scientists around the world began a series of meetings back in about five years ago to try and talk through and think through what the future of modeling should be. And a key players in organizing this for systems aims, which is a section of future Earth it's analysis integration modeling of the Earth's system and Compses net network for computational modeling a social and ecological sciences. And so we started and had a series of meetings in many places in interesting places in France in Boulder in Japan, Germany. Back in Boulder again, right, but we've to try and talk through and think through and envision what the future of modeling would be. And some of the questions that were brought up or how do we, how do we put together how do we integrate modeling human biophysical systems, and all that we know is important but how do we actually do that is a worthwhile. That's a lot more code that's doing things that are difficult. How do we model across scales because some questions are global, some issues are very local, and some are very in between how do we do this. And certainly large or system models are not very good for trying to model what's going on in a given community. How do we diversify intellectually diversify input and and concepts going into these models right now because of restricted access and funding and things like that. It's actually a very small cadre of software engineers and scientists that provide conceptual input and the kinds of modeling they're out there. How can they be flexible for the future. Alright, so a lot of the modeling systems that are white and white use today and IPCC and others were designed long ago years sometimes decades ago. And that means they were designed to answer questions that were important then, and which may or may not be important now not the ones that are in the future. But how do we try to design for resilience scientific resilience for questions. We don't, we can't get anticipate the challenges we have not yet faced, and importantly, how do we democratize this technology and take it out of the very restricted world context, which, you know, which for a variety of reasons that was where it started, and that's why but how do we change that because the challenges we face are our challenges as a species, as humanity all over the world. And, and so it's important that as much as possible, we both provide access to these important tools to help us in prosthetics to our intellect, and get input, right so get get input and contribution from much more diverse group of scientists. And in other words these are big questions and and and so how do we do this with an international scientific agenda, build new capacity, and we need to build from the successful technologies we have now but surpass these. And so these these are we, that's why we had so many meetings that these are big questions and how do we talk about these, and how do we think about it for the future. Consensus, I think that that emerged from these is that structurally a way to do this is to shift from the kind of modeling approach formalisms you might call them of the mid 20th century of monolithic all-encompassing code bases where you keep adding more and more to the same mass code to a distributed evolving ecosystem of models ones that could be potentially interoperable and and this is mirrored in and you might see some of the most successful open source ecosystems for scientific software that have that have emerged like the Python ecosystem the our ecosystem, very different than the monolithic statistical programs for example that that dominated the 20th century, and this would allow us to be flexible adaptable respond to diverse changes changing challenge with right and integrate human and earth systems of different scales and would help us encourage get contribution from a much more diverse scientific and and non-scientific community on how to build these models, as well as make these models of the technology more accessible and usable, so help democratize this technology, and so that's great. But to do that, we need to change how we do modeling. We need to make sure that individual model code is accessible discoverable. It needs to be understandable. So you can't just put it out there. Other people need to be able to use it and and run it and things like that. We need to leverage new technologies that make it easier to reuse model code so that you can make run by different people and put together in different ways and common API right to link models together and in ways that are widely understood and can be applied. And this of course says just to include involved technology but also protocols ontologies, you know understanding of a terminology means and as a lot of you know systems has been instrumental in doing a lot of this kind of work. And but we need these practices to be followed across multiple science domain so for thinking about looking at kind of an integrated view of a coupled natural and human world, then we have to be able to do the same thing with our modeling. And while technology is needed. It's not sufficient alone. That means we have a social challenge of one of my colleagues call collective action, bring together scientists to agree on how to go about doing these things. And, and this means, you know, I say we need to shift from the approach of modeling a team team modeling to solve a problem for that team to creating models designed to be used by others as well as solving a particular problem. And that means we need standards. I mean the other stuff's exciting have integrated models but at the base of it, maybe the boring but fundamental thing is we need community standards about how models are deployed documented described made usable an API eyes to actually do this and we need to have people adopt these so that models created by different developers and different teams. Can solve those problems but can be integrated in new ways that people didn't anticipate even to become building blocks for kind of integrated modeling of human or systems. And so we need new scientific institutions to help us do this. How do you how do you get standards right so we need institutions and institutions meaning kind of rules and organizations. We need to develop software standards right that can apply to multiple global, global scientific communities. Tell people about them, you know disseminate this information, administer the standards. So that means we need to be able to recognize individuals and teams that adopt the standards and be able to certify standards software that meets, you know me standards, we need to be in education. And importantly, we need to somehow create the feedbacks of professional incentives to encourage and enable scientists to actually adopt standards to do the extra work. Why is it worthwhile to model in a standardized way rather than power you want to do it. And so we need to create these kind of incentives to make it worthwhile and this requires a coordinated international effort that spans social and natural sciences. So the open modeling foundation emerged as a meta organization to do this. After all these meetings we had a lot of discussion, but several initiatives to actually put some of the stuff in practice came out of that and the open modeling foundation idea was one of those. They don't have to go beyond the workshops and talking about it. Started with a meeting in June 2018 serendipitously a number of modeling organizations converged on Colorado State University for a conference of the road from you guys in Boulder, and I came up with a mission statement and some ideas and some kind of a plan of action and you know, I won't read this or you can glance at it I can come back to it but the idea that the open modeling foundation would be this international community consortium if you would, of modeling organizations to try and coordinate their standards for modeling that would help create and enable this open ecosystem that we envision and focusing on on what's turned out called fair and findable accessible interoperable reproducible and somewhat different words but the idea is to make things discoverable accessible, reproducible and integrable. And so some in the interim, well, this this organization doesn't officially exist. But it's what we've tried to do now in the past couple of years is start to lay the groundwork for creating this kind of an organization and systems and and ConsistNet and then DSAT Crump modeling group. Got a little bit of funding to develop a couple modeling environment for food systems as an example we did this actually for an AGU paper and a failed NSF grant and then we got some money from the Alfred P. Sloan Foundation. The initiative related to this is a project out of the global land project and aims to try and create large scale behavioral models of land use Mark Lonsaville, who's the chair of aims is heading this. We did a paper in our system dynamics. I want to look at on modeling feedbacks in the social and our systems. We're trying to create these incentives and certification with an open code badge for the testing containerization as a way for reuse. Of course there's been lots of enhancements on API for integration and systems. And then something came out of this was an open letter in science. A little, almost a year ago, trying to call for model transparency in the COVID pandemic because there's very, very little coordination and lots of models, as many of you remember the early days of this. So, and we've had meetings, but the meetings have had a goal in mind we had the founding meeting in June at and for Collins. And then we started the ideas to try and come up with a series of strategic planning workshops to bring in different members of the model and representatives of modeling community to talk about this initiative, get input from the global modeling community to try and build an organization, something that has an organization hasn't existed before. And so, I was in Germany for sabbatical and we had one focus, especially on Europe and at the Institute for advanced sustainability studies of colleagues that were involved in in the founding meeting put together a meeting in Australia to talk specifically about standards for model documentation. The Sloan Foundation money provided funds for a couple of workshops, and we were going to have them. We're going to have one in Boulder in fact. Last May, and one in Brussels in the summer, and then one more probably in Tempe. And as you all know, none of that happened and they've all been on zoom just like this. And so we had one we did kind of did a real quick realignment last spring and moved it all to zoom. And it was pretty successful in May. We had a second one that was supposed to be in Brussels we had that in on zoom also when it was clear that we weren't going to be able to travel. And so the first one was focused kind of on North American, but we've had to do is look at time zones. So now we focus the time zone in particular areas rather than hold the meeting in those areas. So we focused the first one on North America the second one was focused on European time zones. And we have the third one is coming up beginning the end of May and first couple days of June so it's coming up in about five weeks. And that one will be in a time zone focused on Asia Pacific so we're trying to invite organizations there. The idea is to try and make this a global community. And then we hope that we can have a formal organizational meeting and launch by the end of 2021. That's, that's the goal where we are right now. So here's some things about what we're thinking about this comes out of an NSF proposal that is pending and but so far 46 organizations around the world have participated in these these meetings representing thousands and thousands of modeling scientists. Government agencies and labs, national laboratories, universities, nonprofits, laboratories, professional societies, scientific networks, digital repositories so it's been a really diverse and actually very heartening response to this initiative there's been a lot of interest in this since this is sort of how we're thinking about organizing it. And with a council of members represented that would be made up of representatives of organizations so members are organizations on individuals not a society. Models can be an affiliate or join working groups, and we have standards working groups to kind of develop standards, certification working group to help administer those standards by working out ways to certify models and and provide badges and papers and model repositories like the systems and concepts net and find other ways to help incentivize standard based modeling, and then an education outreach group that would both provide training and try to work with groups like journals and government agencies to require or encourage standard based modeling. So this is kind of what it looks like. And, and, and the game we're not making this up there's a number of other standards organization so this is kind of a standards workflow, a standard workflow for standards. You can see how this is supposed to work and the ideas of a standards working group would propose a standard for setting accessibility be looked over by the executive committee, and then be be sent out as an RFC request for comment. Modeling community get comments back does back to the standards working group, they revise it. And, and it goes to the council of members for approval at any given point it can go back to the community or go back to the standards group for visual additional revision, and eventually gets adopted as a formal standard gets published. We have a project more dinner would do that and then the certification working group would work out, you know how we would be badge and certify things of this. How would we actually do this as a global community, well we sort of had some ideas, and then when we were forced to take our workshops online. We hustled and put some of these into practice and it seems like they're working and the idea was to build a collaboration platform based on GitHub. And so we have a pilot version of this that we have now piloted in the two previous workshops and we will again added things to the coming workshop, where people at the workshop, besides just making doing discussion will actually actually put in ideas into what's actually the GitHub issue tracker, and eventually have proposed changes to initial documents like initial standards ideas that go in as the equivalent of the GitHub equivalent of pull request we're trying to avoid some of the specialized GitHub terminology the ideas that you can propose ideas, we can propose edits, people could discuss these, and we can actually be adopted. And the nice thing is this captures the discussions, and the ideas people have and provides a permanent record of how we're trying to create this organization that anybody can go and look at. We're going to work with this. The SGCI the science science gateways community Institute and NSF funded organization, which focuses on gateways that work with systems and concepts that to help improve this gateway the gateway that we've put together a pilot for. So what it looks like to get an idea so a really basic intro screen where you can go to organization government standards and information. And so you can see here's a document here, and you can propose ideas and changes to that document if if you want and you can take a look at this if you'd like. So this is an example standard that was developed in one of the workshops. Here's issue tracker, where we've captured some of the information during the workshops and breakout rooms and things. And, and this is information organizational information about how we would organize this come up with different kinds of working groups and things on it. So, we have a little bit of support from the slum foundation we're in negotiation discussion for maybe additional support we're hopeful on this. We have a NSF proposal pending it's with systems Consisnet kawasi hydrology people aims. So international society psychological modeling and international environmental modeling and software society are all kind of copi partners on this NSF proposal but we also have a number of other organizations research data alliance open geospatial consortium SGCI earth science information partners. Open science gateway that are supporting this this proposal and we hope we get funding for that and we'll see keep your fingers crossed. And so we're doing now as we're trying to get this last workshop going in a month and start to plan for an organizational and charter adoption meeting. The end of the year improve our gateway from this plain vanilla version to a better one, start to think more about standards and certification and badging. And we have a infrastructure proposal in the Consisnet that would provide kind of cyber infrastructure for usability. And we need someone who's really good at designing logos. If you know anyone that does logos, or you do logos, and is willing to work for recognition. We're happy to we'd love to have some input on that. What we don't want to do is this. One of my favorite web comics give you a minute to look at this, and I'll wind up and say, thank you very much. I wanted to make sure that we had time for any discussion or questions. Love to hear what you think about this and answer your questions on things so I'll leave it at that, and we can go to the questions section so thank you very much. Terrific thank you so much Michael for a really stimulating talk. So folks have questions. You've got a couple options you can use the raise hand feature and I will keep an eye out for it and call on you, or if you prefer to type out a question or comment in the chat you can do that as well. And Lynn has given you the power to unmute yourselves. So, so while everybody is digesting and trying to frame their questions I have one which is simply. If there's folks on this webinar or listening to the recording later who are interested in becoming involved in omf in some fashion or contributing to it, or maybe even applying some of the standards as they merge. What are the pathways that they could follow to get involved. At least the way we're envisioning it right now. Individuals could join working groups I mean that's where the actual action is going to take place. And we've got kind of a members council which, you know, overall approves things and strategy and things and that's representatives organizations but individuals can actually get their hands dirty and really work on these things in working groups. And that we just kind of want to stay in touch and get information, assuming that this organizational, this work chart that I showed you earlier is approved. You can be a affiliate. We've actually put up a pilot version of I'd like information on a companion website that I didn't show you here we need to merge these. And I've got a group a bunch of not much but a fair number of requests for information haven't done anything with this because actually people discovered it without us telling anybody so it's kind of a little embarrassing I haven't anything to say to people except stay tuned. But, or maybe I can point them to this talk if you've got it recorded or something like that. The but that would be the ways, you know, you can, you can become a representative of an organization like of systems or some some of their organization represents modelers, join a working group or become an affiliate that's the three ways and actually there's a student group to that we're, we're proposing kind of a student affiliate group also so four different pathways, but we're in all people have another idea. And, and if you really want to join in the workshop in the evenings, coming up, I'm happy to invite anybody who would like to join in the coming workshop also, if you really want to participate, love to have your input. The nice thing about zoom meetings is that the cost of transportation logic and food is negligible for all the participants. The tables can be pretty big. Thanks. So I'm not seeing any raised hands at the moment, so at the risk of seeming to hog the questions. I have sort of a big one which is maybe unfair but I'm, you know, you started out the talk talking about the critical need for kind of multi sector dynamics approach or, you know, let's think about the human aspect of the entire global human natural system. And I'm curious about whether there's an emerging consensus within the sort of broader social dynamics field about how doable that is. I gather that there's some, probably some tension between some folks, maybe historians who would say you just can't do it. It's human beings are too contingent and unpredictable and others who would say nonsense that it's just a bunch of molecules and there's seven and a half billion of them. There's definitely predictability and a lot of people in between. Do you have a sense of where the community is on that and movie that's those are those extremes are exactly what you know, just what you pointed out and everything in between. I think there's there's been a movement toward a larger. I'll say minority. I've begun to think that human societal dynamics can indeed be modeled. And I say that from from the growing popularity of agent based models. Let me like, give me a second here and I am going to show a little graphic just so you get a sense of why I think that there is there is a shift on this. And this will just take me a second here to find here. And this one. Okay, and what I want to do here. Let's go to this screen. And I think you sort of have a similar. So, um, what we've been there, there's a couple of things here one is that this is what I want to see. I don't have this other graphic. Well, this will do. There, there's, I should show you different graphic here, another one. Oh, and I'm trying to I'm trying to find another graph I've got. Well, I just want to point out that we these are the number of models in the library in the concepts library that have been published. And it's, it's, it's growing at an accelerating rate. If one of my colleagues Marco Janssen has done a study of publications out of ISI Web of Science, and just looking at agent based and individual based models. And the number of those is also growing a number of papers about these is growing at an accelerating rate. Rapidly. And those kinds of models have been especially important in changing people's perception of the model ability of human social systems. Interestingly, I think one of the outcomes of the current global pandemic is a increased recognition by people in the social sciences and humanities that not only is modeling some aspects of human behavior doable but is desirable and needed. And that maybe it's actually okay and it's not a front to our humanity to try and do modeling and network science has also been growing equally rapidly so I don't think I wouldn't say that there's a majority of people in in the social sciences and and especially in the humanities that oh my gosh we need we all need to start doing modeling. But there's a greater acceptance of it. And a recognition that it's, it's important and at least somebody should be doing it for some things, and there's real value to it. So, I would say past a decade there's been a real shift in this because I've had more fights with colleagues and social sciences about modeling, then at least as many as I have about modeling people in the natural sciences. No fights but you know trying to explain why it's possible and things. Right, thanks. It looks like there's a question from Marty Taragi. Why don't you go ahead and unmute yourself to. Marty you're muted you have to unmute yourself so I can hear you. Hi, Michael. And hello everyone. I have a question. All we know that we are in an entrepreneur now. And there is a share we need to shift our one traditional point of view to the interdisciplinary one. But there is I have a problem myself in my, in my country, for example and developing countries. And it is that that it is that in our country, for example, our professors resistance resistance in resistance to combine other combine other disciplines in our in our main in our researchers. It is hard to it is hard to persuade them that to use interdisciplinary approaches in our researches. For example, it is hard to say to say them that we want we want to use some psychological or some or some some psychological or some economical economical approaches in our in our researchers and how can how can we persuade them to how can we persuade them to accept this kind of this kinds of this kinds of disciplines for researching it's hard a little bit to persuade them. Thank you. So are you talking about convincing people in natural sciences that we should include models of people, or talking about that people are resistant to using models at all. And they don't believe they don't believe they don't believe they don't believe other other other disciplines. It's hard to it's hard to persuade them to use other disciplines in our researches. For example, I, I'm working in water resource management and I need to use some behavioral behavioral method and I use some psychological factors in my model. But my, for example, my professor don't believe it. She don't believe the survey. She didn't know I just want modeling and she just thinking that some traditional higher hydrological modeling and it's hard to persuade her that that we we should we should consider other other other aspects of aspects to do a good good research. And part of part of my response and this this actually also time talk to social scientists is to point out that we're already doing modeling of those other aspects already doing it. But instead of doing it in equations or algorithms that are transparent and easy to easier. We're harder to understand but they're easier to check and evaluate. We're doing it in pros in writing. We're, we're, we are saying that we think people would do the following things. And we're doing it on a paragraph. And so that's a model to anytime you you make a statement about a system and how it operates, how economics operate, the importance or lack of importance for economic decisions in water resources. Anytime you make a statement about the future of how people or other or plants might respond to to changes in hydrology as a model. That's a model that represents that system. The difference is, it's done in words, nothing wrong with doing it in words, they're understandable, but words are hard to check and test. And so it is useful to say, look, we're already modeling these things. Why don't we also not in replace but also take these words and turn them into a different way of representing a different way of expressing the model, so that we can test it and we can check it in that's hard with the words. And so I think something that might help is to point out that your professor is already doing modeling of these things. It's just in a different format and all you want to do is put that model, the model that's in their head in your head into a form that's easier to test and interact with your model, not replace it. But some of the way that I talk about people go No, I'm not modeling. So sure you are as a model anytime you tell a story, a story is a model. It's not a minute my minute second by second recounting of something that happened. It's an abstraction, you know, you tell a story and it has a beginning and a middle and an end as a model, and it has a goal. So maybe try to rephrase what you're doing a little bit and might help. It's a great point. Okay, let's go to Chris Vernon next and then Albert after Chris. Okay, hey Michael. Thank you for the great talk. So I just had a few quick questions and please forgive me if I missed this in your talk. So we're we're in the process right now of writing a coastal integrated hydro terrestrial modeling workshop report from a workshop that happened several months back. Taking advisement from the original it on workshop that I think many of you on this call may have been a part of as well, or adopting open science by designs approach for multi agency collaboration and interoperability with their models. And I looked at OMF, and you know, we're putting our recommendations forward and I specifically placed one in about contributing interoperability standards to the OMF and like, like foundations that are forming. And so I think I'm curious to know whether you're, you're planning on spawning working groups off of the categories you have now for instance the standards working group, where you have broken out categories driven by interest for things like interoperability standards alone and having a specific working group tailored to crafting those or if it's going to be a kind of a lumped approach and how we should apply to to contribute to this. Well, what I would like to do Vernon is let the community decide what's what's useful in this. Okay, we've, you know, I think it's it's it's useful to make it a simple startup, you know, for organization and things like that, but but then provide a mechanism, whereby a standards working group can become an accessibility standards working interoperability standards working group or reusability standards working group and so forth. If we have enough interest in people, by all means that would be outstanding. I just, I feel like that we're being an overly enough ambitious as it is to set this whole thing up without trying to specify in advance the level of interest and participation that would be required to do that so I think that would be a great idea. And I certainly think Greg and anybody else has been involved this would welcome the community deciding what would be the most effective way to move these things forward in the in a previous iteration of NSF grant. We actually wrote that we really wanted to let the community decide how this would work and NSF wanted something more concrete and and really we need at least a strong man to propose to everybody. You know, or an org chart and all that kind of stuff to get started then people can tear it apart and change it. And that's, that's fine. But I think the more that that the community at large has a stake in this kind of an organization, the more they'll want to participate. So, I know it's a longer answer than what you're hoping for but the answer the short answer is yes, but see what happens. No, that's that hits an L in the head I was just wondering if it was interest driven and that that's good I think that that goes right to it so we will proceed accordingly over. I hope you I hope you indeed participate. It's great. Yeah, yeah, our DOE programs are excited about this so I think we're looking forward to it. It's great thanks for the question Chris Albert. And then we'll go to afterwards. Thank you for your inspiring talk. That was really good. You mentioned this already briefly and I guess this is more like a note to you then then there may be a question but maybe you could reflect your thoughts on this. So, you mentioned briefly OTC sort of open geospatial consortium, which, which is doing basically what you want right but then for data for geospatial data that they they have formed a consortium they, and they develop standards, and those standards are adapted to the community because the community is involved as well. And I'm not sure if you're familiar with their structure how they set it up but in instead of having those, those meetings, you know, once a year or a few times a year. They run something like testbeds, where they actually test standards and try to with they try to get relatively small group working on a detail or a few details of a proposed standard and see to look at, you know, is this holding up does this standard do what it's proposed to do. Or, you know, are there exceptions and how can we get around those exceptions, and they, they put those testbeds in a, that there is a financial incentive but there's also you know interest driven kind of. And those testbed feeds them in working groups kind of and you know those working groups can then work further to to hammer out those standards basically and to to make them more solid. Have you. I guess my my question goes into the direction have you looked more closer into, you know, OGC and how they set it up and are you so that this comes back to Chris's question as well if you thought of a structure like that and setting it up like that. Yeah, I mean, I have a number of conversations with George Percival, who was the head of the science program at OGC until just recently, and he's participated in our couple of our workshops. And I've looked at the structure, OGC, which actually emerged from the Open Grass Foundation, many decades several decades ago, right, starting out as very small. And now it is a large, well funded international organization that has staff and software engineers and things like that. And so they can do that. And I think those are, those are great ideas and as we move from things that are kind of like accessibility into APIs interoperability, those that kind of process will be important. You know, we need to be able to demonstrate to a larger world that certain standards are workable and useful and beneficial. And, and we have to do that. But again, I'm not kicking the can down the road to saying we want to, we want to get it launched with a usable structure, and then see what that structure evolves to need so what does the modeling community need does it need test beds to test APIs, or does it need something else I mean what what's convincing and what's useful and and I think that's a that's a useful model. But I don't, I guess I'm trying to be uncharacteristically modest and say I don't know the, the answer. I don't know the answer, whether that's whether that's the best way to go or not it's worked for OGC and maybe it would work down the road for OMF. And I think, I think we would be wise to keep our eye on OGCs and IEEE and and the other organizations that do standards, and get an idea what works and what doesn't works for them mean that's not let's not reinvent things. If we have to come up with something new that's great but let's not do it needlessly so I absolutely think we should look at these groups closely and talk to people. That's great. Thanks for that question, Albert. And last question to show. Yeah, forgive me if I'm not pronouncing your name right down at the bottom of your screen. I got it. Oh, I wasn't able to unmute for a second. Thank you so much for this talk. I'm a postdoctoral researcher at USGS. And my, you know, foundational work was my PhD research which involves stakeholder engagement and building landscape simulation models in response to specific landscape management questions the stakeholders have. So when you talked about models designed for use by others. I, my ears really picked up because I believe in, I mean they're amazing models out there but people don't know how to use them. And, and, and a lot of times the models don't really provide necessary answers, they're looking, looking for so my question is, you know, I wonder if you have thought about how we might streamline stakeholder engagement or at least. Allow us to understand what are the relevant social questions that we should be looking, looking at as a scientist. So, what, what I sort of would like to see this is my vision at least maybe it won't happen this way is that. The RMF will not do that. Right. That's, that's what actually individual scientists and teams will do working with stakeholder communities. I don't want to say we'll ignore stakeholders I think that's that's a bad thing. I think we want to get stakeholders input on standards and how these things work and interoperable, you know, and how we can organize things, but for particular kinds of questions, those are going to evolve, and I think the community itself is best. positioned to make those decisions, and if we can create an organization that democratizes access to technology and democratizes the ability of people around the world to contribute to that technology, then we have a better chance of communities at multiple scales, providing input to modeling teams, and doing so in such a way that those modeling teams then create tools that other people can use. And, and can, and then because they're understandable and reproducible. Other teams can say well this is close to what I want but not exact. And so I will take this and modify it now. Because it's a standard based model, as opposed to when this proprietary black box, not described, not reusable, then they go well this may as close but I can't use it. So, what I'd like to think is that can we can we create the underlying infrastructure to make this possible to make the community create these kinds of things. Rather than dictate them from the top down. Even though I know we need some kind of top down organization I really, I'm a complex system scientist. And so I'm really committed to incentivizing bottom up collective action to solve problems. So that's, I would like to see an organization that can do that. So I don't want to say what what you're asking is really important. But I think I would rather see omf enable and empower the community to do that rather than omf do that. And, and the lady your last name's Lou, who's without us GS, and is doing their community modeling will be at the next workshop and in five weeks so she's, wait, who her last name's Lou. And she's heading us GS, kind of a modeling modeling archive or access group that they're developing us just is also making a lot of efforts to make model accessible. Yes. So she's joining us with that. That's a great name to end on it. Thank you. So let's thank Michael again for a terrific talk. Thanks Greg.