 Good afternoon. It's nice to have you here at the Energy Seminar. Today we have a special seminar where you'll be hearing from a number of faculty and students from around the country who've been developing the idea of creating a new discipline or perhaps research community focused on something we call macroenergy systems. And I'm not going to describe what this is because they're going to do a much better job than I could. But I can say I'm really excited about this topic because this idea emerged at least in part from a group of students at Stanford who created the energy systems modeling group several years ago. And this was an ad hoc group of students who came together from a number of different departments across the university, energy resources engineering, civil and environmental engineering, electrical engineering, economics, the business school, management science and engineering, and so you might be wondering what they all have in common. And what they had in common is that they were all working on the energy transition. So came the idea of creating a new discipline to describe this work, which eventually came to be called macroenergy systems or the science of the energy transition. So building upon this idea, Patricia Levy, a graduate student of John Williams in the management science and engineering department, and one of the leaders of the energy systems modeling group took the initiative to write a paper on the topic. And we were really fortunate that Joel took a chance on this and they published it. And in the article we invited others who were interested in this topic to join us to help build a community around this idea. And we were really pleased and maybe somewhat surprised that we got a very big and positive response. So over the next nine months, the team represented here today came together to organize the first ever macroenergy systems workshop. And that was held two weeks ago. And so the results of this workshop will be discussed in our seminar today. So joining us for this discussion are Ines Azevedo, who's an associate professor of energy resources engineering in the School of Earth at Stanford University. Then we have Emily Grubert, who's an assistant professor in the School of Civil and Environmental Engineering at Georgia Tech. Oh, yeah. And then we have Benjamin Hobbs, who's a professor of environmental management at John Hopkins University. And Jesse Jenkins, an assistant professor of mechanical and aerospace engineering. And at the Endlinger Center for Energy and Environment at Princeton University. And Ben Lebowitz, an assistant professor in the Graduate Program and Operations Research and Industrial Engineering at UT Austin. And finally we have Patricia Levy, who's the chair of the macroenergy systems workshop steering group committee. And also a PhD candidate in management science and engineering here at Stanford. And if you listen to all those departments that were represented by that list, I think that's really illustrative of why we need something new that will bring together all these disparate disciplines so we can work together more closely on the energy transition. So Ines, I will turn this over to you. So it's great to see everyone. So the organization for the meeting will be that now we'll hear from Patricia Levy on a brief introduction about the workshop and about macroenergy systems. Great. Well, good afternoon, everyone. I'm so excited to be talking to you all today about the work that we've been doing in macroenergy systems. So as Sally mentioned, I am a fifth year PhD student here at Stanford. And I've been working on this effort around macroenergy systems for about the last two-ish years. And it's really exciting to be speaking about this finally at the energy seminar today, in part because this seminar has showcased some of the best and most interesting work in the field of macroenergy systems. So to kick us off today, I want to take some time to talk about, you know, how we got here what we had vision macroenergy systems to be and our vision of what we think this field could provide for us. So Sally gave you a little background, but sort of my take coming into this is that I'd been working in energy systems for about eight years and the whole time, at least for the beginning of that, I felt puzzled by how disjointed and scattered it felt. And I assumed for a long time that that was just my unfamiliarity with the research and the community. And when I came to Stanford, I saw that the people I wanted to be learning from and learning with really worse scattered across several departments and even located up the hill at Slack. So I helped create and organize that seminar that Sally mentioned. And I got some of the best and most valuable research feedback in my PhD from that seminar. And that experience really led me to believe that the broader community working on sort of energy systems transitions could really benefit from more organization at all levels. And that really drove my motivation to help write the paper that we published last summer in jewel outlining our vision of macroenergy systems. And that paper seems to have resonated really strongly with a lot of people who have been looking for a name for what to call the research that they do and looking for a community to be a part of. So I'm really excited to share this idea with many more of you. Hopefully this resonates with you as well. And I'm excited to take away to share some takeaways from the workshop we recently had with a broader audience. So before I get into any more details, I want to take a quick aside to thank the people, many of whom Sally just mentioned, who worked with me to help develop the workshop. We spent a lot of time hashing out new ideas and plotting a path forward. So this wouldn't have happened without the support of the people you see here on the screen. And I want to give some additional thanks to Maxine Lim who has just been our outstanding administrative support at pre-court energy. I certainly would have gone crazy organizing this without her. So now without further ado, what exactly is macroenergy systems and why are we so excited about it? Well, macroenergy systems research supports transitions to more sustainable and equitable energy systems, which I think we can all get behind. So although research in energy systems, at least in the U.S., sort of started with the oil embargo, today this kind of research is really driven by larger challenges, most notably mitigating and adapting to climate change and improving energy access across the world. And macroenergy systems researchers have been working hard to understand these kinds of energy transitions for years, if not decades. And this common motivation has led us all to ask very similar kinds of questions. And so, for example, one group of these types of questions would be, you know, what technology portfolios do we need to meet the challenges of climate change and energy access? And how can society best allocate our resources to support those technologies? So this paper asked, you know, what fraction of our emissions do we not currently have good technology solutions for? And it turns out that about 27% of global emissions are from sectors like load following electricity or iron and steel that don't have a mature technological substitute. And we need more research in these areas to bring down our greenhouse gas emissions from these areas. Another paper looked at specific, you know, following on this idea of load following electricity looked at how much we need resources like geothermal, nuclear or generation with carbon capture and sea frustration to keep costs down in a clean electricity grid. And they termed these resources firm, low carbon generation and found that having access to these resources brings down costs substantially for very clean electricity systems, as you can see, you know, in the right hand side of this graph. Another important body of work in macroenergy systems looks at how we can expand energy access affordably and sustainably. For example, this paper asked what sources of electricity would make the most economic sense to provide electricity and how would that change with location on the continent of Africa? And it turns out that the traditional solution in many developed countries today of extending the centralized grid is not actually the most economical source of electricity in many places. It actually turns out that small hydro or photo voltaic powered small systems could potentially be more economic under a certain set of assumptions. A big group of questions for many macroenergy systems researchers is how policy will affect the evolution and use of energy systems. And this body of research can be quite important for advising the creation and development of new policy that actually moves these transitions forward. So for example, it asks questions like what effect would federal standards have on our national stock of electricity generation? So this paper shows how plants would be retired and built under different levels of federal policy versus existing state policy and shows that a federal renewable portfolio standard would have dramatically more effect on our generation fleet than the existing state based renewable portfolio standards. Another paper in this area focuses on how policy might change the allocation of investment if it prioritizes objectives other than cost effectiveness. So this paper looked at how investment with a fixed budget in an electricity system in a developing nation might change if it prioritized equity in addition to cost effectiveness. And here we see that more money is put into transmission instead of just putting that money into generation, but that electricity reaches fewer people. Finally, a growing body of macroenergy systems work looks at how our energy systems interact with environmental systems, human societies and human institutions. And it asks questions like how does our energy system use and impact water systems and what is the water intensity of different electricity generation technologies? Through this work, we've learned things like natural gas has a smaller water footprint than pulverized coal generation even when that natural gas plant is using carbon capture and sequestration. Other work in this area has answered questions like where does solar generation have the greatest pollution related benefits? And it turns out not in the places that people think of as being bright and sunny, actually a solar panel eliminates more pollution related damages both in the environment and to humans if it's installed in the eastern US. So all that amazing research is really just the tip of the iceberg in macroenergy systems. And I would like to argue to you all that that body of work has enough in common with itself that we can say it is all part of the same field. So what distinguishes this macroenergy systems research as its own field and distinct from disciplines like environmental engineering or energy economics? So we put macro in the name for a reason and that's because these are big transitions and they involve massive scales. And when I talk about massive scale here, that could happen along one of three axes. It could be energetic. So just the amount of energy that's involved, you know, multiple power plants, whole fleets of vehicles. It could be along the temporal scale. So the scale of time needed to have an energy transition. This could be years or decades. And finally, the final scale is the spatial scale. So over a state or a country and, you know, or a national grid. And if you get really big along any one of these scales, you start to talk about these really big systems that have really big impacts on policy, on the climate, on the livelihood of many people's on the livelihood of many people. And also that scale drives methodological choices that characterize macroenergy systems research. Because when you get to scales that are this big, it's very difficult to run an experiment. It's hard to run an experiment with say the entire natural gas supply chain of an entire country. They're not going to let you do that. So experiments are nearly impossible, meaning that we use a lot of simulations and models. And these models have substantial abstraction because we can't represent all the details of these enormous systems, or every single second involved in the immense, you know, time span that we might be concerned with. This scale also drives complexity, which drives interdisciplinary approaches because these large systems are touching lots of other systems. And to ask and answer good questions in a way that's meaningful for society, we really need to understand these other systems and bring in other ways of understanding. Also, this complexity encourages quantitative approaches and frameworks that help us represent complex systems and competing values in the same coherent framework. And I have to note here that quantitative approaches are not the be all end all of macroenergy systems research, but they are a common through line that help us make sense of these enormous systems. So these features of common methodological choices and the aspects of complexity, interdisciplinarity, and quantitative approaches in combination with all of these similar types of questions that we're asking, we argue define macroenergy systems as its own field distinct from other fields of research. And we're really excited about macroenergy systems because it could give us a disciplinary home for this interdisciplinary field. We all use similar tools like simulations and forecasts, computational models and abstraction, lifecycle analysis, learning curves, and other similar tools. And we use them to ask similar questions like the examples I just walked you all through. But right now, the researchers doing this kind of work work in different departments, we use different terminology, and we participate in different conferences and journals. There are some existing groups like the electricity systems integration group and the growing open modeling energy community, which are a major step forward in the right direction. They really don't fully address the needs of our shared research community. And I think that identifying ourselves as the same community as the same field could have so many benefits for researchers and for for research. A new field can support the best possible research for sustainable and equitable energy systems, which I hope is a goal we can all get behind. The commonly shared identity of macroenergy systems could facilitate better collaboration by making it easier to find others in the field, as well as, you know, having the same conferences or journals could make it easier to connect. And these collaborations can support better research. As well, working together, we can improve our tools and methodologies. Better communication means that we can avoid reinventing the wheel and instead collaborate on making the best wheel possible. I can't tell you the number of PhD students I know who have said that they built a tool that already existed, but they weren't able to get access to it, or, you know, they didn't know that it existed. So this could really go a long ways towards accelerating research. As well, macroenergy systems could support better academic capacity, because a commonly shared identity and reputation could help attract more faculty, more students, and the funding needed to support those faculty and students. And we're going to need that academic capacity because there's an enormous demand for education in macroenergy systems. We need to educate a professional workforce needed to advise the energy transition across, you know, cities, states, and nations, corporations, across the world. So this is just a taste of what I think the benefits of macroenergy systems could be. And as I wrap up my little introduction here, I want to share with you all sort of my vision for the future of what this could all look like if macroenergy systems succeeds in maybe five or 10 years. And I'm envisioning that we have targeted journals and conferences that support our research community where we can go and all see each other together instead of seeing some macroenergy systems researchers as AGU, some at IEEE, some at Informs, etc. I think that the credibility and visibility of macroenergy systems could make it easier to find collaborators, make faculty hires, and secure funding, because it'll be easier to explain what our value at is and what why people should care about our research. Macroenergy systems could make it better easier to recruit bright and interested students and make it easier to train them because we will have a commonly established curriculum. When they graduate, they'll graduate into a strong professional community that can support them along their path. Better communication within the community of macroenergy systems could lead to faster advances in our understanding of energy systems and the technology and methodology we need to understand them. And finally, I hope that a more established macroenergy system community could support better cross-disciplinary collaboration with researchers looking at human impacts, political institutions, and natural ecosystems, which I think is really important for enabling this transition to be equitable and sustainable in ways other than just purely cost and the capacity to meet electricity demand. So this is the vision that's been motivating us. And in driving towards that vision, we've been asking ourselves some key questions like where are we as a field? What are the major critiques of work in our field? How can we better educate our students? And finally, and I think most importantly, what actions would support the growth of our shared research interests? And how can we work together to create a lasting community? So two weeks ago, we gathered with some other researchers in the macroenergy systems community to discuss possible answers to these questions. And I'm really excited for you all to hear from our panelists today to talk about our takeaways from that workshop. Also, if you hear this and you're like, this is me, I'm a macroenergy systems researcher and I want to get involved, please, I would encourage you to use macroenergy systems, you know, as a keyword say on your Google scholar page, your journal submissions, start describing yourself as a macroenergy systems researcher in your bio to other people, etc. If we believe in this, this will make it happen. If you are really excited and you really want to contribute your time, leach out to us to help organize events and resources and please reach out to us to keep the conversation going. So on that note, thank you. And I'm going to turn it over to Inez for our panel. Thank you so much, Patricia, for such a fantastic presentation and for your leadership in making all of this a reality, indeed. So now we'll spend about 25 minutes with the panel members on some guided discussion. And as a starting point, I'm curious how each of you defines what do we mean by macroenergy systems and macroenergy systems research, as well as what are some of the key questions that macroenergy systems can address and that other areas or fields cannot address as easily. And so, Jesse, maybe you can start by addressing this question and then we'll open to the rest of the panel members. Yeah, sure. So I think for me, and I think you heard this in Patricia's remarks, the field of macroenergy systems is a very problem driven field. It's organized and its center of gravity sits around answering certain questions and questions of practical importance to the world around us. And that maybe stands in contrast to other disciplines or fields that are organized more around a set of specific methodologies or a disciplinary or theoretical perspective on the world that can be applied to a number of different problems. So because we're problem driven, it requires us to bring a large number of different tools to bear when trying to answer different types of questions as they pertain to what drives change in large scale energy systems or macro scale energy systems and what are the impacts of those systems on society, on the environment, and on other outcomes that we care about. And so it's, I think, an inherently interdisciplinary field and one that requires a strong cohesive organization of researchers who are essentially organized around answering the same sorts of questions, but in strong and consistent relationship with others in associated and allied fields that need to be brought into the conversation and into answering particular questions with their own particular theoretical methodological or disciplinary perspectives and people who kind of come in and out of engagement with these sets of questions. And I think what we're searching for is a greater sense of gravity or center of gravity around the community of scholars and researchers who spend the majority if not all of their time grappling with these questions around the evolution change and impact of macro scale energy systems. Great. Thank you so much, Jesse. I'm wondering if others in the panel would like to address also this question about how do you define what macro energy systems are? Sure, I'll go next. So I think Patricia hit on a few of the really key points, one of which is the stale and scope of the problems that we're addressing. So thinking about long time frames, national coverage, or even global models, and then the interdisciplinarity of the methods and all the things that go into our tools. I think two other distinct features of macro energy systems. One of them is that even though our core focus is on energy, we're also looking at the interactions between energy and other related systems. So a lot of us do work on energy and water, energy and food, or the role of energy in the broader economy. And then I think another feature of our work is that we tend to have a normative bent, where ultimately what we want to do is inform better decision making. So those decisions tend to be policy decisions, strategy decisions, operational decisions, but more so operating at the macro level. But we're not just trying to describe systems, we're trying to actually figure out how to intervene to realize better outcomes. I'll jump in and kind of follow up on all of the comments that have made so far. I think the interdisciplinarity of the system level nature is part of the reason why this is different from a lot of the kinds of energy engineering and that kind of topic that we see in some other disciplinary contexts. I think one of the major strengths of potentially moving this into more of a field or disciplinary setting is just really bringing people together to ask questions in a very particular way. So how does our work actually influence things at scale? What do we mean by scale? And what is it that we're actually looking at? I think one of the examples that people have given in the past about what differentiates macro energy systems from energy research more broadly is, you know, is somebody who's researching solar panel efficiency improvements in macro energy systems? Maybe not, but people that are looking at, you know, geographic migration of people in response to energy transition might be part of the field in much the same way as somebody doing more operational research might be. Thank you, Emily. May I jump in? So, Benjamin mentioned some defining features. I'd like to add one which is a serious attempt to engage with multiple objectives, not just economics or engineering economics, but societal equity objectives, public health, and others to quantify those and really understand the trade-offs between them. So an explicitly multi-objective approach complemented by a multi-level approach. Benjamin mentioned a normative approach. In a multi-level approach, there's some entity that's planning infrastructure, evaluating policies, and wants to understand how the underlying system and markets will respond to that. So the normative part is considering these trade-offs and which of these infrastructure designs and policies might be best, but you don't have control over everything. You recognize there are a lot of things you don't control that are determined by the markets and other entities that will, however, respond to the incentives and infrastructure you provide. Okay, great. So we're looking at problems that are large in scale that involve a multitude of stakeholders and where we have different potentially competing objectives to address. So under that lens, could you illustrate some of the key questions specifically that emerged that you would be passionate about addressing under the macro energy systems lenses, as well as those that you see being at really the frontier of research in this space? Yeah, I can start there. We had a nice discussion at the workshop on some of these key priorities, and I took a little time to try to summarize that in a slide here, which I can share. So the discussion at the workshop really highlighted a range of frontiers of research questions and macro energy systems that a lot of us are grappling with. And, you know, we sort of charted a path of evolving questions from primarily a set of technical questions about how novel technologies would even work and at scale in complicated energy systems. So what happens when you add large penetrations of wind or solar photovoltaics or you rely on off-grid power in developing economy contexts? From a more technical perspective, would this work? To a set of questions that were primarily around economic trade-offs, so what were the least cost or cost-optimized strategies that could, you know, marry economic and engineering objectives to try to meet important outcomes? And I think the third phase that we're in now, which motivates a lot of our work, touches on more complex trade-offs across a variety of dimensions, as Ben Hobbs was just saying. And so some of those questions are concerned not just around the total cost of an energy system transition, but the distributional impact of who wins and who loses, what benefits, what costs are distributed across different populations, different types of people, different geographic scales. And so questions around equity and distributional impacts of energy system transitions and decisions that we might make about how to guide those transitions are, I think, pretty paramount. Evaluating quantitatively and illuminating trade-offs across a variety of different objectives is a key area. So, you know, maybe that certain policy objectives can be achieved simultaneously while others may have strict and important trade-offs between them. And we have to provide light, shed light for decision-makers on those important trade-offs that may not always be intuitive due to the complex nature of these systems. And we're seeing increasing interlinkages of different energy systems. So I primarily study the electricity sector, but the electricity sector is becoming increasingly enmeshed in transportation through electrification of vehicles and in heating through heat pumps in information technology, et cetera. So there's an increasing need to study the coupling of multiple energy sectors and linkages across them to factor in institutional and behavioral concerns. So, you know, we might study these from perspective of an engineer or, you know, perfectly rational economic quant, you know, actors. But of course, reality is much more messy and institutions, their concerns, their limited influence over the system and human behavior, you know, and our sort of all of our quirks and irrationalities are important factors of understanding these systems. And so trying to factor in insights from political science and sociology and behavioral economics and other fields is an important new area. We study energy systems in transition, and no one really knows where those transitions are going to go. And so thinking about dynamic feedbacks and changes and how certain factors may end up driving non-miniarrities that, you know, we might not immediately see. And thinking about the uncertainty that we face in long-term transitions, much of which we can consider as deep uncertainty where the probabilities of certain outcomes are really unknown and really unknowable in many ways. And so we need to try to provide decision-makers and strategists with strategies that are robust to that. And we want to move quickly to address some of these questions, particularly climate change and the urgency of climate mitigation and the need to provide energy access and modern energy standards to our standards of living to billions across the planet. And so trying to understand barriers and bottlenecks to the rate of change is and how we might overcome or accelerate change, overcome those bottlenecks or accelerate change are also key. And I think what links all of these areas is a desire to provide improved decision support tools. So tools that people can use to understand complex systems and actionable insights that can be derived from the practical application of those tools in order to guide real world and practical decisions, whether that's policy or investment decisions or where we should direct our innovation effort and our research effort. All of these factors are critical. And wherever they touch upon the complexity of large-scale energy systems, we can develop better tools and better insights to help guide those kinds of decisions. So, you know, again, as you said before, it's a very problem-oriented practical field, I think, and the kinds of questions reflect that. Would any of you like to react to Jesse's description? I think one thing to kind of flag that Jesse started to mention that really characterizes some of the most interesting questions in macroenergy systems to me at least right now is that we are facing a lot of non-stationaries in engineered systems and in natural systems that we haven't really had to deal with before. So to kind of jump on to that deep uncertainty issue and the just general issue of dynamics, I think we have a lot to offer in terms of thinking through in a very strategic and decision-oriented way how we can plan around those non-stationaries and future uncertainties. So moving, let's move forward. So Jesse and Emily highlighted well some of the interesting questions and frontiers associated with equity and distributional effects, the multi-objectives and the blurred lines between sectors that are making all these questions much harder to address than in the past. So with the opportunity of making a contribution to this area of research with important questions and new methods for research, also comes the criticism of what can macroenergy systems achieve or not. So this is a question for Ben. What emerged so far as the most important critics of what macroenergy systems research can deliver? Yeah, so we decided to have this panel discussion where we tried to survey the many criticisms of our work that are out there. So as a young and rapidly evolving field, it's important for us to realize where we can improve our tools and try to have an open discussion about what are some of the priority research areas. So here you can see that we organized that discussion around 12 common critiques of our models and our work more broadly. And these come from a lot of different angles. So from economists, from engineers, from stakeholders and practitioners. So not only are some of these criticisms maybe controversial, but some of them even directly contradict each other. So for example, this tension between our models being too complicated versus being too simple or not having enough detail. But I just wanted to highlight really four specific areas of criticism that really emerge from our discussion as big priorities for us to address. So the first one of those is the one you see here on tail risks. So what we're talking about there are really the possibility that climate change over the long run could be much worse than we actually expect in terms of our average case. And also tail events such as large natural disasters like catastrophic wildfires or hurricanes and how to actually prepare for those. Another major criticism was the ways that our models represent the economic impacts of climate change. So the overall sense is that we do a much more thorough job modeling the costs of different transition pathways. But of course, if you're doing a cost-benefit analysis, you also need to understand the economic impacts that you could avoid by mitigating or adapting to climate change. So there was some sense that a lot of the models are using obsolete information on the economic impact side or that we're representing economic impacts in too aggregate of fashion using things like the social cost of carbon or a single economy-wide damage function. The third major criticism was that our models and tools don't have enough policy realism. So a lot of us are used to running scenarios based on these sort of economy-wide or almost stylized policies like constrained carbon emissions and watch your model tell you what's the least cost pathway to satisfy that constraint. Or maybe you impose a carbon price, but really what we see on the ground is especially here in the U.S., a lot of the policy instruments that are having meaningful effects are much more granular and sectoral policies. So thinking about efficiency standards in specific energy end uses or even a lot of municipal or city-scale policies that might not even be intended to target energy and climate change, but nevertheless have big energy and environmental impacts. And then the last theme that really resonated and Jesse hit on this too is that our models need to do a better job representing the distributional impacts of climate change itself and of the policies we enact to mitigate or adapt to climate change. So when we talk about distributional impacts, we're thinking about distinguishing impacts on different income groups or racial or ethnic groups. So those are pretty well understood, but policies can also have very different impacts on groups like urban versus rural or homeowners versus renters. So increasingly, I think there's a push to try to understand not just overall costs and benefits, but how they're distributed. Thank you, Ben. And keeping you on the spot for just a little bit longer, what can we do about these criticisms and limitations? Yeah, great question. So that was another theme of this panel discussion. So once we identified these major critiques that we need to respond to, we actually just took stock of ongoing research efforts that are actually making progress toward addressing some of these issues. So just to give you three quick examples on the distributional impacts front, we heard from Dr. Jennifer Morris from MIT, where she was talking about how in their general equilibrium model EPA, they've gone from having a single representative agent that represents each country's consumers. And they've actually started to disaggregate according to different income groups to look at the effects of climate change and climate policies on those different groups. On the issue of tail risks, we had a nice talk by Delavine Diaz from EPRI, where she was showing how they've incorporated uncertainty into models that try to project the impacts of climate change. And these are models where they've actually started to disaggregate damages into many, many sectors. So that work, for example, has highlighted that there are certain sectors where climate risk is really prominent, such as agriculture, cooling and heating, and water where there's big uncertainty about climate change impacts. And then a third area I want to highlight where there's actually a lot of progress being made is on this criticism about the lack of transparency. So I think in the last few years, we've seen this real proliferation of open source modeling tools. So Dr. Joe DeParelis from NC State is leading a group on this Timoa modeling infrastructure. There's a similar model platform called Osmosis, which is based out of the UK. And then there's a big project going on in the EU right now called Open Entrance. And all of these are really efforts to make the models more transparent and accessible and allow people to more easily combine and compare models. Excellent. Thank you so much, Ben. I'm wondering if anyone else in the panel would like to react? And then Ben Ops, yeah. Sure. So many of these are issues that the policy analysis community in agriculture and education and water resources has been dealing with since the 1950s. Perhaps a number of these problems are more pronounced than energy modeling simply because we can build bigger models because we have more data. But we can learn from other fields. And likewise, those fields can benefit from the improvements and dealing with issues such as equity and others that Benjamin just described as well. So these improvements, these enhancements are not happening in a vacuum. And we have to be sure that we retain connections with other policy communities that deal with similar problems. So Ben, it's great that you brought up this issue of learning from other communities and other areas of expertise. So in that sense, what sort of educational efforts do you think would be warranted to grow this area of macro energy systems? You won't be surprised to know that there was a panel exactly in this topic. Thank you, Nez. And this panel included some folks who've been doing energy modeling since the 1950s, such as Fred Murphy. And we also had John Ryan to Bill Hogan involved who really gave us some great perspective on what is best learned within the university and what's best learned outside. I really encourage you, by the way, to look at the PowerPoint slides that will be posted for many of these talks. We also had younger educators talking about some of the innovative methods that are used these days to engage students and really give them confidence in modeling and achieve some of the goals that, for example, we've been talking about an open source coding. One thing that I really enjoyed about this panel is that all 50 or 60 people who were there participated in breakout groups where we asked several important questions about, well, how could your experience have been improved in in your education? And number one was more experience in coding and managing large codes. But the next five topics were all about just thinking. What is the problem that you're really trying to solve? How do you communicate this? How do you integrate different disciplines? The second question that we addressed is we asked everybody in the room, what are the neglected topics and skills that you think should get more emphasis? And the social aspects, whether it's communication and non-modelers or political science, political economy, were voted the top four. You know, definitely coding and the art of model building and things like finance are down there as well. But there was a feeling that these aspects, don't call them softer, not sure what we would call them, are going to be critical to the success of this field. A third question we asked is, well, what were you glad that you learned about? You've told us what you missed. And perhaps you won't be surprised that people feel like we really have great competencies in modeling in the operations research economics paradigm. And so dual variables, we're really glad we learned about those. So nobody is saying that we should give these things up. So somehow we have to figure out how to include the social and broader aspects of modeling and communication and modeling while retaining these skills that really differentiate a lot of the folks in our community from the other folks and teams who are going to be addressing these problems. And finally, we talked a lot about how we should document and communicate the ideas and techniques. And we asked in particular, should there be a textbook? And there was a majority who thought about two thirds majority thought, yeah, sure, it'd be great to have a textbook ask the person sitting next to me to write it. But actually what got the most enthusiasm was a combination of a web resource where people teaching at different places such as Hopkins and Stanford can exchange materials. And also a Khan Academy type collection of videos from people like the Bill Hogan's and John Wyance and Fred Murphy's who can describe their perspectives in short adjustable videos that our students can benefit from all over the world. Thank you so much, Ben. And finally, let's think about the sorts of strategies and infrastructure that could help this area grow and be successful. So Emily, what did we learn about considering the strategies and infrastructure at the workshop? That is a great question. I think one that I've seen coming through in the Q&A a little bit as well is this sounds interesting, sounds like something that might be worth doing. Now, what do we actually need to do for this? And I think one of the real challenges that comes through it's really thinking about what this looks like in practice from a number of pretty logistical frameworks. One being, you know, if you want to have a conference and you want to have a journal or something along those lines, how do you pay for it? I think as we were talking through various options for what infrastructure might look like in the future at the workshop, there was a lot of enthusiasm for kind of emphasizing this community as a thing that exists and connecting researchers. We'd like a lot of ways to find and learn from each other. And so there was quite a bit of enthusiasm about possibly trying to kind of recognize which conferences are already the ones that have great sections with people that are doing any S-type work, maybe before trying to launch our own. Similarly with journals, I think there was a general sense that we kind of have enough journals, but we do need to make a little bit more of a concerted effort to ensure that there are places where we feel like we're getting appropriately critical, but also appropriately fair types of reviews. So I think one challenge that comes up for a lot of us is just you send your stuff out to kind of a methodological disciplinary journal. A lot of the time you get some pushbacks that you maybe would not get from people that are more used to asking these kinds of questions at these kinds of scales. And to follow up on what Ben was just talking about, there was also a ton of enthusiasm for kind of consolidating T-string resources and understanding who else is working on these types of things. I think one of the real challenges that we discovered as we talked through this at the workshop and something that we as a team are still thinking a lot about. So for all of you coming to this energy seminar, please do reach out if you want to actually actively help develop some of this infrastructure. One of the things that we'd like to preserve about this community is kind of the friendly and welcoming culture and very big tent view of the interdisciplinary approaches to a topic. Like Jesse was talking about, it's actually kind of an unusual thing to organize a discipline around the topic rather than a method or core theory. I think one of the real challenges we have with this is kind of navigating what that actually looks like in terms of ensuring that we keep the friendliness and we keep the curiosity and all of that while also acknowledging that we do have some some things to work through if we want this to be a field over the next decades, centuries, whatever. I think one of the things that came out in a conversation that I was part of that made me laugh quite a bit is just thinking about the fact that environmental science, environmental studies, and environmental engineering are all separate. Why is that? When we think about what this looks like from a macro energy systems perspective, how do we actually address the fact that there are reasons that people have split methodologically before and we would like to really think through what that looks like. But I know for me personally a lot of my community in this area does qualitative research thinking about how to include that with the kind of quantitative thrust of what we're talking about is a pretty big challenge as well. I think the overall impression is we do need some infrastructure and we need to think very carefully of the community how we want to move this forward. So you're welcome to join us. Thank you so much Emily. Any reactions from others? Just wanted to have a quick echo of Emily's last point about sort of the need to maintain strong interdisciplinarity with our work in relationship with other communities. I think a lot of us that do this work are really hungry for a stronger center of gravity that can kind of pull us together. But I think it was a pretty consistent theme that we want to make sure that that's not like a black hole, right? It's not a singularity that pulls us to become so inward looking and self-referential that we lose what's so rich about this field and the relationships with all of the adjacent and allied disciplines. And so I think that tension is really central to what we're trying to create and I think it was really great that that was surfaced in the workshop and you know resonated with a lot of the different people and in several of the different breakout groups that we had for this this panel. Thank you so much to all of you for sharing this tremendous information. This was indeed a very exciting workshop and a very exciting seminar where at some point we had 240 people participating so there is definitely curiosity towards what is going on and hopefully also interesting continuing this effort. So we'll wrap up the portion of the seminar with the panel members and we're going to start addressing questions that we've been receiving through the Q&A. So I'll be starting in a minute and actually an initial question to the panel that I saw joining into Q&A is really how are macro energy systems research-related questions different from sustainability system questions and methods that needed to be addressed? I can take that one so a lot of the work that I do is kind of life cycle assessment focused and very much focused on how do we define sustainability and how do we quantitatively assess sustainability across social economic and environmental factors. I think that comes into a lot of the conversation with sustainability transitions. One of the challenges that I think we face as macro energy systems people is taking a very energy-centric view of that type of question. I think what we talked about a lot at the workshop is that we do have a variety of methods and approaches to analyzing some of these things but a lot of the time really truly delving into macro energy systems questions really does require quite a bit of topical expertise and so the types of things we think about with sustainability are very relevant for the types of things we think about with MES but actually having that strong core focus on energy is kind of the the guiding principle here does differentiate this a bit. If I if I can add to what Emily said so great point Emily and I'd like to remind folks that there's a there's a tradition in development economics that if somebody is is trained as an energy person they think the solution to rural development is energy access or if somebody's trained in education is about education or if somebody's sociologist they might emphasize say the role the role of women and it's important to understand that development needs to involve all these things we have to be careful about you know we we have this energy wrench and thinking that it's the answer to all the problems. So the next question will be how are we thinking about best positioning macro energy systems to help policymakers understand its value and put put it to use and before I hear from you I'd like to invite Patricia to also put her camera on and mute as she may have some of the the critical answers to the questions being posed so in terms of interaction with policymakers and informing the decisions being made by interacting with policymakers any specific strategies that any of you would like to propose. So I think it starts with the kinds of questions that you ask to frame your research and to ensure that a lot of that if not all of that for me anyway those questions come out of engagement with real world decision makers trying to understand the questions that they're grappling with and you know the way good ideas good research questions for me typically come about is you know as a you know content expert or domain expert in this area I engage with policymakers a lot and you know strategic decision makers and companies and you know R&D you know efforts and the last questions that you know try to guide their their thinking and I may have a ready answer and maybe from my work and maybe from you know one of my colleagues that I know or paper I read and if so that's great I can pass that along and if I don't have an answer it's often that it's an answer it's a question we don't really have a good answer to I mean not always I don't know every paper in the in the world but I you know have a good sense of where the the state of knowledge is and so when you find a question that you don't have a good answer to that means it's a good science scientific research question right it has interesting academic novelty so that's great and if it's coming from conversations with practical decision makers grappling with that question right now then it's an answer that needs questions yes or needs it's a question that needs answers yesterday and so that makes it a good practical motivating question to work on and so I think when we marry those two together we can ensure that the work we're doing is motivated by and can provide practical you know answers to real important questions and then you just have to keep that dialogue going you know as you complete your work and move on to the next question and evolve over time and so you know that's a challenge for us to be in as researchers to be not just in our ivy towers but in active engagement with you know people out there making important decisions on a regular basis yeah and just to briefly add to that I want to just go back to something Ben Hobb said earlier which is this theme of multiple objectives so I think it's important just for us to understand that the objectives we often have in our models aren't necessarily well aligned with the objectives that policymakers have so you know we might be able to present here's the least cost optimal pathway but maybe what they want to know is you know how many jobs will this create or you know okay uh this reduces emissions but uh will it shorten people's commutes to work and make them happy so I think that we need to sort of look for the questions that are being asked by policymakers and then try to uh yeah just like Jesse said uh modifier expand our tools to be able to answer those questions yeah I'd like to jump in on this as well I think Jesse and Ben raised some great points but uh two two sort of additional ways I see macro energy systems sort of organizing this research community to better help policymakers is there's there's two sort of major things I'm thinking of the first is that um if we have sort of a more of a coherent research curriculum sorry not research curriculum educational curriculum um the people who are being trained in being macro energy systems researchers will hopefully get exposure not just to sort of economics or optimization but also to you know some social sciences and also alternate ways of you know quantifying benefits dealing with multiple objective situations and just the sort of type the type of education that would help them understand what policymakers need better than somebody who's received say a pure or education and then decided to start applying that to energy systems and it's just learning that on the fly and the other way I think macro energy systems can work towards uh supporting policymakers better um is that you know sort of by having sort of a coherent research community and reputation um if somebody say in the California Public Utilities Commission is saying you know I really like to find somebody to work on this this question that I have you know who would be able to answer this question that they'll be able to go and google you know macro energy systems researchers um and actually find us you know we a lot of the researchers I know in macro energy systems as I'm sure you've heard from this whole panel we uh got into this because we want to help change policy um but making that connection to policymakers can be challenging for a variety of reasons um and this is just one way that I think uh you know so having a coherent identity could help facilitate that transition because we're really saying hey we're we're here to help thank you and with that I think we're uh getting um at the end of uh this seminar um I would like to thank each of you for uh phenomenal sets of uh responses in this uh unusual format uh we hope that uh everyone that joined the seminar in addition with uh other folks that are out there will start to actively participate in uh helping us grow uh this uh initiative and a final big thank you again to Patricia for making this happen and we hope to see you in future energy seminars too thank you so much everyone