 All right, good morning, and thank you, Jimmy. Thank you, Steve, for kicking us off. So I'm delighted to moderate the first panel discussion and presentation. So let me just give you the overview of today. So this panel is about understanding the requirements and the opportunities. The next panel will talk about the international perspective and the final panel in the afternoon will talk about the technological options. So for this first panel, I'm really delighted to welcome our colleagues from industry and also from academia. First, we'll hear from Clyde Loutin from the California Independent System Operator. And Clyde is going to overview the current status from the perspective of Kao Aiso and also the requirements looking at some of the recent data coming in. Next, we'll hear from Erin Menier from EPRI. And she will speak on a very important aspect, which is when and how much energy storage do we need? And then finally, Adrian, who is from our department here at Stanford. He is a returnee, if that's a word, from industry. For the past 10 years, he was the founder and CTO of Empower, a battery cell startup here in the United States. And he's going to speak about how fast we're able to learn these new technologies and how can we speed them up. So without further ado, I would like to invite Clyde to come to the stage and then share the Kao Aiso perspective. Clyde. Well, good morning. And thank you, well, for inviting the California AIS to participate. So before we get started, I'd like to talk a little about the California AIS. So what it is we do. So we maintain reliability by delivering energy across most of California on a small portion of Nevada. We also look at the transmission planning process. Look out a few years, sometimes up to 10 to 20 years, and figure out what it is we need on a grid and ensure that we have enough transmission capacity, generating capacity to meet that demand. We also drive innovation as we have more and more renewables coming out to the grid. One of my responsibilities is looking at how well we can form or comply with the NUC real-time control performance standards. So I get a look at the system on a minute-by-minute basis to make sure that as we integrate more and more renewables, we can comply with those standards and also determine what it is we need to operate a reliable grid. For some of you that's familiar with the Western US, we also operate an energy imbalance market, which is every five minutes, we ensure that the energy we serve through the market, through 22 participants currently, they get the cheapest energy every five minutes to meet the demand. And then we also have servers reliable to coordinate in the West, which really is like your air traffic controller that looks at individual planes. So we look at the individual BAs in most of the West to ensure because we can see everything that goes on and direct individual balancing authorities what it is they need to do. Quick, I'm just going to hit on a couple of these points. California IASO peaked at 50,061 megawatts last September, September 6. That was quite a day. We operate a market that's about $729 billion based on 2022 information. We serve about 32 million people in California and a small part of Nevada. We operate one of nine IASO's RTOs in the US, North America, Canada. With that, a little bit about the EIM, EIM is helping a lot with renewables. So we tend to utilize EIM a lot to mitigate curtailments that we see on the system. So currently, as I said, we've got 22 participants. And since the EIM started in 2014, we already saved EIM participants roughly $3 billion and reduced CO2 emission by 800,000 metric tons. Real quick here, as we integrate more and more renewables on the system, what it is we're seeing currently? Well, we tend to see a lot of oversupply during the middle of the day, especially on weekends. And we're looking for ways now, and this is hopefully some of you guys are going to have ideas how we could avoid curtailments or minimize curtailments. We're looking at new technologies, consumer programs. And the storage is right at the top of the atlas. Storage batteries, the IASO, is neutral. So we are agnostic to any type of technology to store energy. The amount of response to what that is is how can we move shape and shift load to meet the current verbal energy supply that we see on the system. Time-of-use rate, we had recommended time-of-use rate where when we have an abundance of energy, we propose the cheapest energy rate where you could incentivize customers to do whatever it is they need to do to help the grid. Another last bullet, the electric vehicles. We're looking at ways now, how can we use electric vehicles to regulate the grid? Because if you think about it, electric vehicles, even though it's on the distribution, it impacts the system frequency. And if we can figure out ways to utilize electric vehicles, I think it's going to go a long way, helping to control or regulate frequency on the grid. By that, what I like to tell folks is if you buy an electric vehicle and you're going to go, let's say you need to use 60% off your charge tomorrow, you can just, an app, you can say, you want to participate in regulation and service. Everything above 60% charge, we can use that to regulate the grid up and down to maintain frequency on the system. It's a simple concept. I think it's easily implementable. It's something we need to start thinking of, how millions of electric vehicles sitting out there is going to be very difficult to control these through a centralized control system. But I think through frequency control, we can make that work. Now, this is a plot really starting off. Back in 2018, we were looking at, well, what it is we need. Now, everything that's shown here is actual generation or production that we saw on the grid for March, April, May in 2018. The blue is the actual demand on the system. The red dotted curve is the net, which is your demand minus wind minus solar. And you can see the abundance of solar we see on the system during the middle of the day. And that dotted line that you see is actually the five minute energy prices. You can see a shape there, right? The five minute energy prices is pretty high just before sunrise and then during sunset. So we had predicted that this is a good area here right before sunrise. It's a good area to discharge batteries. And just after sunset to discharge again. And during the middle of the day, you charge a lot because that's what we tend to see oversupply on the grid. Now, the area between the solar and the blue is where you'd have things like hydro, you'd have natural gas plants, you'd have imports filling in that gap. But this is one opportunity that you can see where batteries could come in or any kind of storage where you can help out during the morning and the evening. Now the reason why this is important is just before sunrise, if you look at 4 a.m. 4 a.m. is when we tend to see minimum loads on the system. So the operators would tend to shut everything they don't need to shut that down. And in the old days, you had that blue curve where from 4 a.m., you committed resources to meet your pick demand for the end of the day. Well, now, you cannot really do that because as soon as you start committing units, you have these solos coming up. And you're gonna have to shut those units that you committed down. Well, that's a challenge because a lot of combined cycle plants, they may have one stop start for the day. And if you need that unit for the evening, you may not be able to get it. So it's really an interesting balancing game you're gonna play here, how storage can come in. And you can see during the middle of the day, that's an area where you can charge quite a bit. Now, this is actual generation for May 25th, sorry, March 25th. And on this day, if you look at the red, this is actual curtailment. That's enough curtailment for roughly 80 to 100,000 homes. So that red area right there is oversupply. And what is this really telling us, right? The challenges you can see right off the bar, the need for storage on the system. We're trying to minimize this red area here. My concern is the imports are coming in. When it's hot in the rest of the West, it's really hard to get that import. And you can see how we shaped the imports today. During the morning, you can shape that. And the previous slide I showed you is where storage comes in handy just before sunrise and then just after sunset. So we started to see that play out. Natural gas and hydro. Again, this is on March 25th. And I'm really excited to see what's gonna happen towards the end of this month with the amount of hydro that we got and the snow melt. This plot's gonna look pretty interesting. So again, this plot shows you the amount of curtailments that we see and how do you really manage that? Well, if you look back about three weeks ago, April 23rd, this net load, which is the red dotted curve, is really encroaching onto you and not as possible generation, right? So the very bottom there is the two nuclear plants we got in California. Just above that, you've got things like geothermal, biomass, bar gas. And you can see how flat that is. So it's really non-dispatchable. Now, when you have a lot of wind and a lot of solar, you have operating conditions like this, during the middle of the day, you tend to have no downward dispatchability. So when, let's say solar kicks up or wind kicks up, you have nothing to back up. So some of the problems we face is things like high frequency, high air control error. You try to export as much as you could and you try to minimize curtailment. So this is what we started to see. What this also tells you is we need essential grid services from inverter-based resources, wind, solar and storage. We need, I'm gonna say essential grid services, we need things like voltage control, frequency control and the ability to ramp up and ramp down. So in other words, if the system operator needs help and instruct a battery or an inverter-based resource to move up or move down, we like to see that unit move at a control trajectory. So the renewables, they're pretty quick today, but being quick is not good in some instances because we get a control frequency. When we control frequency, what we really do is we get benchmark every minute so how well we support the interconnection frequency. So having enough, too much generation is a bad thing. Having too little generation is a bad thing. So you're gonna try to balance that as best as you could and you need things to fill the gap, right? So on this day, we exported quite a bit. Again, we curtailed quite a bit and the takeaway from this slide is dispatchability, during the middle of the day, how do you control system frequency? And at the end of each month, we got a report to the North American Electric Reliability because how well we supported the interconnection frequency and it's not a good thing to have us call less than 100%. So you try to maintain that, do everything that you could to maintain that system frequency. This is looking at system peak on September 6th last year was the highest demand we saw in the system which was 52,061 megawatts. This is an interesting plot when you look at it, California imports a lot of energy. So that very green at the top, it tells you we rely a lot on imports. Well, what happens now is when we have consecutive hot days in the US, where do you get that imports? Everybody need that energy for themselves, right? So prices go high, it's really hard to get that import and you can see we have a lot of solar. But what again, this chart tells you is when it's hot, you get very, very little wind. So when we, like the previous plot, when we had a lot of wind on days when it's really hot and you need that support from renewables, you don't get it. So again, this plot shows you the need for storage. It shows you the need for storage greater than four hours. So during the evening ramp, what we had to do in some cases is quickly charge batteries so that we'd have the batteries available when the solar drops out or drops at night. And again, you hear a lot of folks saying, well, there were days when California ISO served. Well, last Wednesday, we served 100% of the energy needs for 63 consecutive minutes from renewables, which is a huge task. So some folks would say, well, you don't really need natural gas anymore. But on hot days like this, yes, you do need natural gas. You do rely on imports. So we've got to figure out, storage plays a huge part in how we operate the grid. Now we've seen this in the West. Folks in the rest of the country, they still do not have a lot of renewables. So they tend to, we see things when I attend meetings and I say, well, these are the challenges we've seen out West. A lot of folks, they say, well, we don't see that yet. So again, this is a pretty interesting plot. If you look at the hydro, during September timeframe, actually from late June, July, August, you don't have a lot of hydro on the system. Places like Pacific Gas and Electric, they utilize the hydro pretty judiciously so they can save that energy to meet the peak demand where the price of energy is high. But for things like ramp flexibility, you don't have a lot of flexibility. Now, looking in terms of long-term storage, again, I like this plot that it shows you, again, this is actual data. When you look at, in 2019, we saw a whole week where we had very, very little wind and so on. So the challenge here is how much long-term storage do you need? So there are studies ongoing right now with the California Energy Commission trying to figure out, well, do you need something longer than four hours, something longer than eight hours, or can you stack four hours storage or eight hours storage to meet the demand? We do not have a lot of pump storage in California. We, within the ISO footprint, I think we got a little over 2,000 megawatts, but that's not enough to show you the kind of curtailments that we see. Now, again, here you can see we rely a lot on the gas to provide things like ancillary services, but when you have over 20,000 megawatts of wind and solar, your production is less than 4,000 megawatts, you can see that's a challenge. You can see the need for storage on the system. You look to the right, the right shows you a week where we had, in May, you had a lot of wind, a lot of solar, you still had a lot of hydro, and that's what we're gonna see at towards the end of this month, and you back off your gas fleet quite a bit. Now, this also poses problems for system operators. When I say it poses problems, when you back your gas fleet all the way down, you gotta comply with things like frequency response, what that means is if you lose a big unit, and the new standard from FARC now is if you lose anything in and in the connection, you got 52 seconds to help recover, right? Or rest frequency and recover. So having that capability when you're back, your resources way down, sometimes it's not there. If you look at the hydro, the hydro, it's a lot of hydro, when you have a lot of hydro, you get very little ancillary services from hydrodita spilling, which is the operator pretty close to the max, and you get no ancillary service up, and they wouldn't deck. So this is where storage comes in, honey. So this is another area where storage can come in, not only to meet or to generate when the price is high and charge when the price is low, but ancillary services. We're looking at ways where storage can participate in things like regulation, frequency control, and it's something you need to operate a reliable grid. So on this slide again, by 2025, the two nuclear plants we have in California, which is the bottom, that's gonna go, but you can see again how we operate geothermal biomass biogas, you get no flexibility out of those units, and you gotta rely again on storage, you gotta rely on your existing gas fleet, and your hydro, and utilize the imports as best as you can. As I said, one of the biggest challenges we have is getting out imports on hot days when the rest of the west is hot. And most of you are familiar with the duck curve. And moving forward, how are we gonna ensure a reliable future? And we're doing everything currently. We're getting a lot of help by trying to make that duck flat. So I talked about things like demand response, how can you shape and shift load to flatten that out? Because this ramp right here on evenings, we anticipate that this year, not this year, by next year, to hit 20,000 megawatts. And ramping up 20,000 megawatts on a gas fleet is not really a practical to say. So we rely a lot on imports. So that's one area here where batteries can help. They can ramp pretty fast. And so we talked about time of use rates. That's gonna help here. Electric vehicles, so everything that I mentioned before, and whatever you guys can think of that can help us, move from a citizen duck to a flying duck is welcome. You can talk to me after this meeting of open airs. And we wanna make this work. And I'm optimistic that by 2030, we can serve 60% off-demand from renewables. And I'm pretty optimistic that we can collectively operate 100% carbon-free grid by 2045. All right, thank you very much, Clyde. So please hold your questions. We'll do a Q&A at the end, following the two additional talks. So next, let me invite Erin Manir from EPRi to share with us on when, how much storage do we need and how much can we pay for it? So Erin, please. Thanks, hi everyone. Erin Manir with EPRi. I manage the energy storage and distributed generation program at EPRi. We do a lot of research on energy storage from analysis and planning and economics to technology evaluation and implementation. How do you get these systems deployed in the field? So I'll talk a little bit about one of the analysis projects that we did. And just to let you know upfront, I don't have the answers. We are evaluating it. It's a continuous journey that we need to go on to keep reassessing the situation as we get new information. So some of the questions to talk about. What is long duration energy storage? Why do we need it? When and how much? And then what are the performance and cost requirements? So again, it depends. Depends on who you ask. It depends on the assumptions of your scenarios. And it depends on how quickly the market evolves. So what is long duration storage? Stephen defined it as three days. I think we saw in California some of the community aggregators put out a procurement a few years ago and they were looking for eight hour storage. DOE put out their funding opportunity for long duration storage last year and they defined it as more than 10 hours. So I think it's a moving target and there's lots of different flavors of long duration storage that feel different needs. Why do we need it? I think this one is a little more clear. We all know decarbonization, high renewable penetration will require storage. There's also resilience, natural disasters. Again, here in California, we have wildfires with public safety power shutoffs. So there will be needs for long duration storage for both decarbonization and resilience. So how is storage used now? I won't go into this too much because we just heard from Clyde and I took one of the Kyso graphs as well. And right now we're seeing a lot of four hour storage. Four hour storage largely because many of the ISO and RTO markets for resource adequacy capacity have a four hour duration but that's not true for all of them. In some regions they may have a six or eight hour requirement to receive full credit for capacity. And so again, there's regional differences and then just how the grid is evolving, how renewable penetration, how storage is evolving. And so that four hours as the cost comes down and it becomes more economic we'll see longer duration technologies being deployed. And so as we start to flatten or shave off the top of that peak, the peak just gets wider. So we will need longer duration storage to cover that peak. And then thinking about, we talked about multi-day, the sun's not shining, the wind's not blowing. How do we keep a reliable power system during those periods of renewable droughts? And so there's an opportunity for weekly storage that can cover multiple days. And then taking one step further talking about balancing severe and very high renewable penetration scenarios when you have a significant amount of storage, we don't wanna, or solar, we don't wanna curtail all that. We wanna store it, shift it. And so when you get into these scenarios of over generation seasonally that's the opportunity for seasonal storage. All right, so now I'll take a little bit more time to talk about a study that we did at EPRI that looked at when do we need storage, how much. We use this, EPRI has a tool called Regen which looks at the system-wide economy and takes in a lot of different factors about policy, load growth, generation mix and determines what the optimal mix is to serve the needs of the grid. And so the question of when and how much really depends on the scenario. So this is one example of different policy scenarios and how that impacts what storage would get selected in an optimal system-wide perspective. And so this reference case was for 2035 or all the cases were for 2035 but the reference case is assuming today's policy is enacted. So you've got a lot of targets in certain areas. You've got storage targets, renewable targets decarbonization targets. So the reference scenario is what exists today. The net zero is that plus carbon removal. So different technologies are carbon removal strategies and you can see that you start to need longer durations of energy storage when you go from the reference case to a net zero case. So taking it a step further, looking at carbon free. So carbon free does not include carbon removal as an option but does include nuclear as an option. And so again you see as we get increasingly stringent on the decarbonization policies, then you will see longer duration storage needed. And so in this scenario you start to see the daily shifting of storage and then start to get into that weekly and then even some longer duration, 100 hour, I think in this case it says over 100 but in some of the cases up to 500 hours of storage for this carbon free scenario. And then if you go to the 100% renewable scenario you again see an increasing mix of the different technologies that are needed and selected for this scenario. So again we have an answer for a very specific question but this question, there's a lot of assumptions that went into this and so when you change those assumptions you get different mixes. So it helps to inform us because I think that's what we need. We need information to inform how we make policy, how we deploy storage, how we look at technology and develop technology and the different attributes they need. So building on some of the results from that study how does that inform performance and cost requirements? So highlighting a few of the key findings from that. The first one, so cost assumptions can lead to different technology choices. So we had several different technologies in there with estimated costs. We had a base scenario, an optimistic, a pessimistic because we're really not certain how the industry is going to evolve and what the cost might look like. So we wanted to run many different scenarios to understand this. We also came up with this what we call hypothetical battery because we don't know what, we can't pin it down to a particular technology at this point. So if there is a flow battery or some other battery out there that is lower cost on the energy portion what does that look like? It's much lower cost than say lithium ion and lower power cost than we might see with some of the alternative maybe thermal or a pump storage systems. So we had this hypothetical battery and when you include that in the mix so you can see the different scenarios of no hypothetical and then including it in the scenario how that changes the mix of which technologies may get selected in that scenario. The second finding was that round-trip efficiency of the technology also matters. So that hypothetical battery what is the efficiency of it and how does that change how it gets selected? So two scenarios run is if it's 75% round-trip efficiency or if it's 50% round-trip efficiency and so you see again on the bars there's a low efficiency and a high efficiency and again there's a change in how the technologies are selected and obviously it gets selected more in a higher efficiency scenario. Next finding, very low energy capacity costs. We still need very long duration storage with low energy costs to handle the curtailment and the or to mitigate the curtailment again because we don't want if we install all these renewables if we're curtailing them we're just gonna have to significantly overbuild to account if we can't shift it we're gonna have to significantly overbuild. So it's finding that balance of storing the energy and shifting it versus overbuild and so we will need very low cost energy. So that's where we see hydrogen coming into play and then technology availability has system-wide implications. So if you remove a technology from the scenario how does that impact it again if we have this hypothetical battery if we don't have it again it can change the mix and so I think the overall takeaway from this study is that it's just too early to pick winners we're gonna need a variety of technologies that can help and the more technologies we have and with different technology attributes it will enable that optimized system so we can select the technologies that meet the needs and as the needs evolve over time so this is a particular point in time in looking at this reference scenario if we look out even further the needs will change again so I think the study can helps us to inform some of these decisions that we're making and how we approach long duration storage but it's not the end answer it's a continuous analysis that we will refine over time and so we're looking at this study was from 2021 we're redoing it again this year looking at regional aspects so how does that this was a US wide model how did particular regions if you look at those regions how do those compare based on their policy and generation next in their load requirements and then also looking at different weather years and so the assumptions on weather and how weather might evolve we have an effort at every called climate ready that's trying to understand how we can account and prepare for the different climate scenarios going forward so looking at other weather scenarios will also change the results so again the start of a conversation and I think it helps to lead into some of the discussion later today on regional focuses and technology we need to look at it all in order to be ready so I will pass it on and take questions later thanks all right thank you very much Aaron and then last but not least let me invite Adrian to come to the podium and he will share his insights on technology learning curves great thank you great well good morning everyone my name is Adrian Yao and I'm an industry return to graduate researcher but I like Will's term returning a little bit better for the past nine years I was the co-founder and CTO of a lithium-ion battery company called N Power and we actually recently just acquired a Gigafactory in Indianapolis Indiana but I decided to step out from my own company last year so that I can return to academia and pivot my focus towards the problem of long duration energy storage and really take advantage of the risk taking potential and tolerance only afforded by an institution like Stanford before going back out to industry again so very happy to be with all of you today today I'm not gonna be talking about new ways of building batteries in light of this theme today I'm gonna be talking instead about new ways of thinking about what to build and really thinking with the end in mind and this is kind of following on Aaron's point it's too early to kind of choose winners but how do we make sure that what we work on has the best chance of winning and given the short time frames we have to act we really can't afford to wait too long and not think holistically from the very outset so I wanna propose a systems level techno economic framework for building for guiding what it is we should be building now I know that sounds like just a long string of really big words that could literally mean anything so let me start off with an example so that we can all build an intuitive feel for what am I mean by this now since this is a long duration energy storage workshop and Dr. Steven Chu earlier brought up the flow battery let's use this as an example now for those of you that aren't familiar with what this is this is basically a battery cell where you take the energy components out of the cell and basically treat the cell more as a reactor so you can think of this as a chemical plant that can reversibly charge and discharge chemical fuels stored externally and the concept here is that we can begin to separate the storage component from the reaction component and thereby decouple the energy and power components and since these tanks can now be independently scaled we can make them as big as we want if we want longer duration storage we can make the tanks bigger then if we look at the cost curve then whereas conventional batteries like lithium ion batteries have a fixed cost with respect to duration flow batteries can potentially and theoretically approach this electrolyte cost floor that's really asymptotic and dictated by the fundamental materials cost of the electrolyte right and so therefore as material scientists and chemical engineers we should therefore focus on developing low-cost electrolytes and we should be all on our way to solving some of these problems right so just a quick show of hands how many people are familiar with this kind of concept great well this is wrong and reality is actually much more complex and if we look at our cartoon here where we went wrong is actually up here and this is part of the problem with simplified diagrams because the realities of connecting a battery cell to the grid requires a technology stack that looks like this and we need to be able to bring the voltages of DC side up to several hundred to several thousand volts before we can inject it into an inverter and then convert to AC now you might think well if we just take our flow battery with you know maybe a single cell voltage of one or two volts depending on chemistry why can't we just stack it in electrically in series and get our operating current and be well on our way well the answer is you can but the problem is that while this is electrically in series it's also hydraulically in parallel which means that you're going to get the shunt current loss that eats away at your efficiency and the more you stack cells in series the more loss you build up so if you actually want to connect this kind of technology to the grid these you start to see that the system topologies get really complex oftentimes acquiring more than just one pair of tanks and so if you look at any commercial demonstration of redox flow batteries you always see many more than just one pair of tanks right if it were true then we should ideally have just two big tanks and an arbitrary number of reactors but that's not what we see because they actually do need to be electrically isolated now don't just take my word for it look at every single commercial demonstration of redox flow batteries and you always see way more than just two tanks so did we really decouple energy and power at the systems level or did we just increase the size of our unit cell which is now a 40 foot container and that's not necessarily saying it's wrong but we also have to ask the question how quickly can cost fall due to learning rates and here's a general observation from our group that says that typically the bigger things are the slower you learn now this is again just an example my point here isn't to say we should stop working on redox flow batteries that's not my message here it really is that if this is new to you this illustrates the dangers of not doing systems analysis from the very outset and we can't afford to run into these roadblocks when we're four, six, 10 years in right so I hope that illustrates what I mean by this long string of big words and we can dive in so when we talk about long duration energy storage like Aaron and Clyde talked about it off the questions often revolve around the amounts, the duration and the cost now as we begin to build a sense of what these requirements should be and like Aaron said we're not gonna answer those questions today because it really depends on what we're looking at it's important as engineers to not dive and jump straight into our favorite chemistries or whatever we think feeling wise is gonna be lowest cost instead we really have to take a step back and ask the question what system architectures make the most sense and I'd like to offer two components to help answer this question and that is a holistic technical scope which is an example we just kinda walk through but also what I call a learning techno economic framework so let me dive deeper a holistic technical scope just means a multi-level systems architecture analysis where we start to ask questions starting from the macroscopic site level such as when does energy density actually matter and then zooming into the container level and asking questions like how many cells per container maximizes learning rates while also minimizing costs and then zooming in again to the cell level and asking questions like are drop-in technologies or drop-in replacements to existing technologies most cost competitive so this really represents technical breadth this is number one number two is the learning techno economic framework and this is really understanding how technologies actually scale so every technology has what we call a technology curve and you can plot basically the price or the cost of the technology as a function of scale such as cumulative gigawatt hours installed and also with respect to time and it's in my opinion that it's up to institutions like Stanford to instantiate new curves put them on the map and then hand them off to industry to traverse the scaling portion which is arguably the much harder part but when we at Stanford are thinking about instantiating new curves we have to keep four key things in mind because those are the things that we can more or less control the first is the minerals cost floor and that's dictated by the elemental the mineral and the material composition of not just a cell but the entire technology stack the second is the manufacturing complexity or system complexity because that dictates your initial costs the third is a learning rate this is not something so much you can control you can't say I want to learn at 24% per year, year on year it's more so that decisions in our design process can affect some of these things and lastly we have to recognize that the way to pull us down the curve with speed is dictated by the market growth rate and maybe having the ability to service multiple markets can help accelerate the adoption and cost reductions of certain technologies now the most important part though is that we need to recognize that there number one is an incumbent curve and where that incumbent curve sits and given the circumstances of climate change and the short time frames we have to act we also have a very short period to make sure that our new technology curves have the opportunity to intersect and overtake the incumbent curve and if this condition is not satisfied based on our initial analysis we really have to stop and think whether we should continue working on what we're doing now this all sounds well and good but let's actually put this to practice and start off with an example let's start small small in terms of the technical scope and that this is just a cell level comparison but big in terms of the implications of this kind of question now this is a short duration storage kind of example I know we're talking about long duration here but this is a really good example of how to implement this kind of framework so basically we're trying to ask this question are we looking at the condition on the left or a condition on the right, right? now I think I don't need to belabor the sodium ion opportunity I think many of you here are familiar with this it really boils down to the materials availability and therefore potential for lower cost of sodium and also the drop in compatibility a drop in compatibility of sodium ion into existing gigafactories for lithium ion and also the ability to potentially benefit from what is an already learned process right and so if you look at a sodium ion battery it looks and feels just like a lithium ion battery and given the roller coaster ride we just went through with lithium carbonate in the last two years we're starting to be inundated with a lot of buzz and a lot of hype about whether sodium ions batteries will disrupt and conquer and we're also starting to see some very aggressive cost targets so let's actually put this to the test with the techno economic model I just mentioned and we can start to pull in together these four pieces of information now to do this properly and to make sure that what we're trying to do is industrially relevant we managed to convince the industry leading consultancies to hand over literally hundreds of thousands of dollars worth of proprietary data to us so that we can run this analysis and really collaborate in answering this very important question so on the cost four side we have commodity price forecast out to 2040 and 2050 by the likes of benchmark minerals and wood McKenzie to establish the and predict the initial cost of batteries we built our own manufacturing cost model derived in large part from my personal experience building and running a gigafactory but also with a panel of industry consultants and also getting quotes directly from material suppliers to understand learning rates we can run regressions on data from Avison and Bloomberg BNEF and we have a wealth of market growth data so we come back to our two curves that we now know and love just a little bit more scientifically we can import the data on prices on demand and have our model ingest all of this data and we can see that it spits out a curve establishing the baseline for lithium ion specifically LFP lithium ion with pretty remarkable fit and you can see that it's even able to capture the first time lithium ion cell prices increased last year because of the underlying lithium carbonate volatility right so now that we've established a good baseline we can then begin to lay on what we think sodium ion will be now in the interest of time I'm not going to go through all the assumptions but really the point here is that we're taking a very generous assumption base case for sodium ion while also being really realistic with the inputs so we're getting quotes from directly from the current manufacturing base which is largely based in China with realistic quotes on what things would cost today if we were to make this at scale and so naturally with any new technology the curve the initial starting cost is going to be higher but the key question is then how quickly will costs fall right and this is where we have the first word of warning if we just willy nilly throw on some conventional learning rate like we often do and expect us to learn the same rate as lithium ion we're going to paint this overly optimistic picture that says that we're going to hit cost parity within one year of debut and that's kind of a little bit too rosy and that's because conventional learning rates will approach $0 per kilowatt hour which is obviously unphysical so we have to implement some kind of minerals cost floor constraint we also need to recognize that different parts of a battery will learn at different rates so how do we know what those should be well if we look at the bill of materials cost breakdown of the last 10 years of lithium ion we can actually see yes in fact various parts of the battery learn at very different rates and if we look at and for example manufacturing is much faster than for example the rate of learning in active material powders like cathode and anodes so if we look at where we think sodium ion is going to be in 2024 assuming that's the first year in which we start to manufacture this at tens of gigawatt hours we can see that the drop in compatibility allows us to benefit from a very low overhead in the beginning it also means that our learnings there are saturated and that really to approach the minerals cost floor of sodium ion which is lower as expected in LFP but not by much we really need to rely on the cost reductions coming from the materials production and as you can see it happens much slower so if we take an example scenario where sodium ion dominated all non-lithium stationary storage ESS so that lithium ion is going alongside sodium ion we can plot the demand curve in those purple dots on the bottom and you can see that still the stationary energy storage market as it is today is still much lower than the demand driven by electrification of vehicles for lithium ion and you'll start to see the importance of the market growth rate here we can see that and if we basically assume a range of learning rates based on our sensitivity analysis from the prior slides we can see that sodium ion struggles to keep up and in this scenario we're likely not going to have cost parity what if we take the even more generous assumption that sodium ion dominated all stationary storage period including the market share attributed to lithium ion so you can see the curve of the demand curve has increased and you can see that improves the scenario but still sodium ion here still struggles and if anything it might be slightly lower performance, slightly more expensive cousin to lithium ion within this period now this all sounds a little bit out of control but if we then ask the question what about the lithium minerals cost because that is definitely a concern that would be a good question to ask because based on our model we actually do see a significant supply shortfall for lithium carbonate and if we feed this back into our model we can see that the lithium ion price begins to stay high and comparing this back against our sodium ion baseline we start to see that the competitiveness of sodium ion does actually rely on the lithium minerals prices staying high which actually is a sentiment shared with us by the key sodium ion players in the industry that are part of our industry panel of consultants now this you might think this is out of control well there are a couple of things that we can do as both researchers and also policy makers so there are two components to dollars per kilowatt hour there is the dollars per kg portion which is the materials cost and so far we've only been talking about this portion and we can reduce that just by manufacturing scale and that's something called learning by doing but there's also this kg per kilowatt hour component as well which is the materials intensity and we can improve upon this by improving the storage density of materials and therefore require less materials and this is basically learning by researching so if we look at the historic trends of lithium ion we can see that materials have increased steadily over the past 30-ish years and if we assume not the same but more aggressive learning rates in terms of learning by researching for sodium ion we can see that we start to pull that curve in now one more thing that we might be able to do as policy makers is subsidize so if we take the existing assumptions of these materials cost and if you're familiar in this industry you know that these cost assumptions are very aggressive and very generous for sodium ion basically the difference between the red curve and the black curve represents until the point they intersect represents the amount a government could for example subsidize to ensure that the technology adoption can happen assuming that the performance obviously is on par so in this case 26 billion is not that much but something to keep in mind though is that given that the curve shapes look so similar this is highly sensitive to the dollar per kilogram assumption and so just a slight increase in dollar per kilogram could end up with a number way bigger than that now if we come back and revisit our original assumptions in the spirit of this holistic systems analysis we can again see that this is actually not entirely correct because in actuality it looks more like this and that's because of certain inherent material properties inside sodium ion that hurt their volumetric energy density now you may say volumetric energy density doesn't matter for grid scale storage maybe but it is true that to contain one kilowatt hour of energy you need one 290 amp hour ish prismatic cell for LFP you would need 2 for sodium ion and to have a 100 megawatt hour insulation for grid scale deployment you would need 50 of these 44 containers for LFP but you'd need 100 for sodium ion so what's important to recognize is that even a cell level cost parity condition is still not sufficient for sodium ion to take over now what does this mean for sodium ion and specifically short duration storage I think it's kind of clear that sodium ions will not be this ultra low cost savior to our lithium woes but potentially the cost of not doing sodium ion may be actually greater just because of the impending demand shortfall that we will have and what's interesting about this kind of analysis is that we can based on the sensitivity analysis recommend key research directions that maximize the bang for the buck and so I won't go through these in detail but some of these recommendations that we have are that the anode is something that can make or break the technology and that we really need more attention here the cathodes need to keep up at a given rate just to be competitive with the lithium ion and maybe the best way to maintain low cost for short duration storage is to increase lithium extraction but my point here today is really not to convince you whether you should invest in sodium ion or not and whether you should bet big on it but to really illustrate the power of this kind of techno-economic framework in this kind of systems level thinking and apply it to the longer duration storage technologies that we really want to be talking about today because this is really a good chemistry agnostic toolkit for evaluating those technologies so by combining this holistic technical scope with this learning techno-economic framework it helps us really better decide as engineers and scientists in this room what should we actually work on to improve our chances of success for the energy for the I guess the C-suite executives and policy makers in the room where should we invest our resources and for all of us where should we dedicate our time to make sure that what we're working on is actually has the maximum chance of high impact so with that I'd like to end here and take any questions in the panel and chat afterwards thank you Adrian thank you Clyde and Erin if I can ask you to join us please alright well thank you for the comprehensive presentations to start the day along with Steve and now we have about 35 minutes to talk about linking your three presentations together so let me get started here with just a couple of questions to do so and then we'll open up the questions from everyone here in the room so Erin I thought I might start with your talk so you discuss a number of metrics efficiency cost per kilowatt hour cost per kilowatt and there's been one metric that's on my mind which is energy throughput per year so for a particular technology particular installation how many kilowatt hour will the battery discharge per investment per site per year and specifically I'm really curious as to go to the very long duration storage if you were to consider how often that resource will be deployed to generate revenue for the owner what would that look like and how would it compare to the short duration storage yeah that's a great question I don't have the specific details on that I didn't look at that within the study results but what I can say is that I think for all the technologies is that most of them were not discharged at rated power for most of the time so a lot of them were running could had a duration we're operating a duration of two to four times what they're rated because they were outputting lower power limit or they were just used less part of it is you need that resource adequacy so you have assets that need to be there to ensure the grid is reliable but they're not dispatched all the time so I think as you get into those higher renewable penetrations you'll see more of that but when you have that flexibility in sort of reference case like gas and other assets that can provide that flexibility then you storage is the way it's selected is probably more optimally used and maybe just build on a little bit more and also inviting Clyde and Adrian to chime in as well is there a scenario where because of the tail of the distribution of storage duration say for example the 100 plus hour you were mentioning is there a scenario in which they will not be running all the time and therefore leading to a lower return on investment and if so how does one construct the market the policy and the technology so then the return on investment is better over time and I think it goes back to how we how climate change and it evolves over time especially in those high renewable penetration or like 100% renewable it really is going to look at like how what do those future weather years look like and that's going to I think play into the answer quite a bit and so there's a lot of efforts in gathering data to inform those weather models as well to inform going forward Clyde what is the how I sell perspective on this that's very difficult so I agree with Aaron if you look at my my plots it's back in 2019 we saw a week with very little wind and soil right how often is that going to happen is a big question but I think you know like today how we operate the nuclear plants and the other renewables like geothermal and biomass that could be a place where we can operate long term or long duration storage kind of pretty close to baseloaded I do not want to say baseloaded because what the system needs today is flexibility because on one side you have variable supply and on the other side is unpredictable demand so the system is varying so whatever it is we decide to do we got to hinge back on the operational needs so a lot of times it may not be the most cost effective but reliability is going to trump whatever it is that we develop so in other words kind of battery provide the three things we talked about frequency control, voltage control and can it move, can it provide flexibility there's one last point in California today in 10 minutes with the amount of renewables and we didn't even hit on the rooftop PV it impacts the system frequency just as great the system is changing plus or minus, not every hour but in some hours plus or minus a thousand to plus or minus 2,000 megawatts in 10 minutes that's hard to predict it's hard to control can storage fill that gap so it's not just storage it's how do we predict that variability and it's a natural variability so I tell folks you can forecast on an average let's say next 10 minutes or next hour but I get benchmark as to how well I control every 4 seconds and I would have to report to note every minute did I do a good job, did I do a bad job right so whatever it is we design in terms of storage it has to be able to give the system operator the flexibility to operate the grid yeah and add to that is the specifically for the storage technologies that have longer duration that might not be cycled as much I think there are two components that we really need to think about from the economic side of things how are they going to pay back their return investment and that might be dictated by the market and it really depends on where you are to dictate what that cost will be I'm not going to pretend like I can answer that question but something that is really interesting from this kind of analysis that I presented earlier is that the rate of market growth will significantly affect what that cost fall would be and if we are only implementing and building and commissioning these plants out of a sake of more so insurance as opposed to daily cycling needs then there is a consideration there will the technology cost fall it might need to start off with something that is very already inherently low that doesn't rely on rapid learning rates to we can't bank on learning rates in that case so to add to that like you said if they are only there for assurance and you have the opportunity to have other natural gas resources is it going to get selected there is not as much demand so again it really goes back to do you have other alternatives instead of that seasonal or weekly storage this is a great opportunity for Stanford since the new door school of sustainability will have a significant focus on predicting climate change so stay tuned maybe this is a good segue to the technology aspect so Adrian you talked quite a bit about the learning rate and you hinted at also some performance tradeoff as well similar Erin you also talked about the competing requirement of round trip efficiency and cost and these are complex tradeoffs so Adrian I'm wondering if you can comment a little bit on whether there are technologies on the horizon that there is an interesting tradeoff between performance and learning rates so can you give up some performance but have dramatically accelerated learning rates as a way to get things out to the market out of the grid sooner I think some of the challenges with specifically long duration energy storage and energy prices in general is that it's a commoditized area and you're starting with new technology which may not hit all performance metrics or maybe outperforming existing technologies but cost more and whereas lithium ion could enter with electric vehicle space with luxury EVs like Tesla's or really starting from camcorders that have the tolerance to have this cost performance tradeoff here you're really trying to sell something that potentially is more expensive and less performing at the outset but there may be other markets that and this may be require the support of governments or other policy makers is to subsidize the implementation of these technologies in ulterior markets that can affect the market growth rate so for sodium ion for example one example would be like two wheelers or three wheelers in a place like Outer Rick Shaw's in India areas that can really allow sodium ion to at least get a foothold, get a bite and begin to accelerate the learning rate in deployment but the challenge is really this you're operating at the opposite end of the spectrum where you're trying to inject new technology that might perform worse and might cost more than what really needs to be a commodity Aaron how about your analysis I know you created these hypothetical batteries you looked at round trip efficiency and cost did you look at all four combinations of them yes yeah we looked at different scenarios there's other performance factors that go into their self discharge when you look at really long long duration storage self discharge becomes a factor in how it gets used and the ability flexibility is a big thing like can you ramp quickly and just so yes we did look at some unfortunately I can't share I don't have off the top of my head like how all of those intersected but it is something that has been are there strong sensitivity to some of these tradeoffs you know it's interesting some move things more than others and you know we did model a regular flow battery as well sort of at its current state and where we think it might project and you know just didn't get selected we tried a bunch of different scenarios and it just didn't get selected and so we really need to a part of it was probably may have been our assumptions on what is really the optimistic scenario of that and it takes into account not just the technology but as Adrienne said like what's the total installed cost what's the total operating cost and there's you know tradeoffs in you know lithium ion and the degradation that's another factor so you know with lithium ion how we did over build an account for augmentation in that cost so and other technologies not so a lot of assumptions and so I think that's another technology factor that that is plays into it is what is the operating cost as well there are many unknowns and many big unknowns as well that's definitely the central challenge here so maybe just one final question before we open up to the floor for discussion so Clyde in thinking about policy and market drivers for technology demonstration and deployments at just last year we had a great panel discussion here I can't remember I think you might have also been on that panel as well with representatives from PG&E and we were talking about how power purchase agreement for energy storage can be a really big driver I think PG&E went as far as saying well you know give me something reasonable we will have the PPA in place to get the technology deployed how is that looking at how powerful is that as a knob and a driver how more can we do that so a lot of folks look at economics right unfortunately I look at reliability so you can design any market and you can put any kind of objective functions to minimize costs and most markets in the U.S. they operate on a 5 minute basis so if you can forecast what that demand is in 5 minutes you can always find the cheapest energy to serve that when it comes to passing that commitment to the system operator he is faced with things like controllability frequency control, regulation contingency reserve the ability to ramp and the ability to report back to the guys in the east that I do a good job controlling the grid it's not just bouncing it's but bouncing the U.S. has the strictest performance standard in the whole world we got frequency limits that we got to operate to and if you do not have the flexibility you exceed one of those limits either high or low for more than 30 minutes that's just one standard you get hit with a million dollars in fine we have another one where if you lose an element and I cannot bring my system back in 15 minutes that's another million bucks in fine we got another one that this is spread out across a year if something happens like let's say the west from Colorado all the way out to California if we lose anybody loses something I got 52 seconds to help 53 seconds it's too late so if I do a good job in 52 seconds at the end of the year let's say we look at 20 events the median is what I get benchmark on that again comes with fines when you regulate the grid high frequency, high ace you know in other parts of the world if you have high ace or high frequency, that's fine they allow the frequency to deviate you cannot do that in the U.S in the U.S. if you have problems you got to contain that within your system and if I do not have the flexibility to maintain that it's very very difficult to manage a thousand or two thousand megawatts variability in 10 minutes it's not easy we need to change markets one way I need to have something that would tell or automatically determine what's that frequency change because when the way market works today is you make a decision probably 7.5 minutes before you're binding 5 minutes and you instruct units to move for that anticipated load well let's say wind kicks up a thousand megawatts and solar kicks up another 500 megawatts there's no mechanism today to tell those units hey I do not want you to go let's say from 100 to 150 I need you to stay where you are or decrease production these are the kind of input we need to feed back to the market designers to guys like you if you know the challenges we face then you could design something that could help me control that right so most of my talks are going to be reliability what it is I need to comply with hopefully you could design when you design you got to fit in that model not just costs and not just return efficiency but can I vary my production you know as I was saying electric vehicles we're going to have so many in California why can we just use some of those to control frequency and control that in a way where it's transparent to an electric vehicle owner so it's going to be something like you buy an electric vehicle and you would provide frequency control whenever that's plugged in and that's totally transparent to you how nice would it be at the end of the month you get a check from PG&E or your electric provider for 100 bucks or 200 bucks say I didn't see frequency control and you didn't even do anything so things like that we got to stop thinking electric vehicles I think eventually they're going to come up with batteries that could do almost everything but you got to know what it is we need so you could design it you know to help I can see the title of a publication book that says long duration energy storage colon but hopefully through the efforts of folks in this room we will make it less complicated and hopefully converging on a number of key solutions so we have some time to address questions from the audience it's really a privilege to have this distinguished panel here so please so my question on the technical economic analysis have you factored in what a carbon price does to those various models and what carbon prices would really drive more progress yeah that's a great question no the answer is no but it certainly would be a next step but coming back to the policy side of things that would be in the same vein and flavor is kind of like the subsidy analysis understanding what government intervention or what kind of subsidies would help with adoption and for example if you compare the numbers that we I showed earlier with what the IRA has had for cell manufacturing for example the $35 per kilowatt hour credit those do fall within the range of what could be done and the gap that we are seeing so the answer is no not yet on carbon pricing but certainly something we should look into and maybe you can comment from that at the preside yeah apologies I'm not the modeler I'm providing the storage input so I can't speak directly to how that was modeled but if you want we can follow up offline I can connect you to see how we take that in and how it's accounted for in the different policy scenarios Clad I think it's fantastic the way you framed it you know reliability versus economics and so one of the things I've always wondered is what is the value of that reliability is there a price point I mean if you are buying car insurance right if there's a miscalculation you're going to get an adjustment in your premium so how should and this is open to the panel how should we be thinking about the value of that reliability and how would we structure a market to enable that that's a very tough question right I think when it comes down to reliability well let me step back when I was a young kid I grew up in Caribbean back then you know those big TVs we had a 23 inch TV which was low it was 19 inches and we were happy because we had a black and white TV today when it comes to reliability you know young kids you know just when my kid was she was 6 and we had lost electricity I was working for PG&E anytime and she called me up and she goes down I want to see Rugrats can you turn the electricity back on right so the kids today they want electricity almost all the time right and it's how do we maintain that right you know we have like a one day in 10 10 years where it's acceptable from a planning perspective but when it comes down to real time it's very very difficult you know and I look at the system you know as I said every minute if let's say we shorten generation on the rest of the interconnection but for how long do you want to lean when I say lean if I do not have enough generation I can lean on somebody else right but you cannot do that constantly right for the same period every day you know so we started looking at this with a lot of solar when solar dropped off in the evenings we saw a deficiency on the system right I went back to NERC and I said I have a problem out west because when the solar drops off it's very very difficult to make up that energy and back then you know we were forecasting 12,000 megawatts to make up on evenings today it's 20,000 but we figured out how to solve that right the the the challenge we see now is like during sunrise but to go back to your original question I think it depends and if like for me I want to see I want electricity 24, 7, 365 right I know we're gonna have hiccups right but true natural disasters that's something we can we can live with but on a normal on a normal steady state operating condition with everything in service that's my biggest challenge today right and when I say biggest challenge we have we have standards so if you lose an element you have other standards that kick in if you lose two elements we have other standards that kick in and you know exactly what to do but on a normal steady state condition that variability we can't predict that and you cannot control it that's what we need help so I tell folks under N minus zero on a normal steady state condition is the predicting that variability controlling to that variability is really really difficult I want to see as close to 100% reliability as we could provide when I was Secretary of Energy I thought if we would relax the frequency and voltage controls we would have a more reliable system it would be a lot cheaper your standards are very different the people who really care about tight control have their own systems if you're a chip manufacturer is there any possibility that we can relax these controls as you go into the future we will actually need them to relax even more but can we do this so Professor Chu you hit the nail right on his head I think we have too strict of control standards right so the one I talked about where we need or we get benchmark every minute so how well we support the interconnection frequency I've been asking the guys can we relax that to 5 minutes or even 10 minutes can we allow frequency to move a little higher than today we have 36 mHz you got to operate within that well the system is going to collapse if I go to 37 or even go below that so these are the kind of stuff that we need to get academia we need to get to understand but I've been saying this is the exact thing up to last week I was in a meeting in Tucson and I was complaining about why do I need to control every minute when we have variability on the system I cannot predict the output today we got 12,000 mW of rooftop PV I have no visibility no controllability it impacts the system frequency so I agree with you 110% we need to think we need to relax in some of these strict control performance standards we have within the US problem it was that I couldn't figure out where the points of resistance were actually coming from because until you know who is pushing back you can't really change it do you have any clue I mean everything is all about money in the end and so you have to follow the money so who is going to lose money if you relax the frequency and voltage control well I don't think anybody is going to lose money if we relax the voltage and frequency control I think the standards needs to change remember we operated with standards that were developed for controllable supply and predictable demand we operated with these same control performance standards today when you have variable supply and unpredictable demand so we need to change that and the people that can change that is the North American Electric Relability Council we can just say what it is we need even within my company when I said hey we need to relax some of these standards I get resistance why do we want to relax it and again my response is if the system is not going to collapse why do I need to control that it comes back to electricity it travels at a speed of light we control it every four seconds and I think we do a pretty good job control that every four seconds but four seconds is pretty slow and I like to go back and hit on something I think I brought this up the last time when you explain that to a high school student controlling the grid every four seconds they go what's your problem well my response is you try driving down the freeway 60 miles an hour with your eyes closed and every four seconds you open your eyes and see where you're going well you might be able to do that but today with the variable supply that road is not straight anymore it's very winding so trying to control every four seconds now it's more and more difficult right and again one take away and you can quote me is the system is not going to collapse under n minus zero everything under normal steady state operating condition it ain't going to collapse and again why do we need these strict standards it's actually a really interesting point if we did actually reduce those constraints you could promote more adoption of CNI type batteries for the chip fabs that actually do need that frequency control and then they would promote that market growth rate well you know even within the US they allow the frequency to go down to 59.3 before they start dropping a lot in the west 59.5 we start automatically tripping a lot so if it can work then why can it work in the rest of the country so we've been modeling for years showing that we're going to need a lot of long-duration storage and yet almost none has been deployed and because it's not been deployed we're not getting any sort of learning curves and it appears that the reason it's not getting deployed is because none of the actors in the market see any value in deploying it so someone alluded to the fact that recently a bunch of CCAs went out for a long-duration procurement but what they meant by that was 8 hours not multi-day and so I guess the question is what has to change about the way we evaluate RA or compensate people for doing this that would actually cause the utilities and the other people are actually going to be deploying this stuff to start doing so that's a tough one and please don't not quote me the way things work is you have to have a blackout for folks to realize you have a problem so today the way things work the way the operators control the grid nobody sees a problem I remember going into my office one weekend and I told my boss we had a lot of problems over the weekend and he asked me two things did we drop load and I said did we go into a stage emergency and I said no and he said what was your problem so the general public once they have electricity they think everything is fine but we couldn't control the grid for 11 hours and that was a challenge for the operators but we didn't go into a blackout but these are the kind of things I think the general public needs to understand that and I guess most of you understand like today we have to control the grid every four seconds we got a report to NUC every minute so how well can we control but a lot of you didn't realize that that's something you have to do then there's another thing that we do if we operate at high frequency you speed time up and vice versa at low frequency you slow time down we operate to an automatic clock in Colorado and as soon as you hit five seconds either too much or too less you got to correct that so there are a lot of hidden things that system operator has to do in the background this is why flexibility is really really important so whatever it is has to take in mind whatever it is I come up with cannot help the system operator control the grid and again Professor Chu if you could help in relaxing some of these standards I've been trying I think that will go a long way so you just need to know who to call I would add we looked at how energy storage is being modeled in integrated resource planning if you look at different utilities and how they're modeling it some technologies are being energy storage technologies are being included but not the wide landscape of technologies that are under development and there's challenges with confidence levels and the cost and the technology so whatever we can do to keep advancing and demonstrating and really understanding the energy storage costs and performance characteristics in resource planning models because right now you're seeing lithium ion, maybe flow, pumped hydro but you're not seeing the wide variety necessarily in many of the resource plans that the market is that are being brought to the market I think it's a question for Glyder a very simple question as an operator do you treat this as a larger scale energy storage system as an engine market as an answer to service market components it's both because you need energy capacity used to be a problem when I say capacity the industry used to plan to meet that peak let's say for the day or peak for the month what we're finding out now is you could meet that peak but you may not be able to meet the energy needs get into that peak some of them might be 10 in the morning 11 where you have a hard time balancing supply and demand but I could meet that peak so what NERC is doing right now they're coming up or revising two standards I happen to be on both we're looking at energy needs as opposed to just capacity needs right do I have the flexibility in the fleet to meet that variability so that comes with both energy and then ancillary services right so we have a lot of batteries today they move every 4 seconds and they help which is great but you also need the energy component you mentioned just a few you mentioned 5 few times 4 7 is that a scale of thrush no that's not a scale that's a requirement so in North America there's really every 6 seconds we do it every 4 seconds we have other entities I think OCOT does it every 2 seconds you got to balance your supply and demand one of the things we're thinking of doing and you'd see this pretty soon we want to implement frequency control on inverter based resources a lot quicker today because of the variability we're seeing if you see we're going to do this towards the end of the year today we do not do anything within let's say 26 millihertz it pushes the cut down in half and you start helping control the grid from roughly 17 millihertz you're going to start doing something we already started to get pushback to implement this because in the old days you tried to do it conventional units you're going to have to move a lot more to do it today you have inverter based resources everything is software driven why can't they do it but the pushback is why should I do it but they didn't do it so it's a lot of back and forth but it's something I think we need to do because as I said plus or minus 1,000 to 2,000 megawatts in 10 minutes if we can cut that down 50-60% by doing something or inverter based resources helping that would go a long way no that's a new standard that's called frequency response that the federal government impose there's a reason for that in other parts of the world they do not allow a unit to operate a PMAC so if you build a 100 megawatt unit you got to operate like I know in Europe it's 97.5% you got to reserve head room to automatically provide that frequency response in Latin America they reserve 8% in the US reserve 0% I brought that up that what they were doing in order to get that frequency response in you know 2 seconds why don't we allow or impose on units they need to maintain head room I almost got beat for saying that because if I build a 100 megawatt unit why do you want me to hold back 5% or 3% or whatever they build that unit to maximize you know economics to maximize profit so a lot of things has to change like today when we curtail wind and solar they need to automatically provide frequency control, frequency response if something goes bad they need to provide that on service what happens is if you have let's say an inverter base like a wind farm and you got 10 inverter banks and we curtail you what they do is they curtail they take the inverters offline so if it's a 100 megawatt plant let's say you have 10 inverters 10 megawatts a piece and let's say we curtail you 50% they take off 5 inverters so when you think you have 50 megawatts a head room you may have zero so we got to get some of these rules change when we say we curtail you thou must provide these kind of controls so we're working on things like that it's a requirement from fuck that says you need to have the capability to provide frequency control they did not say you need you must provide it so having the capability and maintaining that head room is two different things so it's a good start by having the capability but we need some more brute force implemented in that that says you need to be able to provide that because that's the first slide that defends any system house and the Europeans they were smart they said we're not going to allow you to operate a PMAX provide that across the board for free same thing in Latin America why can't we do it on that note Injury Clyde and Erin thank you very much for sharing your insights I want to thank you Clyde in particular now I think I have my next bumper sticker ready so Clyde thank you very much