 afternoon everybody we're going to get started here. There's a polling question up if you haven't noticed on the screen I'm just asking what your affiliation is so you feel free to answer that and then we're going to get the QR code up so you can scan that so you can ask questions during the session. There we go. Okay well welcome to the NRC's RIC session on risk-informed decision-making. Risk-informed decision-making will be key to navigating the nuclear future I believe and especially in the context of the significant changes that were discussed this morning during the plenary session so I think it fits very nicely. I'm Andrea Cook I'm the Deputy Office Director for Engineering in the Office of Nuclear Reactor Regulation and I will be your session chair. Okay a little bit about the agenda we have an hour and a half session and we're going to take about 50 minutes for the speakers to give their presentations and then we'll move on to a 30-minute Q&A session. A couple logistics the Wi-Fi code is RIC 2023 if you need that and please do silence your cell phones. If you're a virtual participant you can submit questions on the platform under the Q&A tab using the QR code that was just shown and if you're in person you can scan that QR code and use your mobile device to ask questions. This is one of the most widely attended sessions in terms of the registration number so we're going to do our best to get to all of the to answer all of your questions but our panelists have offered to step outside in the hallway after the session in case you have a question that you really wanted to ask and didn't get a chance to they'll stay for just a few minutes and if we have any unanswered questions in the queue at the end of the session those will go to our session coordinators and while we may not get to answer every single one of those we will take those in a consideration for future engagements on on the topic of risk. We'll have a total of five polling questions for this session. For those of you in the room you saw the instructions on the first slide text the code that's associated with your answer to 22333 and if you're on the platform you can go to the live polling question tab and go to the polling section to answer the question. Lastly I just want to put a plug in to provide your feedback at the end of the RIC on this session and others. We do read those carefully in fact your input from last year helped to shape this this discussion today so thank you very much for that. As far as what we're looking to for you to take away I think one of the key things that we'll discover here is that there has been significant progress on risk informed decision-making in the last few years and our panelists are going to give some really good examples of that but to steal from the chairs quote this morning I don't think any of us are looking at it with rose-colored glasses. There are still challenges to be addressed and so what we'd really like to get from this session is how we can build on the successes we've had and address the challenges that we have to move forward and get to where we want to go. Just a few acknowledgments before you start first of all thank you to all the panelists for your preparations and presentations today. I want to thank our session coordinators that's Andrew Mihalek, Alyssa Newshausen, Stacy Rosenberg and Jeff Wood of the NRC staff for coordinating the session and thank you very much to our AV staff who are very busy putting on the first hybrid RIC. I just wanted to point to two digital exhibits on risk informed decision-making that are here at the RIC. The first one is on the impacts of risk informed licensing actions at nuclear power plants and I've actually seen this one and it has some really powerful data on the impacts of risk informed decision-making so I really encourage you to check that out. Jay Robinson is a sponsor of that of the NRC staff and the second one is integrated risk informed decision-making for licensing reviews which goes through a relatively new process we have at the NRC for using an integrated team to look at licensing reviews and help us focus on the most significant aspects. Ian Singh of the NRC staff is the point of contact of that digital exhibit so please check those out. Okay let me kick off the sessions and introduce our panelists. Our first panelist is going to be Mike Franovich. He's the director of the division of risk assessment and the office of nuclear reactor regulation here at the NRC. Secondly we're going to hear from Victoria Anderson. She's a technical advisor for risk and engineering at the Nuclear Energy Institute. Thirdly we're going to hear from Jeff Wood. He's a senior reliability and risk analyst in our office of nuclear reactor research and our fourth presenter will be Mark Wishhart who is a senior technical leader in Epri's risk and safety management team. He's taking the place of Ashley Lindemann who unfortunately could not be here today. Our last presenter will be Amir Afzali. He's a CEO of a newly formed consulting company and he has over 40 years of experience in the area of risk. Okay without further ado let's go on to our first three polling questions. The first one as I mentioned was displayed as you walked in and just gathered a little bit of information on who you all are. So if we can go to the results of the first polling question okay or you can text in your answer and then we'll get the results. Can we get the results? Okay mostly NRC staff and US industry although other is kind of hedging up there. So good to know who's in the audience. We do have a second and third polling question and they're both related to what your views are on where we are collectively on the risk informed decision making journey. So if we could bring those up our hope is to use these questions to help facilitate the discussion during the session today. So the second polling question. So the second polling question is with regards to implementing rhythm industry has needs to do more or has done too much. Okay it looks like we're kind of settling on needs to do more has not done enough with a close second of has performed just the right amount although there's a big difference in percentages there. Okay so that's a good good fodder for our discussion later and then the third polling question. Is with regard to the Nuclear Regulatory Commission and other regulatory agencies same question have we done enough or not. Okay the results there look pretty similar to the question that we have for the industry with most people saying we need to do more. So hopefully we'll get some good suggestions on where those areas are that we can do more and what the challenges are that we can address moving into the future. So thanks for that we're looking forward to a very interactive session and without further delay I'm going to pass the baton off to Mike Franovich for our first presentation. Thank you Andrea on this podium you need to practice risk management because it is a little wobbly up here and those lights aren't exactly that friendly to the speaker but we'll manage through this. So let me go to the next slide. So thank you Andrea I appreciate the introduction and it's good to have the polling results for a little context for the discussion we're going to have today. I'll cover five topics being the first panelists. I'll be briefly cover rhythm history and over a two minute drill of history. It won't do a justice but I'll give it my best shot and cover some of the key policies and actions that shape our work. Rhythm guidance for enhancing greater approaches international cooperation and close with two recent examples of rhythm and operating in new reactors. To paraphrase Winston Churchill the farther back you look in history the greater you are likely to see the future. A search for the true representation of risk has been a quest since the inception of the Atomic Energy Act. Wash 1400 reactor safety study used PRA technology and so the seeds are risk-informed decision making. Its aim then just as it is today with risk assessment is to enable better decisions about nuclear safety. An early example of rhythm was consideration of the wash 1400 accident release sequences as part of the planning basis for emergency planning established in 1978. The accident at TMI accelerated the demand for rhythm. In the 1980s two key policies served as regulatory stabilizers. The 1985 sphere action policy declared that existing plants did not need further regulatory actions unless new significant information emerged. The policy called for a systematic approach to identify vulnerabilities in both operating plants and future designs. The 86 safety goal policy answered the question of how safe is safe enough and further the call for an identify severe accident of vulnerabilities. PRA offered that gateway. In turn licensees conducted individual plant examinations in today's parlance this major stress test actually led to over 500 improvements in the plants. At the same time designers of new reactors were required to develop and use design PRAs and expectations established in the original part 52. Moving forward into 1995 the PRA policy statement proffered that PRA technology should be increased in all regulatory matters in a manner that complements the deterministic approach and supports traditional defense and debt. Another objective is to reduce unnecessary conservatism. In 1999 the commission issued an SRM on a Secchi white paper Secchi 98-144 on risk-informed performance-based regulations. It set the expectations of the staff on how we should regulate. The paper established what a risk-informed approach should look like. This approach applied actually widely to reactors fuel facilities and the materials programs. At the same time the agency issued guidance using five key principles of risk defense and depth safety margins performance monitoring and compliance with regulations. In 2019 the commission's SRM on a new scale block valve kindly reminded the staff that the SRM from 1998 white paper and the risk principles still applies to NRC activities. It's enduring. In the last 20 years there have been so many initiatives and programs implemented it actually looked like an active seismograph if we were to portray it on a slide. That said one of those activities has been 5069 risk-informed SSE categorization rule. The rule allows for a much more graded approach to treatment of SSEs in the four categories. The SSEs in risk three been provides relief from special treatment requirements and allows for alternate treatments. As of 2023 the NRC has approved more than 50 percent of the plans for use of this program offering greater flexibilities while maintaining safety. For new reactors we see movement in that direction as well as new scale has developed a similar approach and lastly but not least build off the 5069 framework the industry led licensing modernization project actually refine the approach tailored to advanced reactors. Turning to the topic of international cooperation the NRC strategic leadership actually an international activity states verbatim advanced risk management concepts is through an international standards and guidance for graded approaches and licensing and oversight of operating new reactors. Why this strategic objective? Because PRA provides a technology neutral means to achieve a graded approach along with risk management. This morning IAEA Director General Grossi spoke of harmonization efforts again PRA's with the use of rhythm provides a means to that end. I like to highlight the recently issued tech doc from IAEA and graded approaches. It provides some excellent examples of member state practices including risk informed programs. It's an excellent reference and should help many member states especially countries embarking on nuclear energy. Now I'll turn to some more recent examples of where we applied rhythm. Some of you may be familiar with a process we call integrated decision making for emergent issues commonly referred to as LIC 504. Actually this process was created as a lesson learned from the Davis-Bessie reactor vessel head degradation event from nearly 20 years ago provides a structured process to document decisions for issues that may warrant prompt regulatory actions. It also provides guidance to apply integrated decision making and has been used frequently for a range of topics both plant specific and for generic issues. We're actually now working on revision six of this guidance. The goal is actually to use best estimate methods and realistic analysis and because this is guidance on decision making we've actually included a section on decision making biases such as confirmation bias or my side bias. Portrait on this slide is actually some examples of recommendations that could come out of such an evaluation from the LIC 504 process. We strive for an optimum NRC response and that's what we're trying to illustrate here. An objective is not to underreact to something that could be significant nor overreact to something that doesn't have importance to it. There's actually a range of potential outcomes from this process. You'll see on the right hand side of this chart examples clearly if we're in a range of where something is truly very significant we may require immediate regulatory actions but going down the scale we could involve formal back fit analysis if the issue is of substantial increase to safety and that's been defined by the commission. Could involve information notices or we can use smart inspection samples in the baseline inspection program or no regulatory action may be an outcome. Again we're using rhythm we're not relying on numbers alone so we are applying the five key principles for risk informed decision making. An example where we've done that work recently is in the treatment of high energy arcing faults. First I must say U.S. plants are protected for heaf events. This electrical hazard is not unique to nuclear however its proximity to certain equipment and nuclear plant poses certain risks. Since the early 2000s we have accrued significant operating experience through the U.S. and international work. Several years ago regulators recognized that the use of aluminum in electrical systems may pose additional risk compared to copper based systems but our state of knowledge was also changing from the operating experience. This illustrates that risk may not be static and vigilant use of operating experience is appropriate. In the U.S. this hastened what became great work done by EPRI and the NRC together through an existing MOU to improve the methods and assess the operating experience. Reflecting the wide electrical configurations in plants and important concepts such as fault clearing times and the variations in breaker coordination studies. In 2022 we did complete our LIC 504 evaluation and we used two reference sites and we appreciate the assistance of the plants that supported us in that effort to apply new fire analysis techniques and risk methods. We concluded that no new requirements were warranted however we did apply the be risk smart framework which you heard of from the chairman this morning in determining that we actually should emphasize the T part of be risk smart which is teaching. That led to an increased outreach to stakeholders and a webinar to share the qualitative and quantitative insights importance of prevention mitigation event and event management. And at this week as of today actually we released an information notice to convey lessons learned from the LIC 504 work. And lastly I'll turn to an example for new reactors. One of the challenging issues for SMRs has been establishing an appropriately sized emergency planning zone or EPZ. Proportional with the risks of the SMRs the size of an EPZ impacts many aspects of EP coordination and in terms of determining what kind of emergency response capabilities should be in place. As an independent layer of defense in depth EP is about dose savings and dose management during a very unlikely but significant radiation release type of event. Examination of scenarios from likelihood timing distance and dose and consequences may require pre-planned prompt protective actions or the EPZ in short. If you participated in the earlier session on EP earlier this afternoon you've heard how the NRC uses our state of the art reactor consequence analysis tools to risk inform our EP regulations and guidance and how modern PRA can provide insights on the spectrum of accidents used to inform EP and appropriately sized EPZs. The challenge was to address emergency situations that could capture a reasonable portion of the residual risk of SMRs. It's a different proposition than just looking at the safety case of the design. In 2022 we successfully resolved intricate EPZ related issues for rare earthquakes other hazards and treatment of uncertainties for multi-module facility. More specifically the NRC approved a novel approach and methodology proposed by Newscale. We believe that this work has promised for other innovative designs as well. And lastly I'll close with we are advancing RITM through our regulatory programs and activities to focus resources on what is most safety significant. We use risk assessment in a manner that complements defense and depth, safety margins, engineering judgment and enterprise risk as well. We continue to develop innovative approaches. We include leveraging international cooperation to increase the use of RITM and more graded approaches to safety. And in closing I'll say when done right RITM clarifies, cuts through and captures the essence and spirit of making great decisions. With that that concludes my presentation and I'll turn it over to Victoria. All right thank you Mike. I'm happy to be able to work with this panel today to deliver some key insights about how risk-informed decision-making benefits the entire nuclear sector. My presentation will focus on the benefits that the operating nuclear power reactors have seen from implementation of risk-informed programs including some unexpected benefits that we didn't originally anticipate when we developed these. As an overview this slide shows some of the safety and operational improvements we've seen as sites implement these programs and throughout the presentation I'll get into some more specifics about how we've seen these improvements. So some of these improvements we anticipated when we developed these programs. So those include the reduced outage duration, averted shutdowns, and reduced personnel dose. There are others that have really emerged as sites have had runtime with these programs and have experienced both the day to day and long term benefits of the use of risk information in making operating decisions. For example we've seen licensees able to make their outages less complex which in turn greatly reduces the risk of their outages. We've seen plants able to leverage these risk-informed programs to complete deferred maintenance and to work with more flexibility so that their maintenance can be more thorough without the pressure of unnecessarily short allowed outage times. These benefits truly result in a safer nuclear industry and reflect the advantages of implementation of risk-informed decision-making. So given the substantial benefits touted on the last side you're likely curious as to what specific applications we're talking about. I'm going to walk through three of our major risk-informed licensing applications and while there are certainly others these three are what we sort of think of as our marquee risk-informed licensing applications for the operating fleet. So first up is TISTIF 425 the risk-informed surveillance frequency control program which at this time has been adopted by every single operating reactor in the fleet and it is the first risk-informed licensing application program that has been adopted fleet-wide. This program allows licensees to move their technical specification surveillance frequencies to a licensee controlled program and use risk information to extend these. Some of the major benefits we see after implementation of this program include streamlining of testing programs shortening of outages with less frequent testing and reduction of frequency for tests that are known to stress plant systems. So those theoretical positives are helpful to think about but what really illustrates the success of these risk-informed programs are the specific operational and safety improvements realized at the plants as a result. We have many of these for risk-informed tech specs surveillance frequencies and there are a couple that I'd like to highlight today. So first at one site using this program to extend the station blackout test allowed for performance of the test on just one division instead of two so it greatly reduced the complexity of the outage it reduced dose because personnel weren't going in there as much and it really lessened the potential for test cause errors. Second at another site they were able to extend their emergency diesel generator operability tests and that supported a train specific outage meaning that only one emergency core cooling train is worked on for a given outage and that also greatly reduced outage complexity and risk. The next application I'm going to discuss is 5069 which Mike touched on earlier. This is risk-informed special treatments. It allows licensees to recategorize some of their safety related equipment with low risk significance to streamline various programs such as maintenance and procurement. So some of the major benefits include that we see a substantial reduction in the quality assurance scope which makes it much simpler and faster to procure equipment that might be needed for maintenance and repairs. It also improves program focus on the most risk significant components. So we've also seen a large number of benefits realized from this program. We have one plant that reported that after several years of runtime with 5069 they saw a permanent one person REM per year reduction and saw that they were able to reallocate over 200 maintenance engineering hours per year when they changed their maintenance rule scope. They also saw that safety related parts procurement lead time was cut in half so work is able to be performed much more expeditiously. At other plants we've seen the staff at the plant use this program creatively to improve plant operations and safety by completing maintenance and testing as scheduled or ahead of schedule. One plant was able to use this program to downgrade a valve so that it didn't have to be part of an IST test and that ultimately enabled the test to be performed. We also saw a plant that was able to streamline piping replacement and conduct it earlier and this was beneficial to safety as there were some known issues with that piping. Finally I'm going to discuss one of one more risk informed licensing application which has been adopted by nearly half the operating fleet at this time. This is TISTI 505 or risk informed technical specification completion times. This allows licensees to extend equipment allowed out each time up to 30 days based on plant configuration specific risk assessment. So there are major benefits of this. One of the ones that's touted most often is the avoidance of unplanned shutdowns which has both economic and safety benefits. This has helped support more deliberate maintenance practices and it substantially reduces the need for NOEDs or notices of enforcement discretion which require some substantial emergent resources from both licensees and NRC. With fewer NOEDs both the NRC and licensees have more staff time to focus on more important issues and ultimately improve operations throughout the fleet. We do of course have some plant specific examples for this application as well. One of these is related to an anticipated benefit ofverted shutdowns that actually revealed an additional safety benefit. In this case a plant was able to extend a completion time that enabled them to complete a repair and keep the plant online. But what was notable about this is that this took place during a time of extremely high regional power demand. This meant that the plant didn't have to go offline and stress the grid. So this ultimately helped enhance nuclear safety throughout the entire region. We have also found that the program does a lot to reduce outage risk because moving work online can really help reduce the length and complexity of outages. So plants have been able to do things like move piping replacement online, move cable work online, and reduce outage length up to a day or two. So while the benefits discussed today are substantial and numerous I really do believe that we're going to see even more of this as we continue implementing these programs and we'll see more refocusing of both utility and NRC resources. I do want to take a moment to acknowledge that we do still have remaining challenges ahead in implementation of these programs and getting full benefit throughout the fleet. Some of these challenges include adoption of these programs throughout the entire fleet because we have not yet achieved that for all these programs and continued education on the nature of risk-informed programs both for NRC and the industry. As we implement more and more of these far-reaching risk-informed programs it becomes more and more important that everybody on the staff at the plant at the utility and at the NRC fully understands what these programs are about and how they work. I'll now turn it over to Jeff Wood of the NRC who will discuss the NRC's efforts on infrastructure supporting risk-informed decision-making. Thank you, Victoria. So I'll be discussing some of the work NRC is doing to advance the infrastructure for using PRA in decision-making. So our previous speakers have given us some great summaries of some of the history of risk-informed decision-making as well as some specific examples of risk-informed applications I'm going to talk about the infrastructure that we built which gives us the basis and gives us some confidence in using risk in our decisions. So I'll review NRC's framework for using PRA and risk-in decision-making. I'm going to talk about some of the work we're doing to maintain and enhance the infrastructure and the guidance for using PRA and also discuss some of the risk tools that we're developing to support risk insights in our world. So the infrastructure that I'm talking about are the activities that NRC, the industry and other stakeholders are doing that build the foundation for risk-informed decision-making. These things give us confidence in the PRA and the results. So these activities include use of the PRA standards, the PRA peer review process, the many NRC regulatory guides that support risk-informed applications developing PRA models and related tools, development of new PRA methods and NRC's approach to PRA acceptability. And together these things build the foundation that we can make our decisions on. So I want to review NRC's framework for achieving PRA acceptability. This is discussed in NRC regulatory guide 1.200. So there are three aspects to the framework the use of PRA standards, the use of the PRA peer review process which provides an independent check of the PRA and also the NRC staff positions which are documented in NRC red guides. And these work together to give us an acceptable process for developing PRAs and that reduces the need for an in-depth review of the PRA for each application. So I'll review what we have planned for coming revisions to regulatory guide 1.200. In the coming years we plan to review for endorsement the scope of PRA standards that are going to be needed for new and advanced light water reactor risk-informed applications. So this includes the ongoing work the staff is doing to review the 2022 revision of the level 1 LERF PRA standard. We also are anticipating review of the level 2 PRA standard which is anticipated to be published this year. And we're also planning to review the advanced LWR PRA standard and the low power and shutdown PRA standard for endorsement and red guide 1.200. And having this scope of PRA standards along with the NRC endorsement will give us the guidance that we need for new risk-informed applications. So we're also continuing to develop guidance for non-LWR PRAs and risk-informed decision-making activities for non-LWR applications. Regulatory guide 1.247 has been issued for trial use and this provides NRC's endorsement of the non-LWR PRA standard. And we're continuing to evaluate other gaps in our guidance related to non-LWR PRA. So for example, we're looking at guidance on the treatment of uncertainties and addressing some areas specific to non-LWR PRA. So we're planning now to make sure we have the guidance for some of the new areas for non-LWR PRA so we can make the timely decisions for those future applications. So in addition to the guidance that we're developing NRC also has some tools that are being developed to help support our use of PRA and working with the PRA standards. One of the things I want to mention is the development of a PRA standards database. So as new versions of standards are issued, revisions and updates are issued along with revisions to the corresponding red guides. We recognize we need a tool to help us work a little smarter with these documents so we can track what has changed and we can work more efficiently and also maintain consistency of our positions on the PRA standard requirements. So Sandia National Laboratories is supporting us in developing this database and we think this is going to be a much more efficient way to work with the PRA standards. So another PRA tool I want to mention is the set of NRC's SPAR models. So the SPAR models are NRC's risk assessment models. They're used in several regulatory programs and they're a resource for the staff to incorporate risk insights into their work. And we have a lot of support from the Idaho National Lab to maintain and update these models. And one of the key inputs to that work to keep these models up to date is benchmarking against information provided by licensees from their PRA models. And we want to continue to emphasize this moving forward because it's important for us to make sure we're maintaining up to date risk insights for our work. And in addition to the SPAR models I'll mention we have a PRA software the Sapphire software also developed by the Idaho National Laboratory and as they're continuing to advance that software we're moving towards more cloud-based tools to improve our computing capability but also to enhance the way that we can collaborate and interact. So one other tool I want to mention is called SPAR-DASH. So SPAR-DASH is our risk assessment dashboard which summarizes the results from the SPAR models. So with this project the SPAR-DASH project we're aiming to make risk information more accessible to the NRC staff including some of the staff who are not necessarily familiar with PRA or not PRA experts. We're aiming to present this in an easy to use in an interactive format. This is another tool that also supports our BRISC framework to incorporate risk informed thinking into our decision making and it's a communication tool to help us communicate risk insights amongst our peers and to decision makers. So this tool summarizes a number of different PRA importance measures and some other PRA results and it can be useful in assisting us with prioritizing and also ranking different contributors to risk that can inform our work in a number of different ways. So this is an example where we're leveraging some modern data analytical tools to improve the way that we're accessing and communicating risk insights. Just to summarize some of the key messages from the presentation so NRC has developed a framework for establishing the acceptability of PRAs for risk informed decision making. NRC also continues to develop some PRA tools to incorporate risk insights into our work and important input into that is benchmarking against the licensee PRAs to make sure we're maintaining up-to-date risk insights. And we're also continuing to develop guidance and tools to support the use of PRA and risk informed decision making for new and advanced non-LWR and LWR applications. So that concludes my presentation and now I'll turn it over to Mark Wishart from EPR. All right. Thank you Jeff and thank you to the NRC for allowing me to participate in today's conversation. I like the title of this it's a voyage and it certainly has been a voyage and the voyage is not without its challenges. We've seen challenges along the way and we will continue to see challenges but I think it's fair to say that the benefits and the value of risk informed decision making is real. I think Victoria did a nice job at illustrating what some of those benefits are and where we see the benefits moving forward. Over the past several decades we've really had to do a lot of work to get us to where we are today and thanks Mike for your timeline it really helps to visualize just how much work has gone into putting us in the position that we are today. We've had to develop the foundation we've had to build upon that foundation we've had to adapt to change and challenges in our industry but through that all we've been able to develop we've been able to expand the risk informed applications and our PRA capabilities. We're finding the right mix between traditional engineering approaches and risk informed approaches as we go and each situation may need a different balance but it's important to note that risk informed approaches complement traditional engineering they are not a replacement. Also risk informed approaches are consistent with NRC principles of defense in depth safety margin managing and monitoring risk. Those still remain very important to this process. As we move forward and look forward we continue to focus on having a structured and transparent risk informed regulatory framework. Jeff, thank you for providing some of the elements the building blocks if you will of what that framework looks like and what we need to see moving forward. We also recognize that consistency and predictability benefit both the regulator and the industry. So continuing to have a consistent and predictable process is also very important. To that end in 2019 EPRI published a technical document a framework for using risk insights. This document addresses the gaps in the overall risk informed decision making process. It looks at both the inputs and the outputs to the process. It discusses the role of PRA in that process and it also aims to support an understanding of how probabilistic and deterministic inputs can be better integrated. If you've not taken a look at this report I invite you to check it out on our website. It's publicly available for download. If we look at the use of risk information and we look at it and the application of seismic hazard analysis specifically in the area of what do we do when hazards change? Our understanding of external hazards is evolving and it will continue to evolve. So when that happens how do we reevaluate these hazards? Well one option is to use risk information via a seismic margin analysis or a full seismic PRA depending on what you may have available to you and use those inputs into the decision making process. This allows us to focus on the most significant hazards and magnitudes and their impacts rather than only looking at the largest hazard and magnitude. Through use of risk information we can identify potential plant modifications that yield the greatest benefit and by that we mean the reduction in risk not simply focusing on deterministic compliance. Also looking at successes in fire PRA risk has helped move fire protection programs beyond deterministic approaches. The industry started with full room burnouts where we basically say the room is lost. We looked at what the damage vector was and then we made exemptions from deterministic requirements. We then got PRA. Early fire PRAs were conservative and that's understood they were a first pass. Several improvements though we were able to refine we were able to update our PRAs and now we have models that better reflect operating experience. But that's not to say that this hasn't come with a great deal of work of effort. It's been an iterative and incremental process and frankly it takes a lot of time. These advancements have been mostly in part to the great collaboration between EPRI the industry and the NRC. Like talked about one of those examples of great collaboration which is the evaluation of high energy arcing faults. Early testing suggested that the hazard zone of influence its impact on other components or equipment within the room was larger than we initially thought. But after a multi-year research effort taking a deep look into the hazard we were able to provide additional guidance for how we might model this risk moving forward. The treatment of uncertainty is also important when we talk about risk-informed decision making. Understanding and appreciating uncertainty and the potential impacts it has on our decisions is something that we need to both account for and address. And I think this is an interesting way to transition into the idea that risk-informed decision making is not only about the numbers CDF and LERF but it's about understanding the why. Why have we arrived at these numbers? Typically refer to that as the insights of the PRA. Why did the number increase or decrease? What were the changes in component importance? Accident sequence importance? Human failure events really what we're saying is we need to look behind the numbers in PRA parlance we talk about the cut sets. Cut sets are great for PRA practitioners sometimes but they're not great in general but still we need to look behind the numbers we need to get away from only focusing on CDF and LERF. Uncertainty is another piece of that insight that we get from risk-informed decision making. I want to quickly go through this slide I think I've talked about the EPRA report previously but again I refer us to the EPRA technical document it's publicly available we discuss defense in depth we discuss safety margin we discuss the use of risk in the decision making process and how all of these elements interact and connect together. I also recognize that there's a lot of information on this slide I think I've touched on many parts of this but I want to point out the second report there the EPRA report in the bullet this is a report on multi-unit risk in this report we talk about multi-unit risk where I should say we talk about risk-informed decision making in the context of multi-unit risk multi-unit risk is an area that we see as being current research and continued research looking forward again this report is publicly available I invite everybody to download it on our website leveraging risk insights for aging management this is actually a very interesting application of risk information aging management presents an opportunity to utilize risk information to support the current fleet as we move towards extended operations we're moving beyond 40 we're moving beyond 60 engineering analysis can help us identify the aging mechanism likelihood how likely is that to occur PRA and risk analysis can help us identify the consequence of those failures risk-informed decision making allows us to bring those two pieces together EPRI has research in this area we have a report on leveraging risk insights for aging management I believe we published that in 2022 and this is an area an application of risk where we continue to work with the industry and the NRC I guess to close I do want to point out that you know what we've presented here today is just the tip of the iceberg there's so much more that we could talk about in the risk-informed decision-making space I think it's also important to reiterate that while the risk numbers are what we see above the surface the insights are everything below the surface and as with an iceberg there can be so many more insights than what is perceived from just the surface I think in closing risk-informed decision making takes time it's an iterative and incremental process but it does add value EPRI is looking for opportunities to apply risk insights where it adds value for the current fleet and the next generation we also recognize the need to help manage the process and look for areas of potential improvements we strive to help make the current and future risk-informed applications both more effective and efficient but also beneficial to the industry with that I will hand it over to Mir Good afternoon it is my pleasure to be here under the spotlight it's interesting to me spending almost 40 years in this field how far we have come and what we have achieved and I was the main focus of this presentation is where we go from here and how do we do it this time around from the beginning the right way but listen to the my panelist here my fellow panelist here I think the case was made so bear with me as I go through this I'm repeating some of the benefits you have heard already just as a heads up as we go through this there are a bunch of slides you have a lot of information on them that's deliberate to document the thought process but for the sake of time I'm not going to cover every slide in details so I believe innovation everybody that I talked to and generally people who are involved in the industry believe innovation is critical for nuclear to be a solution for future energy generation so I'm going to submit to you that let's take that as a face value and go with it that's a given in this conversation so the next question is that so how do we make it happen so this talk is about risking for performance-based approaches using a rationalist approach to regulation is the best way to allowing that innovation to occur but before talking about the premise and the assertion that I provided here I would like to provide some context of the rest of the conversation the first item is what we call by regulatory framework regulatory framework is beyond the initial licensing so this is a picture of regulatory framework that you can find on the NRC website so it's got five components in component one NRC provides their expectations like part 50, part 52, part 53 etc that provides some guidance of how to meet that expectation in component two the applicants licensee provide justification how they meet those expectations in component three NRC provides oversight that you are meeting those expectation in component four the operating experience are evaluated to see the original licensee basis that makes sense and component five you do all the research to support all the other components so the next item is in terms of context is what do we mean by different construct of regulation so the primary the three different constructs one is the what I call unstructured construct is ad hoc construct you just bring your case and you try to make your case and try to prove to the NRC that you meet their expectations and you don't know exactly the NRC doesn't exactly tell you what those expectations are either in a structured approach are two options one of them is called structuralist approach some people call it deterministic approach that basic what it says you don't need to know exactly how I meet it as long as you comply with the structure that I'm providing for defense in-depths in the rationalist approach you provide a rational basis for meeting a defense in-depths criteria and margin so it's the aggregation of all the features that you have as opposed to step-by-step step-by-step process of the constructionist approach so as I'm talking about the rest of this conversation I'm basically not talking about that hoc approach historically that has been proven not to be something that neither the NRC or the regulated industry is interested in so I'm going to concentrate on the structuralist and rationalist approach I'm not the point that I'd like to make that we are talking about currently we're talking about large light-water reactors of only two different type of design PWRs and BWRs but in the future we are going to be talking about a number of different designs, technologies, etc so just this is something I got from ANS which tries to divide the technologies we are talking about in terms of the coolant now within each one of them there are different technologies and within each one of those technologies there are different designs so you can see the number of designs and application that we are going to be facing as we go forward so with this context in mind I'd like to talk about first how do we go about making this happen the first thing is what is the objective of innovation the innovation is trying to make sure that your product is commercially viable because you have to satisfy your stakeholders which are great pairs and potentially some shareholders the product is affordable reliable and resilient to your target is your customers and is safe and environmentally responsible and the target is the public so if you do not if you're innovation that doesn't allow you to get to to achieve all those three I'm submitting to you that you are not going to be able to deploy your reactors so how do we go about doing so a regulatory framework which allows that type of innovation that occur given the number of designs we're going to be looking at we have to have a regulatory framework which is cohesive from beginning to the end everything is related to each other it has to be coherent that means the different designs are evaluated in a different in a same with the same standard and it has to substantially benefit from owner controls program again I'm submitting to you that you have to have all those three goals to be able to have a commercially viable and socially acceptable design so what are the attributes of such a framework such a framework this has to be agile it has to manage different design it has to be predictable so allows people to invest it has to be resilient because we know it's not that we think we know things are going to happen so it has to be resilient to deal with the future potential future events so again I'm submitting to you that all three have to be achieved and I'm submitting to you that the way to do it is through irrationally sub-processor regulation which uses risk-informed performance-based techniques to establish a safety case so what is the evidence for my claim I think my my friend I provided in the slides a bunch of different risk-informed applications under each one of those components of the regulatory framework that we discussed my colleagues on the panel have already covered a bunch of bunch of these and provided the benefits of them the most important takeaway from this slide is that there is a proven it's not the gas it's the proven evidence that they improve safety risk-informed performance-based approaches improve safety they reduce unnecessary burden and they optimize use of staff so that's where we're transitioning from a compliance mindset in a structuralist mindset as long as you are comply you're okay that is not adequate you're going to want to move to the to the excellence mindset where compliance is a minimum objective and that you can do that through a rationalist approach we use as risk-informed performance-based regulation again this is not something that is just a my opinion is I believe that opinion is based on a significant evidence provided by my my colleagues earlier on so where do we go from here given this is the case where do we where do we go from here I provided a number of suggestions on this slide I'm not going to go through every one of them actually I'm almost going to go through none of them except number five why am I going after number five because that's the component five that's where the research enables something to happen so what one of the fundamental things we we talk about advanced reactors is operating experience we have a lack of operating experience we're going to have challenges in user risk-informed application if we combine the PRA using physics of failure to establish the failure probabilities we we will have a better understanding when we talk about risk-informing aging management that basically that's what we're doing in isolation but that's basically what we're doing we are using physics of failure and the consequence so if we embed that it's part of our research program and have it done we can we can achieve that the second component is we know testing extensive and expensive testing to develop your basis for your licensing is not going to allow us to to achieve timely deployment of these these reactors so we are DOE spending billions of dollars millions of dollars at least in modeling and simulation having this research done to make sure those modeling and simulation tools can be effectively deployed in a risk-informed rationalist approach which uses risk-informed performance-based techniques is is going to enable us to transition from current test-driven approach to a current test-driven and empirical approach to a more science-based approach so what are the lessons learned based based on our 30 years of nuclear power operation lessons one is that that a structural approach is likely to to result in unnecessary burden so using risk-informed you are going to reduce unnecessary burden you're going to get rid of conservative siloed conservatism and identify the right risk contributors and you are you are going to prevent having these additive requirements as you go along so that's the lessons that this is a lesson is facts all those things that I mentioned have happened lessons two is that the irrationally approach based on what we have seen in the risk-informed area rationalist approach which uses owners control program and those owners control program are endorsed by the NRC and they are monitored by the NRC to make sure this they effectively are effectively executed provide the best option for moving forward and lessons three is that use of risk-inside in combination with the prescriptive rules because risk-inside is not complete on their own they will result in conservative decision-making number of times I hear this conversation about deterministic is conservative I think is solid conservatism the real conservative decision is made when you have realistic analysis and then robust risk-informed decision-making will result in the right classification as we go forward as demonstrated by watch 1400 so I would like to conclude my presentation by repeating what I said earlier on that the rationalist approach user risk-informed performance based method and programs is the most likely construct to allow time to deploy when the advanced reactors is promised in reactors and I because it provides the agility it provides the predictability and it provides the resilience and I think that all that would directly related to the nurses mission and finally I would like to conclude that part 53 framework A we have worked on it for a long long time provides a starting point for that transition into a rationalist approach it should be in place as soon as possible I know it's not perfect but we should not allow perfection to be enemy of good with that I thank you for giving me this opportunity thanks Amir I think we're going to move on to our last two polling questions and they have to do with we're having a risk-informed workshop this fall about the September timeframe and we wanted some feedback on what topics you might be interested in incorporating in those workshops so we had polling question for please okay looks like some of you already started answering this as the topics are listed there and it looks like there's not one clear winner but the progress on risk-informed licensing applications including lessons learned seems to be of interest part two is the same question I think we just had logistical issues trying to fit into one question so if we can go to part two there's a lot of interest okay looks like a risk-informing advanced reactors is another area of interest although I saw EP and security pop up there too so thank you for that well we're definitely going to take note of that as we put together the agenda for the risk workshops so with that we're going to move on to Q's and A's our favorite part of the panel session so you guys can scan that code if you're participating virtually you can go to the Q&A tab on the platform to ask questions so we had a few pop up already so we'll get started our first question is for Mike I think it's pretty straightforward what is the level of public involvement engagement in the LIC 504 process right thank you for the question the LIC 504 process I should have clarified in my presentation is an internally focused process internal decisions within the agency and how we manage the enterprise risk from those emergent issues again it was a lesson learned from Davis Bessie we had no process to actually document our decisions it was easy for auditors to come in and say what's your process the GAO audit as a lesson learned and we didn't have one so it was focused on internal decisions and be able to document them in a systematic way it does not substitute other processes that may have public engagement though depends on what the recommendations come out of such an evaluation those processes may have a public direct public engagement component others may not depends on the nature of the recommendations but 504 is an internally focused process okay thanks and the next question is for Jeff down toward the end of the table or I guess you're kind of in the middle sorry how will PRA tools and SPAR models impact advanced reactor rulemaking part 53 how will those tools be used for the role so currently our PRA tools our SPAR model are focused on the operating fleet so I think there's always a little bit of a balance in developing those models because we want to be efficient with our resources and not get too far ahead of where the designs actually are so we don't have any advanced design SPAR models right now at the moment but we're planning we're we're keeping stock of what designs are in the pipeline and planning for those for the future but they're primarily focused on operating reactors so our most current model is for the the Vogel 3 and 4 units and right now we don't have any plans to get kind of get in front of the advanced design process and develop those before we have a mature design and really have the basis for developing our own internal models okay great okay this next one looks like it's for Mike the new scale resolution of the seismic risk issue and EPZ size determination has been deemed proprietary if the methodology is extended to other applicants will the NRC allow it to be disclosed to the public a simple answer is that it's not a recall this is a commercially proprietary method by new scale I know there may be some strong interest in it at one point the methodology was offered as a potential used for non-light water reactors a new scale made a business decision to actually not go in that direction and make it tailored to new scale design alone this ultimately will be their decision I find it would be a great efficiency gain if that were to occur because what we find in dealing with and working with other applicants is there's a difficulty of folks transitioning from design engineering into the world of emergency preparedness and it takes a lot of time and effort to to essentially rewire folks mind about what the goal and objectives are emergency preparedness and I find that having to have that conversation on an individual applicant basis is very inefficient but ultimately it is a proprietary document and it's up to the company to make that decision okay there's one here it's for Victoria but I really feel like this is something that anybody on the panel could answer so maybe Victoria you could start and then the rest of you can jump in so where do you see an opportunity for future risk informed programs or other applications of risk information it's the what's next yeah I think there's a lot of efforts to discuss what's next and the industry is sort of discussing that internally and grappling with that there are some opportunities to look at the applications we already have and maybe expand their scope slightly and to look at synergies between the applications so that we could get more out of all of them and there are also discussions about things like risk-informing EP risk-informing security and what does that look like moving beyond what we're currently doing where we talk about using realism and security and EP not all of those will come to fruition but it's good to discuss them anybody else others I think I mentioned it during the presentation using risk-in-sites in the aging management programs we see that as an area that can potentially help as we age beyond you know 40 and 60 years so that's another area that we're performing research in right now anybody else want to jump in on that one before we move on maybe I shouldn't repeat everything I said during the the conversion there's a list of opportunities for risk-informing the regulation for advanced reactors that I've put on my slide one area where I think the immediate needs will get us the best benefit is risk-informing inspection during construction I understand that RC is working on that with two advanced most of the ARDP winners using the LMP approach for for developing the safety case I think we can easily use the risk-information to right-size our inspection efforts and make sure that we create some level of owners control over that particular aspect of the activity allowing a lot of decisions to be made by the owner operators given the margins they have to the risk for the public so that's where I I believe we're going to get the biggest bank for the park if we start now so there there are a lot of opportunities I think if you look sometimes we in the NRC have a unique vantage point that we see different sectors of the industry doing different things and we see overlap and maybe compatible approaches but not always consistent one area is the essentially trying to adopt maybe a more of an LMP-styled approach for operating reactors what would that look like what would it look like for for the non-advanced reactors the new the new light-water SMRs we see some commonalities there there's one industry group that has offered a standard we think that might be an opportunity there to see how risk could be folded into licensing basis event selection ultimately it comes down to what's the return on investment is there an appetite for this will we see applications but we're looking to maybe start that conversation here in this workshop that's coming up later this year the one that Andrew mentioned a lot of good ideas there thanks for that and thanks for that question so this next one is really interesting I don't know that there's one person on the panel who could answer it best let me just maybe start with Mark and then maybe others may have information to what extent does rhythm capture cyber security risk and has there been any advancements again I think it's the where have we made progress in this area and where is there more to be done on cyber security risk cyber security I know Epri has some research going on currently into instrumentation control risk digital risk I can't speak to cyber risk specifically so I'm not sure that I have that level of information but I know that digital I and C is certainly an area that we see risk information being part of the process as we move forward I'll just add to that since I happen to be sitting up here that we are doing a lot of work at the NRC in the area of digital I and C in terms of risk conforming our guidance during the plenary session this morning was mentioned we currently have a paper with the commission looking at further risk conforming in that area so certainly it's an area of great interest we are doing a lot of work in that area anybody else want to chime in on cyber security okay this next question is interesting it's for Victoria I think there may be other panelists who want to weigh in on this though so there are this given the stated benefits of using risk insights that we've all talked about this afternoon why are there still areas where we're pushing deterministic or where are those areas where deterministic approaches are more appropriate yeah I think I'm not sure I may not answer exactly the question that's being posed I'm not sure quite what they're getting at but deterministic and risk aren't risk informed aren't mutually exclusive you have to use both so I think you still have to use some deterministic approaches because there are many reasons that a PRA can't tell you everything there are things we can't model in PRA there's uncertainties associated with it so it gives us information it doesn't tell us everything I think if you look at that slide that Mark had with the iceberg with you know the focus on the numbers and then everything below it that really illustrates that you can't just make a decision based on a specific risk number you have to include many other things and all those programs that I talked about in my presentation have like deterministic backstops and deterministic considerations and things other than just looking at the risk numbers so I think you're not ever going to move away from the deterministic considerations it's always going to be part of it and it should be yeah I'll agree with though what Victoria said and you know we talked about risk informed decisions not risk based decisions right it's part of the calculus but it's not the only piece as you said there's a lot of additional insights that come from performing a deterministic analysis or a PRA or a risk analysis that needs to be part of the calculus before we make decisions I'm going to approach the questions slightly different in a different angle first of all I want to talk about deterministic we use this deterministic work where we really mean prescriptive not not deterministic in a PRA there's a lot more deterministic calculation than in a a structural approach typically you do a worst case in a PRA you have to do a spectrum of analysis so you do a lot of deterministic calculation so with that context you at this point in time because the technical adequacy of PRA is not sufficient to include or or because it's going to be too expensive to include every possible type of a hazard and do a adequate evaluation of it is beneficial to supplement it by some prescriptive work theoretically there is no barrier to fully transitioning to a fully PRA risking for bases well PRA based but there are so many barriers in terms of financial investment you have to make that I think that's not feasible in a immediate or even in a distance future so we are advocating for a risk informed approach to with acknowledgement that the state of knowledge is not complete to do a fully PRA based approach I'll add you can attempt in certain areas to try to risk inform but then you're forced into places where you may not have data or enough experience and you're forced into a situation of trying to use expert elicitation which there is an appropriate role for expert elicitation but with that comes more uncertainty so how you compensate for that is applying some engineering judgment and margins there are places I look at an accident tolerant fuel in particular there are areas you might be able to do work but at some point you have to use more of a without the actual experience you're going to go into the expert elicitation process okay now there's another really good question it's earmarked for mark but I think others would probably want to weigh in on this to in terms of how we use PRA in advanced reactor designs where we don't have any operating experience so the question is how do we treat uncertainty in PRA for new advanced reactor designs where we don't have that operating experience that you know I think the industry may want to talk about how how you handle that and maybe Mike wants to talk about how we look at that from a regulatory perspective but we can start with mark great question I know that within EPRI we've got a lot of research going on right now that is kind of bridging the space between you know where we kind of exist risk and safety management right and then advanced nuclear technology so we've got a colleague of mine looking at these types of questions what is risk for advanced reactors it may not be the figures of merit that we're currently used to core damage and largely release and then you know what do you do if you don't have data right and we know we talked about expert elicitation and some other methodologies that can be implemented but agree those come with a significant amount of uncertainty that needs to be accounted for so I guess I don't have a exact answer to the question but I agree these are areas that we need to start exploring especially as we're starting to use risk insights on areas where we don't necessarily have a large data set yet and it took us time to to get that data set no I had one of the I guess probably misconceptions is that there may not be operating experience for non-lightwater reactors actually there is operating experience for non-lightwater reactors some of it may be a little dated it may not have the amount of experience that you see in lightwater fleet and so but there is data there are data available and the different technologies now how it applies to a unique design and the actual runtime on a plant so it's kind of a different matter there is also another perspective and there's a smaller project that we're working on with research to see if we can cultivate extract insights operating experience insights out of the lightwater fleet that could be applicable to non-lightwater so when you look at some issues where the underlying causes are related to design engineering issues those are commonalities that you might be able to to glean those insights to apply them over non-lightwater work this next point I'm going to put in a plug for a session later on this week I think it's tomorrow on risk informing advanced reactors but we'll ask the question it's about part 53 and the question is for Amir so you talked about what that kind of ideal state would be for the regulatory infrastructure so the question is what kind of advanced reactors designs might meet the standards you discussed for viability within the part 53 framework if I may ask you to repeat the last part of the question I want to make sure that I'm answering the right question what kind of advanced reactor designs meet your standards for viability within the part 53 framework I mean he said another way the way I'm reading it I'm sorry if I'm I'm not converting this correctly but it's what would a what would your ideal framework look like for advanced reactors given what you presented so in a rational ideal approach is a rational approach as I mentioned what the rational approach is is is using an aggregate assessment of a safety case so part 53 a part 53 which is totally risky for performance base would indicate what are the performance objectives of the regulation and allows the developers to show how they meet those expectations through demonstrating that the aggregated safety case meets the in our six expectations so the the current even framework a or part 53 has some construction elements in it so I would basically get rid of all that I would go totally rationalist approach and it it I would give maximum flexibility to developers to show me how they meet the expectations and I give them a guidance of how you can go about demonstrating your expectation right now we have a at least LMP which provides some guidance not complete guidance some guidance in that effect so my ideal framework would be a totally rationalist approach where the safety case demonstration is up to the applicant and the process for developing that safety case is for some type of a guidance is provided for developing that safety case thank you and again the session on advanced reactors is tomorrow it's in this room at 130 so I'm going to skip a few here there's a question about challenges in applying RITM the question is about what are the challenges applying RITM at the NRC but I think this applies to any regulator and where where do we need to do the most work where are those challenges for regulators and the questions for Mike this may take a long time to answer actually I find I suspect the experiences in the NRC and industry aren't terribly different and I talk to my colleagues and industry about applying RITM in different areas and I think the challenge that we've had is taking a compartmentalized approach to doing our reviews the traditional engineering groups do their work the risk people do their work and several years ago we recognized that as a major issue that we really need to start projects from the get go with an integrated team where you're learning from each other and it's that communication that's been a challenge but I think we've got processes in place that have been used and are overcoming the challenge the other dimension of it is sustaining the efforts so by incorporating it into your procedures and the leadership backing it up and reinforcing those expectations helps sustain the type of culture that you're looking for here where you're having more of the conversation because you're learning from each other that's risk analysts or the engineering groups there's no one group that has a corner on the market of ideas or insights so that's the kind of things we're dealing with and I suspect for plants that have transitioned to some of the programs that Victoria mentioned that same kind of conversation happens within the utilities too so it's not dissimilar okay so we have five more minutes we'll take a few more questions I'll just stick with you Mike can you effectively implement a performance based decision without a PRA well you can there's actually a regulation called 5067 the alternate treatment of source term which is a performance based regulation that does not actually require a PRA it's actually a design to be more of a stress test on ESF features for the containment so yes it is a possible in theory the maintenance rule which is a fabulous rule of a performance based regulation does not require a PRA but in practice one is used it's difficult to actually inventory all the components under a PRA you'd have a gargantuan model so it's a kind of a blended approach but that's an early forerunner of a performance based regulation that does not require specifically a PRA and plants have tried to do it without a PRA but ultimately everyone has adopted PRA to help inform those programs okay let's do one more question this one is really interesting and it's for Amir so how should risk communication to the public change under a rationalist framework when that approach may be seen as being more subjective so I think the risk communication is a very important concept and from my personal experience particularly living in California working for the Abla Canyon when I went to the some of the public meeting that the NRC had use of numbers are counterproductive so the way the communication would take place in a rationalist approach in my opinion is that by articulating the margins that you have in a transparent way and those margins are do not necessarily have to be in terms of numbers it could be in terms of colors it could be in terms of some distance from a acceptable level of margins you can demonstrate the concepts and communicate them effectively when you have an underlying method to transparently produce those margins so first thing after the work with confidence in the public community that the staff is going to do their proper job to provide oversight and then you can demonstrate some of those margins in a transparent I'd using colors or etc one of the things I saw I think Jeff talked about is a dashboard dashboard is a way of communicating that that you can formalize and generalize and provide some test runs with the public and further demonstrate that that type of communications are effective I'd add to that that solution said is one of the elements I think that's under underutilized and not tied always very well is the role of performance monitoring and equipment which is help maintaining a picture about those margins where you see early warning of degradation and how to restore the margin in the facilities sometimes in these conversations we we don't talk about and overlay all the surveillance frequency programs surveillance that are done inspections that are done by licensees the conversation sometimes to eat treats that is a secondary matter it eventually comes into the conversation but I think those types of monitoring programs are really vital to show that the health of the plant will be maintained as well as its margins okay so it is right at five o'clock I want to thank you all for your engagement and the questions was really good discussion today so appreciate your participation and thanks again to all the panelists for your presentations and preparation we'll conclude the session now