 Greetings all and welcome to our rick session on the banishment of digital demons. I'm Paul Ribstock and I propose this session to address an important element in the assurance that digital technology employed in nuclear applications will indeed do what it's supposed to do. Digital systems are radically different from the legacy systems that they replace. Everybody knows that. They don't just do the same job differently. They have the ability to do radically different jobs and to allow for and benefit from interactions among tasks that are traditionally being considered separate. Hardware failures still exist, but they're no longer the major source of problems. Hazard analyses offer a way to assess the behavior of systems to uncover subtle behaviors and interactions that can be significant in digital systems but are less important or even impossible in legacy systems. Those are the demons, a wily bunch of imps meant on causing chaos wherever they can. Our panel of renowned experts will explore the capabilities and limitations of hazard analysis and the assessment of digital technology for use in nuclear applications. Now I'll get out of the way and let our experts dive in. Stephanie Coffin, our Deputy Director for the NRC Office of Nuclear Regulatory Research will take it from here. Stephanie? All right. Thank you, Paul. Can everybody do a sound check? Everybody hear me okay? All right. I had to laugh because we're a little late to the session because we were dealing with demons of our own on this side. So very, very apropos the name of the session, Defeating Digital Demons. So I want to first start off by acknowledging and thanking our panelists who are bringing a wealth of experience to this session and our moderator, Susheel Burla, will provide a brief bio for each of them as we enter into the moderated sessions. So you'll get to hear a little bit more about their backgrounds. Now we're taking kind of a atypical approach for the format of this session. And so can you go to the next slide? There you go. And one more. And so we're going to discuss this session through a series of three questions posed to our panelists and moderated by Susheel Burla. Dr. Burla is a senior level advisor here in the NRC in our Office of Research. So there's not formal presentations, but a panelist is welcome to use visuals to support his response. And so you might see some of that as part of the session. So I want to give you a little preview of the format we'll be using. And so can you click one more time? So what Susheel's going to do is present and explain question one and direct it to one of the panelists and then he's going to invite responses from all the other panelists. And then from after hearing from our panelists, we want to invite you the audience to participate. And so as you're listening to this question one in the panelists, you're welcome to submit your Q's and A's on the panel on the right of your screen. And then towards the end, then Susheel will select and sequence the questions that best fit the flow and the purpose of round one question one. Can you show that? And then following the audience questions, you can go to the next one. We want to have a polling question and the idea is just a yes-no question. It's related to round one and it's really to give feedback to the panelists on how well they're articulating their cases. And so then we'll move on to round two. You can click through that. And same sort of format. Susheel will pose a question, the panelists will address it, there'll be an opportunity for audience Q and A and then we'll have another poll and then we'll move into the final round, round three. Same format, question to the panelists, responses, engage the audience on some Q and A, and then a final poll. And then after that you can click one more time. We'll have a formal closure of the session. We'll be mindful of the time and pay attention and if the time permits, we might be able to do some additional questions at the end of the session. So I think with that, I am going to stop sharing and I'm going to turn it back over to Susheel. This session builds on the outcomes of a series of international workshops held in 2020 by researchers from Halden in Norway. That session was focused on understanding the state of the art in the safety assurance and digital safety systems. One of the limitations identified in those workshops was the ability to evaluate the hazard analysis of a digital safety system. This session drills down to understand those limitations through a sequence of three questions. This is a research oriented session. Mind you, there is nothing no direct connection with regulation, regulatory practices. We are here to learn about the state of knowledge. The questions are focused on a small subset of the real world problems to make the research manageable and to fit within the 90-minute time limit of this session. I've already lost 10 minutes out of that. The context is set via this use case. Digital upgrades of reactor protection systems are operating reactors. The reactor protection function is a Boolean function from the control rods when demanded. It's combinatorial logic, computationally much simpler than safety functions in the auto or aviation sectors. The loss of concern is the loss of the reactor protection function. Any condition that can lead to that loss is the hazard of concern. The causes of concern are systemic causes rather than a random hard way of failures such as, and I'll just go through this list quickly. In any combination of relevant hazard analysis methods is within the scope. Within these confines, take a quick look at this first question and revisit it later more slowly. The terms design diversity and hazard analysis have different meanings for different people. We are going to limit the excursion of these terms for the limited scope of this session. Design diversity for the purpose of this session is described through this example. Items A1 and A2 which may be two components or subsystems or systems are performing the same function and receiving the same inputs. Both A1 and A2 are performing correctly. Their output should be the same. A1 and A2 are successful diverse systems if the same common cause does not degrade the performance of both A1 and A2. For example, the same latent design defect in both A1 and A2. Some unwanted interaction between the item and its environment resulting from unexpected signal pathways, unexpected propagation through interconnections and degradation through shared resources, shared memory computing resources, communication resources. A1 does not degrade A2. A2 does not degrade A1. And the panelists are welcome to add any other constraints to limit the scope of their answers. The hazard analysis of interest is shown using a reference model from IEEE Standard 1012. Requirements from the plant level analysis drive the development and safety assurance of a system in three activity streams. The middle one in black is development, the top one in green is verification validation, and the bottom one in red is safety engineering. Hazard analysis may be performed in any each of these activity streams. Our focus is on the hazard analysis performed as a part of safety engineering. The bottom red block. Hazard analysis is performed at every phase of the development process including the planning phase, continuing to the concept phase, analyzing the interactions between the system and its environment there, proceeding to the requirements phase, then to the architecture phase, and there may be many iterations in these phases before clearing the gate to advance to the next phase. At each phase, hazard analysis produces the safety constraints or requirements for the next phase, and also the test cases for verification. Detailed design, implementation, testing first the system itself, and then the system integrated in its operating environment. Now let us walk through question one again to refresh your memory. Can safety evaluation of a reactive protection system based on state of the art methods for hazard analysis be as effective as the current practice based on design diversity? Please be brief. Bottom line of front. Ideally, yes, no answer to start with. I'll introduce each panelist when I invite the response. And at this point, I'm going to share this and invite the response from Matt Gibson. Matt, get ready while I find your slide here. What are you looking for? It's like with which I want to introduce you. Okay. Showing me only a few windows. It didn't show me the windows looking for. And now I'm there. I can see all my windows. Here you are. Matt Gibson. Is the technical executive at every 13 years there. Leading the integrated digital systems engineering portfolio. Licensed control systems engineer, certified cybersecurity professional. He's got more than 40 years of broad digitalized systems experience, including hazard analysis, digital architect role, design, implementation, support functions, human factors engineering, and cybersecurity. Matt, take it away. All right. Thank you. I appreciate you inviting me to share our perspectives. So for question one, I don't want to disappoint you, but I don't have a yes no answer for that. It's an interesting question. The answer based on our research is yes, with some important clarifications. You know, the first one, there's two clarifications I want to get to you. The first one is, you know, modern hazards and reliability analysis combined with risk insights and used within a systems engineering process. And it can indeed demonstrate a high safety assurance. So these structured methods can reach a safety and security and performance conclusion, create analytical evidence to support that conclusion. So they're sort of capable of doing that. So our EPI research on operational experience data and on these analytical methods and the combination of them support these conclusions, which we've done over several years. So the goal of these methods is not to eliminate diversity, but rather to find the right combination of design characteristics within an external to the INC function that will deliver the needed safety, security and functionality. So it's a holistic thing. All right, this may include diversity at one or more levels of the facility design as indicated by the analytical methods. So, you know, visualize the decomposition decision stat. Now, where you put diversity and why you put it there and which part of the stat really makes a difference on the effectiveness and appropriateness of that. The second clarification I want to get to is to expect to use the word comparable. So the expectation of comparability to the existing methods shouldn't be the objective of looking at these new methods. Because these new methods are quite innovative in a lot of ways and should be judged on the own merit. Because they may be demonstrated to be superior to the current approaches in all aspects of efficiency and safety. So that's something we can't assume what we do now is the gold standard. Deterministic, you know, diversity, you know, saying, hey, I'm going to do these set things with this set redundancy is not necessarily a sound position. You know, we looked at a lot of research into that and there's not a lot of real good experiments that support the validity, you know, that really is a good idea in all circumstances. Certainly in a risk informed area, but not in all circumstances. Alright, so what you can do when you say it's got to be diverse a certain way and it's got to be redundant in a certain way is you now make that the design and you may divert your systems engineering analysis away from things that could result in behavior later on when the system is running. So I think that, you know, your modern hazards and libel analysis provide a better understanding of the safety function and they can ensure safety and security in a more efficient and technically sound manner. That's what I would say about. Thank you, Matt. Thank you. I want to ask Mark for Nokia of General Motors GM has a lot of experience with safety critical systems and vehicles. Any particular example that you can share with us after I introduce you Mark is a GM technical fellow GM technical technical fellow is the highest position in the technical ladder in General Motors comparable to the senior technical advisor position in the federal government is the principal system safety engineer for all GM propulsion systems worldwide over 20 awarded parents master's in engineering sciences from RPI bachelor's in mechanical engineering from Purdue both excellent engineering schools professional engineer in the state of Michigan expert systems engineering professional recognized by Inca say he chairs the task force for SAJ 3187 recommended practice for applying system theoretic process analysis or STPA to automotive applications. Mark. Hello. Good afternoon. Good evening. Good morning depending on where you might be and I'm Mark for Nokia and I have a similar answer to what Matt expressed I think it's a qualified yes for me there are caveats along the way as always but one thing that I can talk about within the automotive industry is sometimes the idea of diversity or basically redundancy is not an option we have a lot of it's expensive to do things and it's expensive package and produce and implement redundant systems in all of the safety critical systems that we have to manage so one of the examples I have is looking at my screen here looking at I'll make it big like that and hopefully you can see this is a component of a electronic throttle control device so what happens is in the old days that I can remember you used to have a pedal mechanical lever hooked up to the throttle itself you press the pedal and the mechanical linkage would open the butterfly valve and you'd go faster or slower depending on how it is and so we replaced that 25 years ago at General Motors maybe a little longer with a motor controlled system and you can read the text down below I'm not going to read it for you but it has a lot of advantages to do over a mechanical linkage gives us a lot more latitudes with a lot more features that we can accommodate so in essence it's throttle by wire now the issue is what do you do with the control system that misbehaves so should I put a second throttle body in there should I put a second motor in should I put a second position sensor you know in our world every time we add something we also add the risk that that will fail too and so what we do is we end up we end up dealing with more of a watchdog approach that you'll see in the center the CPU is going to do the processing the watchdog processor is an independent parallel device that will keep an eye on what the central processor is doing as far as its controls of the motor and therefore if we see something misbehaving or unexpected the watchdog can actually pull the plug on the CPU and return us to an automatic condition and a very simple diagram looks like this to go through so we have pedal sensors something figures out what the pedal is doing something then says position the throttle this way that's the request the throttle execution goes out for a command but then we look at basically very simple things like the request was made and the position was commanded how do those things match up if they agree within a certain tolerance things are good if not the watchdog will shut it down and we've done millions of vehicles like this and so I think there's a fairly strong case that you can actually achieve the level of safety integrity you need without necessarily having to have redundant systems just an example here so I'll turn it over to the next presenter to Sushil. Thank you Mark. You're welcome. I'm going to request Captain Sham MomQuest so Captain MomQuest here's let me introduce you. Captain Sham MomQuest is a visiting instructor at the Florida Institute of Technology masters degree in human factors experienced considered an expert in aircraft accident investigations active current pilot for Boeing 777s and on international routes he's instructed in a variety of both general aviation transport aircraft numerous technical publications on flight safety and accident investigation automation and human factors lead for the commercial aviation safety teams joint safety implementation team loss of control working group and many such other bodies that investigate safety like the aircraft safety state awareness working group joint implementation measurement and data analysis team he's a fellow of the Royal Aeronautical Society member of the Flight Safety Foundation and BSE flight deck and handling quality standards for transportation transport aircraft committee. Sham? Yes, I think you're so presenting there. I don't have a slide for this particular item but the issue of the potential for common cause failures even though you have diversity can be seen in examples as simple as the Boeing MAX experience that we had there's been several other aircraft where the exact same thing was implemented where the systems were seen to be diverse and the problem turned out to be in the requirements and those were common throughout all the designs so even though you did have diversity in some of the designs in terms of redundancy they all led to a common they all led to, sorry, led to common I guess you could say fault modes that resulted in the accidents and a large part of that was that the human reaction to the to what they were seeing and to the complexity of the situation was not fully accounted for in the standards and so we really ended up with some accidents that could have been avoided through a more robust hazard analysis method and diversity really had nothing to do with it. It didn't change the outcome at all I can't hear you. You are muted, Susheh. Thank you, Shem. I'm going to request Paul Bouchard. I'm going to ask for your answer based on your experience after I introduce you. Paul is a master's in computer science software engineering. He's an INC engineer at New Scale. He's built his INC engineering on the foundation of three years as senior engineer performing safety evaluation for regulators. Five years as control systems engineer at Idaho National Labs advanced mixed waste facility. Four years as test engineer at INL's advanced mixed waste facility. Five years in the steel industry as INC engineer and 15 years in the Navy as electronic technician reactor operator. So, Paul, what's your answer on your experience? Well, thank you, Sushil. My experience, I believe it is technically feasible that we can use an advanced hazard analysis methodology in order to assist, if not completely eliminate design diversity. The design of the New Scale power reactor is such that we have very limited space. It is a small modular reactor, a fraction of the size of the existing reactors. And as a result of that, we had to limit what instrumentation, what control devices we had. And part of that we used STPA to analyze our module protection system. And with the intent that the analysis we used, we ignored all of the assistance of redundancy, of diverse design of single failure analysis to optimize each channel of our monitoring and control functions. And I believe we are very successful in that. We began our analysis during the planning stages, carried it through conceptual design, detail design and development of our design solutions. And we continue to use it even now as we're going through the process of our standard design approval submission. And I've given presentations at the STPA conference at MIT where I was able to provide a little bit more data to support my claims. And I do have to agree with Matt and Mark. It does have to be taken with some precautions. It's not a panacea. It's not going to immediately solve all the problems. But judicious application of proper hazard analysis can significantly reduce the resource requirements for diverse design diversity. Thank you, Paul. I'm now going to call upon Professor Ellen Walsing and let me search the slide to introduce you. Professor Walsing has over 30 years of experience in safety critical software intensive systems, including development certification, industry academic collaborative research in many application sectors, auto, medical, nuclear, particularly in Canada, Darlington nuclear power plant where digital safety systems were pioneered almost three decades ago. He has run 15 years of software consulting business over 19 years as a professor at McMaster University in Canada where in the past he had been director of the McMaster Center for Software Certification and director for the Software Quality Research Lab. He's the founder and the first director of the Software Certification Consortium. Ellen. So thanks very much. Just getting my screen. So thanks to Sheila and thank you for the invitation. So I listened with interest, but I think I'd already guessed what people were going to say. So I'm not sure if you can see the slides. So I need slides for one reason mainly to remind me what I was going to say. So hopefully they'll also help people look at what I'm saying. So my straight answer is a little bit different. I think I'm going to be the only one here who says no outright because I didn't give myself the chance to say sometimes yes and sometimes no because as soon as I get into that situation I really want to say no. And I don't have anything bad to say about modern hazard analysis methods. I think everything I've heard from the panelists has been true. They've been fantastic improvements in what we can do in hazard analysis. I just didn't see that that was the solution to the problem. Even if I assume a perfect hazard analysis I think there is a role for design diversity at play in being able to assure a safe system. And a lot of my research is related to safety assurance, producing assurance cases and trying to do that without the design diversity and it turns out to be quite difficult. In fact so far we haven't done it. So to tell you where I'm coming from in terms of assumptions and experience so when I do safety assurance or when anyone does it I think it's most effective and I heard from the panelists that they were thinking the same thing. It's done before and during development of a system not after the development. You plan for it to be safe. So you use the safety assurance to sort of evaluate the development of the system as well as the outcome. And what we're interested in is people process and product and showing that all hazards were eliminated or adequately mitigated is just not enough to show that the system is safe. One of the things that we found in practice is you need to show that the system delivers the behaviour that was expected. If it doesn't deliver the behaviour that was expected then people find workarounds and those workarounds quite often disturb the safety that you've carefully built into it. So why do we want those designed diversity? I think people have already mentioned it. We avoid systemic failures or we hope that we can avoid systemic failures in constructing and implementing the safety. So what I'd like to do is have a look at something that was done on Darlington. So she'll mention Darlington. This is what we did. We actually did have two diverse systems but related to what I think that someone said earlier, I think Richem, this started at the very top level of doing the development. So right from the goal level requirements we started introducing diversity. So in fact why do we want to do this? It turns out that we make mistakes when there is complexity in the systems and what we can do is try and use diversity to have complexity in different parts of the systems. I don't have time to go into detail here but when we planned this we actually had different complexity in different parts of these systems and the reasoning behind that is there was less chance then of having systemic errors that linked both systems. So why do we want design diversity? So we have quite a lot of anecdotal evidence and very little experimental evidence. There's no way I can sit here and say I can prove that it's necessary. I also haven't managed to prove it is not necessary. So in a nutshell my question is would it be responsible not to use it when we don't have something definitive in terms of an answer? And I'll stop there, Shil. Thank you, Alan. I'm going to request Dr. John Thomas next and let me introduce John. He's the executive director at the MIT partnership for systems approaches to safety and security and the MIT safety and cybersecurity research. He's the co-director along with Professor Nancy Leveson of the MIT engineering systems lab and he has affiliations with other labs, very interdisciplinary persons, safety and security labs, systems engineering, software engineering, complex systems. He has again multi-disciplinary teaching interests, systems engineering which includes safety critical systems, cyber physical systems, information and control systems, requirements engineering and human-centered software engineering which includes user interface design. John has participated in research with industry, collaborative research in many sectors, automotive, aviation, outer space and nuclear. And in some of these application sectors he's also contributed to the development of standards there. John? Thank you. Well, my short answer is yes. I think that we can use state-of-the-art hazard analysis methods to provide a unique benefit that is not provided by design diversity alone. Now I think I actually agreed with a lot of the previous presenter who said no. But I'm reading the question a little differently. When I say yes and when I listen to all the other presenters say yes, I did not hear an argument to eliminate design diversity or things like redundancy. I think that's, you're going to be hard pressed to find someone who argues that we can eliminate across the board design diversity and that does not play a role in safety and what we need to achieve. But there is a problem. I also think there is sound evidence that design diversity is not enough. So it's not necessarily an either or pick an apple or an orange. I think there is a role for both of these. Let me share my screen. Bottom line up front. Some state-of-the-art hazard analysis methods have been proven to identify specific methods and some are better than others. We're talking about things like engineering deficiencies, human interactions that aren't otherwise conceived and identified and addressed, flawed requirements that aren't otherwise detected and prevented, complex interactions that are obvious to you. This is the unique domain of state-of-the-art hazard analysis methods and design diversity has not been shown to solve this satisfactorily as I think all of the presenters would agree. Ten years of evaluations and trials in the nuclear industry have demonstrated benefit from state-of-the-art hazard analysis methods for these areas. Now I'm using the word some here because I don't want to be misunderstood. There are a lot of hazard analysis methods. Some are better than others. Some have more evidence than others. I'm not talking about all. I can't make that blanket statement. I'll throw in an example from the software development on the space shuttle. They actually used a similar approach 40 years ago to what's being proposed today in nuclear. They were 40 years ahead of the game at NASA. They used multiple redundant computers, a backup flight system that was independent from four systems. It was similar to get us home if we need to. We have five computers, one diverse from the other four. Different software code, different programmers, different contractors. Different development environments, different configuration management system across the board. The very first launch had to be scrubbed because the backup computer couldn't be synchronized with the primary computer because they had different assumptions and different requirements. You could figure out which one to believe. The whole thing had to be scrubbed. Redundancy and diversity, I don't think, is a silver bullet. It does have a role to play, but it adds complexity. Complexity is almost the root of the system's problem. This is known as software diversity. It's also known as end version programming. We have a very rich scientific literature. This started actually in the 70s. It's been studied very hard in the 80s and 90s. We're talking a 30, 40, 50-year-old idea. I've got quotes here from the scientific studies studied at the bottom. The independence of assumption of errors that's fundamental to the analysis of end version programming, meaning software diversity, does not hold. Why doesn't it hold? Because the independence is broken. Even if you've got separate teams, separate requirements, do everything differently, we're breaking the assumptions. Independent teams still share these biases. They share similar assumptions that humans tend to make. They share similar gaps and experiences. The errors that we're seeing are not uniformly and randomly distributed. They tend to be clustered around common edge cases, common gaps in our experience of abnormal behaviors that we're not that familiar with, leading to common faults. Here's a nuclear example. This is a real example. I've scrubbed the details so you can't read this and figure out where this came from. But this is an event that happened at one of hundreds of events that have happened in the last 10 years in the industry. We've got two different supplier modules here highlighted in blue. Different suppliers, different requirements that they each created. It was seen as diverse according to the reviewers at the time. They're both reviewed, both passed every check that we had. Independent requirements and implementation. It was tested and it was put in operation. Months later, and this is the nature of the problem, isn't it? Months later, we've got brand new interaction. Nobody tested for, nobody thought to check for, and both suppliers made this flaw, made this mistake. Both systems interacted in a brand new way that was overlooked until it happened in operation and led to a significant event. Notably, neither one of these failed. So we're going to go beyond the failure problem. There was not a single failure in here. They worked as intended by each supplier and that was the problem. The intention was wrong. So the question is, can new state of the art has analysis methods reliably and consistently identify these flaws? So STPA is one of the methods that have been tested on these cases. Over a dozen of these cases from the nuclear industry using nuclear teams of engineers who learned the process, applied it compared to previous teams who had applied other methods without the state of the art has analysis. And the answer is yes. The empirical evidence exists and the answer is yes. These new methods are effective. The old view is sort of, you know, these are random problems that happen. The world is full of only random problems. And as long as we have enough diversity, we're good. I mean, that is part of the problem, but I strongly disagreed. That's the whole problem. I think we've got very strong evidence that shows it's not the whole problem. The system's view is it's not all about components and combinations of things randomly misbehaving. It's about the interactions between them. The common assumptions and how these are going to have common causes between diverse equipment. New scale. The talker on the call right now, Paul Bouchard, did this in his company. This is data from his organization that he's presented. New hazard analysis methods had some overlap in green with the kind of things that we catch with a diverse approach. But we have brand new contributions to hazards and loss scenarios that are not identifiable and preventable. Not preventable. That's what these numbers are for the preventions that were missing without a state-of-the-art hazard analysis method like STPA. This was submitted to the NRC, by the way, as part of the safety assessment, and it was approved. Embraer did this evaluation. The bottom line here is 44% of the results from hazard analysis overlap with what they could identify today with traditional techniques. But 19% were identified earlier with hazard analysis, which is very important. And 37% were identified only with state-of-the-art hazard analysis methods are overlooked with this assumption that diversity is going to solve a problem. Nobody caught it anywhere until these events start to happen. There are safety standards that are accepted worldwide based on this concept of using state-of-the-art hazard analysis to complement the traditional redundancy and diversity approach. And it's necessary, this is a picture from the standard ISO 21448 which shows this common engineering concept of unknown unknowns. It shows that we need hazard analysis to shift or minimize the amount of unknown unknowns. This is the unique domain of these hazard analysis methods. You don't get that by throwing diversity at the problem. With that, I'll end. Thank you. John. Yes. There's a question and I'm going to read this to for everyone's benefit. Diversity, I'm reading the question verbatim. Diversity is the measure to address all the things that you cannot think of. To say that you do not need diversity means that you think there's nothing you did not think of. But since all systems and how they're used evolve or change over time, is this not an impossible assertion? John. Can you unparse the question? What is the impossible assertion? Well, the question, the gentleman who personally sent the question has a view of diversity and that view is that it's the measure, uses the word measure, maybe not intended, but diversity is the measure to address all the things that you cannot think of. To say that you do not need diversity means that you think there's nothing that you did not think of. But since all systems and how they're used evolve or change over time, is this not an impossible assertion? So the claim. Is that we do not need diversity? I think nobody is saying that we can't, that diversity is irrelevant. Diversity is that we don't want, for example, diverse components in providing a function. I think diversity has diversity, but every industry outside nuclear recognizes that diversity is only part of the problem. We do not have perfect measures of diversity, which is part of the problem. We have a history of believing that these components are diverse and then we have an event and discover we were wrong. They're not diverse. There is something in common that nobody knew about in these industries has been hazard analysis. It's the solution to compliment diversity and you can think of it as hazard analysis. You made the main point that you aren't advocating total elimination. I see that Matt would like to say something. Matt, please go ahead. Good discussion. I do want to come back to that hazard analysis and the use of it and diversity need a context. When we're discussing hazard analysis, we run the risk of discussing it way too narrowly. That's kind of maybe what we're hearing here because you really have to use these methods in context because along with hazard analysis and reliability analysis and overall engineering process, that's the only way you can use it. Hazard analysis is diagnostic. It finds problems with your design. You didn't have to take that insight and change your design. If your design requires diversity to be adequately reliable, then so be it. It can be at different levels of the system stat. You can have a mechanical thing over here that's not related to your system that provides the necessary reliability, total system reliability to achieve its mission objective under all circumstances. We've got to get to a holistic engineering view. Not a very narrow like this is this. I just want us to think about that a little more broadly. Thank you. Thank you, Matt. There were a few other questions from the audience, but they fall into this topic of the subsequent questions. I'm going to defer answering those, bringing up those questions from the audience. I'd like to now conclude the discussion on question one. Stephanie, over to you for the first poll. Okay, so I think we're going to, Spencer, you'll show the poll and the audience should see the poll. Okay. And can you give us a sense for, and so we're asking the audience to show the same similar question we asked the panel members. And so looks like 60-40 split. Spencer, can you tell if the rate of return is slowing down? Okay, so let's close the poll and, Sushio, you'll use that information as you move into round two and Matt has his hand raised. Just something to everybody keep in mind when you're showing your slides, there's about a three to eight second delay between that and when the people see it. I'm getting texts while we're doing this and people say, I'm not seeing this slide, right? So if we just bear that in mind when we're talking from our slides, maybe could slide up a little bit or figure something out about that delay because sometimes we're talking way past your slides for people to see. Thank you, Matt. Good point. All right, Sushio, you ready for round two? So can you give me the score? I don't have that screen. It looks like, oh, it's changing. It's going 50-50 now, getting close to 50-50. Where did it end up? It looks like 54% yes and 46% no. I don't know how real time it is because the yeses don't seem to be changing at all. So sometimes it's good to close the poll and then put the results up. Sushio may be ready for round two. Yes. So apparently a good part of our audience is not as convinced as some of our panelists are and we'd like to now give some more evidence to the audience through the second question and see if we can sway some minds. Does sufficient scientific evidence exist to support the assertion that the hazard analysis can be evaluated independently with consistency for correctness and degree of completeness needed to avoid design diversity? For example, do independent verification and validation methods exist? So from the audience during the discussion on the first question this was one of the questions that came from the audience. How do you perform an independent study to ascertain the adequacy of the quality of hazard analysis? So I'm going to request Paul if he has anything to add here Paul, which art? Yes, yes I do Sushio. Kind of expanding on something that Dr. Thomas presented. I'd like to share the context of the information from NewScale that he showed. When we performed our analysis in STPA, if you're not familiar, you'd analyze the functions of a system rather than the components of a system and you look at the various ways that a specific function can not perform what it's supposed to. This screen shows an example of one of our safety constraints which was developed from the analysis and what we did was we built a cross-reference that compared the results of our analysis with our existing requirements. These particular ones were early on was the functional requirements essentially conceptual phase and what we looked at we looked at the different requirements whether or not they contributed to satisfying the safety constraint and then any evaluation that was necessary to clarify that contribution. And the numbers we ended up with this is Dr. Thomas showed portion of this when we did our functional specifications, we had a very relatively low number of relationships between the safety constraints and our existing requirements. Very few that had more than 10 relationships more so with 5 or less and there was 82 safety constraints that were identified that we showed no relationships to any of our requirements. We took the results of that and rolled it into the next phase of design, the detailed design phase for our protection system and when we re-performed the cross-reference we had a significantly larger number of relationships in total many more with multiple greater than 10 relationships we were down to 20 safety constraints that had no direct relationships and again the number 75 for 5 or less that's primarily because we improved those numbers to 10. When we got to the design solution phase, again a dramatic increase in the number of total relationships the one number that stood out for many people was the fact that we still had 15 safety constraints that had no direct relationships. When we analyzed those 15 when we were performing the analysis we looked not only at the interrelationships between the various components within the system but we also looked at the interactions with the operators and obviously we were not at the point where we can engineer operators and so those 15 safety constraints ended up being administrative in nature and they were incorporated, they've been incorporated into our technical specifications, operation procedures, abnormal operation procedures and so at least in our experience we found a very clear evidence that it can be shown in real numbers what the impact of the hazard analysis is. So that's good experiential data Paul. The question really is if there is a third party evaluator or assessor or certifier are there mature V&V methods for that third party to evaluate an organization's hazard analysis of a digital safety system I'd like to ask any other panelists who might have any thing to contribute to answer that question. I believe I can address that one as well Sushil. We have within new scale we have an independent verification design group and they have been they're in the process now of performing their own hazard analysis of the react protection module protection system and I have yet to see the results of it I have not been involved in it at all which as a very very interested party is rather frustrating but I understand the need for independence but I feel that an organization like this could either be within an organization within General Motors which then Boeing or whoever or it could be an independent third party organization and I believe they would see similar results. So your key idea is that they can perform their own independent hazard analysis. Yes that's correct. Okay thank you. Ellen you had your hand up and then you lowered it. Well it's not exactly on that topic it's just related to the discussion before in terms of hold your horses on that we'll get to the other I don't mean question one I mean what was Paul was saying before that Okay. So related to the question that you asked now I personally don't have any problem with the experiments that are going on in terms of showing how effective STPA is for instance I think it's great going back to what John said earlier I thought that the question really was whether we needed both diversity and good hazard analysis and if you look at this one of the great things that I think that I like about STPA is that it shows me feature interactions which includes the environment so Paul was talking about the operator so what we've had is we've had good analysis for feature operations where the at least interactions are in the system that we're building these are feature interactions outside the system we're building and that's what causes a whole lot of our problems so the fact that STPA and maybe other hazard analyses I don't know about but the fact that we can do things with that is a step in a huge step in the right direction but if I interpret your question slightly differently in terms of is there experimental evidence that just using STPA means I don't need diversity I don't think there is any so I know that everyone or John says that that wasn't the question that was the question you proposed yeah that's fair enough so here's the situation the questions were so interesting and had so much to offer that we went way beyond our time budget for all three questions so I'm going to drop the third question and the third poll but there was an audience question that came in that pertained to the third question and that was directed to Paul but I'm going to ask all of you to think about that question that came from the audience and that's about expertise what kind of expertise do you need to get this kind of quality so Paul showed some data and some results that were impressive but is that expertise replicatable so who wants to take on that question I can start Matt, your eyebrows were clicking I see John but let's go with Shem first Alright so in answer to the question it absolutely does require training and also facilitation particularly the first few times until people get used to it I find working with various organizations and also students that I've been working with depending on their background they have to be pulled back on track multiple times but after a while people start to get it and of course we're able to get really really good results and I know that John Thomas can talk to that more but even with people that don't have subject matter expertise it's really amazing but of course subject matter expertise gains quite a bit there's no way to escape that but it's really one of the things that we're working on at Florida Tech is creating a certificate program in these topics I know that was something you wanted me to talk about but I can hold off on that unless some other people speak to this first you're muted Sushil you're muted Sushil you're muted Matt I want to come back to you I want to add to what Shem said you have to be careful about the Maytag Repairman problem this is an organizational problem you get people trained on STPA and other parts that go with that they need to be engineers, they need to be reliable analysis, they need all that well if they don't do it regularly they will atrophy it's not something like an organization wants to adopt STP can't go out and say well I'm gonna do STPA on this project and I'm not gonna do it for two years they have to create a center of excellence that does it regularly it's like playing baseball we all know the rules but we quit playing for two years golf my favorite I can go way up and down in that if I don't play red it's the same thing so I just want to add that to the the things we have to consider when we adopt these advanced techniques thank you Matt, John you wanted to see something? Yeah I do, questions two and three are related in my mind, question two is about evaluating the analysis can we evaluate analysis consistently and question three is do we know what affects quality of the analysis so they're both really connected in my mind and I want to address them together as we discuss I'm getting a new appreciation for the concepts here and especially appreciate Ellen's clarification that just was very very helpful so I think we've got a sort of consensus that diversity and hazard analysis contribute unique insights we need both we can't get rid of them we can't eliminate diversity across the board but we also know we can't have diversity maybe on everything I know in aircraft there are certain things what physically you can't have diversity you don't want two actuators moving on the same surface of an airplane wing because that's just a recipe for trouble so we're forced to have one jackscrew in certain places and other things like that so here's the question that we're posed it says needed to avoid design diversity so looking at this slide with this example we know in hindsight their strategy to use that design diversity alone to go to two different suppliers didn't help the problem in the same assumption nobody caught it everyone believed it was diverse at the time which is more important than the hindsight saying yeah but it wasn't really diverse that doesn't matter if they have no way to figure that out at the time so on one hand before this problem caused an event what do we do do we throw diversity at it using the current approach or do we throw hazard analysis at this I think the best answer is let's do both let's take that off the table and then we get to the heart of these questions you can't do both you gotta choose one do we throw design diversity the traditional technique for decades at this problem or do we use hazard analysis well in this case we have both halves of the experiment conducted actually we have a factual answer for this case and for dozens of others as well but let's just talk about this one the factual answer is they threw design diversity at it right they threw that approach they strive for design diversity they evaluated design diversity they gave it a green light and it didn't work they also took a set of blind engineers and said let's evaluate the logic the functionality of this thing which by the way shared among both of these so let's just evaluate the functionality shared functionality right you could implement that functionality in one module two or three or whatever he said let's get the functionality right using STPA hazard analysis using state of our house analysis and what they showed is teams using STPA to get one definition of the functionality right we're able to eliminate this problem using hazard analysis alone and so if the team let's suppose the team had had this choice on the table to put first modules sort of blind or in the dark just give it to two suppliers and hope there's no common assumptions or with a choice to give them hazard analysis to do a directed targeted analysis of flawed assumptions that you would make whether we're talking one module or two or whatever and what we found is the hazard analysis found all the problems that were encountered were caught with hazard analysis they could have in hindsight gotten it right the first time maybe without even paying for a supplier too because they had the definition of all the logic and functionality of supplier one that was necessary to get the job done so I think there is an argument that this was tested the hazard analysis was tested blind it caught the flaws tested in a natural experiment because they did it for real and it didn't catch the problem so I think the answer is a resounding yes and this is just one case now part of this problem is how do you review those assessments someone does STPA claims it's done how do you know they did it right well that's been tested as well as part of these blind trials that have been done across multiple nuclear utilities multiple applications you can see on the left hidden flaws that were identified what happened is all teams consistently identified the exact INC errors the exact common cause errors and failures that were missed with a diversity approach sort of a blind diversity approach the unmitigated human errors and crime flaws across the board and we're talking multiple teams independent of each other different geographic locations different backgrounds PRA experts participated although they weren't using PRA they just thought that way digital INC engineers participated and other folks and and every one of these was able to show that they can reliably use these methods to catch the problems that doesn't mean perfection when I read this question I don't read this question saying do we have scientific proof of perfection we don't have that for any method including how we evaluate diversity but relatively speaking relative to other methods the sufficient scientific evidence exists to support the assertion that the quality of the hazard analysis and the conditions that affect it are known to get useful consistent results yes that we can say yes does it pass the bar of providing these consistent results that are understood well enough to provide quality yes thank you John there's another question from the audience is qualitative hazard analysis sufficient enough for the decision makings in designing diversity and redundancy will a quantitative approach like reliability and consequence analysis be useful to support the decision making it would like to take that on the decision I would say it depends a lot on the decision there are some decisions where a probabilistic assessment is uniquely positioned to give you a good answer but not all decisions in fact a lot of decisions don't get any answer from a probabilistic approach or they get exactly the wrong answer and we have evidence we have tons of OE in this industry that have shown that Mark you had your hand up thanks I just wanted to comment on a couple things real quick one of the things that we found and we use STPA within General Motors especially for human machine interactions and interactions is the operative word not interfaces but interactions and something that I believe Paul said that you need to analyze the functions and not the component within GM we have a gigantic history of DFMEAs as the way to demonstrate adequate design integrity but those are all very failure based approach and what happens is we tend to focus on the components do those DFMEAs and then we come around and put all those parts on a table and wonder why they don't fit as a system or if they do fit why all of a sudden new behaviors, emergent behaviors appear and so this idea of having a analysis technique like STPA you want to call it your state of the art approach is an excellent compliment to that you know we have some inertia within our culture that if I had the biggest lever in the world I could never get rid of DFMEAs within General Motors and that's really not my objective my objective is to supplement those existing well known techniques with newer techniques that can address a system level approach to things early in a process when there are still a lot of unknowns and so thank you thank you Mark thank you we are coming close to the end of a lot of time so I want to read one more question from the audience it is really very similar to a previously asked question but I want to honor the person who asked the question by presenting it to you again just to make you aware how significant this question is to our audience what evidence would suffice to claim to satisfy the claim that the people performing the hazard analysis possess the right skills to produce correct and complete results? Shem? I think that they again will come back to the need for training and standardization I did want to mention just very quickly that the use of the hazard analysis method will at times point you in the direction of redundancy in fact that's part of what we're doing when we're using it for aircraft design it's not like we're missing that the hazard analysis is actually highlighting where we need to do that but also showing us where we don't need to but in terms of the system or the training it really is necessary you know we are in the process we've been hung up due to COVID of working on a certificate program at the Florida Institute of Technology or Florida Tech which is again we've run into some administrative hurdles but some of the planned certificate programs would include system safety engineering stamp the overview of theoretical version CAS or causal analysis using system theory SCPA and safety management system approach and I'm not sure to what extent the nuclear industry uses safety management systems but there is a lot that can be done with that as well as cybersecurity so it's something that might be possibly a way forward for some of the people where they are able to enroll in a current master's or doctoral program. Thank you Shem that is kind of information that question was looking for so that concludes the discussion on question 2 you have actually answered question 3 as a part of this discussion Shem so Stephanie over to you for the second poll okay do we have time for a second poll if you can roll that out in the interest of time are we able to discuss while folks are filling out the poll I think we just have a couple minutes left so I'd like to just focus on I'm not seeing any results I don't know if others can see but there we go so to remind folks can the requisite quality of hazard analysis be evaluated independently with consistency so a positive response so you have been compelling in your discussions today I do see it moving back and forth but I think I think we're pretty close again I don't see any positive, I don't see any yeses fluctuating so it's an interesting phenomenon maybe Mark it's the people who believe it jump right in there and then the ones who are on the fence that are coming in so thanks very much for participating in the poll you need to give me a sense for time if we need to move on we should proceed to conclude now Maricio are you able to show that slide I'll stop sharing my screen what's up there is the exact evidence that's required by applications Maricio Gutierrez and Sushio and Paul Redstock for putting together the session and I want to thank our panelists in particular Mark Vanakia and Shem Monquist for bringing us your knowledge from experiences outside the nuclear application sector we have much to learn from you Dr. John Thomas and Professor Alan Wessing for bringing us knowledge from your research across diverse application sectors and Matt Gibson and Paul Bouchard thank you so much for being here and sharing your knowledge in the nuclear field and of course our technical support staff and the conference organizers for making this session happen and thank you audience for your questions and for participating in the polls and if you have any more insights that you want to share with us please there's an email there for Paul Redstock and with that I shall declare the session closed thank you all again for a great session