 about approach for comparative evaluation of the nuclear energy system of scenarios. And this presentation is based on the result of our collaborative project kind. What was done under this collaborative project kind stand for key indicators for innovative nuclear energy system. This product has developed an approach for comparative evaluations of nuclear system of scenarios based on the application of the set of selected key indicators and a selected judgment aggregation and uncertainty analysis method. The project also performed some case study on trial application of this project and the developed approach is recommended for establishing a productive dialogue between energy option proponents and decision makers regarding sustainable nuclear energy options. How the task is look like? For example, the problem of alternative evaluation may be defined as a ranking of the alternative from the best to the worst. You see here alternative one, two and M. These alternatives are evaluated based on the defined set of indicators. Key indicators one, key indicators two, key indicators M. In principle, it's often some alternatives indicator can show better or worse performance in comparison with other alternatives. So the question is how to select the best alternative? We of INPRA does not invite something new. They used the multiple criteria decision analysis making techniques, MCDM. What is it? Multi-criterial decision making techniques are tool aimed at supporting decision maker who are faced with making numerous and conflict evaluation. And it includes two parts. Multi-criterial decision analysis MCDA and multi-objective decision making, MODM. This is two main components of this techniques. In the first case, the MCDA, multi-criterial decision analysis problems, consists of a finite number of alternative explicitly known in the beginning of the process. And each alternative is represented by its performance indicator or criteria. And the objective is to find the best alternative or to find a set of good alternatives. Contrary in MODM, multi-objective decision making problems, the alternative are not explicitly known. An alternative or solution can be found by solving a mathematical model. The number of alternatives is either infinite or not countable, for example. For our problem, for kind product, MCDA approach was found to be the most appropriate. And we are based on this MCDA approach. The MCDA methods may be categorized into following groups. Well, you're based on training, reference-based, and some other. And you see here the list of all these methods. I would like to go into the description of each method. But among them, the MODM method, based on simplified additive baiting aggregation, was recommended for the needs of the kind product. The reason for this, that this MODM method was found, its wide application for different kind of problems in nuclear and non-nuclear fields. And you can find wide experience of applying these methods in many of publications. And this method has its extension of using of the uncertainty estimation. But some of our participants applied, not only MODM, but other methods. For example, Promethea or Topsis. But the basic method applied in the kind product was the MAFT method. This here sees the elements of the approach, of the kind approach. At the start of the comparative evaluation process, interaction among decision makers and experts is very important. The next step is the formulation of the problem and then formulation of the alternatives. Then you need to select the key indicators and calculate them. The next stage is the determining of the ranking of the alternatives. And the very important steps is the uncertainty and sensitivity analysis. And the result is the comparative evaluation and interpretation of the result. Here I would like to show you some specifics of this process. Mainly, because maybe it will be more clear to understand when I show you what is behind this method. And this is valuable or this is useful for the exercises that I will provide you at the end of the presentation. So, let's look at the alternatives and key indicators. Here you see some performance table. When you calculate the indicators, you have the performance table. Candy counters may be calculated in natural value and score and something else with different units. First, you need to transform, to convert this in so-called single attribute utility function. Instead of indicators, you have this single utility value function. And this utility single function has no any unit. And it range from 0 to 1. From the worst to the best. Worst is 0, best is 1. And here you see how it can be done with utility function. For example, here you have indicators and this is utility function. This is a special function, which converts this AX to this utility U. The shape can be different. But this is not the end of the process. Then you need to convert all this single utility function to the multiple utility value function. To 1, all this U, to 1, big U. Or capital U. If you have multi-attributial value function, only one value to each alternative, it is easy to compare them and to rank. But how we can do this? How we can convert this to this? This we need another parameter, weight. And here you see, if we assign some weight to each indicator, we can have this one multi-attribut value, or one value, like this. You need to multiply it X to corresponded weight and summarize them. Then you have this value. This is in principle very simple. Now you see some illustration of this rolling of this aggregated. You have a number of indicators. You need to roll them into one for each nuclear system, for each alternative, and then compare. And this approach may be called as one-level objective tree. But what is the problem here? What is this issue here? These indicators can be from different areas, from economics, from safety, from waste management. But you need to assign weight to each indicator. Who will do this? It is hard to find an expert who has expertise in all these areas and to do this in proper way. Economists prefer economics indicators. Waste management people prefer waste management. And this is the issue, how to assign. So the answer can be found like here. We have the role of the indicators from different areas into this area. For example, here you see we have the assessment area or evolution area, economics. And two indicators within economics. Waste management. And here you see in this example one indicator. Reliferation resistant to environment free and so on. Does not matter, this is only illustration example. In this case it is easily to assign the weight. Because now you need to assign weight to the area. And within area you need to assign the weight. If you have some expert to economics, he will provide the weight for the old economic indicators. Who will provide the weight for the assessment area? Decision maker or something else. Figure like this. But even this area may be too much. And maybe sometimes our participant decided to aggregate them in high level objective. For example, how level objective or your strategy is cost performance and acceptability. For example. And now you see there are three level objective three. And now you need to assign the weight to each level, to high level objective. To assessment area and to indicators. It is easy to done. And it is easy to interpret result with such an approach. You can chose any level. One level, two or three or even four level if you would like. But this is how this approach show. It can be done better. And the result can be interpreted more clearly. Weighting factors. We have here some external things, weighting factors in this method. You need to specify this weight. And this is some kind of subjective things, subjective evaluation. As I understand, somebody prefer this, somebody another things. And what is within our product? It was recommended that the weighting of the indicators or areas or high level objective should be the responsibility of each member state. And it based on this reflects the national priorities of these indicators or other indicators. So we have no any recommendation for the indicators. But if you have any no preference, you can start from the uniform weighting. To see, to provide sensitivity analysis, to assign different weight to different areas, to different indicators. And show how this impact on the result. And understand why. Here maybe I will skip a deep explanation of such kind of the slides. But what should be here to note, to highlight. When you assign weight, the sum of weight should be equal one. For example, if you have two indicators, the sum of this weight here, this plus this, should be one. Here as well, for example, the performance, you have in performance one, two, three, four indicators. The sum of all four weights should be one. This is very important to keep this sum as one. After you assign this weight, you need to calculate the final weight for indicators. And this is very simple. You see, the final weight here, weight of high level objective, assessment area and indicators. Here should be multiplication. You multiply it and you will find the final weight for the calculation. Here you see this example, numerical example. In this numerical example, let's see, let's see green one. You see here, you have four items, so you have four uniform, zero, twenty-five. Some of this should be one. Here you see only one, so here you see one. And so, this is the example. Some technical things, sometimes very important. And here you see the result representation for the five nuclear systems. You see the ranking. The best alternative here is the nuclear system number three. And the fourth and second more or less on the last place, but more or less indistinguishable. And what do you see? You see here is also the contribution of the acceptability, performance and cost to each score. And it's maybe useful, valuable to interpret result. Why this one and this is the last one. What is the, for example, you see there is no any contribution from the acceptability to the fourth one. It's maybe the field for the discussion, for the consideration and so on. So it's more clearly. The sensitivity and uncertainty analysis. This is the very important steps. And we distinguish the sensitivity analysis and uncertainty analysis. Within this project, we mainly focus on the sensitivity analysis as it's more easily and more understandable. But now in full on project, after kind, we try to extend this approach and to add features regarding the uncertainty analysis. What is the sensitivity analysis? The purpose of sensitivity analysis is to examine the change in the model output values. That means ranked results. Owing to the changes in the one inputs. In indicators, we take only one indicator, change it and see the impact on the result. Or we take only one wage, see the change, assign some another wage and see the result. This is the sensitivity. Uncertainty analysis is aimed at evolving multiple uncertainty sources into comparative evaluation to provide overall ranking result with uncertainty. So we have the medium and uncertainty for this one. But it is more complex and we will do this in our ongoing project. General comments regarding the sensitivity and uncertainty analysis. First, we need to identify uncertainty sources. For example, this uncertainty may be in indicators value. Mainly, this is the objective uncertainty. For example, I don't know. Maybe you evaluate cost and you know from what to what this cost is right. Weight. Weight is more subjective. And the what is else, the specific parameter used in MCDI tools. For example, I show you how the indicators converted in the single attribute function. I show you the linear function, but it can be exponential or other. And this shape of the single attribute function is also the subject of the sensitivity analysis. The next thing is to make the informed estimation of a certainty unit to understand the rate in which you can change your indicators. And the last one, evaluation of the uncertainty in the result by performing sensitivity analysis or uncertainty analysis. Here you see the very simple weight sensitivity. How it may be done. But for weight we have some, I think that you need to remember that the sum of weight should be 1. So, if you change 1 of indicator, for example here, from 0 to 1, 1 of indicator you have chose, you need to adjust other indicators. Because if you have some 1, you change your indicator, you increase it, some maybe more than one. So, you need that other weight automatically change proportionally, holding the weight some equal 1, the unity. You should be remember. And here you see, for example, this is the reference case. In this case you have, this is the first place, second place and so on. And if you change your indicator, you say how the rating is changed. And you can identify it. If you have some preferences, here you have one ranking. If your preferences move to some another place, you may have the same ranking or the ranking may change. This is some kind of sensitivity analysis for weight, for utility function. You see here that utility function can have different shape, like linear, like exponential or stepwise. There is some mathematical theory under this shape. For example, I don't go deep, but for example, this shape prefers the people who don't like risk. This is neutral attitude for risk. And this, who prefers risk? Who prefers risk? Who prefers risk? But this is only some small notation. And the shape of this function can also impact. It is important to understand. Is it significant or not? K-indicators. For K-indicators you also can apply the sensitivity analysis, so-called here direct approach, when you change some range of these indicators and you will observe, you will analyze the impact of this change to the final ranking. There are also some advanced tools you can find, for example, multi-attributed utility theory with the uncertainty analysis. The other maybe interesting and important things for the sensitivity analysis is applying different tools. For example, you have applied here the three methods and if all methods provide the same result, that's okay. If not, you need to understand why. This is also one of possible sensitivity analysis. I'll show you this, for example, result that maybe changes the last two, five and four. But within my previous illustration, it doesn't matter, because they are practically indistinguishable. So we can estimate this result as a good result, a stable result. The step for the next comparison includes, I have 10 minutes, 15, maybe a bit more. What is my final time? Okay, let's go. I will skip some very deep technical details. So hypothetical next comparison. Then calculation of performance table is very time-consuming. You need to spend, to invest many times in preparation, this table, in principle. So to see how this approach works, you can do this on the hypothetical systems. And here we have done the convert evaluation for two hypothetical methods and for two, and for five hypothetical methods. And I will show how it was done. For example, you see here the indicators. It doesn't matter what are these indicators. Nikolay system and randomly score of each indicators. This is a value assigned randomly, without any sense. But we have our table. And when can we work with this? We have no, we don't invest many efforts in the preparation of this table. But then we can to assign weight for, for example, uniform weight for the high level as objectives, evaluation area and indicators. And here you see the final weight for the indicators by multiplication of each. But after this, we will aggregate them into the one value, multi-objective, one value and compare. And here you see the result, for example. And the contribution of each high objective, high objective scores to this ranking. This is straight only how it works. And here you see some kind of sensitivity analysis performing for these five nuclear systems. For example, regarding the impact of single attributory function. Linear weight approach to weight, to provide the sensitivity analysis. Here you see this also the hypotheticalness, including twoness, one and two. I would like only to show you one interesting thing regarding the twonesses. If you have twonesses, it is enough to have the quality evaluation, to say this best, this worst. Here, for example, X means this is better. Nothing or zero means that it's worse. And if they are more or less equal, you see that they are equal. And you have this table. You can transfer this to the two-point scoring scale. Assign to the best performance one and to the worst zero. Just easy to do. So, for the twoness comparison, this approach is acceptable. And very easy. Can be done. And here you see some results. Here also you see some sensitivity analysis provided for the twonesses. There are many different interesting features under this comparison. You can only touch one of them to show, to illustrate them. For example, when you transfer the indicators to the utility function in the range zero to one, you can use, for example, different domain. For example, you can vary them from the minimum indicator to maximum. Or you can use global when you see the maximum and minimal possible range of this indicator. This also can be done. So, you can see how it, this is kind of sensitivity analysis to more clearly understand the result. You can do this one or something else. And this here you see, for example, this linear weight approach to weight sensitivity analysis All these two nuclear systems as illustration. Now, because I have, according to the schedule, I have five minutes, but how can I use? What is my, because of this, I maybe skip some slides. Three. Okay. I'll try to be short, but not in five minutes, maybe a little bit more. Okay. So, what I would like to illustrate again, some, this approach was applied to the national case study, performed by our participant. But this is not the full scale of the study. This is the trial application, how approach works. And now we have another ongoing product in which we try to do more complex analysis. But here some kind of the trial application. And our participant includes Armenia. And here you see, they compare, compare Vivian 1000, Kandu, SMR, SMR, Small reactors of 360 megawatt, install capacity, medium reactor ACP, 600 megawatt of install capacity. And here you see, non-nuclear as well. They have a three-level approach, including high-level objectives, evaluation area, and indicators. And here you see, the result for Armenia, they have the results, where this analysis was done under this assumption that the cost has the preferences, has the weight 0.5, performance 0.3, and acceptability 0.2. And they analyzed the ranking of the results. They found that ACP is the best alternative here, formed by the Kandu 6, and very close to the Vivian 1000. This is the result. This is the case study before by Romania. They consider the comparison of the INS, enhanced nuclear system, including PHVR, with a slightly enrichment fuel, INS, lead-fast reactors, and they can do, normal can do, these three reactor types, so-called enhanced nuclear system, innovation nuclear system, and existing simply nuclear system. And they have results, they conducted this result under different assumption on their weight, for example, cost 50%, performance 30%, accessibility 20%, then they changed somehow these preferences, and see the result. And the interesting thing is that the INS shows the best score for all these assumptions. This is some of the results. Here you see the case study from Russia. The participants considered the thermal reactors of three types, but this is practically the same reactors, but different levelized unit cost in the back end, and two types of fast reactors, and their combination. And they considered also the different weight. In the option one, they gave the preference to the waste, in option two, the preference to the economics. And in the first case, when we have the preference to the waste, indicators, the innovation systems based on fast reactors has the first scores, but in the second case, when we have some preference on economics, the answer of the cycle based on the existing thermal reactor shows the best score. From Thailand. Thailand is a newcomer country, and they consider only two options. They consider the coal power plants and the nuclear system. And in the base case, they obtained the results that the CPP has the best score, but during the sensitivity analysis, they found for the weighting accessibility, they found the range where this can be viable, can be useful, or can be implemented. So, some exercises on comparative evaluation of the nuclear system. This is part two of our exercise. If you remember, in the first part, I showed you the calculation of light-water reactor and asked to calculate some indicators regarding the advanced or evolutionary... Advanced, yes, evolutionary, this is a light-water... Advanced light-water reactors and innovative fast reactors. So, let's imagine for this type of nuclear system, we will do some comparison between them for this three systems. And let's consider, for example, this is only just an example, some indicators. For example, economics, this management, environmental, some country-specific, maybe any, a maturity of technology. And assign some scores to these indicators. It's very valiant, not exactly calculation, but let's imagine that we have the existing nuclear system, advanced nuclear system and INS. We can suppose that the lower cost will be for the existing. A little bit more for advanced and more for the INS. This is logical, in principle. For waste management, maybe we need to identify the target goal. We need to minimize this indicator, the target to minimize. So, for waste management, if, for example, we mean the light-water reactors, advanced light-water reactors and fast reactors, the waste management, the specific red-based inventory will be more for the INS, for the existing, less for the advanced and the lowest for the INS. So, the best, if we like to minimize, we can assign 9, 1, and 6. This is also logical, for environmental. The amount of useful energy produced, ah, this is, okay, I saw, this is, we need to maximize. For example, because if we have the more energy from the less of uranium, it's good. This should be maximized. So, we can assign the 2, 9, and 4. This is also logical. Country-specific, it does not matter. We put something. And maturity of technology. We need to minimize, but the best one, the most maturity is, of course, existing, then goes advanced and then only INS. So, this is some explanation for this number. More or less logical. And then let assign for them all this, the equal weight. We have 5, 1, 2, 3, 4, 5. So, 0.2 weight for each indicator. What should we do? We need to transfer, excuse me, please, to transfer this table according to this goal, to the single utility function. And depending of the goal, to minimize the indicator, as a target, or to maximize, we can use the increasing value function or decreasing value function. Because in our final, in our final single utility function, 0 means worst, 1 means best in all cases. So, we use decreasing or increasing. Here you see the explanation of this fact. Then, after we have this table, we need to multiply it indicators to each weight and summarize. And you see here the result. This is the result. And this is the ranking under this consumption. This is the example. But what is the exercise? Let's use the same table, performance table. But let's provide some sensitivity analysis. Let's consider economics with the biggest weight. For example, let's assign this weight to the economics. And then we have uniform weight between other indicators. What will be the result in this case? And this is my question. And I expected your answer. So, thank you very much.