 Welcome to module two, video two in our guide to applying a multi-criteria decision-making framework. And as you said, a very practical guide we are undertaking here. And in this video, we really want to look at the really key issue when we are using AHP because you need to get it right from the beginning and that's really about how you structure your problem. So what we're going to do in this video is consider the process of structuring our multi-criteria decision-making problem. And we're going to do this using the example of the work in New Zealand that we discussed in an earlier framework and which much of the reading considers. So in particular, we're going to look at how we chose the domains and sub-domains, really identify what we mean by sub-domains and domains of course as well. And what needs to be considered when you select them. So again, just to go back a little bit, let's think about what we were trying to do in New Zealand as this drives our choice of criteria, what we call domains and sub-criteria, what we call sub-domains. And again, that's really a key issue. What is it you're trying to do if that then really will determine what criteria are important to you. Remember one example we were highlighting was how close or how well the characteristics of the next generation systems fitted with the land manager motivations and perception. So that was one of the ways we could use our tool, wasn't it? So we needed to think about what is important to land managers and then think about how the characteristics of the next generation systems fitted this because within our overall framework that we were looking at here, our idea about whether or not a new system will be adopted will depend upon how well this new system compared to the old system here basically enables farmers to react to external incentives, price signals, policy instruments, et cetera and the sense of how well it fits with what they want, what their land can do, what's the capability of their land. And again, and then if the characteristics fit very closely with this and match their land, then we're likely to lead to adoption as well. So this was a key driver of what we were looking at in our analysis. And if we go forward a little bit, we show didn't we that then we can identify these next generation systems. And in our example, we use the example of dairy sheep. We then can identify the criteria that are important, the environmental, financial, et cetera. We can see how well our dairy sheep are forming against that and then we can wait those pence, how important farmers found or land managers found those individual criteria to get a waiting for it and see how well it fitted. And so we did this, didn't we? As we said, with two dairy sheep, potential people considering dairy sheep with two land managers, and we were able to score the system that way. Now that's one way we can use it, which is really about waiting alternative systems and seeing how well they fit with the weights generated by the land manager really. Or we can use it a different way. And this is probably what more I'm gonna talk about in this video is we can just use it to just consider well, what is important to that managers? How much weight do they put on these different criteria? And because that gives an indication of, again, what's important to them, what factors they're considering and that can allow us to think about a whole range of issues like we talked about at the end uses of this model. And so we're gonna talk more really about this level, not so much about matching it and comparing different systems in this. So we're really just thinking about using it to help us consider what's important to the land manager. So we talked about this idea of domains and subdomain. So I think it's good for us to go back into some more detail. We just touched on this briefly in our earlier video. So first of all, what are domains and subdomains and what's the difference between them and then really why do we use them? So what we call domains, they're really what high level overarching categories that may be important in decision making. So if you're deciding about a new land use, you will be considering its overall financial performance. You will be considering what it does in terms of environmental performance. You may be thinking about markets, about what knowledge is available. These are high level criteria that are important to you. And we call those the overarching domains. Now clearly within those domains, there are a whole range of other factors that may be important. And these are what we call the subdomains or others may call subcriteria. So simple example, within the financial domain, you may be concerned particularly about the return on investment or the payback period or the profitability per hectare. So they are a range of criteria, subcriteria that fits within that higher domain, the financial domain. So why do we kind of break it down into this domain and subdomain level? First of all, by structuring the problem this way, we keep it manageable. So we can do analysis at the higher level and then go down into each of those domains into more detail, the subdomains into more detail. As it said, it helps us structure the problem in a sort of structured logical way. It also allows us to delve deeper into decision making process. If we stay at the higher up domain, the financial, okay, we can say, yes, from that, we know financial is important, but we don't know what aspect of financial is really important to them. Is it the amount of capital they have to spend or is it the return on this investment they get? Because again, that could have important implications for decision making. A particular factor may perform overall well financially. So you say, well, they should choose it, but actually within those subcriteria, there may be something key that doesn't fit what the land manager is looking for. So the overall thing is, okay, so financial is important, but what aspect of financial is driving it. So the key thing then is, because this is so fundamental to structuring the problem, because this is what you're going to get from your respondent, your interviewee, the pair-wise comparisons are going to be between these domains and these criteria in the subdomains too, then it's really important to be able to identify what are the right ones for your particular problem. So how do we determine? So clearly we're not starting ever from new. Basically, there's been a lot of literature on lots of decision making processes. So I think we go back to the literature and see if people in our example are thinking about changing land use, what factors are important to them. But then there's also value in brainstorming. You can do this as a bigger group or maybe even just a couple of you within our analysis. I spent, we locked ourselves in a room with a colleague and spent an hour or two just trying to work out which were the important criteria in this. And but obviously more importantly, well, not more importantly, but also importantly is discussing with the stakeholders in a sense of the people you're going to speak to or people that have an interest in this, again, because they will have insight. So through this process, you can begin to identify the criteria that are likely to be important in the decision making context that you're looking at. And that's exactly what we did. Now, a question is, well, how many criteria or domains and sub-domains do you need? And of course, first of all, this will depend on what you're looking at. And again, but there's a few things we should consider. How many domains will we use? So how many of the high level domains? In our example, we use six. How many alternatives will be in each domain? And again, there's a challenge, a practicals or pragmatic situation here. Basically, you need enough of these indicators, these criteria to be comprehensive, but you can capture the key elements of the decision making process that's going on. But you also need few enough to make it manageable in an applied situation. Because the more the criteria you put into your analysis, the more pair-wise comparisons they have to make under a HP. So basically, you are then putting more pressure on your respondent, put more time pressure on them, and maybe it's hard for them to concentrate, focused all that time. So there's a trade-off between being comprehensive, capturing the essence of that decision-making, but at the same time not having so many that actually it's virtually impossible to do the analysis. And one of the questions you have to think about is a pragmatic question is, how long are you likely to have with the person you're interviewing? As I said, roughly we had six domains and six sub-criteria in each domain, okay? So it's such six in each sub-domain. So basically, or six sub-domains. So that, when we did this, this took anywhere between one and three hours to complete depending in the sense how chatty, how engaged the respondents were. So basically the more domains and the more alternatives within it, then the longer it takes and the greater the challenge to keep them engaged. But at the same time, the more domains, the more you're likely to capture that breadth complexity of the decision-making process. Now we should emphasize that we use domains and then went down to the sub-domain level. AHP is very flexible and it's perfectly possible just to keep it depending on what you're trying to do at that high level. Is it financial that's driving these people or is it environment? Because that gives us insights that can help us with choice of land management or expanding land like in the Sentinel project, okay? In that project, time and resources were limited. The researchers were generally new to using AHP and therefore they kept it at this higher level. And within that, you can simply document when they're talking about financial or environment, what are the actual sub-domains or sub-criteria they're considering? So they could be talking about financial, they could be talking about profit per hectare. And you know that that's an important driver. You don't necessarily have to quantify it, but it gives you insights. So that has advantages, doesn't it? This idea of staying at the higher level. It's simpler, much less time consuming and you don't need to think, classify the factors quite so much ahead of the interview. You don't have to specify what those sub-domains are, you allow them to do that as they discuss. The disadvantage is that maybe it gives us less understanding of the trade-offs and a little bit less understanding of the decision-making. Well, what really is the crucial aspect of this decision-making process? But maybe it's a good first step in that process of talking about those high level domains and what's important within them to identify sub-domains that could be used in future analysis or approach. So if we go back to the example that we use, as I said, we came up with six domains. And then within those six domains, we had on average mainly most times six sub-domains. And we identify this a lot through the literature, through discussion, et cetera. So our six domains are financial performance, market, knowledge, regulations, social and environment. And then for example, in environment, we found that in New Zealand context, the nitrogen leaching, quality of water, phosphate losses, diseases, greenhouse gas emissions and broader environmental stewardship were important factors that farmers or land managers took in. So these became domains and sub-domains. Domain, environment, overall criteria, environment, sub-domain, sub-criteria, nitrogen leaching, water quality, et cetera. And we can structure it this way. So the blue up here are high level domains and then under that are sub-domains within that process. So how do we sort of structure that problem? What does it mean in terms of how we do things, okay? So practice, you know, AHPs in our analysis was conducted within three stages. So fundamentally begin with, we worked at that domain level comparing financial against environment, financial against market, financial against social, et cetera, et cetera, environment against social, environment against financial, and down, down, down through our analysis. So basically we just did that at that high level and that generated overall weight. So that gives us an overall weight of how important environmental factors, financial factors as well. So for example, environmental factors may come out at 0.2 or 20% weight in that situation. Then what we do is apply within the same analysis within each of the criteria. So within financial, we would then go through and compare how important is the level of capital investment required against the return on investment? How important, for example, is the return of investment against the payback period? And again, you will generate weights to highlight that. So then within that overall domain, you know how much weight they're giving to those particular sub criteria, subdomains, okay? And then the final stage is to weight the criteria at those subdomain levels you've got by the overall weight that you've given to financial. So let's, I mean, sometimes it's a little hard to follow. Let's just be specific here. So if we did the first analysis at the domain and we found that the weight of the financial domain was 0.5 or 50% of the weight given to decision-making went to financial. And then we found that when we went to the subdomain level that the weight given to return on investment was also 0.5, then 50% given to it. This would mean that the overall weight for return on investment in our overall final decision-making would be 0.25. So it's the 0.5 that we've given to the financial multiplied by the 0.5 that we've given for return on investment in that. And that means that weights for individual criteria can go from zero, if no weight's been given to it at all, to one, if it's actually the sole determinant of our analysis. But what it does show us is, you know, within a criteria, the weights are being weighted by how important we view that criteria. So the overall weights that come into return on investment are in part determined by how much weight we've given to financial in this itself. So let's just take a real example again to highlight what happened here. So we went through the process and again, remember these weights will add up to one. I should have summed it here. Sorry, I didn't. So in this case, with this person, we found financial performance had a weight of 0.375, out of one, market factors 0.16, social wellbeing, et cetera. So we see financial performance here. Then when we went into the sub-domains within financial performance, these were the weights that were generated. So for example, payback period was given quite a high weight here, 0.326, right? So then how do we work out the importance of these in the overall decision-making process? We simply weight this figure by the overall weight given to financial. And this comes out, and I've stressed it as a percentage here because it's easier for us to see. So it was 2%, as we see, payback period was higher and about 12%. But once we sum these, it's 37.54, which is the same as 0.3754, rounded to five is the same weight as that. So these sub-criteria will add up to that overall weight that we've given to financial through this process. So just to highlight something of importance to us, is the sub-domains need to be described to respondents in a way that can be sure that they interpret correctly what is being considered. And this is a really key thing. What you think is a criteria, they need to also realize it's the same criteria when they understand it. So if you're talking about quality of life, for example, which is one of our social criteria, what does, you know, make sure that they understand it in the same terms that you mean it, you know, or culture or other factors it, yeah? And again, we also need to make sure, you know, that we understand the way they've interpreted them when they are feeding back to you. And it might be, depending on how you're using it, it might not be, you know, it might be more important that you understand how they've interpreted it and what they're talking about when you write it up for them, okay? But if we're trying to maintain consistency across a group for comparison, then we need to make sure that they've interpreted it the same, yeah? And so the other thing we need to do is a careful consideration to what's within each domain and whether respondents can make meaningful comparison between the options, you know? They've got to be able to trade off. So they need to be able to make reasonable comparison within the financial or the environment. And we can think about this again in terms of consistency levels, our consistency ratios that we found. So for example, if we look at environmental here, we found perhaps the highest levels of consistency in the decisions that were made within that domain. Farmers were consistently able to indicate that nitrogen leaching, for example, was more important than phosphate or phosphate was more important than greenhouse gas emissions, et cetera. And that's because the criteria were quite tightly defined. You know, it was nitrogen, phosphate, et cetera. And farmers were able, land managers were able to sort of trade that off. If we look at the market one in, as a comparison or in contrast, we see the levels of consistency we made there were slightly lower. And I think that's because they were trying, we had much broader character. It's like security of supply, like overall scale of the market. And I think land managers found it harder to consistently trade off those things because they're quite big and they're a little bit divergent. So all I guess we're trying to make here is what's within those domains is important. And also you've got to think about not just what's important to decision making, but can people logically compare them so that they can in a sense rank them in their minds about how important they are to you. So again, this came back to the point just re-emphasizing it here. Yeah, about consistency and inconsistency. If basically we have inconsistency, it might mean that overall, they aren't able to grasp the concept to trade off. Okay, so again, that's not so much your choice of subdomains and domains is mean about the whole process. Yeah, but what's important to us is this idea that I just mentioned, if the subdomains are poorly constructed, for example, if the factors are too disparate, too different, then it may be difficult for respondents to consistently compare them. And that could be a key challenge to us. The final thing about consistency is that, if there's simply too many comparisons to be made, we've got too many domains and subdomains, basically your respondents are gonna become fatigued, they're just gonna start moving your dial one way or the other just to try and get rid of you so they can get on with whatever they have to do. They're very busy people running their businesses. So again, the second and third ones really matter about, so this one is about the number of domains and subdomains you have, could lead to inconsistency if you have too many. And this one is about how you construct those. So again, it is very important about that process. But what I just want to emphasize again in summary is structuring the problem is crucial to success in applying your MCDM. We need to identify the key elements of the decision-making process you're looking at, what's important for this particular aspect that you're looking at. You need to think about structuring it in terms of what are high level, i.e. financial, what are the lower level, the subdomains, the subcriteria, return on investment, capital requirements, et cetera. And again, we're really trying to be comprehensive but pragmatic that we can't capture every aspect of it. In our next video, we're going to move on from this to say how we took the principles of AHP and developed a tool based in an Excel spreadsheet for New Zealand and for analysis in New Zealand and highlight how it can be actually undertaken. And in that, we will again go over some of these key elements of what you need to be confident on before you get to this stage. Again, there's some reading available based on this video. So basically as a systematic review of the criteria used in AHP, and this can help you think about how criteria are being defined and measured. And again, for our particular case, reading from module one on the framework for prioritizing innovation will give you some insights about the selection of criteria or domains in this. Again, just want to acknowledge that the part of the work on New Zealand was funded through our Land and Water National Science Challenge. And I hope you enjoyed the reading materials and look forward to seeing you in the next video.