 Welcome to the fourth video of module two in this practical guide to applying a multi-criteria decision-making framework. In this video, we're going to be really looking at how we work with stakeholders and maybe think about providing useful feedback for them. In the previous videos in module two, we think about recapping what we've looked at. We've considered about how we went about structuring the MCDM for our New Zealand example and the process by which those domains and sub-domains were selected and what we need to think about when we're doing this. And then in the last video, we reviewed the workings of the Excel spreadsheet and highlighted what's needed to undertake the analysis and how it can be used as part of this ongoing interview process. In this final video, we cover some issues around using the approach with different stakeholders, maybe individuals, groups, companies, versus individual farmers, etc. And think about how the information can be aggregated and presented to stakeholders in a useful way. But the key emphasis I think, you know, something we really need to emphasize is that the basis of this process, which is through interviews or workshops or whatever way we want to do it, is that stakeholder engagement is key because you need them to be brought into the process to get useful information. So right at the beginning, you need to know who are the key stakeholders within this problem that you're trying to address. And really, as fundamentally, you've got to be clear in your own head so you can be clear with them in a way, what are you trying to achieve? Why should they help? And what can be fed back to the, you know, so really, you know, what is it you're trying to achieve? Why should they actually help you? What's in it for them in a way? And what can you get back to them that may be of use to them as well? And also, you need to be very clear, you know, about how the process is going to work in a practical sense. If you're going to interview them, then they need to understand that maybe it involves this completion of the spreadsheet, you know, in the approach that we conducted. And the other thing I think we sort of emphasized a bit in earlier videos is sometimes it can get a little bit abstract. And so I think the more you can use concrete examples of where or what possible land use choices might be, or in the possible trade-offs that might be occur, then the better it is, you know. So when you're working with these stakeholders, keep them engaged by relating it to something, you know, to examples that they can relate to, and don't make it too abstract. Otherwise, that may create some challenges for you. So what we've highlighted, and the good thing about multi-criteria decision making, you know, is that you can work with a range of different types of managers with this approach. So, you know, for example, you could work with individual land managers who have, in a sense, the sole control over their land use decision, or you might work with a group of managers responsible for a particular area of land. So it might be a corporation or a trustees of an Iwi like we worked with here in New Zealand. But then you may also work with a group of land managers who share a common issue. And again, in New Zealand, as we've highlighted before, we worked about, worked with members of an irrigation scheme. So they have a common issue, waters coming to them through the irrigation scheme, and they all have a common interest in what to do with that water. But then, of course, you can also have a sort of different type, which may be working with a group of stakeholders who have an interest in a particular area of land and how it's managed. So they might not be jointly responsible for it, but they may all have an interest in it. So, for example, it could be the land manager themselves, but the government may have some interest, the public, and other stakeholders. So again, in theory, you can go through this process with them to understand their views on what they want to see from that land. So they can be applied through different types of managers and different types of stakeholders, really. One of the things we've done within this, and I do think as a strength of this, is that you can apply MCDM to a group. It's often being used to do this, and we did it as well within our study. And there are a number of ways it could be applied, and two possible ways, for example, you could, in a group setting or a workshop, get each member to individually score each pairwise comparison, say at the domain level, at this high level, with finance against environment, etc. Before facilitating discussion between the group, at which time they then kind of settle on an average figure. So you could just have one workshop. First of all, you ask them for their individual scores, and then you get them to discuss with each other, and they come to their agreed score. But an alternative would be to actually not to bring them together to start with, but to do the process individually with each one, and then aggregate their scores. And MCDM allows us to do that, HAP allows us to do that. And then basically, you could either present the average results back, or more usefully present, you know, each results back to the group, and allow them to discuss and come to a decision. You know, there's some pluses and minuses of these different approaches. The first one, for example, where they're in a group setting when they do it, we always have the problem that maybe strong personalities will sort of dominate. So, you know, there's a need for good facilitation to make sure that everybody has their say, and that there's not just being swayed by the person with the strongest voice. Again, the issue about if you're doing it in a group decision, a discussion group situation, and there's lots of discussion, then the time taken is going to extend considerably when compared to a single manager. But however, you know, if you try the second approach where you do it each individually, of course, the time then is going to be extended as well, because you have to do, you know, an interview with maybe five or six people individually in the group before you bring them together. So this may have some implications for the practicality of using it at group level, or the size of groups that can be applied to. But it's interesting, a lot of interest we've had in our approach has been from consultants and people who are working with relatively, you know, larger groups of land managers, and they feel that this would be a useful tool for facilitating the discussion with them. So it seems that it knows an attraction for working in this group way. And remember we showed this graph in an earlier video, and this is, you know, how we did it in our situation where they each ranked, each individual ranked each domain here, and then this red line here is the average of their outcomes, you know, with their rankings. And we can see, you know, this group were very closely aligned, as we mentioned before, in terms of their high weighting placed on the environment and social wellbeing. This is what the trustees were particularly concerned. And also we can see, you know, as I mentioned, they're quite close alignment between them. So it doesn't seem to be there was going to be a lot of disagreement in this group about what's important for the land that they're jointly managing as trustees. And as we said, this group here actually found it useful to clarify their views. And they then actually took it back to the group that they were trustees for, that they were representing to highlight what was, you know, to take that discussion further into that group. And that whole process actually worked quite well in this case. Again, that, maybe that, our example worked quite well, because they generally agreed with each other. You know, they generally had the same views on what was important for the land that they managed. Now, the question would be, would that still work quite well if there was very wide scale disagreement? No, some people really pushing for the financial side, some people saying, no, we've got to protect the environment, etc. Okay. But still, you feel this could be useful in this, and if nothing else, it brings to light those differences. Okay, and also then the discussion around that may clarify the nature and the extent of the disagreement. So it gives them an opportunity to talk about what it is that's driving these differences. And again, it may encourage discussion around what the issues and could facilitate mediation between different views. Now, you could think about it, could also think about it in terms of group situations. If people were thinking about working together, going through this process could actually highlight whether they have sort of compatible values that could be working together. You know, if once you said if one was very, very strongly concerned about environment and the other one was very strongly concerned about financial, maybe there's going to be a lot of disagreement each time they come to make a decision and perhaps they may be better off not working together. So again, it can give us some insights into these sorts of issues as well. So that was the first part, which really was really about concentrating on the stakeholders, identifying who they are, how you get them to work. But a key point about engaging people and bringing them into this process is how are we going to analyze what they've said, what we found from them, and what sort of feedback we're going to get. Because I know my experience on much of the survey work and other work I've done, not just in this area, is that if you can provide good feedback to people, then they can see further the value of, you know, if you can provide useful interesting feedback, they can see further the value of collaborating with you. So we're just going to concentrate that on in this in the second half of this presentation. So this video here, so let's think about how we might aggregate the results from the individual people that we meet with, what how we might provide feedback if it was to an individual or perhaps to a group. But one thing clearly, in order to provide good feedback, we need to be clear as to what our results mean and how we can interpret them. And we'll come back to that as we talk through this. So at its simple level, we can think about aggregating results. So here we have just the example at the domain level, the higher level for a number of interviewees that we, you know, a number of land managers that we interviewed and went through the process. And we can see here, the various weights that they gave to. One thing we could simply do from this is plot the chart of these individual results to get some feedback about the relative, you know, using a spider chart like this gives us some relative feedback, relative position for each other and can prompt discussion on that. Or you simply could average these depending on what you were interested in and aggregate them into one average score and discuss that again, depending what you're trying to look at. I mean, I'm quite a fan of the spider diagrams because I feel they do point us to where the really important things are, but they can get, you know, a little messy if you've got a large number of respondents. And so there may be other ways you want to present the results as well. Now that was at the domain level, but of course we can do a similar thing at the subdomain level. So for each individual component, right through from our capital investment down here, again, we, you know, we generate these weights from our tool and then we can plot these to say again, this is quite a, you know, a common, but there's a lot going on here, spaghetti diagram as we say, but again, it can begin to give us some insights of patterns that are going, going on in this situation. So in again, we can put all the results from each of our, you know, models into a single spreadsheet, each of our interviews, each of our land managers here, and then begin to play with that data to calculate averages and average results or to highlight them in this way. One thing I just want to touch on here is in a sense, you know, what's the sort of feedback? And I thought again, maybe using our next generation system project and the work we did in New Zealand as sort of an example of this. So we provide, you know, reproduced a template, which we would use to go back to those who helped us with the study. So first of all, you know, this sort of thing here is to have the partner who from our team was there, what they were up to at the moment, what were, you know, what was pushing them to try to change their system, their management system, and then maybe what were they interested in changing to sort of a general overview background about where they are now, what they're looking for. And then we would provide some sort of analysis of their decision-making criteria. So we'll talk about the domain levels and then thinking maybe comparing their results to the average here and then thinking about how that fits across different domains. And then you might go down within domains and see the weights that they've been given, you know, a number of charts here can do that in a bit of commentary trying to interpret what this might mean for them. And so again, you could then give them, you know, a final, the final spider graph for themselves, highlighting, you know, for their final results. So we see for this example, we can see that, you know, important factors for them are a profit per hectare they can get, what return of investment needs have a high weight, but they also want to make sure there's advisory supports available to support them in this change. So we can hear a sort of a clear picture of what's important to this individual when they're considering changing their system. Now, you know, it's kind of interesting thinking about what's important in this way, but you can then begin to say, well, how do they vary in a way from the average from the people that we've looked at. So it may be more interesting to provide some comparative feedback. So here we see the orange lines here are the average for all your respondents. And then this is the individual person. And you can see here that they put much more weight on return on investment than the average. And then you can have a discussion with them about that. Or here we see generally, they follow quite closely the average for social on the environment, they are more concerned with greenhouse gas emissions than the average, you know, and again, so just ways you can present the results, which can prompt some discussion, maybe further discussion, or just some thoughts in the person's mind. Again, if you're working with a group that have common interests, so this, you know, potentially this, this was the example of the people with the Central Plains water and irrigation system, we in a sense presented feedback after we interviewed them individually, we came back and provided feedback in terms of the results for them. And again, you can, you know, have each individual and then you can think maybe about the average as well. And again, it just prompts discussion that they can think about where they sit and how they sit with the others in their group, and why you may have differences. And again, the point we're trying to make earlier about one of the advantages of this process is that you're discussing with them as you go through. So, you know, you probably want to record the interviews, and then you can begin to understand what's determining those choices. And so in this feedback, you know, we basically just put some quotes from them to highlight what, what sorts of things were being said in that, you know, we like to think we're more on the environmental side, but it can't do a return. We can't actually do it because there's no point going broke over it. So highlighting the key, you know, got to make a living. Yeah. Here we highlight social well-being is very important. And again, we can go through. So that was at the domain level, but we can go through and do this at the sub domain level as well. And again, it's all about prompting discussion, reflection, you know, consideration in the group and looking at how, you know, different land managers with a similar issue, how are they going to view this? You know, what one thing we're kind of interested in this thing is, can they see opportunities for working together collectively to maximize returns from the water and reduce the overall environmental footprint of those systems? And so, you know, again, you can discuss, for example, do they have common views on working collaboratively or not those sorts of things may be important. We highlighted a bit before about this, what happens if you have very conflicting reviews about what's important. So again, this was from another example we worked with where free groups were potentially considering working together on a land use issue. And once you've been through the process, again, just highlighting the domain level here, we do see they have quite different profiles in terms of their decision making. And what we can say here, it doesn't necessarily mean they can't work together, but it could be a conversation starter. So you could present these back to them in a group and they could then begin to discuss but what do they mean when they're saying regulation is very important, for example. And does that mean they wouldn't be able to work with someone who's put more weight on financial or actually do they concur more than this looks like. So it's a good way of getting a conversation starter in it. So again, by providing feedback, going through the process, providing them feedback, you can use this feedback in a range of ways. And I think this is an important element, you know, an advantage of this approach is that it does provide us, you know, with some form of quantification, but it obviously needs interpretation and the feedback and prompt that sort of discussion that we want to have. So I just really want to sort of summarize where we've been to, you know, in a sense where we've got to, you know, in this module, we've presented a practical application of MCDM using analytical hierarchical process procedure, AHP. We've highlighted in general how it's been used. We've considered how to structure the problem and mainly about the domains and subdomains. We sort of examined the spreadsheet and the tool that we've been using. We've highlighted that there are other tools obviously available, but we used our tool just to highlight the sort of insight you can get whilst undertaking the process. And we just consider some of the issues are working with stakeholders and generating results and feedback. Again, I just want to acknowledge that the work in New Zealand was funded through the MBs, our land and water national science challenge program, next generation systems. Thank you very much.