 Well, first of all, thank you, everybody, for joining us this afternoon. My name is Alice Muller. I work with World Vision. I'm an evaluator that works with World Vision Australia here in Melbourne, but I'm also very proud to be a member of the AES and working on the Victorian Regional Committee, which gives me the very proud honour of being able to host this seminar for you today, which is one of our monthly seminars that we give as all regional committees do across the AES. So there's lots of interesting content being shared. Thank you to all the AES members that have joined. And if there's any non-members, please check out the website and see what you think. We'd also love to see you all here at the AES National Conference that will be in Melbourne this year in our local hood. The conference is in September and I believe you've got one last week if anyone would like to submit any abstracts or concepts to be able to share your ideas and present some of your experiences at the conference as well. So please check out the AES website if you're interested in finding out a little bit more about that as well. Before I introduce our lovely guests who are going to be speaking with us today, I just wanted to acknowledge the First Nations people that we are First Nations owners of the land that we are meeting on all across the country and potentially internationally as well today. I am coming here from Rajui country here that you can see the beautiful Dandenong ranges where I'm lucky enough to be based. But please share with us also whose country you are coming from today and we'd like to acknowledge and pay our respects to all the traditional custodians of the land across which we are meeting and pay our respects to the ancestors, elders, past and present. And we acknowledge the unique cultural and spiritual relationship to the lands, waters, sea and the contribution that is being made to society and we are grateful. Thank you. So today we have Nick and Mal from Day 4 Consultants who is Day 4 Projects who are here today to share some work that I was actually lucky enough to do with them last year. Nick and Mal are both very well known across the AES. You may have heard from them in the past, but they are experts and facilitators and evaluators all in everything collaborative. And so we've been lucky enough to learn from some of their expertise and their experience in projects that they have done both across Australia and internationally and share with you some of this work today. So on that note, I'd like to hand over to Nick or Mal, whoever is going to be driving for us today. I think I'm first. Yeah, thanks. Excellent. Wonderful. Over to you. Thank you. Hi, everyone. Thanks for being with us this afternoon. If you are a person that likes to have your camera on, go for it. It's always nice to see a face in the crowd. But if you're doing something else, more than happy for you to stay off camera towards the end of the session, we'll have time for some questions and hopefully a bit of interaction. So it'd be great if you could stick around if you have time. As I said, my name is Nick and Mal is here with me today. And you'll hear more from Mal as we go through this session. I'm going to share my screen. So we've got some slides to go through together and for the recording as well. We're going to be talking about an evaluation of partnerships for scaling. And in this context, it's for scaling FMMR. So Farm and Manage Natural Resource programs that well-visioned are running in all different countries. But we'll tell you about that as we go through. We want to really kind of make this presentation around some of the techniques and tools and things we did in the evaluation of these partnerships and how evaluating partnerships can be quite useful for informing scaling efforts, whatever those scaling efforts might look like. So we're from day four projects. And as Al said, we specialize in all things collaboration and we do that kind of front end stuff of building collaborations and figuring out how people might work together, what they might like to do together and what they might like to achieve together. And we also do lots of work in evaluation. So this is this is, I suppose, a project that's close to our our hearts and wheelhouse. And so we've done lots of work in evaluating collaborations over the years. And this was a really fascinating one. So hopefully we can share some insights with you today. Just a brief agenda, really, on what we'll be covering today. Alice is obviously a well-contuned and acknowledged country and I'm drawing you from a country. I'm going to hand over to Alice in a moment and ask her to take us through the purpose and overview of this project. And then it'll be back to me, I think, to talk a little bit about learning and evaluation in the context of partnerships and in the context of this project. And now is going to take us through some of the lessons, insights and conclusions that kind of came from this work. And then we'll have a bit of a discussion with Alice about how World Vision are applying insights and recommendations. It's going to be a presentation about an evaluation that's being used. How exciting all the evaluators online. We've actually we've got some real world application of it, which is great. So I'm going to hand over to Alice now to take us through purpose and view of the project. Thanks so much, Nick. So, yes, just to give you a little bit of background here at World Vision Australia, we have a long history of working in community based natural resource management and land restoration, sustainable agriculture and these kinds of programming approaches. And recently, we have made the commitment that we would love to see a billion hectares of land of degraded land being restored. We can see we can we know that land degradation is one of the major drivers to many of the issues that we are trying to address. Food insecurity, poverty, child malnourishment, lack of income sources, lack of resilience to climate change and its effects. There's there's so many different drivers that can all be traced back to the the condition of the land that people are living in and rely upon for their everyday life. And so for us to be able to address the challenges that we're facing every day and to be able to address the growing need for this in the future as we're facing down climate change and as well, we need to really see this this land restoration scaling up to a massive level well beyond the rate of change that we've been seeing in the past. So we've made the audacious goal that we would like to see a billion hectares of land across the globe being restored. Thanks, Nick. So Nick mentioned our favourite acronym, FMNR. If you can take one thing away from this presentation, it's how to wrap your lips around those four yet those four letters. But we call this farm and managed natural regeneration. And the reason that we think that we can actually get to a billion hectares is through techniques such as this one, which are low cost, scalable, proven in a variety of contexts around the world and able to be spread from farmer to farmer, community member to community member and can work in all different places. So farmer managed natural regeneration is basically the pruning, protection and growth of existing vegetation. So existing trees from stumps or seeds in the landscape to be able to regenerate indigenous forests or trees in agricultural pastoral land in the landscape that we're working in. Thanks, Nick. So the way that we're hoping to do this is not just by doing more and more and more projects, which is our direct programming. Obviously, we will want to see that happening too, but we have recognised that we are going to need to activate a lot of different pathways to be able to reach the scale of the programming that we'd like to see happen. So in addition to doing more direct programming ourselves, we also have identified that we need to activate partners. We need to work with partners for them to also take on practices like this and included in their own programming to replicate the work that they're doing in their in their own footprints also. In addition to that, we want to be able to unleash communities to poor organic spread, supporting community members to be able to spread the practice between themselves and improve the enabling environment, looking at policies and markets and all these drivers that can help people make the changes that they need to change. Thanks, Nick. So where are we starting? So what the activities that we've been doing most intensely over the last year or so is starting to work in these four countries. These are the four countries that we looked at in this program in this research as well. So in each of Uganda, Ethiopia, Kenya and Zambia, they each have a long experience using techniques for land restoration like FMNR, there's proving capacity, there's strong government support for restoration, but we also have a strong driver or a strong need in terms of this food security, child wasting is an issue, drought and climate vulnerability are huge issues in these countries also. So we have developed national scaling business plans in each of these countries that calling us to work with partners more and to looking at different ways of engaging with partner networks and different collaborations in the country to support the spread of FMNR to these audacious goals that you can see here on the screen. I've also mentioned in each of these countries, some of those partnerships that we've had experience working with in the past that we wanted to look into more and to ask ourselves, how can we work with these particular partnerships? How can we make them stronger? How can we help them to help us scale FMNR out to the goals that we need them to? Thanks, Nick. So as we were planning these strategies and starting to look at how we work with these partnerships, these are some of the questions that we kept coming up in our head. So how do we how do we work with so many organizations? What do we need to do? What models do we need to use? What structures do we need to use? What should world vision be adding to these partnerships that sometimes exist beyond us and without us as well? What do we need to make system change? What contributions to partnerships need to add to that? And what affects the sustainability of these different collaborations and partnerships? Do they need to last for a long time? And if they do, how do we support them to to adapt and evolve over time? And finally, how do we show the value and the impact of these sorts of these sorts of collaborations so that we can get the resources and the support we need to be able to pass it on to them? We know that it's it's sometimes easier to secure resources in grants and program designs for the activities that we do on the ground. And sometimes things like supporting collaborations and supporting people to come together to talk to meet to build ideas is sometimes not the kind of the most sexy things that donors like to see. So we need to be able to build the evidence to justify these are really important. And these are the changes that we can expect to see as a result of this work. And so to be able to do that, we wanted to know how do we monitor and evaluate these collaborations and how do we know that they're working and how can we show the benefits that we feel are coming from them? So at that point, as we were riddled with these questions, keeping us awake at night, we said, we need to find someone who can help us. So this is where we handed them over to Nick and Mal to take us through some research to help with answer some of those questions. Thanks, Alice. I like that. I was really seamless. That was like that's the hand I was really good. I liked that. So this is where we came in the questions. That we did lots of work on this project and we and we looked at things that I suppose really quite quite deeply located in place and people's experiences. And we'll try and share some of that with you today. But there's not there's not probably enough time to go into depth on all of it. So please do get in contact if you want to know more about the projects more specifically. But I thought what would be interesting for us to kind of start the conversation on today is really a bit about the approach we took, I suppose, and some of the things that we learned about how you might begin to conduct learning and evaluation in these kinds of partnerships. And so we started with a kind of rapid review of current practice and literature in the world of partnerships and collaboration. I would argue that the practice probably leads the literature. And so it's a it's a review of what people are doing and how they are approaching their learning and evaluation within those partnerships as much as it is about what's codified and written down and known about partnerships as well. So it kind of drew from all of those contemporary sources and linked to the questions that Alice had. We sort of started with these three on the left hand side that are related to this rapid review. So looking at what what are the critical success factors for building effective partnerships, specifically to scale initiatives such as FMNR. So if you see kind of Alice was talking about what FMNR is, but these these partnerships themselves don't do FMNR. They do a whole bunch of things that enable FMNR to happen. So how do you scale a partnership type approach to scale FMNR? Does that make sense? That's the first one. The second was related to that kind of question that Alice left us with, which is around how do you monitor and evaluate these kinds of things? So what are some of the indicators that could be useful to measure effectiveness, but also to measure the the role of partnerships in scaling initiatives or in initiatives that seek to scale something, in this case, FMNR. And then another question around why do people even work in partnerships in the first place? So what's the value of working in partnerships and how can this be demonstrated? So what are some of the things we'd be looking for in terms of value add within the partnership world? So we found lots of things and you will know different ways of codifying, classifying partnerships as well. But essentially, we we we found this list, I guess, that the ways that people think about collaboration and partnership differ and they have they're not all of these things are always in the way that people do that, but they do take into account of these things. So sometimes it's about their purposes might be might be different to one another in partnerships. The functions might be different. The members are usually different, even the types of members and the factions that they might work with. The ways that they make decisions and who's involved in those decision making processes differ. So you've seen before Alice named some of the different partnerships that are on the screen anyway. And so there's networks and alliances and multi-stakeholder partnerships. Those kinds of things have different decision making processes. They also have implications for the formality or informality of the relationships between people. So this is this is a network. We're in the Victorian chapter of the AES network. You've come along. Some of you are members. Some of you are not. It's really informal. I'm talking and no one else is. So it's kind of like you don't have to participate in a communicative way to be here. So it's all that kind of stuff is these are the features of partnerships that we're talking about. The intensity of interaction and how you can choose some of that as well. So do you show up to a meeting? Do you show up to a workshop? Do you actually write the thing together and work on the project together? So all sorts of all sorts of ways of classifying and also thinking about the length of time that the partnership is seeking to exist. So one of the one of the partnerships in this project, for example, was was set up really around an event. It was around a conference seminar, bringing together people to speak and be convened at an event. It had a life after that, but it wasn't set up for that initially. And lots of partnerships are set up with different lengths of time in mind for how long they might exist. So lots of things to kind of consider when you're looking at what makes up a partnership. And these four that we had with both vision were yeah, we're all really different and they were different under all of these things. So this is I suppose the first bit of this slide is lots of words on there, but I thought it was useful to label on for the recording as well. Often in the world of partnerships, you should think about things like critical success factors, key ingredients, people have different words for it. But it's essentially like what are the foundational elements that you would you would expect to see in a successful or effective partnership. There are different versions of these lists depending on which source you go to. This is an exhaustive list. There might be some nuance that you would add to this. But generally, these elements are things that you would expect to see in all kinds of partnerships from the from the very kind of fluid networks and alliances and meeting types through to the very structured kind of formal partnerships that exist as well. And so if you have these elements in place, you it's a good signal that the partnership is probably pretty healthy. If they're not if they're not doing really well, it might mean that there's something to explore with the partnership. So the absence of it doesn't mean it's a bad partnership. It just means that there's something to explore So actually articulated goals, transparency and decision making, mutual respect between partners, that people and partners understand their roles and their relationships, that there's mutual trust. That's obviously a difficult one to get to. But sometimes on the journey there, you see mutual trust emerge between partnership partners. So willingness to work together. We kind of get lumped into working in partnership sometimes and values that underpin those work well and then that commitment. So there's a willingness and then a commitment to kind of show up to to behaving against those values. And importantly as evaluators that there's there's a sense of agreement around what will indicate that the partnership is beginning to demonstrate some of its results or accrue some sort of success, whatever we put whatever language put around it. But it's the agreement on the indicators that's really important in the context of partnerships that we all agree on what the signals might be. So in this project FMNR similar initiatives focus on and you see they're regaining the mind of community. There's a kind of community level partnerships. They have an intention to kind of increase or improve empowerment at a community level. Not all of the partnerships had were only focused on community members but all of them had inroads into communities that they're working in. So there's something about the either the participants in the partnership but also the people they were hoping to reach. So changing minds and changing minds and hearts so that individual beliefs and attitudes were shifting because of this partnership. So that was something that we saw in all of these that there was something about making sure there was inclusive community involvement and commitment. So that's related to those roles and relationships being articulated. So being really clear about who's involved how they'll be involved where they'll be able to make decisions and these partnerships that we reviewed with Vision also had an intention to improve policy support. So whether that be about creating kind of enabling environments for discussions or interactions with people who make policy or whether that be around kind of the formation of agreed things that go to policymakers or advocacy elements of it but improving kind of policy support for FMNR. And then the fourth one and I think it's probably goes to us as lost the ambitions of the billion hectares is that there's got to be some sense of ambition that this thing can change that there's a building of sense of hope and self determination to be led by the communities that are doing this work. We looked at kind of how performance indicators within partnerships get are arrived at and how they get kind of surfaced and and agreed between different types of partnerships. So excuse me. And when you when you are trying to evaluate really complex partnerships like these one, I think all partnerships are pretty complex. There's there's some challenges that might arise. So and I'll try and make these I try not race through these because I think they're all really important. The first one is the one that we see the most in practice but also in the literature. There's this idea that you will you will look at the the outcomes of the partnership rather than sorry that you look at the activities and outputs and structures rather than the outcomes. So an example would be that you'll be you'll be looking at how how much how much land has been regrained as an outcome rather than looking at some of those elements of what is the partnership doing? How are people working together? What are some of the outputs that are being created through this partnership way of working? What are the structures that exist between partners? It's less probably like fun to talk about, but it's a really important element of of evaluating partnerships to look at those activities out of these structures. As Alice said before, there's something to do with the relationship between who's funding a partnership and what they're expecting to see and then the different pathways by which partnerships actually influence outcomes. So, you know, tell me how much land has been regrained rather than tell me a bit about who's going to be involved in this partnership? How often are they meeting? Are you growing the base of it? Is it kind of growing a different pocket? So this perhaps we would argue more interesting things to take into account when looking at whether those partnerships can achieve their outcomes. We also found there's no standardized measures, but there's lots of standardized practices. So you'll see, you will have seen things like collaborative health assessments, partnership health checks, the scorecard surveys, and they're all really good. They've all been developed for particular partnerships in particular contexts, but they are really good, nonetheless. Unfortunately or fortunately, I'm not sure that there's no, there probably is no need for a standardized measure about partnerships because they're all so different. But there are some, there are some different things you might look at when you're, there are some standardized ways, I suppose, of accessing and measuring partnerships. We see this interest in most. I'm so all the evaluators online, you would notice this kind of interest in obsession with looking at linear cause and effect to, you know, in sort of a programmatic way of thinking about work. So we did, you know, we put in X inputs and we got Y outcomes and it's sort of, you know, in the world of scaling that also is complex, I think, but in partnerships, the ways that we get to those outcomes, the ways they accrue are not linear. They're often about contributory pathways that partnerships kind of unlock and they might just unlock minds and hearts of people as well. And then the last thing I suppose we've heard is that indicators are still relevant and valuable. So, and I mentioned that on the last slide, that when we agree on what will signal changes happening, they can be really helpful anchors for partners to have conversations about what they're doing together so they can be a source of kind of, yeah, igniting that conversation, keeping it going. So some, I suppose, current literature practice about the world of evaluating partnerships. I'm going to hand over to Mal to take us through how we kind of took all of that and then applied it to the world's work. Over to you, Mal. Thanks, Nick. Hi, folks. And thank you for the very generous introduction earlier, Alice. As Nick said, I'm going to give you a high level overview of the evaluation. And I am joining from my own country today. So the methodology that we chose to apply, I'll give a bit of an overview of that and some of the the findings, the enabling factors, some of which might feel familiar to some of those who do evaluate partnerships. And I'll also talk through some of the limitations of the evaluation as well. So for this evaluation we acquired a multiple case study approach, we chose a multiple case study approach because it enabled us to really understand the four individual partnerships, as Alice mentioned, that were set in for very different country contexts. So we were able to look at those insights from those four individual case studies and then we compared and contrasted them via a cross case study and through which we were able to identify some of those high level patterns and themes across the four case studies. So as you can see on that slide, we designed four main areas of data collection in the evaluation. So the document review where we reviewed supplied materials from each of the partnerships and we extracted the data from those documents into a data extraction template, which we structured according to the research questions, the rapid literature review, which Nick mentioned and we, as he said, reviewed literature from leading partnership practice agencies and then the four within case interactive workshops. So we facilitated four online workshops with members from each of the collaborations and partnerships and we presented the results from the document review back to the partners both to a sense check and to test those results in that work. And we also used those workshops as another data collection point. So in those workshops, we were able to explore members' experiences, their perspectives of the collaborations, successes and failures, like nature of the partnership relationships and the partnership activities. So these workshops were really participatory, the breakout rooms, group discussions and I think this data collection point was particularly important because for us as evaluators, it brought together a more fulsome understanding of the structures and functions and natures of these collaborations, but it also brought to the fore a really broad range of perspectives and experiences from the different and various individuals and organisations who were members of the partnerships which range from grassroots community groups to international NGOs. We then had a cross case workshop. So this is where we were presented the preliminary summary of some of those finding insights. We had 26 participants across section from the four different partnerships and participants had the opportunity to examine those key insights, those high level themes and to challenge some of the assumptions also to further distill and describe the partnership. And I think also consider future options for monitoring and evaluation. What was really interesting was it was a really important point for partnerships to be able to connect with people from other collaborations doing similar work who they hadn't met before. So, yeah, the cross case workshop and the subsequent analysis we were able to identify those high level patterns. And, yeah, we looked for lessons to inform future scaling efforts. So these are some of those key findings from that cross case analysis. They're high level findings and they're specific to this project, but it's not surprising that a number of them are probably commonly found in evaluations of partnerships for scaling. So I'll just race through them, but all partnerships we found that all partnerships had a united and consistent voice and that partners really had a genuine voice at the table and their contribution was really respected. There was a whole broad range of ways that they used this united voice to influence change, but a specific area of note was they used this voice to engage government at both state and national levels. The way government, the way in which government fit into the structure of the collaboration did influence how advocacy was done. So if they remember of the collaboration or not and what the effects of the advocacy were, but we did find that all partnerships were increasing strategic support for their work. One size doesn't fit all. Different partnerships approaches will be suitable for some contexts more than others. The partnership needs to be fit for purpose. Therefore, the structures and processes for each partnership, they're going to vary because they serve the purpose. So funding matters. Collaborations require resourcing and the activities of partnerships require funding. We know it's not the only ingredient for collaboration, but the ambitions of a lot of these partnerships were long term and big ambitions. And so we found that they need sustained funding in order to assist these activities and to realise these ambitions. So some enabling factors. So some of those enabling factors that create an enabling environment for those partnerships. There was an agreement to a shared vision, ambition, common purpose. So whether that was to deliver specifically on FMNR goals or border land restoration or climate goals, the key point is that all partners had a shared ambition in their partnership. They had a real sense of hopefulness to realise that ambition and a clear understanding of that shared purpose. So that was a real commonality. The strategies and approaches to delivering upon the shared ambition, the common purpose, they were very different from each partnership and how collaborations thought to create that change and scale ranged all the way from organic national movement building to more traditional development approaches and inroads. So unsurprisingly, some of the pathways for scaling really varied and the activities of the partnership looked quite different from case study to case study. All case studies had strong thematic expertise within the partnerships. So really strong thematic understanding of FMNR, land restoration, nature restoration, climate. And these were redeemed as really important factors for developing that strong thematic evidence base within the different country contexts. Partnership structures and processes and governance varied from formal to very clearly defined. Some of the partnership had documented partnership agreements, memorandums of understanding, partnership time frames. Others were more informal, more organic and it had been a documentation. Some of the limitations of the evaluation changes to personnel and to staffing. So we found that getting those perspectives and experience within some of the partnerships from people who were involved in the genesis or early development of the partnership was tricky. Consistency and depth of documentation did vary considerably. As I said, some partnerships had very formal field reports, agreements, MOUs and others had more external communication based products and been a documentation in terms of some of that formal documentation. Facilitating some of those workshops across oceans and continents definitely has its challenges when it comes to things like scheduling, negotiating time zones, different working cadences and the big one technological connectivity. Like despite those challenges, the workshops were extraordinarily well attended. There was a mix of partners and voices in our virtual Zoom rooms and some of those voices and folks in the room were funders or hosts of the partnerships. So their actors were funding the structures and activities and so there were times where participants foregrounded some of those perspectives so that was a dynamic to note. So that's my high level overview of the approach. With that content in mind, I'm gonna hand it back to Nick. He might have some additional thoughts to add to any of that but also is gonna take us through some considerations for evaluating partnerships for scaling. Thanks, Mal. Yeah, I think we were just having a chat before the school and thinking about negotiate. When you're trying to do things like this across many different times zones with many different actors which is the nature of these partnerships kind of thinking about how you engage people well, I think as, yeah, I'm glad we shared some of those limitations and maybe I would call them considerations as well kind of things that things to be aware of in this world of evaluating partnerships, you know, there's lots of, there's something in the evaluation of partnerships that you can kind of, you can either improve or dismiss kind of the ways and structures that people have are working together as well. You know, there's lots of power dynamics between partnerships. So some considerations for evaluating partnerships for scaling. I mean, I don't know if that's a very well written sentence, but I definitely wrote it. But some considerations I think when you're evaluating partnerships where the purpose of the evaluating them is to figure out how they might scale. So when you're looking for questions of scaling within partnerships, what are some things that you might want to consider? These are the things that we found in this project but they are also applicable I think to any partnership that's intending to scale. So there's something about investing in consistent evaluation, monitoring, learning, whatever we want to call it. As Mal said, because of the formal informality, formal to informal nature of these partnerships, the ways that they were documenting what they were doing differed. And that doesn't matter except that when you're trying to compare them. So when you're trying to look for comparisons across or when you're trying to kind of look back at what worked and what didn't and under what conditions, some partnerships had more information than others is essentially it. So if you want to know what's working, document it. I suppose we would all say that as some evaluators anyway, that's the first one. The second I think is around clear and appropriate scaling strategies. And this is important in the context of scaling. So you have to clarify what it is that is being scaled. In this project, there was a mix of, okay, we're scaling FMNR practices, we're scaling knowledge about FMNR, we're scaling broader regeneration and regreening processes or we're scaling how to build greater networks. They weren't mutually exclusive but there were multiple things that folks were trying to scale and scale meant different things to different people as well. So that's important. And that's common in every project I think that intends to scale. There was an important finding for us in this one which I think was useful to share in this context. So the size of the network or the size of the partnership does not necessarily deliver in this case FMNR at greater scale. So having more people or more members within a partnership does not make FMNR just automatically scale. I know that it's like kind of breaking some of that linear assumptions about cause and effect, I guess. Facilitating knowledge exchanges was something that we didn't expect to happen but there was actually really, was a really fascinating kind of thing that came out of this project. As Alice mentioned, they chose four at World Vision for us to focus on to start with but there's many more. And one thing we found through just applying this multiple case study approach is that probably the bit that people seemed to like the most was meeting all of the other people and learning about what they were doing. So through the evaluation, they were able to exchange knowledge about what was working in their context and that seemed a really important thing. And I think has had implications how these folks work together. So something to consider I guess. Number four is linked with number one, collect that useful information to guide scaling efforts. So you might need different information. Excuse me everyone. You might need different information. You might need to collect it at different points but if you're trying to understand how this practice can be scaled in this environment then concepts around contextual factors, resource availability constraints, how the partnership is structured, how relationships are facilitated, all that stuff. It's kind of important to understand in order to apply it to scaling efforts, and finally, convening and networking and collaborating. So yeah, it's linked to that number three. There's something around a space that a partnership occupies and we're doing it tonight. There's a group of people that are naturally coming together through partnerships and networks anyway. So find other ways to bring information to them. In this case, convening international FNR technical experts and speakers to share experiences. I mean, you got loved with us but you could have had an international FNR technical expert. But I think that the point remains that so use the convening spaces that are being naturally formed or formally formed and use that as a site to bring other information in. Also I'll hand over to you for how this is being applied in the World Vision context. Thanks Nick. Yeah, this was a really interesting study for us. It was a little bit different to some of the more recent studies that we've done and it was a great journey for our national office teams to really see that we're really interested in these mechanisms and not just the outcomes and the outputs and the impacts that we're achieving but also really trying to understand the mechanisms and how we're achieving it. So for that purpose, that was a really great experience for both ourselves here in World Vision Australia but also our national office teams in each of those four countries that worked with Nick and Mal and the team on this project. There were sort of three main takeaways for me about how we're applying the results of these projects. The first is that we've taken some of the actions about how we can best support these partnerships directly into our project plans and our national scaling plans in those four countries. So in there, we've been able to allocate some resources to some of the funding for some of the core functions of those partnerships as well as looking at some of the different needs that were identified whether that be like communications materials or additional training or additional monitoring and evaluation support, for example, these are all practical things that we've been able to plan for. On monitoring and evaluation, it's also helped us in how we design, how we go about tracking and learning about our scaling efforts. So for example, we have rhythms of regular reviews where we're going to bring people from each of those four countries together to reflect on how we're going about scaling, what approaches are working, what are not and during that we have the opportunity to kind of reflect on some of these different frameworks about partnerships. How are they being set up? What's the goal for them? How are those structures and processes contributing or inhibiting achieving that goal? And then what are some of the intangible things that are benefiting from those or coming out of those partnerships, those relationships and capacities, collaborations beyond the partnerships, for example. So we've all been able to embed some of the recommendations into that, those monitoring and evaluation plans we have for our own program. And thirdly, being able to talk about this work, being able to have some research and some evidence coming from the partners themselves, not just our own staff, but the partners that are working with us or at times independent of us, working together to hear and have the evidence from them in their own voice about why this is important and what working together in these arrangements has enabled them to do and what barriers they're still facing that working in other forms of collaborations may help them with has been really great for us to be able to continue to advocate and leverage additional resourcing and additional support to be able to lend to those scaling efforts and those collaborations. So as an example, this work is being also presented in conferences with lots of other partners who have come together in Zambia this week to talk about how they can accelerate nature-based solutions spread and this work is one of the topics of conversation in there about how can we broker more organisations working together on those common goals and what do they need to be able to do that and how do we know it's working? How can we set that up? So it's been really valuable in lots of ways, but that's just a bit of a taste of some of the different ways that we're using it at the moment. Thanks, Nick. Isn't it nice? Well, I like it as an evaluator, as an external evaluator that it's like, oh, so you're using the results of the thing we did. I love it. That's fantastic. It's not always easy to use these things. I think what we did find from this work was that people are really keen to understand it more, both within World Vision, but also within the partnership. So that's really exciting. It's great to hear. I have some questions as prompts for us up on the slide. You don't have to answer my question, so if you would just like to ask one of us, but that is the end of the presentation. We thought this would be interesting to discuss, but you might have questions. So I'll stop talking and see if anyone does have questions. Thanks very much, Nick. So feel free to put any comments you might have in the chat or if you're feeling brave, feel free to unmute and just yell out and we can keep it as a face-to-face conversation. And you can ask any other question you may have as well. Those are lovely questions. Just come through, Nick, on how do you address power imbalances? Well, Kevin, thank you for your question. I would encourage you to come along to the AES training that we have on this in a few weeks. So we are actually going to do some training on power in partnerships and how you can kind of consider them in both the process of evaluation and also the evaluand. And I don't mind telling you before the training. So I think, I mean, acknowledging and figuring out how you deal with power within partnerships is not one size fits all. There's a real, there's some really unsaid and unknown things about power and how it shows up in partnerships. So formally, it shows up in the governance arrangements that are in place, particularly around decision-making. So whether that is in kind of, yeah, whether there's someone or some group that gets to decide on things or whether people are participating but not. So power shows up in that way. It's really important to kind of call it out and be very transparent and clear with folks from the beginning about how those processes will impact on power. That's one that's kind of, I suppose, lighter touch and maybe not so forth. I would say within these partnerships, you have relationships between funders, people who live in country, people who live in different locations and then people with different kind of stakes in this. So you have government representatives, local land owners, local farming groups, community groups, local advocacy groups. It depended on the partnership. But as I list those names, you can see that they would be inherent power dynamics between those folks, both in the way that they interact with one another and know each other and also their expectations about their involvement in an evaluation. So one of the things that is really important to try and navigate is creating a space where people feel that they can give to the evaluation without it being checked on or audited or assessed so that their participation is welcome no matter what their participation looks like. That's all very well and good except for when you do a joint workshop and you have someone in there with their boss and you don't know that as the evaluator what that kind of dynamic is. So you can plan for it but then I think in reality there's some things that come unstuck. And so then the follow up to it is making sure that you have different kinds of avenues for people to continue inputting, participating in the evaluation without disrupting the kind of representation dynamics that are in there as well. I don't know, that's off the top of my head, Kevin. I think there's about like 10 hours of considerations in powering partnerships but it is a really good topic and within this one it was really polite how power showed up but that doesn't mean that it wasn't there and it wasn't dismantling for some people. Any others? Public report, is that Alice? I don't know if that one's for you. I can answer a little bit. We are preparing a research brief from this work that can be shared publicly however it may not include all the references I suspect you're probably interested in. So that might be one. Nick or Mal, I'm not sure if you wanna add on to that. Yeah, specifically what would you be, what kind of references are you looking for? You're looking for the bit that I was talking about at the beginning or I can point you towards them? Hi, I'll unmute. Sorry, it's very early in the morning here in England so I'm actually in bed and then once this is over I'll go back to sleep. It's also very cold unlike you guys I hear. Look, I'm about to prepare for a study. Now the context is slightly different. I suspect to yours at World Vision and I should disclose that I used to work for World Vision in the advocacy team but not in the valuation. Although I worked with the deliwaters a long time ago. Look, it's a tricky one. I'm about to embark on an evaluation with the new partnership between a range of mental health service providers and a justice sector provider. And I need to try and do some sort of mapping so I can come up with a concept. Now obviously I'll need to talk to people but the way that it works is highly hierarchical and within your executive level to get buy-in from them for the study. And before I can have that first meeting or pull together a scoping document or a mapping document or a pitch if you like to get the partners on board. I need to be, and it is an embedded impact research evaluation that I'm about to start up from five year project. In order to do that I have to try and find out as much intelligence about the different partner agencies their funding, their governance structure, all of the things that you were talking about but a lot of it isn't easily findable in a sort of literature review. So I'm wondering having, I guess, worked for World Vision myself, the sorts of things that they don't make publicly available. And any tips on how to find stuff that you might have encountered giving you done stuff because I anticipate that I'm gonna have real difficulty because there's a culture here, particularly in the UK of very fierce competitive tendering. So a lot of stuff is not as widely available as it is in my studies in Australia. So just any suggestions or tips, but I think this might be a question for others who are about to, who are requested to do some sort of mapping and profiling before they actually undertake the evaluation work. So I know it's a tricky one but I think it's one that could have broader application for other participants in this session anyway. But it's how do you get access to the information you need to sort of pitch, if you like, to get them to buy into the evaluation to see it as worthy and worthwhile. Thank you. There's a bunch of things in there but I've got some thoughts. Do you wanna go first, Alice? I was just basically going to handball it back to you but just to say from our side in terms of engaging with the partners, a lot of that work for us is luckily done in country and we do have processes of trying to understand and profile those partners, but obviously we have a due diligence requirement as well. We wanna make sure we're partnering with the right organizations that are doing things in the right way. And so we do have quite a thorough kind of assessment and screening process and that does rely on having access to data that isn't publicly available but usually that comes through building relationships and then requesting formally and often the respect of World Vision's reputation helps us a lot there and that there is a lot of trust to be able to share that information knowing that it's not going to be shared widely or it's going to be used for the right purpose. So in terms of screening partners, that's how we would do it in country but that is a little bit different to your question about the research. So at that point I was going to throw to you, Nick and Mal, to say if you had any experience and I'm doing this for the purposes of studies and research. I would agree with everything you said, Alice. I think there's, it depends if the end point that you're looking at is to establish a feasible or viable kind of collaboration between folks. I think there's no easy answer to how do you get information that's not publicly available as you've got to ask people directly, I think is the answer. But in terms of understanding what might be underlying or driving particular organizations to participate in collaboration or not, I mean that the simple name for it is like an individual conversation to understand organizational incentives or drivers. That's what I would put under. And so sometimes folks get invited to pitch for a collaboration but it's a foregone conclusion what the collaboration will be focused on. It's actually quite typical, particularly in funded collaboration. So we're doing a collaborative project on X. If you would like to join, these are the terms and conditions sign up or don't. That's pretty typical. In the case that you're giving Liz, I think that's a really interesting piece of that understanding what people are gonna give to the collaboration, what they expect to get from it, what their organization will allow them to participate in, if at all, what can they shift, what can't they shift? Should they actually be in a collaboration? Should they not? Is there any point in collaborating or are they just going to learn about this thing to one another? So I suppose that the simple answer is ask them all individually for a conversation, I think. And if they don't show interest, I would say that's also a signal of their intent to get that they're probably not interested in collaborating. I don't know if that's helpful, but we definitely find you can do a lot by the screening conversations up front with people. And I would say that most of this stuff isn't publicly available because people don't document it very well or there's no incentive to document it well, so it's not there. David, I've got your message in the chat. Yes, I'll put my email in because there might be some more specifics. I mean, I was looking at the literature review that we just did for this project. I'm happy to share the references with you. They're not all relevant to evaluating partnerships and collaborations between state and local government, but I think I can point you towards some stuff that might be more relevant. So I'll just write my email in and feel free to put me a line and I'll send you some references tomorrow. Not, so I've got to eat, so. Great, well, is there any last questions from anyone? No, if there's not, I'd just like to say a big thank you to everyone for coming along and joining the conversation tonight. And for being interested in partnerships and scaling and how to evaluate them and for your excellent questions and discussion and a big thank you to Nick and Mal for coming and sharing some of the work that they've done with us as well. It's been wonderful to be able to share it a bit more broadly and yeah, I hope we get to hear more about this and hope you can all come and join the AES conference later this year and come to more AES events. Keep your eye out. There's lots of other interesting seminars coming up in the program. Excellent, thank you, everybody. Have a very safe and happy evening and rest of the day.