 So welcome and greetings for whatever time of the day it is in your part of the world. I am Gunjan Veda from the movement for community led development and great to have you with us in the session on community led monitoring and evaluation. We have a 75 minute session with some wonderful panelists and we're going to try a different style of running this meeting what we call the fishbowl discussion style and we'll explain that when we come to it. But the idea is that our panelists are going to share with us some information around, you know, our community led mail and they're going to share with us tools but then we want this to be to be a peer to peer learning session. So that's what's going to happen. If we go to the next slide please. So the purpose really after this session is that we know now increasingly there is a realization that you know communities need to be front and center and development that the top down approach development cannot work. But if communities have to be decision makers, if they have to lead their own development and like us, they need to also have the information. And what information is needed for them to design programs also needs to be decided by them. In other words, monitoring and evaluation for community led development programs cannot be the usual monitoring and evaluations that we do with our programs they cannot be extractive and they have to involve the community. So how does what does that mean. How do we do this. What are some tips. That's what we are going to see in today's session. We are going to introduce you Elaine Stapnitsky from Salonga is going to actually take us through the first segment which is, you know, give us an overview of what is community led monitoring mail different components of it so she's going to talk about that and Indiana Delgadello from a hunger project Mexico is going to share with us one section of a tool that was collaboratively developed by a research team from the movement sort of the movement for community led development had a collaborative research team comprising of 35 monitoring evaluation and research learning and program specialists from 23 different organizations. That's our core team. In addition to that we've had a lot of people from different organizations joining and working with us throughout the process. And both the tools that we will be sharing with you today were developed by this collaborative research team and are currently being used by many organizations in different parts of the world. But back to the structure. So, no, can we go back please. So the first segment of the session is really going to be what is community led mail and then the Anna like I said is going to share with us a brief session, a brief section of a tool that they developed talking about how to do community led development. So, and then we'll have a fishbowl style discussion which I will explain when we get there what it means. The second segment, Jennifer Simpson from PCI, a global communities partner is going to share with us a tool that was again developed by the collaborative research team on specifically on CLD evaluation so it's a 13 question tool Excel based on doing CLD evaluations, we will again have a discussion around that and then we'll come back finally for key learnings and tips. So what is a fishbowl style discussion a fishbowl style discussion is someplace is a style of discussion where we are all speaking with each other and learning from each other. What is going to happen is that at some point in the conversation, everybody in the room is going to enter the panel. We won't all enter it at the same time because that's going to lead to chaos. So we do want to kind of minimize noise and make sure it's productive time for everyone. We want to bring people in to at the time into the panel, and we will discuss the issues and whatever is being discussed. You know, whatever has been shared will go with it will go with your experiences questions reflections concerns that you may have. And again I'll explain more as we get to it. Next slide please. Hello to Elaine I wanted to quickly share, you know, who are the what is the team that has been doing this, and what is the movement so the movement for community development was launched in 2015 on the same day that the sustainable development goals were adopted by the UN General Assembly, it was a group of organizations who got together and said, Well, if we really want to, you know, if we really want to attain the sustainable development goals and we need to put communities front and center and so today the movement comprises of over 1500 local civil society organizations from all over the world, and 72 INGOs, including I know some people who are here so mercy core, World Vision Care, all of them are part of the movement. We operate national chapters and working groups and then we have this collaborative research team which I said, as I explained is a global team spread across multiple time zones and countries where we work together in developing these tools and addressing some of our common concerns on community development and looking at its impact. So that's a very quick overview of the movement. And now I am going to hand it over to Elaine to tell us a little bit about what is community led mail, and why does it matter. Great. Thank you. So, I am from Selenga, and we are actively defining working to define community led monitoring evaluation accountability and learning. So we're, we're calling it Coal Mail or Coal Meal. And we do this by developing a capacity building frameworks for NGO staff to facilitate Coal Mail through e learning and virtual coaching components right now, especially due to COVID. We are also compiling a compendium of Coal Mail resources and a Coal Mail toolkit for key stages of the process. And we're fostering learning around Coal Mail in various networks and communities of practice, which is my interaction with the movement for community led development we're hoping to start a Canadian chapter focused on Coal Mail. And so, as such, Selenga is really trying to be a thought leader and innovative practitioner in this area. So what is it? I do want to start off with a Mentimeter question. So if you're not familiar with Mentimeter, if you can open up another browser or you could use a smartphone or device. And you can go to the website www.menti.com and put in the code when you get there, that's at the top of the screen. And it should be able to lead you to this question. And so you can submit as many words as you like, but what is one word that you would use to describe community led monitoring evaluation and learning, either from your experience or if you are new to this concept, you know what words come to mind as you, as you think about it. So I'll give this about a minute. And if anyone's having trouble, please just let me know. I do have the chat up. What is the mentee number? It should be at the top here. 42968061. 6061. Okay, because we are only seeing your slide, we are not seeing the Mentis screen. Oh, that's helpful to know. Okay, let me reshare. I thought I shared my window, but I am sharing. Yeah, and I've put the code in the chat box for anyone who needs it. Okay, are we seeing the mentee? Oh, and I had the wrong code, sorry. Yes, now we can. Thank you. Great. Necessary, challenging, good words. Any other thoughts? Yeah, definitely participatory. Yeah, empowering, action oriented. Nice. That definitely engaging. Great. I'm going to close it off in about 10 seconds. It's appropriate. Yes, appropriate to the context, inclusive, definitely. We're seeing participatory and empowering come to the fore here. Definitely vision based contextual. Yes. Great. So I feel people definitely have a good concept of the basis horizontal. Yes. And a link to that concept of inclusive. Transparent. Yes. In the interest of time, you can you can keep adding thoughts and then we'll maybe if we have time I'll share this out again at the end or such. But just to share a bit more about how we as a Selenga have been seeing community led mal. It's, we, we see it as a number of things and it can affect a number of things. And so it could be a framework. It could be a framework as well as a process where we're enabling community members, including the vulnerable and marginalized to lead the monitoring and evaluation of community projects to draw lessons and learn from the information themselves. And so we're not taking information away from them, but we're doing it with them. It could also be seen as an approach so as we're coming in it's kind of this broad concept of empowering community members to shift their roles from being just subjects of the data collection to becoming the drivers, analyzers and users of the of the monitoring and evaluation data and results. It can definitely be a system and needs to be integrated into systems so that it can complement formal meal systems. Most people will use a log frame basis or results based management if you're Canadian base so that it doesn't function as a replacement for formal mail systems but can definitely complement work together so that you can have accountability both to the community, as well as to donors and services. It can also be a strategy to deepen community ownership and control over the projects, as well as a tool or a toolbox of things to build capacity, as well as a step within the actual meal process. And so, in, in Selenga we've kind of developed a process to mobilize community members, especially the most marginalized to design implement, so that they can see what's happening with their theory of change and share out those pieces. And so we've just outlined the steps that we are walking through with a different organization, different organizations of first budgeting and being ready and making sure it's suitable for their context, as well as then working with them to build the capacity of their staff and then build capacity of community groups to run through all these processes so again, we were a little short on time to walk through each of these but each of these steps should be very familiar if you're a design monitoring and evaluation person and so we build their capacity to do all these steps. And I had the pleasure of seeing that even, even though they may be literate or semi literate, it literate or semi literate they can also if it's designed well, do the analysis by themselves, and then the interpretation and action comes automatic and you want them to be continuing to do this time and again so I'll close with this. These are just a few quotes of what we want the community to be saying. If you're doing community lead now. So we are in the community and we're taking a lead role in tracking results of our actions and drawing lessons that they understand the barriers and are taking action. These are the collectors and analyzers and not just some subjects so I will pass back to Gunjan. Thank you so much Elaine and I know people may have questions we are going to go through a lot of material very quickly. But remember the fishbowl is a place where you to actually bring your questions to bring your concerns. You can always put them in the chat box as well but you will get time in the fishbowl for everybody to come in and talk about the questions that they have as well as experiences and concerns. So with that, thank you Elaine. I'm going to now ask Diana to share with us. I know Diana that the tool that you were involved in developing kind of talks about CLD as a whole in the program life cycle. But what can you tell us about how does this to look at community lead now. Thank you Gunjan. So yeah, now I want to introduce how we see and how we apply community lead into community lead now into basing in a tool. So this is the tool that I want to share is a self assessment tool for CLD. And as Gunjan mentioned before, this is part of collaborative research that is happening inside the movement. The movement for community lead development. And this is a global effort that started, well the movement started was launched in 2015 as Gunjan mentioned but this tool was launched very recently in January 2021 and it was based on almost two year experience of this collaborative global research and it involves 35 people from 23 organizations and this self assessment tool is based in nine dimensions which addresses the characteristics that we identify that CLD have. This tool is available in three languages and two formats and we'll be sharing more about this in the chat box I can share the link and where you can download this tool for for access and for knowing a little bit more about this tool but this tool is not necessarily for for this side if we're doing good or bad CLD this is this is a tool for leveraging an inner and inner process to see in which dimensions we could improve whether we're from an organization and a funder or media community member. And this will help us to have evidence on the impacts of sorry on the impacts of CLD and there are a lot of some more dimensions so but what I want to share with you and let me share my screen for going through the tool very very quickly. And I will share the link in the chat but this is a tool so we have to format one of them it's in in an Excel sheet and we have a mobile version of this and we have it in English and French and in Spanish. And now we're going to to focus on this on the English version. Let me, of course, yeah. So, and we're going to focus just in the, the dimension of monitoring and evaluation practice support CLD. So you can see the nine dimensions over in the list and some dimensions are larger than others but each of them has the same possible answers, which are gravel from and it says from zero to four with zero it's no information one is doesn't try. The two is tries the three progressed and the four means succeed. This is a really quick view of this tool but for the purpose of this session and because we want to have time for the fishbowl. In this age dimension, where we are focusing on participatory, sorry, monetary and evaluation practices for CLD. In this section, we identified three dimensions that are relevant for really say that mail is happening based on a CLD approach. First on participatory monitoring and how it's carry out. So we have these four options. As you can read it's incremental and it goes from no information or the information it's not enough for a judgment to the fourth, which is community participatory monitoring is central to the monetary and evaluation approach of the program. The second question is about how participatory evaluation happens in the program. So in this case what we want to see is that if it's possible to go through no community community participatory evaluation to have participatory process in some stages. It's like maybe you have participatory planning in data collection, but not in any other stage of the evaluation. So you use a two or you give a two you evaluate this program with a two, or maybe they have three to four evaluation stages cover. So we could give a number three, but in the best case scenario, community members are part of and central of the evaluation so we can have a four. And the third dimension, sorry, the third question. It's related to, it's about a money findings and if they are disagreeing based on different and relevant social differences and this is a, and if the program acts upon if they have of the or they take some, some measures about that. In an ideal scenario, we see that CLD mail happens when based in our inner tool. What happens when community participatory monitoring is central when community monitoring is facilitated by community members or structures and includes reporting back to people in a regular basis like everything small six months. And also, these happens when CLD when community members are central in all of the evaluation stages from planning to the action plan. And also we see that we have CLD monetary CLD mail when findings are disaggregated, according to all the social differences that are relevant to the to the intervention. And this is translated into into actions in the in the actual program because we, we can improve that program. So, more in a procedural matter in every question it's important to write down your, your comments and your, your reflections about what you give this information because this can help you to, to use the tool later on. But this is, this is something really valuable for you to identify how you, I, how, how do you conceptualize the mail evaluation based on the CLD approach with this tool and this is the thing that we wanted to share right now. So, I hope that that gives you some elements on how we see the CLD and right now we have I think we have a poll I will stop here and I think we have a poll Sarah if you can launch it. And of course if you have any questions we are going to use that for the for the visual process. So, again, this was a very quick overview we are actually the tool that looks very specifically at the different dimensions of, you know, how to look at community led mail valuations is going to come in the second half. But this was more of an overview of what it would mean to weave this into the design process of the CLD program itself. So people can start filling out the poll, you know just based on what it is currently, do you or your organization include any of these aspects in the work that you do. That would be wonderful. And again, this is just for us to begin to get the sense of what is happening what needs to happen more and what are some of the challenges that we are facing in different segments as we do this. And while people are putting in the responses, I want to give a quick overview of how we're going to proceed in this fishbowl segment. So, I hope everyone knows the raise hand button. So if you go to, if you go to reactions on your screen, you will see a raise hand button. I would encourage you to use the raise hand button because in the fishbowl style discussion, we are going to invite people in pairs of twos to join the fishbowl and the conversation with us. You can, the idea behind the fishbowl is that we learn with each other, and we learn from each other even as we learn with the panelists. So there is a lot of horizontal discussion brainstorming that goes on in that discussion. And next slide please. So just some of the things you can do. You can actually ask any questions that you have of the two presenters in the fishbowl. If you would like to ask questions of the presenters, I suggest you put your hand up first because in the fishbowl style discussion, people keep moving out. So your panelists are going to move out after the first round of discussion and we'll have others coming in carrying forward the conversation. Think about, have you done this? If you had to do, would you like to carry out community led men, right? Is it feasible? Is it necessary? If yes, what kind of enablers are there? What kind of barriers and challenges are we facing? Any concerns that you may have? And I see some people here who were at our morning training session, and I know there was some concerns raised about the time requirements or the burden on the community. So any of those things can be brought into the fishbowl. But before we go into the fishbowl, let's look at the results of the polling. Can we end the poll and look at the results please? I know not everybody watching, but whatever we have. Okay, so based on whatever we've seen, it seems and no surprise that the area where communities are probably most involved are in the data collection aspects of Mel, and I am guessing I may be wrong, but this is probably because we often tend to involve enumerators from the community in data collection. So even within data collection, I would be curious in the fishbowl to see where community members are coming in. My guess is it's probably happening and the collection of the data, not in the decision making on what data is being collected, how? And then the least one seems to be, well, data analysis seems to be the least one, and that there are at least two places where there is no community involvement. Okay, thank you very much. I'm going to move into the fishbowl, so if you stop the screen share, thank you. So can you please raise your hands if you would like to enter the fishbowl? If not, we will get two people randomly into the fishbowl. You will see yourselves in the spotlight. It's not necessary to speak if you're on the panel. It would be great if you speak, but it's not necessary for you to speak in the panel. But the idea is that let's see where we learn, what we learn and where the conversation takes us. So, and I see two people have come in already, wonderful. So, Sarah, let's get the fishbowl started. And I am going to, while Sarah is doing that setup, I'm actually going to invite Zilat. Hello, Zilat. Wonderful. Zilat, do you want to start, do you want to, whatever your question, your reaction, your experience, your concern, please feel free to say whatever you were wanting to say, and we'll get you into the spotlight. Okay, first of all, I'm working with the community who are affected with the flat and cyclone for six and a half years. So I never seen that the community led evolution. So this is something that seems quite new to me and what I feel this is necessary for the community, but they're in reality, I'm a bit, you know, confused and worried that it is actually possible for the community to let the evolution for their actions. Because the community, what challenges I feel that the leadership from the community to make them to make them feel like this is their responsibility and take their leadership to do the evaluation is a very big challenge because we are working with the community and at first place make them understanding the thing is, it took a lot of time. So, and their mindset and they have the, like for having their ownership for the evaluation and carry the action. These are the barriers I think I, if I have to, I have to do that in my with my communities and give them the evolution right to evaluate their work. First I need to change, make them to get that leadership on themselves and change their mentality I think. Thank you so much. You know, those are great points and I wonder if Elaine or Diana you want to respond to them and then a Smith I can also respond to this, or bring up another question but anyone who wants to. Yeah, Elaine, Diana. Anyone who wants to respond to seeing it's concern. I'm happy to jump in. Yeah, no it is difficult and it definitely takes intentionality and it takes further planning a little later on in the tips I'll talk about what are kind of those enabling pieces but you have definitely highlighted what some of those challenges are is that, you know, there's there's a number of things that I mean on one is, if you want to go fast go alone, but if you want to go far go together. And that's a, that's a common African proverb, but it means that if you go together it does take more time. And if you are pressured in, especially emergency type situations, you know the donors expecting certain deliverables within a certain time, it can be very challenging and so in those cases you may want to reduce scope, and just focus on where are the elements that you can bring into the process and make it and just push it along the continuum of being more community owned. Or is there a particular part of the project that you can make participatory or more inclusive or give the community the power to make decisions on particular aspects of the project. And I am going to come to the Anna Smith and then seeing it back to you because in the panel it's a back and forth so it's not like you just say your piece and you are out. But before I get to that I do have a request so we are doing the fishbowl on zoom for the first time we've only done it in live sessions. And we've just discovered that for the panelists to be on the spotlight they need to have your camera on now I understand it's not possible for everyone to have their camera on and it's okay. It's not a challenge we will deal with it. But if you can. Thank you so much, I appreciate it and see that an estimate that. So, I'm seeing it as Mita Diana, anyone wants to see that does that speak to your concerns, is there anything else you want to say on that. Yes, like what he Anna says, if it's not possible to make all the money system participatory we can go for a certain focus to think that we can make it participatory like I am the project I'm working in. We are just started working on a community. I won't say like community led monitoring evaluation system but we are trying to be more accountable to the community in our many system. So, what we are doing is like, we are using community self assessment to realization for further action. It's like, we are the data we collected from the community where giving them back. The same data and we want them to analyze them or evaluate themselves on the basis of that. And what we are doing further is like, after that analysis, like evaluation which they do with their collected data, they make the action plan. It's a process. It's not a tool like it's a process which which we have named as community self self evaluation to realization for direction. So it's kind of like we thought that that were innovation to us but like when knowing from all of you. We have to do a lot on community led amines. So thank you so much for the learning to. Thank you. And I think one of the questions that does come up about community led mail and I know I heard this in the morning session. I would be interested in seeing what others also feel as they come in. Is this constant thing up? Is it always possible? Or are there certain times and places where only certain things are possible and doable? And how does one decide that? And I know Gabriella, you also had a question and will bring you into the fish bowl for that question. But for now I want to go back to the current people in the fish bowl. Diana. I think that if you think if you see the CLD as a process and not only that happens just at some stage, you can see that you really need to see the value on that. I was thinking on the experience on the hunger project on the CLD, the participatory mail process. And this is relevant for people because even though they do have to develop some capacities for doing that because they need to be really aware of the relevance of the things that they are going to do because it is true that it implies different activities, more activities than they have to do. It also gives them the strength to make a difference of their vision or their work or the things that they are putting into some processes or some projects or some actions that they have at the local level. And based on that mail process, participatory process, they do have information and they are able to make decisions based on that with their community authorities and that gives them the strength to act differently. It is also something that came into our attention that not every process can be that participatory because of the, because of local views or local processes, inner ideas. For example, based on women working in decision making spaces and sometimes the local knowledge or local traditions doesn't allow that to do that. But just putting the question in there, just starting a different process on how people approach this idea. So it's a process. I want to just like ask you all one thing, but do you really think that the locally lit monitoring, the donor will be agreed with this? They will, because that monitoring that is start with the very fast, like the planning of the community and designing their own action and plan. So that needs to sustain and without local resource mobilization doesn't like not goes appropriately all the time. So we, they need additional support. And I don't know how the donors will agree with this. This is like, I don't see any door for this plan. I'm just not being negative but I'm here to like, I hope there is always a option. Maybe you can ask. I can see that that's that's actually it's not about being negative at all. I think this whole conversation is about, you know, kind of looking at some of the practical challenges and what is needed. I think the only thing I will say to that is that, you know, it's not easy. And you're right that you know one of the things about community like monitoring is that it does require more resources or more resources at the same stage in the process. Because all at what stage do you need to invest more? What stage more intensity? The community with I'm working with, they plan their own design of action like they need to raise their plane because flat is there every day, every year. They need sanitation support, they need people because women didn't eat for days because they don't have toilet to go during that. So the monitoring and evaluation if they do, but they need support local disability never will like fulfill the need. So, and this, yeah, maybe in it will come with that with time and more effort and more like we need to generate more evidence I think so and yeah. Yeah, I mean some of the donors are beginning to at least talk about community like monitoring and evaluation now. What this, when will this talk translate into actual funding being made available for this is this question that you know that we don't know answer to. But the interesting thing for me has been that in the last few months at least I've heard increasingly donors begin to speak about the need for this. And so now we need to see if people will put there, you know, if they're going to actually walk. I think this one should be one of the principle of local elite adaptation, one of the principle like they have the flexibility and predictable funding. I think that this should be goes there because they've been more like focus will be up on this diet. Absolutely. I am going to win. I know it's time to swap fishbowl because we also have. So what's going to happen is we're going to be able to take two more people into this fishbowl. Diana and Elena going to step out and we're going to have two more people join us with that and Zina to carry this conversation forward. So after which we will go into the next presentation on this actual evaluation to and we will come back to the questions and see is that has there brought in more clarity or more confusion. And then we will have a link the Anna and Jen who is going to be a third panelist come back in the end with tips and to answer questions. So, so it's going to be a different structure, but the conversation will carry on throughout these 75 minutes. But, Sarah, if you can bring in I don't know if anybody has hands up who would prefer to come in and this round. If not, I see two people I see Gabriella had a question or a comment so maybe we can bring Gabriella in and one more person if they want to come in. And I don't think that adults and is on public transport so can't speak that. Oh, can you perfect that L if you can. Oh, you can't speak. We see you and I will be a voice while you are there. So, so thank you, Gabriella, is there anything. Okay. Hi everyone, nice to meet you all. Thanks for the presentation I came in a little bit late sorry I apologize for that and I need to go early as well sorry. So my question is, and well at go I work for go and this is a, and this is our aim to promote and to being may making this monitoring evaluation led by communities. However, as we are in that process. My question is, which are the main barriers that you have encountered when starting implementing this in your organizations. I have few but I would like to know what are like the key actions you have undertaken to address those barriers from top to down, because I know that, for example, meal teams are aware of the necessity of communities leading monitoring and evaluation. They are this external party that can better say we are we are doing what what is expected to be done right. It's like the external level writers but you know the most important ones because they are the ones for which we are trying to build well being and so on. So, which are the main barriers encountered what when trying to promote this within meal teams right or management levels and what are the main actions to overcome to address these. Yes, so those are excellent questions and I think that's again going to about how do we how do we actually put this into practice right, and how do we put into practice comes at various levels so they're on the donor level challenges that we talked about. And then now they're the, their challenges not just at the donor level their challenges also at the operational level within teams right because a lot of it comes down to this question of power dynamics. Right. Because even within between us and the communities whether we like it or not that does exist to power dynamic and being to be able to see that control right and to say that we are going to actually listen and have the community defined instead of us defining what is important in this project, and communicate with them, so that we have really very little control then over the results that are coming up right, or what we want to put out and that does require a large of I think conversation but I think it's also somewhere around the question of, how do we begin to look at CLD as a process overall right, but I know it all you can't speak. I have your question I will and I will say that, and I'll bring it in for you, but us with that in Zenith I if any of you wanted to come in on what Gabriella said, or asked, and again Gabriella will have the panelists respond to it later on when they come back into the room, but for now it's Zenith and Asmita if you want to add to the conversation, while I pull up Adele's comment. I think what we need to make this in process that good intention of the implementer, like us we are civil society because what the result will come from the evolution. So we are not expecting that from the community we are planning something else in our mind. So the intention of except, excepting those, what community is saying and community needs is it should be there, and that community to raise to do the evolution, properly or a structure where they need some knowledge and guideline, I also think that to like make them the evolution what they need and they will be needing in future they need to be trained on that they need to be that those doors should be open for them to understand. So I don't know I couldn't think more about this. Maybe Asmita can add. Nothing on Gabriella's question like I do have a concern like most of the people in our communities are not well educated to know about all this process and just when we go with them to collect some data for the assessment they don't have enough time like they can't give us a half an hour 15 minutes to. So, in that point like how can we engage. What are the things that we need to consider to make them involved in the process of our meal. So, I just wanted to know about all this things. I'm curious about all this things. I think they lost hope that something can happen. No, and these are all very valid concerns so I think you know as with the ends in it and Gabriella to your questions like I said we will come back to all of them. But right now all I wanted to flag are a couple of things and then I want to read adults point and welcome Jen into the panel. Now to your questions you know so so the whole thing about organization sitting control, and to kind of you know, accepting accepting of that that requires a shift in mindsets. Not just so when we talk of mindset transformation and community development we always talk of external mindset transformation right, we talk about going to communities and transforming their mindsets, so that they believe in their agency and ways. We forget that there is also a lot of you know, internal mindset transformation that needs to happen within our teams and within us and as organizations to be able to do this right because still that happens. We may say we will do CLD because that kind of localization seems to be the topic of the day, but it's not going to happen. It's not easy and and Smith about, I think you raised two very important questions right. I think the time question is something that keeps coming up again and again not just about community led mail I would say it's a general question when it comes to community led development work. We just conducted a study of about 100 173 programs across 65 countries that the cells declared a CLD programs right. And all of them depend heavily on community groups and the work of community members. Yet this is all and a lot of them are women. Right. If these are the most marginalized and the most excluded people in the community then they're also the people who have the reason time resource on their hands. And yet we are adding to that. So how do we manage that I think remains one of the fundamental challenges off CLD and something that we are trying to, you know, I don't think we answer yet. But I think we are looking for these answers and conversations like these and with donors and others to say that fundamentally the way we find the team and do things has to change because there are these concerns about the knowledge part and I'm sure that you know about the ability to do this right like do they know this or not is something that I'm actually going to have Elaine come back when she comes back come and address because I know this is something that she has spoken to I've heard her speak to this. So I'm actually going to let her address that part of it, but I think we need to think about what we regard as knowledge and skills. We are used to a certain language of monitoring and evaluation. And certain methods and methodologies and it's true that the communities will not know that. But communities do know that if they are creating a program. What do they need to know for that program to work well or not. And it's essentially that bridge our role perhaps is that role of that bridge that when they identify what they need, and how they want to do it we kind of translate that into the language that funders and donors and us understand. So maybe our job is that translation. I also had the point about knowledge and M&E is political and engaged in power dimensions. Knowledge is the origin of consensus and community are diverse. How do we ensure that community led M&E processes are inclusive. And that the most marginalized needs come to the front in the M&E process. Again, excellent question I think some of that was what I was trying to say when I was also referring to the hierarchies and and the inclusiveness but hopefully a value will soon be at the point where you are able to speak and come into the conversation. For now I am going to move to Jen Jennifer Simpson from PCI who is going to share with us a tool which so far we were talking about in general. And you know some of the challenges. Now what we're going to look at is that there is a tool that we can at least begin to ensure that our evaluation reports are much more congruent with the principles of CLD even if you're not able to do all of CLD. So Jen over to you. Hi everyone, and I really appreciated the discussion on the fish ball and just wanted to add you know, as we're thinking about how we do community led now, and thinking about it's not just about, you know, the communities and stakeholders being trained and engaging but also this mindset for our own program and our own staff who need to be able to do it well and efficiently, and that these kinds of engagements also include, you know, a lot of trust building. And so I think there are some, you know, internal program, you know, for own staff capacity building in that in that light. I'm going to walk you all through a quality evaluation quality appraisal tool. I put a link in the chat, it is accessible online. And as I pull up the tool, tell you a little bit about it. So this tool was commissioned to support the assessment of studies and evaluations. It was designed to appraise the strengths and weaknesses of evaluation reports in terms of their methodology goal rigor and community led development principles and so as such includes all the phases of a program evaluation based on its corresponding report. And it also includes elements of community led monitoring evaluation. So I'll point those out as we go through the tool. Through participatory to phase and a piloting process with a number of Emily and for multiple organizations. So this tool was developed part of the movements collaborative research team that Guggen mentioned at the beginning of the session. So this tool is really meant for users to include M&E staff, but also other program staff. These are individuals who may be commissioners of and or those who conduct the evaluations themselves, and similar to the CLD tool that we got a little glimpse of. It's not Jen, I'm sorry to interrupt you but I feel trying to screen share that we can't see your screen. Sorry about that. So sorry. Thank you. All right, hopefully you can see a picture of the tool now. Thank you. Thank you. All right. So it's intended not just to educate good versus bad evaluations so it's not just about grading and evaluation and this is a bad or good evaluation. So really I hope it serves as a learning tool about how we can improve our evaluation process through using the tool to assess our reports. There are other tools out there that evaluate evaluations or assess evaluations, but this tool is different from others in that while the foundational understanding of a robust evaluation approach is helpful. It's meant to be simple enough for really any of your staff to apply. And the second the tool supports the standardization of best practices in particular for community life development. So as you can see this tool is built in Excel. And you will see that there are two tabs there is a guide tab, and then the tool itself tab. And I'll just kind of give you some quick information through the guide because it goes through the scope and purpose of this toolkit which I gave you a bit of an overview already and how to use the tool itself. You'll also see that the tool ends up providing a score or rating. So this is an overall evaluation report score but we also provide a CLD specific score. These scores are embedded into the available answer choices within the tool. So, whether you answer a particular question in the tool as a yes and no or partial. There's a grade that's given to each question on the tool and the scores are summed and reported in the tool tab so we'll go through that. We also noted that some of the questions are weighted, and though all the questions on this tool are important there are 13 questions. During the development of the tool some questions were determined to be extra critical and therefore they are weighted a little bit more heavily. So this is really what's in the guide to to help remind users of the scoring and the waiting process. So let's look at the tool itself. The second tab, you can put your evaluation title at the top who's actually doing the assessment and when. And then you'll see at the top, there is the actual score so you can go back to the guide to reference you know where you fall in the score, but really again this is about a guide of about how you're doing. And then we have the 13 questions, which are also noted as indicators and true eminy form on this tool. And you'll note that some of the questions are have a little bracket with a CLD besides them so that way the users actually know in the tool which of questions have the community led development relate you know are the development related questions. So the first one, I, you know there are 13 questions but I kind of want to highlight the CLD questions for you. So the first one has to do with how the report provides information to understand the local context, in which the intervention was implemented. So what this question is getting at is really how well the evaluation team sought to understand the local context. As they relate to the program and activities and the evaluation activities pertaining to that program. So the first one is around does the sorry are the data collection tools piloted. Oh sorry MS one does the report discuss involvement of relevant stakeholders during the evaluation design in particular the community stakeholders. So this is looking at during the design stage of the evaluation itself, not necessarily the design stage of the whole program. The design stage of evaluation where the evaluation questions are being determined and refine were local relevant stakeholders engaged in that process. The second, another question is our data collection tools piloted and adapted through a test phase with relevant community members. What is going on to more around the actual data collection tools and the piloting and adapting. So a lot of times we could be developing brand new tools sometimes they're standardized tools. But are we looking at our tools with the lens of our communities in which we will be applying these tools to be culturally actually relevant. Another CLD specific question is around whether the report discusses the involvement of relevant stakeholders during the preparation and implementation of data collection. So this is getting at the contribution of local knowledge to effectively adapt our tools, and this also includes trainings for enumerators. So those who are going out to actually collect the data, how they're asking the questions and what context are asking the question, and also quality control processes but does that translation for for accurate translation. Finally, does the report discuss involvement of stakeholders during the evaluation analysis and review of finding. Another particular question is really trying to get at whether the local communities and leadership have an opportunity to review the findings and contribute to the interpretation of results. So this is really trying to assess are we not just take collecting the data, writing up our evaluation reports and moving on. Or are we going back to the communities in which we were working and actually sharing back and actually asking for their contributions into what the results mean. So these are a few of the CLD specific questions in the school. So your tools are getting more at basic rigor. So there are questions around, you know whether their report is, you know, clearly defining the evaluation questions, whether the analysis was done correctly. Does the reports bring up issues such as potential biases in the analysis. But basically you would go to use this report you simply would go through question by question, you would read the indicator decide based on your definition, a yes a no or a partial score. And the tool will automatically calculate your running total as you go through. And so what you'll see as you answer all of the questions. Ultimately you'll get a final score and a final CLD specific score. And so this is the overview of the tool and how it works and the kinds of content that it takes. So I think it would be really great if you're able to go through download the tool, get a sense and I'm hoping that we can move into the discussion to think about. Yes, this is a tool and Ashmeetah had a comment in the previous fishbowl about, you know, community led monitoring is less of a tool it's more of a process so how does this tool. How could it be used or useful to explore how the process is, for example, so I'm really interested in to hear what you all have to say about the school. So I will pass it back over to Gantan for the festival. Thank you so much and and you know I know Gabriella and a couple of people had to leave but I was going to say to her that this was one of the things you know in almost an answer to her question about how do we begin to do it within organizations. And maybe we can start, you know, small. Jen if you stop sharing your screen, we can still see your screen. The answer is to start small and to start with a tool like this to begin seeing how to be gradually build up right. It's studying questions we wanted to keep it simple so that everybody could use it irrespective of the size of the organization. You know, you are able to do it it's not extremely complicated, but still you start with the basics and it tells you how communities can begin to be involved. You will not have time for a fishbowl but I do want to give people the chance to ask any specific questions they have around the tool to Jennifer, as well as to kind of any concerns comments that they have, or if there has been a very rich discussion I know if, you know, listening to this or looking at the tool some of your previous concerns are getting addressed, they're getting actually even more intensified. So we'll have about, you know, five minutes or so, if people can raise their hands if anyone has any questions, reactions, comments. I think, Gloria. Yes please, Gloria. Hi. Thank you for the for the time I had a question and it's kind of related to what was said about thinking of this as a process. So I wanted to ask to the to the panelists. If they think the tool, or some of the indicators included in the tool could use could be used as a progress report. So when when you're creating policy or programs and especially social social programs, we usually do like a mid review. You know when when the projects have done, or when you've spent the most of the money. You want to see how you're going, how you're doing but from a from an institutional point of view. The organization tends to be on the indicators that you proposed from the start on the number of people that you've attended on the number of people who participated, etc, etc. So, thinking about this tool. Do you think it could be used it could be. Is it appropriate to use like midway or different points of one investment or one project. And thank you for that. Jen. No, I was going to ask you if you want to answer that. Okay, so I mean the short answer is yes I think there's ways to use these tools for that kind of purpose. I've been getting to know and use these tools. I kind of find found a lot of different areas where they can be either applied as they are or elements of them could be adapted for certain purposes. So for example, on this evaluation quality appraisal tool. You know, it can be used to assess your how you did on a particular evaluation based on this report, but also before we even get to the evaluation we're looking at these indicators and seeing okay. So designing our evaluation and creating guidance around, you know, an evaluation scope of work or to our, you know, these elements are going to be included in that evaluation plan from the beginning. And that kind of helps ensure or we hope it'll help ensure that down the road once we do the evaluation, we would actually then see that reflected in the reports a little better, as well as updating like our evaluation writing guidance, for example. And so those are some of like the ways that we're like applying this tool in not just the tool itself as it stands but also like taking from it and and trying to apply it in other ways. I do think that when we're talking about participatory M&E community led M&E. You know, building that and incorporating it within your M&E plans and not like an addendum to the plan, but actually having it sort of embedded foundationally within your M&E plan is going to be really important. And we've had some lessons learned within our own organization around how we do that more effectively. So that when we're reporting on indicators, our IPTP is not just donor indicators or some indicators, we also have these other types of indicators like participatory monitoring indicators, sustainability readiness indicators, but they're all part and parcel for M&E plans. So, I don't know, hopefully that kind of answers your question. Okay, we are going to be we're running behind time and so I want to make sure that and I know people we did the promise tips on how to do this. So I want to bring that Elaine to kind of quickly give us some tips on on, you know, from the actual part of doing this, how do we overcome or address at least some of the barriers that people talked about. And then after Elaine, we will have a minute each for both of the presenters, panelists for any last words that they have. And then we'll come back to the audience, but Elaine, over to you. Great. So I hope I'm sharing my screen to let me know if you're not seeing the prerequisite conditions. Some of these are basically consolidating some of the discussion we had already. So having a commitment from the donor. And if not the donor than the implementing organization or the partners to make community led Mel work is very important. If you're pushing against that it can be very difficult and so it does take time it does lean on your relationships with people and with the donors as to their openness and to build in the time for it. Community was raised during these discussions is that maybe they're too busy. And so if they're, it needs, the concept needs to be introduced to community leaders and the community people who may be involved, because they would need to set aside time. You want to be building into the systems and structures, you know, and considering what sort of compensation, the community values it at and what they can build as resources or what may need to be complemented by organization and external funding to at least set up the initial system. And then the willingness as people were saying to give voice to the vulnerable and marginalized some of whom are often women, or those with disabilities or other vulnerabilities. And then having a strong feedback mechanism so this speaks to a number of organizational enablers, whether within your organization there's already transparency being fostered whether they want to be hearing from from different stakeholders, especially those who are being affected by the programming that they have. And this is, this is a handout that we're developing that will be available on our website soon within the next few months, month or so I should say, but what are some of the key considerations so I'm not going to go with it in detail. But as you see there's a lot around flexibility whether it's around designing the tools or flexibility in the funding or the time that in depth initial assessment that the MCLD has developed, trying to get a full understanding with the community of one on the one side the organizational readiness, but then also what's happening in the community. We do want to make sure we're not leaving people behind. There are, there are questions or concerns around quality of data, but we, this is really building from utilization focused evaluation and so meaning that the more people use the information, the more improved the quality will be, as opposed to kind of our typical M and E which is usually research base how do we get those, you know, control groups in place and trying to get to the end degree of accuracy that we can spend a lot of resources on the focus is more on use. And then, again, some pieces around resources at the beginning of the project period, whether it's staff time or funding or time itself to make sure that you've set it up well. But as we said it may not be right for all the context so just to highlight some of the learnings that we have around what we, what are benefits of this is it provides evidence at all levels predominantly for those who the the programs are effective, affecting predominantly supports good governance fosters partnership definitely strengthens their community capacity which leads to sustainability catalyzes development I have I have seen and trial that, and I'm convinced, and it can be less expensive if designed well, but as mentioned it, you know, it may be more qualitative than quantitative data. There's can be limitations due to literacy and numeracy of the community but I have again you can design these processes in participatory ways, using figures and simple metrics and rubrics, so you can get around that and so I, we do have a number of examples of that as well that I don't have time to share. It may not necessarily replace your formal mel system for your project or for the donor. And if it's not done well, well it can reinforce existing power dynamics of which we've discussed extensively. Thanks for that time I'll pass back to. Thanks so much. So Diana, I'm going to come to you and Jennifer now after Diana for your one quick take away reflection answer response anything to the session. Thank you. Thank you, I think that what it's think what I'm thinking right now is that this session it's intended to have this, this is moments for, for what is community live and what I think it's very pretty clear for me now it's that we do have experiences on community live based on the experience of one organization in this case HP for example but this is something useful on how do we need to build on the community on capacity community capacities in communities for addressing this and also something that I was thinking it's, this needs to be relevant for communities not only for funders or whatever and this is also a mindset shift for everyone who's involved in in in mel in mel systems in mel developing mel systems. We need to, to think differently on why we need these mel, not only for for for accountability for to a funder but also towards the community and also to understand different times in inside the communities. So it's a new way to understand monetary and evaluation it's not only a different way to do it but also a different. It has a different purpose. So I think that's kind of my key takeaway for these because it is a process but it's not only. I mean it's, it's because it's changing constantly, and we need to adapt into that, always based on human dignity and communities in the center. Thank you Diana, Jen, you muted. Yeah, for me I think one of the key takeaways is that, you know, co mel maybe somewhat aspirational right now, but there are steps that we can take to concretely move ourselves into that direction together. We need to consider how we effectively and officially efficiently incorporate these principles of community led development and community led mel into our existing mel plans. So we're going to need to talk with and convince our donors, but we're also going to be need to become listeners with the communities in which we're working. We're not the ones saying well we know what it needs to look like, but actually to become, you know, more of the listeners and, and, and really work with our participants. So I would say to focus on the process and what we can do in the interim and step by step move our ourselves in that direction. Thank you, Jennifer. I, we wanted to keep a couple of minutes and we literally have like one minute left though. But if anybody in the participants and people in the room with they have anything that they want to share in terms of any key takeaway something from learning from the session, then that would be wonderful you can put it on chat, you can just unmute yourselves and speak. And in the meantime, my colleague on the is going to put into the chat box. Email IDs for all the panelists and for me. So if anybody wants to get in touch to kind of, you know, discuss more about community led mel. Share the resources to learn more about the resources so long as a bunch of resources the movement is having very active conversations around this. We have a group that is going to be looking more and more into this. You're very welcome to engage with all of us so we will be sharing the email IDs. But does anyone want to share anything was anything that was new was anything that you learn any key takeaway. Just unmute and speak. Don't even wait for me to call on you. I can maybe quickly. Yes, say a small comments because we are so I'm a lawyer and we're working a lot on governance and trying to help small scale fisheries community to set up management systems and frameworks and we have been starting to think about a project to actually being able to be able to give that time that communities needs to set up and create their own. Management systems and I think some like we're trying to look at blended finance schemes to see if we could find a way to convince investors to really invest in the long term. And I think if I just looking at your tool make me think that it could be something pretty useful to try to convince investors or something. Okay, look at this. This tool we're going to be following up the community is going to be following up the evolution of their own evaluation monitoring system so that they can be independent and they can continue to to have the project running just by themselves. So, thank you. I think it was really, really useful. Wonderful. Thank you so much as anyone else. Otherwise, I know we are at time actually we are one minute over so. And we know it's been a long day but thank you everyone thank you for bringing in all your insights to all the people who entered the fish bowl it was an experiment. It wasn't as smooth as we expected it to be but hopefully it will be better next time. But thank you so much for learning with us for asking very interesting and difficult questions and for everything that you have shared. In both in the comments in the chat box, as, as well as in the discussion section. So, if we can just put up the last slide on here, and we will let everyone carry on with their day but thank you so much. Please don't hesitate to contact us. We will be in touch, if you would like us to be and we look forward to working with you more on these issues.