 Well, hello everyone. Welcome to this AES Victoria seminar today. We're really thrilled to have Associate Professor Amy Galaxy as our speaker. I should say good morning, good afternoon, and good evening. I know we've got people joining us from around the world. So thank you so much for taking the time to join today's seminar. I believe we at least have people from the US, New Zealand, Australia, and I think a fairly large group from the Mongolian Evaluation Association as well. So welcome and it's just fantastic to have you joining this Victorian AES seminar. I'd like to start by acknowledging the traditional honours of the lands that we're hosting this AES seminar on today. So that is the lands of the Wurundjeri peoples of the Kulin Nation. And I'd like to pay my respects to the elders of those lands past and present, extending that respect to future and emerging cultural leaders, as well as extending that respect to anybody who may be joining us this morning who identifies Indigenous to Australia or around the world. Welcome. Okay. Just a couple of housekeeping things for this online webinar. As you can see, the session is recorded. So hopefully you all feel okay with that. We will keep the recording on for the duration of the session, including to capture question and answers. Amy has agreed to pause several times during her talk today to take clarifying questions. But because of that, we really welcome you to post questions in the chat during the presentation. And at times when it's appropriate to take a pause, we'll be able to stop in and address your questions. There will also be time at the end prior to us finishing for you to ask further questions. But again, the chat is the best place for you to communicate with us throughout the session, just because of the size of the group. So please feel free to use that. Yes, Esperanza, you've asked for the closed captions to be turned on. I think we can do that. Just a moment. Okay. Yes, I'm sure I can figure out how to do that. I might just start by introducing our speaker. And hopefully I can figure that out shortly. So look, today's webinar, we are thrilled to have Amy Galaxon as our speaker, as I said, Amy is the director of the University of Melbourne Centre for Programme Evaluation. The Centre has been delivering evaluation and research services, thought leadership and training evaluators and giving qualifications to those evaluators for more than 30 years. She is also a co-founder and current share of the International Society for Evaluation Education. A long time member of the Australian Evaluation Society Pathways Committee and its predecessors and a key architect and leader for the University of Melbourne's fully online multidisciplinary masses of evaluation program. So that means Amy practices, teaches and proselytises evaluation as a trans-discipline, that is, as a discipline of its own and a tool used in many, perhaps all other disciplines. Her teaching and research are focused on creating clarity about what evaluation is and what a good evaluation looks like, credible, systematic and useful determinations of merit, worth or significance through the application of defensible criteria and standards to demonstrably relevant empirical facts. This means evaluation must surface performance and offer clear and transparent reasoning to arrive at judgments about how good that performance is. Following on from that, she is currently studying, thinking about and experimenting with what people and organisations need to know and be able to do to deliver good evaluation. Amy does all of that because she's pretty sure evaluation is essential if we're going to save the world. I think I agree with that. Amy is also a graduate of the Interdisciplinary PhD in Evaluation at Western Michigan University where she learnt the logic of evaluation from Michael Scriven, the program evaluation standards from Dan Stufflebeam and mainstreaming evaluation from Jim Sanders. She's a sociable introvert, asking about what she's reading or about the connections her brain is busy making among seemingly disparate things. So thank you so much, Amy. I'm going to hand the floor to you and really looking forward to your presentation. Thanks Ruth, and thanks for the opportunity to do this again online. So as Ruth said, I'm Amy Galixin, and that was a super academic introduction, but you will find this presentation is much less so. I'm happy to be talking with you today about soul and maturity on being evaluators, which is for evaluators but also for people who just dabble, I suppose, or deal with evaluation in different ways. So let me just find my way back to the screen where I've got this and there we go. I'd like to also acknowledge the traditional caretakers of a variety of lands first the world were injury people whose land. I'm on right now which is in the region that I call Melbourne and where's been my home for about 10 years now. So the Ghana people of Adelaide whose lands hosted the as 22 conference where I did this original presentation, and the Lakota Sue people of the land where I grew up in South Dakota in the United States. I pay respects to elders past present and emerging and to any indigenous people with us today, and those who might watch the recording. I'm deeply grateful for their tens of thousands of years of observations and knowledge about how our planet works and how people can live sustainably within its systems. And I hope the rest of us can learn to pay closer attention to their wisdom. Thanks to say thanks to AES and the AES 22 conference committee for the invitation to present and thanks to Ruth and the AES Victoria Regional Committee for the chance to do it again online. The ideas in this presentation aren't mine. They come from a variety of sources. My thinking has been informed by many conversation partners and observations throughout the conference and you know my practice in life and so if I tried to list everybody that I should think about this presentation to take the whole time that I have allocated. So instead I'm just going to say thank you to all those who have contributed. I just wanted to say something about why I'm doing this online. So I wanted to reprise the keynote in an online space because I'm concerned about equity in evaluation education. And if you go to the IC website you'll see that that's a place where we're also talking about it. Face to face conferences are great and they're great for connections and learning and they're really important to keep our professional organizations financially afloat. But they're expensive and now that COVID's on the scene it's risky for people who are immune compromised to do those kinds of events, no matter how vigilant we are about wearing masks and things. And so because of that they end up excluding a lot of people. And I know I can't fix all the inequities in the system but making this keynote public is something that I can do. And so that's why. As Ruth said I'm happy to answer clarifying questions and so you can just put those in the chat throughout the time I'm going to pause a few times because I'm about to cover a lot of content. And it's important to get the first bit so that you can get the second bit so if you have questions please do put those in the chat and Ruth will when I stop Ruth will manage those for me thanks Ruth. Have you been defeated by the closed caption. Okay, yeah, so sorry, we don't have the closed captioning feature on this. I'm not going to make any promises about that but I'll see what we can do to get that addressed when the recording goes up. So, I wanted to, with ever since I've come to Australia I've been really just delighted by how our neighbors in New Zealand always introduce themselves by talking about where they're from and geographically and also who their people are so I thought I would do a version of that about me for so you can see me here on the map I'm the stick person with the curly green hair. So just prepare yourself because this is the quality of of graphics that you're going to see throughout the entire presentation. So, my people originally came from Norway and Germany about four generations before me and immigrated to the Midwest of the United States. And then in 2012 you can see that little green line that passes through that's me immigrating to Australia on my own in 2012. So my 10 year anniversary is in November. History has a lot of farmers, pastors, teachers and homemakers you can guess which, which sick person is which of those kinds. And when I was looking back through the generations of my family and noticing the kinds of professions people had. The thing that came through the most strongly, because there was at least one in every generation was that home homemaker function that care, that person who provides care. People who nurtures people who make sure that they take care of the sick and who are present with the dying. This is a hugely important role and the reason that I can stand before you today in this online space is because there are lots of people taking care of all the people across all the generations of my family. I've probably inherited a bit of that caretaker gene and so I really see that there are powerful intergenerational forces in my family that are moving me towards positions where I can be responsible for helping other people to grow and develop. Now we're into the, the content of the presentation and so I want you to meet this stick person who is going to represent a person or an organization, all throughout this presentation so the ideas that I'm going to discuss are going to apply the same to persons and organizations, and a stick person was the best way I could think of to represent that. I'm also going to talk quite a bit about soul, and soul has a lot of meetings and I'm going to use it in a particular way here so I'm going to explain that first. Soul is very practical in the way that I'm going to talk about it it's about yourself. And so if the word soul feels strange or uncomfortable to you just think about it as a self instead. I've got identity and purpose I've drawn it here as a square, but with concave sides so like swoop in. It's orange because this outline of the squares orange because it's the fire that's driving that person or an organization, but it has those concave sides because it makes points that reach out. And that's the fire of the thing that the soul wants to accomplish going out in different directions to accomplish that purpose. And I talk about so I'm also talking about deeply held value so the things that drive us, even if we don't talk about them as individuals or as organizations they're at the core of what we do they're at the core of what we think is important. And why we're working to achieve the things that we're working for so again true for people and organizations. And this, but sometimes they're vulnerable. And so here I've drawn a more fragile soul with just a dotted line instead of a solid line as individuals, we don't want people messing with our souls, unless we find them quite trustworthy. And organizations are the same. So you know this is the core of what's important to you. It's about your activity it might be about your research discipline it might be about how you think about evaluation. But however that is, it's going to feel pretty precious and if you're not very clear about what it is, it will probably feel even more vulnerable. And that's what that dotted line so represents. So here I've attached soul to our stick person entity to show how they fit together so you can see the soul is sort of like a cape. So the orange was pointy bits attached to the arms and the legs and they align with that. So it's sort of shows the energy of it with with the context of that sick person. And again it can be that solid line or that dotted line, depending on how sort of clear someone is about what those core values are. So when you add evaluation, which I've represented here with a little green checklist in the middle of the slide into an organization or as a human few had evaluation into our lives. Often you get anxiety and I've drawn that here as a big red a in a spiky circle so the spikes are pointing out because anxiety is pretty prickly business. It's normal. It can ramp up and then it gets to be really unhelpful. So whenever we add a person and their soul or an organization and their soul and evaluation. We're going to get anxiety for a couple reasons one is what I talked about already souls are pretty precious and we want to protect those. And here's because evaluation activity takes away resources from doing soul activities. So if I have to spend money on evaluation and I'm not spending that money or time or other resource on the things that my orange soul is driving me to do so it's an opportunity cost that's real and having to do it can make people anxious. So if the organization's purpose and energy is dedicated to helping vulnerable populations and I've shown those here as, you know, a personal disability, perhaps someone who is elderly and aged care or children. There's even more anxiety and I've shown that with lots more of those anxiety spiky bowls, because vulnerable people are really important and it's important to us to get care for them. Right, we want to do that well. And so the anxiety around that is always increasing. I'm going to talk a little bit about how this works in a system assistance perspective so when there's anxiety in the system. Reactivity is the immediate response. So we have the anxiety ball here at the top. And at the bottom I've drawn reactivity as a tornado or a cyclone, depending on where you're from in the world with a tiny little fish next to it. It's a symbol for what my family calls the frenzy piranha. So reactivity doesn't think it just reacts and it does a lot of stuff, because when you're anxious, sometimes doing a lot of things will feel calming to start with. But ultimately, what happens is it just makes more anxiety so you can see here I've attached that spiky ball to the anxiety ball with another closed arrow to make a loop. So the way that this gets diagrammed in systems land is you have to think about how those two things affect each other so if they're connected by that loop. You have you can have a plus sign at the end of the arrow or a minus sign at the end of the arrow, and you the plus doesn't mean more it means same direction, and the minus doesn't mean less it means opposite direction or balancing. Plus either they all go up, or they all go down. And with minus they go opposite so that it balances out. This is tricky because we I least me as a person who grew up with Western math plus means more. So it doesn't in this case. So the anxiety slash frenzy piranha loop that I've been drawing here is a reinforcing loop and so you can see that from the plus signs. So that means they increase or decrease together. So if you've got anxiety that's going up means you get more frenzy piranha, which means you get more anxiety, which makes more frenzy piranha. And so I've just made both of those symbols bigger to show how that increase reinforces and makes bigger and bigger of both those things. So this is, and I've shown that here on the right hand side of the slide that reinforcing works the other direction so if something happens that allows anxiety or reactivity to come down. Then you'll also get less of the other so less anxiety, less frenzy piranha, less anxiety, less frenzy piranha so we want to we want to make that reinforcing loop work in our favor, but it takes a little bit of shenanigans to make it happen. But when we add evaluation into this loop. We're also often just adding in another reinforcing thing. And so, perhaps you've seen this. I know that I have that increased anxiety increases reactivity those frenzy piranha, and then increases evaluation activities as a reactive response. So when this happens you get, like, would talk to an organization recently that had 1240 indicators, right, huge. That's a, an untenable amount so you get things like that hundreds of indicators, heaps of constructs themes variables of program logics, lots of program theories, millions of dollars and hours and probably days or years of time spent internally and externally on data collection and analysis. And that times invested not only by the organizations that are doing the evaluation but by the people that they're intending to serve. So everybody is making a huge contribution to what may in fact be frenzy piranha activity. And so what happens is that flurry of evaluation activity is reducing the resources that are available for soul activities. And so, even if it's intended to be protecting of the soul ultimately it has an effect of draining resources away from it. And that draining of resources will also just increase anxiety so you can, I've added in here these many evaluation checklists to show that in the circle. There's a number plus sign there to show that everything increases together. So this is a pretty, pretty significant problem and I think my hunch is that in 70 years since evaluation became a discipline that was going to help tackle the world's problems. Not much has shifted and my hunch is that the reason that there's so little change when we have learned so much about evaluation and how to do it and make it useful. And we're in the middle of this frenzy piranha anxiety cycle, and we're increasing activity without necessarily increasing benefit. And so what I'm going to talk about today is what happens if we reframe the situation from this perspective, but that was a lot of concepts. Oh, one more thing. We know organizations exist in emotional fields that can be riddled with anxiety. And so here we've got our person our entity with the soul, and they're surrounded by the spiky balls of anxiety. And that puts we know that puts them in the middle of this frenzy piranha anxiety evaluation loop how do we respond. And I'm going to suggest that our responses to deal with souls to deal with evaluation and anxiety, our response could be maturity, instead of just metrics and so here on the slide on the right hand. I've added in the symbol for maturity which I'll explain more in the next section. And I've also had a question mark because we want to see what happens if we put that in the in the system so do we get a plus sign already do we get a minus sign at the end of those arrows. So the maturity going to a thing that just keeps increasing the anxiety and the evaluation and the frenzy piranha is going to be something that kind of helps balance things out. Now I'm going to stop. And if there's questions in the chat, because that was a lot of big ideas in a short amount of time so please ask if you don't understand something. Is there's no questions yet I mean but perhaps we can pause for a moment or if you if you have a question pop it in there don't be shy. Please don't. It actually makes me nervous when no one has questions. Because I'm pretty sure I don't explain things that clearly. Oh thanks Liam is personally in gross thanks when I'm. Hi Liam thank you. Alright I'm going to go. Assuming that we're. Okay, flag me Ruth if something comes up that needs to be clarified. So next let's talk about what is maturity. So maturity is based on the entity that person or organization or sick person and how they function. I've represented it here as a the stick person inside a bubble. And so you can see the arms and the legs of the person and their head like they're filling up the whole bubble. And that's because responsibility is the ability to hold tension. So it's about the person or the organization holding their own tension so there's things that they're worried about or they might need to do and they're managing that themselves. But they're not going outside that bubble they're staying in it they're responsible for their own emotional being and destiny and so that's why the circles quite close right they've got their own space covered. They're not small they're not big which I'll talk more about in a second. They're not holding their own tension and they're not holding anybody else's. I'll explain more about that as we go. Also remember that this is really about soul and protecting soul and ensuring that that soul can do what it needs to do right so that's that fire that's pushing out. There's a soul over the person in the entity but there's about to be a bazillion different stick figures and colors and stuff so I'm just going to take that out so it's one less thing on the slide to make it so busy. But I just want you to remember the soul is always in there, even if I haven't drawn it. So, how can we understand how maturity functions. So here we've got our mature person or organization in their bubble in the middle of this slide. A field of anxiety spiky balls. And some of those spiky balls are sticking into the bubble of our mature person. And so anytime anxiety starts poking either us as a person or poking organizations, a couple things can happen. First, the first response can be over functioning so in this case the personal organization stretches responsibility out past their bubble and gets into things that are really other people's responsibility. So it's great to take on a role to help other people manage their, be their own being in destiny. So I've just drawn that as this person whose arms and head and legs are sticking outside the bubble. And let me just give you some examples of what this looks like so classically it's international development organizations who think they know what other countries need and then they try to provide it by bringing in their own team to manage the problem. In the house, it might be parents who are chasing up kids to do their own schoolwork. It might be bosses who micromanage or to take responsibility. Like they've delegated a project but then they take the responsibility back. I'm certainly guilty of that from time to time, or often I can be evaluators who tell clients how to make decisions, or implicitly, like not with without sort of naming it they impose their values into the evaluation process. So this is about any time you're sort of out of your bubble and trying to solve other people's problems that really belong to them to solve. And that's again personal or organization. Any time the anxiety or stress goes up and I've just put more anxiety bubbles in here. And those spiky balls and the more they get stuck into that entities bubble. An overfunction or will respond by getting even bigger by over functioning even more and so here I've made the person larger to show that really at this point the only thing that's left inside the maturity bubble is the middle of their body. The other thing else is outside the bubble, and as a person who over functions as a default I can tell you that this is not a good scenario for anybody. So that's the overfunction. Under functioning is the opposite so this is any time a person is in an anxiety field, and they get those anxiety spiky balls stuck into their bubble. They become less responsible for their own being in destiny and so I've drawn that here with this person who's small inside the same size bubble. So you can see there's a big circle of white around them and their arms and legs have a lot of space between where they are in the bubble, and where the bubble is. So what does this look like in life so in an organization it might be that they're passive. You know they're in a sector that needs to grow but they don't want to take a risk. And so they're avoiding conflicts that would help to make that happen. It can be in communities that allow others in to define and solve their problems rather than figuring out what they need to do on their own and using their own resources to do that. In marriages that sometimes looks like somebody who makes all the decisions and somebody who lets them but then is grumpy about it. So we don't want to do their homework is another classic employees who don't want to do their work or just do it badly, so that someone else will pick it up. And in evaluation I think or research it's quite often we have. We have this idea that we're going to go in and bring our questions to the communities and then take the data out, which we say is to solve the community or problems or to help them but we don't engage with that community in any way we really are staying well inside of our bubble, and not interacting in a way that would actually be beneficial to the community. So, again when the anxiety goes up with the, which I'm showing here with the more extra spiky balls. The under functional will get even smaller, they will respond by even by being even less responsible. As you can see I've made the person tiny so it's, it's passivity it's not being connected. You can't connect anybody if you're not touching the edge of your bubble. And you're expecting the under functional will expect that someone else will pick up the slack when there's a lot of anxiety. And often the right because the world is full of under function or so we're just looking for opportunity to do that so they don't have to manage their own stuff. So we've got maturity in the middle. And with the person who's just touching the edges of their bubble, the under functioning person is on the left hand side with a small as a small stick figure inside of their bubble and the overfunctioner stretches out beyond their bubble. And just a reminder that the orange is that soul. And based on reading, not just me thinking is that the those are the under function and the overfunction are probably a bit less clear about what they're about. So that's why they have the dotted line of soul instead of the solid line that the mature person has. And so all this again can be true for people and for organizations. I'm stopping again for question. No questions yet just positive comments. Maybe who's loving the personal and human reflection rather than an academic reflection on evaluation. Well, whenever I've talked about these concepts in an academic way people's eyes just glaze over. So I thought right let's let's get out the stick people there was more fun for me to. Okay, so let's talk about what maturity looks like. It has three parts focus on self staying connected and getting curious. So I'm going to talk about what maturity person with their well defined soul, filling up their bubble as they should. This slide's about the first principle which is focus on self and managing self so here what I did was just make a mirror image copy of the person in their maturity bubble. And I put a magnifying class between the two of them, just to show that that's a focus on self it's that self reflection and self examination. And that's when the anxiety shows up which you know we got the spiky balls of anxiety and we even have a little frenzy piranha inside the bubble of the mature person look out frenzy problem. That means when that all starts to happen and this is again true for organizations and people that you manage your own frenzy piranha, you figure out how to deal with your own anxiety, and you just keep focusing on your own being and destiny. You just really get out that magnifying glass and you look at yourself and you think about okay, what you know what's coming at me, how do I manage this, how do I stay responsible only for my own being a destiny. And you're always sort of monitoring that so that's an ongoing thing, because the only way to keep from getting too small or too big is to really pay attention to how well you're functioning in that space. So that's thing one. Thing two is about the set the set slides the second part of maturity and that staying connected so here we have the mature person obviously this is us right we're the mature person in the middle. We are just touching the outside of our bubbles but our bubbles are connected on the right and the left to the person who's under functioning and the person that's over functioning. And so this shows that second principle of relationship and connection. So those touching bubbles are us interacting as a mature person with these other people. And it's really important to do that particularly with people who increase your anxiety. Perhaps you have some of those because it provides a really great laboratory to practice staying in your bubble. So it might not be pleasant but it's really good practice. I don't like it, but I get to do it a lot. So when there's anxiety around what what do you think will happen, obviously the dang over functioners are going to try to get in your bubble and solve your problems right they want to do things for you, because they don't want to deal with their own stuff. And the under functioners are going to get smaller and smaller and they're going to try to lure you into doing things for them. So here we've got the anxiety around and your frenzy pranas with you inside your bubble mature person, but your goal is simply not to feed your frenzy prana even if you have a little one in your bubble, right, they're not your friends. Don't feed them and make them a pet. So this slide is about the last part the third part getting curious. And so here I've got you again obviously that you're the mature one in the bubble. Large rabbit ears, a magnifying glass and some question marks and all of those three features are both inside and outside your maturity bubble. This is on purpose. So you're staying in your bubble, but you've got your antenna out, you're listening, you're watching, you're observing, you're asking, so that you can learn as much as you can about the situation about yourself, and about the people that you're connected to. Anytime anxiety goes up. So our anxiety spiky balls are back in you in this space where you are, or in the people that you're interacting with are connected to, you just listen and look harder so that's that combination of self reflection but also really intensely interacting with what's happening around you. So, to show how that increases I've made the bunny ears and the magnifying glass and the questions double the size right so anxiety happens and you just get more intensely curious. Instead of letting feeding the frenzy prana or dealing with that increased anxiety that way. So on this side we're back to the mature person in the middle, bunny ears, magnifying glass questions. And I know that they look a little bit silly here when they're attached to the perfectly normal looking overfunctioner and underfunctioner. But the advantage of this is that when that person or entity of any kind organization can stay in their bubble can manage themselves can stay connected and can get even a little bit curious. The anxiety floating around is going to reduce and I've showed that by the anxiety balls, getting smaller. And it's all of them. It's not just the ones that are stuck into people's bubbles. Yours or other people's. So that's the magic of how systems work. If even one person or organization can change their functioning just a little to manage themselves to be curious and to stay connected so only three things. It will affect every entity in the system, even those that the that you as that mature person aren't connected to. So, if there's somebody for instance attached on the left hand side of the underfunctioner on the right hand side of the overfunctioner. And they're feeling anxious, their anxiety will also go down. So what if a response to dealing with souls evaluation and anxiety was maturity instead of metrics so I've just reprised that slide that has that mature person in our loop of anxiety and frenzy piranha and evaluation. So, and I've left that as a question mark. But I think what we can say is that maturity does balance the system. So when anxiety goes up and maturity also goes up, then the frenzy piranha of reactivity goes down. And because the piranhas and the anxiety go in the same direction, if the piranhas go down, anxiety goes down. So it's a way to balance things out instead of just all the time increasing. Still no questions, Amy. So feel free to go and have a drink while you think about writing. Actually, we've just got one from Liam. And I'll repeat it for the recording. Liam posted I'm thinking about how applicable this thinking might be in terms of aspects like developing leadership. Is this something that you've thought about? Well, not me. I mean, yes. But there's a great book in my slides at the end. There's a book called Failure of Nerve by a guy named Ed Friedman who talks about this stuff exactly in the leadership context. So you could go read Ed, which I would totally recommend. And yes, it is usually important for leadership. Yeah, just a couple more, Amy. How would these principles of maturity apply to organizations, James has asked. Oh, hi, James. And let me talk a little bit. I'll talk about that next, and then I'll come back to that question, James, if you feel like I haven't answered it. And just one more from Karina, which you might touch on more as well. She says, don't you still need measurement but less driven by anxiety and more driven by curiosity. It's like, you're prompting me to talk about my next section Karina. Thanks. Anything else? No, nothing else. Okay, perfect. Karina, you're about to get your question answered. So what does this look like in practice? Hmm. So here we have our mature person in this case, it's an evaluator, obviously, because we're talking about evaluation practice but you can replace that with any other kind of person you'd like. Since this was for an evaluation conference I'm talking about evaluators. So what have we thought of our job as evaluators as helping organizations clarify their values and purpose. The reason that they exist and so I've shown that clarification is that that move from the soul with the dotted line to a being a soul with that clear, really strong robust line. I connected them with an arrow but look it's not going to be that linear it's going to be one of those things it's like a ball of hair in the middle where you sometimes get it and sometimes you don't so that process of moving from a dotted line to a solid line and getting that clarity is going to be a bit messy but still worth doing. So then when as evaluators our questions and observations are going to be focused on how that process is going of getting from lack of clarity to clarity. And so I've added in a question mark and a magnifying glass between the dotted line soul and the solid one to just show that we're engaged with that and where our questions and observations are about that process. We'll be listening all the time, because what we want to hear about is what they value and what they think is important. And we also want to reflect that back to them. So we hear it and then we say things like do we hear that correctly. And do you really mean that because sometimes people will make statements and then they think about it and say well you know turns out maybe there's something else that's more important. And so, in this slide I put the bunny ears in between the dotted line and the solid line soul to just show that extra listening. So our observations and reflections with them on soul could also help them identify where they're over functioning or under functioning and so here I've just put both those representations of the over function under function in here because the odds are as an organization that that's probably happening that there's one or the other, and asking about it and observing about it and, and being curious about it as a way to help them reflect on it. We would also be observing inquiring and listening to see where the level of anxiety is in the organization and in their sector and what kind of responses are happening so they can see a frenzy piranhas are on the scene, and how much frenzy piranha activity is happening. So here we've got that progression from the dotted line soul to the solid line soul and I've just put that in the middle of the anxiety frenzy piranha loop with us again as the evaluators on the outside. We're observing that carefully and reflecting that back to them. When we put ourselves in the system as evaluators as this non anxious presence that's managing ourselves staying curious staying connected. We can help the system calm down so that organizations can be clear about what their work is, and then focus on it. So here I've just, again put that arm, the mature person into the cycle. So they're balancing things out, calming the system down, and making it possible for that soul to develop a bit more clarity. When the systems calmer than we can thoughtfully bring in all the other stuff. Karina that you mentioned so you can think about evaluator competencies, one of the things that we know that that can be useful to them. It's heaps of tools I think Michael quim patent just did a thing that found 100 evaluation approaches or something. This might be a sign of frenzy productivity, but that's a different presentation. So we got all kinds of things that we can use, but we have a better chance of using them when the systems calmer because we'll be able to make better choices, and the organization will be better able to say oh you know what this is what we really need. So that when the energies calmer everything can just proceed with a bit more clarity. So when we're functioning this way as evaluators it also might mean that the accuracy of our maps and our program theories might be less important, and they also might be less linear. But kind of that classic program logic and the bottom of the slide with the boxes and the arrows. And next to it I've just drawn, you know, all the same stuff connected to each other and all kinds of ways to show the difference between those two things. This messier system that's on that left hand bottom corner is really a rough diagram of how things work that you can update as you go but it also is focusing on observation and I've put the anxiety spiky ball and the frenzy piranha in the system because we want to be paying attention to that. That's really important information about where people feeling anxious where is the organization feeling anxious, what kinds of activities are happening that might not actually be fruitful in terms of pursuing the organization's, you know, soul, their purpose. And so, if we stay in that observation role we can also use our tools in a rougher way to help do that. So rather than investing in making a perfect program logic which we all know doesn't exist. And trying to persuade clients that they don't need a perfect program logic. We can just say look this thing's going to be messy and let's just keep it messy, because that rough thing will make it easier for us to change things around. It's also going to change the questions that we ask and the kinds of data that we'll collect so here I've put in all of those checklists little icons. But I've connected them to the soul, with some arrows so what we're doing is we're collecting data but we know what we want. And we know how to make better sense of it, because we've decided what's important and we've decided that based on how it relates to the soul and the purpose of the organization that we're working with. Helping them do their own thinking is also really important. So our job here is to provide expertise to them to figure out how to track how they're going to make sense of the data that's coming in, and to manage their own selves in terms of their bubble. So we're going to help them get clear about the core of their being and doing so our for our role it means that we can potentially be more focused, but not in a develop a thousand indicators way. But in a what do we pay attention to which is how well as the organization staying in its bubble. What does the anxiety look like, what kind of frenzy Prana activity is happening. What is it that they want to know and, and do what are they existing for what's the fire that's driving them. And so when we're in that space and the organization's getting that clarity about soul, then integrating into evaluation into their work to figure out how they're going. It will help understand what their relationships the products and outcomes they're trying to create. And so I've shown here I've just moved all the stuffs over to them so the organization itself has the evaluation in their hand that little sign for metrics, and they've got the messy program logic and the other hand because this is their stuff, and we're just helping them work through it. So I'm not going to ask questions because I'm almost done but I wanted to just finish by saying, so this is a completely different way of thinking about how we do our work as evaluators. And why should we bother with changing this. Is it that important. And this slide is intentionally left blank so I'm the middle of this year. And I was reflecting on fundamentally how hopeless I felt as a human. So we've had a plague or two if you count monkey pox, and increasing evidence of catastrophic climate change which is affecting the most vulnerable people in our world. We got nationalism like I'm from the US we had Donald Trump. We got instant news cycles of a population issues. People cities that don't have water. We've got clear signs that governments are failing and social systems are not working. We're deluged in data and in maps but it feels to it felt to me like we didn't know who we are or where we are and I, it made me feel really hopeless. But then I was reminded of Western world history in the book that I mentioned, Liam, the Ed Friedman book. He talks about how in the 1400s in Europe, they'd had the plague feudalism nationalism and then all kinds of wars happening between these little feudal states. And the slide here is a map from the 1400s. And I think the title that usually gets put on it is here there be dragons. So it's simply not safe to leave your little corner of the world because if you walked outside of your sort of small geographic area you would certainly encounter a monster of some kind and probably not survive. These are the same maps that had the world is flat and boats like falling off the edge into the mouths of monsters. So they're in the same kind of mental place I think that we potentially could be right now. And yet from that era, it launched the Renaissance. So art. People who are going out and exploring, let's be clear the colonizing bit was terrible, but the spirit of the time was really important and that was adventure, creativity, curiosity, energy. And I think we're at a similar moment now. And evaluation and evaluators can help kick off a Renaissance if we can shift our paradigm to this thing where we understand how we function in the system in relation to anxiety and frenzy piranhas. And what our job is, is to focus on souls and to help them make that progression from on clarity to clarity, and to help organizations clarify their responsibilities, their purpose their values. And if we can do it by working to stay in our maturity bubbles, be connected and be curious, then our presence in the space will create clarity and reduce the energy spent on frenzy piranha activity, and set it free for adventure and exploring. And so I've replaced competencies in this diagram with the boat, a brown boat and a blue lightning bolt so the blue lightning bolts energy and the brown boat is exploration. And so if we walk through the cycle again you can see. So if anxiety goes up maturity goes up frenzy piranhas go down, and then energy and adventure go up, and then anxiety goes down again. And I just also want to say this isn't going to happen overnight. The maturity bit. I'm now 51. I've probably been actively working on it for 20 years, and the progress is slow, I can tell you. So is for us for the organizations that we work with we have to just keep in mind that this isn't a fast process. And so I would suggest that we need to be a little bit more like my Norwegian and German German farmer ancestors. We need to do our work. Just keep doing it be faithful and watch and see what happens, and then we might need to do some weeding, but no one's ever made a plant grow by pulling on it. You just got to be patient while it does its thing. So every time we work to stay inside our maturity bubble as evaluators to be curious to listen to focus on values and soul what's really important for those organizations. We're reducing anxiety and making a bit more space for energy and curiosity and creative responses to the challenges of our planet, and all the living things that are on it. And that's I think how evaluation might help save the world. Thanks. I think the slides are up, but right you can always come study with us at Melbourne we'd love to have you. And I did at the end of my slides I've got the Ed Friedman book and other things that you could read. Thank you so much. I really enjoyed that. I'm sure others have to and it was a couple of quick hands as well so thanks. Thanks for that. We do have time for comments, reflections and questions so please feel free to post in the chat or perhaps we could even do the raise hand function now that we're towards the end so if you would like to just say something out loud. Please feel free to raise your hand and I'll call on you. Please feel free to raise your real hand. I couldn't see the function so I just had to go the old fashioned way. I mean that was just amazing. I heard that at the conference and found it absolutely probably the best presentation I've seen for a very long time so thank you so much for that. I just wanted to ask you, I mean in the new world it sounds like it's really about a mature person who is a mature person who's listening and curious and I'm just thinking about the traditional role of the evaluator. I mean it's sort of, it's sort of suggesting that it doesn't have to be the role of the evaluator is to somebody who has traits of curiosity and listening and maturity. I just wanted to run that past you. So anybody can be mature. And I think my impression of how the spaces we end up in as evaluators is that organizations have anxiety around a thing. And one of the ways they deal with it is to shift it to a consultant of some kind, which is often us, or if you're an internal evaluator, then the organization will try to shift that anxiety to your function. Instead of owning the work. And so how we manage ourselves in that space makes a huge difference to whether the anxiety goes up, and whether they learn actually how to manage their own anxiety. I think that's for me, it's fantastic when you've got a mature leader and organization all of the literature around that which is what's in that Ed Friedman book is a if the leader can function in a mature way that's clear it actually calms the whole system down in the organization so it's fantastic. But we're not counselors right or therapists. We're just people who know some things about data and so working on ourselves is really I think the way that we can, can do that because quite often if you function that way it's also catching and people become more mature kind of by association because our bubbles are watching. So Stephanie's hand is raised next to me. See Stephanie Bowden. Hi, I am he thanks for that that was really interesting. I was just reflecting as you are talking on the two layers of an organization soul versus the actual individuals that I mean particularly an external evaluator is interacting with an inner sense. In fact, I can, I can see the argument or the usefulness of conceptualizing the organization soul but at the same time the organization isn't a thing like it's just people who are not necessarily going to have all the same. You know I've often worked with people whether that's a sort of handful of key individuals that you're working with on an evaluation or another project all have different values and vision and black a huge amount of the difficulty comes from the fact that they're creating a lot of anxiety and tangle. So how does your kind of recommendation work on that level when there is no like one soul to kind of help people and cover. That's a great question. Thanks Steph. So I think part of it is acknowledging that that's a thing. You know we all have different values in the space if you're in that kind of team. And to, to call that out as a something that's actually impeding progress and to talk about what that means because I think quite often, we tend to function around that, as if it's not real and it's not having consequences and so I think that's something actually that in our role as evaluators we can do to say the evaluations about what's important to you and it seems like we got a whole mix of things that are differently important to different people. So can we put that all out in the world like can we have a conversation about what's important to you and why so we can see how those things relate to each other because I find a lot of folks that have been doing like thesis research and page of varying, varying kinds are in nonprofits, and so maybe they're a bit more, you know, people are in that space for a particular reason I think it might be more challenging and organizations like government where that's not necessarily the case. I think that value conversation still stands is really important because if you can't you're always going to be spinning in that anxiety frenzy crown up place if you can't talk about what's important and why it's actually causing people to do things that are making more anxiety. Thank you. That made me think of the stuff you've written about making values explicit and just contemplating the kind of danger zone between facilitating conversations between people in conflict to make their different values explicit and getting just drawn into this massive interpersonal conflicts within an organization. Well, and part of that for us I think is having a clear sense of that's in our like what's in our bubble. You know, where can we be helpful in observing or reflecting back to them what we see, and where is this outside of our space. And we might say look you need someone who does mediation or conflict management or whatever. But what we can do is surface that that might be needed. But we need to be the ones that do it and that's part of us staying in our bubble we might observe that they've got my goodness. That's a pretty significant conflicts in here. And if you can't work those out then you're not going to be able, we're going to be able to do the things together that you want to do. So, you know, it's because if we can identify that. I mean as consultants right that's nice for us like we can just keep making money while they don't work their stuff out. Fundamentally, I think our ability to observe and tell them that reflect back to them what we're seeing is really important. Two more. I'm sure going to you, Erin and then any giants. Sure. Thank you Amy for the presentation it was very interesting I didn't get to go to the conference so thank you for making this available to us. I was thinking about anxiety and frenzy piranhas and how many of those I encounter and then also my own frenzy piranha and anxiety, which I think it also, you know, feed into the whole, you know, the whole maturity thing really is about our own presence and how we, you know, it, I guess introduce ourselves into organizations as well we're working with. But one of the things I see a lot I'm sure you do too is that a lot of this kind of circular effect, you know is really fed by the way, particularly nonprofits are funded to deliver miracles in a very short amount of time. And so I see a lot of like, you know, even metrics. Before I go in, you know, starting to design evaluations with people that have already been set up by the funder that have, you know, for the very high expectations and then the people I'm working with are like pulling their hair out going how am I going to do this and we're trying to set things up so that they can, you know, demonstrate their achievements but knowing already that they may not. So I'm wondering how we, you know, kind of use these concepts in that kind of environment where, you know, people are kind of almost kind of set up in a way to just feel like they're going to fail to start with it. Well, and not only set up to feel like they're going to fail but probably to fail. Yeah. Yeah. So, again, again, I think that observation is really important to say that. You know that this this ability to look at the thing and say, this is what I see does that feel like that to you is probably one of the most powerful things I would like people to take away from this conversation. Because it's saying that right you've named the thing. You know, so when you name it then it has less power to sort of make everybody crazy and then you can think about okay well how are we going to deal with this. What kind of flexibility do we have with the funder can we go back to them and say look, you know, we have X amount of dollars to do this thing. So how much of it do you want us to spend on doing evaluation stuff and how much of it do you want us to spend on this other things that are more important or not. And if they won't play ball, then you, you make those decisions. I mean it sort of creates a sense where people can be self defining, or the organization can be self defining about what's important. And it will help them navigate that because it might mean that they don't want to take money from that funder again in the future if they can help it. And it might spark looking for money elsewhere that could be a bit more flexible so I think it's part of its opening up options, instead of just spinning in the circle so anything you can do that can increase the number of potential responses to a situation will also help decrease that reactivity, because you're not thinking that we have to do the things, all the things you're thinking about okay well what if we can't do all the things. You know then what else can we do or other ways to do the things that aren't going to be so resource intensive. Does that help. Does that feel real as a useful. And I think it is helpful I mean I think the approach is very very helpful I do think that, you know, a lot of these organizations don't have options. And so their funding is limited, you know, even you know to, you know the government agency that's I think that's something that they are completely reliant on that. But I think what you're saying is important about opening up these conversations, even, you know, we know with the client or with the funder or together around, you know, what is realistically achievable here in a mature, you know, in a way that is, you know protecting what is really valuable here in the organization. And this other thing, sorry James I promise I'll come to you. I think the other thing is the connects you potentially, I feel like nonprofits often are kind of competing with each other because you're getting right you're fighting for money from the same kinds of places. But if, if the, if you all could connect with each other and say are you struggling with the same things we're struggling with. So what would a good solution look like, you know, for the your sector or that particular kind that's another way to open up some possible responses so if you think about connection as a way connections that you haven't made. And you can do that in a way that you know everybody feels kind of safe, because if you know you're not risking each other stuff but you're saying look we're really struggling and usually that means somebody has to go first. I would say this, you know and it might be in casual conversations because you know somebody in another organization that does the things I feel like nonprofit people kind of move around from place to place so there's probably a lot of personal connections that would make that possible we could have that conversation without having to like make an announcement, you know, my organization's having a shit time with these particular indicators or whatever but thinking about those connections can be really in thinking about possible responses. James, I know we're a little bit over but I don't mind. No problem so thanks for that Amy for a really good presentation it made me reflect a lot on what my role is as an internal and an external evaluator. I guess I'm just coming back to my original question in terms of what organizations can do to build maturity themselves because I'm kind of thinking that it's up to us as evaluators to actually clarify their maturity. But I also feel that if organizations can also build their own maturity themselves without supporting them. I think that might be even better. Yes, I agree 100%. And I think that it really depends on thinking about what kind of if you're the evaluator internal or external what sort of access do you have to leadership to have a conversation about this. I certainly don't think it's our responsibility on our own. I just think it's a thing we can help with. But it's really the organization's job, right so it's us staying in our bubble and being curious about it it's not us getting into their business and trying to be their organizers or their value articulators. Yeah, so there's, but I think talking with people in your organization about the difference between what a mature organization could look like and what a less mature organization could look like as a way to start that conversation. Okay. Yeah, that's a good idea. Thanks for that. I mean I'm just saying a lot of people needing to head off. Are you happy for people to email you if they've got a great account. Yeah, I think my email is on the slides. So you can certainly ping me there. If you've got questions. Oh, look, Ruth, you're already ready of course. I didn't want to post it if you just agree. Hi, Charles. Oh, you're muted, you're muted. Thank you for the presentation really enjoyed it. So some autos have been related evaluation with power imbalance. And so I wanted while you're taught around how the concept of anxiety and evaluations related to pie and balance if power imbalance, kind of promotes that whole system of anxiety. And also how does maturity help to in a suppress this by balance within that evaluation space. Not a small question for first thing in mind, but a really good one. Thanks Charles. So the overfunction or underfunction or thing for me is the way into understanding that one of the ways in. So if you think about power and balances. I'm not saying they're not real because they are. And, and I think what Aaron talked about with the nonprofit funding is a great example of that like people have money and nonprofits need the money and so you're in that space of trying to figure out what to do about it when they have unreasonable expectation. So if you're a person or an organization and sort of a low power situation, understanding what work is your work and understanding what work belongs to other people can be really helpful. Having some clarity about what you can also be really helpful. So that working on defining of self for a person in a low power position will often lead to things. That's why I suggested you can start to think about. Okay, well if I don't want to do this thing that they're requiring me to do what are my options. What you know what are the possibilities I have to sort of navigate around that and then you start to think about what are the resources I have to build on what are the strengths that we have as a community for instance or as an organization that would let us be able to do what we want because we're clear about what that is, either without the funding of that person or to use to get clever about how you do the thing within the rules and the rules that of that funding organization, for instance. The focusing on maturity of self and means that you start to have more possible responses to that power situation that can help diffuse it, but it will not be a simple task, which you already know. Thank you. So thanks for you I mean the chat. I really would love to like this, Aaron and Charles both your questions are really important and so I feel like that's the sort of thing that gets worked out through connection with each other. You know how are you tackling this how are we thinking about it, you know, what things have worked for you what things haven't. And the connection with people who are in the same space can also be really important to think because it just helps you see more options if you've got colleagues who are doing facing similar challenges.