 Kia ora kato katoa. Welcome everyone to innovative approaches to mentoring by Dr. David Turner. This seminar is recorded and it will be uploaded into the AES YouTube channel. I'm Marina Sanko, your co-host, and before we begin, I'd like to acknowledge the traditional custodians of the diverse lands in which we all come from. I'm speaking from Wellington, Aotearoa, and I acknowledge the leaders, past, present, and emerging from all of our lands. Today we are very lucky to have Dr. David Turner talk to us about the mentoring pilot. David has practiced research and evaluation across the New Zealand public sector, and he's a member of both AES and ANCIA. David also very kindly offers to volunteer for pro bono research and evaluation services for the NGO and community sectors. I will monitor this chat function for any questions you might have throughout the seminar, and we will have time for you to unmute yourself and ask the participants questions, any questions you have directly. So, before further ado, over to you, David. Right. I just want to note that to my fellow committee members, Julie Elliott and Jill Thomas are also part of this. So, let me just share screen because I've got a here we go. And we're good. This is about a program that that we developed and ran last year, called group mentoring. It's intended to be an ongoing service offered by AES to members. We have a working group that pulled it together and is continuing to do it. Julie is here, me, Francesca, and David Roberts are not present, but Jill Thomas is here. I just want to mention that I'm intending to not take up too much time with this briefing I'm hoping that there will be some discussion and questions and ideas. A brief description what I mean by group mentoring. It's an online program. It's just, it's just through Zoom. In each case, and one experienced practitioner acts as a mentor, you have six people taking part or less experienced. Last year we had, we had all AES fellows doing it. The groups meet monthly. And one aspect of this is that, you know, it's entirely up to the discussion of the individual members. They can communicate outside share ideas or whatever, as they choose that's that's up to group members. In each case, the groups are expected to work out their goals, their interests, what they'd like to get out of it and feet and what they'd like to get from the mentors. Last year we piloted this between January and June. We had four groups meeting online. And because this is an evaluation society, of course, and we advocate better evaluation evaluation. We evaluated the pilot, we had capstone students from University of Melbourne their graduate program evaluation, and the pathways committee was supported it as well. I'm going to talk some today about the results what came out of that evaluation, which we hope to, which we plan to publish. The evaluation drew on mentors, it drew on mentees, it involves surveys, focus groups, a number of, you know, different approaches and methods, and it made recommendations for the future of the program, including the recommendation that it should continue. I'm going to talk a little bit here about what evaluation found from talking to participants mentees. People noted that they that they value the chance to engage with peers, not just mentors. So, so there's a bit of a community of practice aspect of this. And looking for common ground, sometimes the different people in a group. You know they had things in common like the sector where they worked or whatever. Generally, people express satisfaction with the process of matching them to groups and individual mentors. And part of way that works is that the mentors and doing this this year. Right up short bios about themselves so people can express interest based on what they know of someone's background and what they have said they're interested in in talking about. One of the issues in the evaluation had to do with setting goals. The process, the way it seemed to work out is it didn't value that much goals like technical knowledge of evaluation. They did go through a process but that didn't seem, you know, specific learning objectives didn't seem to drive it that much. People like the online approach. The mentee said that they through this process they were coming to recognize their own expertise. Better understand the knowledge and skills that they already had and and reported increased confidence. There was some reports of better knowledge of evaluation theory and to various soft skills involved in evaluations such as stakeholder management. Now, I also talked to mentors and some of the stuff that came out from the evaluation at the mentors. They talked about knowledge building rather than specific skills. They thought the opportunity for their group members to talk with their peers as well as them was important. Got a sense of dynamics in that you know each group had its own dynamics, and that changed over time. Some people may have been more engaged in the process at one point than another for example. They talked about the skills that they were using that the process required of them. And a couple of examples here group facilitation and interpersonal skills but of course they had to have evaluation expertise to answer questions and to engage people. They also found that that they had to be active, ensuring that participants would engage in the in the group stay engaged, engage with their peers. There are a few things that came up that, you know, we just have to consider in this year and future years. What is an appropriate, what sort of goals are appropriate for people to pursue through this through this kind of group process. What do the mentors and mentees expect and and how well do those match up. You know, challenge and in participants establishing establishing that commonality that, you know, peer relationship with others in the group. There are some questions about just how much structure people expect. How much leadership does the mentor have and how much do you just let it flow. And finally, similar to the idea of commonality it's, it's just how do you get created appropriate chemistry in the group. So the, so the group gels and works together well, where we are now. So doing the 2022 round it's, we've got mentors have volunteered and we've added, and there's an added role of an associate mentor in each group that was a bit of an experiment last year. One group tried and associate mentors to the idea is that someone can be involved one year as an associate mentor, and then maybe the next year be a mentor. And that's, that's another part of personal development for them. I say here is members will be asked the fact that is already out of date. Have been asked to express interest. We expect it to run between May and November. The future rounds may run a bit longer, because one of the comments in the evaluation is they'd like eight and eight month program and six months. So, um, that's my overview of how it works. I deliberately did not take up the time, because I would like to do what you think, and you know, Julie, Jill, anything you'd like to add. I think only that that people love the program. They universally said run it again, please run it again. Some people said they felt like they've won the motto when they got selected as mentees into the program. And I suppose it felt it really did reinforce for us. It's so many evaluators stand alone in their own organizations and feel extremely isolated. They have enormous self doubt about what they're doing. And just the opportunity to talk to other evaluators and debate issues and realize they were sharing common concerns and pitfalls and struggles was really, really big to the mentees. The mentees who involved with focus groups were very affirming of the program. Yeah, I'd agree with that Jill. But I think it's interesting in retrospect how we came up with the model. You know, it was sort of in the early days of COVID and so the online approach was quite natural. But it was fortuitous really, because what it meant was that we could bring together people from different locations who have interests in common. And that's not really possible with face to face mentoring. So the online group mentoring is for me the peer support is not just about what happens in the six months. We kind of want to see how we, how we emphasize and get it to work better so that the peer support aspect works a lot better so that people have relationships with their peers that go on way beyond the short period of the six months. So this this mentoring process is really just setting up longer term relationships. People now have peers that they can go to to talk things over. And that kind of talking aspect shouldn't be underestimated. I don't think we really kind of focus and Jill and David are both looking after the evaluation this year and we're really going to be focusing on the program logic. And what makes what makes the difference. So this as these two aspects of the peers, the peer support and developing those relationships and just that opportunity for for dialogue for talking things over thrashing things over that are of shared interest amongst the group. So I'll be looking forward to saying how that really comes through in the program logic and then when we, when we do the evaluation. So, we do feel that this is an innovative model. We've had a scan of the literature. A lot of the literature on mentoring is about youth mentoring or professional mentoring in this sort of workplace. But something like this does seem to be innovative so we think that we've got something useful to contribute to the knowledge base. I'll leave it at that but I'm happy to answer any questions that you have, including anything that some of our mentors that we've got here today. I might like to to raise and discuss and bring their thoughts forward. Thank you. There are two comments. Cara and Marty, would you like to unmute yourself and ask them directly. I was just going to say it would be getting the sense that maybe there's some been some men some of the mentors on the call here but it'd be nice just to get your personal reflections and I think Julie maybe that you've done that possibly a job but anyone else also any mentees it's always just nice to hear from the horse's mouth as the saying goes. I've got lots of other thoughts but I'll save them because I don't want to monopolize the floor. I'll save them if we have time. Thanks Cara. One of the things that we've got asked the mentors this year is to describe their approach to mentoring. On the AES website for the program, each mentor describes their focus of interest as well as their mentoring style and you know there's some sort of differentiation between the different groups but mostly they're really emphasizing flexibility, kind of participation, communication and problem solving. There's a range of things that they privilege that they think are most important for how their groups run. So the mentors that we've got here today are new to the program for this year and we hope to keep on expanding our mentor base, our sort of mentor pipeline perhaps and but actually finding mentors is not easy. People are often reluctant to be mentors. So that's something that the program has to work on as well, finding people and supporting people. This year we really hope to support the mentors a lot more than last year with providing more resources that were developed. So over the years we'll be able to develop more resources and then feed them back and support mentors so that they know that they're not working in isolation that they are part of a program. I'll just ask a follow up question Julie on that because I've watched with sort of interest on the periphery this idea develop and get rolling which was lovely to see through being connected with David. I wonder if there might be a number of people who would be friends that is and I probably describe myself as one of those who I've been around for a while but I certainly don't feel like an expert. But know that you might have something to offer to newer or more emerging ones but not always like I don't know that I would be able to particularly well wouldn't be able to articulate especially what that might be. And so where there might be a bit of a what's the word I'm looking for. A drafting process but almost a bit of a supported process where you know you can say look I'm interested you can take me or not wouldn't be offended but I'm here if someone who might sort of suit me you know my rather than it being a super formal process. Does that make sense and I don't know what what the sort of feedback slot been like so far but I guess I'm thinking to the future because it's always often you get the rush of enthusiasm early but then trying to sustain initiatives and you often have to reshape it a little bit. Yeah. I think that it might not be as intimidating a role. You might fear. I think that it's six 90 minute discussions that you're you're facilitating, and you're really trying to as a mentor, you're really trying to bring out the interests of the mentees and supporting them to be able to engage in in depth discussions. So people with experience in evaluation probably pretty, pretty good at that, you know, so we're certainly not looking at kind of one on one teaching, you know, this isn't in this program is not in competition with the AS workshop program. So it's not putting people up as as the trainer or a seminar presenter at the front of the class where everybody where the mentees listen. That's the opposite of what we're trying to do. So if tricky questions arise in in a session, some of the mentors last year would kind of take that on board as the focus for the next meeting so it's a really iterative process, and that gives them some time to prepare for that session. So you kind of don't need to know the answers to all the tricky questions at the top of your head. Dylan David what what do you think about this this issue of mentors and mentoring for mentors and I think we've tried to address that support issue with mentors through as a mentor support group who meet monthly is their own little community of practice and put in place some structures to help that the mentors share ideas. We also learned a lot I think from the pilot about structures you can use in the group online group setting that really help the group gel and and also really help the men mentees to be really active participants. So some of those sort of hints and tips are things that we've been gathering together as a sort of tool bag. But I think we also really do see that that associate mentor mentor partnering is a bit like a master apprentice partnering, and it helps. It is intended to help people who who think they'd like to be a mentor but really don't want to do that on their own yet to sort of come in and work with someone else and and have that pipeline effect but I agree with you Julie I don't think you need to be you know professorial and have the ability to just you know snap snap snap answer every question in fact I think the richness of the group is about a dialogue that happens in between at the shared community, but also the leadership from the mentor so the mentor certainly did things like share journal articles, you know, prior to meeting so they could be discussed at the next meeting, you know, they certainly had an academic strengths but they also had a really well practically grounded strengths that were really very very helpful. One of the really popular things one group did was they did a monthly by our problems so everybody in the group was allocated a month when they could bring a problem to the table, and that the group would workshop that. And that was really really popular. People got a lot out of that. And because it had to be nominated up front and ahead of time, the mentor could also take a strong leadership in thinking those as issues through or put on the table. I hope that helps understand how it has worked in the past. I think we're getting mentors on board if people are just so busy. That's been some of the feedback we've had this time around. I just like to hear some queries with the mentors from last year and found that it takes a minimum of five hours a month. In terms of attending the meetings and some preparation time and some sort of liaison time. This year we've actually got, we've been lucky enough to secure a part time project officer will be starting with the AS very soon. So that will help us with a lot of the liaison for the working group. There is quite a lot of work up front. There's a lot of resources that we're sort of at in at the moment, and also for the evaluation itself, and at the end closing the program. But during the months that the program is running, we're, we're sort of less hands on we're not doing as much active, active work. There's a couple of questions coming through on the chat. I just asked a question about people dropping out. In fact, we had really quite a low dropout rate. Jill, you might be more familiar with the numbers, but it was very low. With only about, I think out of the 50 mentees are at three, who disengaged, but it has to be said that with the evaluation. In terms of responding to invitations to be part of focus groups, we only got a 50% response rate for people who work in evaluation. It was amazing. And that we did stress it was a pilot and that we expected people to participate in it. And that was really important that that being part of the pilot meant that they would contribute to the evaluation. So that was very hard and Jill and the evaluation team had to chase those people a lot. And we didn't actually get a representative sample in those focus groups. So we've got a few strategies of our slaves to lock people in much earlier for that kind of participation. So in a sense, we don't know about why people did engage with the evaluation. And there's a profile in the participants that we couldn't engage with. So they might have a different story that we didn't get to hear. Yeah. And I read it has asked about the fact that this strikes me as a mix between online community of practice with an element of mentoring. I'd say that's, that's, that's correct. We're not a traditional sort of mentoring approach in that it is online and that the group element is really important. And it's probably more like starting a and supporting a community of practice to get started. So some of the groups us from last year is still continuing to to meet and to engage with each other as a group, as well as, you know, one to one relationships with amongst the group. That is not being facilitated by the mentors anymore. And that's kind of what we would really like to see a lot more of that these relationships are being built and and then can be self sustaining. So Farida, did you have any more questions here you say said you'd like to be interested in how we're supporting mentors. How mentors are supporting mentees through specific issues that they face. And Jill mentioned the the bring a problem that that some of the groups tried the way that that would work with the that a mentee would write up their problem. So really articulate it in a, in a page, which I think is really useful in itself. And then the mentor would have a chance to read through and prepare before, before the meeting. Issues would be brought up, you know, in the course of discussion, but then there was also this more formal process where if where some issues were given a lot more time, and people would take turns as Jill said that mentees would have the kind of the focus of attention for, you know, a period of a section of the meeting, taking their turns once once a month. We did not encourage discussions between the mentees and mentors to address kind of personal issues out of the sessions. So mentors are not on call to answer people's problems to solve people's problems. But that I would say that quite a few of the groups did set up their own WhatsApp groups. And there was quite a lot of between, you know, during the intervals of meetings, a lot of communication going back and forth, and quite a lot of resource sharing. Has anyone got a template for this? Does anyone know about that kind of stuff? So there was a bit of communication going back and forth during the month, but the mentors were only expected to participate, you know, in the meetings. And, you know, we were directing, we were directing at the end of those, at the end of the program, we were saying to mentees, please join a special interest group with the AES. And we have, you know, the AES is not going to kind of structure or acquire ongoing community of practice to emerge, but some groups did that and we're doing it for themselves. So, you know, from the AES perspective, if people wanted to continue to engage, then we were really directing them to that sort of participation that we would. One thing we have not talked about so far is that we have not yet been successful in really engaging Indigenous groups in this, in this program. We did last year have some people working with Indigenous communities who were themselves non-Indigenous. We have, we have tried, we have approached different groups to see about bringing in mentors, finding ways to engage them and have had that dialogue. But we haven't really got anything, you know, it hasn't, it hasn't come together yet. So that's kind of an objective of the program that we haven't got to yet. And I think we should just acknowledge that's a continuing challenge for us. Yeah, that's a work in progress. There are a couple of questions asking you about the themes and topics that were discussed. So we asked mentors to nominate the particular topic that they are interested in. It's really something that they sort of bring great enthusiasm to and, you know, could talk about endlessly really. And so they nominate that topic and write it up in a way that that looks appealing and then people apply to be part of their group for that reason. And then the group themselves decide what issues that they want to raise throughout the program. And so that's a collaborative participatory process that they agree on and they work through those things. And then other things, you know, topics can arise throughout the program and the mentor will say, should we talk about that further next time? And so the list sort of evolves as time goes on. But yes, I would expect that that what Kara has called age old challenges that crop up in evaluation practitioners conversations do appear frequently. And that for, but it's always lovely to talk to other people about them. And that's very much, you know, a focus point of the program that is really important to us. That people, we have found that evaluators often isolated, not everybody can come to a conference, for example. And so to have somebody to thrush these things out with is really great. And people do feel comforted by the fact that other people have experienced those issues as well. It's not just them. So that's, that's a really worthwhile thing. What are the kinds of problems and issues that people brought to the table? It would vary because of the different focus, the focus of the different groups. Most of the mentees would be beginning beginner evaluators of say two or three years minimum experience, some of them will have, you know, 10 years experience. There's quite a diversity there. What we actually found when we did the recruitment process last year at the beginning of the program was that there are many people who move into the evaluation field from other fields. And so they've done very well in their career already. And then moving into evaluation, maybe from a research positions or program management or policy positions. And they're just trying to work out what is this thing evaluation and how do I transfer my skills across. And that's a really important, important segment. So people, you know, as adults, people bring all sorts of skills and knowledge sets. And, you know, it's a multidisciplinary field. So it always makes for interesting conversations, doesn't it? Julie, I'd just like to raise a point, which is that in the work that I did with human systems dynamics, I was actually involved in a group like this. About two or three years ago. And we did actually end up setting up our own self sustaining group, which still runs today. And so I've got people all over the world that I'm catching up with on a monthly basis. And it's been interesting to hear you guys talking about this because it's given me more clarity about the fact that what I did is actually what you're aiming to do here. So I think it's really exciting. And I think that a lot of it will actually just play out as you go because it will depend on who's in the group. And I think that it can actually be really helpful to have people with two or three years experience and 10 years experience in the same group. Because I know you've probably found the same thing, Kara, there's things that you've come across many times. And when someone that's been two or three years and talks about it, you go, oh yeah, but actually in talking about it with them. You find new ways to talk about it and become more competent in the way you're working with it. So it doesn't actually matter if you've got this big mix up of people in the group. In fact, that kind of diversity can be quite cool. So yeah, I just kind of wanted to say that because having heard a little bit more about this from you, Julie and David and Jill today. It's quite exciting what we're signing up for actually. So when you get say yes to something like this, it is a bit of a leap of faith and you've kind of just got to find a way through it all but it's sounding really good. Yeah, thanks Judy Judy. Mentoring a group that's going to be looking at managing evaluation in complexity. It's something that I'm very excited about. Thank you for joining the program this year Judy. We have any other questions. We're also putting the focus areas up on the website. I'm sure that's, that's really useful. If you're thinking about becoming a mentee, please apply. There's still time. This year we've got five, five groups running for those groups have a mental and associate mentee. And the fifth group just has a mental and we have. We have a combination of new mentors and associate mentors and some other mentors from last year. So, so we've expanded slightly on last year and we hope to expand again in in 2023 and get more groups out. Yeah, we should also mention we will be presenting about the program at the conference in September. Hopefully we'll have some more evaluation emerging and more insights into the program theory and how it's evolving. Could I ask a quick question about if whether or not they ever done something similar to this and it's previous and it's previous history. Julie, it's something that's been talked about for a very long time. And it came out of. I think there was a membership survey and strategy document like it was given emphasis, and then the pathways committee were given the responsibility of looking, looking after it. And so we. We, the people on the pathways committee. We had the responsibility of coming up with a mentoring program. And then the design of this program that we actually developed the design emerged from our conversations, which were around the early days of coven I think that's really, really important thing to mention, because the idea might not have come up the way it did. But yeah, it's been talked about for a really long time. I understand that the New South Wales group. Regional group is developing some sort of mentoring buddy buddy system, which might go more along traditional lines of one on one mentor mentor mentor relationship. That would only be for people in New South Wales. And this is the first first sort of program for all, all members. Last year we had a couple of members who weren't not in Australia I think I was in PNG. Is that right Jill PNG. And that was a little bit a little bit tricky. It was very tricky. I think connectivity was such an issue for that person that they eventually discontinued and just, it was just really, really challenging for them. So that was a real shame. But I think that was the only person outside of Australia New Zealand to was. It really is an idea that's taken years to develop. I mean, I know several years ago I actually wrote a proposal for the S board. It was much more along the lines of a traditional one on one mentoring program. And we had lots of debate and, and the thinking just sort of evolved away from that. And I think that's what we do toward more of a group kind of drawing on community practice idea. It just kind of like develop a sense developed over time that that something like the group community practice idea would better suit AES community. That just sort of that that's just kind of how it evolved. It's been quite a while to actually try something. So I'm glad that we finally did. What advice would you have for people trying similar initiatives from the lessons that you have learned so far. It's quite a lot of work. Would you agree, David and Jill on the part of everybody involved the organizing group. And at the beginning, the brave mentors who didn't really know what they were getting themselves in for, but volunteered anyway. So take sort of commitment and courage and a lot of organizing and design work. We have program guidelines and we've had to write lots of other material to the group has put in a lot of hours to to pull that all together and to kind of work out any little issues just as you would with any other, any sort of program design. So this is, you know, pretty small program but it still needs all those design features. I think that that would be why it took so long to get off off the ground. Yeah, I'm pretty excited to have the money for a project office for this year that will really miss the load. There is a lot of work to get it going. Yeah, that's kind of, I suppose that's the initial thought is figured through carefully on this fair amount of work to be put in place. And last year we also had a series of fabulous students from Melbourne Uni. Because they put in a lot of work around the evaluation. It was part of their final subject, the capstone subject, but the work, the amount of work that they did for us was really beyond what a student is normally expected to do their capstone subject. And so we were really, really lucky to get them on board and they worked together and and students came on at the beginning of the program. And as the semesters progressed, the new students would come on and just kept on adding and handing over to the next student and so the evaluation was really important to us to figure out what is working, what is making the difference what we need to do. I certainly think we, as a member, we owe a great gratitude to you all for that because I can only imagine the amount of voluntary time that's gone into it. And the way that it's come together, I think we're quite fortunate with the timing was COVID. But I genuinely hope that we can make it work in a way that's going to be sustainable because the value to members is I will be huge. So, yeah, thank you so much for the time that you've all put into it so far. Thanks Cara. Right well, does that we ready to wrap it up then. It looks like there's no more questions. There's no more questions that is thank you. Thank you so much, David, Julie, and, and Jill for for giving a fantastic presentation on the mentoring pilot and yes I will consider joining and just have to check my calendar sounds like a fantastic group to be a part of. Thank you everyone for your participation.