 I've been here for quite some time now, and there are so many people around, and you know what, they keep on talking about ECG. Ah, ECG, I know. I know what ECG is. Wait a second, you're kidding. And of course I knew, now you can see, you're a professor at the outside world. For evaluation, being involved in evaluation, developing evaluation. Yes. And the director of the center of evaluation. So, you must know, is that right? So what is it? Is it like electronic capacity device? No. No, no, no. No, it is electronic category. Do you know what it is? It's the course running my... Aha. Yes. In course. Are you sure? Are you sure? No, I'm not. Do you think we're going to have college together? I don't know. No, I'm not. Of course I'm feeling good. Oh, you are. Yeah, of course. So what is it then? It has to do with evaluation, yes? What? An evaluation. An evaluation? It's about evaluation. It has to do with assessments, you know? You collect data, and then you're making a assessment, and then you tell the people what to do. You need a presentation. And that sounds to me like all these people do that? I don't know. They have some later, right? I don't know. Yeah. Why are you here? Maybe they're waiting for the pool. They're waiting for the drinks. Yeah, but, you know, it gets about evaluation, and then the ECP is an evaluation capacity developer. You know, there's so many governments and organizations. They want to use that instrument of evaluation nowadays, and it's very popular. If you are a modern organization, you have to use it. And then, yeah, the question is where all these qualified people come from. You must have some qualifications for doing an evaluation. Of course. It should be a profession. Is it not a profession? Yeah, yeah. They're trying to do a profession. That is, it depends on what you're doing, what is part of the profession, what characterizes a profession. I can't weekend. Yeah, for example, yeah. He has a training, a craft training, and of course at the end he gets a certificate. And this sounds like who he ever is. You know, you can have written an article, you can tell yourself, and it's published, of course. You wrote an article and it's published, and you can tell yourself that you are a journalist. And it's similar with us. So, if you have figured out something, which we say it's an evaluation, we can say we are an organization. And that's the problem with the evaluation capacity of other people. They are not courses that are offered. How can they participate in a course? Yeah, but they, I know I have heard about it. I need that, and I must say that that is a very useful course. Because they have so excellent teachers. Like, you know, there are a lot of other courses. It's like, yeah, I would say like these cooking book style courses. So you go in such a cooking course, and afterwards you think you are a good cook. And you are easy to cook. And mostly it's not like that. It's a recipe, right? I do that. I make a lot of bad experiences. You're cooking? Not everybody is cooking. When people who visit the cooking course can see they can cook. And this is similar with evaluation. They visit them in a few days course and then afterwards they see there are evaluators. But as I said, there are different kinds of courses in Canada. And in Canada, of course, it's something very different because it's about two weeks courses and a core course, advanced courses. So I think that people learn really something about evaluation. And as I said, it's very important because there is a great need for highly qualified evaluators. And if there are not enough qualified evaluators, then evaluation maybe gets a bad image and we don't fit the needs of our clients or we produce not the quality they expect. And then there is a risk that they say, well, now I decided to do an evaluation, to say non-evolving evaluation, but the result of it is really disappointing. So to some people I told the story today that I think you attended the conference too, in April, where the health minister informatized for an evaluator and we were sitting there and this guy was giving a keynote speech. He had now health minister. So what people expected of him was an evaluator before and now he praised evaluation and that he really used evaluation for his work and he disappointed us completely. Because he said, well, we're expecting this because I want to informatize an evaluator, but I must say I never used evaluation for my work. And he said, why? It is because they are never on time. This is so complicated to read. You never know what they want. They need current recommendations I can follow. And so on and so on. And actually I must say for some evaluation reports, it's like that. And then it's why evaluation capacity development is so important. And probably not only for evaluators, but then obviously for ones who are supposed to use them. Yeah, I think so. I think it is also important not only for the people conducting evaluation, but of course they must know their business that to get money for evaluation, I think it is also very much important to convince the people on higher decision levels to get money for evaluation. They only will do it if there are police that they can use the results for their decision making. And so that's important. Do you think they should participate in the next week? Oh, I think so. And I know there isn't an excellent course about that. But maybe that is true. But I hope not. Yeah, okay, let's see. Yeah, yes. Any more questions? No, really, really, really interesting. Yeah, I think I can tell some interesting stories. Yeah, I know, because I know what I know is the program manager, right? The program manager. Hello, hello. I don't know. But you know, she has a special talent. But unfortunately, I've had also a life. A life before. Before her talent. It can trust you, it's true. And I wouldn't say that. Some years ago, we were working together. And she worked together? And she worked. No, I mean, long before. She was also working with the Center for Elevation. But then she was the head of the department, which was also responsible for evaluation capacity development. And that's why I would like to ask you what that in your experience is with evaluation capacity development. I have heard that also in your organization, the German evaluation institute, it was not really of high priority to do that. Yeah, quite interesting, right? You have an evaluation institute and you would expect that just automatically everybody would understand whoever is doing evaluation should invest in evaluation capacity development also. But it was quite hard actually to convince the ministry. They looked, we are an evaluation institute, an independent evaluation institute, and we will do evaluations. But at the same time, it's so important to be when you're working in a country and you think about how can you develop capacities within this country on different levels, different target groups, so that actually people are more and more able to do their own evaluations. And not only do evaluations, but when it comes to a more systems level, then what is needed in a country on evaluation capacities, not only on the individual side, but also on institutional and societal levels. So that was quite interesting to see the reactions that then, as we started, and we pushed it through and we started working on it very high, so the more they could see how the impact was on what we did together with the partners, the more familiar they were, and now I think it has quite a high standard within the German Development Corporation. So that's very interesting. You see, that also has to do with awareness building actually in the evaluation institute that evaluation capacity development is really an important task and it's even more important if you have a look at developing countries where they take their first experiences with evaluation. So I will take it out in Costa Rica where you have been for several years. And we had the chance actually and that was really one chance in the lifetime, to implement a pure ECB project. We did ECB all over. And the conditions were really, really good because in Costa Rica already, so I think since 1995, they have a massive evaluation implemented. So already quite good capacities and the awareness was there and we started with for the trainings or the target groups, but then it became quite clear very soon that the training itself was not sufficient. Now, we had a very, very good relationship with different organizations within Costa Rica. So that started with the government but also we worked together with NGOs and with the universities and with evaluation associations. So we formed a national platform where we all sat together. We had the chance to work in one country with the awareness there. So that was also really, really nice. When I went to Costa Rica, I was always so happy that we don't have to dismiss anybody about evaluation, about evaluation, capacity development. Everybody knew what we were talking about. So it was kind of the question to go back to Germany where you can pretty much determine if there are still people and if they are really kind of hesitant. So that was really cool and fun to work there, especially with this platform. And now they are, because I'm a project standpoint, I'm not working there anymore. But they are developing our regional platform where more and more actors from Latin America are coming in. But this platform really changed a lot because for the first time the main actors would sit at one table and would talk to each other about evaluation and how the country could proceed and the awareness of the people that they need to work together in order to get the best out of it really, really raised and changed a lot. So imagine you sit with the minister or vice minister in Costa Rica. Costa Rica is small, right? Very nice. And still, you can talk to ministers directly. So they are involved and then you have NGOs, smaller ones, bigger ones. You have representatives from the associations, universities and so on. So we actually could develop together a theory of change that you mentioned that was hard. But it was really, really worth it. So together we developed this theory of change and together we decided what action should be taken in order to reach the different outcomes of this project. And you were there, too. I mean, the experience. Two weeks ago, I think, from there, it's always worth to visit Costa Rica. I can recommend it to you. Especially if you want to learn something about how to implement the monitoring and evaluation system on a national level. And I think what is more special with that kind of action there was that many different actors were in the same direction. So we have had this, it was also called a fossilile project which was sponsored by the German government an ECB project. And beside that, we implemented but also a university cooperation. And this is, I think, a wonderful combination because we mentioned that topic several times before to bring the news into the evaluation. And, of course, there are very well trained people graduating from a master course in Costa Rica, but having only little chances to practice evaluation. And so, during this university cooperation, one of our objectives was to conduct together with young students and master graduates an evaluation. And out of that we got another experience which we are using now for the second round together with the fossilile and with able partners to bring the young people into practice. And this is, I think it's also something you can recommend if you want to establish an evaluation capacity in your country. I think it's a good idea to cooperate with universities there and there are a lot of young, motivated people with creative ideas and they're very cheap. This example shows also that you have a very good training offer for years and but then still applying evaluation is for some are missing. So people are eager to get into business to do evaluation so from what you want to be established as consultants but also at the same time within the organizations and here comes to the second part we should talk about the institutionalization of the evaluation. So within the ministries within different public administrative organizations and also the NGOs they get quite interested in looking how can we actually institutionalize evaluation so that it's not a one-shot thing and that was another really, really, really rich experience where you sit together and think you have to do capacity development on an individual level also within an organization for example there's a management chain for institutions as public administration and then for example for some NGOs we get together a practice called evaluation in a very remote area of Costa Rica where we worked very, very close together with people involved in the health community system and from there we could disseminate not only the results but the lessons learned from these processes so that was also quite an interesting experience to have on an individual level capacity development now they can apply these capacities within the institutions they develop the different capacities necessary to commission to do evaluation practice and to use of course evaluation and then we worked together with the Latin American Association of Evaluation in fact we probably heard about them it's such a famous evaluation association and we developed together evaluation standards so that's also something important which you can then include in your work in the different institutions and to have a framework built up so that all kind of you know, linked up together which was very, very interesting I think you see with these examples that evaluation capacity development is not just about training if you talk about that in a country or in an organization there are different dimensions of course for example you have to establish some structures and some processes to anchor evaluation or a monitoring evaluation in a country or in a country and for that you can very different qualifications on the different levels or positions where they are worked so for example some people have to be able to write an evaluation policy if I want to establish monitoring and evaluation system in my organization how can I do that? which steps do I have to follow what are the lessons learned out of similar activities and for that I think we need also different formats for the evaluation capacity and it was actually a little bit a fun thing because we started in Costa Rica with operational courses on the reparational level so it was a three weeks course three times one week and then after the course people said well now we know actually we know now how an evaluation should be conducted but yes sorry we are not the ones who decide about it that's right we have to climb up the level and that was the idea comes up with the evaluation course for an upper and middle management it's only a three days course and part of it is to be aware of the topic of evaluation but also to give them some insights how they can use evaluation for their work and of course then how they can commission an evaluation to know how to ask evaluation questions write with terms of references and you find the right budget according to their questions like when it's very beautiful for many people that they balance the questions they have the evaluation questions that they balance it with the money they have and my experience is especially in Costa Rica as more the people are motivated as more questions than they have we are now convinced to do evaluation and they just do it and then they have a bundle of evaluation questions and then most of you have disappoint the people that didn't pick because then you have to ask about financing how much money do you have and then they send an analysis this is now to come and answer 100 questions you have to focus on the most important ones and to pre-decent the balance is something you should also learn in this agreement and upper management course but the final thing was again after the three days course we understood how we can use the evaluation for our work but none of the ones who decide and so we went a step further and said ok that's true so we must have the format for the top decision makers and they have not really much time so this is only a course it's a two or three hours mostly combined with a nice event having a practice together and then we transport our message to build awareness for the evaluation to make that interesting and in Costa Rica it was really perfect and then I had the chance to talk to the complete cabinet of the country and that happened in Costa Rica with the president and all the nurses to talk with them of course in 20 minutes but to talk with them about a monitoring and evaluation system in their country and I must say that it is also fun to think that a guy from Germany is telling the Costa Ricans how they should implement the monitoring and evaluation system on a national basis and we in Germany are far away from that and nobody is listening from the political level and hopefully this is going to be so this is it but then it sounds like Costa Rica is paradise for evaluation so is it really that great is it are there any problems nothing I don't think so no of course everywhere there are problems because Costa Rica is a wonderful country but of course there is a big population in the government as you know there have been and then the government has completely changed the people completely so we have to start with this awareness coming again and it is not the same in the ministry a lot of the people on the upper and middle management level they stay but of course the ministers and second players stay like stay but what they see hopefully is that that this is really sustainable is that they started to create this national evaluation plan where they decide which programs they will evaluate and now they combine it also with the SDGs and they will evaluate programs which are connected to the SDGs and that was their idea to do that so I think yeah I think it could be really a success story even if there are also some risks because the main point is always the money they wouldn't like to do it unless they have an idea for what they are doing but in the moment I do not see very clearly if they really use the evaluation for really changing the policy but there are also some sign some papers that this could happen but the main thing will be and that's actually the proof of the pudding you can say okay I'm so in favor for evaluation and you do it and have an evaluation policy but at the end it always can it comes to money and then if it is all is only rhetoric that people say it because they think they have to say it no at the end they have to give money for it and they will only give money for it permanently if they will take use out of evaluations and this comes again then to evaluation capacity development because it is our responsibility of all the people since the years our responsibility that we do not disappoint the people who decided to use evaluation for their policy and they do not come to the to the result of the decision well I send money for that and they don't give me the proper answer to these results and then could something happen what happened two decades ago two decades ago there was no proof in that we had the first proof in the 60s and the 70s in the 80s and the 90s evaluation was declining and now we have the second new proof but there is no guarantee that this proof will continue and it's our responsibility that this proof will continue to do a lot not to disappoint the ones who saved money for evaluation that's our responsibility so it actually really makes sense that if that exists and that we are all here and that we are going to develop that further is there anything you would say because learn from this experience what really comes to my mind is immediately because we implemented in specifically mentioned this morning the blended learning program we started in Ecuador and implemented it in Uganda so that idea occurred to me that it would make sense to integrate it to IPDES so that for if you could use some different formats online formats and then combine it with presence and here we could also work together with centers and partnerships in different parts of the world because obviously we can learn a lot from each other and it's not that we are I mean you made your evaluation professors but it's not that we know everything and we of course don't have all the experience so what do you think about that and what else could we learn from this experience is for IPDES I think we can learn a lot for IPDES so I think more different formats for different target groups because as I said we cannot only focus on the ones who should conduct evaluation and who need to see depth courses knowing how to develop a site how to know how to use methods approaches and all that this is more for the people conducting evaluations but as I said development of evaluation in the country is very important to convince the management and also the talk management the ministers in the government and not forget also the civil society organizations so that was also a very interesting experience in Costa Rica that there the civil society is very much interested in evaluation also if you look at the European countries there is not much interest in the civil society about the results of evaluation and less in participating in evaluations and so that is true so that we can learn especially from countries in the south and especially in the field of the participation of the civil society I think we have to look very carefully what is developing for example in Latin America countries which I don't know what is going on there in the civil participation in evaluation but that we can learn there how they do it and of course other forms that will be necessary so for example we take with the civil society with communities in the health sector in Costa Rica doing self evaluation doing under competencies and qualifications as an evaluation if you want to consult such kind of people or such kinds of evaluation as with our traditional evaluation concept and that means we have to integrate different approaches and we have to focus also on different target groups in order to establish monitoring and evaluation in the country and the other thing I think it's it's a good idea to connect with the programs with master programs so that in the end you are able also to get the chance to get the master degree as you know there is also a big discussion in our professional communities and associations about certification and if they make an association and they present with one solution and in Britain they have a solution but I'm a professor at the university and I believe in academic certificates and I think in the future these academic certificates will play a major role in the professionalization of evaluation