 Boy, thanks everybody for coming out on a cold afternoon to talk about Program for results two-year review I don't we had 200 RSVPs for a topic that I thought was a little bit esoteric and obviously there's a lot of energy around this this conversation For a number of different reasons. I think one of the issues is is how does the World Bank remain relevant? in in a world of Expanding aid architecture and also making sure that It's demand-driven in the sorts of instruments and approaches that it's providing And I know that we're sort of this this program for results instrument is in essence been beta tested It's had a couple years. It was allowed a There's a 5% ceiling on the actual total of of lending that could be used under this There's been a review an initial review But because instruments are three to five years in length and takes a while afterwards to actually do a full-on evaluation We're sort of really a technically speaking serve in the third or fourth inning of a fully understanding this but My interest in convening this was okay in a world where there's a big reorg of the World Bank There's these other actors instruments like the Asian Infrastructure Bank that are complementary to perhaps or or perhaps competitors to traditional Bretton Woods institutions depending on your perspective that It seems to me that there's going to be an ink There's a lot of energy in the bank system to be relevant and also be responding to clients And so I think there's a sense that a belief that this instrument is More responsive to clients and so one of the questions is going to be is this instrument delivering more results than traditional instruments is this is the contact, you know If the and then the other question over time for the bank is will bank shareholders including the United States including the US Congress and others say well You can do more of this and have the and I think folks in the bank would say we'd really like to have our ceiling Listed listed from 5% to something higher than that And I think that's the really I think the Jury still out and what's going to happen there. So without further ado, I'm gonna turn the panel over We're gonna ask Fadi Asada who's with the World Bank to run us through a video to kind of frame up the conversation and then we're gonna hear from Fadi Asada who's from the World Bank and then we're gonna hear from mr Jorg Frieden who is the Swiss Ed but also represents a number of developing countries as well as Switzerland and Then you're also gonna hear from my friend Craig Albright who used to Oversee government relations for the World Bank group and so he's a former World Bank staff He also worked on Capitol Hill. He worked in the executive branch of the US government So is gonna be able to provide a little bit of a perspective of of a particular shareholder But also some of the the politics of this if I can put it that way so without further ado, Fadia the floor is yours Thank you We'll just we thought we should hear from clients first and to start this conversation and frame it since this is an instrument that was designed Mostly in response to clients. So if it's okay with you, I think this video self-explanatory We will hear from three countries about their experience with P4R In early 2012 the World Bank Group introduced a new lending instrument program for results Program for results also known as P4R disperses funds based on the results achieved P4R helps governments improve the efficiency of their expenditure programs and achieve results by strengthening institutions and building capacity in addition P4R maximizes development impact by leveraging the World Bank's resources to ensure that the resources of all development partners are Used to achieve the desired results Well for some time our clients were asking for our help to help them to implement specific government programs that they had They were looking to us to help them think through how do they strengthen the quality of the programs? And how do they actually improve their capacity to implement them? But our existing instruments really weren't very well suited to help us to do that. So we introduced the program for results Over the past three years P4R has made very good progress Across the globe spanning all six regions in a range of environments from fragile states to middle-income countries 22 operations are currently underway in a variety of sectors For example in Kenya program for results is currently supporting the Kenyan government's national integrated safety network program really the key attractive thing about it is the fact that You can focus on things that you want To do yourself like that social safety net. It's a policy Priority for us. We had already as a government consciously decided that we want to do that If it's something you own you want to do then P4R becomes Fairly easy to implement because you're not doing anything else other than what you wanted to do yourself By fostering engagement at the program level P4R creates opportunities to improve coordination and effectiveness among all parties involved It focuses on key Concrete measures that you take to advance the project or the program that you're implementing So they are clear measurable indicators of progress in Implementation and that's a very important aspect of a project management and if you do that again When you have concrete measures like that you enhance development effectiveness in Vietnam P4R is helping the government improve long-term planning and the delivery of infrastructure services and This before our system in required The commitment not only at the working level who are directly in born-in-the-investment But also the upper levels. I mean like central levels have to be more responsible and Also have to have a longer plans to ensure the smoothly runs of this The output expected P4R's inherent flexibility also helps foster partnerships towards the achievement of results And I think with this one now this implementation of before you also enhance the relationship between the bank and and the government through the common Output which is have been agreed with between the bank and and the government But it gives room for Flexibilities and also more responsibility from both sides in Morocco P4R is helping improve access to social services and economic opportunities for sitting an example, I'd say If you are talking about the PPD La generalisation de la scolarisation, for example, it can be a text of law and we have it available Well, there the political will of the government is assured But the critical effect of the generalisation of the scolarisation if we do not give the means at the same time Institutional regulatory but also technical Qui nous permet d'atteindre cette ce résultat donc le P4R nous donne cette possibilité de pouvoir atteindre ses objectifs Donc la question de l'information devient fondamentale These three examples have shown how P4R has been used to strengthen the link between financing and development results P4R is a significant step for the World Bank in the direction of providing solutions to urgent development challenges Well, the program for results has really shown some promising early results We think that this is an instrument that has great potential in terms of really helping countries Achieve better development results. We're learning as we go along And so we really look forward to partnership with you as we take this instrument going forward For more information, please visit our website at www.worldbank.org forward slash P4R Thank you. It's a pity we cannot take you all to see some of these Live because I think this is when the concept really comes alive But I'll try to take a concept that's quite vague and share with you some of the lessons and experiences Why did we introduce P4R as you heard many countries? So we have a very good instrument that we did projects the World Bank Projects you go any country and you see them and we had the very good instruments for doing budget support and supporting policies But many countries were coming to us with this question like you so now is that they have in morocco For instance, this is several billion dollar initiative for the government to not only improve transparency But also community engagement, etc They didn't want the world bank is supporting one part and the european commission another and Someone else assert each one to create a little island They really wanted help to get better results for their program and more transparency And this is what the type of initiatives with program for results can do because instead of us going with our own set of rules We go to a program and work with them to look at two things Program on the institutional side. Do the institutions have the ability to work? Well, can they really deliver the results? Can they plan? Do they have also the checks and balances for addressing some of the fiduciary issues some of the environmental social issues? And once they do that are they what are the results that provide the incentives for moving this program in the right direction? So at the bottom of it really it's about building capacity and focusing more on results And this is something that you have seen in some of these examples So p4r in a nutshell has four features if you want to just take the take away for you is one it disperses against results not expenditures or reimbursement And it disperses to a specific overall program But it really the disbursement the result has achieved two things one something very Hard to measure which is it has changed the conversations But maybe that's the most important thing it's doing and we've done interviews with clients We have also even out our board Looked at the discussions in our own management when you're discussing it before are the first thing Everyone talks about is what are the results? How do we get there? Do we have what it takes? And once we do that it's the whole conversation comes around that Which is really a a different focus all our instruments try to achieve results But by linking this financing to results it takes it to a very different level The second is we don't work on specific procedures, but rather on improving institutional functionality Which is again something we aspire to do in all our instrument, but here it is the core of the instrument That's why it's called program for results The first part of it is the program how does the program function well and second is how can we achieve the most efficiency and effective results from this program It's now three years old and in three years. We were testing Does this work? We all have aspirations and put things on paper and then you never know how it works in practice We have some good news to report here is that after three years 22 of these have been approved and about 19 more are under preparation And those that have been approved have really done well We have done surveys and they looked for results and talked to clients and talked to development partners and the feedback is encouraging but I sort of always like how the private sector thinks sometimes and if we think about demand you can ask two people come for seconds That's really a good test of demand and what's interesting is where we're happy for are The countries that have started that are now under second and third and some on the fourth one in three years So as you come back for a second take probably it's not so bad When we ask clients if they like this about 80 some percent of them said that they would like to use it in the next two to five years So this is encouraging at the same time We have also adopted a very interesting learning approach here Which is good because we when you leave your procedures behind you really are a lot about judgment and how you do things And how you work together. It has changed the work we work with countries And the feedback from that has been positive in Tanzania actually one of the countries in the interview said to us We used to dread the world bank missions And now actually we look forward to them because it's kind of helping solve our problems We don't see these as your requirements anymore, but rather how we can make our Systems better now. They don't always still like world bank missions, but at least that's encouraging to see In terms of where we go with this We looked also at the demand moving forward when p4r was introduced Some of you have been monitoring this. I see some of the names, but many may not know this We were asked to move cautiously And we are we're asked to limit the commitments of p4r in the initial two years for five percent Of our ida ibrd commitments. We are now in three years We actually have maintained that and have been able to implement it well But now the demand is exceeding that and actually we expect sort of demand to At least in the next fiscal year to be somewhere and even this fiscal year to Be above that five percent and next year to be closer to about 15 percent or so And moving forward demand is of course a function of supply But clearly countries are interested to explore this further In terms of how it's doing on the institutional capacity, so we talked a little bit about the results On the institutional capacity We've also seen that p4r have included a lot of actions and improvements that have had spillover and Effect for many aspects of the program For example in few countries we've seen monitoring and evaluation systems put in place for a very large program in which we are adding a very small proportion of financing We've seen environmental and social units being institutionalized and road maintenance Programs that will now play a much more critical role Not only for the world bank financing or specific activities wearing fans, but much broader We have seen supreme audit institutes being strengthened so that they can play a bigger role in audit and implementing audit requirements But to bring it back to results as well We have also seen that the results have been also a little a big talk among governments and and In our review missions when you go there everyone seems to know what results they have to achieve and why And it's interesting the dialogue within country between ministry of finance central ministry or sub national ministries has been But these are the results that we have to achieve So again, this has become something that's very commonly exchanged And we see a range of results. We see outcomes If you call like child immunization, I know that there's debate. What's an outcome there? We have seen improvement in nutritional status We've seen anti natal care coverage improved and we've seen some things that are more simpler outputs depends on the sector We've seen agriculture yield in some we have also seen some of these institutional type of improvements and capacity building as key results to be achieved So it tailors to the range of development programs depends on the level of sophistication and maturity they have And it adapts to each program But it is in its early days It's actually encouraging that we have been able to adapt to these and achieve the desired results And we hope that as we move forward we continue to learn and we continue to sort of see progress in terms of expansion Okay, so faria. Can you just explain to me? What is the difference between a p for our Loan or instrument if i'm tanzania and just a straight up world bank loan and what's the difference? And why is this why is this so great? Can you just explain that I saw the video and I saw Happy folks, but I don't fully understand just to keep it really basic and really simple for me As to what what exactly is this thing and why is this so much cooler? Than than a straight and a straight up world bank Okay, let's maybe take an example to do that So in tanzania we have an education project and now we are supporting their program Which is called achieving big results when we do an education project we agree with them on a set of investments They wanted so we rehabilitated some schools. We got some books. We trained some teachers, etc We finance against each of these activities when they are done And we monitor how they are doing and they have a set of rules They have to comply were for each and every activity. So they don't see the money They don't see the money until they finish the project No, they see the money as long as they reimburse they spend on a training or a contract or a school They we reimburse every time you do it we reimburse an expenditure item And they really the results we monitor, but they are not linked as directly to the financing Now again a good a live example in tanzania the government has launched this thing called achieving big results They want to elevate the student achievements across the country. This is their national program. If it is involved We are involved many others. So instead of us looking at each specific contract or expenditure What we have done now is work with them to see what are good results to measure What are the changes they want in the education system for instance institutionalizing student testing? Introducing some of these changes changing the way schools are managed So this is going to be across the whole sector And the disbursement will happen only when the attainment of school students is achieved. I hope that explains So Let me try to say something and then fight it can correct me if I get it wrong So There are three kinds of loans, right? So there's development policy loan, which is budget support There's an investment loan, which is project specific and then there's p4r, right? So I'll use a different example or we can use education in in tanzania. So development policy loan will be Uh, we the world bank say we want you to change x policies, right? It's very specific and certain policies need to be changed and as policies are changed the money is provided, right? That's basic development policy loan, right? And then there's the investment loan, which is designed heavily upfront Uh, very very specific, you know book this big that unfortunately executive directors have to review Particularly the u.s executive director because that's to comply with about 150 different laws and Then that gets approved and then they you know spend based on that and they follow Follow the playbook and then p4r is sort of in between right where where tanzania says we have a 500 million dollar education reform program and the bank says we're going to finance 50 million of it And your whole program has to comply with the certain rules that the bank has in order to invest in that larger program So the bank is able to then get into Uh, that larger program and that larger program has to comply with world bank rules and up front there's uh analyses of systems that are made public And it's it's a it's a uh, you can kind of think of it is in between You know budget support and an investment loan, but it is really a different animal, right? And so that's I think kind of what it is for a non-technical person Is that correct fari? Is that does that capture it? More or less Lynn does that that can No, I mean, I think the only thing I'll just add is is p4r works I mean one way to think about it just to complement this is one works at the transaction One works at really trying to say You know, does this does this dc city have good systems in place when there is snow One is about sort of blowing the snow one is about the system And one is do we have the policies in place sort of that one things need to be changed that We have them on the book. So it is a different level of operating But the It is sort of a good way to explain it. Yeah, there's there's one other thing That's really important about about p4r that it's attempting to do and it's early, but we'll see The it's you know the are as results right so that so the origin of this is to get better results So an investment loan the climax of the experience is getting approved by the board But it's it's approved by the board That's the big sort of point of victory and then the program runs Uh with a p4r what you're doing is you're creating what are they called the disbursement linked indicators or something like that So you you create a set of these right and then you release the money when they hit those various benchmarks And so the climax is throughout The life of the program at least that's the way it's intended to be so that that intensity of engagement is is there throughout And and obviously it's linked to the results. So achieving the results is important And then you're it's it's more empowering for the country to be able to figure out how to get to the results And deliver them so the the results piece of it is Important to understand at least what it's attempting to do and I think The the instrument should be judged over time on whether or not that Is is happening Thank you. Thanks. Thank you craig. I appreciate that so You're you're the swiss executive director, but you represent a number of other countries on the board Can you talk a little bit about how? How you're thinking of how you're as a board member you're thinking about this you've obviously this was presented to you several years ago Switzerland's a major partner of the world bank. You also lead on financial innovation as a swiss as the country of switzerland you or You know there's But there's there obviously a number of concerns that have been that have been expressed about p4r There have been some people have have concerns about this and But could you talk a little bit about both from a swiss perspective and some of the other constituencies that you represent? How they look at p4r Is better. Yes So the first issue is the relevance and I would address this as a member of the board concerned about The impact of the world bank worldwide Then I would look at it from the point of view of of the country I represent the country of central asia a different level of development that why it seems to me The instrument is appropriated and needs and then few minutes on the controversial issues that is That is that is making difficult to extend the use of p4r that are issues around Safeguard environmental and social safeguard on one side and procurement on the other side and what kind of debate we are having in the board about those issues on relevance the fact that Reduced substantially the relevance of the bank is the fact that the bank is financing today a very tiny Portion of investment programs and expenditure A public expenditure in large part of the world even in the poorest country like Kenya so You cannot exercise influence on development policy and on institution By implementing in a corner of the country a perfect project I mean if the bank limits itself To implement project today The influence of the bank will be limited to small corners of the developing world So the issue we have had always Is to establish linkages between the implementation of a specific projects and what The country intends to do in that area in that sector in terms of policy institutions Measuring results and so on and so forth So that's the challenge and that what these instruments try to answer to To put that project that activity of the bank in the context of the policy Of the country it is you know building roads Promoting primary education improving else Where the bank finances three five percent of the old expenditure Finance by the country And tells the government fine our five percent is there if You achieve with your own program a certain number of results. We agree together So that's really the central approach and therefore for us The importance of the instrument to have a substantial dialogue with governments About critical policies and sectoral interventions So that's why we see it as extremely important to avoid a marginalization of the bank in small areas That are not that are not in a in a position to really influence the global picture I go back to to the country I represent even the poorest Kyrgyzstan Tajikistan. These are countries of five six million inhabitants Lost in the middle of central Asia almost no natural resources Live out of migration and export of some commodity gold for Kyrgyzstan aluminum for Tajikistan But with a heavy inheritance institutional inheritance of Soviet republic A difficult transition Into more open economies and more open political systems These countries are not used to measure results. They are very heavy input oriented You know that you know the tradition the Soviet tradition of you know, uh, uh, measuring Like the world bank, uh, you know, uh, uh activities by Uh, by inputs, right? Uh, you have the input. This is what the projects are and, um In in this in this country where China is investing billions and the bank, you know I give you the example of Tajikistan today speaking of relevance The IDA program for Tajikistan is is 60 million a year China has signed recently with Tajikistan an investment program of six billion 100 more and and this frankly this is true for commerce. This is true for For Guinea-Bissau this is true in many places relevance, right? In this situation, what is the best approach for us Is clearly to help government To implement policies with some with the little money we have That influence also the use of the Chinese money So we would like to engage In a in a rational transport sector planning of roads maintenance of roads that we'd Apply to our five million dollar investment But we'll apply also to the 100 200 million investment that the Chinese are ready to, uh, to, uh, to finance now Will the Chinese apply the same standard and I'm back to the other issue. I'm not sure maybe not but if Let's let's keep in this example because this for me is at the heart of the debate on standards and safeguards The the traditional approach by the for the bank is to say look in my projects Everything must be clean And effective so in for on my 10 kilometer kilometers roads We apply the highest international standards The environment will be perfect. We have environment engineering No force labor everybody will be paid on time Women will have the share This is for five 10 kilometers, right and we apply the standard We make sure exactly that all standards are maintained You have the Chinese down the road. They built 100 kilometers Following government standards or their own standards Can we have an influence or what is the traditional response that some of the board members would like us to apply Don't build the five kilometers Well, will this does make a difference? No, obviously not so the issue is Standard could not be should not be applied to our own island or perfection today We should introduce our the challenge is to influence standard that could Be applied also to the Chinese investment to the government's investment to to all the investment in the sector and those normally Must be objective to the achieving the sector must be Must be standard defined in legislation Must be supported by monitoring of results and so on and so forth And this is the approach we have Part of us some others are more concerned consider That if the bank dilutes its intervention In the program where the Chinese and the government and the bank put the money We cannot ensure that in the old program our standards are fulfilled, which is true Now here you have exactly this option either You you you opt for having perfect procurement and perfect standards in your own small corner Or you try to engage to improve standards across the sectors And you will certainly have problems because here and there Some of your standards will not be fulfilled and we have a conflict Thank you your the penny dropped for me now that you've explained. Thank you very much. Okay, so craig I'm a cognizant of You've got you if you got We've got members of congress that have to be explained this to them. We have A u.s. Treasury that has to be explained this that the united states is particularly sensitive to issues of standards Whether it's labor standards or environmental standards And so there's probably a temptation to say we really ought to just focus on our five kilometers as opposed to the 5 000 kilometers They're going to be financed elsewhere And so how How has how has how has the u.s. Government responded to this? How has the congress responded to this to this dilemma? Thanks yet And let me just say thank you to you for arranging this panel and and I think you've you Particularly in in washington you've done Probably more than most if not all and trying to get the bank out and talk about some of these things Which provides a service so thank you for doing it and thank you also for giving me the platform This is the first time I've spoken on a panel as a former Something usually I'm speaking as a current employer. So you don't have to stick to the talking points I do not I feel like totally liberated I'm gonna leave here and go to a meeting that actually is going to matter for my current job This is like totally no no stress and sort of just say what I think kind of thing. So I guess I caution My world bank friends who are in the room because we'll see what comes out A couple of points that I'd like to make The uh So I had a really Unusual world bank employment experience I frankly had never set foot in the world bank until I went into interview with bob zealik And I went to the security desk and I said hi. I'm here to meet with president zealik And the head of security came out and took me up to Zealik's office and he hammered me very hard for about 30 minutes And then I had to do some more Interviewing to see if I was good enough for the job And then once I was hired I was De facto kind of senior staff at the world bank because I didn't cover much of the world bank's portfolio at all and don't think that I'm a development expert The part that I did was relations with the united states and particularly at the political level So on that very narrow piece of what the bank did I was Basically an advisor to the president and the in-house sort of political guy for the world bank So for the four years that I was there. I had a pretty good grip On what senior management was thinking and what it was doing and the decisions that were made And I was fortunate enough to have jim kim be excited enough to keep me around so Was grateful to be able to Play this kind of role for two different presidents and in the world bank And I got to know a lot of the senior management that way so so that's the Experience that I bring and then I left and I'm doing something very different But for those four years I was there at a high level And normally people have to work for 30 years to be able to get into the meetings with senior management Just the first day I showed up. I was there and I was like sort of the main thing that I was doing when I was there So I could say a couple of things One is the world bank is absolutely terrible at explaining itself and engaging the outside world. It's it's horrible at it It's one of the worst The IMF is worse But the bank is really bad and frankly Too much of my time and energy at the world bank was taken up by fighting the bank To go out and explain itself to people and it's a it was a big frustration of mine And a lot of people who worked with me know that including some people in the room and That was that it was An endless amount of energy needed to be taken to try to get the bank to go out and sufficiently explain itself Which made it very unusual in washington most institutions in washington are here Designed to explain themselves and they have you know, that's like their their purpose is to engage the hill and lobby the administration and You know participate in the process Where the bank's view is it removes itself from that process And it and its interaction is through the through the board and through the executive directors And then it's up to the us to deal with its own system Which to me is ludicrous because you can do it that way if you want But nobody else does and you're missing a whole huge zone In washington dc that is going to impact your policies And it's and the bank has been very bad. It's so so bad at it that Activists had to shut down washington dc in order to get it to respond, you know at its at its Now the WTO seattle. I'm not saying it's this is all specific to the to the world banks multilateral institutions And the bank is probably better at it than than the WTO or imf for others But it's still really bad and they had to shut down the city to try to open it up and they had the epiphany Oh, hey, how about if we just set up a program for these guys and they can come in and participate in the meetings Brilliant now they're not shutting down DC anymore, you know, but it was it was that hard and I think jim wolfinson made some really important Steps and really tried to You know break that up and get the bank to explain itself more I think bob zealock did a fantastic job in opening up the institution from Data and information and he I can tell you from being in daily meetings with the senior management He was hammering them to go out and it was an expectation that they would Go out and explain themselves to the world In order for him to feel like they were doing their job sufficiently And jim kim has has maintained the focus on and engaging externally also so To the extent that I was successful at all It was because I knew I had the support of the president of the bank and if I had a problem I ultimately could go to the president and who would generally support me. So it was In a unique position. I say all of that because when it comes to p for our That's part of the problem Another piece of the problem is that developments not particularly Like sexy political issues. So a small percentage of the people On capitol hill or in the administration are thinking about it And so when members of congress have to make decisions about it, they're, you know, it's there's not really that much Experience, you know embedded into that. They're all very intelligent people and actually You know very very skilled people who are in these positions, but they don't have any time to do anything and so You know, unless it's sort of shoved in front of them. They just don't absorb it So you have this environment where people don't really understand what p for r is so You throw the instrument into the mix that That had to be approved by the congress and the administration and people don't particularly understand it and um What they what they're concerned about is that the bank is trying to get out from under its procurement rules and its safeguard rules um And part of that is because people just don't trust the bank because they don't have that experience of engaging That you that you do with other, uh, certainly u.s. Government agencies where you can go in you can talk and have an exchange and There's a you know drafts that you can review and you can red line and you know, there's leaked copies or Circulated copies and it's a very iterative process The bank's perspective is we deal with the executive directors and that's it Part of it is understandable because there are countries where if you engage with the parliament You are actually are putting your staff at risk and so it's hard to have a policy that's you know Engaged really heavily in the united states, but don't do it in in this other country and have you know Sort of a global policy, but I think that's that's part of the problem. What I'll say about p for r is that One of the one of the other things that that's frustrating is that people just don't appreciate The the where the intensity of support is for this instrument My experience in working with the people who are advocating for its creation and and its success A lot of them are people who are governance and accountability people You know they were saying this is investment lending on steroids because it gives us more ability to engage these countries and Effect their systems and sort of get into the mix of the things that they're doing like yorgos saying I mean linda van gelder who's here ran the governance and accountability program for the world bank She's now Fadia's boss helping to implement p for r because this is what she likes to do you know, so The the support for this is coming from that perspective Not from the perspective of I hate these procurement rules. I hate these safeguard rules. How do we get from under them? It's the relevance issue that you're very Accurately laid out which I hope is being captured here because that was about the best. I've heard it explained Is is sort of the motivation for this, you know, that is what the bank is frustrated about and it does want to be more relevant So, you know, the the safeguards is a box checking exercise, right? So we have to bulletproof these programs and do everything that the safeguards tell us to do and make sure it's like Bulletproof that's not what they want to do They want to go out and have impact and change systems and make things cleaner and make things better And and if you stay within a box of you know box checking system to make it bulletproof So I don't get heavily investigated that's not where they want to be So it's not that they want to get out from under them It's that they want to be more impactful in the way that that yorgs said and that's not understood So I should probably stop talking because I'll go on forever. But those are a few thoughts Thank you. Craig. I think I think this is Thank you to all three of you. I think this is uh, I now I get the joke about what this is about and I think that It's not something you can put on a bumper sticker But I think the interest let me try which I think is we can either choose to do pristine five kilometers out of 5,000 kilometers of roads that meet a whole bunch of standards in the other 4,995 kilometers don't respond to that or we can Bring our brand our expertise our standards and a little bit of money to influence the broader 5,000 kilometers now There's a slight trade-off in terms of saying we may not have total control over all the standards in terms of how or But we're going to have a big seat at the table and how how those 5,000 kilometers are Procured or the standards that are operated on but we're not going to have absolute control We're going to have to give up some of our controls that up. Is that a fair Description of the of the trade-off that that that we're trying to get with p4r Yes, because the the result against which the financing of the bank Is is is paid out Is the result of the old program or the on the 5,000 kilometer and not only the five So we have to agree on the result of the old government program And if that if those Program are achieved or the result are achieved with these birds So which may it's also, you know the negotiate the government must Must want that because we don't have financial leverage You need a government committee to that and uses the well bank to get competence And to have a I would say an objective assessment of results that it uses against its own vested interests All right, I just I'm just think cognizant of the time and I know So I want to call on a couple of folks. I like I said 200 RSVPs for A two-year review of the program for results I mean it's obviously that there's obviously a huge energy and interest around this more than I expected So in a cold day no less. So I suspect there's some opinions in the room So could I see some hands, please? Uh, I'd like to call on a couple of folks. I see this gentleman up here I see this woman here and I see this woman here. We're gonna have these three for this So who's got microphones? Michael Jacobs. Can you come up here? Please first start with this this gentleman here with the the coat and tie Then this woman back back there the second to last and then this this woman here in the middle Okay, just if you so in honor to honor everybody if you just name and just keep it very brief, please Yeah, I'll be very brief. My name is george couture from seal of hope My first question. I have two questions in fact. My first question will be directed to George Friedman In terms of relevance, I want to know how the world bank will still be relevant in face of the power financial power of china in these small countries around the world and look looking at the west perspective Is that the money that is put in place can compete with china and from the other people aspect Oh, is it important to turn again again toward world bank to get money? That's my question The second one will be for craig albright You are talking about the fact that the world bank need to reach out to The opinion outside of the world bank and discuss I would like to know if the world bank is doing the same thing From what I know, they don't but if they are doing the same thing in other country around the world Because the program in fact is put in place to Satisfy or come to help people in these countries Are you reaching out to these people to know how the program is impacting them? And in order for you to adjust your way. Thank you Okay, I'm gonna have craig jump in and respond to this now and jump the queue because I know you have to go soon Yeah, uh, I'd say the bank's not doing a very good job of it anywhere. So I don't think that it's It's it's not a strong suit of of the bank in any particular place I think that there's a there's a very significant program in in europe and the united states and in in japan in terms of The donor countries In terms of the developing countries, I think it's it's a it's a different kind of challenge You know, yeah, I was fortunate enough to visit some of our projects and and some of the countries in africa and meet with The people that do the external work in client countries and it's hard for them because if you're in Pick any country, I mean use gana as an example because my one of my guinean friends is here and I fell in love with gana, but It's it's relative to that particular part of africa You know, it's it's more developed But if you're going to go out and explain World bank projects to people the techniques that you You know would normally think of In in the headquarters are irrelevant For the client country. There's no twitter You know, there's no facebook although it's more now, but you know, it's not those aren't the tools you use They're more like development practitioners than they are communications people and going out and talking to those folks is is a challenge I will say that the bank has Part of what you're talking about is the realm of beneficiary feedback And I will say that the bank is a pioneer in the realm of beneficiary feedback in many ways And has figured out ways to do this more effectively over time I think the open data initiative was part of that I think being able to do the mapping work is part of that You know, everyone talks about the example of I forget the country that the you know, they put the On the school they they put what We use Uganda. Maybe they put What's supposed to be in the school and the teachers and all that And so the parents could go and look at the list and say it's not here and then demand better results There's a story about a young woman in in the philippines. I think it is who Through the world bank's mapping effort With the with the program and was able to petition the government Based on that information that was made made public through, you know, a transparency effort You know, and and there's many other people who can speak more eloquently than me about the beneficiary feedback programs That the bank has tried to pioneer and push forward So I give the bank credit for being one of the best at moving that forward But it's still really hard and it's embryonic But the the the point that beneficiaries Uh Empowered is a necessary element of the ecosystem for successful development. That point is definitely Believed at the world bank So that's an achievement that I don't know how long ago that happened but not that long ago and in the You know trying to develop it and make it more effective is is the challenge but accepting the importance of it is is Was done some years ago I'm gonna thank you. Thank you, Craig. I'm gonna we're gonna ask I'm gonna ask my other colleagues fadia and yorge to To I know they're gonna have responses to those thoughtful questions as well But I wanted to make sure that some other folks had a chance to ask some questions. So this woman back here Hi, my name is frances seymour from the center for global development And I'd like to hear a little bit more about the actual experience of implementing the program Um, and maybe fadia for you To be credible a program like this also has to not pay if the indicators are not achieved So i'm curious about what the disbursement rate has been compared to expectations And if this hasn't been a problem to what extent might we be suspicious that the indicators have been renegotiated or that the grading has been Easy to move the money this has happened in the past like with structural adjustment And then um, maybe for yorge I'm curious about the degree to which the kind of nightmare scenario that was in the minds of the ngo advocates has come out to play. I mean Have we in fact found out that forced labor has been used in building the roads? And therefore a program has had to be suspended or you know Do we feel like we know enough about what the experience is in these programs about whether some of these bad things that We feared might happen have actually come to pass and if not why not or if so, what have we done about it? Thank you Great, and the this woman here in the middle Jeremiah if you can get this Hi, i'm jennifer bremer from arizona state University and um, I guess sort of sort of not similar but sort of related questions. I'm wondering um, How the um, how the bank is engaged in in the implementation? I mean the the traditional lending activity that would often be a technical assistance trust funded kinds of things that would go along As well as bank staff engagement, and i'm wondering, you know, what how what form does the support actually take? And then the second question is how does How does the um, how do you think that that the pfr a p for r program is affecting the other the banks other operations? I mean, how is this feeding into the approach and it seems like there's still p.m. U's etc They're all kind of heavy-handed implementation m use a project management unit. Okay, just Sorry, that's like a separate unit that's set up outside the government basically to implement the project Rather than being really deeply embedded as you're describing this system is So i'm wondering what kind of uh, how you see these the things that interacting with each other Okay, great. So really good thoughtful, uh probing questions. So Uh, why don't uh, fadia? Why don't you start in terms of responding? I know there was there was three sets of questions I know you wanted to respond to then yorg I know you'll want to to respond as well Okay, thanks. I just want to add first a couple of words on on consultations I guess one way to explain it is also there are several levels that it takes place at So first there is the whole bank engagement in the country There's a very usually extensive consultation when you do a country program and to agree on what is in it And usually these have three to five year horizon And this not only with the government but also with other stakeholders and uh, it's usually quite extensive and wide When you come to the design of an operation again depends on the specific tool you're using But there's also involves a consultation process Uh, the one that's very uh, known and usually standardized the ones related to environmental and social aspects But there is also a broader consultation with stakeholders on on the the whole design of the operation One then the third level is as you make progress and you achieve results and sharing and making accessible information That's I think what uh, Craig touched on in terms of beneficiary That's been an area where actually there's been a lot more science and with it There's been a lot of more interesting Tools that have become recently available. We've seen in pakistan using cell phones For instance to report on things we've seen in others where we have used a lot of Pictures and available information But this is an area you can never do enough of so I think this is an area where you have to accept from the beginning that The more is better, but also how you get there, but I think it's at least very well recognized But it differs at the different levels of engagement and the type of products just want to clarify that On um experience with the probe for our to date. It's actually quite encouraging for the initial Now we have about two and a half years of implementation and the number of them that are like a year plus of implementation more than 10 um And the disbursement is we have some sort of projections for each week when you plan you say what do we expect it to be which are You know taking reality into account So it is actually quite on on target But your other part of the question if it doesn't happen do we disperse and then we have a couple of test cases Where things did not happen and we did not disperse a penny one of them 18 months and until things happen Then the resources so it is again changing how how we work as well and how you sort of also value some of these things Which is also Why the discussion on results and targets probably occupies The the largest part of of the negotiations and discussions And this is a good tension to have we get always the that question So do people then low ball this or high ball it because if it's linked to money then everyone will want to have Very easy things with actually a couple of cases where the government you know strategies are political statements and people put Taiwan I just Eliminate maternal mortality and you come and saying well if you're going to get paid if you eliminate that you are You really serious about this target and then you go back and you realize this is actually way out there It's nice as a speech, but it's not actually going to be realistic So we've seen both extremes and this is part of the dialogue is to really see what's the international experience What are the trends what type of analysis what type of data? It's pointing to a lot of problem in data issues in countries Which is again important We've seen actually now there's a new people are being prepared in a country to improve data systems and statistics Which is for me it was actually i'm quite excited about that because it means now we're valuing this enough to invest in it Um, so it has shown that it does have the teeth when it doesn't work But that it actually the myth that results based doesn't disperse is also not true Because for middle management, they know exactly what they have to achieve In terms of what happened in life if it doesn't match So we have a few things when people can't just change indicators these indicators link to resources are part of legal agreements So yes, you can change it if life changes, but you have to go through a formal restructuring of the operation So it's not like on this mission. I find this is low. I'll just Slash it down by 10. That doesn't happen But you know if there is a reason and the logical reason why you need to do it Then you you have a means, but it is a formal process that's subject to reviews and And explanation of the reason and what is what's needed there The nightmare scenario I I think one thing we didn't mention one before I was introduced actually There were some exclusions imposed on before art to avoid I mean because there were some who thought we will build dams with before You will not want to do that or build an airport was before art because this is where you want to have your control on the inputs You don't want to have an airport that is designed wrongly because what does it matter if you don't disperse? You've already spent a huge amount of wasted money So those were excluded a priori from our policy So some things are very huge contracts or that have irreversible impacts on environment or people we do not allow for Because we still at this stage believe at least that we really would like to have the a priori Controls on on those even though you could do that post but it is was the decision the institution made and in terms of implementation The best way to describe it is it moves a lot of like pre design issues to much more problem solving hand holding issue a lot more Capacity building we just did a round of interviews with countries a lot of requests for more capacity building and engagement from the banks So it's in some way a bit heavier but different in nature Because it moves from looking at transactions to seeing how what are the problems why you didn't achieve this result How can we work together and technical assistance can come in many ways either through the program Or with sometimes a lot of partners who are supporting that it depends on on each country situation Okay, so far you know no nightmare scenario so far clean clean bill of health on on p4r projects So far. Well, I find nightmare, but so far no no nightmares, right? So no nightmares. Okay Oh, you're sleeping. You're sleeping well at night. Okay. So so your Can you just I know these were some very thoughtful and probing questions Could I just hear your your thoughts from the board, please? Yes to intuition public policies and on on safeguard I mean This program this is about improving public policy and and and therefore maximizing the development impact of investments independently of who finance them Mainly they are financed by all resources. I mean I speak about the Chinese in the case of infrastructure But that's not the usual case the usual case is budget finance expenditure requirement expenditure in health and education and so on The precondition is the existence of a reform minded Uh Technocracy in the ministry we work with I mean these are demand-based. They can't function only if We have partners that want to improve the situation and looks for Technical support and supporting legitimacy in order to implement reforms And and that's that that's the critical point. I think it also what probably makes technical assistance and institutional support more Promising is the fact they are demand driven by a group of people who want to achieve those results And the fact that they shift financing on results give them in their own system Legitimacy is an leverage discussing with politicians other part of the administration and so on and so forth That's the realistic issue. We are dealing with is to put our intervention in the economic But in yeah in the political economy of the country We are dealing with so of course The risk is that if you if the if the reform mind is minister goes Or if there is a change in the people that initiated this the things could die or or lose Lose relevance and impact that's clearly but that's frankly is true of so many projects that last for years We know very well how many changes in pmu and other You know personally or in in a secretary of of a certain ministry a bad on our project That's a vulnerability. That's unavoidable in public policy. Anyway, you are changes in administration in your country and so on On safeguard, I mean, let me speak about an example that I know relatively well. It's about forced labor in Uzbekistan This has been a very controversial issues as you know, we have had an inspection panel on that that have just concluded We set a number of recommendation The bank faced to the challenge of Child and forced labor in Uzbekistan Comes up with a project. This is it is a traditional project that says in the x thousand actors We contribute to cultivate through mechanization and others there will be no forced labor Okay, the government sign off and we have An independent monitoring and in that part of the country we are safe Now is this enough? I mean, I'm not happy about that because I Forced labor will continue elsewhere and I don't have an instrument to discuss with government what happens outside the perimeters that I'm irrigating So my concern, you know, I recognize the problem But I'm not happy with the solution that a known that a do no harm save god gives me I'm more and more interesting to see if the government is is is ready to try to To eradicate forced labor in Uzbekistan and I'm looking for an instrument For instance a program of mechanization of the cotton sector linked to certain results That would address at the country level with inspection by ILO In the an international convention that prohibit forced labor that will address that problem You know, I'm just giving you this this example because it's the way I look at it Again to have clean to have to have no forced labor in in what the bank does is not good enough at the end of the project The story is ended. So I I'm that's the kind of dilemma we face in the area of safeguards we want to improve Standards in the country in the legislation in the way the government deal with these critical issues And we must find a way to engage the government at the level of its own Global policy beyond what we can ensure in the perimeter in the places where we do our own projects and we check everything York, thank you very much. I I promised everybody we'd end at 2 30. We're going a little bit over I think this has been very enlightening. I appreciate the world bank colleagues coming I want to thank yorg in particular for coming from the the board and thank you fadia and Really appreciate craig also coming back to reprise his role of And it's reprising his past life role at the bank. So please join me in thanking the panel