 I want to start this talk by asking you all to cast your minds back to 2013. As you do this, you'll probably remember that 2013 was a year in which two very significant events occurred in the world of Australian aid. The first of these, of course, was an election in which this man was replaced by this man as the Prime Minister of Australia, and then shortly afterwards this man's government started enacting very radical changes to the Australian government aid program. First, AusAid was disintegrated and all its functions were reintegrated into the Department of Foreign Affairs and Trade, and second, over a subsequent year or so, more than one billion dollars was cut from the Australian government aid program's budget, so greater than I think about a 20% cut to the Australian government aid program's budget. The other significant occurrence in 2013 was this was the first year that the Development Policy Centre ran its Australian aid stakeholder survey, and the 2013 stakeholder survey was run just prior to the election, and as such stands now as a snapshot of Australian aid as it was. It stands as a snapshot of Australian aid prior to the disintegration or the end of AusAid, and it stands as a snapshot of Australian aid when budgets were growing or at least static and not being cut. Of course, the reason I'm standing here in front of you today is that in 2015 we repeated the Australian aid stakeholder survey. We used the same method. Participants filled out a survey form online, very similar to that one, and as in 2013 we ran the survey in two different phases. In the first phase, we targeted senior executives in all of Australia's large aid NGOs and all of Australia's large aid contracting companies as well as senior executives in a sample of smaller aid NGOs. The second phase, we opened up to the public, anyone with internet access and an interest in Australian aid was able to participate in the first phase. So the first phase was a targeted sampling of experts, the second phase was an attempt to broaden the reach of our data gathering to gather the opinions of a wider range of Australian stakeholders. If we did that in 2013, we did that in 2015. Importantly, in 2015 we used almost identical questions to those that we used in the 2013 survey. And because of this, we're able to compare the 2013 results to the 2015 results, and that's what I'm going to do in this talk. By doing so, I'm going to try and answer questions such as whether the dramatic changes to Australian aid in the last two years have made the Australian aid program worse, whether they've made it better, whether they've had no impact at all. So in a moment we'll see what Australia's aid stakeholders think. But before we do that, I just want to speak to a couple of methodological points. The first of these is I want to address an objection that some of you may have about the type of data that we're using. And this is that our data come merely from the subjective assessments of Australia's aid stakeholders. They're not some sort of perfectly objective gauge of the quality of Australian aid. And up to a point, I think this is actually a fair objection to the process that we've undertaken. However, it's worth remembering that there is no perfect, objective way of gauging the quality of an aid program. You can't run a randomized control trial on an entire aid program. The sorts of reviewers that DFAT or the OECD conduct are themselves subjective. They're based on the opinions of a few reviewers. And then indices such as those used by people like William Easterly and the Centre for Global Development have subjective decisions underpinning them, what to include, what to exclude. And they're also only very skeletal gauges of the quality of an aid program. What we have with our stakeholder survey is a much richer set of data. It's also important to note that the data that we're drawing upon comes from the opinions of experts. So I think something like 94% of the participants in the first phase of our stakeholder survey this year have been working in the world of aid for more than five years. And these are all people who interact with the government aid program on a very regular basis. So they're people who know what they're talking about. As such, their opinions may not be the final word, but they're definitely worth paying heed to. The second methodological point I want to make is just to state at the outset that today this talk is going to be based on the data from the first phase of our 2013 and 2015 surveys. I'm going to call this phase one. And that, as you recall, was the phase where we targeted the opinions of executives from large aid NGOs, a sample of smaller aid NGOs, and large aid contracting companies. Now, those of you who filled out, who participated in the second phase of the stakeholder survey, don't despair. Your opinions have been drawn on in a number of places in the report. And also, and this is quite interesting, and readily apparent from this chart behind me, right? Well, I thought it was clear. On average, the responses that we got to questions were very similar when they were answered by phase one respondents and when they were answered by phase two respondents. So there were some small differences, but on average, the overall responses we got to the different questions were very, very similar. And this means, in effect, that if I were to give this talk today, based solely on phase two data, or to give this talk based on phase two and phase one data combined, I'd be delivering a very, very similar talk to the one that I'm about to give right now. All right, so on to the results of the talk. First up, I need to forewarn you. This is going to be a fairly gloomy talk. The bad news will outweigh the good. And so because of this, right up front, I want to emphasize one very important piece of good news. And that is that in 2015, when we asked stakeholders how they would rate the effectiveness of the Australian aid program, the majority of them, a clear majority of them said that the aid program was quite effective. You can see this behind me in this chart. And because this chart is going to be as fairly typical of the charts I'm going to use throughout this talk, I'll just explain it very quickly. The chart is a percentage bar chart. Each block is associated with a particular response to the question that we asked. The color of each block is tied to the answer that you can see in the legend there. Dark red is always the least positive answer. Orange is kind of a moderately negative answer. Gray is associated with neutral responses. Light blue is associated with quite positive responses. And dark blue is associated with very positive responses. And the relative size of the blocks in the bar chart give you a sense of the percentage of respondents who answered in each of these ways. As you can see in this chart, the light blue block is by far and away the biggest, reflecting the fact that most stakeholders, when we asked them in 2015 about the effectiveness of the Australian aid program, said that it was quite effective. And to my mind, this reflects a real achievement on the part of the aid program. Given the bombshells that have been dropped on it in the last two years, the fact that most stakeholders still think that it's running quite well is unquestionably good news. And it's a testament, I think, to the hard work of aid program staff and DFAT staff more broadly. So if you're in the room and you're from the aid program, quietly pat yourself on the back for this. You help deliver this piece of good news. Unfortunately, however, the good news is not the only news. And I'm going to now shift into a slightly more gloomier phase of the talk. So perhaps the most significant piece of bad news that we received as we analyzed this report came from the question when we asked stakeholders whether they thought the effectiveness of the aid program had improved or decreased over the last two years. Here, as you can see by the size of the dark red and orange blocks in the 2015 bar, almost three-quarters of stakeholders said that they thought the aid program had become less effective over the last two years. And in 2013, on the other hand, when we asked people whether they thought the aid program had become more or less effective over the previous 10 years, as you can see from this chart, there's a striking contrast. More than three-quarters of respondents in 2013 thought the aid program was improving. So what we see is a really dramatic turnaround in the space of two years. An aid program that had been seen to be improving is now in the view of our stakeholders seem to be getting considerably worse. What's more, we didn't just ask stakeholders about the effectiveness of the aid program overall. We asked them a series of questions about just how well the aid program was performing with respect to specific attributes of the work that it did. And their responses to questions about these specific attributes are charted behind me in this chart. I know it looks a little bit mystifying at this point in time, but I'm about to clarify just what it shows us. So each point on the chart is associated with stakeholders' responses to the specific attribute that we asked them a question about. And don't worry, I haven't attached names to the points, but all the names of the various attributes that we asked about are attached to the points in a chart in the report. What's more, I'm going to discuss some of the individual attributes in a second. The main point for now is that the location of the point on the chart is a function of how positively the stakeholders appraise the attribute that it's associated with in 2013 and how positively they appraise the attribute that it was associated with in 2015. So points situated over here towards the right of the chart are associated with attributes that were appraised fairly positively in 2013. Points over here to the left are associated with attributes that stakeholders appraised fairly negatively in 2013. Points up towards the top of the chart are associated with attributes that stakeholders appraised fairly positively in 2015. Points lower down on the chart are associated with attributes that stakeholders appraised fairly negatively in 2015. And what you're really interested in here is the light blue line that cuts diagonally across the chart. This is a line that plots a one-to-one relationship between the two years. Any point that is below that line is associated with an attribute that was appraised less positively in 2015 than it was in 2013. These are the points colored in red. Any point above that line is associated with an attribute that was appraised more positively in 2015 than it was in 2013. Can anyone tell me anything about the distribution of the attributes on this chart? Yes, no, thank you. More than three quarters of the attributes that we asked about were appraised more negatively in 2015 than they were in 2013. Another thing that you will want to note is that the distance of a point from a chart gives you a sense of just how much the attribute that is associated with changed between 2013 and 2015. Attributes that are on the line changed very little between 2013 and 2015. Attributes that are further away from the line changed a lot. And as you can see, there are a few points that go better, but most of them didn't shift by a heck of a lot. Whereas on the other hand, quite a significant number of the points associated with deterioration lie quite a long way from that line. So, well, the three quarters of attributes got worse, and some of them, as you can see from the location of the points on the chart, became a lot worse. So here are some examples of attributes that became worse between 2013 and 2015. The first of these is funding predictability, and this really dropped off a cliff. So this is the extent to which stakeholders felt like they could rely on the aid program as a predictable provider of aid funding. As you can see in 2013, something in the range of, something in the range of about 40% of stakeholders thought that funding predictability was a weakness or a moderate weakness of the aid program in 2013. So this isn't a sort of particularly positive appraisal, but when you contrast it with 2015, you see a real change, right? So over 90% of stakeholders in 2015 thought that funding predictability was a risk or a great, or so it was a moderate weakness or a great weakness for the aid program. So here's an example of an attribute that has become a lot worse. Of course, you could say that funding predictability is one thing, and that most of the deterioration here is associated with something of an outlier event than cuts to the aid budget in 2015. Of course, it's worth mentioning in response to that, that there are further cuts to the aid budgets slated for the next financial year too, so it's not as if we can be that confident that we've moved beyond an era of budget cuts. But more importantly, we saw deterioration in a whole range of attributes that have nothing to do with funding. One example of this is transparency. As you can see in 2013, transparency was appraised pretty positively by stakeholders. Only 21% of stakeholders thought transparency was a moderate weakness or a great weakness for the aid program in 2013. On the other hand, in 2015, 58% of stakeholders thought that transparency was a moderate weakness or a great weakness of the aid program. And it doesn't cost a lot to be transparent, right? And yet, here's an example of an attribute which ought to be easy to uphold to keep, you know, it's ought to be easy to provide quality and it's an example of an attribute that has become a lot worse in the last two years. To me, this is concerning not only because transparency is an integral component of a well-functioning aid program, but an aid program ought to be transparent because it's taxpayer's money that it's spending, right? And yet, over the last two years, if stakeholders are to be believed, the Australian Government Aid Program has become considerably less transparent. Another example of an attribute that's got worse is strategic clarity. In 2014, in 2013, only 14% of stakeholders thought that the aid program's strategic clarity was a weakness or a great weakness. In 2015, on the other hand, just over 50% of stakeholders thought that strategic clarity was a weakness or a great weakness. So to me, this suggests that the intellectual cohesion of the aid program and the mechanisms for translating ideas at the high level into practice within the aid program have become significantly worse over the last two years. Another example of deterioration comes in the form of staff expertise. Now, to be fair, as we discussed in the report, responses to some staffing-related questions actually improved slightly between 2013 and 2015. In particular, in 2015, stakeholders were more satisfied with the length of time that their primary point of contact for aid program-funded work had been in place than they were in 2013, which is good. Indeed, to me, that seems like a real achievement given the termult of the merger. However, staff expertise, on the other hand, is an area where we see deterioration. So in 2013, just over 50% of respondents thought that staff expertise was a moderate weakness or a great weakness. In 2015, on the other hand, nearly three-quarters of respondents thought that staff expertise was a moderate weakness or a great weakness. And this is a real issue, because giving aid well is hard. And if you don't have sufficient expertise within your organization, all the beanbags in the world won't save you, right? So this is a really concerning change. Expertise is crucial. I was wondering if you're gonna laugh at that one, where the tic, expertise is crucial. Yet if stakeholders are right, and in this area, I certainly think they are. Expertise is on the wane in the aid program. Now, I was encouraging to hear Secretary Vargase talk yesterday about a recognition, of sort of explicitly recognizing his talk that staff expertise was important and that the DEFAT were gonna work hard to retain experts within the aid program. But the question that needs to be asked is how did this ever deteriorate in the first place? Who let this slide? Like, what sort of attitude to aid comes accompanied with a willingness to let experts depart from your government aid program? So the aid program is seen by stakeholders to have become less effective overall and less effective across a range of specific attributes. What's more, the stakeholder survey in 2015 pointed to changes to other areas of concern. Foremost amongst these is a perceived change in the ethos of Australian aid. In 2013 and 2015, we asked stakeholders the extent to which they thought that Australian aid was devoted to helping poor people in poor countries, vis-a-vis the extent to which it was devoted to advancing Australia's strategic or commercial interests. And I won't go into this chart in too much detail, you'll be pleased to know, but the light blue line reflects the relative frequency of responses in 2013. The dark blue line reflects the relative frequency of responses in 2015. The x-axis goes from zero to 100. So the further you get over here, the greater the extent to which respondents thought that the aid program was devoted to addressing issues associated with poverty as opposed to advancing Australia's strategic and commercial interests. The peak of each of these curves shows the most frequent response that stakeholders gave in each of the two years. In 2013, the most frequent response we got when we asked this question was that about 50% of Australian aid was devoted to tackling issues associated with poverty. In 2015, on the other hand, the most frequent response was that 20%, just 20% of Australian aid effort was devoted to issues associated with poverty rather than advancing Australia's own interests. Now, to my mind, both of these assessments are a little bit uncharitable. If you'd asked me this question, if I'd filled out the survey, and I can assure you that I haven't, my estimates in both years would have been somewhat further over here. I don't think that just 20% of Australian aid is devoted to helping poor people and poor countries in this day and age. However, setting aside absolute assessments, the relative change is still striking. And I think stakeholders are correct in their, at least in the terms of the relative difference between the two years. It certainly seems from the outside as if there is a lot less focus on tackling issues associated with poverty in 2015 than there was in 2013. And importantly, this isn't what the public wants. Whenever they've been surveyed and asked what they want Australian aid spent on, the vast majority of respondents say that they want Australian aid devoted to helping people in other countries rather than advancing Australia's own interests. So that's what taxpayers would like Australian aid spent on. However, if our stakeholders are to be believed, the current government is doing a lot worse at that than its predecessor was. So that's quite a lot of bad news, right? The big changes of the last two years have exacted a real cost. Nevertheless, I just want to emphasize this one important piece of good news. Most stakeholders still think the aid program is quite effective. And this is definitely good news given all that has happened in the last two years. Not all is lost when it comes to Australian aid. However, the flip side of that equation is that there's still a lot left to lose. And given the trends of deterioration, there is clearly no cause for complacency when it comes to Australian aid. Real effort needs to be made to turn matters around. Which begs the question, what is to be done? Needless to say, the report, our report has a long list of recommendations, but rather than recite the list to you, I want to close in a different way. I want to talk about the sort of work that's required to help repair Australian aid and the sort of work that's required from all of us, not just people in the room who are from the aid program. Certainly some of the issues that we have identified in the report are the sorts of issues that can be tackled by DFAT themselves. And hopefully some of them will be addressed by DFAT. Certainly in the last couple of months, we've seen improvements in the availability of data and transparency on the DFAT website, which is very encouraging. Hopefully that will continue. And it was also encouraging to see the here secretary Vargas' comments about this newfound recognition of the need for staff expertise. The proof in the pudding has yet to arrive in this area, but hopefully it will. So some of the changes that need to be made can be made by DFAT. However, some of the problems of Australian aid require a recognition in political circles of what's gone wrong in the last two years and a real desire to rectify matters. To really improve Australian aid, change needs to come from the political sphere. Now here, perhaps stakeholders' assessments give us some cause for hope. When we asked stakeholders in 2015 about the quality of the political leadership the aid program had been on the receiving end of, as you can see from this chart, most stakeholders thought it was poor or very poor. But as you can see from this chart, when we asked stakeholders for their assessment of minister bishop, stakeholders were substantially more positive, which suggests that stakeholders view many of the big problems that the aid program has faced in the last two years as being born of other parts of the Abbott government, rather than being minister bishop's fault. Which other parts I'm not sure, but I'm just trying to pass the stakeholders' opinions here. And of course we have a different prime minister now, so maybe I do have an opinion on that. And apparently minister bishop is considerably more powerful. We also have a new minister for development in the Pacific. And these changes may really lead to improvements in Australian aid. There's also an election coming up, which might change things further too, who knows. However, I think the Australian aid community would be very wise not to assume that the politics of Australian aid will become better just like that. Rather, all of us in this room who care about aid need to work to create a political environment where there is support for giving aid well and an understanding amongst politicians of just what good aid needs. Our work doesn't necessarily need to be oppositional. Hopefully there's fruitful collaborative engagement to be had with the government and with future governments. But one way or another, those of us who want better aid have to exert more political force. And if you ask me the big lesson of the last two years, is that we all have a lot to learn about making Australia a country whose politics points in the direction of good aid. It's possible, as we can see from the United Kingdom, but it's not happening at present here. And we don't really know what we need to do to change that. I'm not saying that we're currently doing nothing or that we know nothing, but there is a lot of room for improvement at our end. Improved understanding of and engagement with the domestic political economy of Australian aid. Good aid does not happen. It does not just happen. It has to be made to happen. And we all have a role to play in this. So the story I've told you today sounds like it's the story about the aid program, but would be wise not to conclude that it's all just the aid program's fault and the rest of us can be absolved of what's happened. We need to work hard to find ways to help foster positive change. And on that note, I'll finish by saying that if you share my interest in how the aid community can become more effectively involved in pushing for better Australian aid, there's a panel on the stakeholder survey straight after morning tea. Stephen discouraged you from coming, but I want to encourage you to come. And it's going to take place in this lecture theater. We also have another panel on the topic of the Drivers of Aid Policy, which is taking place in the Western Theater at 3.10 p.m. today. This is a panel chaired by Chris Roche. So those of you with an interest in these matters, I'd really like to encourage you to come to that panel too. And then hopefully the Development Policy Center will be holding further similar events later in the year. And in these four, and once again, I'd encourage you to come to these four, because hopefully in these four, we can learn more together about how we can work together to create a better future for Australian aid. That's all for the future though. In the meantime, I'm very happy to take your questions. Thank you. The 2014 ANU poll, and I can't remember the exact numbers, but it was just an overwhelming response in favor of aid being given for the purpose of helping other countries rather than advancing Australia's own interests. Indeed, even when I filtered for a political party and looked just at coalition supporters, the vast majority of coalition supporters wanted Australian aid to be given for the purpose of helping people overseas. Patrick, have you got any money? In the second phase of the survey. Yeah, the second phase of the stakeholder survey, we did encourage people from the Pacific to participate and we got some responses, but not enough to drive this survey results at all. And the challenge is, as I'm sure you'll know, having worked in the Pacific, trying to elicit a conversation, particularly as you start from the outside, and this is the first time you've engaged with a particular person or a particular government department. It takes a lot of effort and a lot of time. We'd love to do it. I completely agree with you. It's important, but we don't have the resources to do it at this point in time. With respect to the vested interests comment, I'm not that sure that we can dismiss the responses of the stakeholders as being the responses of vested interests. Certainly, perhaps when you ask stakeholders about whether they'd like more aid delivered through NGOs or through contracting companies. Yeah, and maybe when you ask questions about aid levels, but what's the vested interest in asking that your aid program be more transparent or that it have more experts in it. So to my mind, vested interests don't really skew the responses we get there. But even then, right? I mean, it seems like a reasonable thing to ask for as a taxpayer that your money's been spent effectively. Then on the I-word, we didn't include the I-word in the survey. However, we did have a question about sectoral focus and the vast majority, how to frame this. There was a clear majority of respondents, both be they from NGOs or from aid contracting companies thought the Australian aid program was focusing too much on economic sectors and that it wasn't focusing enough on social sectors at this point in time. There was one really bad result in both surveys. Bottom left hand corner, your diagram. What was that? Welcome to my appendix slides. You know you're dealing with a real nerd when they have additional slides in the appendices. That was to do with predictability of funding. That was appraised. That was... Oh, staff continuity. So predictability of funding was appraised quite well in 2013, but very poorly in 2015. Staff continuity was appraised very poorly in both years. Jonathan Pryk from the Law Institute and the Sustainable Co-Author of the First State Court Survey. Yes, sorry. So we have a baseline now and we have what I'm hoping, and I think everyone in the room here hopes there's a negative outlier in the results from this survey. Do you plan on continuing the survey in the future? Is it even given the limitations that you highlighted at the start? Thanks. Duncan McIntosh from the Internet Registry for the Asia Pacific. Do you see any value in trying to do some comparative work on government aid versus private aid, private sector aid, and seeing what the perception is of the success or effect or focus of privately funded aid coming out of Australia? So you mean individuals' donations to NGOs? Yes, foundations, organisations with their own funds who are doing development work. I mean, produce the survey. What kind of discourse do you plan to have with DFAT and with other stakeholders about the results and the implications and the forward agenda for change? OK, so starting with the question on what we plan to do in the future, nothing is certain and everything is funding contingent, but I'd love to keep doing these stakeholder surveys. I think 2015 in particular has indicated just what a valuable tool they can be as we've captured stakeholders' perceptions during a time of dramatic change. My personal opinion is that we don't need to do them every two years. We should probably do them every three years, once in an electoral term. That seems to be a natural cycle. And the other thing I'd like to do is refine the questions slowly over time. We don't want to abandon the existing questions because that's what gives us our time series. But I think each time we do the survey, we learn something new and we can think about ways of asking questions in an improved manner. And so on that note, if any of you took part in the survey and found that there was a question that was particularly hard to answer, by all means email me and tell me about it. Government versus NGO aid. Obviously, there's not much point, perhaps, no offence to anyone in asking NGOs' stakeholders about just how good their own aid is. Indeed, we did sort of ask that. You can find the results in the report. Not just NGOs, also contractors. Clearly, what we'd like to do in our public opinion survey work is learn more about what the public thinks about aid that flows through NGOs. That's a good point. Thus far, most of our public opinion survey work has been about the government aid program, and we should definitely start including more questions on NGOs. And also, yeah, I mean, on philanthropies and other sources of funding, good point. And the final question, you obviously come from the other part of CAP if you're using words like discourse. We are very happy with the engagement we've had from DFAT thus far, and find that very encouraging. And we're certainly planning on entering into further discussions with them about just what our findings have shown. Certainly, that's been a change from the atmosphere at the time of the previous survey, and it's really good news, I think, that DFAT are willing to engage with us and have had very productive engagement with us thus far.