 Now I'm deliberately going to make you a little bit uncomfortable and ask that you open your mind and allow yourself to perhaps consider evidence in a different way than you have before. And with that in mind, I'm going to tell you that I may not answer the questions but rather pose a lot of questions that you'll need to consider as time goes on. So with that in mind, let's consider the idea of evidence. We know that evidence concerns facts intended for use in support of a conclusion. Now I'm going to present to you the idea of evidence, the creation of evidence and how we might test and assess evidence. So get ready. I think therefore I know. We're often commonly and even as academics, we think therefore we think it's fact. But in actual fact, our thoughts aren't fact. And some even argue that our thoughts aren't even reality. They're not real. Now when many people, I did an interview this morning, call us called in and they said, I think where does that come from and how powerful is it when it comes to us exploring our world and understanding the ways that we know something, the evidence? Can we really ever know something? It's dynamic. Knowing is dynamic. So we've got to be careful that we were open to that, the nature of evidence being dynamic, that one day we might find something out and the next day that might be completely reversed. But I think more importantly is how will we know about something? How will we know the best way to transport Canberra into the future? Is it by light rail? Is it by buses? Is it by cycle? So what are the ways of knowing? You've all come here and you're all eagerly awaiting my opinion, my thoughts. Are my thoughts reality? So here are the ways of knowing. Authority. So the commonly held notion of knowing things is through authority. We may be told by a teacher or by someone in authority. A tradition. When I was little I never stood on cracks because I was worried we all knew it was, you know, you'd fall into the pitch in the pavement as the cracks opened up and swallowed you in. It's common sense, of course that's common sense. Sounds logical to me that I'm going to be sucked in if I tread on the cracks in the pavement. The media, the media has a lot of control when it comes to framing our ways of knowing because they offer bits and pieces, the whole of the story or little aspects of the story to frame our notion of what's going on. But experience. And this is one I'm more and more coming to as someone who was a quantitative scientist to begin with, quantitative social scientist. I would never have said to you that how powerful experience was ten years ago and I'm going to ask you to reconsider the power of experience. That is the way that we live our lives, the things that we're exposed to that are unique to us actually frame the way that we think and are real forms of evidence to us. But on the other hand we have the idea of scientific evidence, scientific ways of knowing and that is that there is an approach to knowing and that that is through a systematic collection of data, a translation or analysis of that data to inform and then of course an evidence base. What I want you to consider, I want to push you out of your notion of perhaps thinking that scientific is the only way to know and consider this that rather ways of knowing is along a continuum. On one hand we might have the extreme of the common ways of knowing, I think therefore I know and on the other hand the scientific idea of evidence. Now as I promised I'm going to push you a little bit further and consider the idea of lived experience and expertise. So our experiences and observations inform facts, inform facts to us and perhaps to a group of people. Now consider this, when we go out and survey people or we're asked to interview respondents through qualitative studies our respondents are actually experts. They hold a level of expertise as data points if you like. But they should be approached as such. They carry with them a lived experience of difference and in particular this notion of diversity and adversity. Take for example navigating railway stations in a wheelchair. The ABC at the moment is actually launching a survey to consider whether or not transportation in New South Wales, rail networks are truly catering for people with mobility issues. How would we know there's an issue if we don't have to navigate Redford Station which only has one lift for one platform if we're not in a wheelchair ourselves? So that lived experience does grant us an insight or knowledge that is powerful and should not be underestimated. But on the other hand, perceptions are more powerful than reality and that is a worry. Take the idea of African gangs. Perception is more powerful than the facts, than the data. Why? Because our perceptions influence our Bolivia. The question man comes to while valuing lived experiences how do we undo biases formed through them. So I offer you my sniff test, a toolkit for assessing evidence and so I've been thinking quite a lot about this idea of well how do we find out if something's right or wrong? And I offer you the idea that we should consider the claim the who, the what, the where, the source, the platform, the publication, reporting of the author, the context, the remarks around it, the response, the prejudice, the fear, the politics and then the qualifiers, buzzwords or scientific language whether there's language that then goes into moderate or mediate what's being said or offered as a fact. And then consider what are the consequences? Are there any harmful consequences? Is it helpful? Does it hinder? Does it promote? And then we can come to a verdict to challenge either facts that have presented to us or facts that we believe through our own experiences that then go on to influence our behaviour. I'll offer you an example. So Lucinda and I recently worked on a fact check for the conversation and the claim was where the highest growing country in the world with 1.6% increase and that's double than a lot of other countries. This was offered by the One Nation leader as a fact. Kind of looks like a fact. Has data in it? Looks pretty legit. So let's consider the source. A spokesperson after being contacted by Lucinda at fact check said that well actually the World Bank data says that Australia's population growth was 1.6% and that's much higher than the comparable countries with immigration programs like Canada, again another statistics, the UK and the US with statistics. It's quite legitimate. So what's the context? It's being used, these data, this fact to argue for change, to argue for a popular vote on the future of our immigration program. So it's a high stakes game. The qualifiers. So as I said that 1.6% looks quite legitimate. It is in fact legitimate. And then we kind of consider them what else might be going on there? And the spokesperson said the talks about population growth and the context of our high level of immigration because in recent years immigration has accounted for around 60% of Australia's population growth. So the concern is not about growth but something else. Something else within that component of growth. So the sniff test, the consequences could be harmful or they could actually spark a conversation about the future of Australia. The verdict, there were half truths and the half truth is a pretty good lie because it has borrowed legitimacy. And that idea, well it's half right, yes 1.6% but there's more going on there. We need to unpack that. So in conclusion there are myriad ways of knowing lived experience is evidence and shouldn't be discounted but the level of evidence and the fitness for purpose and what we accept along that spectrum should be dependent on what we have on offer. And I offer you today the sniff test and hopefully you'll be able to use that to challenge your own experiences but also to hold others to account. And I'll leave you with that. So I'm the fact check editor at the conversation. My job is to work with academics like Liz who's one of our superstar fact checkers and take claims that are made by politicians and other people of influence in society as you saw this example and test it against the best available data and research. We've been doing this for five and a half years and checked hundreds of claims right across the political spectrum and all sorts of issues from immigration to education, health funding, terrorism I couldn't name on subjects. Now so obviously I've had a lot of time in my role to reflect on how successful fact checking is and it certainly is successful in some ways. We know that fact checking operations all over the world overturn harmful false stories every single day and we know that there is a hunger for people to read and share these articles and we know that people appreciate having that kind of clear evidence on the table. What we do is we test claims against data or very high quality research so there's got to be a strong foundation to test the claim against and we need to strip out opinion, conjecture. So hopefully that is a very high standard that kind of rigor and we invest a lot of time in these fact checks and we would hope that that would be enough that bringing these fact checks to people would be enough to convince them of the evidence. It's not our job to convince people but obviously we do want people to be making decisions based on the best available evidence. But it's not as simple as just giving people the facts and that is because we are humans and we have feelings and we're all subject to a range of psychological and social factors that can hinder how we take on-board new information. So one of those factors is called the illusory truth effect. The more often we hear a statement, the more familiar it is to us, the more likely we are to believe it's correct even when it's not and that's because new information is quite hard for our brains to process. So the more we've heard it, the more likely we are to believe it's correct. We're all subject to these by the way this isn't just our foes, it's all of us. We're also subject to confirmation bias and motivated reasoning which is our tendency to seek out the evidence that supports our existing views and to reject or reinterpret evidence to the contrary that challenges us or question it to the point where we just disregard it. We also have a tendency to accept facts when they come from people we trust, people from our own communities, people from our side. Everyone doesn't take sides in issues. And even those same facts when presented by someone coming from a different point of view we could reject exactly the same facts so that's something we all need to be aware of. These sorts of biases don't, they're not overcome by expertise or education or intelligence. It can actually be the most educated people who can be the most polarized on a subject. We're also as Liz mentioned really influenced by our life experiences and our feelings about what's happened in our lives. So if you've experienced something for yourself and then I bring you a fact check that says that's not really happening because it's only happening in 1% of cases but you're in that 1% then that fact check isn't necessarily going to speak to you. Now we've all been in that situation where we're having an argument with someone or we're reading someone's Facebook post usually someone's aunt or uncle they seem to always be the culprits and we're reading it and we're getting our hackles up and your instinct is just to jump in and tell them that they're wrong and try and convince them of your point of view and hopefully reaching for the best evidence to do that but even so our motivation is to change their minds. Now I would always encourage everyone to reach for the best evidence in a disagreement but I think where we're going wrong and where we can be doing better is in acknowledging other people's feelings and perspectives before we reach for the evidence. So acknowledging things that they've lived through seeking out points of commonality so often a fact check is commissioned because there is a point of disagreement often there will be a point of disagreement perhaps there is genuine confusion about the facts or perhaps there's a lot of fear a lot of fact checks are based on feeding into people's fears and those fears are very real so if we can find out what is behind the other person's belief where did they get their information from in a kind way like acknowledging the legitimacy of their perspectives but is there something they're afraid of or worried about and acknowledging that thinking about finding common ground you know for example vaccinations the two sides of the argument they're based on fears for children so if you can find that common ground beforehand and thinking about the fact that within our social groups there are often commonly held beliefs so if you're challenging somebody's commonly held belief and asking them to take on new evidence that challenges that just thinking about the power of what you're asking them to do you're asking them to kind of step away from something that might be part of their belonging in a social group which is an absolutely huge thing to ask somebody to do lost my spot now so while we're having these disagreements acknowledging the other person's point of view also acknowledging how you came to your perspective and only once all of that groundwork has been laid with the compassion with the empathy with the time investing time in people finding that connection only then really is it time to bring in the facts and the evidence because we really need to lay that groundwork if that seed of evidence has any chance of taking hold in someone else's mind and importantly not trying to win arguments on the spot just leaving that little seed in there hoping it's going to grow but just walk away because we know that we double down on our positions when we're challenged too firmly I can see through our readership and through all the academics who work with us there are so many people working to bring out the best evidence and people who are responding well to the best evidence and where we can probably be doing better is just in our interpersonal discourse to make sure that when we're trying to spread the best evidence to other people we're giving it the best opportunity by connecting on that human level Thanks Well what I'm going to try to pick out of this wind energy conflict is a story about the politics of contested knowledge and contested truths it certainly was the case in King Island and so all of this started in 2012 in King Island which is a small island about halfway between Victoria and Tasmania where a very large wind energy facility was proposed to the community and so this would have been about 200 turbines the numbers weren't fixed but about 200 they would have been about 150 metres tall each and this is a photo of Curry Harbour so this is the main town of King Island it's a beautiful place and it's not terribly developed in the way that we picture a lot of our landscapes so the reason I say that is that 200 turbines that are quite tall would have had a significant impact on the landscape well man it's so interesting from a research perspective is that the company who proposed the development did what we would call folk interested in community engagement as being best practice community engagement from a distance and on paper at least it looks that way I won't go into what it was about it that made it look like best practice but you'll just have to trust me that it was because I'm an expert and so where that became extra interesting was that despite doing all of these things that it meant to limit conflict we saw pretty disastrous outcomes in King Island in terms of the impacts on the community so there's under about one and a half thousand people in King Island it's a regional community and in places like this it's especially important that social cohesion and interconnections between people as individuals or citizens or business owners stay strong because it's often those close ties that are so critical but this is a quote from interviews that I did with people who live in King Island this was from 2015 about two and a half years after the proposal was announced about five months after a subsequent announcement that the proposal would not continue so this is somebody talking about their experience during the time not that a wind energy facility was being developed but during the time that there was a discussion about whether a wind energy facility should be developed so there was no material change to the island but nevertheless there were substantial impacts on the people in the community there we can develop these things wind turbines, wind energy facilities and people can be very happy with them but the complication that happened in King Island was that not everybody agreed with the company that it would have been a good idea for the community so I'm going to take a little side step to give a small amount of insight from social psychology so this is also my background understanding I'm conflict using social psychology and it speaks to some of the themes that Lucinda mentioned but when not everyone agreed with the company that this proposal should go ahead an opposition group formed what we know from social psychology is that when people are in groups, they behave in a particular way if you identify with me we're part of the same in-group because we're together and we see ourselves as being similar but if we see people that are different from us we see them as an out-group so what happens once we get this intergroup situation is that there's a lot of trust within the in-group there's a tendency to see consensus with people on your in-group as opposed to disagreeing with them even if they might be wrong there's problems between an in-group and an out-group where the leaders of those groups become more polarized because they start to define themselves not in terms of necessarily representing the interests of their group but in terms of finding distinctiveness in the group that they see as being different from them but we see toward out-groups a lot of hostility suspicion about people's motives mistrust of what they're doing but also stereotyping of who they are and that leads to dehumanizing them so when this sort of dynamic happens it can be very problematic for coming to some good outcome this was certainly the case in King Island the conflict was described as being polarized very wind groups and anti-wind groups formed and people were divided and so this wasn't insignificant there were stories of people that wouldn't go into the main town anymore because they were being hassled business relationships that were fractured as an example in King Island there's two supermarkets there was a perception that someone that was managing one of the supermarkets held a particular position on the wind energy proposal so anybody that disagreed with that person then had to decide do we keep giving them our business or do we stop giving them our business this led to substantial boycotting of 50% of the supermarkets in King Island so that's not a small impact on a small community and so despite what appeared to be a good community engagement strategy instead of seeing a few different groups as part of a whole we saw this distancing and so we saw the people who were supportive of the proposal creating this vast social distance between themselves and the people who were in opposition to the proposal and that social distance exacerbates those issues between an in-group and an out-group before I move on I just want to make a quick note about empathy so I think a difficulty that we have when we talk about how we make decisions about what we should do with our environment our land and our resources is that certain groups of people or certain types of people who hold an opinion on an issue sometimes get characterised as being villains in this case with wind often people who oppose wind energy are characterised as only caring about themselves not caring about the greater good and so on but I'd just like to point out this is a quote from somebody who opposed the wind energy proposal and they really suffered they suffered from the fear of what this proposal might mean for them in their community and they suffered from what they felt as being negative stereotyping in the stigma that was applied to them so I really would like to encourage empathy toward people that we might disagree with on these sorts of issues but the social distance that occurred in King Island allowed different things to happen so I'm going to talk a little bit about what the experience was we saw different understandings about why there was this discussion happening in King Island interestingly the company went to the community and they said we don't yet know whether this project is fully feasible so we want to ask your opinion on whether you want us to proceed to do a feasibility study that's quite unusual for a corporate entity to do this sort of thing with the community what we saw was that people who were supportive of the proposal said yes of course we should move ahead we should have a conversation about how we want to do this feasibility study to get the information we need but the opponent said we see this as just being one step along an inevitable pathway toward this thing being built we don't want to go to the feasibility study because we think that that's just signing away our future there were different truths about what was going to be going on there so some people who opposed the proposal felt that it was the first step on a sinister plan to build 600 turbines in King Island and completely depopulate the place and turn it into an energy generation site meanwhile people who were supportive of the proposal saw it as being a boon for the economy that had been struggling and the first step along a high-tech future because there were rumors about some of the big companies like Google and so forth hosting cloud service there there were different narratives that emerged some of Matt's comments about the power of language were relevant here some folks talked about a wind farm others talked about a wind factory and that shaped how this was interpreted but there were different experts as well so there were experts from anti-wind lobby groups who were seen as being relevant to particular people and folks from other organisations that were pro-wind who were seen as being relevant to different people I'm sure you can guess what sort of stance those different people had at the time evidence was contested too and that the experienced, the lived experience of people that felt they experienced negative health consequences due to turbines were seen as the most important evidence to some folks whereas others said all we're interested in is the work of a particular academic who doesn't entertain the idea that turbines cause any health effects and so of course this led to different legitimacy about how the proposal should develop who's in the right position to make a call who gets a say and who gets excluded and at the end of it, as I mentioned it didn't get developed it didn't address any of the economic problems for King Island but there was a divided community as a result and so this was just the effect of talking about something and so it does make me fearful this idea of a plebiscite about population it's very powerful to suggest an idea and then ask for opinions on it as well so I'll finish up here this work is from a couple of papers and I'd be happy to share if anybody's interested but I know I have one over time I work as an economics advisor in Bill Shuntz I was mainly working on tax policy but a bunch of other areas as well I've worked previously for Andrew Lee I've been working for the Labor Party for the entire time in opposition for my sins I can't comment on the merits of Labor's policies today I'm not a spokesperson for the Parliamentary Labor Party but I assure you that they are very meritorious but I can talk a bit about how we develop policy options for our decision makers within the Labor Party and luckily or depending on what you think of my talk this seminar sounds sufficiently dry that I was actually allowed to come so much of your stuff let me come so I just want to give you a brief bit of context about what it means to be a policy advisor for the opposition and what the implications are for how we view evidence the policy battle between an opposition and a government is asymmetrical the government has access to thousands of policy experts they have hundreds of political staffers to assist in selecting the policy options presented by their guests to call the relevant fads up and ask them about things when the minister has questions and naturally of course stakeholders are keen to inform the government because they're actually in charge there's probably more people working directly for Tim than there are in the entire opposition and he's in charge of social services policy we're in charge of everything policy for the Labor Party on the other hand we do have one key advantage over the government which is that we can be much more selective in the positions we put forward the opposition doesn't have to develop policy on every worthy but uncontroversial challenge facing the country we develop and announce policies for a political reason essentially we want to highlight why we believe the electorate should support Labor at the next election and what we will do if we win because we're not in charge of anything there's no need to announce policy if it doesn't have a political purpose the whole purpose of developing policy and announcing it is political that has huge implications for how we look at things it's much different to how a government policy is done so how do we evaluate suggestions for policies that put forward to us how do we look at the evidence what evidence is useful for a party political adviser like myself and what's less useful I've come up with a bit of a hierarchy of evidence for a political staffer particularly an opposition political staffer which I hope is at least mildly controversial and ranks from ranks from bronze for the less useful through to platinum for the most useful and you can see that there's not necessarily a relationship between rigor and usefulness this hierarchy is about what's useful for policy development and it's political as I mentioned but it's not about spin I'm not talking about how we justify policies that we announce that thankfully is not my department that's the job of the media office and there's a lot of cynicism regarding policy based evidence about cherry picking evidence to support already announced policies I want to talk today about how we develop the policies before they're announced and then go on to justify them to the public so as I said there's not a relationship necessarily between rigor and the tier ranking that I've given these things there are things in the bronze tier ranking for example which are extremely rigorous academic papers that have required a lot of work that have considered all the evidence and have taken months to develop and probably years to finally get published but the reason that they sit along some of the other less rigorous things is because they're not particularly useful for us. Now why aren't academic papers useful? Generally because a lot of the time they do have very interesting implications for policy however generally that's buried in a couple of sentences in the conclusion after 30 pages of literature reviews and methodological discussion which while I'm sure is fascinating to the academics who are researching things not particularly useful in terms of developing a policy agenda to convince people to vote for the Labor Party in the next election. They also generally do not take politics into account, academic work is not renowned for its acknowledgement of the limits that are placed upon political parties in a federal political system and you know then you see it next to other things like push-polling and you think it's obvious that push-polling is not useful for policy development purposes but we see it all the time where I have meetings with stakeholders who put forward their latest wisdom poll about why we should do policy X generally they're well meaning but when they commission polls they're not careful in asking questions and so their findings are unreliable. There's a conception I think that as political advisers we don't really care whether or not the evidence is rigorous just as long as it says what we wanted to and then we're good to go but actually if we can't rely on the polling to be a reliable indicator of public opinion it's not useful for us in developing policy. We want to highlight we want to announce policies that highlight issues with important constituencies and we can't do that if the polling questions actually don't make any sense. Not all policies we announce will be popular. I do tax policy almost none of the ones I announce are popular but we do want to know in advance whether or not they're unpopular. The silver tier is where I think actually almost all of the kind of public conversation in Australia happens around policy and it's potentially useful for policy development. The main reason these types of it is don't make it higher up the tier list is that they generally don't provide suggestions as to how to address problems. They're generally more talking about the fact that there's a problem rather than how it is that you would go about addressing it or if there is a suggestion how you might address it it's generally at the end of an op-ed and it's again a couple of sentences. In many cases this is deliberate so you know the micro simulation modelling that Professor Gray's team does is intentionally factual, apolitical and descriptive. It provides for analysis of a policy issue. It may be useful for policy development but its objective is not to recommend a particular course of action. In other cases people haven't worked out a way to solve the problem they're describing yet and in that case it's very difficult for us as policy advisors to suggest ways forward if the experts themselves haven't thought of possible solutions. The gold tier provides for the most useful evidence I think to develop policies for announcement. Evidence in this tier is relatively rare but its impact is the highest and I explain why I'm saying that even though it's not the highest ranking in a minute. In my opinion the best work of this kind is in Australia's produced by the Grant Institute who regularly release work that is both rigorously researched and comes with costed policy options. For example their negative gearing report was instrumental in our analysis before we announced our policy on negative gearing. Now we didn't have the same policy as what Grant recommended necessarily but the fact that they had costed options that have been thought through was very helpful in terms of us developing our thinking that we then used to inform our policy. Costings are not strictly necessary I don't think but if your policy is going to have an impact on the budget it's helpful for us to have an understanding of at least the order of magnitude effect that they will have so that we can consider what we would get the parliamentary budget office to cost. I think government agency reviews are also important but I think they've disappointed in recent years in terms of their ability to impact change. The PC has written a lot of extremely detailed reports that have reached a thousand pages which I have disappointed in terms of policy impact. So the Platinum tier is that it's kind of a unicorn that these things actually really exist but I wish there were more things in the Platinum tier which is the policy pages with cost of recommendations and analysis of public opinion. Generally speaking in Australia which has a pretty small policy community, particularly independent of government the most rigorous work in Australia is done by organisations that are generally apolitical and so they tend not to want to get involved in discussions about what may or may not be electorally popular for either side of politics. If that work did exist though it would be really useful. So I think if there were organisations that would do more of this work it would be much more likely to become part of our agenda going forward. So this is my subjective tier list of usefulness I'd be happy to hear how wrong you think I am whether you disagree with the placement or the concept is offensive to you generally but it was I found fun to sort of have a think about it I'm not in a particularly reflective job so it was fun to to have the chance to sort of think about this in a more detailed way.