 Welcome to this session about failure and why it's good, we think, for the future of our research libraries. What we're going to do is do this session in a few different parts. There'll be a presentation to start with, then a discussion element, and then some breakout rooms in the third part. But if the discussion element looks like it's going really, really well, we might just continue with that. So bear with us as we flex the plan for today, really around you and what you want to talk about. This is what's called a provocation session. So really would encourage you to be part of thinking with us, talking with us about this topic of failure, drawing on the results of a really fantastic project called Failspace. I'm the Director of Libraries and Museums at the University of St Andrews, my name's Katie Eagleton. And for those of you who would find a visual description useful, I've got sort of mid-length red-brown hair, and I'm wearing a black and kind of acid green checked top today to bring a bit of colour into the session. Now, the idea for this session came about when I realised there was a lot of people in RLUK talking about failure and getting better at learning from it. And I started to think, okay, that's brilliant, but how do we move that from being a buzzword to something more meaningful? How do we really think about how to do this in a sector where it isn't always easy to talk about failure? And then I found the Failspace project online and thought it was just a really interesting way to do that. So I invited Professor David Stevenson to join us today, and then two colleagues, Katrina Cannon and Karen Crouch, who are fellow library directors. And what we plan to do is have a really open and honest conversation about this topic and then hopefully you'll all join us in that. So we'll move on now to the first part of our session. I want to introduce to you Professor David Stevenson, who is Dean of the School of Arts, Social Science and Management at Queen Margaret University. He's also a Professor of Arts and Cultural Policy and Associate Director for the UK Centre for Cultural Value. As I mentioned, he was one of the investigators, one of the principal investigators on the AHRC funded project, Failspace, which you can see a slide on here. So over to you, David, to tell us a bit about the project. Thank you very much. Thank you for inviting me to come and speak to people today. It's a pleasure. So yeah, just a quick visual description for those of you that would find it beneficial. So I'm a man in my mid 40s. I'm bald. I have very heavy black framed glasses on today. And I am wearing a blue smock over a white t-shirt, which I've realised also makes it look a little bit like I'm wearing a kimono. Now that my head's been kind of chopped off at the top. So I'm standing in my office today in Musselbra in Edinburgh. And I'm here and for the first kind of 20 to 25 minutes, I'm going to talk about the Failspace project. And I've been doing this project over the last three years with my collaborator, Professor Leila Jankovic, who is in Leeds. And this project started from a shared interest that we both had in cultural participation and in terms of patterns of participation with cultural organisations. And that's cultural organisations in the widest sense, but in particular cultural organisations that receive some degree of public sector funding. And part of this is because for many decades now, it's been such a flashpoint of debate around about the extent to which the organisations that receive public funding are legitimate recipients of public funding because their audiences, their staff, the people that use those organisations most regularly have consistently been shown to be relatively lacking in diversity. And overall, they are coming from what the Warwick Commission found continues to be the wealthiest, best educated 8% of the UK population. And so there's a real sense around about the potential that some degree of public sector funding is actually functioning as regressive taxation, taking money from those that can afford it, at least in moving it to those who have the most. And Leila and I have done different things in the past and looked at this in different ways. And fundamentally, there's a long standing debate and discussion around about the problem of cultural participation. And I've talked elsewhere around about the extent to which it's more of a problematisation than a problem. Because if you look at a lot of the high level statistics like taking part in the Scottish household survey, really we don't have a problem of people not participating in culture. Across the board, there is lots of evidence that people have rich, complex and diverse cultural lives. What we have is a problem of people not participating, engaging, attending, visiting certain organisations and certain types of cultural practice. And again, this is something that we've known for for some time. But this debate flares up every now and again. It was particularly big in the 1960s. And then laterally towards the end of the 1990s, it became an issue once again. Some of this was an economic driven from a kind of right of centre political argument around about waste and expenditure on organisations that could essentially be self-funding. And then increasingly as you moved into a slightly more left of centre kind of political environment at the turn of the millennium, an argument that became around about kind of social impact and social return. And the benefits that we could get as a society from people going to taking part and engaging with these organisations. And academically, there's enough of argument around about this. The two perspectives is should this really be around about access and opportunity? So should our focus be on getting more people to come to these organisations that are real bastions of value, real bastions of cultural value, and our primary function needs to be around about overcoming barriers, bringing people in, providing more opportunity? And then there's people on the other side that says that really, this is a kind of a false approach. And that what we need is structural change that recognises diversity of value, that we need to redistribute funding, that we need to share the funding out in different ways to different parts of the country. And we need to recognise that people are already doing an awful lot of cultural work. But what happened when Leila and I came together and looked at this is that despite all of the academic work that happens in this area, actually there's been very little progress in either side of this argument. And most notably we've had recently the debate around about Arts Council England and the redistribution of funding out of London. And that has been quite a significant shift. But even then is relatively small scale. So on one hand, we still see that most of the public sector support, most of the public sector funding continues to support the same types of organisations, the same types of cultural practices that are to a certain extent not widespread or utilised by the broadest spectrum of people. So there really hasn't necessarily been redistribution to the same degree or scale that people have called for. On the other hand, you also have a situation whereby we haven't managed to do a significant amount of work about diversifying the audiences. So while on a micro level individual organisations might have projects that they perceive have diversified the people they're working with, if you zoom back out again and you look at things like taking part in the Scottish household survey, there hasn't been any statistical shift that is significant over the last two decades of work that has been around about widening access. So whilst we may be getting these kind of small localised wins or successes, cumulatively, this work isn't necessarily delivering us whole scale structural alterations and changes. And I think what really struck home this whole argument to us, and again there's a question around about the academic practice involved here, that one of the people that I was reading just before we started this project, published a paper about cultural participation. And I think this was in the United States context. And I think the paper was published about 2017. And I remember that this author has been writing about it for a while and I went back to look at their previous work. And what was really interesting was I found a paper they wrote in 1996 that had a section in the paper that almost word for word was exactly the same as the paper that they published in 2016-17. Now leaving aside questions of self-plagiarisation, the fact is it was a peer-reviewed paper and the stats still pointed to the same conclusion, which was that we did not have diversity in terms of audiences at these organisations. And so that was where we started to say that was 1996, we're now approaching 40 years later and yet despite all of this work, all of these projects, all of these narratives and conferences and events where people talk about their commitment to diversity and inclusion and widening access, shouldn't we be saying we failed to move the dial at a national structural level? And why are we not saying that? And that really is where this work started. And so that brings us to fail space. It was a three-year project funded by the AHRC and as I say, this really wanted to understand this distinction between all of the success narratives that we saw. So a lot of discussion around about the extent to which there was, you know, best prospects, there was lots of good examples, there was lots of arts organisations we saw celebrating their attempts and the work that they were doing to diversify their audiences. But the extent to which they're doing the same things that they've been doing for 40 years, they maybe have repackaged them, they thought about them in different ways, that fundamentally we still aren't that much further forward. And this also goes for workforce, it goes for the boards. And it always seems to be a problem that is in the process of becoming, we aren't getting closer to solving it. And then every now and again, it gets re-highlighted as though it's a new problem. And so the purpose of the research was to say, why is nobody talking about failure? Why is this not a word that tends to get used? And why could we benefit or how could we benefit if we actually start to talk about failure more explicitly? And so what the project involved, there was various elements to it. We did workshops across the UK in different places. We asked people to self-select on the basis of how they identified, where they are participants, where they, somebody that represented a kind of cultural organization, where they are thunder, somebody that supports the work, or where they are professional and artist, or somebody that particularly works in outreach activity. And we spoke to those people individually about what failure meant to them in this work, what failure looked like, the extent to which they felt it was a useful concept. One of the consistent things we get throughout this project, and often when I speak about it, is some degree of hostility around about saying, failure's quite a negative term, it'll make people feel bad about themselves, or the sector is in a really difficult state right now, it isn't helpful for us to be talking around about failure. And then people will often say, we don't use the word, and it's just too absolute. But what's really interesting is that nobody felt that success was too absolute a term. Everybody was more than happy to talk about their successes. And also, as we did this work, despite everybody saying that failure wasn't a word they found helpful, what became very clear in both workshops and the interviews we did, is that actually, once we moved deeper into them, and we stopped talking about their own practice, and we asked them about failures they experienced elsewhere, everybody was happy to use the word failure. But what it was, was they used the word failure to point the finger or to blame other people. And so they were happy to talk about funders failures, they were happy to talk about managers failures, they were happy to talk about the people who had failed them in other parts of their organization. Artists talked about participants failing to understand the work. So failure is a word that we use. But it's a word that we use very explicitly and quite regularly in regards to blaming other people within a wider cultural ecology that we're all a part of. And so that was where we then started to dig in. And the extent to which what people find really difficult is to talk about the failures that they had some degree of involvement in. And part of what came out was because of the extent to which people absolutely conflated those failures with their own identity. And a lot of people said to us that they didn't want to describe themselves as a failure, they didn't want to describe themselves professionally as having failed. And so because they felt unable to divorce themselves from this sense of being a failure, if some of the work they were involved in didn't deliver on the results, they didn't feel able or confident enough to talk about it. And once we didn't gather the right type of data to kind of confirm this analysis, what was really kind of, I guess, anecdotally clear was that a lot of the people who felt that most strongly were those that really felt that they were working in the sector as a vocation. They had a real connection to the sector. And that sense of their ability to divorce themselves from it was far harder for them. Hence why they brought the failure essentially to themselves whenever they talked about it. So we also did anonymous surveys and we did over 100 interviews with people again from across the country and in different roles within the system, including people who did evaluations. We did some deep hanging out at certain locations that we feature in our book, which is basically a sort of projects that are rhetorically spoken about as being successful participation projects. But we spent time hanging out in the areas to explore the failures that sat alongside those successes. And we also did workshops to develop and adapt tools as we moved forward with this. But fundamentally, we had lots of failures in this work as well. And we got very similar people that came to certain events. And we felt that we should separate people out because we wanted to protect them. But actually what that did was double down on the whole position that people told us, which is they won't speak outside their stakeholder groups because of the relations of power between those different stakeholders. And we didn't really challenge that. We really struggled to get organizations to allow us to name them in our research. And so we have a number of case studies in the book. All of them are from England and Layla Oaksay is at the University of Leeds. I'm at the University of Queen Margaret in Edinburgh. And I have to say my Scottish contemporaries let me down in that I could not get one organization who was taking part in the event to agree for us to be naming them specifically in our book. We also consistently failed no matter how much people told us they loved the work to get people to talk about failures on their websites. Even though the participants that we spoke to told us that the fact that those failures are not acknowledged feels like people are covering things up. And the sense that they said to us, they're always putting things out that saying how successful they are at reaching out. But they're failing to get to us. They're failing to understand us. And they just seem to ignore it. Now that's not true. The organizations inside were often, you know, thinking about it, talking about it, discussing it, but they were just not prepared to stand up and say, we recognize that the way we're working in this particular area is failing to deliver the results we want. It doesn't mean that we're not committed to those results. But it does mean we need to think and change. And one of the other big things that came up is that we were patting ourselves in the back because we thought we'd run really great workshops that felt like really safe, honest, kind of collaborative spaces. We ticked all the boxes in our methodology. But one thing we had, which to be honest, I come up with as a bit of a novelty, was an honesty box. And that was in the corner of all of our workshops that people could put into. And it was basically a big egg shed because I took the idea from the farm that where you can pick up eggs and you can leave your money behind. So we called it the honesty box. And what was really striking was that despite the fact that we thought in our workshops, we created really honest spaces where people felt like they were talking to us honestly about their experience, that honesty box was always full at the end of our events. People were always putting things into it that they just couldn't say in the room despite the fact that they were in a room of their peers. And we've got lots of these and I picked some here that I would use. One there you can clearly say says, I lie on my official evaluations all the time. It's bullshit. I know policy workers and funders don't look at them either. I lied to get money. I think everybody does. And this is something that came up quite a lot in the postcards, the anonymous postcards of the work was the sense of either lying, adapting, framing the narratives that people were putting back into the work. And what was really striking is when we spoke to funders, they knew this was happening. They were aware of the fact that the work that they were getting back in terms of valuation either wasn't particularly good quality or probably left lots of things out. And they were frustrated about this. But equally, they weren't doing anything about it. And they were accepting that they were getting poor quality of valuations in and to one extent or to a great extent, they were just putting them to the side and ignoring them. We also got cards that we use that people talked about the failures they just felt they couldn't acknowledge. So the one on the left saying, we were over optimistic about what we could deliver in the budget and the timeline. So we were really hopeful. We were really keen. We were really committed to this project. But instead of fully acknowledging the negative impact on our participants being fast tracked to unrealistic processes, we focused on some of the positive outputs and not the outcomes. So actually, again, this sense of people feeling that they consistently have to frame their work in a very positive way and overlooking the failures, even if those failures were unintended or even with the best of intent when people started the project, they genuinely wanted to try and address this, but they just simply couldn't do it. And what we found overall was that there was a landscape within the cultural sector that was just not that honest. It wasn't open to truly critical reflection. People were editing pieces out. They weren't mentioning things. And so what that was was creating false narratives of success that were actually suggesting and creating problems. And one of the good examples that I have from that is talking to a funder who says, well, if the art sector is telling me that they can address a social inclusion problem, a really ingrained social improves and problem in a local community for a £25,000 one off project, and they give me an evaluation that say that they've delivered social inclusion in that community. Why can I go back to the government and make an argument that actually what we need is community workers and social workers? And so leaving elements out, we had artists constantly saying that they were involved in evaluations that were framed as successful, but they were projects where they hadn't paid themselves or they'd taken their pay and they'd used it to utilize or to pay for the equipment that they used. And yet those were being framed as successful projects. And I guess to use a really sort of bad example, it's a little bit like, you know, not recognizing in kind. If you're doing a budget for an activity and you don't recognize the in kind input, then actually, if somebody comes along to do that project in the future, you don't have a true understanding of the costs of running that project. And this absence of failure, this absence of the narrative of failure alongside the narrative of success is giving us partial evaluations that are not changing practice, that are not helping us be explicit about some of the problems and are trapping us in this loop of every few years, somebody pointing out the problem once again, despite it being a problem that everybody knows about. Because what was really striking is that all the stakeholders we spoke to recognize the difficulties, but they felt totally unable to talk to one another about it because they simply didn't trust each other and they were much more interested in pointing the finger and blaming. But because they were interested in that, that's what they assumed that people would do to them, that they would blame them for the failures that they had had. And so the biggest failure is that people are not learning. A good example again is around about fundraising, sorry, I'm not fundraising, funding for tickets. Lots of evidence has shown that giving away free tickets does not help diversify audiences and very often continues to benefit the people who would have paid anyway. A lot of the funders we spoke to said, we know that that's the case, we've evaluated this and we know that that's what happens. And yet they were saying they continue to get applications that ask for money to support free tickets to overcome barriers to participation. But what was striking is that they then funded it. And so there was this constant sense of going, we know it's failing but we've probably not talked about it, we've probably not been explicit that the failures are happening, so we'll just keep funding it because actually that will keep us moving forward. And that came across from the participants as a real sense of stop consulting us, stop asking us what we want because we keep telling you and yet you're failing to change or you're failing to do things differently. And what was also striking is that a lot of the participants we spoke to recognize that these changes are not easy. They don't expect things to be fixed straight away but what they do expect is honest recognition of the real difficulties and not having their voices and their stories used to present something as a success in a way that benefits an institution, an organization or an individual and doesn't benefit them as individuals themselves. And they really felt that that co-opting of their narratives and presenting them as wholly having been a successful participant was a frustration. And one of the organizations we looked at was Slung Low in Leeds, really incredible organization, does brilliant work and was one of the organizations that involved in flood and one of the big pieces as part of City of Culture. Framed is a huge success but we spoke to participants who said actually wasn't that interesting, didn't feel that involved, felt very artist driven. We spoke to lots of people who weren't involved. Likewise, I've done work with Arts Council Wales. They were really doing incredible work in schools with young people. A big part of what they do is about bilingualism and making sure that everything that they do is available in Welsh. That's absolutely great. But all their evaluations recognise that bilingualism and entirely ignore the fact that they cannot do all the other languages of people that live in Wales because they don't have the funding, they don't have the resource. And yet their aspiration is to be as accessible as possible. And so that ignoring of the failure and it's a pragmatic failure, it's a financial failure, it's a failure that is a structural failure in terms of what they can do is ignored. And that's the frustration the participant said is they were like it feels like they're saying that they're succeeding in being fully inclusive when me as a speaker of Farsi or as a speaker of Polish doesn't feel included. But they recognise that there are challenges. What they don't like is the fact that they get ignored or overlooked. And there's various things that I'm not going to spend time talking at today. Everything is pretty much open access apart from one book chapter that was there. And so there's a journal and there's also a book that we've done with Springer and the QR code will hopefully take you there and not to test for home shopping. And what we're advocating for is a real sense or a commitment to social learning of people working and collaborating together in order to re and understand the problems that's there. And to move towards a situation where we're more talking around about improvement rather than accountability. We want people to acknowledge that failures are happening. What I can tell you is every person I spoke to said we know failures are happening. Nobody was surprised by this. So everybody recognises them and was up for talking about them. And yet everybody wants someone else to take the first step in having that conversation. We can only build trust by making these narratives more visible. And I would argue it builds credibility. If I read an evaluation that tells me that somebody has managed to include a disenfranchised group on the back of one project that's been relatively minimally resourced, I wouldn't read that as a credible explanation. But if I looked at an evaluation that recognised partial success and partial failure and steps about how they could adapt in future, the evaluation feels more credible. And so really this is where we came up with the tools and there's a whole range of tools that we've done. All of them are open access. All of them can be downloaded. They're going to be hosted on the Centre for Cultural Value site going forward when our site gets closed down at the end of the project. And they're intended to help people open up conversations about failure. And I'm not going to go into them in depth but again there's videos on the website that can help talk about them slightly more. But we've had real challenges in this. As I say one of the big difficulties in this is that we've ticked all the boxes in terms of impact work. We've got champions. It's open access. We can get it downloadable. And on one hand it's brilliant. People love for me to come and talk about it. And actually I've had a lot of invitations to talk about sectors outside of the art sector as well. But what I'm really struggling with is to disassociate myself and Leila from this project. Is that everyone is inviting us to facilitate their conversations. And that's great but that's not changing a narrative. That's kind of turning it into a technocratic solution driven by particular individuals. And again it's that sense of I'm gaining professional advantage. I can put in an academic promotion application. I can look to get rewarded for that work. I can get a good impact case study. But what I wanted to do was to change practice in the sector. And thus far I am failing to do that. And so the key takeaways and the tools are first of all that project and policy outcomes are not binary. They are not going to be a success or a failure. But what we don't have is a model that allows us to talk about them very well. The prevalence of smart objectives and I have no problem with smart objectives. I get what they're for. But it drives you to a situation that says my objective was X. Did I meet it or did I not meet it? If people didn't meet the objective what they tend to do is rather than write we didn't meet our objective. They ignore it or they kind of reimagine it in their head slightly to a different objective. And they point towards something else and they say well we didn't reach that community but we reached a different community. Now that's great. There's absolutely nothing wrong with that but we shouldn't do that to the detriment of the objective we set out for. Particularly the work that's around about social exclusion. There is a moral obligation on us to recognize if we fail to deliver on some of these things. The next thing we're trying to argue for is that success and failure almost always coexist because sometimes some of your successes means failing for other people. You know somebody that I find hugely inspiring is Nina Simon who did a lot of work around about the notion of all by and for all and relevance. And Nina does work with organizations like the National Trust and I know that one of the first conversations she had with them is in order to diversify your audience you're going to have to lose some of your existing members. And sometimes there's a choice to happen because actually it could be seen as a failure to see their membership drop but that failure is a failure that they're happy to accept in order to have a different success. And so there's two things sit side by side and you're always trading off between the two elements but if you're not talking about them then you're not necessarily making those fully considered discussions. And then the final point and this came across really strongly is that this whole push about what is the value of the arts what is the value of the cultural sector what is the value of museums galleries libraries that sense that people are always defending the value is conflated with success. Something can be valuable without having been successful and my really bad metaphor is if you invite me to dinner and you put on an amazing roast for me that will be a failure for me because I'm a vegetarian that doesn't mean it's not valuable it's hugely valuable there could be five other people there who love a roast the food has nutritional benefits and qualities it's helped the farmers because they've got an economic benefit from farming that animal but for me dinner party failure but that doesn't mean that the two things are the same success and value are not equivalent and we need to learn to talk about the value of our sector the value of our organizations without them feeling like that is going to question or requires them to always be successful and fundamentally that brings us to one of the big key tools in the in the whole fail space work and this is around about the the wheel of failure and essentially it's a wheel of failure and success that we're really focused on getting people to talk about actually the word failure and what we did was looking thematically at what people spoke about we identified in these projects there tends to be different facets of the work and there's five different facets we identified and that in each facet there's varying degrees of success or failure and that actually when you talk about or evaluate a piece of work you can have had success in one area and a failure in another and that often it's not at the extremes often you may have accepted a tolerable failure in one component of the work whilst gaining a kind of resilient success in another and it could be that the work didn't deliver on what it was intended for but it really helped develop your professional practice or the professional practice of the people involved that's a success but we need to have an evaluation framework that allows that success to be spoken about alongside recognizing the failures that are happening even if we don't have a solution straight away and so when we work with people we use the grid at the beginning to really talk about at the outset of this type of project let's talk about the nuance between each of these different gradients and to really think about what would be closer to our perfect vision of success a simple example could be if we got more people would that be better or worse than a more diverse audience so would we prefer having 100 visitors or would we prefer having 30 visitors who were more diverse if it was a choice between doing more outreach events and diversifying the audience on existing outreach events what would be better or worse for us is this more about doing activity or is it more about diversification and we do this work with people at the beginning because it can really help them make pragmatic choices as they move through a project and they face constraints about budget and pragmatics and resources is to remind themselves which thing did we say was slightly closer to our perfect vision of success because everything is essentially some degree of compromise and the whole model of this is around about saying you're probably going to have to compromise you're probably going to have to accept failures in some part of this some facet of the work but doing that in advance helps you really think about what choices you're making and when you come to evaluating it really helps with the credibility of quality to the valuation because you wrote these states of being at the beginning of a project so rather than retrofitting something onto the end where basically and I do a lot of valuation I can make anything sound like a success but actually it forces you to say upfront if this looks like the last project we did then actually that's a failure because the purpose of this was to move us on it wasn't to keep doing the same thing so for me that would be a tolerable failure a bit meh and we did this for our own work and we did it at the beginning of the impact stage and Leila and I mapped out different degrees for ourselves of our project and where we wanted to be and outright success is the dream the rhetoric the narrative that I've sold to the funders but actually there's varying degrees along there and pretty much this is where we're at with the project at the moment and it's a tolerable failure in a couple of areas and it's a precarious failure in another and it's a bit of a conflict of success in some that doesn't mean that this work isn't valuable and what's been really interesting is in credit to them and AHRC have been really open to us going back to them with this and indeed have asked me to do work with them about their own relationship because they as a funder recognize that work will fail what they want to know is what have we learned what we understood and how does it move us forward because the challenges so many of us are saying that we are trying to do work to cut to tackle really complex social issues and yet we're pretending in evaluations that we are succeeding in these outright success is a long journey and actually part of this is around about saying we've had some failures we've had some successes and we're partially on way to get there and but it's a hell of a difficult to go back to a management and to a thunder and say I need more support or more money if your evaluation is said this project was a success because if it's a success you've finished and but if there's still work to do if there's failures to be overcome then that opens a narrative around about how we can build and how we can go forward and and so that's my whistle stop tour of the end of the background and to fail space and where it works