 Hello everyone, my name is Celina Killock and I'm delighted to be able to present today at the Library Assessment Conference 2020, even if it is from a very socially distanced location here in the United Kingdom. My presentation today is about how I've been putting a library assessment culture into practice to form our strategy at the Open University Library. For those of you who are following us on Zoom live, you can ask Q&A now or throughout the presentation I can pick them up at the end. For those of you watching this as a recording, I'm available on Twitter at Celina Killock if you want to ask me anything. Alternatively, if social media is not your thing, I've got an email address coming up at the end of the presentation. For those of you who haven't heard about us, the Open University is the UK's largest higher education provider. We were formed over 50 years ago now with a mission to open up education for everybody. In that time, we've had over 2 million people choose to study with us. At the moment, I've got about 170,000 students predominantly in the UK and Ireland all over the country studying with us online and that's always been our model. As my Vice Chancellor put it, we were the world's first online university waiting for the World Wide Web to be invented. 27,000 of our students have got to declare disability. To be clear, we don't insist that they declare a disability to us, but if we need to make learning adjustments, we know that information for those purposes. That makes us the largest provider of higher education for people with disabilities and accessibility of information and learning is business critical. They can choose to study a range of qualifications, 180 at last count, made up of 600 modules of study. And when I say we're open, we really mean it and we're really passionate about it. OpenLearn is our free platform of learning materials available to anybody on a Creative Commons license. When lockdown, the first edition happened in the UK. We were quick to maneuver onto that platform and move courses into that space to help the whole world. We launched courses on how to teach online, how to cope with mental health and some learning materials aimed at school children as the UK schools were closed. In that period, we had over, well, close to 800,000 courses were completed. The library then, as you can imagine, is very digital. 80% of our books are electronic and 100% of our journals are too. For those of you who like your gate counter statistics, my equivalent is a unique visitors to my website. Half a million every year come to our website to use our content and resources and skills materials. We make all of our content available open if we can within the terms of our license. And that results in normally about 10 million page views per annum on the library website. We are 365, 24, 7. And for those of you who take part in the spring share web chat, my team and I are very grateful as our students for your help and support for our students when our team are asleep. We've had a very strong strategy for digital capabilities at the Open University Library and we've worked well with our faculties to embed library content and skills into the curriculum. 91% of our students now study on a module where there is library content and skills embedded into it. And the value in that is known. Students who use library resources in 10 library tutorials get a better class of degree at the end of the day. Now, our work is part of our library learning analytics research, which I've presented on at this conference before. I'm not going to dwell on it too much today. We are multi-award winning and very passionate about what we do. And we also practice what we preach. On the Open Learn, for example, this year we relaunched our digital skills, badged open courses, succeeding in a digital world. Feel free to take a look and feel free to use it if you want to as well. My presentation today is actually about how we pull together library assessment culture into forming library strategy at the Open University Library. That combines with three key elements. The overall university strategy, insight, which is an umbrella term I use internally to talk about library assessment, because library assessment isn't the most common term in the UK as it is in the US. And expertise, and I'm going to cover all those three elements, library, university strategy, insight, and expertise in this presentation. Talking first about the university strategy, I'm not going to dwell too much on this, because I know this audience well enough to know the insight in that assessment is obviously the interest of the day. But it's vital that we always recognize that we don't exist in a vacuum. We are always attributed to accountable to our university stakeholders. And so it's our role to understand the university's strategic aims and missions and interpret that into designing our services aligned to what the university is aiming to achieve. However, conversely, it's also our role to convey what the library does and how that does support the university endeavor. And so the senior stakeholders can see a clear tangible link between what they want to deliver and what we are doing. And it's a really interesting model. I call it strategy and interpretation. That's a work in progress. I'm not quite there yet with that one. Moving swiftly on to insight, because I know this team. Insight for me is an umbrella term that I use to talk about all the various different components of insight into my users to understand the service and design it with a service to support students' success. So for example, the library help desk we offer has a customer relationship management system it uses. We know who's asking for what information on what topic, when, how frequently. Is it an isolated request for help? Is there a consistent problem with one module where they're consistently asking for the same thing at the same time do the training session be more useful? Is it a consistent problem with one publisher and there's something we want to talk to them about changing on their platform? Variety of different insight into our customers and when they're getting stuck and when they're contacting us will help so we can understand what we need to change to make things better to help students succeed. We also obviously look at closely our resource usage counter statistics and the like. I'm sure you do similar. And we have data on our internal systems about where content is being embedded and where skills have been embedded into the curriculum so we know who's using what, when and how. We look closely at our statistics on training attendance and combine that up with student attainment to understand actually is our training providing a successful learning experience for our students. Within the UK we also have the National Student Survey. This is a compulsory survey that goes to all undergraduate final year students at every university in the UK and it is benchmarked between each other so we know if we are improving on getting better than somebody else. I'm not a fan. The thing with it though is it is business critical and it is strategically aligned so we have to look into that and make sure that we're trying to improve our scores in that space which we are which is nice. Underlying all of that though is a good team of could we do an inter-user experience research and understanding what our customers want from us and what our services need to offer next. For example we did a large library needs project a couple of years ago now into finding out what is our students want from us and where we can do it through a directed storytelling approach. Core to making all of this work is our award winning library student panel. Now this was something we developed in 2012 and it was lovely to see Lara Miller's presentation at the previous session of the library assessment conference talk about just this and hear that the University of Arizona library has similar. Back in 2012 we were concerned we were hearing only from a very minority group of students so we were either very satisfied with us or very unsatisfied with us. For those of you who are library assessment professionals you know that those tend to be the people that want to talk to you. We were hearing from the same quite vocal minority of students and we were concerned about the people in the middle. We wanted to hear from them how are we doing and what we could be doing better. Now obviously we don't have the same fortune of you the people walking in the door to offer them pizza too. So we set up the panel. How this works is we send a invitation to take part to about four and a half thousand students per annum twice a year and that normally results in about 500 people agreeing to be on the books with us any one time. We commit to them that they will never have to do any more than four studies in a year. It's only four a year they'll get dropped off after a year. They don't stay on the panel forever and when they can leave at any point that they want to. We send them a welcome pack and then when they take part in any research with us we tell them afterwards what we found out as a summary of the results and also what we're going to do with that information and how it's going to be used to improve our services. For those of them who don't take part in a specific piece of research we send regular newsletters anyway to all the panel members so they know what's going on. We don't always sample every four or five hundred for every piece of work. We don't tend to incentivize too much. We don't feel we need to. We've worked with the panel to understand what works for them and what doesn't. If they're doing something long, a half an hour-ish, we might send them, we will send them an Amazon voucher but generally it's on goodwill. The representativeness is an interesting one and it was how we started initially we were worried about that like vocal minority. What we did was make sure the panel was composed of a population similar to the university. In the last year or two we've moved our thinking on this and we've started to weight the sampling to specific criteria as well to increase students from a BAME community and also those with a declared disability. BAME is a term that's very commonly used in our cultural setting in the UK university sector. It's not necessarily one that I'm always comfortable with but it is what my institution and a lot of the UK universities are talking about at the moment. Equality and diversity is a huge strategic priority for us and we want to do more in this area and we want to be better. We've waited the panel slightly to try and understand more what more we could do. The methods that we use with our panel vary everything from the classic of surveys and focus groups through to directly storytelling, use touchstone tools, observational studies, love letters and breakup letters was quite fun, a variety of different creative techniques to try and understand what's going on with them. Now all of that understandably is at a distance and that's come to some creative thinking for us and how we can do some of these things and we're always going to be innovative in this space. For example, last year we had a project looking into the information architecture of the library website. To do this we set the panel members a number of different challenges and tasks asking them to go find where they would look for this information and what would be more successful routes. We used an online package called optimal sort trees testing for this. That's just one of the providers in this space. It's not necessarily the only one. Feel free to find who works for you if you want to do similar. It helped us to reform and refine the library website structure but also work out which terms are more successful, which words resonate more with students and less and which phrases were tripping them up. So it was good from that point of view. From a more qualitative point of view we also do diary studies and this has been a lovely little piece of work into understanding how our live training sessions could be better or could be strengthened and what works well obviously. We sent home to panel members just a very simple sheet of A4 paper that A4 paper in the UK, A4 paper size, letter size in the US, sorry. Very simple sheet of paper to them to print out and have alongside them when they were taking part in a live training session with one of our librarians. Throughout the session, various intervals, they just kind of made a little pencil note of what they were thinking and how they were feeling about what was going on. Took a photo of it and sent it back to us for us to have a really rich idea of what's going on with the training sessions and how they can be improved. All of this insight comes together, variety of different sources and methods to then help us improve our services to improve student success. And it's important that you never look at any one thing in isolation. It is the whole collective. What are your users saying to you? What are you hearing from them? So going back to my model, we had university strategy. We've had the insight. The most important thing though, and the one thing we're not loud enough about in our sector, I feel, is our expertise. We are professionals. We are library professionals. And I've been advocating for loud librarianship for years now. We're not bookstackers. We're not working in an oversized Amazon warehouse. We are the ones that take that insight and that university strategies and turn it into a library strategy. Without us, there's nothing. We're the most valuable resource a library will ever have. So that brings me round to our model. How I build the library strategy is working with the university strategy insight and our expertise. If you have just the university strategy and our expertise, but no insight from our users that this is something that they want and need, you're not going to have any interest in the service and it's not going to get any take up. You have to have all three elements. Similarly, if you have the expertise and the insight, but it doesn't align to the university mission, you're not going to get the resources, whether the direct or indirect, you need to drive that strategy forward. Everything costs and you need to be aligned to what the university wants you to do. Finally, if you have the expertise and you have the, sorry, you don't have the expertise, but you have the university strategy and the insight. You don't have the capabilities to deliver on what your university wants from you and you'll need to build that insight, that capability within the team to drive that strategy forward. We are professionals. We are continually improving ourselves and our services and that capability is something I will always see and grow within every university library I've ever worked for. I have a lovely team at the university and I'm very proud to be one of them, and I would like to just acknowledge a few of them that have helped me with this presentation, so thank you very much, team. And thank you very much, participants. If you have any questions, Selina Killick at open.ac.uk is my email address. These slides and my presentation is also available on my website SelinaKillick.com. Thanks very much.