 Welcome to Anticipating How Knowing Will Lead to Action, part of the Research and Assessment Cycle Toolkit Project offered by the Association of Research Libraries and made possible by a grant from the U.S. Institute of Museum and Library Services. This presentation is part of a module that focuses on identifying the needs, context, and goals of library assessment projects. It describes strategies for beginning library assessment projects with the end in mind to increase the likelihood of actionable results that lead to positive outcomes and impact. We hope the content is useful to library practitioners seeking to conduct assessment projects. At the close of the presentation, you will find a link to a feedback form. Please let us know what elements were useful to you. One of the first steps in identifying the needs, context, or goals of a library assessment project is anticipating how the results of an assessment may lead to action. Through a process of answering basic questions, articulating outcomes, and planning ahead for impact, library assessment practitioners can develop well-thought-out assessment plans that better ensure that projects will result in effective decision-making and action-taking at their close. First, let's start with a guiding tenet and some basic questions that can help assessment practitioners think through ways an intended library assessment can lead ultimately to positive actions. Perhaps the most important idea for guiding an assessment project through to a meaningful and successful close is this. Begin with the end in mind. For assessment practice, this means a variety of things. First, keeping a focus on the outcome or impact of an assessment process is essential. What is the overall intent and purpose? What learning do we expect to gain? And what positive purposes can we serve without learning? Second, beginning with the end in mind means anticipating possible final scenarios for the assessment results right from the outside of a project. What various scenarios might appear in the assessment results? One might not be able to anticipate all of the possible resulting scenarios. And of course, one must remain completely and carefully open to ensure that one's expectations don't bias the assessment process or results. At the same time, thinking about possible results can help assessment practitioners look ahead to how the results may be used and use that awareness in ensuring that the assessment method and process will provide results that are relevant, valid, and useful in achieving positive change. Given the importance of beginning with the end in mind, a few questions can help center that focus throughout the early steps of the assessment process. One question that can help library assessment practitioners keep the end in mind is, what do you hope knowing will enable you to do? Possible answers include engagement and communication with users, positive decisions and actions, service resource and space improvements in libraries, which might involve increasing their effectiveness, improving equity, or decreasing costs. Aligning library work with values and user needs and goals, and generally making a positive difference through increased understanding. A second question that helps us shape assessment projects with possible result scenarios in mind is, what consequences might occur if you don't learn or know more than you already do? This question puts into relief the gap between action without additional information gained through assessment and decisions and actions that result with more information at hand. Without the assessment, might there be missed opportunities to make connections and differences for users? Might we fail to walk our talk, our values, and our commitments? Might we waste time, resources, or labor, both for ourselves and for our users? Might we miss out on partnerships or collaborations, continue harms or exclusions that we are or perhaps are not even yet aware of, but could learn about and redress through the assessment process? Thinking through these questions helps assessment practitioners design processes that ensure that these negative consequences can be avoided altogether or in part through careful assessment design and deployment. Finally, assessment practitioners can think through actions that they might be able to encourage through increased understanding gained as a result of assessment. Like the first question, this one focuses on what assessment can make possible, but further focuses the goal on specific actions, such as co-creating library offerings with users, improving marketing and communication, defining problems and ideating solutions, monitoring progress toward goals, informing ongoing strategic planning, ensuring careful stewardship, aligning library worker effort with maximum beneficial impact, and so on. In thinking through how information gained through an assessment will lead to positive decisions and actions, another important step is to articulate outcomes. In this section, we will focus on outcomes based on user benefits from engagement with library services, resources and spaces, but it's worth pointing out that outcomes could also be used or based on individuals and groups other than those we typically describe as users or even center on the library offerings themselves. One way to think through outcomes for library assessment is to borrow from backward design, which is often associated with instructional design. The basic structure of backward design, as popularized by Wiggins and McTie, is arranged around three questions. First, what do we want users to learn, experience or engage with? Second, how will we know if they've in fact learned, engaged or experienced the things that we hoped they would or in the way that we hoped they would? And third, what activities will help them learn, experience or engage? For our purposes here, we want to focus on those first two questions. What do we want users to gain from library engagement with our services, resources, spaces and so on? That gives us our outcome. The second question of how we'll know if they've had that experience points us toward an assessment approach. For now, let's expand on the first item. What is the outcome we want users to come away with? By focusing on outcomes, we prioritize users from the start of an assessment project. When we clearly articulate an outcome, we clearly state the goals that we want user engagement with the library service, resource or space to accomplish. This clear statement of benefit serves as a focus for the assessment. We then ask, did the user's experience with the service, resource or space accomplish that outcome? Or stated another way, was the user able to achieve their outcomes through the experience with the library, service, resource or space? It's important to clarify that this outcome focused assessment approach is different from other assessment approaches. Outcomes are not inputs. Inputs are resources libraries use to provide services. Outcomes aren't outputs either. Outputs are what libraries do with the resources. They aren't process measures like how much time or money was allocated. Inputs outputs and process measures are all library-centric or staff-centric and generally focus on effort and efficiency. Those measures are valuable in their own right and can be essential for managing library resources. But from an outcomes assessment perspective, the focus is on the user and the outcome of their experiences. Therefore, assessments that are intended to lead to action for or on behalf of users may best be guided by outcome statements. These outcome statements can serve as a guiding focus for assessment. Other outcomes in this context are statements of what a user will know or be able to do or feel as a result of the library engagement, likely in the form of library services, resources or spaces. Outcomes can also be used to express what the library, service, resource or space will accomplish in terms of impact on the user. Generally, these statements can be written according to a formula, often structured as the user will be able to, followed by an action verb. You can continue the statement with an explanation or measures or indicators. So you might expand that formula to the user will be able to do a thing in order to why they'd want to do that or what they might get out of it. You could also add some additional language around specific indicators of the user, whether they are able to do what they are able to do as a consequence of library engagement. So how does this help? Learning outcomes related to the user experience and the outcomes of engagement with library services, resources and spaces help focus library assessment on the ways in which library offerings should or do impact users. These outcomes can be used to emphasize user feelings, perceptions, gain knowledge or skills, behaviors, condition or status. The focus here is on effectiveness for users, not efficiency or effort from a library-centric perspective. Because traditionally so many library measures have focused on inputs and outputs, a bit more time might be needed to think through and structure assessments that focus on how library services, resources and spaces impact users. And writing outcomes explicitly and clearly can really help an assessment of this type stay on track. Now what can we do with assessments guided by a focus on user outcomes? First, we can begin to answer so what questions about library services, resources and spaces. We can learn more about what difference libraries make for users in a variety of ways. Knowing about the difference made, so to speak, by libraries and librarians gives us practical information for service improvement and meeting user needs, allows us an opportunity to clarify our aspirations around support for users, provides us with information that is essential for building partnerships and maintaining effective user communications. This kind of assessment keeps us on track toward our goals and what we hope our services, resources and spaces will achieve and, importantly, by centering users, we keep our mental doors open to getting new information and perspectives. Having said that, librarians seeking to use outcomes as a focus for assessment work should be aware of a few common outcome mistakes. These include using verbs that aren't active and therefore not, judgeable, measurable or observable. Being short on the expected outcomes for users, usually out of a fear that the user experience won't deliver on a more ambitious or desirable outcome. Or missing the outcome focus entirely, usually due to focusing on something perceived as easy to measure. For those who wish to include measures or indicators of an outcome in guiding the assessment project, a bit more should be said about how to do this well. First, outcomes should be focused on the intended result of library engagement. The indicator piece describes to what degree the outcome, that is result, is achieved. Adding indicators can help clarify what meeting an outcome looks like, makes clear how we as librarians or assessors know an outcome has been met, or provides details about what we will measure or judge to know the outcome has been met. If desirable, a target can also be added to an indicator embedded in an outcome. Target setting can be tricky for a number of reasons. First, targets must be able to be observed in some way. Second, setting a reasonable target can be hard. Finding the Goldilocks, that is, not too easy and not too hard, of targets is challenging. Historical data, if it exists, can be helpful to get a sense of what a reasonable target might be, and setting a stretch goal might be, at least at first, means setting the target at a bit more than normal. Without historical data, staff experiences and observations or input from users are also incredibly helpful for setting reasonable targets. Whatever the targets that are initially set, they can be updated in subsequent rounds of assessment. The key, for the first time, is to be able to explain why the targets are set as they are. After that, past observations will help you set new targets. A third way to work through anticipating how an outcome will ultimately lead to action and beneficial change is to plan, from the outset, for making an impact. This changes, this takes the process one step past outcomes to a more lasting change. While not always possible to do perfectly or predictably, keeping impact as a value helps one identify the needs, context, and goals of library assessment projects. In order to design assessments that have the potential to make an impact, one needs to plan for and anticipate that impact from the start. This project begins with outcomes. This process, rather, begins with outcomes but doesn't stop there. Outcomes, in general, are defined as more specific, concrete, and immediate than impact. Impact, as a term related to assessment, is broader and more comprehensive. It's more about long-term growth, change, development, and improvement, and usually takes longer to emerge. To simplify, impact is a long-term cumulative effect of outcome achievement. It's also important to convey that many outcomes can converge to create one larger impact. The process from outcome to impact is not necessarily, or even usually, one-to-one. There are a variety of models to use to internalize the idea of impact and use it as a guiding framework for assessment projects. Nearly all are based on the notion of a continuum. Here you can see a continuum from inputs and resources to activities, to outputs, to outcomes to, eventually, impact. The logic in this model is based on if-then statements. Let's walk through it. This model begins with resources or inputs used to create a program of some kind. To get to the next level of the model, activities, you use an if-then. If you have access to the resources, then you can use them to make activities, and it continues from there. If you accomplish the activities, then you can deliver the service output you intended. If you deliver the service output you intended, then users will benefit in certain ways, that is, attain outcomes. If users attain outcomes, then over time changes for the users might be expected to occur. Let's look at this a slightly different way of expressing the same basic idea. In this model, there's a differentiation between planned work and intended results. Again, starting with resources. If you have access to resources, then you can use them to accomplish activities. If you accomplish activities, then you will deliver a service. Again, this is the output you intended. If you deliver the service, then users will benefit by attaining outcomes. If users attain those outcomes, then you might expect long-term changes and benefits to occur. This can be hard to internalize, so let's go through it two more times. Like the previous framework, this impact model also separates planned work and intended results. This one provides definitions, so to speak, of each element, followed by suggestions of the form these elements come in. For instance, resources are usually nouns, such as staff, facilities, time, money, etc. Outputs are often expressed as a quantity of work accomplished, and outcomes and impacts are typically changes, with the difference being largely a scale issue. This last version of an impact model is more complex, but perhaps more helpful as well, at least once you have an idea of the basic idea of an impact model in mind. This version includes the situation or context and priorities that frame or feed into the model to begin with, as well as assumptions that are being made and external influencing factors. It also clarifies inputs as investments, separates outputs into activities and participation, and subdivides outcomes and impact into short-term, medium-term, and long-term results. Outcomes and impact are further clarified into learning-focused outcomes and action-focused outcomes, with change in condition being the emphasis of the impact element of the model. I find this to be the most helpful of the set of logic models, as it really can serve as a guide in thinking about the intended process and products of a particular library service, resource, space, or even assessment activity. And going through that process is essential in anticipating how a project will lead to action. Thank you for viewing this presentation on identifying the needs, context, and goals of library assessment projects. Please use the link provided to complete a feedback form on the usefulness of this information for your purposes.