 Welcome to discovering what you need to know, part of the Research and Assessment Cycle Toolkit offered by the Association of Research Libraries and made possible by a grant from the U.S. Institute of Museum and Library Services. This presentation is part of a module that focuses on identifying the needs, context, and goals of library assessment projects. It describes motivations for library assessment projects, including responding to imposed questions, connecting values and practices, responding to changing trends, reallocating resources, closing gaps, better understanding users, and aligning strategic priorities. We hope the content is useful to library practitioners seeking to conduct library assessment projects. At the close of the presentation, you will find a link to a feedback form. Please let us know what elements were useful to you. One of the first steps in identifying the needs, contexts, or goals of a library assessment project is discovering what the library assessment is intended to reveal. What do you need to know or find out? Having a full understanding of the information need, the information the assessment is intended to uncover, is essential to making decisions throughout the assessment process. If the assessment is imposed by an individual or group who has an information need, that can make a difference in the approach assessment librarians need to take. Understanding the key drivers or motivation for the assessment project, likewise helps inform decision making throughout the process. Is the assessment to help close a gap, better understand users? How is the assessment related to the library strategic priorities, other needs and goals? All of this information influences and informs the process. First, let's start with understanding imposed assessments. Many assessments are what we might call imposed. That is, the assessment question, project, or task is assigned by someone other than the library assessment practitioner, committee, or team. This might include administrators, leaders, managers, or supervisors who are seeking information that is expected to be revealed through an assessment. This might include individuals and groups outside the library who have information needs that an assessment might uncover. Essentially, someone, not the assessment librarian, has a question they would like an assessment to answer, and the question her, him, or herself might even be one or more steps removed from the origin of the question. One of the challenges with regard to imposed assessments is that the information need that is intended to be addressed by the assessment may not be understood completely by the assessment librarian, or even the person who brings the question to the assessment librarian. Alternatively, the question might be misunderstood in part or in whole. Getting to the source of the question might be difficult, unlikely, or even impossible. Difficulty getting to the core of what we need to learn from an assessment project has cascading impacts. Assessment librarians may then be challenged to fully gain an articulation of the project need or goal, and that challenge may persist throughout the assessment process, or even derail it. There might be more anxiety or concern throughout the process as well, and possibly a murky beginning may make the whole process feel clouded. Whenever possible, seeking out the origination of an assessment question, need or project is ideal. When that's not possible, assessment librarians should expect that a lack of clarity might challenge their process and results. If assessment librarians can gain access to the originators of an assessment question, need or project, asking questions can help reveal the motivations, needs, and anticipated uses of the resulting information. Using open-ended questions to probe this information can provide critical pieces of the full picture and context for an assessment project, particularly one that is imposed. A second step in discovering what you need to know or learn through an assessment project is the key drivers for the assessment. Assessment drivers can come from a variety of places or motivations, and often more than one. For example, many assessments are values driven or focused on priorities of individuals or organizations. In libraries, assessments may be focused on articulating the value of users or library workers, understanding how or why various activities are accomplished, discerning what work is valued, or demonstrating how assessment moves libraries forward in their values and strategic goals. When library assessments are driven by values like equitable access, confidentiality, social responsibility, diversity, or other values, it's important that those values align with and are visible in the assessment process. From the behavior of the assessment practitioners, the assessment design and methodology, the reporting of results, and of course acting on them at the end of the assessment process. Many assessments are driven by a need or desire for change or a variety of trends that the library is impacted by. These might include changes outside the library and within the overarching institution, the community, or within a specific area like technological innovations. These changes might be observed through connections and engagement with the community served by the library, trends observed by library staff, or reports of changes in library or related industries. Trends might also be revealed internally by statistics, observations related to transactions over time, or a SWOT analysis conducted as a part of strategic planning or periodic environmental scanning. One way to think systematically about changes and trends that serve as drivers for assessment is through the use of frameworks like this one. The acronym PESTLE reminds us that trends include political, economic, social, technological, environmental, ethical, and legal contexts. Changes in one or more of these areas may necessitate an assessment project that will provide more information about how to engage with, accommodate, or inspire changes in library services, resources, spaces, and so on. A third example of assessment drivers is decision making around resources. Assessments may be motivated by the need to monitor or reallocate budgets, making persuasive arguments for one time or strategic investments, providing evidence for budget requests from institutions, funders, or donors, or evaluating the use of resources and the degree to which resource allocations match with organizational priorities and outcomes. Any one of these purposes might depend on assessment information. Budget requests, for example, might include assessment data in nearly any area, a statement of need, goals, articulation of impact, etc. Sometimes a key driver of an assessment project isn't starting or continuing a library service, resource, or space, it's sunsetting one. Thinking through the possible discontinuation of a library offering can be tricky and highly charged for a variety of reasons. Assessment data and evidence can provide structure, logic, and reason to the process, and certainly sunsetting a library offering without sufficient information is more difficult than doing so, or deciding not to proceed with relevant information in hand. Some of the questions that need to be asked about discontinuing a library service, resource, or space are provided here. Many of these can be augmented using assessment information. The range of drivers for assessment projects is broad, but generally an assessment project is undertaken to close a gap between what is currently known and understood and what needs to be known and understood. Into the breach, we launch assessment projects to fill those gaps, at least in part, with data and evidence that helps us make more informed decisions about the question at hand. A third step in discovering what you need to know or learn through an assessment project is fully understanding the gap the assessment information is intended to fill. This use of information and assessment to close gaps is closely related to the information need fulfillment that librarians engage in every day. Let's take a closer look at how understanding gaps can help us conduct assessments. Some of the key information behavior work done in our field is focused on closing gaps. Brenda Durvin and Carol Colfow are two information behavior researchers that can be drawn upon in understanding the role of gap closing in library assessment work broadly and understanding what needs to be learned from an assessment project specifically. Durvin describes the idea of sense-making as the process by which people construct information needs and uses for information in order to understand their own and others' worlds. As librarians seek to understand the worlds, so to speak, of their colleagues, institutional partners, users including various groups of students and faculty, the process of articulating an information need and securing information that can be put to use is central. Librarians, like other humans, live in a gapy world. There are always gaps between what we need to know to make good decisions and provide the best possible services, resources, and spaces and what we actually know. And once a gap is filled, the terrain can and will change so that ongoing information, that is, assessment data and evidence, is required to keep the gap filled. As librarians and assessment practitioners, there's a constant need to understand what our users and our coworkers think, feel, want, need, and aspire to. Without actual information, the tendency is to view others as we ourselves are. Being aware of the gaps between what we know and what we need to know and continually striving to fill those gaps, understanding that the job will never be finished but can still be done well. Being aware of those gaps helps us break out of seeing others only through the lens of our own experiences. Developing a clear sense of the gap to be filled by an assessment project means understanding the history and experiences of our users, who might be individuals who are seeking to fill a hole in their knowledge or understanding by using library resources. Our users and library assessment could also be the library colleagues who are seeking to close a gap in their knowledge and understanding of users or library offerings. It's also essential to understand the outcomes that users, library users or library colleagues want to achieve as a result of closing a gap in knowledge or understanding. In this way, assessment projects serve the all important need of helping users or library colleagues make sense of their own worlds in order to achieve goals, fulfill needs, solve problems, or otherwise make an impact. This process is inherent in both information seeking and assessment. In fact, the similarities between the work of helping users find information and the work of helping individuals gather and process information in order to make decisions and take action can make assessment a little less muddy to those who are new or resistant to it. And so, as we think about assessment as a sense making process, we need to remember that the mandate for change is directed towards us as librarians and assessment practitioners. How can we design library services, resources and spaces and yes our library assessments to be maximally useful to our users as a group and as individuals? How can we change and adapt ourselves to be useful to them? Because the goal of library assessment is to inform change that directly helps users. We also need to keep as a central tenant that the users we reach out to via assessment are co-creators of the process and not subjects to be analyzed without benefit to themselves. Carol Calfell's work is also helpful in determining what we need to know or learn through an assessment project and also somewhat reassuring in recognizing many of the stages that an assessment project and indeed a library assessment practitioner experiences. Calfell's information search process describes assessment projects as much as any other information seeking process. It begins with initiation when an individual or in our case a library assessment practitioner realizes a need for assessment, a gap of information needed to fill. The process continues along a familiar path of focusing the information need, designing a plan to acquire information and collecting and presenting the final results of the information seeking or for us assessment process. As in information seeking, one of the most valuable parts of this model is the affective realm of the process. As you skim the feeling sections of her model, you'll see a movement from feelings of uncertainty to optimism to confusion, frustration and doubt to clarity, a sense of direction and ultimately either satisfaction or disappointment and finally a sense of accomplishment. Experienced assessment professionals will likely recognize this cycle well. A fourth element of discovering what you need to know to begin an assessment process is understanding the users who form the focus of the assessment. With a library assessment project, the user may be a library user who is seeking to use a library service resource or space to accomplish the goal. The user might also be a library colleague who is seeking to close a knowledge or understanding gap in order to make decisions about library offerings. The user could also be an individual or a unit external to the library who requests information about the library from a non-library centric point of view. There are likely other possible users as well. For whatever user you need to focus on in a particular assessment project scenario, it's critical to understand the user's goals, purpose, mission or need within the context of the assessment scenario. What are they trying to achieve? What do they actually achieve in the current state of things? And is there a gap between aspiration and reality? It's worth noting explicitly that we need to consider the user's goals, purpose, mission or need both within a library environment and without. To focus exclusively on a library context might result in limited learning. For example, users might be seeking to attain goals that are not at first glance directly related to libraries or may appear to be more closely related to some other unit or perhaps not related to an institutional context at all. For example, a user who is focused on financial aid might not immediately think of the library as a resource for their affordability needs and goals. However, many library offerings might be helpful to them in attaining cost-accessible educational resources. Thus, it's important to consider what's important to a user without constraining that thought process to a library's status quo. Another important step in understanding users is an assessment process to think both broadly and narrowly about users that might be included in the assessment. From a broad perspective, it's important to consider the many users, individuals and groups that might be relevant to an assessment process. After a brainstorming process, it might be necessary to focus on users in a more targeted way, segmenting beyond what is often considered a user group. For example, an assessment project that focuses on faculty users might need to narrow to adjunct instructors, or even further to adjunct instructors in a particular school, college, program or discipline, or even further to adjunct instructors who teach graduate education courses and are also K-12 teachers during business hours. So, expanding and narrowing the funnel of who an assessment project focuses on as users is essential to the process and should be undertaken with care. Once a user group is identified, learning about that group is crucial to the success of an assessment project. In some cases, that learning might be in the form of the assessment project itself. In other cases, learning in advance about the user group might be essential to the assessment design process from the start. Understanding a user group impacts the ways in which populations are conceived, samples are generated, methodologies are selected, instruments and processes are designed, communication is conducted, analysis is planned and undertaken, results are interpreted and reports are disseminated and acted upon. Failing to understand the users that are the focus of an assessment project, again either library users or users who are decision makers seeking information to take actions based on results, can introduce critical flaws throughout the assessment process that may yield flawed and unusable results. While this is not an exhaustive list by any means, this chart can serve as a starting point for considering what might be areas of interest for various user groups, broadly speaking. Institutional users such as administrators or financial decision makers might be interested in affordability, efficiencies and opportunities for development. Faculty might be interested in tenure and promotion, research productivity or teaching effectiveness. Considering what's important from the user's perspective is a key step in understanding users as individuals and communities. It bears repeating that the users we seek to engage, understand and benefit are partners, not subjects in the assessment process. Users should be involved throughout the assessment cycle, including in the formulation of the problem or need to be addressed, the interpretation of findings and actions or decisions based on those findings. The goal is for users to benefit directly, immediately and throughout the assessment process so that the process overall should serve the users as individuals and communities. This inclusion of users as partners in the assessment process should, over time, be part of a plan for continuous improvement and communication leading to positive change for users. This perspective can be a bit at odds with some traditional approaches that view users as subjects to be studied for the benefit of those doing the studying, or in our case, assessing. These traditional views can be so entrenched in some assessment practices that it is essential for assessment practitioners to ask themselves early and often, whether they are developing and maintaining partnerships, communication and a focus on benefit to users throughout the assessment process. Designing milestones for check-ins on this front is an essential practice. One way of thinking about the intersection between what users need and want and what libraries do or might do in the future is expressed in the library impact map. In determining what an assessment needs to help elucidate, library assessment practitioners need to consider and pinpoint what library users, potential or not yet users, stakeholders or others need to know, understand or achieve, and consider intersections between those needs and library offerings, such as library services, resources, spaces and expertise. The intersection between user needs and goals and library contributions is one major source of library impact. Through library assessment projects, librarians can document the impact the library makes or fails to make in supporting users in meeting their needs or accomplishing their goals. In sharing the degree to which the library makes an impact or not, librarians and users together can engage in a reflective and continuous improvement process, which, over time, should bring changes to library offerings and increase benefits for users. The entire process, of course, begins with understanding user needs, goals and outcomes. Thus, the need to understand users to discover what you need to know or learn through an assessment process cannot be emphasized enough. One key strategy for understanding user needs, especially when the user is a library or institutional colleague, is to leverage strategic plans. Strategic plans often reveal what an organization cares about, so to speak, and can help guide understanding of the need for and goals of an assessment endeavor. When the user, who is the focus of or audience for an assessment project, is a member of an organization like a library or overarching institution, there are a number of questions an assessment practitioner can ask to gain insights into the user's needs or goals. Some questions, like those about an organization's mission or purpose, or the ways in which the organization allocates resources, can help uncover what is important and prioritize by individuals within an organization, assuming that the individual's needs and goals are aligned with the organization, which sometimes may be a faulty assumption. In cases where the user's perspective is aligned with the goals of the organization, assessment practitioners can critically and actively read and analyze organizational documents, such as strategic planning documents, publications and presentations, or other artifacts to discern organizational values and priorities. The purpose of a strategic plan is, in general, to set a direction for organizational achievement, articulate plans and strategies to achieve stated goals, and monitor performance, often going to the extent of indicating markers for measuring success. The intent of strategy in an organization is to define what the organization is and is not, does or does not. It sets the direction for the organization as a whole and focuses the effort of those within the organization. While the solidification and centralization of a strategy can have a chilling effect on diversity of thought, and sometimes strategy can lag behind reality, the documents and artifacts that convey an organization's strategies can be useful in discerning organizational goals as related to assessment projects as long as their limitations are also recognized and factored in. For determining the context of a library assessment project and discovering what a project needs to help uncover, understanding the library or institution's strategic plan can be essential. Sometimes, though, strategic planning documents and artifacts might become so out of date or invalidated for some other reason, such as a leadership change or change in the overall environment, that assessment practitioners might be left to piece together what an understanding of strategy through observation. Who are the major players, so to speak, in the organization? How does the organization allocate and use resources? What goals are actually achieved versus stated as goals but unachieved? What goals and values are reinforced in the organization? Which ones are rewarded? When current strategic plans are available, several elements may inform library assessment work. These various components of an assessment plan can also elucidate priorities within the organization. The values, stated customer groups, strategic foci, initiatives, success factors, all of these things can help assessment practitioners understand the foundations of assessment projects. While the details of strategic plans sometimes change or lag behind, mission and vision statements tend to have longer staying power, in part because they are general in language and content. Even with these limited statements, however, some information can be gleaned from a library mission or vision statement about what the library does, which users are prioritized, what benefits are expected to be outcomes of engagement with library services, resources and spaces, and what the long term goals of the library are intended to be. One of the most revealing aspects of strategic documents are the choices that are made and represented in the documents. Because resources are not infinite, no library can be all things to all users, and so choices must be made. Should a library focus on being or making things better? Or on innovation, change, and a laser like focus on the new? Other choices are revealed through differentiation. How does the library, in its strategic statements, separate itself from other similar units or options? Are their choices made based on services or resources, geography, campus or community driven, or user audiences? If a library seeks to differentiate itself by focusing on excellence for particular user groups, service, spaces, or other hallmarks, that will likely play a role in any number of assessment projects as the library seeks to determine whether, or to what degree, they've achieved their goals or have farther to go. For example, assessments might be framed around a library's operational excellence, innovation, or rapport with users. Understanding the framework for strategic work in a library, or what frames take priority, can help assessment practitioners understand what they need to know or learn by the close of a library assessment project. Examples of each of these sample frames include, for operational excellence, measures of quality, or seeking input for service improvement. For innovation, assessments might be focused on gearing up for projects that were previously only imagined or aspirational, or seeking to sunset offerings that no longer best serve users, but could, if decommissioned, be reallocated to provide resources for new options. For user rapport and engagement focused assessments, assessment librarians might seek to understand the needs of segments of the user population that have not been prioritized previously. Depending on the strategic intent, assessment decisions may change, not only at the start of a project, but throughout. Oftentimes, library assessments have what appears to be a simple purpose, to increase use. While simply straightforward, these assessments can rapidly become complex. In discovering what you need to know or learn from an assessment, it's important to consider various ways of increasing use, increasing frequency of use, quantity of services, resources or spaces used, better communication, or in drawing in new users. Again, this complexity under the surface of a quote-unquote simple service use assessment reveals that understanding what you need to know as a result of assessment is best probed thoroughly at the outset of a project and at checkpoints during the project as well. In closing, some of the questions one might consider when analyzing a library or institutional strategic plan to better discern the information an assessment project needs to reveal are included here. This is far from an exhaustive list, but uncovering the assumptions, environment, goals and aspirations of an assessing organization can and should inform assessment work. Time invested on clearly understanding what you need to know from an assessment project pays off throughout the entire assessment process, as well as being a major contributor to the likelihood of the final results being informing and actionable. Thank you for viewing this presentation on identifying the needs, context and goals of library assessment projects. Please use the link provided to complete a feedback form on the usefulness of this information for your purposes.