 Welcome to Considering Your Sample, part of the research and assessment cycle toolkit offered by the Association of Research Libraries, and made possible by a grant from the U.S. Institute of Museum and Library Services. This presentation is part of a module that focuses on collecting data, evidence, input, or other information for library assessment projects. It describes populations and sampling for library assessment projects, including sample sizes and types. We hope the content is useful to library practitioners seeking to conduct assessment projects. At the close of the presentation, you will find a link to a feedback form. Please let us know what elements were useful to you. Once one has selected a method that elicits the information necessary to address an assessment focus, question, or problem, a next step is to determine the sample of participants to engage in the assessment process. To think this through, it's helpful to consider assessment populations and sampling generally, weigh various types of sampling strategies, identify a workable sample size, and anticipate possible response rates. First, let's address populations and sampling generally. A population for an assessment project is the total number of individuals or cases that conform to a preset criterion or description that defines the issue under study. It's possible to apply an assessment approach to an entire population if the population is small enough to include everyone in the assessment, or when there is such diversity in the population that sampling of any kind would skew results. If, for example, an assessment project wanted to learn about the needs of every faculty member in a small department, then a reasonable attempt could be made to include the entire population. This is an example of a population being small enough to include everyone in the assessment. When such a situation is not the case, the population is used to draw a study sample from. For instance, a study focused on transfer students entering in a given term at a large institution might need to use a sample rather than a full population, given the large number of students involved. Thus, sampling is the process of making a selection of participants from a total population. In order to do sampling, it's ideal to have access to a sampling frame, or a list of all units, or usually individuals, in a population. Typically, this is a source of data, or list, from which a sample can be drawn. In any situation in which individuals are being drawn from a larger population as a sample, there is a possibility of bias. Many strategies for selecting a sample are intended to decrease the likelihood of bias, but an awareness of how bias can be introduced into a sample, and therefore a project, should be top of mind for library assessment practitioners at this stage. Once a decision is made to use a sample rather than a population in a study, the next decision is to determine the best and most appropriate sampling plan for a particular study. There are two major types of sampling, random sampling and non-random sampling. Let's take a look at each of these two types in a bit of detail. In a random sampling scenario, the goal is to have assessment participants that are representative of all the characteristics of the population as a whole. Strategies for this approach attempt to ensure that every member of the population has an equal probability of being included in the assessment. In sampling, the intent is to ensure that the actual participants are indeed a random sample. To achieve this intent, it's important to be hands-on. If left to chance, chances are that the sample will not be random. In other words, if participants are allowed to self-select or are drawn from, say, library users found in the physical library, non-randomness is introduced. Some people are more likely to volunteer than others. Selecting from individuals in a particular physical space already deselects those who aren't likely to be there. And unless that is by intent, that is, the population is space users at a particular time, then such a practice would result in a sample that is very unlikely to be a random sample of a more general population. There are a variety of ways to achieve a random sample or some version of sampling that is considered an approximation of a random sample. To achieve a random sample, a sample frame of an entire population is necessary. Then each individual in the population would be assigned a number, and random numbers could be selected until you reach your desired sample size. Using a formula designed for this purpose or a random number table is standard practice. Some approximations of this process are often used, though they do not truly yield random samples. For example, in systematic sampling, selection could begin anywhere on a list of individuals, and then every ninth, fifteenth, or some other number individual might be selected until the desired sample size is reached. This approximates random sampling, but is not the same thing. With stratified random sampling, a population might be divided into groups, and then random sampling would be applied within each of those groups, usually taking either the same percentage from each group or the same number from each group. This is also not true random sampling, and every decision made in this process, how to form groups, how many to take from each group, influences results, and can introduce bias. Cluster sampling is similar to stratified random sampling and may be used if no sampling frame is available. This is often associated with sampling from various geographical areas. The decisions made here can also strongly bias the process and results, and again, this is not truly random sampling either. Non-random sampling is often used when representativeness is not an issue. Such cases, which can happen quite often in qualitative studies, but also in quantitative ones. Judgment is used to select respondents who are likely to be able to inform the assessment project due to expertise, experience, or other criteria. Let's look at a few varieties of non-random sampling approaches. Oftentimes, in practical assessment, a sample of respondents or participants is selected because they're there. In other words, an assessment might happen using individuals in a space or at a particular event. An assessment might be distributed to those who are easy to access, separate from some other plan for sampling. These samples can be called a variety of terms, convenient samples, accidental or haphazard samples, availability sampling, etc. Another non-random approach is quota sampling. In a quota sample, someone, perhaps the assessment practitioner or some other decision maker, decides how many people with specific characteristics to sample. An example might be X number of faculty from each academic department. In purpose of sampling, an assessment practitioner might deliberately choose respondents or participants based on advanced knowledge of their characteristics. And so each respondent is selected on purpose for particular reasons, typically at the individual level. Three other non-random sampling methods include snowball sampling, self-selected sampling, and incomplete sampling. In snowball sampling, initial participants suggest others who may be willing to participate in the assessment. Self-selected sampling is what it sounds like. Participants volunteer to be included in the assessment. And incomplete sampling can refer to a variety of kinds of incompleteness, such as when individuals in the sample do not participate, or the sampling frame was not a complete census of the population. Once a sampling approach is determined, and there are many others beyond those shared in this presentation, a next step in informing the sample is determining an appropriate sample size. This process varies depending on the type and approach of the project at hand. There are a few things to keep in mind when considering how to decide upon an appropriate sample size. First, generally speaking, the larger the sample, the more likely it is to be representative of the population as a whole. Of course, larger samples are also more resource-intensive to solicit, manage, and analyze, so a large sample is not always possible or desirable. Second, if a particular assessment project seeks to attain statistically significant results at a particular confidence level, this will impact the size of the sample. And third, sample size tables are incredibly helpful tools in determining the size of a sample and understanding the implications of the sample size for a particular study. In general, consultation with statistical resources and personnel is advisable in advance if a library assessment project seeks to demonstrate statistically significant results to help assessment practitioners think through or double-check the intricacies of the sample selection and decision process. Having said that, some general reminders are in order. First, sample size decisions should ensure that small numbers of outliers won't skew results, while simultaneously seeking to ensure that all characteristics designed to be studied are present enough in the sample not to be overlooked or ignored. This is a balance that requires careful consideration. Second, there are many general principles regarding sample sizes that should be examined before adoption. Some general rules suggest 15 participants per variable, others a minimum of 30 participants, or for surveys a minimum of 100 for each major subgroup surveyed. There are, in fact, so many so-called rules that one should dig into the needs of a particular assessment design before prematurely deciding on a sample size according to a rule, because the implications of the rule might not be fully understood initially. Third, whatever the sample size is, non-respondents should be factored in mindfully. If a certain number of responses is required and not all of those invited will respond, there's a calculation to be completed in determining the number of participants to invite or elicit. And fourth, and this is true regardless of sample size, it's essential to pilot test assessment strategies and approaches. Given the work that goes into proper sampling and design, one does not want to discover halfway into a project that there are problems that a pilot could have revealed upfront. It's worth pointing out a few additional ideas related to sample sizes that are specific to qualitative research. Oftentimes, qualitative approaches seek to gather complex, detailed information to understand a situation or population rather than come to generalizations. In these cases, sample size calculations designed for studies seeking to generalize don't necessarily fit. As a result, setting a sample size for a qualitative study can take a different approach. Some of the criteria for determining sample size in this case focus on saturation, or the point at which no new information is being uncovered, and the overall purpose of the sample. Is the sample designed to elicit a maximum variety of perspectives, extreme experiences, or is it a snowball approach that's part of the design? Any of these or other purposes will necessarily impact the development and construction of a study sample. Finally, in considering a sample for an assessment project or study, response rates must be configured into the overall calculation. In general, response rates are often not what a library assessment practitioner would like them to be. This can be a result of the distribution or communication of the assessment instrument or event, assessment fatigue, unreasonable expectations of participants, or disbelief of potential participants that engagement will lead to benefits or change. When a response rate is so low as to diminish the validity and or utility of an assessment project, this problem needs to be thoroughly understood and reported. To not do so is to fail to recognize bias that may have been introduced into the process and results, which could ultimately negatively impact the validity of decisions or actions based on the study results. Therefore, planning in advance to ensure that even lower than hoped for response rates will still yield a sufficient sample for a project is essential to increase the likelihood of usable results at the conclusion of the assessment. Thank you for viewing this presentation on collecting data, evidence, input, or other information for library assessment projects. Please use the link provided to complete a feedback form on the usefulness of this information for your purposes. Thank you.