 Good morning and welcome to Encompass Live. I'm Emily Nimsikant, filling in for your regular host, Krista Burns. Encompass Live is the Nebraska Library Commission's weekly online event. It covers NLC activities as well as other library-related topics. The one-hour sessions are free and they happen every Wednesday morning at 10 a.m. central time. And they include a mixture of presentations, interviews, book reviews, web tours, mini training sessions, and Q&A sessions. Today we have with us Katherine Rockmire, who is going to present about conducting surveys. Take it away, Katherine. Thanks, Emily. Okay. The topic for today is conducting surveys. This is the first in a three-part series, one a month from now through July. And today we're going to have an introduction and talk about questionnaire design. A little bit about me. I work here at the Nebraska Library Commission as a research analyst and special projects associate. Some of the hats that I wear, I'm a great writer. I'm also an evaluator. And there's my contact information if you need to get ahold of me. And who is with us today? Perhaps we have a library director, library staff members, trustees, or friends and foundation board members. And what are your credentials to be able to attend today? Just the point that you have curiosity and you're willing to navigate. We have people here from Nebraska, Wyoming, Virginia, Pennsylvania, Kentucky, and Georgia joining us today. So thanks for finding us and attending. We really appreciate your interest. And what are we going to cover today with this presentation? One, we're going to talk about the reasons for conducting a survey. We're also going to talk about introduction to questionnaire design data collection and analysis and reporting. We're also going to talk about measurement, question content, and structure. And then I'm going to give you some question examples. I'm going to try to get to the first two parts, three parts of this so that we can really get to perhaps the reason why you're all here, which is the examples. But I'm a fast talker. And so if you need for me to slow down, just go ahead, don't feel bad, just go ahead and write in the questions area that I'm talking too fast. I will not be offended. So what are some of the reasons for conducting a survey? Perhaps you're building a strategic plan, a marketing plan. You need to justify your budget to your funding sources. You want to do a return on investment study. You might be writing some grants or doing some fundraising, and you need some background information so that you can state your case. You might be looking at capacity building. You might be looking at a feasibility study for construction or renovation. You might be conducting a needs assessment or just doing some program planning or program evaluation. And what's the goal of why you're conducting a survey or why you are doing some of the strategic planning? Perhaps you are hoping to build a customer or patron loyalty. You're trying to find new patrons or customers. You might be focusing on the ones that you already have that are highly engaged and use the library frequently. You might be trying to predict future usage patterns. You might be trying to improve on your return on your investment. Or you might be identifying new services or programs that you would like to offer. And what are some of the questions that you want to keep in mind as you put together a survey? Opportunities, problems, challenges, barriers, your services, programming, accessibility needs. You might be asking about satisfaction or performance, how you are doing as a library. You might be asking about the expectations or the reputation or value or worth. You might also be using the survey data to compare to existing data such as census data or other state data that you have handy. And you might be looking at usage patterns or you might just be doing a snapshot of your library. And who are some of the respondents that you are going to target? And we'll talk about this a little bit later too and also in sampling in the second installment of this three-part series. But you might be surveying patrons, visitors, people who come just one time. You try to catch them while they are there. Maybe you are regulars. You might be looking at certain ages, ethnicities, gender, disability, geography where they live or perhaps types of income. You might be trying to catch people by the types of checkouts that they have or the types of questions. And non-users, you might be looking at potential users, people who hold the library in high esteem. You might be looking at people who are aware of the services and programs you provide or those who are un-informed, unaware or uninformed. And you might be looking at computer owners versus those who come in and use your library lab or book buyers versus those who come in and borrow. What are the pros and cons of conducting a survey? Some of the advantages are if you are doing a paper and pencil survey or online survey, there is no interview or bias. So the questions are put out there and there is no mediator. So people who read the questions get the questions directly and there is no middle person. They are relatively inexpensive. They are repeatable. They often gain thoughtful answers. You can catch more respondents with a survey or questionnaire. If you ensure confidentiality, you can get some pretty candid responses and they are fairly easy to analyze depending on the types of questions that you ask. Some of the cons to conducting a survey is that you are not insured qualified respondents, especially if you do a convenient sample and let anybody just pick one up. Also, the response rate for surveys can be kind of low depending on how you sample. If you have got 3,000 patrons or 20,000 patrons and you only get 20 people to respond to your survey, that is not a very good response rate. The responses are not going to probably be very representative of your population as a whole. Also, if you do a one-time questionnaire and they don't leave their name or anything, you have a hard time following up on any of the responses they give you. If you do a mail out survey, sometimes it is a poor turnaround time. You mail it out. You hope they get it back to you within three weeks. Sometimes they don't send it back. You send a reminder and by then you might be six weeks out. We are going to talk about survey research for a little bit. This is an introduction to survey research. I have a background in survey research and methodology. Whenever I would tell my friends or colleagues that I have taken classes in survey research and methodology, they usually say two things. Wow, that is a mouthful and two better you than me. If we are going to talk about survey research, just think of this as an investigation. Please don't let the term survey research be daunting for you. There are ways to break it down. For your purposes, this doesn't need to be completely clinical or academic. We are going to try to look at the basics of survey research so that you have an understanding of what needs to go into conducting a survey so that you are thorough, but we don't need to be completely academic about it. I don't think anybody is going to write a thesis after this. What is survey research? It's the study of attitudes, beliefs, and behavior of people in their settings through questionnaires administered by mail, email, handouts, personal and telephone interviews, and the internet. The difference between surveys and questionnaires. Survey is the overall investigation. Questionnaires are the actual product that you're using through mail, email, handouts, etc. I'm going to talk about questionnaires in a little bit. Survey is the actual investigation. What types of surveys are there? There's the questionnaire, which is you can send out by mail or you can do paper and pencil. In other words, have it available at your front desk when people check out. It can be group administered. If you're doing a focus group or you have a group that's already together for a program, you can hand out the survey and ask them to fill it out right then and there. You can do it over the internet, such as SurveyMonkey, which we will talk about in part two of this three-part series. Or you can do it by email. You just send it out in the body of the email. You can also send it out as a Word document attachment or PDF attachment and have them fill it out and email it back to you or print it out and mail it in. The other option is to do interviews. You have somebody at your front desk and you just ask them three quick questions and you fill out the paper while they answer your questions. You could also do a telephone survey where you call people up and ask them the questions over the phone and you fill in the answers for them. Survey research design. Some things to keep in mind is the ease of use, depending on the type of questionnaire that you're doing. You want to make sure that it's as easy as possible for people to fill it out. If they get frustrated at all, they're going to quit. You want them to get to the end of the survey and send it back in. Length of survey, that also, length of your questionnaire that also ties into the design. You don't want it to be so long that people poop out midway and then you don't get all the responses that you need or you don't even get the questionnaire back at all. Sensitivity. If you're asking sensitive questions, you are more likely not to get as many respondents. They'll get to that question and they might opt out because they don't... Instead of skipping the question, they may just decide not to complete the survey at all. The layout of the survey, we'll talk about this in part two of this three part series, how it's laid out, makes a huge difference on how easy it is to get through. That's a helpful thing to keep in mind. Then also, what language are you going to be using? If not all of your patrons speak English, you might also consider a second language survey, which is time intensive and requires quite a bit of forethought and quite a bit of work on your end, but it can be done. Then once you've got your survey designed, please don't forget that you need to have it pre-tested by several people. Have a couple people on staff look at it. Have a couple patrons look at it and you have them go through it and you sit there while they're doing it and you know what happens and you might ask them questions along the way or afterward. Then you might have a couple of non-patrons use it, do it, especially if you're going to be surveying non-patrons, just so that you get some feedback because you might be using some language. You might use the word reference and the non-patrons or even the patrons may not have any idea what you're talking about in the question, so then you need to reword your question. That's where the revision comes in. Then data collection, you need to start thinking about your target audience. We talked about users versus non-users and even within users, you may just want to do a sample of people who attend programs or people who are checking out items at the desk, that sort of thing. Sampling, that goes into it too. That's how you decide who's going to do your questionnaire. Incentives, sometimes if you put people that please survey in for a drawing for a book or a gift card or something like that, you might have, especially if you're looking for a larger sample and you really want a lot of people to participate. Sometimes it's not so bad to do an incentive. Then if you ensure confidentiality or anonymity, you might have more people who will be more willing to complete the survey. Confidentiality means that their responses will remain confidential. In other words, any information that they give, their personal information will not be attached to any of their responses and their responses will be compiled with like responses. Ananimity is if that you, for example, send out a paper and pencil survey and they return it, and there is just no way that you would know who completed it. Most of the time, we survey researchers try to ensure confidentiality. It's pretty tough to ensure anonymity. And then survey research is method of administration. We've already talked about these a little bit. Paper and pencil, electronic and online. And we will talk about those again in part two of the three part series. Once you have your information, you need to look at your data analysis. And it helps when you're forming your questions to keep your data analysis in mind. Because if you want it to be easy to do data analysis, that will determine what kind of questions you ask and what kind of response categories you have. Descriptive statistics are frequencies, means, modes, averages, and that sort of thing. Qualitative versus quantitative data. Qualitative is descriptive. So if you have different categories, you ask them, you know, which would you prefer? Would you prefer self-checkout or check out at the desk and then choose one or the other? That's one example of qualitative. Or if you leave an open-ended question and then you look for patterns, that's also qualitative. Quantitative data is, if you ask on a rating scale, say one to ten, it's numerical. Or even on a scale from not at all satisfied to very satisfied, you can kind of put that from, if you had five categories, you could do one to five and rate it that way and find averages. So quantitative, you're able to run, usually run averages and different kinds of descriptive statistics. Qualitative, it's a little more difficult. And then coding, say for example that you did have not satisfied, all the way to very satisfied and there were five choices in there, you could code it one through five, something to that effect, so it makes it a little bit easier to run your analysis. And I think we'll get into that in part three. Reporting, which we will also talk about in part three of the three part series, this is just a real basic way to report your information. You have a title page, table of contents, introduction and objectives, executive summary or highlights, which is always very popular. And conclusions and recommendations, then sometimes you could, not sometimes, but if you're doing a full report, you have the complete findings of the study, maybe even question by question. And then supporting charts and graphs and appendices, if you have anything else you need to add. Just one moment, I'm going to take a quick break. If you need to check to see who's here, give you a chance to breathe. Did you find the 13th person? This person? Sorry, we're just trying to take attendance here. Okay, all right, make sure that we don't have any questions so far. Nope, we're doing good. Okay, I know I talk really, really fast, so just want to make sure everybody's keeping up with me. Okay, let's, this is the meat of this presentation. We're already 20 minutes in, and I just hope that we can get through all of this. This is the third part of this today's presentation, and we're going to talk about questionnaire design. Again, I already talked about the difference between a questionnaire and a survey. The questionnaire is the research instrument. The survey is the research, and the questionnaire is your instrument or tool, and it consists of a series of questions and other costs for the purpose of gathering information from respondents. And when you're talking about questionnaire design, you talk about measurement, and one of the things you would need to consider is operational definition. So, operational definition means what are you really trying to get at? Are you trying to get at satisfaction? Are you trying to get at usage? Are you trying to get at people's intent, how they would intend to participate? Are you trying to get at people's behavior? And then two other parts of measurement are validity and reliability, and if you would look at that bullseye, validity means that you're getting at what you're really trying to measure. In other words, you're hitting the bullseye, you're getting right in the middle every single time. Reliability is that you're doing it consistently. So, if you ask three different questions that try to get at the same concept, then you've got a reliable, you've got a reliable instrument. The only thing is, let's say that if anybody else knows what, in playing darts, let's say you get a triple, you don't get the bullseye, but you get the triple three times, you're very reliable, you're getting it consistently. But if you're not hitting the bullseye, it's not valid because you're not getting at what you really want, what concept you really want to get at. So, that is something to keep in mind, especially if you're pretesting and you ask people, so what does this question mean to you? What do you think we're really trying to get at when we ask this question? That helps you with your validity. And this is survey research at its most detailed, and there's analysis that you could do, but you know, I think if you do pretty good pretesting, you'll find whether or not you're getting at the concepts that you're trying to get. Okay, so when you're measuring, again, this has to do with operational definitions. Are you measuring people's attitudes? Are you trying to find out what they're doing, what they intend to do, some of their attributes, other characteristics, and classification or demographics. In other words, who are they? Are they male or female? That's one of the demographics, gender, age, race, ethnicity, income, education, those sorts of things. And then we look at the question content of the questionnaire. And as I give you examples, we'll talk about these in greater detail. So dichotomous, I asked you, if you ask people a question, would you prefer to use the self checkout or use the face-to-face checkout? That's a question, that's a dichotomous question. They have to choose one or the other, or you can ask them about their behavior. Do you use, most frequently, which do you use the self checkout or the face-to- face checkout? And they have to choose one or the other. So dichotomous means two. Multichotomous is, if you ask them from, say, if you give them five options, ask them which one they're most likely to do, or which one they most likely, or which one they most prefer. So asking them, which are you most likely to check out books, books on CD, DVDs, CDs, and then pick a fifth one, and ask them to choose which one they think that they use the most, or something to that effect. That's a multichotomous question. Scales, that means that you're trying to get at something that's varying by degree, and it goes from one end to another. So for example, from not at all satisfied to very, from very dissatisfied to very satisfied, and you have a few in between, perhaps a scale of five, or from poor to excellent, and you have four on that scale. That's the kind where you can you can code them from one to five, and then you can find averages, and that sort of thing. And then there are these open ended questions, which you get into the qualitative sorts of analysis, and these are pretty difficult to, pretty, there can be somewhat difficult to analyze, you know, depends on what you're trying to get at, and how much detail you want to go into with this, with this survey. For example, I just sent out a flash poll, and I just asked everybody what type of social media they participate in, and that they follow social networking. And so people were sending me back information like I follow this blog, and this blog, and I follow this group on Facebook, and I follow this person on Twitter. But then some of the people also gave me even more information saying, and this is what I like about it, and this sort of thing. So there are ways to kind of code that, and look for patterns, or maybe you're looking for certain catch phrases, or something to that effect, and so that's one way to analyze it. But open ended questions can yield quite a bit of information, but they can be somewhat difficult to analyze. And now what is the structure of your questionnaire? I'm going to kind of walk you through one, but you want to start out, general to specific, because you want to kind of get them in the mindset, but you kind of want to funnel it from the most general questions to the most specific questions. And also, I don't have this on here, but you also want to leave your most sensitive questions to the last, because if you ask the very sensitive questions at the very beginning, especially if you're talking about money, or budget, or taxes, or anything like that, you want to say that toward the end, because you don't want to put them in a somewhat precarious position at the very beginning, because they are less likely to complete the rest of the questionnaire. If they've almost finished the questionnaire, they might just skip that question and submit it anyway. So that's one thing to keep in mind. And then that goes with question sequence, again, with general to specific filters, or if you're trying to just find non-patrons, and you ask, do you have a library card or not? And if they say no, and you just want people who don't have library cards, then you can set aside the people who already have library cards. Skip patterns are, say, for example, you ask a yes-no question. Do you use the self-checkout? And they say yes. Then you ask them certain questions that have to do with using the self-checkout. And so there's some skipping that's involved. And it can be somewhat difficult to do that on a paper and pencil survey. It's a little hard. It's a harder on the end of the interview if you do a telephone survey. But it can be helpful because people don't get lost as easily. But if you do it on paper and pencil, they might get lost. And then the length. Again, you want to keep in mind how long your survey is going to be. You don't want survey fatigue. It's called respondent fatigue to ask so many, so many questions. You know, it's not such a bad thing to ask a five-question survey. You're more likely to get people to fill it out because you can promise them it's not going to take them very long. And they're done in just a matter of a minute or two. And you can glean some excellent information from just asking five or ten questions. What's the structure of a questionnaire? Usually you have an introduction. You tell them what you're going to ask them. Then the body part. You ask them. Along the way, you might give them some encouragement. You know, we're almost done. Thanks for hanging in there with us. And then at the end, you give a note of thanks. And then don't forget at the very, very end that you give them a referral for questions. So if they have any questions about the survey itself, they have somebody that they can contact. So now we're going to get to the meat of this presentation. And I just want to put a disclaimer out there. I am going to give example questions. And to the best of my ability, I have created questions that I believe to be valid in terms of concepts. But I just want a caveat here that if you're going to take any of these questions that you pretest them, and you make sure that they are useful for you, and that if you have any questions about them that you contact me. If you have any questions, I'd be happy to field questions about that. But again, I mean, I have a background in survey research, but these questions have not necessarily been tested and pretested, and that sort of thing. So I do want you to take them with a grain of salt and be sure that you do some homework on your part. So I'm going to give you a couple examples of an introduction, and I'll just read these out loud. Happy Place Public Library would like to find out where we are now as a library. Let me stop for just a second. I want to let you know that I'm creating a question bank, and it's going to be a word document. I will email this out at sometime tomorrow, and it will also be available on the training page where you signed up for this. So you don't need to write all of these down. I am going to send out a question bank with all of this information, so don't furiously write all of these examples down. You'll be receiving those by email. Okay, introduction. Happy Place Public Library would like to find out where we are now as a library. So you're telling them kind of what you're hoping, what you're doing. We would like to learn a little bit about your library experiences at Happy Place Library. So you're kind of telling them what your expectation of them is if they fill out the survey. The information gathered from the following survey may help us, you can pick any of these, evaluate, plan, and shape some of our future thinking about our library and the various needs of our patron. In other words, tell them what you're going to do with the information that you gather. It might be used for something else, and just it's good to let them know what you're going to be doing with the information that you gather. Then the next part is kind of the hook. Your opinions are very important to us. You want to let them know, you really need to know from them, you need some feedback from them, and that their feedback is important. And then the next part is about the confidentiality. Your responses will remain confidential. I think that enough people have filled out surveys in their lifetime as adults that they understand what that means. You could go into greater detail, but you will also, there's an opportunity toward the end of the survey to do that as well at a point where sensitive questions come up and it's a good place to ensure confidentiality. So then you ask them how to fill this out. Please take a moment to fill out this questionnaire and then you tell them what to do with it. Return it to Happy Place Public Library in the accompanying envelope by February 29th. And then there's your thanks. Thank you. Here's another example of introduction. This is one that I used for scholarship recipients. The Nebraska Library Commission is in the process of gathering information. So what are we doing? From whom? The 21st Century Library and Scholarship Recipients. And what are we doing with it? To find out how the scholarship program worked for them and if they have completed their degree program. So that people who are about to fill out a survey kind of know what the questions are going to be about. Here's the instructions. Take a few minutes to complete this questionnaire about your experiences. Now what are we going to do with it? This is for evaluation purposes and we will use some of the information in our federal grant report where we will discuss the outcomes of our recruitment activities and share the stories of how the scholarships have contributed to the developing library careers of scholarship recipients. And just reading that sentence I have realized it is too long. I needed to break that up. You're going to loosen people along the way. So lesson learned for me right there. And then one final instruction. When given the opportunity to comment, please do so in whatever detail you care to provide. And you might include that if you're going to ask for comments along the way and you'll see a couple of examples where people are invited to comment. Again, be sure to assure confidentiality. Even if it's not a very sensitive question there, I would do that. Some of the ways that you could say that is your responses will be confidential. No identifying information will be attached to your answers and your response or your responses will be combined with like responses. So we've got about a half an hour to go through these and I'm going to go through them pretty quickly. I think I have about 30 questions. So we're going to do about a question a minute here. And I'll try to breeze through these so that there's room for questions at the end. And just for that reason, nobody has any questions so far. Okay, here we go. The first question here is a scale question. And overall, how satisfied or dissatisfied are you with the services the library provides to the community? Okay, there's three things I want you to notice here. First of all, this is a general question. This is a good way to start out your questionnaire. Overall, in other words, generally, you can use word generally, overall. Next, how satisfied or dissatisfied? You've got to remain neutral. If you ask them how satisfied they are, you're not asking them how dissatisfied they are. You might give them the options below, but it's best to show both sides of a scale if they're polar opposites. In the question, how satisfied or dissatisfied? Now, what are you asking about? Services the library provides to the community. Now, note here that you're asking about the services the library provides to the community. Not to you. You could ask that as another question, too. So you can really break down these questions. And this is a satisfaction question. So when we're looking at operational definitions, you're looking at satisfaction of services provided to the community. Now, these are response categories that I highly recommend. They're used by the Gallup poll. They're used by nationwide polls. And they've been found to be very usable and quite trustworthy. Very satisfied, somewhat satisfied. Then you give them an out to be neutral. Neither satisfied nor dissatisfied. Somewhat dissatisfied, very dissatisfied. Now, note here that I didn't give them a no opinion. You might be able to do that on a telephone survey when somebody said, well, I really don't have an opinion. You might probe and say, well, just generally where would you be on this scale? And then if they really don't have an opinion, then you have a sixth category that's not offered and that's a no opinion category. But I would not offer that on a paper and pencil survey or in the interview because what you're really looking for is opinions and people usually have one. If they want an out, they can be neither satisfied nor dissatisfied. Okay, and that's a scale. In other words, you could rate this. You could code it one, two, three, four, five. Here's a dichotomous question. Do you have a library card? And it's either yes or no. In this option, I would give them a don't know, but it's still pretty much a dichotomous question because you're going to throw these people out. Here's a multi-cotomous with a skip. Multi-cotomous means that you have more than two response categories. So here's a good question. When was the last time you visited the library? Today in the last week, in the last two weeks, in the last month, in the last six months, in the last year. It's been over a year since I visited the library and I've never visited the library. So you've got your non-users right there at the very bottom. And then that one, if you're going to ask questions about the most recent visits to the library, you don't want to ask the people who've never visited the library about the most recent visits. They've never been there. So you would ask them to skip to a perhaps a different question. Please note that what you're trying to do is you're trying to frame your questions. You're trying to get people to start thinking about the most recent visits to the library. And you don't want to ask like in the last three weeks, you might not even ask in the last two weeks because it's kind of hard for people to count 14 days or you know that sort of thing. Again, this has not been tested. I have not tested this question. So these, this is a suggestion of my part, but you might even take out in the last two weeks because it might be hard for people to remember if they've been there in the last two weeks versus the last three weeks. And they go, oh I'm not sure. And then and then they're going to get a little confused and they might not want to complete the survey or they might get a little bit fatigued just trying to think too hard. So you might take out in the last two weeks and go from last week to last month. And then six months, that's a good amount. You don't want to ask one month. Again, people are going, oh that's 16 weeks and that sort of thing. Six months is half a year. And then in the last year. So you see it's fragile, but it's not a scale because you're, they're not equally, they're not equally separated. It's not today, yesterday, the day before that. It's today in the last week and so on and so forth. Okay, and one reason to ask this question is to help people start to put themselves in a frame of mind. So, multi-conomists, if you recall, what was the purpose of your most recent visit to the library? You may include today's visit. So this is, if you're asking questions of checkout, if you're asking people to fill out a survey while they are present, or if you're approaching people and filling out the paper and pencil survey for them. So these are examples. You could choose other ones. I would not, I would not make it too long. I wouldn't give them too many options because there's some fatigue that goes with that and can be confusing. Here's one that's, check all that apply. So you're not asking them to just choose one. You're asking them to go ahead and check all that apply. You do have an option for them to fill in under other and then you also give them an out. They don't recall what was the purpose of their most recent visit, especially if they haven't been there in over a year. Here's a scale question. I'm going to give you, I think, three examples to get at this one concept. To what extent was there, and it's about expectations. To what extent was the result of your most recent visit what you expected? Another way to ask this is how did your most recent visit to the library meet your expectations? So we're not asking, if it's, you don't ask, was this better than you expected? No. We're just asking what their expectation was and then you give them some options. Better than I expected, just as I expected or worse than I expected. And so that's your scale right in there and then if they don't recall you throw them out and if they've never visited the library you throw them out. So you still have a scale of one to three. Here's another way to get at it, in terms of expectation or satisfaction. Overall, how satisfied or dissatisfied are you with your most recent visit to the library? Now you've already done this one you would ask, if you've already asked people if they've never visited the library, they've probably skipped to another question. Otherwise you could put at the bottom, I've never visited the library. So this is a scale of one to five. Be sure to give them an out in the middle. Sometimes people use the word neutral, I would just rather use which has been tested neither satisfied nor dissatisfied. And here's another way to ask about satisfaction. On a scale of one to ten, with one being very dissatisfied and ten being very satisfied, how would you rate your most recent library visit? I can't, don't know if you can see in my mouse but another thing that you could do is under here you could write very, very satisfied and under number ten you could write very dissatisfied. So they don't get confused and think that ten is perfect and that means very satisfied. So that is another suggestion that I would make. You go ahead and type very satisfied under the one and very dissatisfied under the ten and then they can choose between one and ten. And note here it says circle one. So you're giving them instructions on how to answer this question. Scale, here's another scale. Please rate the following library services offered by or following services offered by Happy Place Public Library. And here's a very good scale. Excellent, good, fair, poor. Excellent, good, fair, poor. This one has been tested and retested and has been been tested for validity and reliability. It's a very good scale. This is where you would choose some of the services that your public library offers. I would go from most general or most used to least used so that you don't confuse your patrons in a library loan. But what's that? Say that one to the end. Over here if they haven't used any of these things give them an option where it doesn't apply. They've never used it in our library loan. They have something to check otherwise they're trying to rate something they don't use and we don't want that to happen. So here are some of the different things that you could choose from. Again, I would keep your list short. I wouldn't have it very long. But bookmobile service, save that one to the end. Assistance with finding a book or other materials. That's good. Catalog system. There might be a better way to say that. Assistance with using the catalog system. The checkout process. You might even use self-check. That might be one of your options here. Notifications about fines, customer service, in our library loan service, programming for a bill. So you can see it here. And then at the bottom, I'm going to go back here for a second. Along the bottom, you might write comments and then an underline so that they can comment or write, please comment on any of these services. Something to that effect if you're looking for some sort of follow-up and you glean a lot of good information that way. Here's another scale. How satisfied or dissatisfied are you with the variety of the following items? So in other words, you've already asked a general satisfaction with somebody's most recent visit to the library. Here, this is just a general question about their satisfaction with the following items. With the variety. Now, notice that's the concept. We're looking at variety. You might ask something else instead. You might ask about the condition of the following items or something to that effect. But be very careful when you're using these questions that these are asking questions that you're trying to get at. So here we go again with a very good scale, very satisfied, somewhat satisfied, neither satisfied nor dissatisfied, somewhat dissatisfied, very dissatisfied. This one, I wouldn't give them an out if they're not, just be sure to or their out might be, I don't check out these items. I don't browse for these items. That might be your sixth category. Adult fiction books, children's, and let me give you the examples. Adult nonfiction or self-help books, DVDs, children's books, teen books, books on tape, books on CD, magazines, and papers. You could probably come up with a few other things and then give them a chance to ask comments. But again, I wouldn't keep your list very long. I'd keep it short. Now, here's a place to put in a transition. You're asking them to change gears. You're asking them to change a little bit of direction and it's good to give them the opportunity to do that, kind of put things in context for them. So we'd like to know some details about your experience of programming at Happy Place Public Library. Or we'd like to know a little bit more about your experience with using our computer library lab. Something to that effect. If you're going to do that, you probably have a filter question earlier on or a skip pattern that says, do you use the computer's library? Do you use the library's computer lab? Yes, no. If yes, then you ask certain questions about it. A way to do a convenient sample is that people who are parked at a station in your computer lab, you could go around and ask each one of them to do that. Or you could have it come up on the computer when they first log on. Something to that effect. And then you could ask mostly questions just about the computer lab because you know they're using it because they're sitting there. Okay, so there's a transition. You're helping them change gears. So how likely are you to participate in adult programming? Notice adult programming in the next year, or you might ask in the next six months, you've got to give them some sort of frame of mind. And what you're asking is likelihood, intention. This is also a very, very good scale, very likely, somewhat likely, somewhat unlikely, very unlikely. You're going from one opposite, you're going from opposite, very likely to very unlikely. How important is it to the adult programming be offered to the community through the library? Okay, so I made this question up. Again, this is not been tested, but you're asking about importance. And to that person, to you. Of what? Adult programming. To whom? To the community. By whom? Through the library. So I made this question up. This has not been tested. So take that with a grain of salt if you happen to use this question. Here's another very good scale. Very important, somewhat important, not very important, not at all important. Notice I didn't do very important and very unimportant. You could do that. You could do very important, somewhat important, somewhat unimportant, very unimportant. But this scale that I've provided you here actually has been used a lot more and probably the validity and reliability of it are pretty strong. Okay, likelihood. This is again, you can either use this as a filter question because if they don't, you'd ask, do you have children living at home? And if they say yes, do you ask this question? If they say no, you have them skipped to another section. If you have children under the age of 18 living at home, how likely are they to participate in children's teen programming, children's or teen programming in the next year or maybe you want to say in the next six months? There's the likelihood scale and at the end, if they don't have children living home, they can opt out and then you can cut them from your respondents. Here's another way to ask a question. Do you have children under the age of 18 living at home and then whoops, I should have cut off the rest. Don't have them say, don't say how they participate in the summer reading program in the past. But I'm sorry, I have a typo here. The only question you would ask is do you have children under the age of 18 living at home? Either yes or no. And if yes, then they continue and if no, they skip to the next, to question 16 because the next five questions are asking about your children living at home. Another way to do it is if you have children under the age of 18 living at home, so here's your next question. Have they participated in the summer reading program in the past? There's your dichotomous question. Sorry about that. I hope that wasn't too confusing. If you have children at the age of 18 living at home, have they participated in the summer reading program in the past? This is another way to ask it. And actually, this is a dichotomous question because you're wanting to kick out the ones that don't have children living at home. This should be a yes, no question, so this should say dichotomous at the top. Another typo. Could you right now 11a and 11d for me? Thanks, Emily. Okay, how important is it to you, to you, that programming be offered to your children, your own children, through the library? So you notice how important is to you, the respondent, the programming be offered to whom? Your children. How? Through the library. We're talking about library services. We're not talking about community services. We're talking about library services. And it's good to couch that for them. Or how important is it to you, that programming be offered to whom? Children in the community. How? Through the library. And then you might here's an example, you could have used the comments follow up on the previous question. But there you go. Glean some really good information. What you can do with it or how you analyze it can is up to you. How? Now, yes, about children. You didn't say what age those children were. You might do that. You might say children ages zero through 13. Children ages zero through 12. This helps you give a better operational definition of what you mean by children. Teens, you might say 13 to ages 13 to 18 in parentheses or something behind the word team. Again, this is through the library. And notice, if people had said you had a filter question earlier, do you have kids living at home? And they say no, they're not going to be asked these questions. And then scale again with the open ended with the comment section. Teens in the community. You could ask some of these questions of people who don't have kids at home. They may have opinions about this. They may think that programming for teens is important. And so you might ask this question, you know, you might have done a skip that if if they don't have kids living at home, you might ask this question and this question about their own children. And then everybody lands later on this question, children in the community and then teens in the community. So again, this can get complicated, but please take this information and scrutinize this information that I'm providing you and use it to the best of your ability for what you're looking at. Here's another scale. So if you have children living under the age of 18 living at home, how likely are they to participate in the upcoming reading program, summer reading program? And that gives them an out at the end. Okay, here's another way to transition. Now we have some questions about and then whatever you're going to ask about. The examples I have here are for your if you have a website. So when was the last time you visited the website and then give them the address of it? And if you're doing an online survey and you inviting them from from your website, give them the option that they visited the website today. Again, you might skip from in the last week to in the last month. I've never visited the website. That's if you're doing a paper and pencil survey or doing a telephone survey or give people an option that they don't recall. You can always filter them out later because you're trying to get meaningful responses. You don't want responses from people who don't recall because then they might give you answers that aren't very valid. If you recall, overall, how satisfied or dissatisfied are you or were you with your most recent visit to the library website? And what was the purpose of your most recent visit to the library website? This is an open-ended one because they may know exactly why they were there. And you can then you can code that later. Another way to ask is what was the purpose of your most recent visit to the library website? Check all that apply. And I just gave you some examples here. Again, you don't want your list to be very long. You can always have other at the end. They can fill it in. You can code it later. And please rate the ease of use. Now we're looking at ease of use of the library website too and then use the word either find or access. I wouldn't use both because then you're asking this is a double barrel question and you're trying, you're either one to find out the ease of use of finding information or the ease of use of accessing. Not both because they're a little, they're asking two kinds of different things. And there's a good rating scale, excellent, good, fair, poor. And then here's where you give some encouragement. Let them know they're almost done. You want them to make it to the end. You don't want them to be fatigued. So you can say either almost done one more page or almost done just a few last questions or thank you for your time. We have just a few more questions. Are you a fan of Happy Place Library on Facebook? Yes, no, don't know. I don't have a Facebook account. You've got to have all these options. If you don't, if you just have yes, no, I guess people who don't have a Facebook account say that they're not, but you're not probably getting the information that you really need to know, which is how many people are on Facebook and have those. How many are fans? Here's another question. These are kind of out of order. You could have put this toward the beginning of your survey, actually. What would you say is the library's reputation in the community? Excellent, good, fair, poor. You're toward the end. This is kind of where you might ask about, if you're looking at doing some sort of tax levy or something to that effect, this is, you're going to save those very, very sensitive questions toward the end. And then to what extent do you agree or disagree with the following statements? Again, there's strongly agree, agree, neither agree, nor disagree, disagree, or strongly disagree. And here are the examples. The library provides value services to the community. The library has a good reputation in the community. The library has provided valuable services to me. See, there's a difference, valuable services to the community. And that's where you might bold the word community. And then you might put the next question right after is the one that the library has provided valuable services to me and you might bold me or to me. I would recommend using the library to my friends and family. You could break that out. I would recommend using the library to my friends. I would recommend using the library to my family. But basically it's people that they know if you could keep it together, friends and family. And then give them an opportunity to say something to agree with. There's room for improvement in the services the library provides. Because if you're only asking positive questions, you're more likely to get only positive answers. And if you're really starting, if you really want to know, get the pulse of people in the community. You've got to ask the hard questions. And then we're going to transition here to demographics. And this is where you lose some people because they don't, they think that you're going to figure out who they are. So you don't ask, you ask the very basic questions and you also assure them confidentiality. So we'd like to learn a little bit more about you. You remind them about the confidentiality. And you might say that the following information will be used for demographic purposes only. That phrase has been used fairly often so that people will kind of understand or usually understand what that means. So gender, male, female. That's a very basic question. Age, here's a good, here are some good categories. Under 18, 18 to 24, 25 to 34, 35 to 44, 45 to 54, on and on and on. If you do something like under 18, 18 to 34, then I would do about 20 more years, 35 to 54, 55 to 74, and 75 or over. So that you have the same amount of space between each of your categories. So under 18, 18 to 20, I'm sorry, 18 to 34, 35 to 54, 55 to 74, and 75 or over. You have a question out of the second hour where you could throw your examples. Okay, there is a question and I'll get to it in just a second. We are almost done. This is good to find out where people live, do you live within city limits or you could say in happy place and then whatever county it's in. So they can check all its applies. So whatever county happy place is in, in happy county, perhaps you have people that you know that are patrons that live in counties surrounding you and you might give those examples and then other county and they can write it in. And then you can code that out later. Length of time that you've lived here might be a good question to ask. Again, don't ask questions unless you really need to know or you're going to be, ask questions that are useful to you. I'm just throwing these out here. Highest level of education attained. These are fairly good categories, less than high school, high school diploma, GED, some college, associate degree, bachelor degree, graduate degree. Now if you have 17 year olds they're going to say less than high school. So if you're looking for adults you might filter out later on 18 and older and then find out what their highest level of education was that was attained. Race, check all that apply. I got this from the Census Bureau. White, black or African American, American Indian and Alaska Native, Asian, Native Hawaiian, another Pacific Islander or some other race. And they can check more than one. And so we'll talk about coding and analysis in part three of this three part series. And race and ethnicity are two different things. Race is these previous categories. Race is not Hispanic. Hispanics can either be white or maybe American Indian. And this is the way that the Census Bureau breaks this out. If you do want to put Hispanic on this one that's up to you. This is what I suggest. So then you have ethnicity, Hispanic or Latino of any race and not Hispanic or Latino. This is the most sensitive question. You must be very careful and make the case very strongly for why you would ask this question. This is where you're going to lose a lot of people if you ask their household income. And along the way you can ask open-ended questions. For example, saying comments or other please specify. And then toward the end if you have other comments for us about your experience please provide it below. I would say this to the last because if they breeze through this thing and then they want to go at length that's fine. If you make this your first question they might go on and on and on and then you actually cover some of those questions later on or they go I just wrote a diatribe. Now I've got to answer all these questions. And then if you want to know their names so that and you might say why. So that we can give you feedback on and you answer any questions that you've had respond to your comments. Then you say your name optional but give them an out. Don't ask them to provide their name unless they want to. Then that's one way to assure confidentiality. Be sure to give a note of thanks. Thank you for completing this questionnaire survey. We appreciate your feedback. And if you have any questions this is where you give the referral for questions. If you have any questions regarding this survey or you could say questionnaire please contact in whomever. So one of the next steps the next steps are to discuss you know what would be the use of a survey and how would a survey be useful to you? What kind of a questionnaire would you be doing? I recommend that you possibly look at taking participating in the next few workshops to give you a bit more information. And also you can contact me at any point in time. Here's your badge of courage. March forward. Keep going. There are some print resources here. These are survey research books. Survey research methods, sage publications. Good, good book. Internet mail and mixed boat surveys. Don Dilman is one of the foremost survey researchers in the business. Online resources. And if you have any questions I have one question. I wasn't able to open the document that was available for taking notes today. Can that document and the examples given today be set in another format? Yes, I will convert those to PDF and send those out. They will also be available when you search after today. This workshop will be archived and you can search for it through the trading portal again. Just search surveys and then you'll see surveys one and then all of the documents that I make available will be provided as links as well as this this slide show, this slide presentation. Thank you. Do you have another question up there? Oh, okay. Let's see. On the scroll on the bar above it, yeah up there and then click on that. Okay. Are there terms that have been tested and found to be better to use clearer than others in looking at academic library surveys of their websites? Terms like database and reference aren't understood by the students. We don't think that of these words as jargon but many loud librarians, the majority of our patrons do. Exactly. That is a very, very, very good question. Database and reference are very, very difficult. Reference, I would use the maybe the and you might just look at the literature about what people are using for signage in their own libraries. But one way to talk about reference is information. People who are trying to gather information or customer service, perhaps customer service. And then in terms of database. Yeah, that's a tough one. That's a tough one. And pre-testing might be involved in that to see what your patrons, what they recognize. Right. And another thing to do is if you're asking about databases, give them an example. For example, for example, and then give a couple of the databases that are most frequently used, that might help you out. Let me think about that one. And if I come up with something, you might give a definition of a database. That would help. It's always good when you're doing some of these questions. Just go ahead and give the definition of a database. So which databases do you access? By database, we mean and then give the definition. Yeah, I guess just go look at whatever you have on your website. I know I think like in city called them research resources or something like that. Yeah, resources, research resources. That might be good. Go ahead and maybe look at a couple of major websites. Seattle Public Library, New York Public Library, New York City Public Library, Chicago Public Library, and see what kind of jargon they use. If you're from Nebraska, be sure to turn in your CE credit and also as an evaluator and survey researcher myself. I do have an evaluation for you guys to fill out so that I might improve my delivery and make some informed decisions about future workshops that I give and I will email that evaluation out to you and ask you to complete that and return it to me. And I'd appreciate your feedback. Some related topics. We have a couple of upcoming workshops. One is about data collection and that is we're going to primarily talk about paper and pencil surveys and online surveys. And then the other one is analyzing data and reporting the methods. That's connecting surveys part three. Something that's archived that might be of interest to you. I covered surveys, focus groups, and observation in general, all three of those. And that's an archived workshop that you could attend. And just plugging our Encompass Live, we have a couple of upcoming sessions, cataloging with RDA with Emily, our very own Emily Nimscott, Tech Talk with Michael Sowers and then communication. Getting the word out. Does your audience does your audience hear what you mean? End of show. We made it. And it's only five after and you guys stuck around and I really appreciate your your time. If you have any other questions, I can field those now. Maybe spilled food. Let me see if we can Oh, let's see. There's one here. No, I think we're good. Okay. Oh, I got to thank you. Thank you. Thank you very much. Thank you for attending. My door is always open. My email door is always open. My phone is always on. Just give me a call or send me an email. And I will be able, I will be happy to help you out. The question bank, I will email that out to you. And I will also again email out the notes. And this time I'm going to add all the links that were that I listed because I didn't list that on the notes for today. Let me see. Right here. Print resources and online resources. I will add that to the to the notes page and send that out again today or tomorrow as a PDF. Thank you very much for your time. You are free to go. Thanks. Thanks, Catherine. Thank you all for attending and we'll hope to see you again in another In Compass Live.