 Hi everyone, I'm Sue Boffman from the Association of Research Libraries and let me take this opportunity to welcome all of you to our workshop on the focus group method. Thank you for joining us for the first of a series of workshops that we are offering to our colleagues who are participating in our research library impact framework initiative. This is an initiative that's been underway for a little bit of time and many of the colleagues on our call today are members of teams across our libraries who are working on five different research questions. They've been busy during their research on a variety of topics including space, special collect special collections, productivity issues, diversity, equity and inclusion issues as well. We decided to open up our workshops to our teams colleagues and it's nice to welcome all of you today with us as well. We're really pleased that you could join us and spend part of your day with us. As I said, this is a series of workshops that will be offering and all of you are welcome to attend all and any of them will provide more information about the series after today and provide you with information about the different types of workshops that we have coming up and how to register for them. Our workshop leader today is Margaret Roller, one of our consultants who's been working with us on this initiative and I know she's looking forward to talking with you today and sharing a lot of good information and practices and tools and tips with you about focus group methods. So, Margaret without further delay, I'm going to turn the microphone over to you and put myself on mute. Okay. Okay. Well, thank you Sue and hi everyone and I am excited to be here today. Let me go ahead and start sharing my screen with you. Hang on. There we go. I think I get go that I have a lot of content here today, and I'm well aware of that I'm purposely kept it kept it here in here for today because, because when this is all said and done probably probably tomorrow I would guess I'm going to send the slide deck to Sue. And I want, I want you to have this information so yes, there's a lot here. I'm going to begin by talking about qualitative research and the focus group method kind of overall some kind of general comments then, then get into quality considerations, moderator and considerations, and think into online mode and these first three areas I'm going to cover today really I think provide the needed context for because I know that the online mode is something that many or most of you are interested in, like most people these days. So I think what I'm going to go over before I get there is going to be useful when we get into talking about the online mode so a lot here, let's get started. And let's begin with what is qualitative research which may seem a little nuts to some of you but I first of all I have I've learned that people have different definitions of what qualitative research is but more importantly than that, really, is that you need to understand where I'm coming from when I talk about qualitative research you need to understand where I'm coming from to, to give context and understand what I'm talking about as I go along in our discussion today so this is what I mean by qualitative research. It's really obvious in the XPD and it is not quality qualitative or survey research for sure and the focus really is on context and interconnections which is another way of saying that that questions are related to each other so an answer once I get to any single question really more more contextually lies in the responses on the feedback I get to a whole bunch of other related questions. Here are 10 attributes that I associate with qualitative research and I have highlighted here. Four of them, the importance of context meeting the relationship between the participant and the researcher and the researchers and instrument and I, I've done that because in my mind those four really serve to drive much of what qualitative research is but of course there are six other unique attributes as well as you will see there. The focus group method now focus group discussions and the old days nobody do what a focus group discussion was now everybody knows what a focus group discussion is. But, but the reason for this slide here is to emphasize the, the point of interaction and that the whole point of focus group discussions are one of the main key points of conducting a focus group discussion is to understand and to realize the interaction among participants and and what added information that we get as qualitative researchers when we, when we see and hear that interaction and key when we look at strengths and drawbacks to the focus group method that indeed, one of the key strengths is the is the dynamics is the group dynamics and, among other things can stimulate recall you know you might have a group discussion with with students let's say and and it's only in the discussion that somebody says oh yeah I remember I remember now going to the library website I had forgotten I had done that, or things that nature so it also spurs spontaneous ideas and thoughts. It's a moderator to hear people think out loud, and that goes back to what I talked about earlier in terms of interaction, and being able to be attuned to and listen for the interactions among participants, which indeed may do what may change people's be thinking about something that would be human nature but that would also be interesting for the researcher to understand how maybe that came about so thinking out loud and listening for that is important it can create a supportive environment. If you are conducting group discussions with students with certain learning disabilities, it can create, again, a supportive environment which may provide good information for the researcher as well as a positive experience for the participants and clearly can offer a wide range of attitudes and behavior. Drawbacks for limitations to the focus group method right on the top you'll see exactly what I just said are strengths, which are group dynamics group dynamics can also have the kind of the drawback of stifling different differing attitudes and personal input can lead to group think which I'll talk about in a minute. For the moderator can be difficult difficult to control. It may not also be appropriate for certain types of participants, such as executives might not be appropriate for certain topics, and there may be ethical considerations. So for instance, you know what you talk about in a group discussion should ideally stay within that group discussion, but the reality is is that you have no control over that, and what participants talk about outside of the discussion. You never know. So that has to be a consideration as well. I want to shift now into quality considerations and I'm going to draw a bit on this, this, this book that Paula practice and I wrote back in 2015. And the whole point was to kind of talk about qualitative research in, in the terms of quality and specifically something we call it total quality framework which is, I'm not going to go through this framework but I'm bringing it up because I want to draw on that first component of credibility which has to do with data collection, and credibility has to do with a few things. One has to do with the scope of your research which has to do with coverage and sample design. So the scope simply is asking the researcher to think carefully about, you know, how well do your participants actually look like or characteristic of the population that you're interested in. If you're working with lists, how complete and accurate are they. We usually use purpose of sampling and in qualitative research and again if you're working with lists, you know, are you able to and if you are do you stratify them. And when you recruit, are you working through the entire list. These are all the kinds of things we also think about and when designing quantitative or survey research as as well. One of the things that falls into this category of scope is the question of how many focus groups do I conduct. Well, unlike what some people may tell you, I guess, is that it depends. And when people say, oh yeah, you need to recreate you need to conduct four groups to do this or six groups to do that. I would suggest to you from a quality standpoint that the number of groups you conduct needs to be considered at a couple of stages within the process one is at the design stage. So, where are your, your participants where are the people that you want to draw into your focus group discussion. What is the depth of your discussions do you imagine them to be. You know, how, how much alike or not alike do you want the participants to be within a discussion. What expected variations do you think are going to be the outcomes of your research and of course scheduling and money and human resources are always, always a consideration. I would suggest to you from a quality standpoint that you also think about how many groups to conduct when you're actually in the field when you're actually in the field conducting groups. And when you look at the groups that you've conducted to ask yourself questions such as, gee, you know, have I covered and addressed my research objectives and the key constructs and focus groups I have conducted if not do I need to go back and maybe add on, you know, conduct more groups or somehow, you know, or conduct more groups or go back maybe even to the groups that you already conducted and complete what didn't you wasn't completed in terms of objectives and constructs constructs. Does the moderator. Does moderator really understand the outcomes of each discussion and that may sound a little crazy but if I can't when I come out of a group discussion if I can't basically in my own words tell you what I learned in that discussion as it relates to the objectives and the key constructs, then, then I need to go back and revisit that group or maybe we need to add on and conduct more groups. Now, have there been variations in the data that can or cannot be explained has have all of the participants and all the groups we've conducted so far, participated in and shared their experiences. And how have the moderator skills have they met high quality standards for all the groups that have been conducted or not or not. Another aspect of scope has to do with non response now non response you're going to hear a lot of course and survey research and non response error where we talk about it a lot. But we also talk about it in qualitative research from a quality standpoint because have the participants that showed up to our group discussion. They the same or different than, than the participants people who who who didn't show up. So, gaining cooperation from participants is very, very important. And there's a number of ways to do it which I list here. The other aspect of credibility or data collection within total quality framework has to do with data gathering and here we're concerned about validity how valid our outcomes. And there are three ways to think about that one is in terms of the content by way of the moderator's guide. Another is by way of the researcher effects and another by participant effects. And concerning the guide, the guide development is is is critical for the validity of your outcomes and the guy. What's important here is that the guide is a guide. It's not a script. It's to give you an idea of the general flow of discussion that kinds of topics and questions you're going to cover kinds of probe probing questions you might consider as the moderator. And it has a funnel design and what I mean by that is that the moderator's guide goes from from broad to narrow. I think of it in terms of four stages with the first couple of stages where we're gaining general information on the on the participant and the topic of topic of interest, and then moving into stages three and four, where we where we're narrowing and we really want to talk about to our, our, we're really our focuses of the particular research we're conducting. Here's a quick example of a focus group study I did for Michigan State University, for example, these were actually online groups. They were asynchronous groups with faculty and staff, concerning the university's outreach and engagement and what we wanted to we wanted to learn the kinds of outreach and engagement activities that faculty and staff were engaged in. So we're trying to understand what resources they were using to support them but but our, but, but, but our real goal where we go where we, where we were going with that is to understand what the MSU office office of outreach and engagement was doing differently or better to support faculty and staff so we started off with just understanding their definition of outreach engagement their activities and then we moved into stage three where we asked about their resources and then their awareness of of the MSU engagement office and how this office has or has not assisted them and then stage four what we really wanted to know which is how could the office better support them and what would make a world class office and engagement resource for for these people. Moderator bias is another consideration when you're considering the validity of your outcomes and there's a number of ways that the moderator can bias the outcomes clearly. I've just mentioned a few here and there's more to come as as you'll soon see a giving selective attention to participants in a discussion or you know voicing their own opinion, or losing track of group dynamics. can happen, of course, even their physical appearance can have an effect. What can help mitigate. moderator bias is something called a reflexive journal. And we're not going to get into it a lot right now, but I just wanted to throw this on the screen to give you an idea of the kinds of things the journals basically a diary that the moderator can reflect on. and complete after group discussion and this is really true for any qualitative method. So the moderator in this case can can think about well what you know how did my personal values and beliefs, maybe impact the questions I asked or my listening skills or, or things of that nature. This can be a way for again to help mitigate moderator bias but really reflecting on the moderator's behavior and thinking. Moderator inconsistency can also impact the validity of your outcomes and, and I will tell you, and any of you who have moderated know this, that it's a big challenge to fully engage a group of participants with a very limited time and, and, and therefore, it's not so unusual that group dynamics may not be well managed, or that group dynamics might actually get away from the moderator a little bit. And in when that happens, one of the outcomes one of the things that can happen, of course, is that the guy as just one example, the guy is not fully covered in a discussion. And maybe the moderator had to jump the midsection of the, you know, maybe of the, of the guide, and go straight to what I just showed you stage four, and get to some of the ending questions. And if that's true that really has that really has really jeopardize the integrity of your outcomes. So you need to be very careful of that. Now the last point on this slide you will see is, is, is the idea that a moderator inconsistency can happen because the moderator has not any in a consistent way presented concepts of definitions. For instance, if you were doing a group discussion, and the main focus objective of this discussion was to understand participants reactions to let's say, a new onsite library service, and that really was a big objective of your research. If the moderator from group to group does not actually state, or in my case I would always I read something read the exact same description, or definition of this new new onsite library service in order to gain participants reactions, then, then the moderator has jeopardized the validity of the research. Simply because group participants from group to group are are reacting to different versions of what this concept is. So you have to be very careful about that. Our personal qualities the moderator, I think goes without saying and but why is it so important why is it so important to have strong interpersonal qualities it's because it's those qualities in the moderator that really help build support, and a trusting and that's important because that is how the moderator is going to enable and motivate participants to share their candid thoughts, which, and that's important because that is how the moderator the researcher gets closer to to the research objectives. Four key broad moderator skills, one is being highly organized, maybe that seems kind of obvious. moderator needs to be very well prepared, there's a lot of little things that need to happen such as confirming participants and making sure the platforms of the facility is all set up and moderator has all the materials. I've highlighted here this idea that the moderator has internalized the guide, and I can't can stress that strongly enough it's important for the moderator to come into a group discussion, having really absorbed the guide and absorbed the key objectives and concepts that need to be investigated in that discussion so that when things happen in the discussion group dynamics and all the things that can happen in discussion, which will happen. The moderator is prepared because the moderator has again internalize the guide and exactly what the moderator needs to be getting out of this discussion. So clearly being able to manage time the discussion to cover all the topics and to cover the objectives is is an important skill the moderator and again I would suggest to you part of that if not most of that comes from having internalized the the guide has to be a leader without being overwhelming it can exhibit control, but and knows how long to stay on a topic and when to move on and importantly what to move on to now it becomes very important when participants are making maybe taking the discussion in in other directions that you hadn't planned. But because you have really absorbed, you know, the, the guide and the objectives and all of that, you're able to say okay I need to move this over here I need to move this conversation back and I need to move on to whatever it is. You can do that effectively if you, if again if you absorb the guide and all, and how to speak firmly but respectfully and showing a lot of respect is, of course goes back to building building rapport, being open minded and flexible, being objective, you know that goes back to not interjecting your own values and the reflexive journal and all the things that we've already talked about. And to be able to deal with unanticipated events because, believe me, it's going to happen whether it's something just participants, not all participants show up. A participant gets angry or upset and trust me, this happens. And it changes to the guide there might be any number of things that the moderator needs to be flexible enough to adjust to and to deal with, which is another way of saying they need to be good logistics whether it's setting up the room or dealing with technology issues or whatever it is. The moderator needs to be attentive and I say here thank on your feet and I mean that regardless of mode whether it's in person or online. The moderator needs to be able to, to, to listen carefully to each participant while at the same time is following the each participants thread. The moderator is being clear about each person's kind of life story as it relates to our research objectives well at the same time. Staying attuned and listening carefully to the interaction between the participants. And what is going on with that interaction. And as I've already mentioned the ability to identify and assess unanticipated topics as they as they show up. So those are kind of skills but what are some of the techniques. I've already kind of said and here's maybe just a different way of saying it. The moderator needs to be very clear about about each participant's attitudes and behavior related to what we're talking about in the discussion. At the same time, be very clear and understand where the group overall is coming, coming out in terms of their attitudes and opinions and behavior with respect to our research objectives, you know, verbal and nonverbal techniques are obvious ways to get at any of this Now in terms of direct questions. There are, you know, three, the three C's of direct questions context. I just threw up here just a couple of examples of these of the three C's of direct questions. What is, you know, context what is your process for exploring scholarly literature relevant to your research. But from there we might want to actually have participants compare and contrast for us so we might say to what extent, if at all, do you use specific scholarly databases. The college university library website website or an all purpose search engine in your process. And then you might even want you know want to some clarification. And I've the example I put here is you know you mentioned earlier that you use whatever sources were mentioned in your process to explore relevant literature. Are these also the sources you use to stay current with scholarship in your field. I've done a whole webinar on the why question and we're not going to get into that right now. Let me just I'm going to just throw this slide on the screen and just mention and just and just make the comment that the why question is not always the right question and it's interesting because many people all the time will tell you that the reason we conduct qualitative research focus like focus group discussions is to get at the why question, you know, survey, survey questions and survey research tell us the what and qualitative research tells us why. And I'm just here to just mention to you that the why question is not always the right question and think carefully when you are writing your questions for your discussion guides. The reason is is that the why question can put participants on the defensive. You are now asking them possibly to justify their attitudes and behavior for you and so they're having to kind of search for a sense of response to you that makes sense, which is going to stop the flow of conversation. It also can cloud the meeting and the example that I use here is, do you ask the question. Why is the library important to your research. And I would suggest to you that that is a confusing question. But if you ask what are the specific aspects of the library that make it important to your research that has that is something that the participant can grab hold of they can understand that say oh you want specific aspects that can now think about specific aspects. Also the last bullet item I would suggest to you that when you rethink your why question that you consider the idea that you may not be asking the question that you think you are asking. The example I use here is if you ask the question why do you use Google scholar. I suggest to you that may be essentially a different question than what are the benefits you derive from using Google scholar, compared to other databases so I just point here is when you do your moderator guides. Think carefully about the questions of course you will. And when you're just about to put down a why question just to think very carefully about that I know I do and let me say, I use my question everybody uses the why question I understand that and I do. But I want every time that I am inputting a why question into my discussion guy. I am thinking carefully whether or not it's the right thing to do. Another way to kind of indirectly ask questions are under the category of what I call or what a lot of people call enabling techniques. So enabling techniques include sentence completion word association. storytelling, and I give you a few examples of what that may be. What about nonverbal techniques, and there's a bunch of nonverbal techniques, one is just silence and allowing allowing participants to think about your question before answering. So silence, which is another also another way to say be patient for slow talkers for shy participants who I'll mention again in the minute. And for maybe you've asked a pretty personal question that's difficult to answer so be patient, and actively listening what's actively listening, it's, it's making eye contact. It's not in your head it's smiling. It's all those kinds of things. When I do, particularly when I do in person groups, I use the easel, just your basic easel a lot, or whiteboard if it's there. I kind of like the easel. It's very tactile but the whole point is to use it to kind of. It says to the parents that appear, it says to the participants I hear you. I'm listening to you and it really helps kind of facilitate and create energy in the discussion. And I say here be approachable, which basically means again smile, and, and I've mentioned before address appropriately. I'm a big believer. I've been saying this for years now that whenever I do an interview or a group discussion, unlike today, I wear an open collar. Just because an open collar, I believe says to whoever it is you're talking to your participants that hey, I'm here. I'm open to, I'm open to what you have to say to me today. I'm going to not spend much time on this at all I'm, I'm going to kind of flip through, you're going to have this in your slide deck. I want you to have it, I'm going to flip through various participant types and you will see that I will mention the participant type and then provide a way to maybe as a moderator to deal with various participant types so one obviously is the, is the dominator person who wants to dominate the discussion. Kind of what I put a hostile argumentative participant, someone who has an axe to grind and I, if you have not had that experience, trust me. It happens, and there are various ways to deal with that. I mentioned, I mentioned just a little while ago the shy participant. And I have many experiences I think we all do whoever has moderated of shy participants and there's many ways to, to deal with shy participants. And that group think is, and I've mentioned this before, but it's something that the moderator really needs to pay attention to because all of a sudden you go, wait a minute. The group all seems to be agreeing with whatever is we're talking about how did that happen. So there are, and this happens on a fairly regular basis as well so here I offer a few ways to deal with that. And praying from the guy. This happens to and, and in this can actually be a good thing. Because maybe they maybe participants have straight off the guide into a territory that you hadn't anticipated. But hey, it's really interesting and you as the moderator thinks, hmm, I hadn't anticipated this but this is interesting and I think ultimately relevant to our objectives. So there are some decisions that moderator needs to make, you know, in the moment, but be aware that this can happen, obviously. Now for an in person discussion, it's not, it's not unusual for side conversations to pop up. And there's a number of ways to to deal with that. Okay. Online mode. Let's talk about that a little bit. There are. I've just put down here synchronous and asynchronous focus group discussions. synchronous is probably more common so video conferencing and asynchronous being called bulletin boards or discussion boards. There are a number of advantages to the online mode and you know, it used to be used to be. Not that like a go like just maybe a year ago, that I would have this discussion and this would be all be news to whoever it is I was talking with. But now after a year of the pandemic and a year of many of us moving to the online mode. Many of you are probably already aware of what I'm going to mention now. So, clearly, some of the advantages can be that you get greater coverage. And therefore you can have a more kind of representative on looking group of participants in your discussion it can be more convenient for for participants. It can be more appropriate for sensitive topics. And it could be a relevant form of communication, depending on on the kind of participant that you are, you are after in your research. It can also provide greater accuracy I put down here and more in depth information. And partly what I mean by that is, for instance, in the online mode you can gather multimedia you can get text, you can get video you can images you can get, you know, a multimedia approach to the responses to your questions. So particularly for the asynchronous mode. You can get very detailed thoughtful responses which also I'm here to tell you can also be a disadvantage, just because it is overwhelming. And then you have to deal with it. But it also committed mitigate particularly in asynchronous mode can mitigate recall, recall area error. The last point I have on this slide is that it may be an efficient use of resources now the reason I say may is because indeed you will not have travel expenses you will not have food expenses. Because we, we feed participants and observers. However, the cost of the online platforms can be the same as the cost of a focus group facility. So for the asynchronous mode when you, when you conduct bulls and boards or discussion boards that can be twice as much as a in person facility rental so it's not necessarily a better use of resources and also if you're paying an incentive or co up to your participants that is pretty much going to be about the same you're not going to save any cost in the online mode. So online mode. It may not help you in the resources you spend, but I would be very careful about that because it may not be true drawbacks to the online mode. There are a few managing the engagement takes a lot of skill and a lot of time. I want to pick up cues, and in the online mode, I put asynchronous here but even in the synchronous mode. It can be difficult to pick up. It's not like being in person. So it may be difficult to pick up the kind of visual and verbal cues that you would expect to normally get in the in person mode. I just as I just alluded to the analysis can be overwhelming and that you may have lots of data in terms of just volume, particularly again asynchronous lots of text data. But in synchronous or asynchronous, you may have in text but then you may also have video and images you may have the in the online work I've done participants have sent me documents for me to read to support the things they were saying to me in the online mode. All of which becomes part of the data so it can become overwhelming fraud or misrepresentation can be a problem security internet access technical glitches and difficulties and for the asynchronous mode text based communication can be an issue because it that is its primary and key form of of input. Now I have put some of these strengths and limitations into this schematic you see here and and remember when I talked earlier about the total quality framework and the credibility or data collection component of the framework. So you'll notice the aspect scope and data gathering and here I've just simply helped visualize a few of the things I just talked about. You'll notice that one of the things I have there I didn't mention earlier as being a an advantage under researcher effects are the platform features, which can help mitigate bias and inconsistency. These platforms have a number of features and, and, and as you see here, the, the, the number of qualitative research day dedicated platforms are many, and they're growing every day and just in the last week I have gotten that on a brand new qualitative research online platform that I had never heard of before. So it's, it's growing all the time, and they are continue to grow in terms of particularly in this past year, out of demand. Their features have continued to grow. Now, this is not something that wasn't, you know, that's new to in the past year, but as an example of some of the features as they have markup features where you can actually have participants markup an image of some kind to gain their feedback. They have heat maps that you can have participants engage in they also have other types of participant exercises, like collages and mind maps and just your video feedback and visual elicitation and things of that nature. So this, I conducted online focus groups for the EPA in 2011 and this is just a screenshot of where I manage the participants. And I have to tell you, I spent a lot of time on this page, because managing participants in the online mode and this is in this asynchronous online mode can be very time consuming and difficult. If you don't have a good efficient way to understand who's been in to respond to your questions, who has not been in to respond to your questions, who do I need to follow up and who do I need to respond to. And that kind of thing who is who has, you know, and actually it was this study I alluded to a few minutes ago when I told you. Some participants gave me documents to read this was a study with behavior and social scientists and a number of them gave me research and literature and things that they had been involved in for me to read in response to some questions I was asking, asking them so a lot to take a lot to keep track of him, but these online platforms do a good job of helping you do that. They also this is from I tracks those screenshots I just showed you a few minutes ago up until now, we're all from 2020 research. And what I'm showing you right now is from I tracks research. And just to show you their observer room feature, and how the observer and moderator can can talk to each other and this is this is from I tracks but other platforms have the same thing all these platforms are offering very similar features maybe done slightly in a different way but very similar. These, these on these dedicated qualitative research platforms also in addition to kind of features I've already mentioned also provide full tech support. You get automatic transfer transcripts and translation. They have multiple video security features, and many, most of them have panels from which they can recruit so they can also if, in your case you ever you needed recruiting. They can also handle recruiting so they're multi featured in terms of what they can offer the qualitative researcher and focus group researcher. Let's quickly in terms of a few of the design considerations. I've put up here so that you can kind of contrast and compare in terms of number of participants, the in the in person mode, we may be. Anywhere from two to 10. Now, grant you there are some moderators that might bring in 12 participants or more, maybe I doubt more but they might do more. My, my kind of limit is 10 and my, my favorite number is eight for the number of participants in an in person group for the online synchronous mode. Most qualitative researchers, at least in my network and it's in what my recommendation is is four to five participants. Of course you can have more you can have fewer but four to five seems to be a sweet spot in terms of getting a good interaction and engaged group but at the same time being able to manage it in the online mode. In the asynchronous or Bolton board or discussion board mode. I, you know, I'm putting I'm recommending here anywhere from eight to 18 and with the caveat that fewer is better. Again, in terms of being able to manage it. And having said that I'm saying that because if you went to some of these online platforms I've already mentioned and told them that you want to conduct a discussion board. I might say, Okay, fine, you know what we can. You can have as many as as 30 participants in your discussion board, because our online online platform will handle 30 participants. And I would just suggest to you, and I hope it's obvious by now that 30 participants is just of a just of a two or three day discussion board is just overwhelming not only to manage and to facilitate, but then in the end actually be able to do an analysis and do an analysis and a report. So length of discussion in person is usually 90 minutes to two hours. The, the synchronous online mode, usually 60 to 90 minutes. And asynchronous as I just as two to three days or longer or shorter. I mean it could be a day but usually they're two to three days. Cooperation is important in in in all of our research all the reasons I mentioned earlier in terms of the importance of gaining cooperation, but it can become particularly important in the online mode because we are also dealing with this whole issue of technology. So the, the idea here is that it's important. So the idea is you need to over recruit, and we over recruit even in person mode, we always over recruit, because for whatever reason not everyone will show up. But communication here about, among other things about the technology to is important so to include in the recruiting process, not only through screening participants concerning their qualifications for the discussion. And things of that nature, but also about your intended use of technology their comfort with that, that technology you're going to use their comfort with be having their webcam on and using video in the discussion needs to all happen at the recruiting time, but also in the communication you will do after recruiting and she's do a lot of communication after recruiting terms of confirming the recruit reminders in all those types of communication. It's important in addition to to confirming and and mentioning again about the date and the purpose of the research and why it's important that they show up is to also mention again the technology and to mention again that your intention is if you're conducting synchronous online discussions that you expect them to have their webcam on the, in terms of moderator considerations, I would suggest to you that the qualities and skills that I mentioned to you earlier, really don't are equally important in the online mode as they are in the in person mode so everything we talked about before, I would suggest to you again are just as important in the online mode. In terms of techniques, the moderator needs to pay attention to the to the fact that there will be differences in the online mode in the participant interaction. Typically the interaction participants will be less interactive, which means that the moderator is going to need to have to call on participants. Particularly if the moderator has not upfront made the participants comfortable and how they should interject a comment or interact. So if I'm a participant, and I'm a little bit nervous being in this discussion with some people I don't know any of these people I don't know the moderator. And if I don't know exactly how how is am I supposed to just talk, or am I supposed to raise my hand, or what am I supposed to do. What I'll do instead then if I uncomfortable about all that is I'll do nothing is and I will just sit and be quiet. So participants need to understand how they can can interact. The moderator will need to pay more attention than in the in person mode to the flow of discussion, the flow of the conversation because it will be more difficult and they may indeed need to interject some more some exercises or things of that nature in order to stimulate and maintain some energy. And maybe it goes back to some of the things I've already mentioned to you when I showed you the screenshots me you know maybe it's marking up an image or things of that nature. And the moderator needs to pay attention to the fact and be aware going into the discussion that participants may be distracted. I mentioned earlier this idea in the in person mode of side conversations. Well on the in the online world participants just may be distracted and need to step away. And the moderator needs to be okay with that. It needs to be okay with allowing the participants to turn off their video. The moderator cannot rely so much you know I talked about earlier about active listening and kind of nonverbal techniques. The problem here is that relying on eye contact or just just just the feel and the sense of the group that you, you get when you're doing discussions in person may not be there in the online mode and probably won't be and and the moderator will need to again look at some kind of an exercise or something or be very explicit in the in the questions that are asked in order to feel good that that the moderator is gaining back from the participants. What what is going on in the discussion, it may be, I put down here a poll, it may be that the moderator wants to interject a poll because you're not quite sure how they're feeling about something so you might put a poll on the screen to get a feeling about that and then and then jump off from the poll with a whole slew of various questions. And as I've alluded to and just say here specifically the moderator needs to be more direct, all this is to say that the moderator in the online world is going to need to be more direct and be more focused. Okay, so that kind of comes to the end of my slide content I have here this is a blog that I have about 250 articles in this blog. I have a few, a few addresses here that you can go to one having to do with the framework I mentioned another having specific to focus groups, but there's, there's a bunch of other stuff there to you may or may not be interested in. And here's my contact information and I'm happy to connect with, with any of you. So, that's it for me. If anyone has any questions, comments, we have a few minutes left in our session. I'd be happy to talk about them. Thank you. Thank you Margaret. So, everyone if you do have a question. There's two options one you could pop it into chat and we can all see it and I will work with Margaret to make sure we share the questions or you can raise your hand. If you'd like to do that. If you hover over the bottom of your screen. If you haven't done this before there's a icon that says reactions if you click on that there is an option to raise your hand, but yes please jump in if you have some questions. So you're all thinking and getting organized about about your questions. Margaret, could you talk a little bit about the guide and do you have, and perhaps even an example of a good one that you might be able to share or could point people to some references where they might get some tips on what what a guide looks like. Sure. Let me make myself a note. And I can send that to you Sue. Sure, and we can make sure it gets distributed or everyone has access to it. Sure I think that's a that's a good idea I will do that. And give you a feeling for that. Um, I'm thinking maybe what I might do Sue is send you an example of an in person discussion for the in person mode for in person discussion, and then for the online mode. For, and I'm, well I'm thinking about maybe the Michigan State example I used maybe I'll send you the guide for that. That would be great. Thank you. Sure. Any questions. If you don't have questions now, you know fine. Happy to address any questions or comments after today. So, feel free to contact me. Okay, I see Michael has a question so Michael and why don't you unmute and jump right in. Thank you. Thank you so much Margaret. I'm just curious. Especially since the pandemic has be fallen has come to all of us. A lot of online interaction has involved really large groups and I know I've participated in meetings with 200 or more people and I'm You know these are like some of them have a discussion format and but I'm wondering what your thoughts are on this reality of some of those kind of interactions involving so many more people and the unique challenges are and you know I realize those are not framed always as literally a qualitative research project. But we seem to be doing it a lot more I'm just curious on your thoughts on those large groups and, and how we bring some do we bring some of these ideas to that or, or not you know what what are your thoughts on that. Well, little difficult to answer only in that I, you know, so what I'm going to say Michael is that it will depend of course on your objectives. Clearly, if the objective. And I don't know if these are group discussions where, in the end, somebody has to report back what went on in the discussion and some kind of reasonable way and, you know, do and research make interpretations that kind of thing then, then clearly, you know, we go back to what I've already said about keeping the group small and that kind of thing for a group of 200 I'm just, I don't know what the purpose of a group of 200 would be is struggling with. So I don't know is so is there. So there's a moderator or a facilitator for these large groups Michael. You know I think. I think in higher ed, we are seeing, you know, a faculty meeting would be caught like either in a large academic department or an entire group of faculty. Yeah, and I, you know, I really appreciate I think when I think about it. The question seems to come back to what is the objective, when that is happening. I haven't studied it but but I just know that a lot very large groups are getting together in that way. And also as we know because often in the past they would have been inside a physical space and yeah so they're online in large numbers yeah. And one of the considerations there may also be that the person organizing that discussion is really one of the areas of focus is being all inclusive. I don't want to leave anybody out. And so I want to create this inclusive environment and it winds up being 200 folks on in this discussion. Which is, of course, a very different thing than when we have very specific research objectives and we're very targeted towards those objectives and, and it's all about how we're going to answer the research question. That's based on the objective. And that is our goal and that is our, our, our focus going into the discussion and, and as the researcher we're saying well how is that going, how am I going to make that happen. Because in the end, I, in the end, I have to analyze that. And I have to make sense of it. I have to provide my interpretations and recommendations based on that and write a report. So that's where my focus is, if my focus is to is to have a discussion among faculty. My objectives are very different, I think, and one, and one of which may be this all inclusive idea. Right, right. Yeah, thank you. Yeah. So can I ask a question, can I, can I get a feeling for how many of these folks are, or have conducted folks through discussions. Why don't I, let's see if I can set up a poll, why don't I do that, and then that can spark, maybe we can get people to respond and in the meantime, we have a question from Gordon. That is, which is, what is the best way to select participants for a focus group. Hi Gordon, I. Unfortunately, participants. Come in different shapes and forms and so it's difficult to, to say there is a way a best way to to recruit a focus group participant. It will really so going back to what I said at the beginning of the discussion. If you have a list of students of faculty of community members, possibly you would utilize that list in the ways that I've already talked about, and you could, and then, and then you can send out emails. I say that, but what's very important is that you also contact them by phone, and I'm hesitating here because typically what we've always done is we've done it by phone. We, we usually have some kind of list even if it's the, the qualitative research providers panel database. We would begin by getting on the phone and recruiting participants and going through that list. So these days going by way of email may be more appropriate to start, but even if you do that, you need to follow up with phone. So that there is a connection between participants in the moderator. Gordon, I don't know if that answered your question but that's a very broad answer to your question I guess which is start with email or phone. Make sure you follow up with phone. I don't know if that helps or not. So Margaret, I have put together a poll. So may I launch it real quick and then we can get back. There's another question that surface to see I'm looking at chat I do see. Go ahead, go ahead. Here comes everyone. I didn't title the question. Answers are still coming in. Looks like everyone has noted that it's planning to. So let me end the poll. I'm going to share results. And you see that most of them have yes I do see that. Yes, most of them have. Okay, great. Okay, good. No, but would like to so we have what almost 80% saying no, yes or no I would like to. Okay, great. Great. So you do have a question that's come in from Beth, do you see it or would you like me to read it. Let me see, I see something from Jessica. Oh, maybe that. No, she's just just I say, okay. Beth, could you talk a bit about using snowball sampling as a method for identifying participants in a focus group situations where it may be useful technique and those where it may not be a useful approach. The problem with snowball sampling and why we use purpose of sampling and discourage convenience or snowball sampling is for the obvious reason that with snowball sampling, you're, you're jeopardizing the integrity of your data. If you're, if you are simply recruiting people who know people who know people who know people. Because you. It's which is a very different approach right then. Looking at a list of potential participants and then systematically going through that list. I fairly want to have in my discussion, people who were recommended by one person, let's say I recruited one person in this one person now has brought in a whole bunch of other people that is not. That's not a quality approach obviously to the focus group method in that now I have people who may be of the same way of thinking may not be very dissimilar in their what we call lived experiences. And then maybe whatever character their lifestyle, whatever it is. And that is not what we are looking for when actually when we conduct any piece of research. When it may be useful. It may be, you know, having said all that it come if it has and it may come into play when you are dealing with what I'll call scarce or hard to reach popular segments of the population. People who there's maybe there may not be great numbers of them you don't have a list you don't have any way of that contacting them. You have a few and far between, and you have to go through targeted groups or individuals in order to gain entry to, to the kinds of people who you are looking for, you know, this in a way goes back to the very beginning I didn't leave it on the screen that much but I think one of the last items on that gain, get how to gain cooperation. I have on that list gatekeepers and gatekeepers can be very useful in helping to helping the researcher gain access to population types, particularly those that are hard to reach, or just are very few and number. The problem though you have to always keep in mind is that you could be jeopardizing your research. Maybe you're not maybe you are dealing with such a selected group that it is very appropriate to take this method. But you just need to be very, very careful about what you're doing and need to document it. Not only for the analysis to be thinking about that but actually in the reporting function need to document that that's exactly what you did in your sampling. Thanks, Margaret, there's I think one last question before we run out of time this afternoon from Michael about the facility moderator, do you see that. I will also share your opinion that if a library is holding focus groups whether it makes a difference for the moderator to be directly affiliated to the library or third party moderator okay. As a, you know, a flat answer here. My recommendation would always be to. Okay, would to bring in a train skilled outside moderator. That is who is going to do have all the qualities that I shared with you today in this session. That is who you. Yes, I am my recommendation is always for bringing in a skilled moderator from the outside. Having said that. And I know that sounds self serving because I am the kind of outside moderator, but I don't. So, but having said that, the reality is that for because of time because of money, because you just don't know. You don't know an outside skill moderator. You may lean towards using someone in the library to moderate. And that's just life that's reality I understand that I get that. But what I would suggest to you and strongly suggest to you that when you go that route that you are very careful that that that moderator is trained and is skilled and all these things that we've talked about and that, that, including the guide development and all the things we talked about we haven't talked about analysis reporting but we will talk about that later and other workshops but for for today just to make sure that that whoever you take on board is skilled in those areas as as much as possible. Thank you Margaret, Beth you are appreciative of Margaret's response and said you would like to make a comment and in our the last couple minutes that we have left. Thank you so much I was just rapidly trying to type it. I worked with a team on a study a few years ago where, and this sometimes happens I think in the library world where you're trying to, you're trying to gauge needs, or, or something about a community of of scholars that is is really far flung. But it's hard to, there's no list, there's no, you know, yeah association that their members of her, you know, and there are no gatekeepers. One of the things that that the team that I worked with did was to to we identified a few people did individual interviews with them and used those interviews to seed a list. We carried out a scoping review on that topic so that we could uncover as many names as we could who'd been publishing in this area. Great. And so we kind of use that to to sort of hold up against the list we were coming up with by talking with people. So that was something that that's a yes it takes time to do a scoping review, but yeah, you know, if you just work up your Zotero and get your team together. Yeah, that's one thing that, you know, we librarians sort of know how to do. That's great. I love that. How creative. Great idea. I love that. It's not my idea, but regardless it's a great. I love it. Thanks for sharing that. Thanks. So, Margaret, we've come to the end of our time. Is there any last comments or words you wanted to share with the group before we say thank you and send everyone on their way. All right, I just well I do want to say thank you and I'm glad you're all here today and I would just repeat what I said is, if there's any follow up discussion or comments or questions that any of you have please don't hesitate to get in touch and I'm happy to talk about it. Thank you, Margaret. Thank you again for your workshop on the focus group method. Thank you all. I'll add my thanks to all of you who attended. Thank you very much. We will follow up by sharing the recordings from today's session as well as last weeks and Margaret slides and she's offered graciously to share some examples of guides. We'll be in touch soon and we'll be sending information about our future workshops coming up. So, thank you everyone. We'll say goodbye. Thank you. Bye bye.