 Hello, everyone, and welcome to Data Sharing for Qualitative Research. My name is Crystal Stelten-Poll. I'm with Dartmouth Center for Program Design and Evaluation and the Center for Open Science, and I'm very excited for this panel today in which we'll talk about some common concerns around data sharing for qualitative and mixed methods researchers and also some potential solutions. Today, I am joined by Kristin Elden Wiley, Sebastian Karcher and Rachel Renbarger. Just as some folks are coming in, I'll go ahead and give us a look at what we'll be doing today, and then let folks introduce themselves and then we can go ahead and get started. So first, we're going to have some comments about the importance of open data from a funder's perspective, and then we're going to follow up with some discussion about some common considerations from a qualitative perspective when it comes to data sharing. And then we'll talk about some potential solutions that have come up, and then hopefully we'll have plenty of space for folks to ask any questions. There's the chat and also the Q&A function through Zoom, and so feel free while the presenters are talking to ask any questions that you may have that come up, and then we will try to answer those during the discussion section and also allow the panelists to discuss with one another anything that comes up during the panels. So with that, let's go ahead and let folks introduce themselves. Kristin, why don't we start with you? Sure. Thank you, Crystal. My name is Kristin Elden Wiley. I am a Senior Program Manager at Templeton World Charity Foundation. So I will be here talking about the funder's perspective of the importance of open data. I as a part of my Senior Program Manager role, I'm responsible for promoting best practices of open research both within the foundation and with our grantees. Thank you. Rachel? Hello, everyone. My name is Rachel Rinbarger. I am a U.S. Education Researcher at FHI 360, which is a global nonprofit. I am currently excited to talk to you about open science, mostly because I was trained as a quantitative researcher and then have transitioned to qualitative and mixed methods, right as the open science movement was happening. So I have a lot of thoughts from my people and others around this topic. Thanks. Thanks. And Sebastian? Hi. I'm Sebastian Cartier. I'm the Associate Director of the Qualitative Data Repository, and I'm also a faculty in political science here at SirQ's University. And so we archive qualitative data, and I also do a lot of work on research transparency and qualitative and multi-method research. And so I have heard a lot of the concerns that many of you probably arrived at today. And I'm not going to dispel them all, but I'm going, I also don't like the word solution. I would maybe have called it approaches to common concerns had I raised that. And that's kind of the perspective that I'll try to provide. Awesome. Thanks, Sebastian. All right. And with that, I will, I will get off a video and I will cede control to Kristen. Thanks. Thanks, Crystal. Okay. Okay. Okay. So I'm probably preaching to the choir here a little bit, but I'm just going to go briefly over why we think that open data is important. So as a funder, we want our research that we find to have an impact and accelerate discovery, probably nothing new. And we also, but we also want to know that if the research data we fundish, but we also know that if research data we fund is shared, the potential to accelerate discovery grows even larger by adding transparency, sharing data enhances trust in other outputs and then increases reproducibility and reuse and makes it easier to get to new discoveries by building on existing knowledge. There's also another reason we want data to be the data we find to be shared ways to increase equity. We want the research that we find to be available to anyone with an internet access and data sets are part of the research that we fund. Go to the next slide. But we also understand that it's not enough just to put the data out there and that there's no point in sharing the data that's not transparent and reusable. We know that managing, preserving and when appropriate sharing data sets is hard. It takes planning and it takes resources. Guidelines we think can kind of move us from the values and principles of open data to the practice of open data. It can provide practical tools and training and facilitate and build infrastructure. I don't think there's much impact in saying, you think it's important, but mapping out how it will be done takes us a step closer to actuality. We think that guidelines have the potential to provide a push to plan appropriately. It can help the researcher determine if they have enough capacity or enough money or enough resources to meet the guidelines for sharing. Guidelines also set clear expectations for both sides. Applicants and grantees will understand clearly what we want as a funder. And then once these plans are outlined in an application, us as a funder will understand clearly how the project team plans to share and manage the data sets. But it's time for me to get off my high horse. As a funder, we can come with a big picture and say all of this, but I'm not in day-to-day managing a pilot pile of data. At TWCF, we fund different projects from hypothesis testing research projects to projects building theoretical models and defining measurements. And we've heard that pre-registering qualitative research projects where sharing qualitative data sets may be different than working with quantitative data. We've heard a little bit about the challenges of sharing data and participant consent. How can these two requirements be met in tandem? We've listened to the concern about the challenge of finding the balance between the context and confidentiality. How can you share enough information about a qualitative data set so the user has enough context, but the identity of subjects is still protected? Hence, I'm going to be quiet now and really grateful to listen to the real experts in this area and this important topic and eager to hear about their experiences and advice. Thanks. Sorry, it took a minute for me to stop the remote that I gave it. Oh, there we go. So thank you for that introduction. I do have a lot of thoughts about this topic and this is my perspective as a researcher and only my perspective. I want to give you that caveat. I've been talking to a lot of my friends about this topic, though, and so I've been incorporating some of their feedback and their questions and their concerns about this into this talk today and something that I just want to start with is what even counts as data. I know Sebastian will be talking about what we do with some of these questions, so look forward to that. But some of the perspectives that I've heard is what do I even count as a qualitative researcher? So we might be working with interview participants. We might be thinking about focus groups. We might be doing observations. We also do a lot of background work as a researcher and qualitative work. So we take researcher memos. We talk about our own experience. We provide thick descriptions of where we're at when we're providing the focus groups, where we're observing. So we have both the participant side of things. What they actually say is that audio is that video. We have the researcher perspective too of what the researcher is adding because in qualitative, the researcher is a part of the study. They say that we're an instrument because our biases and our perspectives can absolutely influence the data that we collect and it's important not to say that we're getting rid of bias but thinking about it, thinking about how it might impact data collection, data analysis, modeling, all these sorts of things. And so when we're thinking about doing open data and open data sharing, what am I allowed to put online? Is it just stuff that I've collected and that I'm bringing to the table or am I also thinking about what participants are saying? Oh, and there's thousands more from this. I was trying to think of the most basic and the most common examples but there's a billion more. Other questions that come up in conversations with other researchers is what did we say to our participants whenever we started this? So we have obviously our IRB requires consent form language around this so did we include that before we started collecting data? Did we tell them what the topic really was or did we have some sort of deception involved that can be common? In qualitative research, more than quantitative, I've seen much more consideration and understanding of asking members about their feedback of the data. So they call that maybe member checking or validation. So thinking about that as another step of this, involving participants in the data sharing process and asking them what the topic is. Are they comfortable? I was talking to a colleague who was talking about their experiences as a faculty member in university and they knew that they were likely going to be outed if they even said their university or their gender. And so they did not want to be given any piece of their information online. They were very scared because they did not have tenure. And I think importantly from a researcher perspective, we want to tell them what the point of open data is. Is it because our committee said that we need to? Is it because the grant funders said that we need to? Is it to help create access for other communities? Personally, that's more of why I have done it in the past is to help other people who are interested in our types of research. Look at the interview questions. Look at what a code book looks like. There are many young researchers or early career researchers who may not have had experience in specific methods. And so for me, it's really important to help the next generation see what is possible in that area. And so all of these different rationals that we might have for sharing our data are important to our participants and we need to make sure that they know what that is so that they can make an informed decision about data sharing as well. And then obviously telling them what would be available. Is it to other people like me, the other Rachel's who are not concerned about what university you're at? Or is it to people who might have a malicious intent? It's very likely that in this day and age there's a lot of polarization and politicization over topics. And for many people, they are not willing to have that additional risk. But maybe they would if they knew it's only available on request. Maybe they would if it's not their video, but maybe it's the audio that's being shared. Or maybe it's not the question they're concerned about is not the one that you might actually be sharing. Maybe you're sharing the data about a description of their setting, but not necessarily their experiences with that setting. And so thinking about what participants might be comfortable with and what you're comfortable with also your knowledge and your why I think are really important here. I want to have a very quick example. This is a video that I pulled from TikTok, one of my favorite places to be whenever I'm unwinding. And this is online footage of a woman in an airport pretty recently within the past few months. And so as you start watching, I'd like you to think about if you were going to share this data, what might it look like? What options do you have in terms of communicating what you need to communicate about the essence of the video? Which options might be better for sharing with the public? Would it be transcript? Would it be video? Would it be audio? And then which ones might be best for private use? So I am at home enjoying a nice evening of scrolling social media when this comes across my timeline. So I am at home enjoying a nice evening of scrolling social media when this comes across my timeline. So thinking about this video, let's say you are researching, I don't know why you'd be researching TikTok videos, but maybe you're thinking of a dramatic example and an observation that you had. Right? If I'm the lady in the white sweater, do I want this shirt publicly? Most likely I do not. Do I want a transcript of the experience? Maybe I would be more comfortable with that. Do I want this TikTok commentary included in what is shared where a man is eating popcorn is possible entertainment around what may or may not be a very difficult event for this person. So thinking about what aspects that we share and in which capacity are very important for us to think about before we even bring it to the participants. Right? If we have participants who might be going through mental health challenges, political issues, relationship issues, I'm not saying that we can't share anything. Right? We can be cognizant and propose a solution that provides a feeling of safety and security for our participants, and it might require more work from us to add an additional layer of transcription, anonymity, confidentiality. But is that worth your rationale or your why? If it is to make sure that other people trust your study, that might be important to you, but maybe it's not worth the risk of your participants not feeling okay. So here are just a few examples that I thought just spark ideas and then pass it over to the expert who can tell us now what do we do with all of these potential problems. All right. So while I wait to get control over the slides, so there's essentially three main areas that we hear at QDR that people are concerned about when they think about sharing qualitative data. And the first one of that and that loomed large and entirely unsurprisingly, though very originally in how she presented it, but entirely unsurprisingly is the question of ethics. And there's questions exactly what Rachel brought up. It's about consent, it's about confidentiality, and it's about access. There's also the second large kind of group of concerns and questions that we hear came up perhaps most clearly in the first slide that Rachel presented. And you hear that especially from people who situate themselves more strongly in kind of a social constructivist tradition of qualitative research is broadly around context, or if you want to be a fancy about epistemology, to what extent can others understand the data or understand them in a way that's meaningful in the context of my research. And then thirdly, they're just practical questions, right, like how do I do this without it sucking up a prohibitively large amount of my time. And as I said at the beginning, I don't really want to give you solutions, solutions in the sense of that I think these are solved problems. I think these are good questions and I think people are right to ask them. I think there are approaches to engage with these questions and not say, okay, ethics, human participant qualitative data can't be shared with Don. So, we can do better than that, but it's important to take these questions seriously and to think hard about how we address them. And we're also going to address them differently for essentially every single research project, although otherwise it wouldn't make sense for me to talk to you about for 10 minutes. There are obviously some general ideas and things that we can do. So first of all, the big questions about ethics. First one is about consent, right? Do I have to get consent? That one has a clear answer. You have to get consent if you want to share data in almost all situations. We effectively, for example, won't take human participants' data, whether it's identified or de-identified, if you don't have consent from participants who are sharing it. Won't that deter participation, right? If you tell them I'll share your data, will they still want to be in your study? And even if they will, won't it change how they talk to you? Secondly, most IRBs in the U.S., ethics review boards and other places include promises of confidentiality. It's not always the case. There's a big debate, for example, in ethnography, whether that makes sense. But I would say 90% of the time we promise confidentiality to participants. How can I keep that promise when sharing data? And even if I de-identify the data, and I'm going to talk a little bit about how we might approach that, are they perhaps still too sensitive to that? Just publish them on, essentially, the wide internet. And does that prevent me from sharing the data? That's kind of a broad swath of the ethical question. There's more, but that kind of seemed to be the most important set for me. So the first thing I want to talk about is informed consent. And I want to start with an example. This is from a study that Alicia Fandavusa and Jen Miller at the Gutmacher Institute did. These were cognitive interviews on abortion. They interviewed about 50 people, 50 cis women to be precise, half of whom had had abortions, half hadn't, half of them in Wisconsin, half of them in New Jersey. And the interviews were designed to kind of get at conceptual questions of measurements. So they were working to refine the measurements that Gutmacher uses in their large abortion quantitative studies, but they're still fairly sensitive. And we worked together on this consent form, and then they worked internally with it and with their IRB. And what they came up with is that they mentioned as part of the consent language that if you agree, the transcript of your interview may be shared with researchers at other organizations. Note the language researchers at other organizations there. We will take out or change any information that could identify you before sharing. You can be in the study whether you agree to data sharing or not. And then a little further down, people gave their consent to be in the study, and then right after that was an opt-in. Do you agree to allow a written copy of your interview to be shared with other researchers in the future? So fairly specific about what would be shared too. Now, the cool thing that Alicia and Jen did, they actually rolled this larger study, which is also, I think, not published as a preprint. They rolled into that a study on informed consent that we published in qualitative health research last year. And so I'm going to invite you in the chat if open, I believe, to guess what percentage of participants knowing what you know now about the study opted into data sharing. I'm going to just give you a minute, just put a percentage guess in the chat. All right. First of all, no one has read my study yet. Go read the paper. But so we have estimates all over the place. I think I saw 10 to 100 percent. Vicky always the optimist. Hi, Vicky. And the actual number is 92 percent of participants opted in, right? And I think that is remarkable. And those that kind of there is a willingness of participants to have data shared even on fairly sensitive topics. They were allegorized asking whether, in the chat, whether they had to do two IRBs, they were able to wrap the entire thing into a single IRB application because the informed consent part of the study was fairly short and essentially no risk. So that didn't actually add up, add a lot of overhead. And you read the paper, there's actually a lot of some concern that they're raising how well people understand data sharing. So I'm just presenting you part of what they found here. And so I do think we need a lot more research on informed consent on data sharing. And I encourage you to do things like that and better understand if you're asking these types of questions, what participants understand, how they react, etc. This is just one study. It's not the definite final result on the question. So some considerations as you craft informed consent, be really transparent about data use. It's about informed consent after all. But remain intelligible, right? So Gudmaha, I think, has a rule of thumb that they want to be at a third grade, I want to say, reading level. That obviously depends on who you study and who your participants are, right? If you do elite interviews, you probably don't have to do that. If you interview people with very low education levels, you need to adjust accordingly. But you have to say what you're going to do. Don't try to skirt it. That's unethical. Don't go in there expecting that participants are not going to share. A lot of participants and other studies have found that in both Quant and Qual are very interested and willing to support science. Many, most of the, a lot of the time at least, that's why they're in your study in the first place, why they're volunteering or at least with a relatively limited amount of money we sometimes offer are in the study. In some cases, they may just trust you. In a lot of cases, they trust you as a scientist. Opt in consent as I just, so I think it's a really great option, especially for qualitative research, right? It prevents that risk that you have of people not wanting to be in the study because they have to share their data. So that's good for the validity of your study. It also is, I think, highly ethical to give your participants more agency over the research process and what happens with their data. So it's good in that sense. IRBs are familiar with that type of thing. It's fairly common in medical research. There's a reasonably large literature on this that typically refers to this as tiered consent. So with most IRBs, you won't run into problems. Sorry for the non-Americans for the IRB language, just replaced by ethics board or whatever it's called, where you are. What I would say is be careful in using this for quantitative studies because having people individually opt out of quantitative data set ruins your computational reproducibility of your data. So I've done this once before a survey and I've regretted it. But for qualitative data where we don't run computational reproducibility in the first place, I think it's really terrific in a lot of situations. Confidentiality, right? What we actually see is that it's exceedingly rare for interview or even audio data of qualitative studies to be shared. There are a couple of examples and they tend to be highly respected. ICPSR, for example, has a large classroom study with lots of videos and that's quite difficult to get access and approval to and only through their virtual enclave, etc. So it can be done, but most qualitative data that we have is in textual form, which makes it, of course, much more realistic to de-identify theories depending on the type of research you do, of course, a loss of richness there. If you de-identify qualitative data, I have a couple of general guidelines here. I'm just going to kind of let you read through this. I'm happy to answer more questions on that. One thing that I want to highlight though is that there really is a balance, right? There is no hard and fast rules. You have to do that. You have to do this. You have to kind of think about how much can I take away keeping the data as a valuable rich research and how much do I have to put away, how much do I have to take away for it in order to be de-identified. Thinking about this a little bit further, I often think of data as kind of a two-by-two spectrum between identifiability and risk. And those who don't necessarily go hand in hand, right? So identifiability is how easy it might be to identify people from the contextual information that is often invigorated qualitative transcripts. And the other part is how sensitive is the information. And so to go back to a study, for example, a qualitative study on abortion, given the debate in the US, it's quite sensitive, right? The risk is fairly high, especially for people with abortion histories. But since there are so many people with abortion histories, if you interview fairly randomly sampled people with abortion histories, it's relatively easy to de-identify that data properly. So in the case of that Goodmauher data, we had that kind of roughly qualified in that bottom-left quadrant where it's high risk data, but we are very, very confident about the de-identification of those data. And there are other cases where things do go hand in hand, right? If you do elite interviews with Chinese dissidents, it's highly sensitive and the identifiability is high. So at that point, you know, maybe consider this data that really needs sharing, right? Not all data can be reasonably shared and the safety of your participants is obviously way more important than sharing the data. There's also trade-offs within the data, right? And that's tricky for qualitative data if we think about reuse, because what you would do typically when you share the data is you may remove a little bit more about say you don't care about geography, so maybe you remove a little bit more geographic information, say you don't see the city where an interview was in, but then you can add maybe a little bit more about the profession someone has, something along those lines. And there's a trade-off and that's obviously not always ideal for reuse, because your user may care about something else, but that's the best you can do and just kind of be aware of that trade-off. You look as you de-identify to your data as a whole. And finally, de-identification works together with access controls. So for example, if I do interviews in a small town, people would recognize their neighbors, even if I remove a lot of information just, you know, by terms of phrases that they use, etc. But if I then add some level of access controls, maybe only for researchers or some other, you know, reasonable thing on top of the data, other researchers would not be able to re-identify that person. So access controls and de-identification can work together to maintain the confidentiality, assuming again that that's actually what you want and what your participants more importantly want. Talking about access controls, right, there's this phrase that gets cited over and over again by the European Union as open as possible, as close as necessary. We really at QDR very insistent about qualitative data doesn't need to be open data and the strict terms of the term, right, like open license, everyone can just go and download it. Talked about de-identification, if in some cases you may think about like a strongly de-identified public use data set, something more respected that may be at more risk of de-identification, but it's has significantly higher hurdle for access. A common thing that one can do is just put an embargo on data, sometimes just doing one or three years maybe enough to get people to get people out of risk, like if you're having, we have one data set, for example, that was on court diversion processes for sex workers and the data came in, I think, five or ten years after the study was done, so people had kind of long, most of them long cycles out and would have been much harder to identify them, so lower risk. There's access by application and that can be very simple, right, in some many cases for QDR data, we just check, are you who you say you are, do you have a reasonable plan, what you're going to do with the data, do you have a reasonable plan to keep your data safe while you're using it and then finally will you get rid of it once you're done with your study and you need to confirm that in writing, but that's a process we can turn around in 24 hours, other things are more complicated, if there is personal information still in the data, we'd require you to run your own IRB to get access and so on and so on and then finally there is access using virtual or physical enclaves that ICPSR offers, we only offer the physical version, which means you'd have to come travel to Syracuse, which is beautiful, but very few data sets up with that level of dedication to get access. Some other things kind of that that people have requested, we put into access conditions for data, most of the time we would manage the access to the data based on based on what depositors tell us, so we always have conversation with depositors and they're kind of in the driver's seat and we make recommendations, but they have the last word. We do for some studies offer depositor-approved access, I've seen Vicky is in the audience, so she deposited data together with a group where they interviewed people working in libraries and we're very concerned that there could be retribution, so they wanted to have, they wanted to know who request access to the data to make sure that those aren't safe, people link to their supervisors and so that's the case where we've wrote depositor-approved access into the access conditions. Then looking at the time, I probably want to wrap up in five minutes, the context is really important, it's always important when we share data, like if you just get a quantitative data set, you get survey data without questionnaires, etc, it's also not understandable. I would argue it's even more important for qualitative data because as Rachel says, there's so much going on there. Here are some of the things that we're suggesting. We always ask for descriptive documentation, right, with the project background, what are the methods used, how did you recruit your participants, etc. We also try to get as much of the materials used in that process, so questionnaires, focus groups, guides, and send scripts when people are willing to share them, recruitment flyers, online ads, etc. And if anything that gives you kind of that tangible feeling, okay, this is what actually happened, this is how people got into the research process, etc. And then there's method-specific documentation, that's one of the places where it's going to vary a lot how your data and documentation look depending on your methods, right. So that could be a sense of the reflexivity and positionality that went into the data that could very well be part of the documentation. It could be code books or coding that manuals, and many more things like, again, depends on the method. One thing that we highlight is, and that also kind of ties back to the opt-in consent, is be specific about what isn't included, right. What can you not include because it's, you didn't get approval from the participants, you didn't get approval from the IRB, you didn't personally feel comfortable with it. For example, field nodes in many ethnographic traditions are incredibly personal and are intended to be incredibly personal. And if we kind of told people upfront, you know, you have to share them, they wouldn't work the way they're intended to work. And that's perfectly fine. Again, you don't have to share every little bit, but it makes sense to say, okay, this exists in principle, but these bits of information are not there. I'll point out that a lot of good descriptive documentation is essentially long-form versions of what's required in many reporting guidelines such as CORAC. Those aren't without their justified criticism, but they're kind of good guidelines about the types of information that's useful to collect. And CORAC kind of works a little bit more checklisty than it would like, I would like it to be, but gives you kind of a good sense of what type of things that you'd be looking at. I also want to raise that even with all of this, that doesn't mean that data re-users necessarily will have the same understanding or let alone be able to, you know, reproduce your results, right? A, you will always know most about your study, B, you know, positionality specifically means that other people won't interpret the data exactly the same way. And that's fine. In one piece that I wrote together with my co-curators, we refer to this process of kind of adding documentation as enabling epistemically responsible reuse. So this idea that we put people in a situation where they can make an informed judgment about what they understand about the data and what they don't understand and can reuse it responsibly, you won't be able to do the same study, but you may still be able to do interesting work if you have enough context and also if you know what context or what additional information that was present for the original researcher is not in your study. And finally, practical considerations. I want to be very transparent, this does take time. If you have rich qualitative interviews and you have 50, 60, 70 interviews, the identifying them takes time and there is no, you know, there is ways to expedite it a little bit. People are developing kind of machine learning based tools, but even the most kind of optimistic of those tools say, you know, this is a human in the loop tool. So you have to accept or reject every option. You can make it easier, right? And I've seen a lot of data librarians in the audience. They're the type of people who you consult about, how can I build something like data sharing into my process? How can I do good data management, those sorts of things? So talk to your local data librarians. You can also come talk to us, but local resources are awesome. So things that could include is think about de-identification right during kind of the first pass to the data, kind of take some notes on that so you don't have to have an entirely separate read through all your transcripts for de-identification. Think about, you know, how am I going to share this as you manage and organize your files, kind of think through the entire process as you set up your original organization. So it's relatively straightforward to pull out the files that you're eventually going to share. Very importantly for all sorts of purposes, don't procrastinate on the documentation. Write things up as you do your research, both again for your own sake, but also because it will make the documentation easier to create in the end. And then importantly make sure that the various things that you're planning to do and promising various entities like your funder and your IRB actually match, right? We've seen less of this now, but we still occasionally see people promising one thing in a data management plan and another thing in an IRB. And that's obviously a bad situation. You have to go with the IRB because I think some transparency in that case, but it's a bad look if you made a promise to your funder and you can't comply with it because you made a different promise to your participants. And that's it for me and I saw lots of questions coming in. So I hope Mark is keeping them organized. Yeah, really briefly before we get into that, I have a quick commercial break and then I'd love to stop sharing my screen so that folks can focus on the panelists. On March 9th and 10th, we have the open scholarship practices and education research on conference again. Registration is open for this. It's a free virtual conference. So for folks that are interested in learning more about open science practices and the social sciences, but especially in education research, please take a look at the website and see if you'd like to join us for that. And with that, let's stop the share. And yes, Mark, if you could start us off with some questions. Yeah, so we ended up getting two main questions that were in the chat. First is in timed embargo, is it common to have the data sharing be timed differently from the time of publication or were the timed embargo included the time of publication? And I think that was directed to Sebastian. Yeah, so that really depends. And I think time, and I mean, time of publication, I assume that's the time of publication of the associated work. And so how an embargo looks like is that the metadata, right, is public. So you see, okay, there is this study, and it has these files, and the files have kind of a clear label, let's say, won't be accessible until January 2024, or something like that. That's how this typically works. Whether that's acceptable for a publication, most journals currently don't have super strict rules for staring qualitative research. So for most purposes, it probably would be okay. If you want to publish in cross, which has strict rules, it probably won't be. So that's kind of the rough guidance on that. Publishers, you'd have to talk to most publishes kind of want their data published sooner rather than later. But if you can make a good case, and it's right, like, you typically kind of say like 12 months is reasonable, if it's just, you know, for your own sake, do you want to get your publications out? If you want to go longer, it would probably have another reason, like an ethical reason why you can't bear it yet. Awesome. Thank you. We have another question from Agnes, which actually generated a little bit of conversation. Her question was the opt-in consent form asked about sharing the data with other researchers. But how do we know that other researchers would access, excuse me, that only other researchers would access the data? Couldn't anyone get access and then use the data against someone? Like, for example, if the participant shared negative perceptions of their employer, the employer can seek out the data. Katie Corker, who is one of our colleagues, responded that some repositories have a gating process so that other researchers must authenticate prior to access and use ICPSR, for example. I didn't know if you guys wanted to elaborate on that or give any thoughts. Yeah, so every repository handles this somewhat differently. But so if you specifically say other researchers, you should, that should, in my opinion, be reflected in how you share the data. So the data of that study are actually shared on QDR. And we have access conditions, access controls on those. So if you want access to the data, you need to essentially fill out a form, tell us how you're going to use the data, also sign a contract that binds you to destroying the data and not sharing it with anyone else, right? So you can't kind of distribute it on. And that still leaves a minimal risk, right? Like, if there is a really floppy or dishonest requester, it is a contract and not, you know, something that we can enforce beyond that. But so that's kind of what that promise is built on. There are additional technical means that you can take, but they're costly and high efforts, like the vertical enclave. And even there, you know, you can always take screen shots if you're malevolent. But that's kind of my general thinking on that. Your consent form could match what actually happens with the data. And so if you're planning on access controls, you can write kind of more restrictive use cases in there. And if you're planning to put it on a public access repository or make it public access through ICPSR or QDR, that would be reflected. Awesome. Thank you. And I think a follow-up question we had from that is, what are the panels used on share upon request as a mechanism for data sharing? I don't want to do all the talking. I'm sure other people have opinions on this. I'm not technically on the panel, but I can pop in as a mixed methods researcher and open science advocate and just say that I find that that is often less than ideal because as we've seen, a lot of folks that have been working in this area, even for just quantitative data, they find that researchers often don't respond to those requests or they start adding in, like if it's adding in, I guess, requirements. And those requirements might not be consistent across requesters depending on what their feel of why the person is asking for data. So I think if you're going to have a share upon request policy for your lab, you need to outline specifically under what conditions you're going to share the data and what your responsibilities are going to be. I will respond within five working days or whatever. I think that there needs to be a method for you to hold yourself accountable or for another person to hold you accountable for sharing that data because otherwise, it's way too easy to be like, I don't like the vibe of this person. I'm not sharing my data with them, but I'm going to share my data with this person that I know agrees with me or whatever. I think that that can get into really murky territory, which is why I like the idea of repositories where those access controls are set up and then the repository handles those requests. Personally, I don't know if others on the panel have different thoughts. I've had pretty similar negative responses where I've either been ghosted or the email was no longer valid in some instances because they changed institutions or what have you. So I think the idea to have in a repository and say what the situation will be when someone requests, I think is very strong. And I think coming from a researcher, when funding ends, I don't know how to take on that responsibility after funding ends to, let's say someone contacts me five years after still planning for that while you have the funding so that you don't maybe get too much of a workload on the back end when you don't have the capacity for it. Yeah, and I mean, just to add to that, even if you're the best intentioned researcher, we spend a lot of time thinking about how we make sure our data never gets lost. We have, I think, five copies of every data, three different regions, three different servers. So even if you have the best intentioned, keeping data usable and safe for 10 years, you know, stuff happens and you're no specialist in preventing stuff from happening to your data and you don't necessarily need to be for those kind of long terms that we want data. Be available, which is, again, where the repositories come in. Awesome, thank you. And then the last question we have, which was directed to Sebastian, but anyone's open to respond, could you summarize some of the criticisms of core queue, CREQ? I hope I said that right. It's correct. And those are the consensus requirements for evaluate and qualitative research or something like that. It's someone confined. If you look on the Equator Initiative, they have kind of reporting guidelines for all types of different research and COREQ is the most widely used for qualitative research specifically for focus groups and interviews. And it's like cited, I don't know, 50,000 times or so. And there's essentially two problems with this that are kind of increasingly voiced. One is that it's kind of muddled, which methodological perspective it represents. So some of the questions kind of, they claim to be methodologically neutral, but then you find things like saturation in there, which is specifically a concept from grounded theory, those sorts of things, people who are kind of more strongly constructivist again, like Victoria Brown of thematic analysis, same, that one has 50,000 citations, so I guess she wins. She is very down on that because these things kind of put this people into a more positivist kind of checklist approach to evaluating qualitative research than is reasonable. So that's the one set of concerns. And I think those are the more important concerns. There's also a recent article, and I'm sorry, I don't have the citation right here, but that right to replicate what they claimed they did in terms of fine. So how they arrived at criteria is essentially sort of systematic review, where they claim to search all sorts of criteria that people used and then use kind of a coding and collapsing process to come up with their 32 points. And they try to replicate the even just the search strategy, and that fell apart completely. And how to get from the outside, I didn't try to then replicate this, but was a fairly compelling criticism of the methodology of arriving at those 32 guidelines or checklist points in the first place. I still think it's really useful to kind of look through and see are there things that maybe I can include in my reporting. So I would mourn against using it as a peer review guide, but I would encourage you to have it in front of you when you're writing up focus group or interview based research. That's kind of my personal perspective on it. Awesome, thank you. And that was the last question that we had in the chat. Were there any questions that we have more panelists to each other that you guys come from different backgrounds? They're all back to you. I have a question. Just Sebastian, your talk and your presentation that was very informative, but it just makes me wonder if there's a guide that exists that qualitative researchers can use an access easier than maybe this webinar recording or is there any kind of support system that? Yeah, there's various options. And I know there's a bunch of expertise in the audience, so please put your favorite resources that I'm not thinking of right now in the chat. So one that we created is that website managing qualitative data that we created for the SSRC for their then still existing international dissertation research fellowships, doctoral students, and that essentially goes through the entire research process about managing data, but has a dedicated module on data sharing that has a lot of the things I talked about today covered with exercises and further resources, etc. So that's one kind of nicely comprehensive place to look. And then again, there are lots of people who want to help you with this. If you are at a university, you have a data librarian, they will want to help you with this. You can contact us at QDR. And so there are definitely help out there. Right, there's the resource that Rachel just shared that I think Samlify has worked on gotten about that. Awesome, thank you. So can you, Kristen? Yeah, awesome. Thank you panelists for coming today for sharing this. I always really enjoy webinars like this because I feel like even though I'm steeped in a lot of these conversations, I always learn something new. So thank you for sharing audience members for sharing papers and for the panelists for sharing your expertise. It's great to know that funders are paying attention to these conversations and that people are open to having talks about how people manage different kinds of data and the different kinds of concerns and questions and considerations that qualitative researchers have when trying to engage in open science practices. So thank you all and we really appreciate you guys for coming out today. This was an excellent audience. Thank you for all of your questions and a video will be provided on the YouTube channel for the Center for Open Science with a recording of this webinar. So if you missed parts of it, it's definitely here. You want to reference it back or share it with friends. Thank you all for coming. Thank you. Thank you everyone. Bye.