 to all three of you for what I thought were some really fascinating presentations. I'm gonna ask a couple of questions just to make sure that we get a little bit of LC dialogue going on in the particular sort of data collection and analysis category that is our, that it's our, it needs to be our focus. But then we'll also leave plenty of time to open it up because all three of your presentations are quite provocative and not necessarily just limited to data collection and analysis issues. And I suspect that there will be people in here who will wanna ask you all questions. So thank you all very much. I wanna start with a data collection question and think about sort of, I don't wanna make a presumption, but when I, in listening to all three of your presentations, there did seem to be a focus on the type of data collection that would occur even if it was done in a citizen science project, but in sort of more of a traditional research project, right? Where you have a question that you're trying to answer and you gather data in response to that question or if you have a database gathered already, such as a database of genomic data, you're asking a particular query of that data, but you've got sort of purposeful collection, active collection of that data. Is there a difference in terms of the data collection that occurs in that way and what I might call more passive collection of data, right? So it's data that's out there in the wild and it's not passive collection on the part of the scientist, but it might be passive collection on the part of the data subjects of the participants in terms of they may not necessarily know that you're collecting the data for that purpose and does that make a difference in what kind of difference does it make? So anybody wanna get started? Well, I think that one of the things that characterize at least genomics and genetics research and maybe a lot of contemporary research is that it is a hypothesis generating. It's not necessarily hypothesis testing, right? So we develop these big biobanks or data sets or we collect data not in the interest of a particular research question, but to just have this data at our disposal, right? And so I think that that is a really big shift and it really changes our relationship to participants, what we tell them we're doing, why we're doing it and for what larger purpose. Like can we tell them how useful it's gonna be? No, we can't really tell them and that is gonna be used kind of in perpetuity, right? That we're having it now. It might be used by various researchers over various venues, locales and over time. I would say the differences from my experience are number one, that the data collection is profoundly influenced by what the research question is and the difference in those environmental and occupational health studies where the people who are experiencing the problem are involved is that those research questions reflect their interests, concerns and needs rather in the case of occupational and environmental health predominantly the needs of the employer or the producer or the regulator. And so I think so in our particular examples related to industrial animal production there's been a lot of involvement of the exposed communities. We in order to design survey instruments or figure out what sorts of instruments could be used we needed tremendous collaboration to figure out what kind of language to use to elicit the information that we all agreed needed to be obtained or what sorts of instrumentation would be possible. And so I think those, that's where if the exposed population has an involvement especially when I don't live near fecal waste pits where the waste is being sprayed on my home. I don't know anything about that personally and neither does anyone else who works at the School of Public Health where I work. And so if we wanna get information about that it's helpful. In fact, it's almost a requirement to talk to people who experience it because otherwise we don't know what to look for. I mean we can surmise some things but the research process is very much affected by the involvement of the exposed population. I'll leave it at that. I mean the thing that I think there's good comments there the thing that strikes me about the difference between sort of purposeful and passive is the dynamic that that creates around control of the data and control over decisions about it. Is the collection passive or is the person passive that it's being collected about? That's more. Fair question. Well I think, so I think I was initially thinking of something that none of you raised in the presentations but that certainly has come up in terms of science and collection of data from people and examining it is the what can you get out of what you might just see publicly, right? What you could mine from Twitter feeds is certainly an area that people have looked at. What you can mine from social networking data to the extent people expose it to the public where you might be able to do an entire research project and the persons who use data you're collecting may never know what kind of sort of LC issues does that raise. But I think you all also, I mean I wanna get back to some of the points that you did raise which seem to sort of suggesting a thread of, you know well it's whether you have a hypothesis where you're collecting data from people or whether you've pre-collected data like in a genetics context and then the actual queries that you're putting forth from that data that there might, that there is some value to having input from the data subject about the questions that get asked. Not only that you're sort of putting them on notice that you're gonna be asking questions of your data but some input from them about that. So I'd like to sort of explore that a bit more. What are the sort of upsides and downsides to, for example, having the communities, the rural communities in North Carolina that you're studying not only to assist you in the data collection but to sort of tell you what kind of data you should be collecting in terms of their own experience. And similarly with respect to the, you know some of the citizen scientists that you spoke to in terms of sort of how they allow the participants to shape their research even if they themselves are not citizen science or vice versa. So certainly there are problems and we just, we have had these discussions at great lengths. So for example, it's well accepted in a study if one is measuring an exposure and trying to figure out whether the outcome is more common in the exposed or in the unexposed that if you have correlated measurement error. So for example, if you know you're exposed and you report more because you know you're exposed that that's a bias. So we had discussions with the community members about if this research is to be useful for you. For example, if you would wanna be able to go to court with it or if you would want it to be considered strongly by a regulatory agency that you might be petitioning to reduce the contamination levels by requiring pollution controls, you wouldn't wanna have your study biased in this way. So we would have to collaborate on how to protect. And so the community members, if they didn't already, I mean mostly they're very quick to understand those kinds of things. And then we could work together to try and figure out ways to protect the data collection from this bias. And I also wanna jump in on one thing but that Jason has brought up like who owns the data? Where did the data end up? Because the experience I described with my university wanting to turn all the data over to the industry that exposes people led us to have as a policy in the next series of studies we did was that all the records that we collected, we actually told the university at the beginning of the study that we were sending the records to our community partners for safekeeping because the university had proved itself that it was not trustworthy for keeping the data safe. I wanna know how that story ended, how you resolved it with the lawyer. Oh, how did the story end? Well, I did get a lawyer to represent me because I was given advice by other faculty and the IRB and so on to do whatever the university attorney told me but I just couldn't do it. So I was able to get a lawyer and we made a deal with the university that we would redact the documents. And I do believe that it is fair and part of science for another group for the industry should be able to hire epidemiologists to reanalyze our data and see if we made any mistakes in our calculations. That's totally fair but they should not be able to identify the individuals in the study nor in this case should they be able to identify the communities. And this is something where I feel a lot of IRBs are a little bit, have some growing to do because they may protect the individual participants but if those participants belong to organizations that can be targeted as communities by powerful institutions then we have a responsibility not only to the individuals who agree to participate in the study but to the communities that can be targeted. So we actually got the university to agree to redact the identities of the rural areas where the study was done and our argument to them was that if you know the participants was a woman age 33 with three kids and was on well water and worked at a nurse at the local hospital in a rural area you know who that person is. So it became an individual issue but I think the issue is broader because people are organizing to protect their communities and therefore have a larger interest that represents a collective interest not just the interest of the participants. So something that you mentioned in your response to that I think ties in nicely to Jason's presentation which is about returning the data or allowing the participants in the research to have that data and as Jason said to the same degree as they have the right to that data under HIPAA well those of us who work on HIPAA think that that right is not as strong as it should be but it certainly is right, right, right, right. Well yeah and he's certainly on paper but certainly a greater right than is the case in research data. So let's sort of explore what the ramifications are of that where again we're in the data collection and analysis category so I don't want to trump a later panel about post research so should people be getting this data while the research is going on? Like you could do that, that's your HIPAA right you can have this ongoing right to get data even as it's being collected. Is that what we're talking about or is it more of an after research sort of thing? I don't, I feel like there certainly are certain forms of research that would not be possible if the data were immediately available to everyone and so in terms of sort of if this were to become a policy I think there would have to be some amount of discretion or I don't want to, I think discretion is bad like I like the system of sharing by default and if you really don't want to share and you have a really good reason you have to go petition for reducing somebody else's sort of rights to access. On the other hand to throw on another thread I will pass this to the other panelists here but one of the thread that I'd like to revisit which is sort of part of this is with the current dynamic of a group of people being charged or maybe a research scientist and her ethical oversight boards what I see happening a lot is not a very good recognition that risk is heterogeneous and that people both in terms of really being heterogeneous risk in terms of what people experience in their own environments and then also their sort of willingness to take on risks is also heterogeneous so on sort of both sides and what I see happening in terms of data sharing and people's a person's ability to share data if that is held, if it's somebody else's job to hold up a hypothetical patient or participant and say should we give this data to them should we give this data to this person over here that isn't necessarily in sync with the entire cohort of people who are involved and it's somebody else's judgment and I think that's a really interesting problem and equal access is sort of one way to get around that problem as it does with medical records. It's not perfect, obviously there are problems and we have to have some tolerance for as a society of bad outcomes just as we do with medical records and people making bad decisions based on incomplete information or whatever but anyways, so I'm gonna stop there throw a few things onto the fire. You know, I certainly have been in many conversations where the idea of sort of returning data to participants is somewhat... Can I jump in on that? So in terms of terminology I think there's a really big difference between and particularly in a researcher or a clinician's mind really different of returning some sort of report versus providing access to data like should you want to go and like request and download your data it is available to you although it is not necessarily it's a very different thing for a researcher or a clinician to go generate some kind of personalized report and deliver that to the person and say here it is and I think that often like sort of the return of interpretive results versus access to the raw data as another researcher would are very different things and I think that's a nuance which is important in the discussion. Yeah, which way does that go? Equal access that not everybody... So what I hear most often and it is in the discussions of somebody's ability to get access to something data or report in a research project that it's too onerous for the researcher to become a personal counsel to the research participant and explain to them the interpretation of the data and the results and like delivering a clinical report and I do think that's too onerous but I think that a prerequisite is just access to raw data. And then a person can turn around and donate that to another research scientist another organization, make it a public resource pursue a first opinion or second opinion depending on what part of the research project you are. Wow, the audience is ready. I'm fine with that. Go right ahead. So, but let's say... Oh, Elaine Collier in Cats. And I assume this will come up in the next session but returning the interpretation of the results of the study is not what you're talking about. No, not the individual level. Right, and I think that's an important distinction. Yeah. The other question was to go back to your earlier thing about being able to go to somebody and say I don't have to release the results on an ongoing basis at this time. So who would you go to in your opinion and who would be the right decider in those circumstances and what would be the criteria? I actually didn't quite follow it so it was who would be the decider of what? So if you could say that the results don't have to be released as the study's ongoing because it may affect the integrity of the study, that might be one example, but who would you... Who would be the persons or the body? Well, in my ideal scenario, it would be federal law like where somebody has to make a decision somewhere that says there's a grace period. For HIPAA, after request, it's 30 days. And that's not going to work for everybody, but you have to make some decision somewhere. And there would be other people in actual the policy world who would probably be better suited than me to know how that federal law has been implemented. And we have a wonderful expert here who could probably, yeah. If I could just jump onto that question. Based on some experiences I've had at my university, I would ask about the context in which the information is being released and whether that could lead to harm to research participants. So for example, suppose a company funds research at a university which uses a screening test to look for some trait that would indicate the possibility that a person has a serious disease. And it was part of the protocol that in a study, perhaps thousands of people would be screened and they would be told, they would be given this information that they could have a serious disease. And the company that funds the study is knows when it will happen, they know where it will happen, and they can position themselves to sell a screening tests, to send their sales reps to all the local clinical offices where people might present. And they could potentially profit greatly from sales of either the tests or the treatments that would be delivered following positive tests or further diagnoses to this population that had participated in research. Now, we might say, actually, well, that's what we want because we want patients to have access to care and to treatment if they're sick. And that might be great if the tests were 100% specific and 100% sensitive. In other words, there were no false positives or false negatives. But there are lots of tests where most of the people who get a positive test are never gonna get the disease. And so I would ask if the researchers are being, especially if the researchers are being funded by an entity that stands to profit, is that an ethical problem that we should deal with that people could then, I mean, people would rightly be very fearful and scared and maybe even panic. And if they have no information about the prevalence of false positive tests or maybe they have no information about the occurrence of untoward side effects from screening and further follow-up, they could experience harm as a result of this study that wasn't intended, but actually that was also profitable to the funding organization. So what are our responsibilities? I mean, I like the idea of providing as much information as possible. However, if the information is unaccompanied by some education about what are false positives and how often they occur and the negative consequences that can occur to people who actually don't have, will never get the disease, but what might actually be harmed by invasive follow-up, I would really ask, are we creating harm and what's our do-no-harm obligation? So it seems that I think one of the issues is who gets, who is entitled to open access, right? I think, Jason, you were saying that the participant, him or herself. Equal access, open access. Equal access, right, so there's, I think there's a, it's interesting to think the participant, him or herself has the right to access to his or her own data, but not the data set, right? And that seems to be perhaps a different issue than the funder having access to the open access. And I wasn't, I'm assuming the funder does not have access to the data or the data set. The funder knows the study is being done. Right, well that already, that is already the case. So the funder knows the study is being done and is in any position to profit from the fact that participants are being given information that would scare them and that would bring them to a clinical setting where they would present for diagnosis and treatment. I think it, I think the question there is just the proximity of the profit, right? So either way, the funder's hoping to profit, right? Yes. And so is the question about the proximity of that, the participant profiting from the very participant's use of healthcare or just profiting off their participation? Profiting from the researchers conducting a study in which the patients will be informed that they have a trait that predicts disease. But it's interesting, I think. So what really is concerning is the idea that people would profit off of fear-mongering versus necessarily profiting off of research. And also off of the fact that, I would argue not just members of the public but many clinicians are not very appreciative of the common situation where tests have a lot of false positives. And this is true, for example, in cancer screening. Many of you are, I'm sure, more familiar with this literature than I am, where the majority of screening tests that are positive are not turning up people who are gonna get cancer. But they are still subjected to the psychological and sometimes the physical consequences of becoming a patient and so on. And are people informed? Are they able to make a decision that is gonna be the best for their health? All right, we got a lot of hands up. I saw yours first and then we'll keep going around the room. So before I ask my question, just a relevant case study that seems to come to mind is those full body scans. I don't know how ubiquitous those are anymore. People would get scan and see all these little tumors that would never have been impactful to their health and then they go and get surgery and have complications and that kind of thing. But the actual question has to do with sort of grounding the conversation in citizen science. So what aspect of what we're talking about right now pertains especially to citizen science? I understand to some extent we're trying to generalize from the way we do science and LC in the context of science to citizen science. And so I wonder what you think about that. And I also wonder if one possible answer to that has to do with not so much a qualitative difference but a quantitative difference, that the scale of participation, the scale at which each of us is becoming increasingly and I feel like it's a continuum part of the data producing machine, that our data is constantly being collected in one way or another, many times involuntarily we're being surveilled constantly for security reasons but those data can always be used to triangulate with other data that's collected for other reasons. So I'm just wondering if is what's different about this for citizen science that it's involving more and more people to some kind of universal asymptote or do you have other ideas about that? So I'm not even a panelist but one thing I will say is I actually see the connection between the data collection conversation and the data access and citizen science and the idea is if you are providing participants with access to data then they can either use it or donate it for science that they wanna see pursued versus just relying on the science that is being pursued whether you know the example in North Carolina or whether in Jason's different model so I don't, I get it and we should stay on citizen science but what is citizen science, right? Is it about who contributes the data? Is it about who formulates the hypotheses? Is it about who actually conducts the analysis? So and you may have had an earlier conversation that it's all of the above, right? Or it could be, I mean I missed earlier conversations so forgive me if I've tread on some territory you've already mined but I don't think it's disconnected but now I want the panelist to maybe add some. Right, so it sounds like what we're talking about or what Jason is advocating for is a way of incorporating some of the principles of so-called citizen science to science, right? The scientific research that we're conducting so I don't think we have to call what you're doing citizen science in any way to say that this is the way, is this inflecting all of science now? Is this the way that human science should take place that no matter what the context that there should be access to your own data that you've contributed? And I guess there is an assumption here that it's some way the science we're talking about is that you've given up something of yourself. Like it's that kind of human, something of your human has been given up whether even if it's social and behavioral or. I mean there's a threat of this that is interesting to me that we haven't really touched on is the exclusivity of access to data and who controls that access to data in a more sort of traditional model. And on the optimistic side of this conversation, sort of imagining, so one question is, how many, so we should do a survey in this room. Data collection, conform consent. How many people are currently enrolled and currently as of today, you go log in somewhere or call a principal investigator or send the IRB an email, are currently enrolled in one organized health research study? Okay, so one, two, three, four, five, six, seven, maybe seven in a room of 30. So I mean, optimistically I think that it would be great to live in a world where everybody is simultaneously enrolled and I don't know what the magical number is. 10 organized health research studies that everybody is sort of contributing. And I don't think that we're ever gonna arrive that world that if the only thing that a participant in a research study can demand is from the research enterprise is the sense of altruism towards advancing human progress or whatever, curing disease. And that if an individual is going to invest their tissues, their time, their money into research that they should try to invest appropriately and maybe not necessarily tie everything up in an exclusive relationship with a principal investigator that they hardly know. And if they do invest in tracking themselves over time their health and their health status and their environment, that this is a resource that has incredible societal and economic value and that only having a model where you can have a little bit of data with this research investigator and a little bit of data over there, you can never connect them and you never have any say in your ability to sort of accumulate and donate that data and make ongoing sharing decisions, seems like a broken model. And to me that's obvious. Like a personal data tax. What's that? A personal data tax. Yeah, yeah. I submit my personal data, that's a tax. Yeah. So that's a sort of provocative statement. So I see hands raising up. I'm gonna try, I had a little, a small cue in my head and I wanna clear that and then we've got lots of other people who've raised their hands. Dave. I guess I just wanna ask, I'm not sure that given an individual, her data makes her a citizen scientist. What I'm, so what I'm wondering is if you guys can think a little bit about or what you think about, I'm sure you can think about it. Will you please? What do you guys think about if you've got citizen scientists as your partners, who gets the data, who controls access to the data, who controls publication of the data? Steve, you said it's part of the process. There's the right to reanalyze. You've got 10 citizen science partners and it's your data, you think. Whose data is it? Yeah, I mean, is there, that's my, that's my. Well, I mean, what I learned from the experience I described was that they're definitely not my data. I, that was, that was clear. And I suspect that's true for most of us who work in institutions, that the product of our work belongs to our employer. And my employer is a government school. So it's subject to a public record statute that actually doesn't protect research participants. And there is a clear conflict between state law and the rules for conduct of human research that my university has to follow. I guess what I'm wondering is how can you be inclusive with your data sharing to the citizen scientists in your study while still protecting, and I'm talking about the aggregate data. I personally- While still protecting your participants. So aggregate data, I think yes. If they're totally de-identified, I mean, the kind of data that we redacted and turned over to the North Carolina Port Council, I felt that's fine. I can tell you that the people who we participated in basically they hired me to do this and they don't have any money. Actually, no, they didn't really hire me. They got funding from the National Institute for Environmental Health Sciences through Environmental Justice Grant so that they could be involved with this. And they identified the research that they wanted to be done in order to advance the interests of their community. They didn't have independent funding like a corporation does to hire a private research outfit to do independent analyses to try and undermine what we found. Now, I'm very pleased to report to you from a personal standpoint that it's been many years now since this Public Records Act request was filed and since our data set was turned over and there have been no letters to the editor saying that we made any mistakes in our calculations that come from the researchers hired by the companies. But the fact is that where the way it works now is that although the university owns the data, the principal investigators make the calls about how to share that information, the IRBs are involved. I do feel like my experience working with community members who are experiencing environmental and occupational health problems is they for the most part aren't interested in raw data because they don't have the resources to do something with it that they would find useful. And it's not really a big deal. Nor do they care about being co-authors on manuscripts or a whole lot of other totems of professionalism that are foreign to them even though they are precious to us. They care about their quality of life. They care about their dignity. They care about the self-determination of their communities. And that's why they want the research. And what they ultimately want is a partner to help them reach their goals. And they want partners that they can trust and so on. And we need partners we can trust. So I can trust my community partners to protect individually identified records because I can't trust my employer to do so. And so I think there's a little bit of reciprocity there that even a big powerful institution that has hundreds of millions of dollars in federally funded research can't do what a little community-based organization with a staff of two people can do. There was a question in the back. I don't, in the purple blouse, I think it's purple. I just wanted to get back to the original question about, I'm sorry, I'm Laura Cleveland, I'm a patient advocate. I'm from Columbus, Ohio area. So I wanted to get back to the point of the data, personal data, personal results. And the thing that I was thinking about in everyday life is say something like a pregnancy test. So we know that there are false positives and false negatives in pregnancy test. And let's say somebody goes out, decides they're gonna party hard for a weekend or something. They have a pregnancy test that says they weren't pregnant. They've done all these drugs, they've done alcohol, that kind of thing. And then you have a baby with issues and problems. So there are things that we do know. And I think in sharing a person's gene information or study information and then giving them information about what we know about this and what we don't know about this, those same kinds of things are listed on the instructions in that pregnancy test. Now maybe this is a simplistic example of that. And but I think there's something to be taken from that that we do know that that happens and we do have the decision to rely upon that test or not to. And I think some of the same information can be applied to studies. I mean, I think that research literacy is incredibly important. And I think that there is sort of a starting point for that is transparency. Return, the lawyer want to say something about that? Said not particularly. But it does occur to me that I'm not sure. You know, if you're getting a, are you talking, Jason, about providing people with the data, whether they ask for it or not? So if I had a pregnancy test done as a part of a research project and I never asked for it, okay, but okay. I must, I'm just getting confused. I'm confused as to what's specific to citizen science. It seems like we're dealing with a lot of issues that are in general to any science at all. And I guess I'd like to maybe change a little bit and ask the panel, maybe push a little bit more on why is this specific to citizen science. If I hear more and more control over, I don't want that person to have my information, et cetera. When you asked how many people are engaged in research, everyone should have raised their hand because like it or not, someone's pawned through your medical stuff and looking at your specimens. But instead, seven of us raised our hands. So a question I have with citizen science, is there gonna be a potential of increased control that will hence deep six an awful lot of research that goes on? And I think many of us look at research, not so much what's in it for me is for the general. It is a tax of being a citizen. A lot of public health issues. You know, you get bitten by a dog and get rabies. Sorry, you're getting reported. Yes, that's not research, but we don't ask permission for that. So a bit of my fear, maybe pushing this conversation a little bit, will citizen science and whatever it means, and I must admit I had a better idea earlier today than I have now, will it have an impact, an unintended consequence for general good research in terms of large populations? Are we gonna just bias those who say, oh, I hate oncologists, so I don't want any oncologists ever touching my data? Or, I mean the flip side will also make, not flip side, but another side. Jason, I'm a little bit worried on the open access so I can share it for other research projects. Equal access, okay. Its context is so important. How is that information gathered? Under what parameters? And to say, oh, here's my sodium. I wanna share my sodium level for this other study on renal failure, maybe totally meaningless and in fact harmful. So I guess that's like too many things. I'm just kinda getting frustrated as to what is specific to citizen science here and are there things we need to start looking at? And what are potential harms for research in general? So that might be too broad, but it's the end of the day and I need a cabernet. It better be good. I think, I tested it with you, you'd like it. So another thing maybe that goes along these lines is when we are having the citizen science project and let's say it involves genomic data. And in the standard system, it will be exceptionally careful with genomic data in terms of who's gonna get access to them, et cetera. But now the participants have, the citizen science project, they have access and they can take them into whatever they want with them including that they can post them somewhere publicly or sell them or whatever you want. Now, what about the information that one can infer about these people's relatives from that? In the standard system, we would have given some thought about that because of course we have those who protect this data or they keep them safe or whatever. How do we, is that a special issue? Is it a special issue here or is the same issue? So to be sure, I think it's a throwaway line to say if it was in the research community, like we would have thought of this. There are so many problems and no matter what model you have, it's not gonna be perfect, you're gonna miss things, you're gonna be accidents. Henry Lacks, hello. Researchers publishing the genome without the family's consent. This goes to, these are just sort of not issues that are unique to citizen science. And it goes back to, the reason why it's confusing is because it is at all of an interconnected web of dependencies and relationships between actors and processes and all of this. And I think we're in the part of sort of flushing out, getting into the weeds where things are just not clear and there are directions that we could go. But I think it's so easy. The thing that I'm reluctant to allow happen is to have so much anxiety about disturbing the current piece or trying something new out of having a zero risk tolerance or something like that, that we're still very much in the prototyping stage of citizen science and let's be clear about that. I mean, in my view, in that there is no sort of right answer and a well defined, this is how it works and how it should work, that's why we're here. And to explore and plumb those and then also look at how to refactor components of institutional, traditional, conventional research enterprise to accommodate those people who wish to play a more active role and contribute in more ways than a purely transactional, beneficent, Icelandic consent form given to a blood and that's the extent of my participation. Who have more to give. And so I think sort of at this point in history in some sense, you just have to be satisfied with a degree of fuzziness because that's where we are. And I hope that you all have a certain extent of excitement and tolerance for people who wanna try new models and do new things and fail. And maybe possibly we disrupt some aspect of traditional science and hopefully half of the people in this room are there to sort of track that and discuss it and talk about it and get involved. I mean, sort of like a living, breathing thing. And I know that this is sort of an end of the day statement. So that's just what it is. I think this goes back to Dave's question too. And it's just reiterating what you said, Jason, that we can't make any grand theories yet, right? There's no, like this is the way it's always going to work or the way it should work. And that science and science ethics is always contextual, right? Research ethics, so it'll be different for genomics and it will be for, you know, yeah, exactly. And so that we can create different standards for how much data we wanna give back or how much citizen might be a citizen might be involved and that there's some negotiation that's going to happen along the way and should still continue to happen. I don't think anybody in the room would ever want any blanket statements over what can and cannot happen. All right, I got a pair at this table here. I saw your hand first and then your colleague to your left can go next. You first, Sally. Who's left, are we talking about? That's okay, so I'm Sally from Patients Like Me. You know, I think I just want to throw out something that's a bit provocative and certainly we've published on and we are confronting all the time in the data collection and analysis part of it. So patients who are participating in trials who are entering their trial drug in the treatment evaluation process on patients like me under the code of the treatment. We ran into this with dexaprexamol with ALS. So we have patients who are actually putting in information about the side effects they're experiencing, their perceived efficacy of the drug they're on and then other people on the site who actually are also in the same trial who are now starting to talk to each other. We actually were transparent about this when we identified what was happening and really understood it more with the sponsor and said the day before their publication of their results, we went to them and said, you know, we have some results here that you might be interested in as well. Anybody interested in this has been published in a fig share document. But I mean, I think that gets out of the control of the researcher. It sort of says I as a participant have access to something now that actually engages me with other people who could potentially be in the study and we actually have the ability to be able to understand each other's experiences in ways that the researcher doesn't have any control over. So we're struggling with this and I think it's something that will continue. That is unique, I think, to citizen science and it's something that I think is gonna keep coming up. And Sally, these are not blinded trials then. They are blinded trials. So people don't know if they're on the active agent but they're reporting. That's exactly right. They're making some guesses. They're guessing and that's fair. And they're guessing because of some of the comparisons they're making with other people who have also identified themselves as being on that same trial. And did the funders find that threatening at all to their data? They founded a TAD, threatening? Yes. Yes, indeed, of course they did. However, I think that we also were very sensitive to it and we also were communicating with the patients themselves to help them understand the implications. But it is going to continue to be an issue. I think we have about 400 clinical trial drugs being evaluated on our system today. So it will continue and it will continue to proliferate but I think we are trying to understand what the implications are but at the same time educate the participants to the risks that they're taking by exposing too much. So I think what it raises is can we expose some without being taking things out of the blinding? Can we provide people a little bit of light so that we're not totally in the dark? Will that help them feel a little bit more engaged in the process but at the same time they are going to continue to do this sort of thing? We can't control that. I know that I'm not supposed to be asking the questions but just to be clear, this means possibly that someone on a placebo is reporting effects of their treatment and then subsequently would be told they were on placebo the whole time and everyone else would know that they are saying, oh, this drug is great or it's drug is terrible but everybody finds that they were on placebo or the participant only finds out which arm. It does raise some interesting possibilities. Oh, it totally does and we're certainly looking for insights into thoughts and raising it up in this community to say this is going to continue to be a question that will come up. We haven't come up with a solution necessarily. We're trying to navigate it, you know, sensitively with patients as well as with our, with people we know we work with regularly in the sponsor side too so. Sharon, so that person has to defer to you to let you chime in. It'll take me one second. That is already happening in traditional research and clinical trials already. Forget about citizen science, that's happening now. So I've been raising my hand for quite a while so I have three comments now. I had one in the beginning. So I don't think has anybody actually thrown up a definition of citizen science during this meeting because I have one from the federal community of practice and we developed this one by looking at the literature and two over the course of three or four months, the community bounced the definition back and forth and tweaked it. So I can't say I'd call this a working definition. It may still change over time, but citizen science is a form of open collaboration where members of the public participate in the scientific process and we were very deliberate process, not research because we're also thinking of folks who are doing operational environmental monitoring or health monitoring, et cetera, including identifying research questions, collecting and analyzing the data, interpreting the results, or problem solving. And to Steve Wing's comment, oh, so on that front, that implies an active participation, not a passive, I've got a Fitbit and somebody's off collecting my data or I'm posting to Twitter about something else and somebody's off collecting my data. I think that passive data collection falls more into the big data analytics than a collaborative citizen science process. To Steve Wing's point, I think your community privacy comment is spectacular. The participatory mapping community over the last 20 years has at times talked about a sense of community privacy, particularly in the context of data sovereignty. So when I'd worked with tribes, they don't necessarily hold the same open data perspective that our new millennial generation does and it's over generations of being, their resources extracted, et cetera. They also would add who controls access, who controls publication, who controls the interpretation. Lastly, on the disasters, sorry, on your open data front, you talked about the motivation of volunteers and that being disruptive, open data, open source, let's make it all open. So we saw the same dynamic in the disaster response in humanitarian aid community who were doing crowdsourcing and citizen-style projects and they were throwing this data up on Ushahidi, public websites and putting humanitarian workers and the people who were providing the information at risk and there were some incidences of actual human security. So going beyond privacy to human security and I would say your community privacy also gets at community security. So does anybody, it seems like that's a really helpful conceptualization of citizen science and I suspect that it's much broader than what people envision because I think when I first turned about this workshop, I'm like, I'm thinking citizen as the scientist, your definition arguably includes that but has a spectrum, yeah. And to interrogate that, you used the distinction between active involvement and passive involvement and with that then in the Venn diagram we saw in our first session this morning, would that distinguish the part of the crowdsourcing circle that is within citizen science from the part that's without the active versus passive participation? Crowdsourcing I think can be active or passive but you're right, it's those interlocking circles. For pointing, I just, I cannot see name tags from this distance. That's okay. Mildred Cho at Stanford and to get to this definitional issue, I think one could argue that if the data that are being collected only go back to the participants, that's not really new. That's just kind of traditional and maybe one way to think about the definitions is to think about the responsibilities first and then use those to lead into the categories of different types of citizen science which probably has lots of different bins within it because I was thinking about the example Steve had where in that case, it sounded like the communities that include the participants but probably also include non-participants or other people, they're not the exact same set were stewards of the data, right? So they actually had like the, they probably had the spreadsheet or whatever, right? Like they didn't, it wasn't just I have my own data but there was a community that had the whole data set they had access to everything which is very different and I think that is more kind of, that's more different than the traditional model but I think you could think about it in terms of not just who has access but who has responsibility? What are the duties and the responsibilities and use that to define what is this activity? Because I think if you have your own data that's a completely different thing than I've got the whole thing, right? Did you also have a question? We could probably just in about five minutes if we want to bring this to a close. Yep, yep, yep. I just wanted to make sure I wasn't misinterpreting that you didn't also have a question. Okay, go ahead, go ahead. Didn't know if I had been excommunicated for my earlier outburst. Oh no, no, you weren't. I just, I always like to get people who haven't had a question, you had a chance to ask. Absolutely, I'm just joking. Sort of. So just, and I love some of the more recent comments in terms of sort of distinguishing the aspects of this that are specific to citizen science but having sort of shared this frustration with Pearl about why am I having such a hard time sort of getting which part of this is citizen science and which part isn't. I had a couple of thoughts I thought I'd share. And the main thing is I think that there's an evolving context that's conflating citizen science with other moving targets that are moving just as quickly as citizen science. And so one of those that comes to mind is these are all things that we've already talked about but the ubiquity of data collection technology and that we're all quantifying ourselves, it's the fad. We have the bracelets, people are doing this. And another is what Leah just referred to as sort of this millennial culture, the culture of openness. And if I want to know what you ate five minutes ago, I just look at your Facebook page and at the same time I find out all kinds of things about your relationship I really didn't want to know about. And so that's part of the moving target and then we have citizen science and I think this definition is helpful. So one aspect that I think has been consistent in this discussion is this notion of quid pro quo that comes up more in citizen science than in traditional research contexts. So if I'm going to submit to traditional clinical trials, it's a more sort of abstract notion of making a difference I think that there's the sense that this is a drawn out process. I may or may not benefit from it. I may or may not be in the placebo group, but my fate is kind of sealed anyway and maybe this is the way I can contribute for the next person who comes around. Whereas in active citizen science and I'm talking about in the healthcare space, I get the sense that there's more of this aspect of, and this came up really in your talk and all those quotes that you had, the sense of empowerment, that every single quote, that was the sort of defining theme is there's something about my own advocacy of what questions are being asked that I'm harnessing this magical power of science and now that I've tasted this magic, what else can I do with it? What can I get from it? I mean, it's sort of the modern day magic. We can make big metal objects fly and we can see molecules. So for someone who doesn't understand what's going on inside the box and to begin to peek under the lid, this is very tantalizing. So I wonder if that's part of the quid pro quo, to be able to peek under that and start to actually have control over that. It's interesting that that's what's evolved over the day because I think coming into it, if I had said what is citizen science or what I thought everybody else thought citizen science, I would have thought it was more like the ornithology projects, the things where you're just contributing very altruistically for the good of science. So the fact that it's evolved now, that citizen science in this realm at least, the sort of orientation of it is about reciprocity. I think that that's, I'm not sure where to put that. I'm not sure if we want to say that that is necessarily one of the requirements of a citizen science project, that there is something reciprocal about it because I think there are plenty of things that are very famously labeled citizen science that don't have that element of reciprocity attached to it. I don't know, I don't disagree that that's what's emerged over the day and that that's maybe something that as an ELSI issue that we want to see addressed, but I said that's developed. Do I still have time? I still have hands. Just take one or two more comments. Yes, I will. We should bring this to a close. Yep, yep, I'm looking at my, assuming that clock is right, we literally have four minutes, so. Yes. Okay, so I want to take us back to some of the things that we have discussed during the day and part of it is one of the themes, I think that this conversation is illustrated is that there are many different types of citizen science that we're talking about and that it would be very important to categorize those at least in some way because they are very different. The type of work that Laura and I and many others have done where we've been fully integrated into the science before the reports and before publications and all of that is very different than what we've been talking about. But I also bring that up because there's a piece of this back on the, we also discussed how to orient people in the different categories of citizen science that we're talking about. And one of the very first things that, and this goes against what we've talked about, I'm gonna get to both points. One, first of all, for an individual that has data and equal access to that, I actually agree with that. Not everyone, very few people are going to do that, first of all. Second of all, one of the concepts of ethics in the past has been anonymized data and many of us have gotten up in those meetings and said, but what happens when we start to get information about all of these different genomic markers and all of these predictive markers that could happen and if people had access to their data, even if it was pieced, they could then pull that, take it somewhere and find it. They're gonna want to find out if they have certain indications, predispositions to certain diseases as we find this information. So that's on an individual level. That's not necessarily citizen science unless you use a super broad definition of the whole thing. On the part where we're actively involved as partners in the research in some way, one of the key factors is that in training or orienting people to understand more about research and research literacy is that one of the principles that some of us have operated from is that they are representing a group of people. They are not representing, it's like physician heal by self, it's survivor, for instance, heal by self. You're not there to represent your own situation. You're not there to think about your own situation. You are there to represent as large a group of people as you can, which means you have to stay connected to your community so that you understand what the issues are that people are facing today, not just the issues that you faced however long ago that was. And I think that's actually an important concept to keep in mind as we start to actually take action and put some of this down on paper and where we go from here, that's one of the examples of criteria that may be important. And I think there are equally important things on the research side of this as well. I haven't seen another one, I didn't. I'm gonna stop pressing that darn button. But I think that's a nice way to close. So I wanna thank my panelists for terrific presentations and a great discussion. Thank you all.