 Well, I want to thank you all for joining us this afternoon for the health policy and bioethics consortium and looking forward to today's discussion of research conducted with exceptions to informed consent bringing waivers. And so before we get started, a few housekeeping rules on the next slide. I want to remind you to submit any questions you have throughout the event using the Q&A function on your screen, not the chat feature. The chat feature is for any technical issues that might come up. You're welcome to follow us on social media. We use the hashtag policy ethics or HMS bioethics. And please do be sure to subscribe to the bioethics newsletter for any upcoming events and to know what's going to be coming next. So welcome to this consortia and the objectives for this series are to discuss and identify key issues in health care systems and public health, and especially those that are raising challenges for ethics in policy or in practice. And we bring together experts who have different perspectives on the issue to define the issue, understand some of the problems and start to think about what those solutions might be and we invite you all into this conversation with us and we hope that this sparks further conversation and discussion in your own work. So today's event is on emergency research coming up. We have an event in December that we'll be looking at the World Health Organization's essential medicines list. And we will start the new year in January with a discussion of clinical trial participants and participation. Thank you for joining us today. I am pleased to introduce today's moderator, William Feldman, who is a pulmonologist intensivist and health services researcher at Harvard Medical School and Brigham and women's hospital, where he has a joint appointment in the division of pharmacoepidemiology and pharmacoeconomics and the division of pulmonary and critical care medicine. His research focuses on drug pricing FDA regulation pharmaceutical policy and research ethics. And he shares the ethics committee at Brigham and women's hospital. He has completed his medical degree at the University of California, San Francisco, his doctorate and political philosophy at the University of Oxford, his MPH at the Harvard T.H. Chan School of Public Health, and his internal medicine residency and pulmonary critical care fellowship at Brigham and women's hospital. So we'll thank you for joining us today. I'm going to turn it over to you. Thank you so much Leah. I'm just going to share my screen. So thank you so much Leah and thank you all for being here today I'm so delighted that we have Carrie Sims and Neil Dickert with us I can't think of two better people to talk about these these really challenging issues with so thank you to both of them. I'm just going to talk very briefly about exceptions from informed consent, a little bit of background and then I'll introduce our guests for today. So I'll start with an example just to highlight the challenges of conducting certain types of research with consent so this was an exception from informed consent trial that was published in the England Journal in 2016. And it was looking at interventions for out of hospital cardiac arrest. So looking at at whether amiodarone lidocaine or placebo was was effective in the setting of out of hospital cardiac arrest. And then the basic protocol was the paramedics would get a call would get to the patient who suffered a cardiac arrest would immediately start chest compressions check the rhythm defibrillate if indicated and then give one of these interventions. If the defibrillation did not restore circulation so just to kind of highlight how how tight the time window was so the average time from call to the paramedics to delivery of one of the study drugs. 20 minutes, and that includes getting to the scene, starting chest compressions checking the rhythm, trying to defibrillate and then getting IV and then giving one of these these study men so very tight time window, and also to highlight how how challenging consent could be, because the patient of course by definition has no capacity the patient is is is incapacitated. And with this really tight time window, even if there was a surrogate decision maker standing right there. So much is happening that it'd be really hard to conduct this type of research with informed consent. In 1996 the FDA created this pathway and that's what we'll be talking about today, which is abbreviated with eFIC or exception from informed consent. Pre 1996, there was, there were trials like this happening but we're really going to focus today on kind of. After this has been codified over the last, you know, few decades, the, what these trials have looked like and what ethical issues have come up. A handful of requirements that I just want to go through so that we're all on the same page. So trials that are conducted with an exception from informed consent have to be for a life threatening condition where consent is infeasible because the patients can't have the time window for surrogate consent is too narrow. There's no way to identify the patients, respectively. There has to be this, this prospect of direct benefit to participants based on animal or other pre clinical studies. It must be the case that the research couldn't otherwise practically be carried out. So the investigators have to be committed to contacting surrogates before the intervention of possible and giving them a chance to consent and the IRB of course has to have have signed off on the consent plans and on all of this. So you can see how the trial for cardiac arrest that I mentioned before really satisfies these conditions and in the kind of paradigmatic way where the really narrow time window for intervention. Life threatening condition and I'd be hard to envision this research happening without an exception from informed consent. And then there are these additional protections that are also outlined in the code of federal regulations that are, I know our speakers will talk about a little bit that I want to emphasize because they're so unique and distinctive to this setting. So one is community consultation that researchers actually have to go out into the communities where these trials are being conducted beforehand and engage in a back and forth with representatives about the research that's being planned. There is also a separate requirement for public disclosure. So prior to trial initiation, the investigators have to disclose the risks and benefits in sort of a wide scale fashion so it has to be typically by radio by television on billboards, but engage in this kind of community disclosure to the community ahead of time. There's public disclosure of results afterwards that are required. You have to have an independent data safety monitoring committee. And then there's this requirement that's that's a little complicated to understand but basically when when consent isn't feasible, and there's no legally authorized representative that investigators are also committed to contacting a family member when possible. And then there's a therapeutic window to allow them a chance to opt out. So, what did these trials look like over the last few decades. This was a review that was published a couple of years ago but basically looking at 41 EFIC trials from from 1996 to 2017. About 40% were for cardiac arrest, but quite a number of other conditions that that were studied so so hemorrhagic shock traumatic brain injuries that is epileptic is schemic stroke, respiratory failure and acute coronary syndrome. About half the trials were conducted in the pre hospital setting and half in the hospital setting but I'll just highlight that the pre hospital trials are much much bigger so something like 80 90% of participants were enrolled in pre hospital pre hospital trials. And just to show on the right that about a quarter of the trials were studying devices, about half were studying drugs or fluids like vasopressin or hyper tonic saline or anti epileptics, and then a quarter or a little bit less than a quarter we're looking at different types of process So, there are a lot of ethical questions that I'm sure are coming to people's minds just even hearing about some of these requirements and I'll, I'll point out a few and the this list is by no means exhaustive. And, you know, I'm sure we'll hear from from our speakers about other kind of ethical worries that have creeped in but but one hard question is. How should community consultation guide trial investigators there's this requirement but what should trial investigators do with the consultation. Another is, is how far should investigators go and locating legally authorized representatives or family members prior to an intervention. I gave the example of cardiac arrest where the time window is very narrow but there are other conditions like stroke where you have not just minutes but maybe even a little bit longer. In the order of hours, though earlier intervention of course is helpful but not always seconds to minutes. What does it mean for consent to be impracticable in some cases you see trials conducted with an exception from informed consent at some sites and not others which raises the question is it, is it truly impracticable if it's actually being done at sites with consent. And then are these trials equitable so. Recent work has shown that black patients are disproportionately enrolled in trials granted exception from informed consent relative to population numbers in the US. And something we can talk more about but again just for challenging questions that put on the table but I want to hand things over now to our, our wonderful speakers. We've got Carrie Sims and Neil Dickert, I'm just going to introduce them let me stop my, my share. I'll introduce the two of them and then hand it over to carry so Carrie Sims is the director of the division of trauma critical care and burn at Ohio State University Medical Center and Professor of Surgery at Ohio State University Medical Center, where she holds the Olga, Jonas and professorship and surgery. She leaves a lab studying mitochondrial dysfunction in late stage hemorrhagic shock. She earned her medical degree from the University of California San Francisco her master her MPH from the University of California Berkeley PhD from the University of Pennsylvania. She did surgical internship and residency here in Boston at MGH and fellowships at Beth Israel at the Harvard Center for Mendelian based of surgery and at MGH and Penn. And then Neil Dickert is an associate professor of medicine in the division of cardiology at Emory University School of Medicine. He holds a secondary appointment in the Department of Epidemiology at the Rollins School of Public Health, and as a senior faculty at the Emory Center for ethics, and a member of the Emory Clinical Cardiovascular Research Institute. His research focuses on ethical issues relevant to cardiology practice and clinical research. Dr. Dickert completed his medical school training at Johns Hopkins and a PhD and health policy at the Bloomberg School of Public Health and did medicine residency at Johns Hopkins and cardiology fellowship in Emory. So, I'll hand it off to Kerry and just want to say thank you. We have two sort of preeminent folks in the field who think a lot about some of these other glitches and I'll hand it off to Kerry. Let's see if this is working. Hi, thanks so much for having me. And will that was a very lovely introduction thank you so much. So my task today is to talk about exception from informed consent and sort of really how this affects the community or how do you define the community. My objectives of my short section are to discuss the Epic guidelines and requirements, describe the challenges with defining what constitutes this community and propose some recommendations for community consultation and then hopefully this will inspire a really interesting and hopefully thought provoking discussion from the group. It's hard to imagine but the concept of informed consent is actually relatively new and really builds on the Belmont report which came out from the Tuskegee experiments in which they enrolled African American people, primarily men with syphilis and sort of just watched the progression of the disease, even when we knew that there were therapies for late stage syphilis. And there was a lot of outrage about this as there should have been. And so this instituted the Belmont group they put out this report and I think these the three things that came out of the Belmont report have really been guide points for how we look at informed consent, and, and actually even without informed consent, as long as these, these are held valuable. And that those were the respect for persons, really autonomy people have the right to choose what they do with their body or don't do with their body and when we don't have autonomy when that person has diminished autonomy, it is our obligation to actually minimize these individuals. The second principle is beneficence and really minimizing harm and maximizing benefits, and finally justice that that people who should share the burden of the research as well as the benefits of the research and it shouldn't be in just one group and you know, bearing all of the harm and none of the benefit. And so this is a slide, then in 1991 with the, the common role and this became the federal policy throughout the land if you were receiving federal dollars for you came from an organization that was federally funded. But it really has become like how we do all clinical research so even if you're not receiving federal dollars. This is really how we, it's the, it's the book of how we do it and it's all about consent and who and when and how. And it's really important to realize that people have the right to mull over the data, think about whether or not they want to participate, and then even go back to talk to their, their family physician, and then decide if they're going to participate in the research and we've made really amazing strides, particularly in the realm of cancer with this type of informed consent and have benefited many many people. The problem is that in my field that trauma is really unplanned and time sensitive. And so just as well I discussed that it's, it's challenging to get consent when there's really the life is on the line, there's not a lot of time, and there's a very short therapeutic window. And for me, you know, trauma is my life, it's the leading cause of death in the US for for young people those under age of 44, and most of those deaths occur secondary to hemorrhagic shock or brain injury. And we have a very time sensitive moment to actually intervene and have some meaningful impact called a golden hour so you would think that we would have lots of therapies that would be driving how we do with best practices and the truth of the matter is, we have very few proven therapies that have been investigated in any rigorous way in clinical trials to actually guide the way that we take care of trauma patients and much of what we do is the dirty little secret of trauma is that we really what we do is based on a lot of animal data, gut sense and sort of what has worked in the past, and that may not be the best way to actually improve lives. So, by definition, most of our patients are unable to provide consent, their LARs are not available. And so, how do we make, you know, how do we make advancements in trauma that benefit all of society as a whole while continuing to protect the individual patient. So, they recognize that this was an issue in the standard research makes it impossible for things like cardiac arrest stroke and trauma. And in 1996, they developed the exception from informed consent requirements for emergency research which we're going to talk about today a little bit. And just as we'll mention it, it really has very tight criteria for who can be involved in this kind of research. It has to be a life threatening condition. The available treatments have to be unproven or unsatisfactory. And there has to be a very narrow therapeutic window. Otherwise, your research doesn't qualify for this exception. And as we go through it, you can see that the people who created the exception from informed consent requirements really tried to adhere to those same Belmont report principles so that we could protect our patients. And so, consent's not not possible. So how do we protect autonomy for those folks. So patients are lacking their decisional capacity. And the intervention has to be done without the person being there or their LAR being there. And you can't identify those subjects prospectively so if you don't fit into this category you are not epic yet you can't really apply for epic. And then there has to be direct benefit to the patients themselves. So the life threatening condition needs some kind of treatment that is, you know, absolutely necessary we have to have very good animal and pre clinical study to indicate that there's going to be a benefit to that person, and the risks of therapy have to be reasonable compared to the standard therapy and the usual outcome. So, as mentioned there needs to be a consultation and that the consultation the wording is that it has to be representatives of the communities in which the research will be conducted, and from which the subjects will be drawn. There has to be a public disclosure, both pre and after the study and then there has to be a way for folks to say I don't want to participate in this research, and how to opt out. In general that usually ends up being these bands which have been shown to be pretty ineffective in terms of getting people to opt out. But if you really feel strongly that you're not going to participate then you can usually get an opt out band but there's really no great way that we've identified for people to say, I don't want to participate in epic. So, unfortunately, when you look at these guidelines, they're actually, they're not very standardized. And so IRBs there have or have variable interpretations of, you know, what does it mean to consult, who should we be consulting. How do we disclose both pre and post how do we have people opt out and so there's no there's no defined way of doing that and then fortunately, I think that has led to the minimization of the ethical imperative to adhere to these rules entirely. I'm going to throw that out there and, you know, hopefully that'll be a point of discussion as well. But there's no there's no real guidelines and unfortunately because there are no strict guidelines. I think that has actually led to a lot of the wiggle room that Dr Felvin was sort of talking about where you use epic and another institution is doing it without ethics so is it truly epic if you can get away with it. So when I, when thinking about the community and who speaks for the community. I found this study to be very interesting was like one of the first studies that looked at automatic defibrillation and they did a study in New York in which they got a bunch of focus groups and these high rises and they had these structured interviews, and they were really trying to figure out, you know, who, who is this community that we're supposed to be consulting. And what they found is that there is a lot of this this congruence between what people think as the community for the general community when you describe it as what is the community. It's not a demographic and geographic, but when you start to ask them about what their their community my community looks like you can see that there's it's very different. So the general community is marked out in the white bars and the views about my community are actually in the black bars and you can see that there's a lot of this congruence there. So we did a study at the University of Pennsylvania looking at vasopressin, whether or not you could give vasopressin and help decrease blood loss and trauma patients who are experiencing life threatening hemorrhage. And, you know, we wanted to do it in a way that honored our community thinking that we are primarily in a, in an under privileged environment. We want, there's a lot of stigma attached to research, particularly in the African American community. That's where our patient population comes from from from for the most part. And so we wanted to make sure that we were actually really honoring this concept of community and trying to figure out like, who is the trauma patients community. And so we set about to look at focus groups trauma patients and family members. We did some structured interviews to talk about the research that we were going to do when we started to tell this into our public disclosure concept. And then we also gave some surveys out. And what we found was that patients compared to the trauma the trauma patients compared to the focus groups or the community members, or the family members were less willing to support or participate in and that if they had the family members or the patients had experienced a violence that actually also negatively impacted their support of exception from informed consent. So variables that are actually really important for trauma patients for this particular study. We also said, I mentioned we had a bunch of questions about like exception from informed consent and would they want to, you know, would they be willing to etc. And so one of the interesting questions that we got was, if the family were not available. Who would be the best person to consent for you to participate in emergency research and we wanted to look at what the patients thought the family thought and what the community thought. And we got very, very different opinions and beliefs. So if you were the patient or the family, you were more likely to believe that the doctor should be the one who actually makes the decision about whether you should participate in emergency research. Or if it wasn't the doctor then nobody should do it because it wouldn't be right for someone other than my family to consent for me. So it was these very, very different opinions about whether or not I'm doing this kind of research would be reasonable with, you know, almost a third of the patients saying that the doctor should do it, and a third of the patients saying nobody should do it. This isn't right. If you looked at the community, however, they had a very low belief that the doctor or nobody should be consenting for exception from informed consent research. What they believed instead was that the religious leaders of the community should be the ones who would be giving consent on behalf of the patient. So very different beliefs very different ideas of who should be consenting if your family wasn't there. So, in another very interesting study. They looked at these 27 trials that were FDA approved, and they looked at the 42,000 different surveys that were available to review. And these folks, Dr. Feldt and Dr. Kelzel have who I honestly, I don't even think I looked at the authors until like, literally this morning, that these illustrious investigators are actually on this panel. They did the study so 42,000 surveys these were, and they looked at the difference between random digit that are random, random surveys versus convenience sampling. And what they found is that as you sort of went away from the community and got more personal, the, the acceptance of enrollment actually decreased in these emergency studies. So, you know, 86% of the folks thought that they that you could enroll anyone in the community and that would be okay 73% that they that approved of their own enrollment but as they got to their family. They were less likely to be supportive of it. And in fact, if you actually looked at whether they approved of enrollment without consent, almost a little over half of those did not believe that that was actually acceptable. So when they looked at things that might influence that acceptance male gender if you were had enrolled more males in your group you're less likely to have epic approval. And if you had more African Americans in your group they were less likely to support epic, suggesting both gender and race actually influence your support of exception from informed consent research. There's also a discordance between who was surveyed, and who was actually enrolled and I think Dr Feldman sort of alluded to that, that they, they surveyed a number of folks, but there was actually a disproportionate increase in enrollment of people. So for example they interviewed, they interviewed a certain amount of African Americans but that the percentage was actually much higher and those were actually enrolled in the studies. That's why it's not going down. There we go. So in another review, it goes to this issue of justice and whether or not the burden and the benefit of research is actually shared. And even whether or not exception from informed consent research actually benefits society at all. In the study, they looked at the 29 epic trials. They had nearly 47,000 patients. And what they found was that of those 47,000 patients, 96% were actually were enrolled without consent which is very high and suggest that that we really are relying on not consenting folks, rather than actually getting trying very attempting to get consent from family members. And then there was a 1% withdrawal again that seems very low in terms of, you know people participating in research. They found that of these 27 29 studies only four studies demonstrated any benefit. And if I were actually associated with increased harm. And of those groups, one third of the participants for African Americans, even though the US population of African Americans is roughly 14%. So again this imbalance of people being represented in these studies, when they're not actually represented by the US population. So is this actually adhering to the principle of justice. So what you know what can we do to try to improve the community consultation. There was a summit in 2011 of groups that actually are interested in emergency research and most recently there was another summit through the national trauma research group. The report is not out yet, but they were interested in trying to figure out like what do these groups believe and how can we actually do this better to define community. There we go. So what they found at the summit was that the definition of community is very multi dimensional. It's not a one size fits all definition. And that it's very specific to each site, and to these location, the type of study and the location so who is being studied what is the patient population, where is it being done and all of those factors need to go into how you consider community. They also felt that the segments of community that would be overrepresented or disproportionately enrolled should have a seat at the table and they should be specifically contacted with the consultation process. So, for example, in our patient of hemorrhagic shock and at the University of Pennsylvania, where penetrating trauma, we were going to enroll. Most of the patients ended up being from penetrating trauma. Most of the patients ended up being African American that were young that we needed to absolutely over represent our community consultation and include those folks into the discussion. And importantly, this should really be a discussion. It should be an open ended two way conversation with the study. The senior staff or the study PI, and that the community should have the opportunity to give their opinions, ask questions and influence the design of the trial. And they felt that the mean the methods that met this requirement for two way communication would be focus groups meetings within the community with leaders and representatives although I have to say in our study. The representatives of the community did not match what the patients in their family said, and that they recommended in person interviews. These methods, which are often use, including random digit dialing surveys that go out to a large group of people. And more, you know, there's even some studies that have actually looked at social media as a way of interacting with the community to get quote unquote community consultation. And those may in fact, distribute information to the community or the people who may be involved. It's not really a consultative process, and that these methods should be used as adjuncts rather than the sole method of consultation. So happy to entertain any questions and thank you again for the opportunity. Thank you very much Gary and thanks to folks who are who are putting questions in the Q&A. I am getting them and we'll I'll turn it over to Dr. to Neil Digger next, but maybe we'll we'll sort of take questions at all at the end that sounds good with you with you guys. Thank you very much for for having me, and thanks to to carry for a great first talk, and will for a really nice introduction to set the stage for this. Sorry. By way of disclosures I have research funding, much of which relates to this content area and a consulting arrangement without be a man related to an ethic trial. What I'm going to do in this talk is talk a little bit about sort of where, where have we come, very very briefly, one slide worth, and a little bit more than about where we are going and what we don't know yet about ethic. As, as you all have heard, we have now 26 years of experience with the effect regulations as well mentioned there was some time beforehand but really, that's when 96 was when these regulations were were passed specifically, setting out the parameters for even more specifically, requiring community engagement as Kerry pointed out we do have an established but I would say pretty significantly evolving set of tools for community engagement, and a significant number of ethic trials that have been conducted, as well highlighted, and so sort of very similar data here, you can see that the, the, the trials that have been done in the effect space have really heavily concentrated in cardiac arrest hemorrhagic shock and traumatic brain injury. I'll go through what I think are some really interesting challenges as we think about situations like stroke respiratory failure and acute coronary syndrome as one of the, one of the interesting challenges we're facing in towards the later end of my talk. So, what I think in structuring this for this session I thought what might be most interesting was to think about what I what what what are the real challenges and questions that remain and I would add, I think the equity piece is a very interesting one I didn't focus on it in this talk but I hope we'll have a chance to talk a little bit about that in the context of a discussion because I actually think it's a it's a really interesting and important topic but what I wanted to focus on here are what three challenges I think are really important one is how to make community consultation public disclosure both meaningful and efficient and Kerry got at some of this in her talk and I'll talk about some, some supplementary issues in that space. A second has to do with defining the scope of the effect regulations and particularly studies that involve a mix of populations who can and can't be involved prospectively. And then a last is something I think that often doesn't get talked about. But really is how best to respect people who are enrolled in EFIC trials, and that's really built on what I think is a really important notion that respecting persons goes much beyond autonomy and thinking about how we execute that is an important area for So, so the community consultation I you know this is a pretty really low budget graph I made a long time ago that that I still think is helpful and and Kerry alluded to some of this but but it's a really important issue I think when we think about community consultation. There can be a tension between depth and breath, and that gets to really important issues related to representativeness as well as whether we've really had a chance to hear from people and what's intended to be a two way interchange. These are absolutely not mathematical locate mathematical notions here but but I think when you look at the different tools that exist. You can have a really good potential to reach a large portion of the community. Others will be small, but in the in being small they also offer opportunities for having a much more substantive discussion and the kinds of information you tend to get are going to be very different in those different kinds of encounters and I think, thinking carefully about what we're trying to get out of community consultation is really really important. It's very easy to focus on numbers. There are both acceptance numbers and demographic numbers and other kinds of numbers and lose sight of whether you're engaging in substantive discussion. The social media space I think is actually a really interesting one in this way. I put that on if you can make a big circle. That's one where I think, you know, social media has so many different potential ways to engage people and we're just beginning I think to scratch the surface of ways to use that. A graph that is a little bit related to to the something that was shown earlier. It's adapted from a table in a paper that we published looking at acceptance. And I'm going to get into a couple of what I think are sort of nuanced issues that carry also raised. When you look at the surveys that are published related to community consultation there's a sort of trend basically between the mid 60s and the low 80s. In terms of the percent of people who tend to agree with it being okay to be enrolled in this study if they were in that situation so usually these are phrase related to personal acceptance and you can see there is a, there's a range but there's a very clear clustering in that sort of space, and that tends to get repeated over and over and over again. But what I think is interesting is, especially when people are focusing on on acceptance, it does matter as Kerry suggested and showed the paper from the work that will and colleagues had published. These are different ways of asking questions so this is just from a study we did that was embedded within a large traumatic brain injury trial. And I think it's helpful to put a little meat on the bones of what it means to ask people these different kinds of questions. So one is, you know, would you be okay being enrolled enrolled in this study without consent that's what we call in the papers that we've published in this space personal acceptance. A second one is about you know is it okay to include patients in the protect study without consent so that's a general thing about other people. And then there's a you know if you look above that. So that's the test is medication and traumatic brain injury patient so that's just a general question in, you know, a very general one. So depending on how you phrase the question and anyone who does anything related to survey work. Depending on how you phrase the question you get different answers. And I think that really matters and we try to put things together and and you know, people who are clinical trial list in the EFIC space or generally not necessarily survey experts. So, so it's a basket of lots of different ways to ask questions and make it a bit of a hard, a hard nut to crack in terms of what to expect. This is the piece that again, Kerry showed this slide from from Will's review of the FDA docket data. And I think it's important to recognize that these really are asking fundamentally different questions about general acceptability about family member and isn't even with without with or without consent it's these these are very different kinds of questions and people often in the context of doing this may or may not be real precise about what they're really getting at. So, I think it raises really interesting questions in part, because we don't always know what to do with numbers that you get right what does it mean that 68% of people said okay that's still, you know, 32 people, 32% of people who didn't say okay. And then at times you get, you know, studies that show 92% of people say okay and we all know they're more than 80% of people who might not say that. How, how to interpret this what kinds of thresholds ought to be used I think raises real questions, but more importantly I think it raises important questions about whether the goal of community consultation is really a quantitative representation of acceptance. There are numerous other uses that may be that community consultation can be put toward one of which, for example is designed helping to design or frame communication or consent materials really thinking about how you're going to execute the study particularly if you're an interactive discussion. And I think we need to think more broadly than just it's not community consent it's not a vote. It is really substantively engaging people about an activity that you think is important to conduct. The second piece that's, you know, with regard to sort of subtlety aspects of community consultation is thinking about context specificity, and the extent to which the requirements for engagement really are impacted by things like the level of risk the extent to which the interventions, depart from normal care or not. These are a lot of really subtle aspects that I think can go into how extensive we want to be, or need to be in community consultation efforts but there's a lot of room in the regulatory space for that. And I think IRB struggle a lot with regard to how to address those issues. One of the things that's coming that's happening now that I think is actually quite promising and interesting is this phenomenon a single IRB oversight of locally conducted activities. Community consultation always will be done at a local level in some way. You're supposed to talk to the people in the geographic area where you're going to be doing the study. But when you're a central IRB and you're seeing, you know, 45 different sites you actually gain really interesting insights across sites. You see which ones actually represent aberrations that might warrant further work. You see the ability to contextualize findings that a local place doesn't. And you also don't have many local conventions or just that their local conventions because that's what they've gotten used to doing. And single IRB offers a way to be a bit more potentially standardized if that's appropriate the flip side is that single IRB overseeing locally conducted activities may or may not have as much of a finger on the pulse of the local community. So with regard to the public disclosure piece right that's the other arm of the community engagement and that's explicitly intended to be more of a sort of one way of communication right that's about notifying people of what's going to happen. This I think there's a lot of questions about number one there's no regulatory basis for determining sufficiency we struggle with that with regard to community consultation but I think it's even more nebulous with regard to public disclosure community consultation we're basically trying to get a good sense of what we think the key stakeholders think for public disclosure there's no there's no real conceptual basis even beyond thinking that people have put forth a good faith effort. We know that in the best of circumstances it's almost impossible that that any really appreciable portion of a community is ever going to be aware of an individual trial being conducted in their area. So figuring out how to judge sufficiency is very difficult. Sometimes people get confused about the distinctions between community consultation and public disclosure. Some efforts can do can serve both functions carry mentioned the fact that large surveys, for example, will in will end up notifying a large number of people and that's certainly true that that efforts can serve both functions but thinking about them separately and recognizing what they do and don't do is really important. I personally think that the impact of public disclosure practices is very hard to measure and and and really, you know this this idea makes sense and we need to be transparent. But what exactly we're getting with the public disclosure activities I think is important to assess. So I want to switch now to an issue that that that I think has been alluded to a little bit. And this is about defining the scope of ethics this is what I what I really think is a second big challenge related to research and one that often hasn't been recognized right so this is. I like the picture of the study that will present it like the Alp study was looking at, you know what drug is being given in the context of cardiac arrest. And, and, you know, obviously that the guy with the paddles is not going to consent the person that that is actively in cardiac arrest. And this guy is a little different right so this is a guy that you know I'm a cardiologist so I have to show a picture of someone having chest pain it's like part of what I'm supposed to do. But this is these are the kind of people that I take care of who are having a heart attack and, you know, in will or carry space and acute respiratory failure or trauma we also see patients like this who are in distress. They might talk to us and we might have a very time sensitive intervention that needs to get delivered. And some patients may look more like him and some patients may be more or less responsive. So, so there's a wide range of people with acute presentations and a wide range of interventions that have different kinds of time pressures people have different kinds of symptoms that could get away or get in the way of their being involved. There's a high level of decision there's high levels of stress different conditions have variable levels of neurologic impairment and obviously the lack of familiarity with research we know is pervasive. The other thing that I think is really important to recognize is surrogates in acute care situations face a lot of the same barriers. So there really are I think a wide range of situations where we face that are high stakes situations and we have acutely ill patients that need immediate treatment. And I gave an example of some of these things but it includes septic shock acute MI cardiogenic shock trauma. And these can span context from the pre hospital environment to the emergency department to ICUs and ORs and cath labs, as will mentioned, these are. There's a there's a variety of settings a variety of time sensitivity, and in different kinds of milieus in which studies that plausibly fall under ethic are going to be conducted. And I think it's important as we're as we're seeing you know this clustering of work in the cardiac arrest traumatic brain injury and major hem particularly hemorrhagic trauma have been the focus of a lot of this work but it really applies to a very broad range of clinical situations. So the evict regulations, interestingly, do have something to say about this and it's important, but there's a lot that they don't say so what the regulations do say about about this issue is that we are supposed to get consent we're required to get consent in capacity or an LAR is available and capacitated. And interestingly, there's a requirement that gets doesn't get much airplay but I think it's really interesting that we're supposed to offer an opportunity to object to enrollment. In cases where consent isn't possible but someone's available so it's like an opportunity to say no, even if it can, a real consent process can't take place. And the third piece that that's been described by my colleagues is that the study obviously can't be practically done when enrolling only subjects who can provide consent or who have an LAR we could do this. So, so what's left kind of unstated is what this threshold should be for practicability. What does it mean that you can't do the study only enrolling subjects who can provide consent how many people have to not be able to provide consent for the study then to either not be doable from a time perspective or not be doable from a scientific perspective where you can't generalize to the population that you want to be able to generalize to those, those boundaries are not well defined. So a lot of questions I think about what it means to communicate with people in a way that's that I would say is short of consent but where you offer people an opportunity to object. So, so in terms of thinking about this, I think, you know, we don't, we have this notion of what a full informed consent process is supposed to look like but we don't have a great notion of what a sort of partial involvement or opportunity to object should look like. We don't have really fully worked out criteria about who ought to be offered an opportunity to object and that can raise interesting issues about a friend that's there or, you know, somebody who may not be a formal formal LAR. And we don't really know much about what patients or surrogates actually want to happen in these kinds of situations and as a as a cardiologist who takes care of these patients interestingly one of the things that motivated me to do some empirical work in this space is I thought that must be a terrible thing to be in a situation where you're not really you can't really be told much about something, and you have to make a decision about an unfamiliar activity like research. So, so we actually asked people, not in the context of an EFIC trial but in the context of stroke and MI trials. So, where people had to be enrolled very quickly and and some people would argue EFIC may actually be the right paradigm. We were curious, and I'll just say, I didn't show it in this slide but many people enrolled in these trials this was a little while afterwards, had very little idea of what they actually were enrolled in that's just, I think an important thing for us to recognize and acute settings people have a hard time understanding complicated information. What I think was really interesting though is that most people in both stroke and MI trials and in the stroke case this was usually a surrogate who was asked to make a decision. Most people were glad they were asked before a patient was included. Not that many people were angry about having to sign a form. And most people didn't think that they would have preferred if the doctor treating them had made the decision for me. We generally did a study that was using a general public sample that was built on a trial that was conducted in the UK that was that actually did enroll people under under their version of EFIC and this was patients coming in with acute myocardial or just being randomized to one of two blood thinners basically and ask them, you can sort of split this into group A and group B about whether they wanted to be give written consent versus being notified after enrollments or sort of an EFIC kind of set up or in the second group they were asked would they want written consent or sort of a brief verbal consent and without dissecting this too much I think what you can see is people were kind of split on the notion of of written consent versus a notification after enrollment. And in general people prefer to sort of brief focus process versus written consent so so I think these these and other data this is just two examples of studies I think that suggests that we should and the regulation support doing this we need to figure out how to involve people even in situations where we know they won't have full understanding. And I think that's actually really promising I started with this notion of thinking that people would be really angry at that idea they'd be asked about something they couldn't possibly understand. I think people appreciate being asked and take it as a sign of respect that they're asked, even if their engagement in the full decision as is because of the circumstances not possible. So the last piece that I want to focus on is this notion of respecting enrollees because I think there's a lot about that's there about acceptance of the EFIC trials from the kind of community consultation perspective which are by definition not people who have been enrolled in the trial. We have gathered some data on looking at the differences between people who have experience with the disease and people who don't. And I'd be happy to talk about some of those data in the discussion. But what I think I want to focus on here are really the experiences of actually being in an EFIC trial. I'm using how little data we have with that. I'll share some of that in just a second. But learning what we can do to make that better and to help people feel respected in a process where we don't have opportunities to enroll them, prospectively, I think is really important. So one of the first studies I did when I was on faculty and actually started as a fellow was in to embed some work within ongoing EFIC trials and actually talk to people who were enrolled in this. These are just a snapshot of some data that came from the protect three trial which was a trial of traumatic brain injury was a placebo control trial of progesterone in this space. And you can see so so just looking at the core questions the first one asked do you think it was okay for researchers to include me or my family member in the protect three research study without asking for permission first, and you get 77% of people saying and then we asked this more general question of do you think it was okay for researchers to include people in the protect three study without asking them for permission for more of a gen kind of a general questions not personal. In this particular case there wasn't that big a difference, but in the print in the community consultation questions there often are. Importantly when you don't sort of really focus on the consent piece most people were pretty accepting of having been enrolled in the trial what I think it's really important from this is it shows that, at least with regard to people's reactions post hoc. One, the level of the number of people who were upset about that obviously we can't talk to everybody in this study, the number of people who are really upset is is not super large, but it's there. And the numbers of people who say okay to having been enrolled is actually quite similar to what we see in community consultation efforts. And this is another study we did that was a much more quantitatively representative study within an ESET within the ESET trial, which was a trial of status epilepticus and everyone who essentially was in that study had had the opportunity to fill out this survey. And this was asking I think importantly different questions so one, just was it okay to include them in the study 77% of said yes and I think what really is key is the number who disagreed more so than the number who were neutral so 13% of this population were sort of not excited about having been included. And then when you when you sort of focus in or hone in on the notion of asking permission first obviously those numbers shift a little bit so I, so I think that's an important thing to think about again from a from a methodologic perspective which one matters the most my own view is what matters the most is is the first question of whether people were really upset by it. More than when we sort of specifically hone in on questions about consent that's a that's an important method logic issue I think. So, so I think knowing that this happens and knowing sort of what that there are a number of people who have concerns and then there are a lot of people who may not. I think it's important that we need to do we as a field kind of turn our efforts into what we can do to enhance the extent to which people feel respected. Right now, almost none of the scholarship in this space is ever focused on how to optimize communications afterwards beyond things like the fact that we have to get consent for continued participation or data use. No one's really studied how best to approach a family member and tell them that they've been or a patient and tell that they've been included in an effect trial. We don't know what they want in terms of subsequent information about the study how that should be presented in ways that help them to understand it. And I would say also help them to understand the value of what of what happened. I think one of the promising avenues for community consultation is actually to help develop some of these strategies, one of the criticisms of community consultation is that it's almost never resulted in any study. Not being done right they generally go forward with a set of data about how many people say yes. Okay, but but it usually doesn't alter the way a studies done. I think if we think about using community consultation to help to enhance communication strategies and treat it as a genuine form of engagement about how to do the study. The best way you can. We actually might learn a little bit more and have some things that would be more directly implementable. So where do we stand. Hopefully we can talk about that in the discussion I think we have a significant accumulated experience conducting these trials. I think there's a lot that we've learned in a lot of ways that they can, that they can happen very effectively. I think there are important challenges related to the scope of ethic and how to approach perspective involvement I didn't talk about this specifically and I should mention it many of the studies that are done so so I know that the, you know, Kerry described that the piece from the FDA docket data that that will and colleagues reviewed. A lot of those did involve a large portion of people were enrolled without perspective consent but there are a number of trials where where a substantial portion of people are enrolled with consent so a mixed, a mixed pattern that is not about sites but is actually just about a mixed population. I think is the nor is maybe not the norm in some conditions but is in many conditions and is going to become a more common thing for people to think about. We need to have clarity I think regarding the goals and best strategies for both community consultation and public disclosure and I think there's a lot of spaces in which that hasn't always been done in ways that are particularly efficient or meaningful. And I think we really do need to shift our turn our eyes towards what it is to to really try to enhance respect for individuals who are enrolled I think in general you know that I was glad to see Kerry mentioned the principle of respect for persons. And I think a more holistic understanding of what it is to respect people and we've done some empirical work on this to involves a lot more than just whether they can say yes or no, and thinking about how to interact meaningful and meaningfully and respectfully is really important. So I just, this just lists a lot of our collaborators we've done a number of projects in the space over the years. And it's been a pleasure to work on on that with them. Thank you so much, Neil and Kerry just really terrific talks highlighting so many difficult issues and we have a ton of folks on this call and we have gotten a ton of very very challenging questions so I'm going to try to synthesize these as best we can and sort of move through them in an orderly way. But a lot of these are really really hard questions that I think we all scratch our heads over. So the first set of questions that I'll put to both of you has to do with kind of definitions and terminology. So one person asked specifically about this distinction between community consent versus community consultation and if you all could speak more about that if community consent is ever sort of inappropriate way to think about this, or how you distinguish the two. A second definitional point had to do with the use of the words enrollees rather than subjects and this thing somebody picked up on the fact Neil that you were referring to enrollees and not subjects and sort of what the right language is here. And then a third question that I'll kind of lump with this in this category is we have all mentioned how community consultation and public disclosure happened before the trial, and then there's there's sort of disclosure afterwards but what about during a trial is there. What what activities are happening during a trial. I guess I'm happy to start. I'll see if I can maybe maybe I'll answer one or two of those questions Kerry can pick up on others or add to it but with regard to the consent piece. I think that's, that's pretty straightforward the regulations and guidance are very clear that this is not a community consent process right this is not a there's no this isn't a vote. I think at the same time if you were to find overwhelming opposition, you know I think most people would say it's it becomes difficult to justify what I think so. So it's intended to be a consultative process where you're engaging in discussion. And you take seriously what what people tell you. So it isn't it is not community consent, in large part because most of these communities are not defined communities with hierarchical structures of decision making right there's not. That doesn't exist there isn't a body you're going to in most of these cases so it so it is not like, like certain kinds of the term community consent tends to exist when you have more sort of decision making kind of hierarchies and structures you can go to. That's not that's not yet. The subject enrollee thing is interesting I I don't know whether I. You know I think sometimes a lot can be made of that distinction I didn't intend to make a lot of that distinction. They, these are people who may or may not have chosen to enroll. You know, they are subjects in an experiment, you know, in an experiment or study that that is that is absolutely true. You know, we are there also patients who are critically ill and family members of patients who are critically ill that we need to treat with humanity and and engage them in that way so and many of them if they're followed up they are they become enrollees study right that's that's an important. That's an important issue. And maybe carry or either of you just on this question of kind of activities related to public disclosure or community consultation. As the trial has begun it's not something I've thought about but it's an interesting thing to ask about I'm curious if you all have thoughts about that. I think I think I love the idea actually about ongoing community consultation and information. I think it's hard to have that it was probably more of the disclosure as opposed to the consultation aspect of it, because I think it's intended that the consultation actually does inform the researchers about how they're going to conduct the research. And so if you're changing your experiment, you know midway that poses a lot of problems, but you know informing the community that you're doing this is actually super helpful for our ongoing public disclosure for the overt shock trial. We had this huge poster in the emergency department so anybody who came into the emergency department who was there for any reason, you know, a foot fracture or, you know true trauma or, you know heart attack anybody in the emergency department could see that we were doing this research and had it all laid about about what we're doing what the background was how was going to go down. We gave out flyers to our patients when they were admitted to the trauma for any reason they got a flyer to describe all of the research that we were doing in our division. But we did not go out, you know, more toward the community and sort of continue to meet with people and talk to them about what we were doing but I do I do really like that idea a lot. And should be something that people should consider doing not as consultive because I think you can't change the experiment but from a ongoing public disclosure and I think it would actually that idea would actually foster a lot of trust between the at least geographic community and the institution that was doing this kind of research. One of the things we found when we were going on and meeting with the focus groups and as well as the patients was that families, patients and community don't really realize that we just, we don't just treat people we actually try to advance science and improve the way we care for people. And so it was actually an educational opportunity as well about like, this is this is what our institution does we don't just care for you we want to make sure this never happens again that we continue to try to save your life in different ways. I think this point is really important. I just to add on to that what one of the things that's within what Kerry said that I think just incredibly important. We think about this as isolated to an individual study but but I really think the right way to think about all of this is as a as the way that an institution and a set of researchers relate to the community and it. It goes over different projects it spans different conditions. This is about how we interact with people about how we make progress in medicine and. And so, you know, I think it we the way it's structured it's it's all about an individual study but it's really it's really about much more than that if we take a bigger picture view. And, and I think for the most part most investigators would love more opportunity for their work to be more public I don't think there's anybody out there doing studies where they're trying to hide anything right they they want they want the the discussion and and and you talk to many people in this case they're like I couldn't get the newspaper to cover it I couldn't get interviewed on TV I couldn't get anyone to talk to me on radio, you know, it's, I think, I think people want that so I think it's, they're open to it. And related vein one one person asked to hear more about this the wristbands and the experience with with opt out wristbands that both of you alluded to one question and come up in the chat about, should there be opt in wristband rather than opt out wristbands and about how, how it could be improved this idea of opt out wristbands but you know as both of you mentioned, they are many trials, or all trials give them out but they aren't giving out that many and it's not clear. In many cases how many there are giving out or folks are actually showing up ever with the wristband on and are therefore being excluded from trial so just if you know, either or both of you could speak to the wristband question. So the, I think this is a really great question. So the opt out wristbands, you advertise them and then people don't pick them up or don't ask for them and those that do ask for them. Really do not want to be in your trial so you know that from the beginning, the I think that challenge with doing the opt in is that these disease processes are not predictable. So, if they were predictable, then you would be able to go out to those groups and sort of advertise your study and get enrollment in advance, which is one of the requirements for epic you. So it's, you have no way of predicting, for example, who's going to suffer some of these emergency events and if you if, if we have the opt out and it's poorly attended you can imagine how poorly enrolled the opt in would be as well. I think it just adds to the impracticality of the whole research. So, I don't think we would get any of it done from a practical perspective. Now does that make it ethically murky, possibly, but it does make it almost assuredly that you will not get your study done. Yeah, yeah, I would just say I agree 100%. I don't think it makes it ethically challenging in the sense that there's just no way you can possibly reach. You look at what what most people don't know about what's happening in the world and to think that we have any realistic expectation of making most people aware of a of an individual study is just is, I think not practicable. The other thing with regard to the opt out piece right. I mean you can imagine, if we really rely, you could imagine people having like a long if you're in a place that does a lot of emergency research you'd like your whole arms going to be covered with opt out bracelets. And these things are really hard some people have said should we have a blanket you know don't include me ever in emergency research. I don't think offering people something like that is also. I think if anybody were enrolling and they saw that they wouldn't do it right you wouldn't enroll someone who said that and that would be appropriate. But, but the idea is really trying to promote kind of blanket research refusals when it's really a contextual and and not related to the study or the condition. I don't think that's productive I think what we what we do we have a burden of communication and and I think appropriate oversight to be trustworthy right I think we just have to own it. We were as institutions as investigators as ethicists. We have to own that we're taking a big responsibility by doing a trial that enrolls people without their prospective consent and and really take that seriously. I think also if you look at, you know, one thing that is given me sort of encouragement I guess is that when you look at the people who were enrolled, and you ask them, are you glad you were enrolled in this trial. Overwhelmingly people are in our study for the overt shock trial. We enrolled 100 patients and contacted every one of their families at some point within that time. We made it some patients didn't make it and only one family member actually withdrew consent when we approached them so I mean that's a remarkable enrollment and I think what it goes to show is that there is despite what we say, there is some trust between the medical community and be the patients that we serve. And that in fact people when they're in in these emergent situations, they really do want either standard care which is the placebo part of this, or they want something better that we think might be better. And, and so I think that there is a there is that trust. And that that's I agree with you 100% I think that the piece that was most that we wanted to focus on when like within the ESAT for example was the, the people that that disagreed with being enrolled not necessarily people who are neutral or agree it's really what we want to know is if people are upset why are they upset right what made that experience something that they didn't want to happen. And, you know, from other studies we've done it might not have anything to do with consent it might not actually even have anything to do with the study. I think we really need to start to study what that is, and see what's addressable, because it may be communication barriers we could help to, to demonstrate trustworthiness and tell people what went into the study rather than you know, like misperceptions right there's a lot of ways that I think we can, we can help and there's no regulatory component to that either right I mean there's lots of ways we can communicate, and we can find ways to be effective in that. So if someone asked about the, the, the concern of survivorship bias that it, you know for interviewing people afterwards who made it. Then, are we, is, are we, are we doing it right and a question that I have for you all is sort of the research out there on interviewing families after the fact when when it's, you know, folks, when it's a loved one who passed away and if there's, there's sort of good data on families versus versus the person actually experienced the, the event himself or herself. We did so we did that in our study so first of all, in our group, we, we, for the protect study it wasn't just people who we interviewed whoever made the whoever was was initially contacted as the person we talked to and in many cases, particularly in traumatic brain injury the patient's not eligible to be included we talked to the patient when the patient was alive, if, and available if the patient was not was neither alive nor available we talked to family members and and most of those in that study that we talked to were family members and, and many of them had people that didn't survive. But there's clearly a bias in our sample in some ways that I don't know how big it was because a lot of it's not a response but, but you know, I can tell you from that kind of study people had different kinds of responses right I did the very salient statements and people who had a loved ones that passed away who said well I hope that we learned something from this or I'm glad they had a chance to try every potential therapy right I mean so I mean you can imagine and whether those are super well informed or not you know it is, it doesn't matter but but I think their experiences are what we were trying to study. And, and, and many of them I think found value in that process, especially if it's communicated well. One, one question to pivot a little bit had to do with the, the idea of anonymity in community consultation and so you had that that nice graph meal of sort of the different ways in which the sort of depth of communication you might have with with a one on one interview being sort of the quite a lot of depth, and this person was asking about social media, and, and I wonder what you think about social media as a, on the one hand, having some anonymity, good. On the other hand, you know you may not have the kind of depth of a back and forth that really gets into a person's sort of personal history and views as well so I'm curious what you all think about that. I mean I guess the question is like, what, what is the, what is it that you're trying to get at exactly, and often like it's, I think, usually in the context of these community consultations, you want to explain to the community, what the study is exactly and sometimes just if you do that from a written perspective I'm not sure that people actually get it because there's no opportunity to ask a question. So I think I think you'll get answers, you'll get some information. The question though would be, is it truly an informed conversation, because there's no opportunity to ask for clarifying clarifying questions, and there's no opportunity for the researcher to to probe hesitancies or to make sure that there are no miscommunications so I like it in the sense it would really fill the public disclosure I think anonymous surveys are helpful for the disclosure you will get some information you may get broad swaths of, you know, information but that again the context and the question how the question is phrased you know what it just poses a lot of issues in my mind. Yeah, I think, I think the key is to understand for people thinking about the social media piece that that that this is not a that this is a very heterogeneous set of strategies right and some of which has really been developed in the context of COVID where people had to move forward with things and couldn't have in person meetings or so. So if you're doing a Facebook live session with you know members of a particular group that's a that's a substantive discussion that can happen. If you're if you're saying I've consulted you know 80,000 people who have clicked on a website once and nobody had an objection so they think that's okay like that's you know we know that's garbage right. And people accidentally clicking on things doesn't necessarily constitute real notification right so I think. I think there's subtle uses of this is probably a whole lot of tools that could be really powerful. One thing I've seen some people do for example are you know focus groups are hard to do right their labor intensive, not everybody has the expertise to do this. You can do that remotely and do it virtually and have actually people who are more qualified to do a focus group, moderate a focus group in an online format and you might be able to reach people that you couldn't reach if you were doing in person focus on social media but, but this notion of, there are new tools, and there's lots of ways I think carries exactly right that you just need to think about what you're trying to get out of it. And, and, and whether the method is suitable to the goals. I really do like the 2011 recommendations from the mercy medicine group, because it really does provide this concept that the consultation should be a two way communication. And then if you want to couple that with an anonymous survey that people get afterwards and they can turn in the information without being identified in any way. I might get some real information for our trial we did give out surveys they were, you know they were all collected they were anonymized. In that respect, whether folks felt they were anonymous might be different. But, but I think really looking at the heart of what the community consultation is supposed to be and it's supposed to be a, a consultive moment where you talk to the patient or the community the community talks to you it says give them the information that they think you learn what their values are you get clarification. And I think when we don't do that, when we just do random digit dialing or we do like these, you know very surveyed approaches. I think that we miss out on what might be valuable to the community. And then if it if we're not really listening, are we then really. I don't want to get that confused, but and it's not really substantive judgment, but there's something very valuable and I'm not sure what the word is maybe Dr. Dr. Goldman knows but that there's something very valuable in that interaction. And I think we need to preserve that as much as possible. I have a very thoughtful colleague who describes that what he sees is a lot of the value of the community consultation process is just having to put yourself in front of people and, and it's like a mirror right you you want to be able to ask people for consent and this is what you want to be able to do and you need to be able to describe what you want to do I mean all of us who have been clinicians that it's one thing to talk in the background about something but presenting people with different kinds of choices and having to stand in one and say you look you know your loved ones very sick and we want to randomly assign them to one arm or the other like that's, you need to be, you need to be ready to do like the process itself I think makes you think about what you're doing and makes you understand that humanity of the people you're interacting with and, and you can lose that if you don't if you're not present in some way, there's lots of ways to be present but but I think that's important. So, one, one just last quick question, before we shift gears a little bit but one person who isn't who doesn't you fix studies is on the call and actually something that came out of their community consultation process was the idea of having a registry for opting that they would then check, you know, against the registry when whenever anybody was was enrolled in this person was asking how common that is and whether you all see that as a, as a, maybe a better step as opposed to Neil you're sort of wristbanded up person you know with like 10 wristbands on and that sort of thing. Yeah, that that process of a registry actually for the protect trial that grew out of community consultation it was one of the few one of the things that the actionable items that came out as people asked could you do that. So that was established for that trial and I don't know whether the person who's asking was affiliated with that trial or with others and probably has come up out of other community consultation sessions to. The biggest thing is it depends on the study right so you're doing a study like Alps where you're randomly assigning people who are being resuscitated in the field to different things that's not that's not doable you don't even know their name half the time when you're when you're doing that. In contrast if you're doing a study where you know it's emergency department based and you've got an hour or so between the time that you know that you first have an interaction and when they might actually receive a study intervention. You, you may have that opportunity so I think it's super context specific as to whether a registry is a workable solution. We probably have to promise people we can't promise that you can always know it, but there are certainly our studies where it, if it fits with the therapeutic window to check a registry. Then I think people. There's a number of cases in which that's been done. It just doesn't always fit. In terms of looking gears we had some questions around equity and around some of the, the sort of ways in which, especially black patients and one person asked specifically about young black men, you know, which Kerry alluded to in one of the trauma trials. I mean, just, you know, if we could talk a little bit more about where to go and sort of what maybe the effect trial is should be doing differently, what the, what what the key issues are when it comes to equity. So, Kerry, go ahead. Yeah, so I, you know, I think, I think, you know, again, we want to over consult the groups that have are likely to be enrolled, so that we get their opinions and I think for our trial. In all trauma patients, we had, you know, we could predict potentially who might be more likely to be enrolled in our trial just based on our geography and mechanism for large volume blood loss. You know, in West Philadelphia that happens to be young men who were shot. And so, you know, because we enrolled more young American Americans who were shot. They were the people who were presenting with the disease process in retrospect what I wish we had done is not done our community consultation with, you know, the studying the trauma patients the families and the community. I wish what we had done is look at the trauma patients who were victims of gun violence, the family members who were victims of gun violence and community members who lived in the geographic community because I think that would have given us a better representation. That being said, you know, I think that when we looked at the people who did continue enrollment, we only had one participants family members decline continued participation that wasn't an African American guy that was a gentleman who was in a motor vehicle crash. And so, you know, I think that representation matters. What I think we need to be very careful about is if the study happens to be on traumatic brain injury and we know that the most likely participant in this trial is going to be, you know, old men who fall, old white men who fall that we don't over enroll old African American men who fall like we really try to figure out who is the patient population who's going to be enrolled in that study and then target in a very focused way, the community consultation to include not only geography, but the patients who will be enrolled. And I then I think we will be really trying to adhere more to the principle of justice. So, I think this is such an interesting area and one that is more complex than maybe it gets credit for. So, I, for example, and part of a network to increase diversity and clinical trial participation cardiovascular disease right we have a historic problem of under enrollment of underrepresented populations. And the ethics base, the criticism that you know that the will you and others have raised is maybe we're over enrolling. So it's a funny, it's a, it's a funny tension that we're that we're facing with regard to with regard to enrollment here. I personally think the other piece I would say is that when you look at historic, there's some really interesting historical cases. There was a TBI trial that was happening before and after the EFIC regs were enacted before the EFIC regulations were enacted they were enrolling basically all white patients. I make the numbers wrong and then after the EFIC regulations, they basically enrolled whoever went came through the door, which was whoever came through the door other and other, and that became much more racially diverse. We've looked at individual the participation to prevalence ratio at sites, where that's been looked at. EFIC is sort of an equalizer in the sense that it enrolls whoever comes through the door with the condition right it's not selective. There's not issues about differential refusal. The issue I think, and I think what you guys found in your paper will is is probably less about the EFIC mechanism as it is about where we're doing studies right we tend to be doing studies at sites that are you know major urban centers and and this is the kind of so. So I actually, I don't know what what you guys thoughts are but but my view is that's actually not a problem with EFIC. It's a problem with site selection. And I think we need to be careful about that because we don't want this to be about the mechanism. It's, it's about the, it's about who does research and where we do it. That that probably isn't in many ways is not intentionally discriminatory but it's it's where it's where for a whole host of reasons it's where research tends to get clustered. I personally don't think it's really the mechanism. The other piece I would say is with regards to the engagement. I worry that people that are really focused on representativeness and community consultation can be over counters and if we focus on the quantitative representation of who's there rather than the quality of who we're talking to and whether we're really legitimately hearing under represented voices, then we could miss the boat in a big way. Thanks. And I think the point somebody else have asked this question in the chat about, is it exactly the point you raise Neil is this about where the center of academic big academic centers are. And you know I think it's an interesting and important question for the for EFIC trial list and for ethicists and all of us to think about this question of burdens and benefits and do we need to, you know, be more selective in which sites you know if we've been doing this in West Philadelphia for the last, you know, for, for a long time, do we need to really maybe spend the extra money and do the trial and in, you know, a place with a different sort of demographic makeup so I think it's like, you know, this seems like a sort of unanswered empirical question in a way but but one that probably needs more, more attention because it's a challenging one exactly as you said Neil it's sort of both ways you, you, you, it's good to be doing research on diseases that, you know, where trauma for example that you know great that we were getting testing therapies and so it's certainly challenging. Well we're just up against the hour and you know I just want to thank Neil and Kerry for a really wonderful discussion I learned a ton. And from all the great questions there are probably a dozen more that we just couldn't get to because of time I tried to prioritize, you know, ones that were clustered together but thank you all and Leo, I'll hand it back to you for any, any last thoughts. I just want to echo your thanks and thank you Dr Sims Dr Dickert Dr Feldman for this really wonderful discussion today and to remind everyone that we will be back in December with our next event and we look forward to continuing the conversation then. Thank you. Thank you. Thank you.