 at the microphone, please. So these have been fantastic talks. I wanna thank you all for what you've done. And I wanna talk a little bit about the bugs that are still hiding under the rocks. First of all, I think it's tough for all of us to ask questions at the end without having been able to address you one at a time. I had questions that I've forgotten already, but I remember some of them and I will ask them. First of all, I belong to an IRB for a longer time than I can recall before the IRBs were even constituted. And it's very difficult for each IRB to come up with a policy. I personally believe that there should be an NIH-wide policy in these areas that IRBs can handle. I compliment the Genome Institute for coming up with, I guess, their policy positions that are handed out to the IRBs for each IRB to think about, but still you don't know what the IRB is gonna do with it. So I think that's one problem. Another problem is that a lot of members of the IRB are totally ignorant about what's really going on. I have to argue with my IRB all the time because many of the members and the bright ones believe that there are ways that one can only selectively extract the information from whole genome sequencing that they want and that there's information that can be ignored, that it's not even there, that it's not even seen. And I sometimes dissent with my IRB because of that because I don't think that's true. I think the information is there, it can't be hidden, and people are kidding themselves that they can take it out selectively. I had another comment and I've already forgotten it. Let me just think for a moment. Yeah, why don't we get some answers first? Sure. I think that's one for me to take. So I think you're hitting on two very important points there that it would be useful to have resources or policies or some more uniform tools that across the NIH would be available to investigators so that they can know how better to anticipate what is gonna be expected of them at the IRB. And I agree with that. I think there's a balance between giving IRBs the latitude to have some flexibility to consider the particulars of a given study. And that's actually the concept behind the IRB system is to have each institution to have its own board that can consider local factors. At the same time, the consent form has a boilerplate on it. And so I was gonna, yes, I agree. I'm anticipating what you're gonna say, and I didn't mean to cut you off. But if we could develop tools and templates or examples of language that have been approved by the IRB, those are some of our goals. I forgot to mention in my talk, and this is why I put others in my acknowledgement because I knew I'd forget something. There's an entity springing up across institutes at the NIH that's known as Protocol Service Centers. So the idea, and we're working very closely with the new one that was just formed with NHGRI. The idea is that this is another entity that was created to help support clinical researchers as they develop protocols for the IRB. And in our case, we're in very close communication with them in hopes of developing these kinds of tools, in hopes of helping them help investigators anticipate what the IRB might be looking for and to help them respond. So there's some interesting new, and I think I'm optimistic that some of these resources will be forthcoming. But at the same time, I think there's a problem with assuming that we can have a one-size-fits-all approach, a standard template that will work in all circumstances. And so the tools have to be interactive and flexible. And this is challenging and takes time. The second really important point that I didn't want to lose track of is the idea of IRB training. I think that's a very valid point. These are complex issues. Even in our own specialized IRB, where we're familiar with genetics, we've evolved in our thinking. And this is why we've done additional training. We've dedicated our retreats to inviting investigators to talk to us about the realities of this kind of research. And that's reflected in my comments. So it's been this give and take. And I would encourage other IRBs to consider ways to bring their members up to speed and the sort of standardized training that they receive, the required training, isn't really gonna be robust enough to get into some of these nuances. So I agree that that's a problem and something we should pay attention to. So I can come up with a third problem. And it's one that wasn't addressed today and probably for good reason. And that is the nasty subject of direct to consumer testing. This is going on in the genome field. The FDA and the other agencies have so far stuck their head in the ground, but patients think they can bypass the physicians and the companies think they can do it. Some companies have panels that are supposed to be well constituted. This is totally unregulated. So I'd like some comments on that. Well, we both, I think we both would have, I think some related comments. This actually happens to be an area that I've carved out and thought of as separate from what we were talking about today, but that I've worked on it and care very much about. But in an effort to connect it to what we've been talking about, I'll say two things. First, we make darn sure that our protocols aren't offering services. So let me put it a different way and much more politely, I think. Some subjects might request to receive all of their results on a disk. Some investigators with the best of intentions and wanting to meet their subjects' expectations have talked to IRBs about doing this. And we don't allow that because we don't think that genomic information on that scale is ready for primetime. There's no way to CLIA certify, I think, an entire genome test. I might be wrong about that, but we really care about the kinds of careful interpretation and considerations that Julie talked about so ably. So we treat research quite a bit differently than the DTC world that makes these things available to people at a price that some can afford and some can't. What I worry about is that once people know that these things are available, they might be less willing to enroll in trials that help us learn so many important things about how this will play out in clinical practice. So I flag that as a concern that I'm aware of that bears on the research context, but that IRBs and ethicists aren't necessarily well positioned from the inside to deal with, but I think issues of regulation, those are all ways to go about doing it. Yeah, and I think the only thing that I would add to that is that this is part of the reason why we have incorporated social and behavioral research to try to understand the actual choices that participants make, the meaning that they attribute to the kinds of genetic variant results that we could find and how those choices actually play out in terms of their actual behaviors about what results they'd like to receive because it's really hard to predict the uptake of these kinds of tests and what people are going to do with the information and how they're going to approach their clinicians with the information. And so I think that's part of the reason why we have built these procedures in these very labor intensive procedures into our protocols so that we can acquire that data to try to understand more about what people are bringing to the table because the kind of intensive and high quality annotation that we do, I'm not sure is necessarily going to be adopted by industry. And so that's a good starting point for us. It's a very interesting research question for us also. Thank you very much for your talk. So my question sort of approaches the issues of the longevity of the data and issues that may not even really be ethical, but maybe more medical legal in nature, but I think they're related. The first question is the issue of re-annotation and the expectations as far as how often that needs to be done, how long that needs to continue. I mean, so as we learn more about all kinds of diseases, is this the burden on the investigators to continue to re-annotate for 20 years after we have this data on a person? And the second one is about the data itself and it's related also in that, what happens if a PI retires and this information is there and I mean, what are the access issues and we're not giving them their information on a disk. How do these patients, I mean, what sort of issues have you guys discussed in relation to this among your institute? So we've signed up for, I think that when we've talked with our participants, we've kind of signed on to hoping that they continue to stay in contact with us, at least for their minor children until their children are 18, but we do put the ball in the participants' court. That is that we would like for them to initiate contact with us in an ideal world when their child is in a position to make these kinds of decisions and I think that this is another thing that has been a nuanced part of our conversations, at least from my perspective with participants, is that we talk about how we plan to do some re-annotation. Maybe it'll be every six months, maybe it'll be once a year, maybe it'll be every two years, maybe it'll be when more tractable techniques come out to make sense of the data and I think that's part of the reason why we alert them to the fact that this is not hitting enter on a computer program and some sort of output results. That would be great and would make this all more tractable, but right now there are really serious considerations when we think about the kind of the interaction between our time in the laboratory and what our goal is. So I think that most participants would really much rather that we devote more resources to trying to uncover the genetic cause for the disorder of interest than in re-annotating exhaustively their secondary variants. I think it's a great question. I can add to that, I think there's been some different approaches that have been proposed and that have been approved and in some other studies with more focused objectives, it's been argued that the obligation to continue looking at and interrogating and disclosing incidental findings goes for the duration of that primary study and no further. So if a study is gonna take three years to generate the sorts of results that we've been hearing about and that's when investigators gonna be actively using the data, that's how long we would expect the obligation and it's practical in a certain way and we can't expect that every year, every few months or every few weeks, somebody is gonna realistically be able to look back into the literature, update their knowledge, update what they're finding and how that relates to individuals and how the resources to disclose it. So there are some practical limits that have built into studies that look roughly like that. On a related subject, is the subject's decision on receiving or not the results a legal obligation for the researchers? I'm imagining a situation where a subject agrees not to get results and later on he develops a disorder. Is this person able to go back to the researcher and question about that? So I'm not a lawyer and I think the legal status of a lot of questions around this area is uncertain. I think some people argue, they're worried about or they make decisions out of fear of litigation and practice standards emerge based on their decisions, based on that fear of litigation and things evolve in that way. I would much prefer a very careful and transparent approach that draws on ethical principles that's well designed and sound and I would hope that case law and other things flow from that. But the truth is I'm not an expert in these areas. My colleague, Ben Berkman, actually is trained as a lawyer and he is with some of his legal colleagues working very hard on some of these questions so I would also encourage you to stay tuned. I think it's a really good point and that's about the best answer or at least the best punt I can give you on that. Thank you, they were four excellent presentations and I'm gonna ask Sarah another one. So one of the things that jumped out at me from your example case that you used was one of the stipulations that the pseudo IRB gave to the researcher was to identify genetic counseling services and I think this is an important thing for the clinical center and the NIH to be thinking about and talking about because all the genetic counselors I know are very busy and overworked as it is with their own protocols and responsibilities and the genetics consult service which is a very active service and has quite a presence in the clinical center that I participate on doesn't, isn't staff sufficiently to go around and be able or I would even argue isn't appropriate to be consenting people to other people's protocols about sequencing so I don't know what it means to go and find a genetic counselor to help. I think that that's somewhat of a meaningless recommendation unless we come up with some sort of systematic way where there's a service or a person who's actually available to them or design position statements or something for people to hire them in order to really implement this. I'm so glad you called me on that because we've learned that you're right. The IRB was trying to help and certainly felt it didn't have the expertise to weigh in on things like which variants or to really dictate how disclosure would occur and we're aware we even have members who are genetic counselors and we were scrambling for ideas to try to help a bench scientist who had never submitted a protocol to the IRB, much less had never worked directly with genetic counselors. I have since learned a couple of things that yes, the consult service isn't available on demand to do this work. They are available for the kinds of protocols where it's relatively rare that something is going to come up. There's been a willingness as I understand it to be called in in these much more sporadic kinds of cases, much like the ethics consult service interestingly has also been called in to help weigh in on very difficult decisions about disclosing certain kinds of findings. It's prompted us to really think about how we might build some of these resources and one idea is to have to employ a genetic counselor in the context of these protocol service centers who would be available to the institute and that's a great idea that requires funding and so part of the rate limiting step there is figuring out how to fund another position. So there's ideas like that but it's one of the bottlenecks that we really need to be aware of and that I urge funders of research and institute directors and clinical directors to be thinking of because this is a very real need that's not currently widely available to our investigators. So we've realized that and I think that might be motivating in part why the IRB and why protocols have pulled back a little bit from such a robust requirement for disclosing these findings because in reality we're just not sure if people have the capacity to identify them or disclose them responsibly. But your point I think at the end of your talk was you want the investigators to be thinking through the ramifications and what resources are gonna be available. Right. Yeah, so my last point is not originally mine so I'm probably voicing other things that other people have said to me but it really resonated with me this afternoon around the semantics of variants because even though Jennifer's talk was an entitled incidental findings she never used that phrase throughout her talk and Julie used variants the whole time and Sarah I noticed you used incidental findings and I bring it up because that is still the verbiage in the bioethics literature I agree with you there's some excellent writings related to it but I think that Jennifer and Julie embody the fact that these are no longer incidental findings we need to get rid of that language we need investigators to acknowledge from the beginning that they are going to find variants and how they're gonna manage them isn't incidental or surprising it's anticipated it needs to be anticipated. As Susan Wolf says we need to stop unanticipating it. But no I take that point and I think I have to use more care I sort of hinted at that I think this is where it's going but I'm willing to take that suggestion and we'll be more mindful of that. I wanted to make one more comment as somebody who's also on the console service is that at this point we're starting to get people that are asking if we can provide exomes or how they could go about getting exomes done in their patients and it makes sense if you're looking down the barrel of $5,000 or $10,000 worth of molecular testing that you might wonder if you couldn't get something a little less expensive that would answer more questions for your patient on a clinical basis and I think that in NHGRI I guess we have to have a discussion about whether we're gonna take a leadership role or just sort of watch things as they evolve with regard to the transition of exomes into the clinical realm so I think that's something that's interesting to observe on the ground. I have a few more comments. First of all, you raise the issue of exomes and targeting things. I think that that's irrelevant to the topic of whole exome sequencing or whole genome sequencing. That's the traditional way of targeting a specific gene or a specific set of genes that are within an exome. That can be done but that's not what we're talking about today. The second point is that I think all of us here today are on the side of the gods. We're saying that the ethics are moving in a certain direction that we really should give all the information possible to the patients but that's not what's happening right now. There really are still protocols that are coming around and being approved that say we are gonna hide information and that's the way it is and the IRBs are allowing that and passing it. So I think that's a really nasty situation that we aren't dealing with at all. And what was that third issue? Well, let's answer the first two and then we'll get to the third. So I'm not sure what the first point you were making. The first point was that you mentioned targeting an exome and I think that that represents an example of the traditional therapy of having basically a targeted gene. Oh, so I think that there was a semantic error there with verbiage. I don't use the word whole very much because of what Les Beesekker talked about is the fact that whole is a little bit misleading. A, it's redundant. If you're saying, oh, it's all of them. And if you're saying whole, you could mislead people because you're obviously missing some things for technical reasons. So I'm just assuming when I use the word exome that we're talking about all the genes or as many of them as can be captured. And then what was your second? Well, that in fact, in September of 2011, we have groups that are saying that we are planning to give only selected information and we plan to hide the rest. That's happening although most of us here feel that it's unethical. So I want to clarify that my intention was to acknowledge that there are those categories of studies that are being approved and at least try to characterize arguments that support those models ethically under certain circumstances. So I, although I think there's some emerging trends, I think they might vary here within the Intramural Research Program where we do have certain kinds of expertise and research is available to us much more so than in the outside world. I think there's some trends emerging but I don't think it's universal and I don't think it's universally agreed upon and I think there's room for multiple ethically supportable approaches in this research area. That said, I'm sure there are as many people as are sitting out there who would disagree with at least something that I said about those ethical arguments but I didn't want to suggest that there's a agreed upon absolute consensus about one way that this ought to be done. Even though we heard pretty strongly people trying to encourage us to go in that direction and I think those worlds can coexist in an environment where we're developing empirical data and trying to learn what best practices are and how to share them and communicate them outward and generate resources for them. So my third point is kind of a summation of what you just said and what was said over here, the issue of resources for counseling. Here at the NIH, all of you, the whole training program, we're really blessed with a lot of resources. Outside of the NIH, I think it must be a minuscule representation of what we're doing and the counseling services must be overburdened to an unbelievable extent. Agreed. Can I just get back to your second point just playing devil's advocate a little bit and I kind of, can you hear me now? Is that better? Okay. Just to kind of follow up on Sarah's point, you know, I think it is, again, important for us to keep in mind that people are voluntarily wanting to participate in research and this hasn't happened to us because I do think that for our exome protocols we are selecting for people who are really interested in this kind of information and what they could learn. But, so I don't think we have a whole lot of people who sign up and have, who are expecting to never get results. However, I think that it's possible to design a protocol where that is very clearly communicated to participants that this is really more like giving blood or participating in a biobank or tissue bank and that that could be okay if participants really have a thorough understanding that that's what their sample is going to be used for and that there will be no way for investigators to return results to them. So I think that's kind of what your, thank you, yeah, that was a good clarification. So I haven't thought about these issues a whole lot in the past, but you know, we have genetic disorders, I think similar to what you're talking about where a patient is coming in, a child who has a disorder and the family's goal really is to diagnose that disorder and they don't wanna find out that mom has a BRCA mutation and just from the standpoint of resources, it seems to me, if you're gonna do WGS on the three people and you're lucky enough to get a result that means something, that's $3,000. If now you find out mom has BRCA and you wanna go back and tell mom has BRCA and you need a CLIA test, my impression is that's relatively expensive and you're talking about adding a lot of money to inform her of that, which she may or may not even wanna know to begin with. So that, I guess I would take sort of the opposite finding, which is really we have to conserve resources and we have to communicate clearly with the participants our goals and I wouldn't consider it hiding information if you find something like that and don't report it. It's, there's no harm, no foul I guess as Sarah pointed out. Mom didn't know that to begin with, mom didn't come to the NIH for that purpose and how is mom harmed by not knowing? Cause she could die, I guess is like the very, so again, this is totally anecdotal and David you probably have a lot to add to this too, but I think that when we were thinking about designing this protocol to elucidate the molecular etiology of rare disorders, my initial thought was kind of along those lines that perhaps this is almost coercive to ask people to participate in this study and that there's almost this coercive aspect to ask them to even think about all of the other information that they could learn, but as I talk to participants, I have to say that all of them are tremendously grateful to have the opportunity to learn that information and then the case of actionable variance. And again, I think that on the individual level, these are families who have ill children, they're young, the parents are in their 30s and 40s right at the time when some screening and intervention could really make a huge difference for them and their main goal is to improve the life of their child and to be around, to take care of their child who may need lifelong goals. So I think it's almost the opposite of no harm, no foul that we really have the, that we can't pretend that such a variant doesn't exist and we have a huge obligation to provide that information to the family and that it is critical health information that they view as a huge potential benefit and I think that's what many of them say when they talk about any of their secondary variances, yeah, tell me anything about my health because I wanna be around for my kid and if there's anything that I can do or any health implementation that I can put into place, that is extremely desirable information to participants. And we say that. Use your repeat that for the record. Oh, sorry. So I think your question was that it's, then it's somewhat incongruous of us to say that we're gonna sequence the minimum number of people because if we don't get back to participants then they're assuming that everything is okay and we do explicitly talk to people about how if they, that this isn't, again, just because we haven't told them about something doesn't mean that their risk for that doesn't exist. So I think that that's one very real way that we kind of address that specific consideration. Yeah, I would also say, you know, as far as cost, we're, you know, for, Clintseco where we looked at 572 individuals and I think if you want to look at things like BRCA-1, BRCA-2 variants and, you know, Malignant hyperthermia and things like that, we're probably gonna end up with, I don't know, 20, 30 variants that we feel like if we don't give these back to these patients they could die tomorrow or get breast cancer tomorrow or whatever and, you know, it's what one in 10 maybe, I don't know exactly what it breaks down to but it's not like you're gonna have 10 variants to return. So the cost, I don't think it's that much and it's also, it's not a whole gene sequencing cost, it's a single variant cost. So the test, I mean in our lab we actually have CLIA certification in our lab so it cost me very little to run that test and return that result to someone but even if I was gonna outsource this it's probably a few hundred dollars for me to get a result to return to someone and if it's, you know, say 300 dollars for a single variant in one out of 10 studies or whatever, it's not that much money, I don't think, the cost of annotation you could argue today with the time it takes is expensive but I really think that the tools are gonna continue to get better and while today it takes hours to do a pedigree, a trio say, I don't expect that to continue, that cost to continue. The cost of CLIA validating will still continue but I think for these things like Bracka variant, I don't know, what's the value of someone's life and someone's, you know, knowing this information and being able to actually act on it. Hi. I wanted to ask a question about a somewhat different ethical issue that I think Julie and Sarah you both made some reference to but we haven't been talking about what little closer to the mic I'm saying. So another ethical question that you both made Julie and Sarah some reference to but we haven't really been talking about which is the possibility, the practicality of calling these data confidential. And I guess Julie, you were the one who said that some people are excited about participating, having access to a new technology sooner and that we can envision a time when it will be part of clinical care but the difference of course is that the clinician is not going to be entering the test results in dbGaP and I wonder if you all could talk a little more about that as an ethical issue and then secondarily perhaps about the possibility that informed consent could ever be. I mean, as you said, it's a torrent and we're only at the beginning of understanding ourselves what it means but these records are going to persist long as we do understand more and more. Do you think participants can understand just how confidential their data are? So this is a really interesting and complicated and evolving area, the issue of identifiability as it relates to genomic data and there's a certain sense that, except when you have identical twins, each genome sequence really is identifiable to a person. If you have that reference sample, you can figure out exactly and not even a whole reference sample, some genotypic information can help you pinpoint who's in a pool of data. So that's true. That calls for more sophisticated security measures, more controlled access measures to control who actually has permission to get into the computers that can access those data. It calls for paying attention to regulations that prohibit people from using those data for, I'll call it nefarious, for inappropriate goals and it requires being transparent with people that yeah, there are some risks here and we can't guarantee absolute confidentiality. There's also some interesting questions that even when we do our best to de-identify data and if we follow HIPAA regulations, there's some inconsistency across all the different kinds of security regulations out there. So if we harmonize those, do a perfect job, there's still ethnic and racial identifiers attached to it. So even if we can't identify it to an individual, we might still implicate groups and that has a whole separate set of very important implications that I and my colleagues are interested in. But I think people are focusing in on informational risk and revising the regulatory framework that applies to these kind of data to ensure that nobody thinks that anything can be completely de-identified and therefore not subject to say oversight and human subjects regulations. There are some proposals afoot that are being discussed to help fix some of the gaps in oversight that come about from sort of loose definitions in this area. So that's sort of an overview of I think some of the current discussions and thinking going on but maybe Julie wants to talk more about the specific challenge of informed consent. Well, and just practically speaking, I think that is one of the, so I think that part of the reason why we try to have two conversations with people about participating in our study. So over the phone where we send them the consent form and then kind of a summary sheet and then more detailed in-person conversation is because in between that time people pick up on different parts of the protocol that are of most concern to them. And so for some families, it's the kind of information that they may learn and for other families, it is the fact that we really try very hard to guarantee and to protect their data in every way that we can. We have a certificate of confidentiality and we have all these procedures in place, but at the end of the day, it's hard for us to make an iron clad promise because of the inherent nature of genomic information. And again, this is very anecdotal, but I can think of one family where that interval was nine months and during those nine months that was the big question that they were grappling with was this really information that they wanted to have generated on them. And I guess something happened and that after nine months, they figured that this was a risk worth taking, but I do think participants understand it and can grapple with it. And some people elect to not take that risk at the end of the day. Thanks. Maybe one quick last. Yeah, this is a quick question and it's, I think, for you, Les. Yes, as we're moving towards the thousand dollar genome and as you're nodding your head that CLIA certification is pretty cheap, out there in Utah, Myriad Genetics is charging $4,000 for BRCA-1 and BRCA-2 or something close to that. How can they do that? And how can we or you measure BRCA-1 and BRCA-2 if they've got a patent on it? I actually am going to suggest that you might wanna raise that question with the Office of General Counsel instead of us because for lots of, is it not on? Oh, I would suggest that you raise that with the Office of General Counsel. Pat Kochak does have thoughts about how we should be behaving with respect to patented genes. And so I encourage people who have questions on that to get an official opinion from her office in that regard. So that's, I'm glad you raised that, though, because it is something people do have concerns about. So I think from there, I will thank all of the speakers from today. It was a really great effort. I hope that you have found these talks to be useful to you in thinking about how you're going to go forward with your projects in the future. And all of us are happy to consult with and talk with you further if you have questions individually, as well as, as I mentioned in the email, we will be sending out a follow-up survey. We really want to know from you, our colleagues, how this was useful, what could be done better if it should be done again in the future. I think the technology changes, the IRB landscape is going to change probably every 15 or 20 minutes. And so I think there will be an ongoing need. So you tell us how we can serve your needs and we will be eager and happy to do that for you. So thanks, all of you who are here and thanks for our web viewers and as well, the people who watch the recorded version of the proceedings. So thanks for coming. Thank you. Thank you.