 So I think there's a lot of synergy with the first session's overview, and I think you'll see a lot of themes that are the same, and I think that's good that we're all sort of thinking the same way. So Sharon and I were tasked with the breakout session on clinical genome sequencing at scale. I should mention that we conducted a little survey of five questions a couple weeks before our session to get a sense of where people's initial priorities were coming in. And I think that underlies the first goal, and we have about six goals that we articulated after this breakout. And if you look at the survey, we asked people to rank NHGRI's priorities, or where NHGRI should prioritize areas, across four different domains, interpretation, support for interpretation of genomes, the functional understanding of genomic variation, bioinformatic tools for sequence analysis, and technology development. And you'll see that the majority of people thought that the higher area of priority was in the interpretation realm, although I think one of the things, as Sharon and I read some of the free text comments on the technology side, was that there was a clear dichotomy of opinions in whether the commercial sector was sufficient to drive technology innovation versus there needed to be continued investment in this area. And I think there were complete ends of the spectrum there, and we did talk a little bit about it in the breakout session itself. And I think the end feeling was that there was still a need for NHGRI to focus on improving technical sequencing platforms that it wasn't just going to be solved in the commercial sector. And a few sort of tactical goals below this overall is obviously focusing on improving accuracy, decreasing costs, and turnaround time. The one I underlined I think is the critical one, that we really need to detect all types of clinically relevant variation in a single genome scale test, which really can't be done today. So I think that's one of the most critical items, but also the last bullet really increasing the spectrum of tissues undergoing clinical sequencing, including analysis of circulating in single cells. I think in most of these areas, a lot of competition in both the academic and commercial sectors will drive this, but I think there's a couple of items here where competition isn't necessarily the best approach, and that is really in the second and third bullets thinking about standards. And the discussion earlier about the Global Alliance is one mechanism to help coalesce standards around the international community, I think is certainly something for us to participate in and think about. We also felt that harmonizing the technical aspects of research and clinical sequencing, perhaps not when innovating on technology, but when doing it at scale for research studies, if we could harmonize research and clinical approaches, it would be easier to pull research studies into clinical analysis and vice versa. So that was one comment that was made and I think echoed. The second goal was to focus on improving our understanding of variant and gene disease relationships, gathering data from multi-ethnic populations with geographical diversity. I think this is an area where leveraging other omics is critical in terms of thinking about how both RNA sequencing, proteomics, metabolomics might help inform genomic variation and its role in pathogenicity. I think a critical element is being able to leverage the large amount of accumulating clinical sequencing data for research use. There's a lot of data out there and it's not being captured effectively. And I think this whole area really requires a laboratory physician and patient participation in all aspects of the clinical genomics enterprise. So a few more specific details related to the tactics for this, that prior goal number two, certainly developing robust approaches to determine the pathogenicity of variants and I think that really highlights some of the second breakout groups talk in terms of different approaches for this. Number two is really related and this is partly from my experience in the ClinGen project trying to get data out of every corner of particularly clinical labs where the infrastructure they have to capture and store and therefore share this data is really poor. And if we could focus on developing a distributable platform for both clinical laboratories as well as others for physicians to share genotypes and phenotypes and clinically interpreted variants it would be really effective in harnessing a lot of the work that is ongoing as well as the data that's out there and we just need to make it really easy for labs and doctors to do this. And I think a third critical component particularly in the deep phenotyping areas of discussion is engaging the patient population both improving their education of the importance of genomics and health, developing a multi-use longitudinal cohort for all patients undergoing clinical genome scale sequencing to enable both re-contact and deeper phenotyping and of course targeted re-phenotyping based on the sequencing results. So that was, you know, some overview for our first goals. I'm going to turn it over to Sharon to talk about the subsequent ones. And I would just say the longitudinal cohort reflects back to the conversation that was had after Eric's talk of really trying to engage all patients sequence whether they were sequenced because their doctor ordered it for as part of a clinical trial. So the last two goals have more to do with really issues related to clinical utility and clinical trials. So the third goal is to determine clinical utility, value and cost effectiveness and we talked a lot about those are all really different things of genomic sequencing in a variety of medical settings. And that point will come up multiple times that we discussed that there really may be significant differences depending on the disease or health setting that you're investigating. And again, our theme about ensuring equitable access to genomic medicine across populations and healthcare settings and we really felt this was a critical role for NHGRI. So with regard to tactics, we really are going to need randomized trials of genome scale sequencing in a variety of medical settings. We've got to, but we have to really define and one of the members of the breakout group, I forgot, use this term, the evidence development paradigm. You know, we really need to think about what are adverse events, what are positive outcomes as we do for any other clinical trial to actually demonstrate the clinical utility of testing. We're going to have to develop what are appropriate cost effectiveness and value trials when you're talking about genomic sequencing and again ensuring that participants in these studies are not just the early adapters of technology, but actually reflect patients in healthcare systems across the United States. And it's really going to be important to partner and also cost share because clinical trials are very expensive with other NIH institutes to really select what's the most appropriate clinical setting, what really is the challenging disease where there's a lot of morbidity in the United States and for which genomics might be appropriate, and how do we design these trials? What measures have really been validated for that disease to show efficacy and can we really show that genomic sequencing impacts efficacy in a rigorous fashion? The last two goals have to do with really how do we, okay, so you do a bit clinical trial, you show that there's utility, but how do you then implement it into clinical medicine? And that's actually probably, I forgot who in the group said we don't do that well for anything in medicine, so why should we be different? But, which is true, but we talked about trying to identify efficient and effective methods for implementation of sequencing into routine medical practice. And these are just examples, developing clinical decision support tools for ordering and applying genomic information. There's a lot of data in physician education literature that it's best to teach the patient at the time, the physician, I'm sorry, at the time they're thinking about ordering the test, not trying to teach them a year in advance. We really need improved methods, again, point of care education for physicians, and we have to implement our experience with return of results, so we've now, through CSER and the ROR consortium, generated a lot of return of results data, but what does that look like when you're really going out into clinical practice and not in a clinical trial? And related to that, we really think that NHGRI needs to think about how should we be doing implementation research in genomic medicine. There are a lot of such studies out of the LC portfolio of grants, but if we're really going to be doing genomic sequencing in clinical medicine, what's really the best way to study how to most appropriately implement that across many different healthcare paradigms? We had a very interesting discussion, I think, kicked off by Steve Jaffee talking about the children's oncology group and really the huge impact of cooperative groups across many years and other disease paradigms. And so, in terms of trying to summarize this, our overarching strategy was creating a learning health system for clinical genomics, for patients, physicians, insurers, and regulators. And one might want to consider, as a very large-term goal, development of really a national cooperative group for genomic medicine. Question? Come on. Your light's on. Do you want to say anything? Yeah, no. Yeah. Yeah, and I forgot to mention I was at the group yesterday, but I believe the emphasis on clinical trials might be a little lessened if you think also about on comparative effectiveness research done on observational studies, if the numbers are large enough, because I think that's a trend that the learning healthcare systems are adopting right now, and also a trend towards pragmatic trials of the cost is not as high. Great comment. Yeah, David. So, I think the goals that you've laid out for the field are very good and appropriate. I guess one of the questions I'm struggling with is, what's the role of NHGRI? And so, NHGRI funds research, right? And it's not clinical implementation, and it's not regulation, and it's not these other things. And so, some of the things you talked about, it seems to me, I can imagine how NHGRI would fund a pilot that would move us towards something. But when you get to clinical implementation, a lot of these are issues of regulation and reimbursement and incentives in the healthcare system, and I'm trying to figure out what's the role of the research institute with a limited budget in achieving those things. Well, I'll just come in then, Heidi, Ken. There was a lot of discussion, especially among many of us, in the group who actively try to order genomic testing about the, and Gail may want to comment, who talked about this very eloquently, about the fact that if we don't start generating real data that this is useful, that there is utility in a clinical setting, we're never going to get to the regulators. And that's why there was a lot of emphasis on really thinking, and perhaps as the prior comment said, more creative ways, and we didn't talk about study design, but of really developing studies that test. When is a genome scale test really effective in clinical medicine? And I will also add that executing on clinical utility and even observational studies are not going to be effective if the results that we are returning are inconsistent and incorrect, right? So, and right now, as we have engaged in an early stage laboratory sharing data, it's become very clear that there is great discrepancy across what's being returned to patients, how that information is conveyed and how it's understood. And so, if we're really actually going to succeed in these clinical studies, we have to have a core component of the interpretive process be correct and consistent and accurate to what we understand. We need to derive further understanding. So, you know, I don't think that just letting the marketplace and the clinical and commercial sectors compete and do it all separately is going to actually advance us in critical ways. And I think NHGRI by aggregating and developing standards in consistent ways and really cataloging the resources and the tools to be able to do this effectively, particularly the knowledge side of it, so that we're all using consistent knowledge to support the integration of clinical genomics. I think we'll then support the outcome studies that we need to achieve to show that this is useful or show in cases that it's not, but answer the questions. And I would say that I think we tried to put our slides in an order that go from things that are more, really obviously what NHGRI thinks about doing in terms of improving the test and improving the sequencing to things that we would really like to push NHGRI to consider doing. And so that maybe, as you said, is not currently within their purview. Gail? So our goal is the National Institute of Health. We are trying to improve outcomes for patients and we can do a lot of basic science, but if we don't get that into the clinic, we have fundamentally failed at our major goal, which is to improve health outcomes. From my standpoint, I negotiate with healthcare systems. I'm talking to University of Washington, I'm talking to group health and trying to convince them to implement genomics in a way that isn't one off. And they say, where is the data? And to answer that, we've been doing cost modeling, we've been doing some other outcomes research, but fundamentally they're not gonna make these changes until they see that there is some utility, not just to save money, because that isn't actually the overall goal of the health system, but to improve health, right? And so we have to show them that using genomics in medicine improves the health of some patients, which patients? So, Eric? I share the passion for ensuring that this gets out to patients, but the conversation begins to worry me a little bit because you could say the same thing for drugs that until you can show the efficacy of drugs, they're not gonna get adopted by the healthcare system and many other things. And yet on the research end, the NIH doesn't generate most of those data. They get generated by the sponsors who bring them to the FDA and bring them to the payers. And we have a system where those who would provide that to patients generate that and pay for it because they in fact will be reimbursed. It's possible to burn a huge amount of money demonstrating pharmacoeconomics or the economics of this. So I agree, but I wanna, I mean, we need to be doing some of it, but I'm worried that if we take it upon ourselves to generate the whole economic and efficacy argument, we'll burn the NIH budget doing it. So, we have to have some line there. Eric, could I ask you one question about your model? Unless we're going back to patenting genes, right? Who's the payer then, right? Well, the payer is clear. The payer is gonna be the insurance companies and CMS. No, I don't mean that. They're not gonna, they're not gonna. So you wanna know who's the sponsor? Who's gonna sponsor the company? Who's gonna sponsor this? Well, the question becomes post-gene patenting. I'm very happy about, we're in the post-gene patenting Europe. What's the business model? So there are very effective business models and diagnostics without patents around the analyte. We have large companies, lab cores, quests, whatever, they do it by virtue of lines of distributions, connections to hospitals. There is gonna be a shift in the business model and it's not gonna be around the patent on the analyte. It'll be on some package of services, some lines of distributions, probably larger organizations. I don't think we are gonna replace the need for an economic model for diagnostics. No, I guess what I'm saying is at least up until now those groups have made no effort, none that I'm aware of to sponsor this type of effectiveness research. So I agree. And I think if we're saying, how do we drive the creation of effectiveness research? I agree. How do we do some models of effectiveness research? I agree. I'm just, and I didn't mean to suggest that the suggestion was that we go all the way, but I wanna note we do not have the critical mass to do all that effectiveness research so we can be catalytic and should be catalytic and set the bar, but we've gotta actually have an ecosystem out there that's gonna drive an awful lot of that, that's all. I mean, I agree with you. And I think one of the reasons we emphasize the need for harnessing all of the data that's already being generated and paid for by insurers and the healthcare systems is because we can't take on the entire cost, but we have to do it in a coordinated way and we have to be able to harness that data for multiple uses to understand whether different paradigms have evidence or outcome. And to the point Eric Green raised earlier about are there fellow travelers who might bear some costs in this? I think this is a great case where insurers, both CMS itself and others and maybe PCORI, ought to be picking up a lot of this and NHERI ought to be the tip of the spear organizing these studies, but we ought to be figuring out how others are really providing the financial fuel for those studies. Right, and that was, we were trying to make the point about partnering with institutes, but we didn't talk about other sources. Jim and Richard and you. Yeah, I agree with Eric that it makes me nervous when we start talking about this, but I don't think we can avoid talking about it because ultimately, like Gail says, this is the end goal, right? And to me, what we have to do is partner, right? And it's not gonna be lab corn, it's not gonna be GBX who pays for it, but we're gonna have to pick and choose and be very careful about how we do this and the beauty of it, I think, is that I know that I don't know much about outcomes research, but what I've also learned from working with people who do know a lot about it, they don't know anything about genomics. Right. So it's a really good partnership that we need, that NHGRI needs to kind of facilitate. Ken? Yeah. Yep, just to emphasize that point, some of us sit in other roles and as you know, Moon Curry's been at this for some time and we've been working until the funding went down through the CDC program, but it looks like some of that funding may come back. But just to under, first to say, I really think this is great what you've come up with the translational effectiveness stuff can clearly be co-sponsored. PCORI, Eric mentioned quickly, is a huge funding resource and then these EGAP initiative, Knowledge Synthesis workshops are there. So I think we don't, NHGRI doesn't need to bear the bulk of the burden but absolutely should work with the other institutes to do that. One thing I might also add is when you think about what we saw yesterday related to gene therapy trials that are already ongoing and some, you know, more than 8% of those are monogenic diseases that have been genomically characterized. And to me, that seems like that's genomic, clinical genomic medicine already happening that if you can tie some of that to knowledge around costs and cost effectiveness, you really have a good case for, you know, where does genomic medicine really make a business value, a patient value, and physicians. Thank you. Robert, I think you had your hand up. Yeah, yeah, I think a lot of initiatives in medicine did diffuse out into medicine. I'm thinking about coronary artery bypass surgery, lots of technologies before they demonstrated some sort of cost effectiveness and I'm afraid that that's sort of out of the bag it's gonna be happening here. So I think we just need to be careful we're not trying to say, you know, genomic medicine can't happen until we demonstrate this. I think that would be counterproductive. But I do think that. Right, and I would just say that was not our intent at all and that's why we talked about there is a lot of sequencing going on and we need to harness that information. Right, and so I think that's a great idea. I also think that somewhere between the sort of harnessing in centralized or federated fashion there is a danger of premature consensus. I like that phrase that Maynard used before in the clinical realm because we really don't know how these can be modeled and utilized. So I think NHGRI can play a role in permitting and encouraging experiments in clinical utility, even absent large-scale downstream proof at this moment in history that that clinical utility is somehow cost effective and then the larger marketplace can take these experiments and try to work them through. So for example, I'm just talking about different several of us are already doing randomized clinical trials in CSER. We do have outcomes teams associated with some of the CSER grants but some of the things I'm curious about are sentinel studies. Just like Rick had suggested, you have protein categories. You can have categories of trials in particular types of genomic information that stand in for the rest of them so you don't have to do thousands and thousands of trials about thousands and thousands of genes. I mentioned N of 1 studies yesterday and then clever ways to do longitudinal outcomes trials. All of these can be supported from a methodological point of view by NHGRI without having to take on the societal role of regulation. I would agree. I would just argue that I think we do also have to learn from our colleagues and if we're gonna do trials, we need to define what outcomes are. We need to define. Absolutely. We need to define adverse events. I mean, if you really think about this CSER paradigm and I'm one of them, few of us really are agreeing on even definitions of what is an adverse event and I think those are the kinds of things that as genomicists we really need to start defining so that if another group or institute's doing a genomic based trial, we have some definitions, we've had some experience with how to do that. And I think we also need to define how to aggregate because there will never be a trial for every rare disease or every gene. And so figuring out ways that we can aggregate when we're looking at cost effectiveness and utility across groups of areas, rare disease diagnostics as a group and things like that. And I think having a discussion about how do we aggregate different elements of the clinical utility analysis spectrum would be useful for us. I don't know if you got this granular, but if you broke into diagnosing and treating disease per se versus sort of prediction aspects, which is in some respects more futuristic, but in other respects, it's very front and center in the mainstream population. And so the considerations aren't exactly the same, I guess, and so I didn't know if you broke it down. I think the main reason we were saying in a variety of medical settings is for exactly that reason. We did have a very lively discussion about putting the money into sequencing people at birth and seeing how the data is used over time. There were real supporters and non-supporters of that idea, but clearly there's a whole different design for prediction and would require a very different kind of study. And I think utilizing these large-scale patient cohorts ascertained by genotype, not by phenotype, will help inform the predictive nature of genetics and penetrance values and things like that that I think will then feed into our ability to do predictive medicine as we understand the full spectrum of how genomics influences the development of trades. Eric? So following up on Mike's comments, did your committee discuss the role of healthy genomes? It seems more and more of this is gonna be driven by the population, the individuals, and less from originating from the physician. And I think that was this element of critically engaging the patients in longitudinal registries. There are many healthy genomes already out there and there will be an increasing number if we can engage those patients and their genomes in longitudinal phenotyping. And we're developing a registry, there's other registries out there, and I think that will become a critical element. I mean, I think a relatively short-term goal is this idea of trying to engage the population that if you had an exome or a genome ordered for any reason, that you agree to join a cohort. Because even the term healthy genomes, which I always find kind of funny, I was one of those, I was in a healthy genomes study. I have multiple chronic conditions. I mean, I'm up here, so I guess I'm healthy. But you know, so even the definition, even the definition of even getting real data on people who had healthy genomes done will provide us a lot of information. But I do think really there was a lot of concordance around the idea that we need to be capturing all the data we can from individuals who've had their genomes or exomes done for clinical or, you know, or for whatever purpose so that that data's not wasted. Debbie. So I just wanna say in bringing up this idea of genomics and health and people getting their genomes, how diverse are those genomes? They're not. So we need to push projects also that actually bring genomics into diverse populations through a variety of, yeah. Are we? Well, I think they had, I mean, if I... Sorry, sorry, this has to be the last question because we have to move on. Yeah. No, I mean, just adding my voice to the course, I mean, but I think you guys had that on there is that in fact the whole notion of equity in genomics research is one that is totally worth, you know, sort of thinking about particularly in this context and implementation and in particular reimbursement where you can imagine different populations are gonna be hit quite differentially. Andy and Evan.