 breakout session summary about some of the goals and some of the priorities that might be considered in the sense of genome sequencing in a clinical context. So, you know, in this session, I think what we're supposed to do is really to think about the mechanisms that might be employed by NHGRI and the different strategies for accomplishing some of those goals. I don't think that discussion of those goals is off the table, but I think that what we probably want to do is discuss mechanisms and structures if we can. Lucy and I and Adam spent some time trying to think about framing this discussion, so spend five minutes here just trying to frame things. It's important to recognize that there are these disparate clinical genome sequencing efforts that are going on now. They overlap in some ways. They hopefully complement each other in many ways, but they are emerge, which of course is designed primarily to take advantage of the electronic medical record, phenotyping that has been carried out in a clinical context, and engage in discovery as well as clinical implementation. Another is really a deeper kind of vertical dive into the use of genome sequencing, whole exome at most of the centers, whole genome at one center at least, to really examine its clinical role, its applicability, its impact, very significantly also with LC as part of that. The undiagnosed disease network, and I think I got that acronym wrong there. It should be UDN, is kind of self-explanatory. Insight is the effort to use sequencing in the newborn period to see how it might augment newborn screening, and then finally ignite are the demonstration projects. The questions that I think that we feel like ought to be considered, ought to be very front and center, include some very familiar questions that we've talked about for the last day and a half. One is what will the landscape look like if the NHGRI doesn't do this or doesn't do that? Also an affiliated question, where is coordinated action necessary, right? This gets to this point down here, which is one to think about organizational structures, and those can vary, for example, on really at least two axes. One can be investigator-initiated, all the way to consortia, but the other has to do with what the scale needs to be to answer these questions in terms of the number of people sequenced, for example. Something that has also come up in many contexts, and is definitely worth considering in something the NHGRI will have to grapple with, is when should the market be relied upon and when shouldn't it? There's some sentiment, but this is controversial to some extent, that when it comes to certain aspects of technology development, perhaps the market can take care of that. I would, for example, argue that especially in the clinical realm, where the dollar stakes are very high, that might be the case, but I think that obviously is open for debate. I personally feel there are other aspects where the market is particularly ill-suited to answering questions, and that has to do in some ways with just pure implementation. The market, if they can make a profit, will implement things. The PSA is a great example of that, even when perhaps from a rational, logical standpoint it shouldn't be implemented. Partnership opportunities with other ICs we've talked about, and that's really important to get at. LC integration is part and parcel in all of these efforts, and one of the geniuses really of genome in my mind is that it's made LC integration explicit, whereas in other dimensions and other ICs that hasn't been the case. What type of evidence is needed? Do we really need randomized controlled trials to roll out everything? Can we do implementation with evidence development? Are there other intermediate endpoints that can be used? The duration of programs is important. You don't want a situation where you create a program and it lasts forever. They have to be able to morph to meet new challenges. Where that's coming up in a very practical sense, and one that is kind of open for discussion, is the fate of Caesar. It sounds like there will be some kind of extension for a short period, but the fate of or existence of a Caesar 2.0 I think is a totally open question. Then finally, the really difficult thing in all this is you sit here and you listen for a day and a half, and you hear fantastic ideas that run the gamut, and they all almost all sound great. Unfortunately, then HGRI is going to have to make decisions that involve prioritizing, and opportunity costs are a really big deal. If you do one thing, you don't do another. With that, I will kind of, I'll end, and this isn't a group that is probably reticent to talk, so I don't think we'll have too many awkward silences. What are people's thoughts about both the goals and especially the organizational approaches to achieve those goals? Maybe I spoke too soon, okay, Howard. Jim, there's been a lot of question about how far into the clinic HGRI should go at all. To me, the reason why HGRI should be in there is because of the lack of direction that there is currently. The marketplace is not currently solving it. It's providing solutions in terms of sequencers. The groups that are providing the services are marketing based on very little substance, even things that we use, literally at our center, doesn't always have the greatest substance. It's the Wild West. As someone who was born in the West, the Wild West was not a safe place. It is very wild. Yeah, it's very wild. The Wild West was not a safe place. It was not a place you could have a well thought out lifestyle. What future do we want? I think some of these examples need to be done. We don't have to drive the whole field, but at least some of these need to be done. I would chime in with something that Sharon Platt has mentioned on many occasions. That is that once in a while you hear this idea that, well, everybody's getting sequenced. Everybody's going to be sequenced. Why does an HGRI have to have a leadership role in that? I would compare it to many things that have been implemented in medicine without data, without rigorous evaluation, that then we much later found out probably wasn't a great idea. The time to try to develop evidence to understand whether things are worth implementing seems to me to be early in the game. In my mind, that's a critical role for an HGRI. That's one reason that I'm fond of the idea of outcomes research with all the caveats that Eric brought up earlier, that we have to be careful. We aren't, you know, that isn't our bailiwick, and we have to partner with the right people, et cetera. I do think one thing we probably didn't talk enough about is the incredible potential for new types of sequencing or types of sequencing technology to drive clinical medicine. So if we think of non-invasive prenatal testing, which is now a huge industry, and rapidly changing how prenatal diagnosis is done, that was based on some very important or difficult originally bioinformatic approaches to next-gen data. And so I think we probably didn't talk enough at this meeting about are there other such really new or different approaches to sequencing or sequencing different clinically relevant tissues that could drive medicine that doesn't involve, you know, is it the early stage, not at the clinical trial stage that maybe NHGRI doesn't want to get into? So Jim, just to follow on on that, what's a little bit scary to me is if we look at non-invasive prenatal diagnosis, in fact we've gone backwards in what we could do. So when that was introduced by industry, it could fine try something 21, but it missed all the copy number variants. And even 13 and 18 couldn't be found. So we went backwards in what the technology could actually do. And that's what scares me about industry jumping forward with some of these technologies, not doing the science. Right. So I guess what I'm hearing is you feel there's a role for NHGRI in figuring out how these things perform and how they can really inform clinical care. Yeah. We shouldn't think, we should not forget also about the area of pre-conceptual screening, so we stop having to make the diagnosis of at-risk couples for lethal newborn diseases by having them have a kid with a newborn disease. Right. And for example, the CSER project at Kaiser is doing exactly that. And when I think about the penetration of various genomic technologies into the population, that's got to loom large. Even though 50% of pregnancies in the US are accidents, right? So immediately only half of pregnancies might be up for that. So the other thing that I would throw out there that we don't talk about much, my guess is that when it comes to genome scale or at least hundreds of genes sequencing, the oncology world will dwarf the germline analysis world. That's my guess in the next 10 years. Maybe we'll be in a world where everybody's getting their germline genome sequence, but I suspect we will be in a situation where most tumors are sequenced, and we don't tend to talk about that. And I don't know what the role can, you can speak to that. Absolutely. I mean, we've been chatting about that. I mean, the numbers of normal TCGA alone is huge, but the numbers at the centers right now are extraordinarily large for normal tumor comparisons. I was going to make the comment, though, in the context of this vision for NHGRI going forward. I'm on the advisory group for CSCR, so I have a good insight into that. I was very impressed at the meetings. But at least speaking in this format, with NHGRI and thinking of going forward, one of the things that I remember fondly about the early NHGRI LC projects, this is in the 90s, where you could be at the table of these consortia if you were not NHGRI specifically funded. And I've thought of this because the CSCR is operating as a very small, enclosed unit, and I may have a chance to say this in my advisory role at some point, but there are lots of others doing what's going on at CSCR. And I can tell you just among chats that I've had with folks here, many more exomes to communicate incidental findings have happened at some institutions in the Northeast, okay, that have happened in all of CSCR to date, and many of these are peer reviewed and funded IRB approved studies. So you may want to think, and this is what Eric was talking about, Eric Lander, in terms of the synergizing with other institutes in the context of the other groups, think about the synergies with NCI and other supported investigators doing the same sorts of things as you're developing these consortia, so that the consortia are not limited to the PIs of these few NHGRI funded projects. This will give you a much greater power to do this type of translation and not have to pay the dime, because much of this work, for example, at our own institution, we just have to do this. We have this incidental issue, it's not an option for us. And I think this is true of the other large cancer centers. So programmatically, I think with a relatively modest support, you could broaden the attendance and the involvement in CSCR 2.0, which is one part of the landscape, but I think a very important part, as we go forward and think where we're going to be over the next five years. Yeah, I just, I feel like we really do need to do a better job of engaging the oncology community, because I really do think, you know, if you think about a path towards clinical utility, tumor sequencing is you can really envision a path there, I think in many ways much easier than you can in a lot of germ lines. So I'll take that baton as a member of the oncology community, as well as the germless community. But the two areas that we've seen, and I think this was discussed in the session yesterday as well, that are the biggest areas of need where potentially NHGRI could have a big impact are first, the, yes, genomes and targeted panelists that are being sequenced, but there's no right now cohesive interpretive framework for, or even hypothesis, you know, a sort of candidate interpretive framework that we can hand to an oncologist and say, use this framework as a decision tool to think about how to enroll patients on clinical trials based on the genetic information. Right now, they get the data in sort of a scalar report, and I kind of like the challenge yesterday, it basically is a PDF, and then it's more or less, with some descriptions, but then it's more or less up to the oncologist to figure out what to do. But in fact, there could be an entire, and this is true for not just cancer, but for other diseases as well, kind of a, one can imagine a Bayesian framework based on disease characteristics, et cetera, that sort of pushes up or down how seriously you take genetic alterations in the context of that particular patient, which is different than what we often do. We sort of just take it as a yes-no with ignoring kind of the disease context. But actually, clinical medicine is really based on context. You have the context, you have your patient in. You know, the test that you've done is either relevant or not relevant, depending on the clinical context. So we need a formalism for how to think about that, and I think that's something that NHGRI could conceptualize, could push. And am I right? You know, my impression is that a lot of the somatic sequencing efforts, like Foundation Medicine, for example, Commercial, that they actually eschew analysis of the germline, right? They don't want to go there, right? Well, absolutely. And actually, and for good reason. And because I think this is exactly the thing that comes up. When you look at possible or likely pathogenic variants or variants that are putatively damaging, I mean, it's one of these things where, unless there's a family history, as everybody in this room knows, you don't even, you want to avoid reporting something because you could easily open a Pandora's box, especially in the setting where you're taking care of a patient with advanced cancer. And now you're raising questions about the family. You don't even know the family history. So you'd like to go in the other direction. You know the family history. It gives you kind of Bayesian priors about things. And then when you see stuff, now you're prime. You think about it differently. The other area that's just emerging now, and it relates to Aviv's challenge talk last night, single cell sequencing. So when she described the cell atlas, that was sort of the normal cell atlas. But one can easily imagine a disease counterpart to that cell atlas. And there are a variety of diseases where the pathogenesis is still obscure. And it really can end up being cell state readouts that give us a clue how to think about both the biology and potential therapeutic implications. And you could imagine, this is something where it's, this is a longer term view, but the technology has arrived to allow us to start thinking about those kinds of questions. So that's an area that could be very exciting, very catalytic from NHGRI as well, but it's a longer term view. Right. Heidi. So in a lot of these clinical implementations, a lot of the clinical arenas, whether it be the physicians or the laboratories, look to their professional organizations to, in the end, dictate how this happens. And I think one of the weaknesses I have seen a little bit in some of the NHGRI funded efforts is really ensuring that there's a connection to the professional societies so that when the guidelines are developed and dispersed, that then to some extent dictate how things get implemented, that there's really a strong experience from these feasibility and demonstration projects that is informing the development of those guidelines. And I think really engaging in a much stronger way, and it's not to say this hasn't happened at all, but I think in a deeper, stronger way to help and have NHGRI actually participate in the development and the initiation sometimes, because a lot of these professional guidelines get developed a bit in silos, where it's the pathologist versus the geneticist versus the cardiologist, and an ability to bring these clinical communities together around genomics in a more cohesive way I think could be beneficial. Yeah, and I would just add that from the standpoint of individual programs, it probably is important for NHGRI to push those programs a little bit for where guidelines, where some consensus can be arrived at so that then the professional organizations have something to sink their teeth into. So we've got, yeah, Robert. So I think for the CSER consortium, which is the one I'm most familiar with, the most exciting scientific results of that are not even been seen yet. I think each of the individual nine centers in a bottom-up fashion have a specific set of questions, which are gonna be truly exciting and I think hopefully rock the country a little bit as these come out. And I disagree slightly with some of the formulations we've heard in that we can't hold back genomic sequencing. I think just as the way we've been surprised by the adaptation of sequencing technologies, we're gonna be surprised by the diffusion of sequencing into medicine, and we're gonna rail against it, but it's gonna be there. So what is it that we can do from the bottom-up? And I think that we can do novel ways to educate physicians, novel ways to create these sort of consortia-based registries, novel ways to automate variant classification, and things like that, that might function better from individual sites and even our one-type genomic medicine initiatives. So although I think themes can be set, I think we ought to rebalance slightly in the direction of investigator-initiated genomic medicine initiatives. Yeah, so I wanted to make sure and get there. I would mention that I don't think calling for evidence early on means holding back, right? So I agree with you that things will permeate the field and the market will drive things, but the earlier we can get a handle on evidence, the better we can stop an inappropriate train from gaining momentum. Again, I think of PSAs. So your last point I think is one that we should get some general feeling about. There's a constant tension about how much should be top-down directed at NHGRI, how much should be bottom-up investigator-initiated research, the history of the genome institute, of course, makes abundantly clear why there's a very heavy top-down emphasis, right? Start out with the Human Genome Project, it had to be. And I think there's a constant kind of reanalysis and recalibration of where that should be. Do people have, so Robert has expressed one opinion, and I certainly have sympathy with the idea that some of these hard decisions that are, after all, to some extent impossible, because we can't tell the future, could be eased a little for the NHGRI by basically enlisting the community by slanting a little bit more towards investigator-initiated research, but what do people think about that as a general organizational principle? Should we double down on the RFA's and top-down stuff, which have been very successful? Should things be recalibrated somewhat the other way? Yeah, sorry, I just wanted to remark that when I think about this question, when I think about this question, I think, so are there a variety of approaches that need to be explored? That's sort of one axis. And the other is, is this gonna get us where we need to go, or is it gonna be too diffused? So if you can think about that when you answer that question, it'll be really helpful, I think. Gail. So I think CSER actually has a very interesting model in that each site has a very different project that they are taking charge of, but yet there's this incredibly active group of work groups that are working on common problems. What about sequence quality? What does the report look like? What are the LC issues? And I think that nice balance, if you have everyone working independently, you lose that ability to learn from each other. And that's been an incredibly valuable component of CSER and then develop a common set of understanding to present to the community. So I like that model of the projects can be very independent in their aims, but that the investigators are tied together and thinking together in ways that benefit each other and then the global community. I just follow up on that. Jim, can I just follow up on that quickly? I completely agree, and I would say moving forward, probably the one balance we might want to strike slightly differently. I think we've all actually really appreciated that we could present the clinical challenge that we wanted to address. And the nine projects are all very different. It would have been, now that we've done this and we've all been involved in these working groups, it would have been good to early have identified common elements and common data structures just to maximize the sharing. But I think it's been incredibly powerful to allow people in the community to say, this is an important clinical question where genome scale sequencing could have an impact and not to have a central RFA that says, we've decided these are the, right? Because we all came up with different settings. I was just gonna echo the same thing. I think that's the right model, is to have a structure and some targeted questions and then allow investigator-initiated solutions around it. But if you just do individual RO1s, you have a scale problem and a cost problem that I think is gonna make it quite difficult to be competitive. And that gets to the other dimension of scale that I didn't put up there, but kind of the other axis is, how many people do you need to sequence to answer some of these questions, right? And we see even within these different projects some tremendously different scales. I think in CSER we're probably talking about, well, certainly it's gonna be fewer than, certainly fewer than what, 7,000 people among the nine centers, whereas others like the next iteration of Emerge we're talking about, they're saying 25,000 or so. So is that a question that you feel is answerable? Is that one that is addressable at this point? How many people denotes large-scale sequencing, right? Is CSER a large-scale sequencing endeavor, right? David's shaking his head saying no. Or does scale depend on what you're trying to measure, right? I think the other dimension to this, which is what we were talking about earlier is, how does NHGRI position itself to lead in analysis or at least inference from sequence genomes when it's maybe gonna control five to 10% of the sequencing capacity, right? How does it create incentives for data sharing or analysis tools or whatever that can, you know, eke out from the other 90% even if it's half of that or something like that. And I think that's part of the, you know, there's this huge amount of sequencing that will presumably happen in the context of care over the next, right? And years, and it would be a tremendous waste if we couldn't figure out how to make use of that in broader context. Yeah, so I think that's a perfect way of putting it, right? I think that we, NHGRI will account for a small percentage, but we have to make sure that it's a strategic quality kind of sampling, if you will, right? So it makes it meaningful. David, did you? No, I was just processing this discussion. One of these that strikes me that is going on, it was commented on earlier, and I'm wondering how it relates to the current activities. It may be, it is, and I don't know is, it seems like the thing that's happening really on its own is this prenatal sequencing. And it seems, my understanding is it's ramping up, and because- It's my understanding. It's my understanding, without actually NHGRI making it possible or without necessarily this community, maybe some people are, it's an interesting test of what happens when actually the market starts to succeed, and it's no longer can we promote the use and can we pave the way, but actually it just starts rolling along because there's some value proposition. What's our role in that? What are we doing to respond to that, if anything? Is anything needed? I'm just curious, because what if we found ourselves like three years from now, and it is happening, are we going to even have an evolving system that will make sure we're doing the things that matter as the use cases develop on their own? I would argue it took off on its own because there was this really clear, transparent need that people wanted it, right? Whereas I don't think there's a lot of sequencing where people don't have as, they aren't as driven to get that information. But at the same time, right, you can imagine that people would like some degree of clarity as to, well, what is the value of this sort of self-re-DNA? How well does it compare? The information that's out there largely produced by the companies, so of course they've got incentives. And what we're finding, of course, is that it isn't quite as good as we thought was, right? So that's a very good example, right? If one could create incentives for Netera to share their data or something like that because they would see value in opening it up for the community to develop tools so that it could improve and so on. The research, though, that's actually trying to... So it's occurred to me for a while that the thing that we can do is ensure quality. One way to ensure quality, it's great it's getting picked up in the world and often that means good things and sometimes it means that the lowest common denominator wins out. What this community could do and what I think some of us were thinking back doing at our place and it'd be good to do together might be to have some challenge problems that actually put out 10 genomes where we know the right answer. Yeah. And we just say, anybody who's hanging out at a shingle right now interpreting genomes, go interpret these genomes. But that's how you do. You may find that people are pretty embarrassed or you may find that some folks won't sign up to do it in which case they should be pretty embarrassed. So we can play a role of setting high standards. In fact, we've been talking about setting a challenge problem if others wanted to join in and a couple of groups wanted to set some kind of an annual QC exercise that would let groups know and thereby let patients know whether you should trust Fly By Night Genetics, Inc. to do this. Probably not. Well, yeah, probably not, but they probably have a better branding than that. No, or Google, or Google Genetics. Or Google Genetics, or anybody genetics. Yes. If we did that, we get to set a very high standard for the whole field. So I would like to propose that folks who want to discuss that, we could get a little group, whether NHGRI does or doesn't wish to be formally associated with that as it might be viewed as a regulatory role or complicated or might induce companies to go complain to Congress. Or on the other hand, it might be a really good thing for an NHGRI. I think NHGRI should work out whether it wishes to be in the middle of it or not. But certainly a handful of groups here running a well-defined challenge like that would ratchet up this field's quality a lot. Yeah. And you could imagine even sort of a component, like sort of like the genome in a bottle consortium that NIST is sort of setting up for sequencing, but sort of an interpretation. It's an interpretation one. And of course, it's different in context. Cancer is a very different problem than a newborn, than a newborn with some disease in the family. You could also imagine if we had a gold standard, eventually doing the same thing for variant adjudication. Right? Yeah. So there's an organization, excuse me, there's an organization doing this, the critical assessment of genome interpretation. It's basically they keep recused genome and send it out and different groups will interpret it. And Steve Brenner at Berkeley has run this for at least two rounds and it's ongoing. It's not been particularly funded. I know they've approached NIST about funding a couple of times. It's based on the CASP model where there's a sort of recused set of reference information about the genome and then. So it could be that by expanding a little bit the set of groups behind such a thing, it could gain traction. So Steve Brenner's running that. Steve Brenner. Steve and the dream competition as well. Steve's involved in that and they're coordinating with that effort. There's a great somatic mutation calling challenge going on right now that was supported by TCJA, ICGC, and deployed by the Dream Challenge organization and the Global Alliance for Genomics and Health is also working with them as part of a benchmarking and ongoing competition activity. Yeah, whether dream is the right branding is an interesting question because it may be that you want payers thinking about this and whatever, but it's the right place to start. You should get a conversation going. And the X Prize people had talked to some of the community about the adjudication issue, but that ultimately is a lack of a gold standard. I guess just one more comment and then we're out of time. I was just going to raise the point that I think came up is that I think one place that we really could have an impact is assembling cancer genomes and thinking about the large-scale structural variation in them. And it's a very, very hard problem and a very fundamental genomic level to put these things together. And I think that we could develop ways of doing that. All right. So it's all dollop NHGRI's problems now. That was easy. Thanks.