 So, for today, for May Council, we've invited presentations on the CESAR program and the Mendelian Genomics program. Jim is here to present on behalf of the CESAR program. And of course, you know him as a member of the council. He's also a grantee of the CESAR program. And for the sake of people on the web, Jim is a member of the advisory, our advisory council, and the Bryson Distinguished Professor of Genetics and Medicine at the University of North Carolina. So, Jim, floor is yours. Okay, great. So I think this does flow pretty well from the last conversation, and I'm trying as best I can now to take off my hat. And this, the information here has really been the work of many, many, many people so I'm just trying to kind of present where we've been with CESAR and what the needs and challenges and opportunities are going forward. So, I wanted to start with the heat map, both because I think it, we come back to this all the time because it nicely summarizes the general gestalt of where we're going, but also to point out that as we move towards the right, I think we see a qualitative shift in the demands and the needs and the challenges of research. And what I mean by that is that we really, there's an inflection point somewhere when you move from basic science to clinical care because there are very different demands on it. There's no question that basic science is the foundation of medical practice, but the endeavors that the two things, those two endeavors demand very different approaches. Theory alone is really insufficient to guide clinical practice, and that seems a lesson that we should have learned a long time ago, but I am absolutely amazed at how poorly we oftentimes learn it. And the balance between premature translation and inappropriate delay in translation is a really difficult one to achieve, and the stakes are very high. On the one hand, when we prematurely apply things into the clinical sphere, we regularly cause profound harm. We cause it to patients and we cause it to society at a minimum through increased cost. On the other hand, when we miss opportunities and we don't translate things quickly enough, we obviously miss opportunities. And then finally, there's a big difference too because medical practice inherently involves individual values, which really can't be gained side in the end. And we have a long history of premature translation. So in the therapeutic sense, it wasn't too long ago that reflexic hormone replacement therapy after menopause was absolutely what was done. There was a time when we gave anti-arhythmics like IV lidocaine to individuals who were peri-MI, and we learned, of course, and same with strict glucose control and diabetes. These were all compelling ideas. They seemed like no-brainers, and yet we were killing people when we did them. And it wasn't until we went back and studied them in a systematic manner that we understood that. In screening, we have seen millions upon millions of men subjected to an uncontrolled experiment that we're only now realizing was probably not a very good idea. I would tell those of you who would think that whole body MRIs would be a good screening procedure to have. You probably don't want to do this. And I say this because I think there is, in our minds, oftentimes a really compelling rationale for the implementation of genomic medicine, and yet we really need data before we do that because we make very serious decisions when we apply genomic knowledge. When we are wrong and we give people false positive results, we subject them to unnecessary surgery, years of unnecessary screening, a premature end to a diagnostic pursuit, and therefore they cannot avail themselves of the real answer. And when we give them false negatives, we forego necessary preventive and therapeutic modalities. And all of these ills are complicated and amplified by the fact that genomics is obviously has to do with families as well. And they make decisions about family planning and abortion, and not to mention the psychological damage of misinformation. And finally, the other kind of qualitative difference that I think is really important as we consider what are the needs of a program that is primarily aimed at clinical investigation, clinical implementation, and ultimately clinical value, is that our values regarding the same genetic information vary. So I love asking this question to people, so let me see a show of hands. If you look at this list of really bad diseases, which I think their very existence has profound metaphysical implications, but we won't go there, if you carry a mutation that essentially guarantees that you will develop a severe, unpreventable, untreatable disease by age 60, would you wish to be told so who here would say yes, they want to know? Who here would say no? Who here doesn't know? I think that I don't know is have it, and you're actually right, because we see that people don't do what they say they're going to do. And the bottom line is we need to understand how heterogeneous values can be accommodated, because they are inherent to the clinical pursuit. So this all gets to what the underlying rationale of CSER is. And what I want to do is frame for a minute the opposing view. And it's a view that we were just discussing. I think that David articulated it, Jill articulated it, Eric Lander at our last council meeting articulated it. And really in a way it is, look, we have lots of sequencing going on. It's already being performed. It's getting really cheap. Why not just introduce it into clinical care and learn that way? And that's not an unreasonable position, right? But I think that there are some very strong arguments to not go down that road. And really that's what CSER is predicated upon. And it is that the unsystematic introduction of new modalities into clinical care, no matter how good of an idea you think it might be, generates poor and misleading data regarding its appropriate use, its utility, and its applications. It also shifts costs to an already constrained medical system. If we start doing these things as because hospitals are competing to do it and everybody expects their genome to be sequenced, we really haven't done the medical system any favors. And I would emphasize that nothing is free in the world of clinical medicine. So even if a test is free, it's very, very expensive if it's not the right test. It makes patients de facto research subjects, and that's not what our patients are looking for when they come to the doctor. And privacy concerns are especially resonant in this field. And finally it undermines the systematic collection and sharing of both genomic and phenotypic information, which is critical to understand. So this is from the original RFA of the CSER program to address critical questions about the application of genomic sequencing to clinical care. And really in the end, what we would like to do in CSER is establish best practices for the translation and the implementation of genomic medicine. So it's trying to sum up the major focus, the major goals of the CSER program, I think it's fair to say, that one would be to figure out what are the patient characteristics that signal potential utility of this test. This is a complex test. We don't get MRIs on everybody who walks into the doctor's office, nor should we. We have to figure out at some point when it makes sense to get an MRI, even if that MRI were free. We need to understand diagnostic hit rates in different populations. We need in the tumor arena to understand when somatic analysis is informative to guide therapies. How should we analyze large data sets in a clinical environment? Things like quality control, timeliness, confidentiality, those become exceptionally important in this realm where they aren't as important in a much more basic context. There are special considerations in different populations. For example, minorities and traditionally underserved groups, and that's not only because there are different values in their different social contexts, unlike Howard, I'm a big fan of LC. Different allele frequencies in different populations, they really matter. And I know from our Sino conference, we'll see an unusual allele, and one of the first questions we have learned to ask is, what is the ancestry of this individual? How do we deal with the plethora of highly heterogeneous, non-target DNA, incidental findings that have you, that are going to transpire when we perform genome-scale sequencing? And I would add that gene discovery is part of CSER, but it really is incidental, no pun intended, to the main goals of CSER. So turning for a moment to the aggregate accomplishment, CSER has been in existence now for I think it's two and a half years. One is simply the implementation and integration of genome-scale sequencing into a number of different clinical arenas. Secondly, the design and implementation of a variety of, and I put in parentheses, relatively efficient analytical systems and workflows. This remains though an inherently difficult and time-consuming and resource-consuming endeavor, and it's something that we really need to work on if the dream of using genomic information in any kind of routine way is to ever be realized. We've made progress towards open-source analytic workflows, the implementation of facile reporting mechanisms and formats to both providers and patients, because explaining this to both is challenging. On a more nuanced level, for example, the demonstration that Holoxon sequencing from formal and fixed paraffin-embedded tumors seems to be equivalent to that from frozen samples. And here are just some numbers. Total enrollment in CSER as of about two months ago is 1,500, with about two-thirds adults and a third children. You can see the racial and ethnic background that we do need to work on, I would say. And the total sequence that is as a subset of those enrolled is just over 1,000. The majority of those being germline-focused and a minority of those being in the tumor-specific efforts. I couldn't agree more with David, so I'll skip over how many presentations and publications there have been. Here is one representative sample of tumor results. And I think the really important thing here is to think about what these results mean. Only 2% of the case is there a slam dunk while there's a mutation in this tumor that is of established clinical utility. On the other hand, in about a quarter of patients, there are mutations of potential clinical utility. Now, there's a big gap between those two, and some percentage of these will truly be clinically useful, and some percentage will not. The return of results and the clinical relevance in the germline reveals that somewhere between a quarter and a third of these individuals have had positive diagnostic findings. That is, these are people who did not have diagnoses before, didn't know what they had or why they had it, or even necessarily if it was genetic, and whole exome sequencing gave what appears to be a pretty good answer. We see that the incidental findings that are generated are somewhere around 3%. If you use the ACMG list, if you use a little bit of relaxed criteria, you get up to around 5%. This is a representative sample in which one sees whole exome sequencing being applied in a variety of different clinical settings. What one can see is that we're beginning to see some divergence. For example, I think many of us have been somewhat surprised that cancer, that is, individuals who seem to have a dramatic predisposition to cancer, presumably because of a germline mutation, we're not really not getting a very high rate of answer. On the other hand, individuals with hearing loss and retinal disease, in about half of those cases, we're getting a likely reason for why they have those. I think there are some important lessons here for guiding the clinical use of this complex technology as we go forward, but we're still in early days of those results. I think that there has been a huge and important effort on the LC-related component of implementing this kind of information, and early results in this sphere include the whole issue of informed consent, really what that means, and there have been some efforts to really formulate what informed consent should mean in a genomic test. In addition, there are specific challenges related to clinical genomics, like what are reliable measurement tools, how do we develop ones that are needed, et cetera. The incidental findings, as Terry mentioned, have been a focus, and they're really a focus in a way, because of that question I just asked you, and the fairly dramatic divergence just in this room that was shown by the show of hands. That is, we've got to figure out what we tell people and what we don't tell people and what we look for and what we don't look for, and who pays for that when we do this kind of analysis. We know that between 3% and 6% of people will have what we consider at some level quite medically actionable incidental findings, and the degree of choice provided to patient has been a contentious issue. I would say that the CSER sites have been really instrumental in shaping this debate, and that the ACMG's recent shift in the policy or adjustment in the policy was really driven in large part by the CSER endeavor. We're beginning to accrue data addressing what patients really want to know regarding this, because in the end, with those things, that's an incredibly pertinent issue. If you just ask people, what do you want to know? They say, I want to know everything, but then if you put even minimal real-world kind of structures in place and ask them, for example, to make a phone call, that the desire for that information dramatically decreases. Like any economic question, right, it's really a question of competing interests, and we're always judging in comparison with other things. CSER's actively collecting data on measures of patient satisfaction, distress, et cetera. I want to get in the last few minutes to what the challenges and the opportunities are going forward, and the very first one, in the end, the only really, really important one is outcomes, and that's what we ultimately have to get to. We've erected this integrated infrastructure to implement genomic medicine, and it's been, you know, I think, very successful in that sense, but the central challenge going forward is to assess whether this actually improves patient outcomes. We need to answer that if it's going to be a useful quiver in our, you know, in our armamentarium, or arrow in our quiver would be the metaphor, I guess. So in terms of outcomes, what are we interested in? Well, we're interested in morbidity and mortality, obviously. We're also interested in patient satisfaction, values, their well-being, and we're interested in economics. Many people believe that prevention saves money. It actually hardly ever saves money. It's very expensive to prevent disease because it's cheaper if patients die, but of course that isn't the final arbiter of what we do, and we need to figure out not so much do we save money, because we probably don't, no matter how attractive it is to think we do, but we need to figure out how much do these improved outcomes cost, right? We need to explore those outcomes in diverse settings, including ethnic, socioeconomic, clinical settings, and I think that there is an important role for CSER and for the NHGRI as a whole in all of this because the market doesn't really focus on outcomes. The market focuses on uptake, and if you can convince people to have a PSA, if you can convince people to have whole genome sequencing, the market loves that, whether it actually improves outcomes or not. Going further on outcomes, we're going to need new approaches. We will need qualitatively different approaches. We'll need new cohorts of patients with ranges of diagnoses. Who is really going to benefit from this and who isn't? We need to engage in some version of comparative effective analysis. I don't really know whether it makes more sense in this situation or that situation to apply whole exome, whole genome, or multiple gene panels in a given situation. Again, when I say this, people immediately think I'm saying, oh, we shouldn't do whole genome sequencing. We should do lots and lots of whole genome sequencing, but as we roll it into the clinic, we need to be careful to roll in the right things and not necessarily assume it's going to be, for example, the most comprehensive test. And I think that at best, this kind of effort in CSER will then feed into efforts of things like ignite and emerge. We need systematic quantitative extension of existing cohorts. And this is a rather boring one, but nevertheless a very important one. You need a lot of patients to answer these questions. You need longitudinal follow-up of those with positive results. How did it impact their care? Was there dissemination of family members? You need longitudinal follow-up of those with negative results. Everybody who we give a negative result to, those are actually the interesting and the difficult results to give back to people because we have to try to interpret, did this negative result mean you don't even have a genetic condition or does it mean we aren't smart enough in 2014 to figure it out? We need optimal approaches to dynamic reinterpretation. Our patients do not want us to get a negative result and quit there. They want us to find innovative ways to reexamine the existing data and come up with an answer. We need to have outcome measurements in the LC arena too. And I just have to say I didn't mean to pick on Howard because he is very committed to LC stuff. I listened to his counsel. At least I want people to live. That's right. As opposed to me, let them smoke, right? So, analytical approaches are another huge horizon that needs exploration. We've kind of gone by mouse dictum, which I probably shouldn't say here in this time, let 100 flowers blossom. And that is each site has established inventive ways of analytical workflows and engines to analyze these genomic data. And we really need controlled investigations to determine the optimal approach. And it may not be the same in every clinical setting in which genomic approaches are examined. We're going to have to make difficult trade-offs between sensitivity and specificity. And a good example of that is, do we start with a gene list that maximizes specificity and that's predicated upon what we know about that patient's phenotype? Or do we take a gene agnostic approach, which arguably will maximize sensitivity but almost inarguably decrease our specificity? And I think that's likely to differ in clinical contexts. I think that in the realm of intellectual disability, one can begin to look at the CSER data and make a persuasive argument that a gene agnostic approach is right. Whereas in the cancer realm, we might be far better off with a gene list-driven approach. But we need to investigate those questions systematically. We need to do this especially because we have limited time and resources in the clinical setting that go far beyond just funding limitations. So variants of certain significance in my mind are the single biggest challenge to the implication of genomic medicine. And you see this over and over when you talk to people in the CSER site, when you look at Terry's Emerge presentation, when you sit in our sign-out conference. And I think it's because everything comes back to evolution in the end. We're evolutionarily predisposed to tolerate false positives. That's what we do as a species. That's why we see the man in the moon. And that's very, very dangerous in a space as large as the genome. Coincidences are going to happen all the time. And we're going to hurt patients if we don't figure out better ways to adjudicate these variants of uncertain significance. And if we don't maintain an appropriately high level of skepticism. Because if you give me a variant and you give me a phenotype, it's like six degrees of separation. I can always come up with a story that will connect them. Our protocols are inadequate. The stakes are very high. And what makes it especially hard in genomics is we rarely have gold standards for adjudication. If you get a newborn screening result that suggests that a newborn has, say, an enzyme deficiency, you can typically go and you can adjudicate that by, say, measuring enzyme levels. You can figure out if you have a false positive or not. We can't do that in genomics. And that's a huge need. And I suspect, because we're not going to be able to attack all of these things functionally, that much of the answer will come from statistical approaches like calculation to mutational burden and that type of thing. And thus, we're going to need large numbers. There has to be planned overlap between, for example, the clingent efforts between Caesar, between a merge. And we have to figure out which of those efforts is best designed to do what. We need integration and sharing in the future. The only way we're going to get real progress is through really seamless integration and sharing. And we have to develop formats that facilitate this among all kinds of different entities, commercial labs, universities, different clinical settings and vendors, et cetera. And we have to incorporate data science to meet this expected kind of crisis of scale that may occur as we get more and more information. We also have to do it while protecting privacy and complying with HIPAA. And I would argue that the market is really poorly equipped to integration and sharing. And in fact, as we've seen with the myriad story, it's often antithetical to sharing. And I think that that is a real problem if we don't address that in a proactive way. On the other hand, I think it is well adapted to a consortium type of model. We need to figure out how and when to integrate genomic data with other omic type data. I think we should do it where clinically promising in this kind of context. And that's a really tough question. It's in microcosm or in parallel really to the question of whole genome versus whole exome, et cetera. And we have to integrate with training programs if we ever hope to have people who will interpret this. We need rational and targeted expansion of the group sequence, new diagnostic categories of patients. And I think that while, so CSER has dipped its toe into healthy individuals, most of CSER has been largely focused on individuals who are sick. And I think that going forward, what we hear from CSER sites over and over is that there are opportunities to apply genomics in ways that may ultimately benefit outcomes of human health in the general population. For example, the Kaiser group is looking at carrier screening for interested individuals. We have a lot of interest and have a pilot program looking at targeted sequencing of selected genes in the general population that strongly predispose to severe preventable disease when mutated. And we need to facilitate the longitudinal use of this information through the lifespan. Someday we really will get to the point where you can get your sequence and it'll be worthwhile 10 years from now, 20 years from now. We aren't there yet. Think about, would you trust a sequence from 2011 versus one from 2014? The answer would be no. But we will get there at some point and we're going to need to figure out how to synthesize that. We need to expand venues and populations to ensure broad diversity beyond the academic setting when it's warranted. And we need to investigate the cascade impact on family members. And finally, I would just mention, and this gets to the point we were just discussing about, what do we mean by genomic approaches? And NHGRI is going to have to think about this hard and is going to have to grapple with this. It's my own feeling that clinical genomics does not necessarily equal whole genome sequencing. Again, I want to put up a giant asterisk there saying that, yes, we need to do whole genome sequencing on tons of people. And we need to do it systematically. They need to know their research subjects, et cetera. But in the clinical setting, focus testing is usually optimal. When somebody comes in with a headache and they need imaging, we don't get a whole body MRI for very good reason. Unnecessarily broad testing can lead to waste, defaults positives, and to downstream harm and increased cost. When I hear a lot, well, everybody's going to get their whole genome sequence because it's cheap. And it reminds me of that great financial, personal finance aphorism that an elephant for a nickel is only a bargain if you have a nickel and you need an elephant. And I don't think that whole genome sequencing is necessarily an elephant that all of our patients need. We need to do it in the research setting. We need to figure out what it means. But I think that NHGRI needs to be open to the idea that our genomic tools and massively parallel sequencing in particular may be useful for improving outcomes in targeted fashions. I'm not saying that's the case. I'm agnostic about that, and I think that we need to at least just simply keep it as a question. Cesar has appropriately focused on whole genome and whole exome sequencing. It's now well equipped to start to determine where the balance is. When do patients benefit from a more targeted or a less targeted approach? I think our early data probably suggests that it's going to be clinical context dependent. And then finally, I would just end with the observation that in all of our large scale sequencing endeavors, we're wasting some important information. We're not harnessing some really important stuff. We don't, it dismays me. I mean, I understand why, right? But I find it dismaying in 2014 that we don't even know the true penetrance for BRCA1 and BRCA2, the most sequenced genes in the history of biology, regardless of species, right? And there's a reason for that. And that's because we have ascertained those data in an extraordinarily skewed way. We have overestimated penetrance and we've underestimated new mutation rate for arguably every condition that we deal with in medicine. And yet accurate figures are gonna be critical to both the clinical and the public health application of this information. The good thing is we're collecting those data and it wouldn't be that hard to harness them by creating a registry for the reporting of off-target mutations. That is, when you found a BRCA1 mutation because you sequence somebody due to a retinal disorder, well, now that's unbiased ascertainment. And if you can get their personal medical history, their family history, and long-term follow-up, you can come to definitive answers about things like penetrance and new mutation rate. And I think that caesar and emerge are probably logical places to start such an effort. And it could be expand to other NHGRI mediated sites as well as non-research venues as well if those could be incentivized. So I will end there and I'm assuming I should sit down. Okay, all right. While we're transitioning, I can't help but tell a true story. And because... Oh, no. Because you may know it. So remember I told you in my director's report how there's been all these associated Smithsonian programs and the evening and stuff associated with the exhibit. And we did one that Jim participated in and happened to be one that he brought his daughter to, his grown-up daughter, and I met her for the first time and I'm sitting next to her during the event. And then Jim, in the middle of the event, did the thing he did and raised your hand and he gives all the different diseases. Would you want to know? Would you not want to know? So he did all that. And his daughter, it turns to me, said, you know, when I was growing up, he used to do the same thing at all my birthday parties. True story. I'm not a popular father. True story. So. Yes. Let's ask two quick questions. So the first is when you said you had positive result, possible and negative, what's your definition of possible? Because you threw that into positives somehow. And the second, just let me get my second question on them. And the other thing is, incidental findings, five to six people, findings five to six percent of cases, doesn't that seem like a lot to you? So let me, yeah, so let me take those in order. Okay. You touch on something really, really important and really difficult to define. And we've spent a whole bunch of time recently trying to define when you report out, when we report out a variant, okay? We really have these three categories. That is, we found the reason for your disorder, okay? We didn't find the reason for your disorder, right? Those are the two easy ones. The one in the middle is really hard, and that is we possibly found a reason. But then you have to break that down into what you mean by that. So that could be that you have a variant in a gene that you know if it was a deleterious mutation would cause that patient's phenotype, but you just aren't sure if it's deleterious. That's your classic VUS, all right? You also, however, see people who have a phenotype that fits perfectly with the recessive disorder. They've got a clear deleterious mutation in one chromosome, but you didn't find a second one. Well, did you not find it because you missed a deletion, et cetera? So that's a possible, all right? Another example would be, you have a clearly deleterious mutation. It's a truncating mutation early on in a gene. But you're just not sure if the phenotype that we normally associate with that gene really fits with that patient, right? So in other words, the very non-certain significance is a highly nuanced result. And what I was trying to represent there is that in that slide you're referring to is that it's kind of easy to come up with negative and with positive, but then there's that possible. And different labs have different standards. So one of the things that I think the Caesar groups have been good about is not inflating that. But you can imagine what we see in some of the commercial labs is they say, oh, well we have a very low rate of variance of uncertain significance. Well, yeah, that's because you're not reporting things out in an honest way. You're reporting things out as positive or negative when you should actually admit that you don't know. So your second question is, the issue of incidental findings, that was all, in some sense, everybody has an incidental finding, right? Because they're gonna have variance in genes that you didn't predict. What we're asking here is what percentage of people have what appears to be a pathogenic variant in a gene that falls into this category that is medically actionable. And the reason you see the variance from, say, two and a half percent to five or six percent is that different people have come up with different lifts. So there are some people who have a more expansive view of medical actionability. There are some who have a more constrained view. So it's all what you kinda wanna make of actionability. It's more the definition of actionability than the identification of mutation. Which turns out to be tough, right? It's not easy, yeah. I was gonna let this topic go, but you invited further discussion. So I agree with you, and I'm a strong proponent of CSER for implementation research. It's desperately needed. But then to take the step and commenting on Dr. Lander's statement and then going into a comparison with hormone replacement therapy, I think is just wrong. I mean, I think there's a difference between erroneous use of information. That would be the hormone replacement therapy versus maybe premature using, I think your work, premature use of information, which maybe hold exomes. Those are hugely different. Wait, let me finish. So I think, in my opinion, in terms of cost benefit or pain in the game, and also being an advocate for genomics going back to the previous discussion in Emerge, which I think we have to be responsible advocate for genomics in translation. I think there's little harm relative to the game of pushing these technologies into the translational setting. I think to hold back using, if we limit ourself to 100 genes, we're gonna make these grandiose conclusions about a very small proportion of the genome and we're gonna miss what is likely to be the most informative part. I think if you think- I fundamentally disagree with it. If you think we understand that the 100 genes are the, using your words previously, the most important genes, maybe we should rethink having the institute. Do we know the most important genes? No, no, no. No, we don't know the most important genes. I fundamentally disagree with you on this. I don't agree with you on one aspect. And that is- That's one of two comments, by the way. Oh, okay. Can I answer? All right. So wait. Go ahead. So I would say that you are overly optimistic about the responsibility, the level of responsibility that the medical profession has in its use of information. I think that there's an unbelievably seductive tendency to use information that we have to make more of it than we know. And think about the individual who is told that you have this variant and it causes X disease and therefore you need this, this, and that. And you're wrong about it. When you're wrong about it, you subject family members to unnecessary procedures, you subject that person. I agree with that. So I just, I don't see the- That's the reason we need better implementation research on the whole genome. I think what we need is we need research on the whole genome, okay? And when it comes to implementation, we have to be conservative about what we implement. And I think that we have to strike that balance. I would actually say, Eric's gonna kill me probably. I don't like characterizing myself as an advocate for genomics and medicine. I'm an advocate for better outcomes. And if we can demonstrate that genomics gives us better outcomes, then we ought to implement the hell out of it. But you know- So you think we're gonna have better outcomes in the medicine? Is the percent of the genome than 100% of the genome? No. But the problem is in clinical medicine, you end up hurting people when you implement things prematurely. No, that's erroneous implementation, not premature implementation. They're different. They're different. Well, but I would say premature implementation is oftentimes erroneous. And- I'll give you that. Can I just say, I think you're both right, but- You sound like my mom. I think you're both wrong. No, because James' argument about the whole body MRI whole body MRI is definitely correct because we MRI'd people all the time and we know what's in the other parts of the body and we're interested in the head. And the genome is a little different because if you can take the opportunity to- And again, I would say on the discovery side, we should do that. We should hold genome- Absolutely. In the implementation side, I think over time we will know better. But again, if we target just a few, then we want no better. So that's why I think you're both right. I think we need to target in order. In other words, I think we need to investigate the whole genome and we need to skim off what we know and we need to implement what we know and then we need to continue to skim off what we know. I just think it's, and again, this is just my personal opinion. I'm not speaking for Caesar. In fact, there are people in Caesar who would very much agree with Eric and not me. I just am, I think we need to be gun shy about what we do in the clinical arena because the stakes are very high. My second comment is not controversial. Oh, good. I just think we have to be careful about estimating penetrants from clinical-based samples. They're not necessarily independent of what we're trying to ask. You used the word independent. That's just not the case. Okay. I think we have to be very careful about estimating penetrants from clinical-based samples. Right. I'm just trying to find out where do we have relatively unscued ascertainment? I would say it's the population-based epidemiologic studies that are very deeply fined, but much more deeply fined. And that would be ideal, but I would also argue that maybe letting all these other data go to waste that could be useful, right? But with the limitations that you point out. So if I understood you correctly, you're basically working on the implementation side. You're not really working on the discovery side. So that seems to me right, that if you're working on the implementation side, you wait for the discovery before you do more implementation. I mean, that's, and as a consequence, I think you'll be able to discover what whole genome sequencing really costs. It's probably not a nickel right now. In fact, it's, because whole genome sequencing, the cost of it is not just doing the whole genome sequencing as we all know. It's all the sequelae that follow from it. And so you will get, and I hope insurance companies are interested in this, is a shot in time of how much it costs right now to do it in certain categories and how much it costs in other categories and what the benefits are relative. And then as more discovery happens, presumably, maybe it'll become more cost effective because the benefits will be much greater. But right now you'll get a slice in time kind of perspective. Is that accurate? Yeah, I think so. And again, I don't want to discount or artificially deflate the fact that discovery happens in Caesar. It does. I mean, we've found very cool stuff. We're expanding phenotypes and all that. But I think it's fair to say, and Lucia, correct me if I'm wrong, I think that the raison d'etre of Caesar is not discovery. It's figuring out how best can we use this new technology in to ultimately improve patient outcome. How do we implement it wisely? Because, I mean, that strikes me as much more similar to what some of us call outcomes research. Yeah. And so that makes, to me, perfect sense because outcomes research is very different from discovery. And that's why I put the outcome stuff in the last part of this presentation, which is, okay, what's the future? Because first you have to create the infrastructure and you have to kind of cut your teeth and figuring out how to analyze these things and et cetera. And then you have the opportunity to start looking at outcomes. The intellectual disability, the hearing loss and eye problems, I mean, those have been known for a long time to have huge amounts of genetic, heterogeneity. So what are you discovering or planning to implement that really changes what we already know about these disorders for the patients? Is it prognosis? Is it, it's not treatment, is it? Right, well, sometimes it is treatment. I mean, you know, I couldn't work this slide in because, you know, unsurprisingly, but you know, sometimes making the right diagnosis genomically has a tremendous impulse impact on treatment, not usually, okay? And I think that what we in some ways have to settle for in a majority of cases is that diagnosis is important, it's a lynchpin of medicine that helps people with prognosis, it helps people with family planning, et cetera, et cetera. But if you're asking, does making a diagnosis, is it a game changer? Not usually, right? It's just that that's the reality we live with in the 20, I would say this entire, you know, we'll be living with this whole century in medicine. We do our best, we make diagnoses, sometimes that makes a huge difference. For example, in long QT, you sometimes, certainly knowing the mutation and knowing the gene affected affects the threshold you have for whether you put an implantable defibrillator in, whether you use medications, et cetera. We have a patient where, who had for 30 years been confined to a wheelchair and crutches because she'd been diagnosed with hereditary spastic paraplegia at very good places. And Holexome Sequencing showed a mutation in the GTP cyclohydrolase 1 gene, which is responsible for DOPA responsive dystonia. Six weeks after being started on Cinemat, she walked into her appointment with no crutches, right? So once in a while it's gonna matter, but the fact that making a diagnosis is not a game changer frequently, I think is more an issue of where medicine sits in the year 2014, not a germane criticism of genomics. We need better, better diagnostic modalities. We also need better therapeutic modalities and hopefully the latter will come, right? Once we're better able to understand these diseases. But Dee Dee and then Rick is gonna have a question. Yeah, so you mentioned the need for other omics data. Could you talk about how you see that being implemented in a seizure context? Yeah, I think that probably the best example of that is in the realm of tumor sequencing, where for example, adding RNA-Seq and that kind of thing could inform whether a particular mutation actually is going to cause increased susceptibility to this agent or that agent. The rest of what I'd have to say would be hand-waving and I'll leave it to investigators who know more about proteomics and know more about looking at epigenetic changes to figure out whether incorporating some of that information into these kind of things could actually inform us in meaningful ways about patients. But I think the one where we may be poised to begin to make a difference would be in tumors and looking at expression data. Because that can be a real game changer in whether a given agent would be expected to have an impact for the better on that patient. So that's the big one. Thank you. Terrific talk. We all have the experience of seeing patients who come to us as the nth court of last resort. And one question I have for you is making a diagnosis that frequently puts an end to that futile cycle. And I'm wondering how you capture that as an outcome and the value of that to the medical system and to patients. Yeah, that's a really good question. I mean, I think so the term in our field is that you can end the diagnostic odyssey. And that's such a great expression, right? Because these patients are really on an odyssey to go from doctor to doctor and have millions of tests. We have not scratched the surface yet in documenting and trying to really figure out the true value of that. We've begun to be able to assess through just talking to patients, following them up, surveys, et cetera, what it means to them in kind of a personal way. And that's not trivial. But I think your point's a really good one. It does arguably have a beneficial impact on, for example, economics, et cetera, when people aren't getting unnecessary testing, et cetera. And I think that's part and parcel with what CSER and these other efforts need to do in documenting whether what we're doing makes a difference, right? And one of those differences could be, yeah, they didn't need all those other tests that they otherwise would have got. All right, thank you, Jim. All right, before they run out of caffeine upstairs, we need to get you up to the cafeteria. So let's reconvene at 320.