 So I just want to emphasize that while it may have felt negative what we were pointing out all the challenges, I just, I think that's just where we are. It's the dynamic nature of the data and the data isn't just a research study. It's actually being delivered in clinical care. So this is just where we are and I think the point of the discussion, I'd like to stay focused on well how do we try to solve some of these? Where are the gaps and are there ways that we can use existing cohorts in existing studies to help understand this or do we need to do some additional studies to help solve some of this dynamic nature and reporting back and the care? So with that, I'll open discussions. Rex. So the other thing I think that we constantly struggle with here is what the definition of action ability is. And so I was really struck several of us had the pleasure of being at the ClinGen meetings just a couple of weeks ago, last week maybe. And there was a presenter there who doesn't routinely come to these meetings who was a parent who talked about a sort of crowdsourcing approach to help get information about variant in his child for which there was fairly little known. And the take home message from that which I think would be great to stimulate some discussion is he argued that we should get rid of the term action ability because his argument was by sharing that information and just making it available, he was able to engage a basic site, you know, back to a virtuous cycle, he was able to engage a basic scientist who actually helped them figure out what the biological basis of the condition that his child had was, he was able to identify something like 50 other people around the world that had this condition. And I think if I remembered it correctly and somebody who was there could correct me, he's actually even gone so far as to think about how you could produce some of the biological material that if provided back to his child might actually have a therapeutic, a positive therapeutic effect. So I was really struck plus it was an amazing presentation but I was really struck by the case that maybe we're not even thinking about action ability correctly in a lot of these cases and that goes far beyond whether we can just sit in front of somebody and say oh well, you know, you're at increased risk for this disease or we'll give you this drug but this guy went out and did a lot of action based on something where many of us would say there was no action ability. Howard and then Mark? Yeah, I wanted to respond to that because I saw the same presentation. I mean, and I understand where he's coming from and I think those of us that are in the clinic never would say there's nothing we can do with this information but I think this gets at what Jeff had introduced as the concept of, you know, personal utility which obviously was incredibly rich for this particular individual and utility that payers and others pay attention to. So as much as it's really enticing to think about jettisoning the idea of action ability, clearly there's going to be some utility to using action ability or utility for certain audiences. So again, this comes back to the idea of we have to understand the context with which we're using it. I would also just point out that a lot of the points that we're bringing up around the role of the patient here are going to be explored in panel 9 and so I think that it would behoove the panel 9 folks who I know are paying attention to this to perhaps pick out some of these comments. We not spend a tremendous amount of time discussing them now but that could reframe our discussion at the end of the meeting tomorrow. So on a different aspect of what you're presenting, I was trying to think of examples where this problem has already been solved where there's data today, the data tomorrow maybe or maybe not different. We have to be able to circulate it and figure it out and, you know, on the genomic level we're trying to figure out for our own institution because of both the clinical part and the liability that is there. But there aren't that many examples in medicine where the quantity of data has to be longitudinally reassessed. You know, yes, if someone has poor kidney function you note that when they come back to see you later but it's not the same. And the only thing I could come up with was around what the grocery stores do where they, when I go to shop, but yes, every once in a while I do. It, they know a lot about me, my past and they see what I'm doing today. They make changes in their algorithms based on what I do and feed forward. And yes, my purchases for my 16-year-old daughter does make my profile slightly different than it would be otherwise. But it's still, there is this kind of iterative loop that is in place. Now, there's monetary reasons why they do that. But they're handling mass data in terms of, and I'm wondering whether we need to look outside of medicine to get some examples on how to handle this. Because really that's where we are is this sort of loop. But it's, there's just not a clear path to follow. The EHR is not going to give us what we need for this. The data, the way we're doing it today is not. So we need something much more creative than just a small iteration. So I think that's a good point. And I do believe there are other places we can get help in this space. But I think the challenge is the reporting side of it on the clinical, right? That's a little, so I think there's a little bit of a difference in that, you know, you should be buying self-magazine today, Howard. Versus, oh, that gene that we reported out is not the same one. So I mean, but I agree. I mean, I think we should look at some of the other technologies. We had a question here, comment. I was just going to comment that the space between marks, following up on Mark's comment, that the space between clinical action ability, from a payer standpoint and personal action ability is potentially filled by research action ability. So that's, I think it's quite relevant. And then the second quip is that, you know, in terms of all that high data, amount of data that there may be some NSA servers available quite soon. Yes. One of the big problems that I see in achieving all of these goals, whether it's, you know, longitudinal data, preventing health disparities, re-contacting patients when new information is obtained from, you know, about variants, is the need for unique identifier numbers for babies born in this country. You know, speaking for newborn screening, you know, we have problems reaching families two weeks after their children are born, you know, about their newborn screening results. Because of the transient nature of a lot of our population. And so I know this is something that, you know, I've been on group studies that have been talking about this for years now, and it's still not been implemented. And I really think that, you know, not that it's something that we can solve here, but on the national level as we go forward with personalized medicine, you know, there's a real strong need for that. And maybe I could ask if the slides could go back up. Howard asked, in regard to this specific question of the dynamic nature of these data, and I think we actually need to get to Howard's slides. The one where you asked a series of questions about, you know, where you store the data, who does the reanalysis, what's the rate, what are the criteria, who pays for it, that sort of thing. I think there are some labs that have some experience with this that could comment. But it may well be something that we want to struggle with a bit. And again, where are the research questions there, since that's, you know, sort of our bailiwick. So Rita, you need to bring up Howard's slides if you could. I know you're probably extraordinary. But maybe Heidi could, or Sharon or others could comment on how you deal with that. I can comment from the Baylor exome experience where we, and we, some of it was in our JAMA paper. It actually is a reasonable proportion of the diagnoses are made from very newly discovered genes. So that's in children, predominantly a pediatric population, although not entirely, and again, predominantly neurodevelopmental genes. And we definitely do routinely re-evaluate the data and report back. But I would say that I think if you look at medicine overall, we tend to just re-order tests. Because, no, but I mean, for good reason. I mean, you wouldn't want to re-evaluate an MRI from five years ago. The MRI resolution is so much better now than it was five years ago. And so I do think we spend a lot of time talking about re-analysis of data, but I actually think the exome certainly continues to get better, and even the genome will get better in terms of our ability to detect triplet repeats and things like that. So I think we're more likely to just order a new test with better technology than to try to find data that's five years old. So my, our impression is that the re-analysis is relatively time limited within the first couple of years and with response to the other comment, because it's also extremely hard to find people. They don't have the same doctor more than two years later. So I'll add to that. I completely agree with Sharon when you're talking about re-analysis of raw data that you might as well in most cases re-order the test. But another paradigm is when you've already reported and interpreted a variant and that's sitting in a clinical report, how do we maintain knowledge on those reported variants over time? And, you know, about five years ago we launched a new software system so that I don't even have to re-analyze the data. If my variant database updates, it will automatically send an email to the physician, update the report. It could do thousands of reports behind the scenes without me doing anything. And that has been an incredibly valuable system. It's been the subject of an NIH study and quality care that David Bates led. But the single biggest barrier for us rolling out this system was getting a lawyer to sign off at each institution that they would accept the fact that the system did work automatically. And we could not get most institutions just to have a lawyer sit down and sign a contract. That was the biggest barrier to launching this widely. And it was, I sat there and said, the liability you could incur, because you are denying patients to get the reports updated is far worse than what you're looking at. But it's been a huge barrier. I think this sort of comes back to an assumption that you made at the beginning of your talk, which was that everything is gonna move towards a whole genome sequence. And I just wanna kind of push back on that and maybe we should re-evaluate it. I think as we learn more and more about the biology of disease, we won't need to do whole genome sequencing for more and more types of patients. I think even hereditary cancer is one example where these panels that we have now are effectively as good as doing whole genome, whole exome sequence. We don't learn anything more from doing genomes, at least in a clinical setting. Now this is putting aside research questions, but I think the same thing will be true for many different Mendelian disorders as we answer, as the Mendelian Center solve all of them, we just need to have a really good clinical panel and don't need to then generate all of the rest of the data, which then causes all these problems. And so I think that's something that we ought to keep in mind is that whole genome sequencing is not necessarily the answer to genomic medicine and implementing it in the right way. So it might surprise you that I slightly disagree with that. I think the big issue is that there is a transition, in my opinion, away from we test for specific purposes till we do a broader test that enables us to use it for more purposes. So as an example, if you only use a panel for a specific indication, it is only useful for a specific indication. If you spend the resources to do the whole thing, then you have the ability then to leverage that across multiple indications because you don't know what you're gonna be back in for. What we do know is our model of the past has been tests for what I'm seeing and reimbursed for what I'm seeing. And I think the challenges we're talking about here are really around the fact of, hey, let's do one wet lab test and then the follow-up analysis is now going to be on the analysis side. And so the testing then is done slightly differently. The wet lab isn't re-incurred over and over again. So I think that's a price point, but as the price gets lower and lower, I think you're gonna see increased use of this for the purpose of using it for other indications down the road, as an opinion. Certainly understand that perspective. I think that as more and more applications of genomics become clinically useful, right? And there's defined advantages to doing them, then those will become part of clinical care. And we're gonna have a much better time showing defined evidence for use of a particular type of data than we are to say that whole genome sequencing is useful, right? And so I think we really do need to still focus on the indication-based testing and not just assume that we can solve all the problems with one test. So I think that's a great discussion. I'm not gonna comment on that. I'll leave that for others. Heidi, did you have a question or you wanna add on? I was just gonna build off that comment because I sort of agree with both perspectives. Technically and economically, I actually think moving towards the larger platform is the way to go. But I think there needs to be a much more definition between what is ready to be returned in a clinical context and what should really be saved for research analyses and then later when clinically relevant could be pushed back into the results. And so I think I'm all for using the full technology at the start once it makes costs sense. But I do think we need to do a better job in deciding what is ready to go back to a patient at each point in time. So I agree with what's ready to go back for the patient at each time, but I do wanna remind everybody that the limitation we have on variation is not having enough variation measured. So as long as we keep delaying sequencing more and more, we have less power to answer some of these questions. Question? I was just gonna say that I don't think that the two approaches, the entire genome approach and the indication approach are mutually exclusive approaches. I think you can design a system where both will work effectively. Other comments, questions? Terry? So I wonder, and I might ask Gail to comment. You made it, what to me was a bit of a surprising comment earlier this morning, which had to do with insurers would rather pay for a single test rather than multiple tests because they're going to incur longer term costs, which is the first time I've heard somebody talk about longer term costs when it comes to paying. But it seems as though that would be a question that an insurer would ask then, oh my goodness, if you're just testing the X gene, it's a whole lot easier than if you do the whole sequence and I have to deal with that. So is there research to be done around that question to demonstrate that yes, it actually is either more cost effective or that it improves outcomes, which is really what they're after from what we've heard. Yeah, so I think there absolutely is research to be done because again, the insurance companies respond both to the research, but also to practice guidelines. And so when the GI doctors publish practice guidelines or the cardiologists publish practice guidelines, they need evidence to base that on. I will say what a lot of the labs do now is I think our cancer diagnostics, we run one panel. We just, you order two genes, only those two genes end up on the report. But if you, it's sort of like when you order a sodium, they actually run the whole chemistry panel, they only tell you the sodium, but if something else is abnormal, they do tell you, we do that. So it makes it hard to do cost outcome measure if on the slide, usual care is actually adding those incidental findings in, but there are ways. And I think it's a really important question because the insurers are so afraid of VUSs. I mean, it's just a fascinating conversation. They're afraid of not just testing, but inappropriate operations, et cetera. I mean, they have some reasonable concerns. Katrina? Yeah, I just wanted to add, Terry, that I think that is an opportunity for the idea of coverage with evidence development to have the payers pay for the test for the thing that they wanted to learn about. And oftentimes it's not really very much more expensive to get a whole bunch more of information. But as long as they don't get it back and that information is being used for the research purpose, then they may be willing to pay for that expanded test. There may be a way to capture what's already going on in labs because many labs have this model. I assume Heidi has a similar model of they're running more than the person ordered. And honestly, from me in colorectal cancer, we were ordering Lynch genes and we have so many TP53 mutations. And that's one of the things that worries me about these neuro panels. You're only testing the genes you know, you're missing the genes that you don't know. And when we started doing a broader panel because we were doing Broca with the breast, then oh my God, we're hitting all these colon cancers on TP53 and it's become significantly recognized that that's an associated gene, but that's not something we thought about. Well, and I think while it's important and reassuring to know that we could get some help in paying for the research, I think it would be helpful to have this group really help us to address what is the research that should be done. So what is the question that we would want to address and can you kind of flesh that out for us? Yeah, okay, yeah. So I mean, I think the very specific question of what is the added benefit of having those other genes versus what is the added cost of follow up? And so the added benefit, you can find an existing data, but those VUSs are not reported and followed up under existing care. So you would need to randomize people to beginning return the VUSs and not beginning return the VUSs, for example. So in the UDN, our proposal had put out to do whole genome sequencing on everybody, but only return exome sequence initially and then reflex into whole genome if we didn't find it on the exon. But what's interesting is most of the clinical sites are saying, well, we actually don't want the exome then, we want the whole genome. So I think there's a question of how do we test this? So if you have more data, are people gonna want the more data versus if you only have a limited data, well, you're gonna take what you can get. So I think there's a challenge in exactly how do we test those things. Just in our case, I thought people would say, great, just give us the exon data and if you don't find it, then we'll reflex into it, but we're finding people want the whole genome. I would also say that I think there really is a need and this is, I think, directly in NHGRI's neighborhood of the best way to educate physicians about variants. I think that many people have argued that they shouldn't be reported at all, but the current ACMG guidelines still recommend that variants be reported but characterized that way. And so really figuring out how physicians should understand that they really shouldn't act on them. What is really the most effective way to have physicians really understand the meaning of them outside of the core medical genetics and probably cancer genetics community is a huge issue and will decrease costs if we could better explain to our colleagues, these mutations are only here because in the future we may learn more about them and they're not something to act on now and I think there really needs to be research in, there already has been, but I think they're in the context of large scale testing, there needs to be additional research because the panels do generate more variants. I mean that data has been clearly documented in many scenarios. Cardi? One thing I will say is that one of the reasons that we recommended returning the VUS is because there are things that can be done, for example, segregation studies in hereditary cardiomyopathy is one of the best ways to rule in or rule out a variant could be done in hereditary breast cancer and other things and so there is action that a physician could take to say, oh no, that was the wrong one, it didn't segregate in the other affected family members or yes, that's it, we've now tested 10 affected family members, although the families aren't often that large, so I think, but I think the laboratory has a role and responsibility in being very explicit about what those actions could be that would help inform and I don't think that's always done. Mark? Yeah, I'm gonna presage a little bit of panel four here because there have been a number of themes that I've been writing down here that have been coming out that there may be an opportunity for a synergy in terms of the right place. So we've heard about the difficulty with the longitudinal nature of patients and how to capture data, how do we utilize quality improvement. We've talked about economics which I would prefer to characterize perhaps as a value proposition, training and now we're talking about testing what is the long-term cost of returning variants versus not returning variants and the challenge that we have is where do you do that and we have an opportunity and have had some preliminary discussions with the HMO Research Network which will soon be called the Healthcare Systems Research Network since HMO is becoming an anachronism and the Genomic Special Interest Group that's been organized as part of that. HMO Research Network consists mostly of integrated healthcare delivery systems that have embedded research within the operations and all of the different issues that are coming up here are already incorporated into that and so I think if what emerges from this meeting is that these are the areas we need to focus on then the question really becomes well, who has the best ability to get all the data that's needed to answer these questions and that may be the direction to guide conversations with how we might engage with the HMORN to accomplish some of these important research question. Anything else? Yes. I think on the educating clinicians about variants it's not only the VUSs but it's basic patterns of inheritance. Just as an example I was contacted last week by one of our neurology fellows who said oh can we get genetic counseling for this family, a baby with intractable seizures and the genome panel returned a result in a known epilepsy gene and when I looked at the report it was a heterozygous variant thought to be pathogenic but in an autosomal recessive condition with no second variant found in the other allele so I think there's a great need for genetic education. But I mean to be fair those are ones that I look at and scrutinize because you can't call the copy number variant on the other allele out of the exome data and if there's a tiny deletion there that involves the other so these are important signals so it's really not straightforward so I would come to the defense a bit of the clinicians in fact I would much rather have them ask that question so that we could do the appropriate clinical interpretation I don't remember who had mentioned this earlier but this is a huge area where the people that are in the clinicians that are returning this there's a huge interpretation thing where we can't take what the laboratory is sending us as gospel it's up to us to also do some additional interpretation in that context. Teri. And maybe to follow on Mark's comments as well as those of the study we should do the added value of the whole genome testing to some degree we'll address that in UDN but it's my recollection that at least a couple of the end site projects are asking this question and maybe MedSeq as well so aren't some of you randomizing people to either the screening panel versus sequencing or the limited versus general and yet those are small studies so can those be sort of pilot data that could then lead us to and maybe you could talk about what the outcomes or what your end points are for those studies. Yeah thanks on MedSeq I didn't mention it but we have an arm where we're taking cardiomyopathy folks and looking at them with whole genome sequencing versus the more directed panel testing and at the moment we're not finding greater pickup but we are certainly generating more cost because there are secondary, yeah with whole genome because there are secondary findings that the cardiologists and then the cascade of physicians around them feel must be followed up on as well so that's only one area and I think we're gonna have to look at other areas where there are known variants and where there are more novel variants and there's but I do think that such sentinel projects are useful so that we don't have to do every single variant in every single gene. Yeah so we are also doing randomized control trial with economic outcomes and we have not analyzed our data completely but the difference really is more in the incidental findings because the usual care for colorectal cancer is a pretty big panel right now but we actually have data that we just submitted at SHD on novel genes that look like they're involved but for the incidental findings the number one thing you find are BRCA one and two and whereas there is certainly some cost into following those up there is a benefit to having followed them up. If I may follow up, this conversation reminds me to have a point I make often which is the difference between discovery and clinical care and we are so much at the forefront in our respective institutions that we are often blending the clinical care and discovery so when we talk about finding a novel variant and then going back into the family and trying to discover by segregation I don't think that's a scenario that typical clinicians are going to be faced with when with typical integration of genomic medicine so I mean we don't want to think about homogenized bad care going out into the world but on the other hand I think that we should make some distinction in our minds between what we would expect clinicians and their patients to be dealing with sort of across the general standard of medicine versus what we do at the cutting edge. Although I would argue that in the implementation space that we, these are questions that are gonna have to be tackled because it's not business as usual where we just send it back and we don't expect the clinicians to do this we don't know what the right mechanism is to address the issue that we were just talking about a couple minutes earlier but that's a very important issue to develop and ultimately it will have an impact in issues that really do matter to healthcare systems which is what is my liability for delivering this kind of care and what's the standard of care and we have a role I think in terms of defining what that standard could be. Up front. So, in the back, up front please. So, Wendy, sorry. Yes, since this is so much a matter of education I thought I'd put a word in for our work group which is the, related to the ISCC work group which NHGRI convenes ICs and professional medical education, medical groups. So one of the groups that presented to us was called TRIG, the Training Residence in Genomics and it's largely speaking, I believe, to pathologists. I know Terry went to one of their workshops and they have the very impressive way of teaching residents how to interpret genomes, well, variants at least. And I don't know where the answer is as far as whose role it is, it's at the very, very, very specialists to deal with these variants and have wonderful knowledge or do we need to, I think, disseminate it into trainees. It does worry me a little bit as a geneticist that this is going on so strongly in the pathology world and I don't see a correlate in the genetics world. But at any rate, I think there's a lot of commonalities that we could learn and it strikes me if this group or people who are deep into the projects could see how this is being disseminated, you could inform each other. So we'll talk more about that later, I think. Mike. I disseminated along the dissemination pathways following that train of thought that I think as a practicing cardiologist, and where do I get most of my education and where are guidelines or guidance comes from within this, often within the specialty. And so some of these things are not, there's certainly universal concepts that'll be fine for all of us to better understand Mendelian genetics again. I don't know if the acronym TRIG might scare some of us. But I thought we were past that. But I think that a lot of us do need to integrate the utility of particular variants within the context of our specialty practice. And I wonder even if there might be the same variants that would have a different context would get a different set of educational priorities for a different clinician. And we're gonna talk about that a little bit in the tomorrow and in a panel eight discussion here. Thanks, Howard. You wanna summarize a little bit? Or are you done? Or are you done? No. Lunch. No, I think the conversations were very helpful in thinking about the whole range of activities, ranging from what types of testing are we doing, from panel to genomic data, to looking at what's axiability and utility. I think those definitions, in context of who's gonna be providing the care, is really what's gonna help determine how we're gonna do the reanalysis and provide the data back. I do think it's clear from the discussion there's a lot of work that needs to be done in looking at how do we do this, looking at the cost effectiveness of this. And I do think some of the programs currently in place have the ability to do that. I guess my question then is, is the field moving fast enough to pick up on some of the return questions down the road? And I'm not sure how many people have that, but I think that's an area where we for sure could leverage the existing cohorts and studies that are going on as to how does the data come back in. Thanks to the whole panel. Appreciate it. Do you wanna mention the... Okay, so yeah, so thanks Howard. You're allowed to sit down now. Thank you very much. And thanks to all of the panels. This has been a lively and fast-paced meeting and that's a good thing. In terms of the lunch, we actually would like to take a picture of everyone. As you see, we show those at multiple meetings, et cetera. And that is on the way to get the lunch. So that's a good thing, because you'll all be heading in that direction.