 And our discussion will be moderated by Howard Jacob of the Hudson Alpha Institute with Discussants Lesbysiccer from NHGRI and Ned Calange from Colorado Trust. Colorado Trust, thank you. Well thank you very much to the speakers, I thought that was very informative and should I think engender some good conversation. We'll start off with Les and then Ned. Great, thank you. We're not. I have notes. Oh right. Great, so thank you for the opportunity to comment. I had a couple thoughts based on the presentations. I was really intrigued, Jonathan, by that model you brought up, the Frybeck-Thornburg model. And I thought you didn't go quite where I was thinking on this, which I think is another thing to think about, which is you talked about that the delta really of the clinical utility. And what's interesting to think about is that in fact opportunistic screening or population screening for disorders through genomic testing will actually yield the largest delta. Because if that's not why the test was ordered and that's not what the clinician is even thinking about when they ordered the testing, the change in how you think about the patient is going to be the largest because the converse of that is what you did mention, which is the confirmatory genetic test where really you're changing nothing and if you meet the criteria for CF, two delta 508 alleles doesn't do very much for you. So it's interesting to think about that, you know, where we may have our biggest impact on clinical care is taking a patient for whom nobody is currently suspecting that disorder and completely changing our thinking about that patient. So that has I think a number of implications. The other one is in thinking about evidence and that came up in all three talks of generating evidence and of course everyone agrees with that. I think we have to also keep in mind what should be the comparator and I often talk about what is called the nirvana fallacy, which is comparing some real thing to an abstract ideal of what that thing should or could be. And we often do that in genomics and say, gosh, the penetrance of that variant isn't 100% or gosh, we don't know everything about the clinical utility of finding that variant than a patient. When in fact in current practice we are currently using many tests to evaluate the same patients for the same indication that have clearly terrible utility and very little evidence. And so an example of that is imaging for intellectual disability or autism. That is done all the time and as far as I'm aware it is routinely reimbursed by third party payers and you could probably generate some numbers pretty quickly to say that genome or exome sequencing may cost five to seven times what a cranial MRI costs. Mark, what? The same but then you look at our yield is probably ten times higher, right, of what we find. Thirty. Okay, there you go. Mark has done the numbers. And so that should be our comparator and then we say to our colleagues and to the payers we think you should move imaging off the initial evaluation of this patient and substitute it with this other test because your value proposition is much higher. That's the right kind of comparison to make. Not all this whining about it doesn't meet some abstract ideal of what a perfect test should be. And the third one relates to both Julie's and Jonathan's talks about what kind of an evidence base we need and one of the things that wasn't mentioned which I struggle with a lot is now not all of what we're talking about here is exome and genome scale sequencing but I think that is an important consideration. And one of the challenges is that test, if you can think of it as a test I'm not sure it is, will be economically justifiable I think in the end for relatively few single indications. It's pretty expensive and maybe for undifferentiated intellectual disability or autism might be one of them but not a lot of others. When you just think about it as a test to answer an indication. What it actually is is a resource that will be amortized across many downstream uses over a long period of time and Julie's comments about we need to do more studies across networks thinking about how to design studies that can begin to get a handle on that long term utility of this as a health care resource is an even bigger challenge. I think we'll need to figure out how to do it and we don't have to wait until that's done to use the test but I think that's one of our utility evidence challenges that we face. Thanks Les, Ned. Thanks Les. First of all I really want to agree with Julie. I mean I just, and Jonathan and Roger because I actually said everything that you should say so I'm going to try to just make a few other points. First of all I want to recognize that when I started in, I'm not a geneticist and I always start with that I just, I'm an evidence person and when I started in genetics it was all association and someone said association isn't enough. So then we moved to actionability and what you're hearing today is actionability isn't enough. That what payers, if I think about sustainability of implementation you only have two problems right, the payers and the doctors, just two problems. So I'm going to take them separately and the payers say you need to change an important health outcome. So you've heard that so perfect and just kind of keep that in mind important health outcome. I loved reading about Ignite in preparation because you're doing what we need to have done, you're generating new important clinical information. So the Clopridigrel example is fascinating to me because I saw the survival curve separate and it made me happy until I read that well we're just not using that drug anymore because we have alternatives. So hopefully there'll be some economic pressure to say we still should test and use it because it's better in some situations. I think the other thing to point out is that when you're talking about Delta it's not just benefit, it's the marginal benefit. So how much effort are we going to put in for marginal benefit? So I know Dr. Borewinkle works on genetic testing in the cardiac space to be better than Framingham and the problem is Framingham's not bad and so that marginal benefit needs to really say I'm going to treat people that I wouldn't have treated otherwise and have important health outcomes and I'm going to not treat people that I otherwise would have treated and therefore save money and not have adverse outcomes. So thinking about the size of the marginal benefit's important, that phrase medical necessity you should go to sleep with, right? I mean I don't actually know what it means but it's the way decisions are made and so thinking about the inputs to medical necessity I would think, have you think about it another way? What would happen today to the health of the people of the United States if genetics went away? We just said Terry all this investment, whole institute, criticism in JAMA, right? We were wrong, let's just not do it. What would be the impact to the overall health of the population? And I think you want to think about that in terms of medical necessity that if we don't do it then people are not going to be as well or live as long. Less than 1% of publications are T2. So maybe this is just a natural lifespan of a new science but you got to change that because if we aren't getting to the place where T2 is driving the bus, right, we're never going to get this implemented. I didn't hear other things like Jonathan came very close but you know I'm a preventionist and for years we had this thing called the Wilson-Jung criteria for screening for disease and the last element of their criteria is it should compete favorably with other needs for the same resources. So you should think about if we weren't spending money on genomics where could we spend it? And I got to tell you there's a lot of areas for benefit. I think I said this at an NCI meeting that if we could just increase the number of people we screened for hypertension and hyperlipidemia from the current incredibly low levels, incredibly low levels. That's the stuff we know works. We'd say thousands, tens of thousands of lives in every state and to me that's an opportunity cost. That's saying should we focus on genomics? Should we focus on other things that we know work? That's kind of the Wilson-Jung issue. Please don't forget about harms. We had very little discussion from anyone on evidence of harms so we always think we're like all clinicians. We think about new interventions in terms of benefit and we never think about the potential downside. We had a little bit of that in talking about diversity but I'd ask you to remember no medical intervention is without the chance of doing harm. And then finally on the payer side, the David Eddy phrase of evidence-based medicine remember he says when there's insufficient evidence be conservative but if there's harms or costs don't do it. Now not everyone agrees with that rule but I think that the payers thinking about medical necessity kind of adhere to that. If I could turn quickly to the doctors and then I'll stop. They are not thinking about the initials APOL1 or CYP2C19. They are thinking about MACRA, MIPS. If you don't know what those mean, that's what's driving primary care, BATI. They aren't thinking about genomic medicine. They're thinking about the new way that they're going to be reimbursed for services and those are MACRA and MIPS and the MDP, the medical diagnostic, no it's not diagnostic, it's the data set that they're going to be looked at for their quality. And I just looked through it in preparation for this meeting and it only talks about genetics in terms of medical geneticists. So you're not on the radar screen for CMS from a quality standpoint and these acronyms, these payment reform systems, they're keeping genomics off the radar screen. So they're looking for survival. I'd ask you to think about talking to clinicians with the probability that implementing genetic medicine will help any of their patients. I know that sounds crazy but if you think about mammography, mammography will save seven lives out of a thousand women screened for their lives between 30 and 74. That means most primary clinicians will never have a patient whose life they save with mammography unless it's an all-woman population. Now we do mammography and we all do it and we do it based on more of a population approach, advocacy and the way that our society looks at breast cancer. I think if you want to convince clinicians to do something new, tell them it's going to help them patient and help them understand it. I was talking to pediatricians recently saying we'd like you to screen for mental health problems in children. Pediatricians are my favorite clinicians because, sorry, no one take offense. But the sense of patient advocacy is just woven into their souls. And what they told me was, oh my God, don't add something else. I can't do anything else. I had one of my residents when I was in family medicine faculty saying, Ned, I love you talking about all this prevention stuff, behavior change, people should stop smoking. And I got to tell you, I just don't do it in practice because I don't get paid for it. And so if I want to put a positive spin on that, the clinician is seeing an opportunity cost. He can spend his time more useful doing something else with the patient and get paid for it. My last thing is really heresy. So Kaiser put together a DVT lovin' ox program. So I was sitting in the clinic, had a patient come in with a hot leg. I said, I think this person has a DVT. I sent him up to the radiology suite. They did a study. And next thing I know, the clinical pharmacist is saying, Ned, your patient has a DVT. We're going to start him on lovin' ox. We'll follow all of their numbers or switch them over to warfarin. It'll be great. And I didn't have to do anything except smile. Remember when I was an intern, I would spend nights, right? You all did, chasing PTTs, right? So this was a godsend, what am I trying to get to? When I was designing prevention programs for Kaiser Permanente, I came to the conclusion I could be most successful if I just designed the doctors out. The system will work best if you don't add something for me to do. So all the great clinical informatics, there's a way to have a nurse or a pharmacist do it. It's going to be easier to get implemented in clinical practice if you don't rely on me to change my behavior. So I think I tried to add some new things to what you guys said. And then I got to tell you, you nailed it when you're in your presentations. I'll give the speakers a chance to comment first and then we'll open it to the panel. So I mean, to the rest of the floor. So I have a few comments. First of all, to Dr. Beeseger, I kind of, I don't actually agree with the statement that it's not going to be cost effective to do, for example, exome sequencing, to do exome sequencing for a specific clinical indication, which I think you said you didn't think it would ever be cost effective. I said I didn't think it would be cost effective for very many individual indications. And I guess I would take issue with that because I believe that the costs of doing it are dropping so substantially that in my view, it will be cost effective. I think some of this discussion has not sufficiently broken down context as well in terms of what the value of the testing is, what the evidence needs to be for it based upon that particular context. For example, testing a person, for example, with developmental delay, a child, to find the answer to that question is will abrupt a life, years, years of investigation that will be extremely expensive. It will give a satisfaction to patients. And I think for, for example, in the inherited disease context, I do believe that finding the answer, as any geneticist will argue, is a utility. Whether or not to give a drug based upon the test, that's going to require a higher level of evidence. And again, then that's going to depend on the particular context in which the drug is given in the particular setting, whether you're going to screen something, that is a very different issue whether you're going to screen healthy people. So screening a healthy person is completely diametrically opposed. It's at the other end of the spectrum, for example, what I was talking about with basically a patient who's looking at death. And I think the decision making process needs to take those elements into context. And I think the value has to be assessed in that way. And I think the evidence that's required needs to be placed within that framework and within that context. If you're preaching, I'm the choir. I don't get accused of being underexuberant about genomics very often, but you did it. So congratulations. I was actually thinking, my denominator there, I completely agree with everything you said. I'll just explain my comment. My denominator was more based on, if you think about all of the clinical encounters where a clinician then orders a test, if that's your denominator, I don't think exomergenome is a very big numerator for that denominator. But I completely agree with everything you said. And I think there will be plenty of indications. It's just not a lot of them. But what I'm excited about is the fact that once that data set is acquired, and I was very excited about Josh's concept of having yet, what was the word you used, Josh? Escrow. The escrow, we will then go back in and dip out of that over and over and over again. And every time you do that, the effective cost of that exomergenome is falling. So that's great, I agree. So I just, so thanks Ned, because I think your comments were fantastic. I'll just clarify on the clopidogrel. So I think, and Josh didn't show these data, but I think they do have data on clopidogrel use. I believe it represents still the majority of anti-platelet therapy. But the other important thing, and we didn't really address this, is that the cost for a month of clopidogrel is about $4, the cost of the other two drugs is about 200 per month. And even with most prescription benefits, the co-pays in many are $50 to $70 a month. So there are huge barriers to access. And we think even if you wanna start with the alternative, having the genotype, and then if you switch after the first month, which is the high risk period to clopidogrel, that's still probably a bad idea if they carry a loss of function. But, and I also wanna comment, so a lot of the success in our program at UF is because we indeed had clinical pharmacists sort of driving that, facilitating that drug switch. We've just implemented it in another hospital. And the other way we've gotten around it, because we found that doing anything that impacted workflow for the physician is almost a non-starter. And so the other way we are doing it in this new hospital setting, as you and using the Spartan Genotyping System, where basically they have it in the cath lab and it sort of doesn't change anything about their workflow. They make the drug decision. But I think you're exactly right. And we've actually experienced that in our implementation. One of our biggest barriers to implementing was when we, especially with an interventional cardiologist, we talked about impacting the workflow. It was a short conversation. Okay, I think we'll, okay, John, go ahead. One very brief comment, I wanna get to the audience. I mean, one thing that none of us mentioned that came up a little bit earlier was education. And I think sustainability is about bringing in the new crop of physicians and other medical specialties to be able to rapidly adopt things as they prove useful. And so that might be another consideration as far as the sustainability piece. Okay, we'll open it up. Mark, you had a... Yeah, two quick comments. But before that, I wanted to solve Ned's problem about what the definition of medical necessity is. As a former medical director, I came up with one. It actually dates to Lewis Carroll through Humpty Dumpty. Medical necessity means precisely what I say it means neither more nor less. So that is the definitive definition of medical necessity. My two comments. One was related to Roger's discussion of the rare variants. And I think we have a sustainable model. Now, whether we could replicate that model, I don't know. But I would argue that the children's oncology group has taken the approach from decades ago where they said all of these pediatric tumors are rare. The only way we're gonna learn anything is by aggregating data, treating kids on protocols. And that protocolized care is something that is paid for by third-party payers. All the data is aggregated. About 85% of the kids in this country are treated on a COG protocol. If we could somehow replicate that type of group in the somatic variants, I think you could actually have a sustainable model. Now, whether that's possible, I think is a completely different question. The second thing relates to what Julie had said about reimagining Ignite as another network like some of the other NHGRI networks. And I would argue against that because the thing I see as valuable to Ignite is that you had the opportunity to let different groups sort of innovate in the space. And what I would argue is what you might want to look at is even going further down that road, which is to try and create a system of innovation where you could identify groups that could go in and either fail quickly or find success quickly that could then be moved out into a network type of a model much as you did with the clopidogrel. Now, again, that is a completely different type of infrastructure to think about. But what we don't have is really any space where you can kind of innovate quickly, find out what works, what doesn't work, and then move the successful things forward while moving away from the things. We just say everything's gotta be four years or five years or three years and so you spend four years on something that's not gonna work. Other questions or comments? Jeff. I just wanted to put a finer point on something that somebody said I'm not, I can't remember who it was, but as we think about the paradigm of generating evidence, which I fully embrace, we should also be able to articulate what waste are we actually taking out of the system? Can we actually talk about not just adding another technology or test to an already overburdened system? I think the healthcare systems and the payers will actually look for the exchange of those for something that is not gonna be effective. So can we define the goals of the future of Ignite perhaps in some of those contexts? I would think that would be very positive to do. So I'm in this mode of thinking about Ignite too as a council member here and I thought I hit it, I'm just not speaking into it, sorry. I'm in the mode of thinking next steps on this and it's the button's on, the little guy that's sneezing. Speak to it. My light's on here, it's on down here. It's shorting, it's shorting. I've clearly spoken too much today so I apologize. No, it's a quick question for Julie actually on this. So you gave us a hint on the Clopeta-Grill larger study by saying much of the data's from Vanderbilt and it's in the packet so I'm looking at it. My question is one of, you made a strong case for need for further evidence. There was evidence there at 400, there's evidence there at 4,000 that looks largely the same. So is there a lot different other than a few zeros in front of the p-value and how many times do we need to overpower things in order to exemplify that significance level scales with sample size? Well, I can answer that. She said it in one word, she says it's not a randomized controlled trial. So of course when you showed the first survival curve and I knew it wasn't randomized, I said, was there something different about the patients that they didn't follow the recommendation? And that's why 400 isn't enough. So I think that you could imagine a different population, you could imagine some inherent biases in the clinicians and it's hard to imagine that in two settings. So the directness of the evidence, we talk about direct evidence in terms of benefit. The directness is somewhat influenced by numbers. So if I said, well I think cervical cancer, screening prevents deaths from cervical cancer and I can show these nine women or 400 women, that's not randomized control, it's not direct evidence. When a country like Denmark starts screening and the death rate from cervical cancer goes to zero, that's compelling. So it's a number and a directness and a worry about bias when you don't have that RCT out in front of it. I don't know if 4,000 is necessary, but I would be, I was already thinking about problems when the study until the end got big and more than what's said. It's a great response because that's an important distinction that the larger sample is giving you greater confidence. It's not just a p-value that's making a wow factor here. Those aren't the same. Well and I really do think it's two things. Because we could have had 4,000 at UF and it would be a different level of evidence. So it's that we have, or 4,500 patients across nine health settings. And that the completeness of that, including the sample size, I think sort of tells a story that no matter what the sample size is in a single center, I think you wouldn't get over the questions that people would have about a single center data. Bob? I was gonna bring up a completely different subject that we haven't talked about yet today, which is what do you think the impact is gonna be of things like the five state lawsuits against the maker of brand name Clopidigal? And whether that kind of trial as opposed to randomized controlled trial is gonna start having an impact on people's need to actually utilize genomic information in their clinical practice. I should mention that I'm thinking back decades ago when some of the original prenatal down syndrome screening was driven by an unfortunate lawsuit. As opposed to our highest level of medical evidence gathering. Yeah, so, and I don't know what the five states are, but I think the first state was Hawaii, which has high Asian ancestry and Asians about 60 to 70% carry a loss of function allele. And their argument is that the drug maker knew that these people had reduced benefit and they still marketed in that population. I mean, I think in the pharmacogenetic space, that has sort of been this argument, maybe lawsuits will drive some of these. I think we haven't seen that TPMT, which has been used clinically to drive type hering therapy for a long time. I don't think really seen that. So I think as a genomic medicine research community, if we wanna count on lawsuits to drive things forward, we may be here in 30 years, not making any progress. So, I mean, I think it's possible, but we really, I don't know that we've seen much of it. That doesn't mean we won't see it moving forward. Any other questions or comments? Can I just make a comment from here? So I just couldn't let it go past that diversity and inclusion and health equity such a big part of this project. I wanna thank and recognize NGHRI, N-H-R-G. N-H-G-R-I. See, that's the problem, Terry. You guys created an acronym that's different from all the other institutes. The Genome Institute. Yeah, just for me. That's remarkable. I wish we did it in all medicine. I think the US Preventive Services Task Force often got criticized for saying, well, the prevalence is higher in different races or ethnicities. And so why didn't you comment on that? And it's because the studies didn't include sufficient numbers to make any specific recommendations. And so by starting out that way and being a leader for the rest of the institutes, which I honestly think you are, that's really great. And for those of you who don't have the internet, MACRA is the Medicare Access and Chip Reauthorization Act. MIPS is Merit-Based Incentive Payment System. It should scare you if you're a primary care clinician. And the NDP is a Quality Measurement Development Plan which then generates the PQRS, the Physician Quality Reporting System. And these are the things that scare family docs, general internal medicines, and pediatricians because their livelihoods depend on them. Well, I would like to thank the... I'm gonna summarize, because we're running out of time. So I would like to thank the panel and I'm gonna summarize what I heard. We have some notes and we're gonna put this back up on the table at the end of the meeting. So first of all, in order to get the major issues around utility and evidence-based, if we're gonna have sustainability, we're gonna have to convince payers that this is an important endpoint. So I'd like to make sure that in this session, although it's separated, that I think we need to, in the next round, have the payers to the table at the same time. I think the need to have a larger numbers is very important. But I think, as Julie pointed out, it probably comes down to the different clinical sites also being involved, more clinical sites. And I think there is also an opportunity around the different clinical sites is having a different payer mix. And that different payer mix, I think it goes back to that first principle of we gotta get the payers involved. So you have different payer mixes, you have different economic models around your fee-for-service, as well as around your accountable care side in managed care. So those pieces I think should probably also be looked at. I think a very interesting point that was made is that, and Jeff brought this up, and I was also gonna emphasize this, if you can actually start with the principle of some of your tests for your renewal, that, for example, children with disabilities getting screened with an MRI, which is cost per cost about the same, and you could actually go in and show that. You actually have a targeted strategy of replacing one set of costs with another. I think that could be incredibly creative as you go forward for your renewal. To that point, there's that balance that Mark came up with, which is we wanna be able to fail early, fail quick. So if you could have some of those that you could target, but then also have the ability to network to expand relatively quickly, I think that's incredibly important, because that is gonna be the name of the game, I believe, is having the payers involved, having the numbers and having the way to show that you can change medical costs. I think that's really an important opportunity. The other part that is important is that there's not gonna be any single test comparator. I think Les's point, if I understood correctly, is that if you look at the fact that we are going to make a diagnosis today about 30% of the time, we could argue that, well, we're gonna fail 70% of the time. I think the point, if I understood, was that the value proposition is not the failure at that individual test, but the value proposition comes in later. Again, that's been a discussion point through the day, which is how do we capture that inside the EHRs and other strategies for managing this? We still have the problems around medical necessity, even with Mark's clear definition. We still have that challenge, and I do also agree with the T2 studies being less than 1%. We can all stand up and argue all that we want, but that T2 studies are really critically important as we go down the road and get physician buying around that. The payment reform, I think it fits back into that other line. Are there ways to do like what we were talking about with the MRI? Could we look at some of those payer mixes and come up with strategies that would enforce that, that would now help that tool become more sustainable with those physicians? And I don't know enough about the acronyms, but I'm going to look to see whether or not any of those pieces have a value proposition around that. Otherwise, it won't get on the horizon for reimbursement and payment. I think the other issue is that education is also a critical point, and then I'm going to finish by saying as Mark's point, you know, the children's oncology group has really shown that these types of strategies can be really important. Now, is that a unique subset? I think we don't know enough around that. I'm going to say it certainly isn't unique by itself, but I think that model and looking at that as a positive reinforcement is the way to go. So with that, we'll conclude and move on to the next session. Thank you very much. Thank you. Next, we have the economic considerations panel. We'll have State of the Science gaps by Bob Nussbaum and Mark Williams.