 The Director of Pediatric Ethics at Mercy Hospital in Kansas City, and today, John, I don't know what you're talking about. Genomics. Genomics, okay. And what happens when genomics leads to a false positive result? Exactly. Thank you very much. Thank you. And thanks. It's great to be back and always humbling to hear so many great talks. It's an astounding record of success here at the McLean Center. Those of you who live here may think that this kind of stuff happens everywhere else. It doesn't. It's an amazing place. My talk will really be focusing on this. We have a solution, but now we're trying to figure out the problem. How many babies will die because of false positive genomic tests? No conflicts, but we'll be discussing off-label use of things that the FDA calls devices, but I'll explain that in a little more detail soon. So in 2010, my hospital in Kansas City, Children's Mercy, started a Center for Pediatric Genomic Medicine. And the next year, the NHGRI had a call for proposals to look at ethical issues in newborn genomics. Like all good ethicists, if grant money was available, I became very interested in the topic. We had a director with an audacious idea, Stephen Kingsmore, whose idea was to sequence genes in 20 hours, whole genomes, provide interpretations to doctors within 48 hours for sick babies in the NICU, and try to show that such results would actually improve the care in real time of sick babies in the NICU. At the time, whole genomes usually took about three months to go through this process. The problem was that sequencing was getting easier. In fact, there were machines that could churn out whole genomes in 12 or 18 hours if you had enough money to buy them. But interpretation is pretty hard. Leslie Beeseker wrote in a paper about opportunities and challenges. You get the result, but it's overwhelming because the variants you see range from those that are extremely likely to cause disease to those that are nearly certain to be benign and everything in between. So how to speed up interpretation? We did what most places have started to do, which was to develop our own proprietary software or informatic devices or interpretive pipelines. These things have lots of names, and we, like everybody else, gave them cute acronyms. As far as had sort of a viking theme, S-Saga, Runes and Viking, none are validated for FDA approved because all are constantly changing and being updated with every new interpretive discovery or report. So how does it work? What does S-Saga do? Software by which clinicians can describe the signs or symptoms that the baby has, and the software matches those symptoms with previously reported candidate genes. So clinicians would go to a software interface that looked like this. You'd enter a new patient. You'd get drop-down menus for cardiovascular, endo, ID, constitutional, metabolic, et cetera. You'd put in the signs and symptoms that the baby had, and the software would spit out a list of genes that had been previously associated with that. Then the whole genome would be sequenced, but only those genes that S-Saga said were likely to be associated with the baby's symptoms would be analyzed and this shortened the time necessary for analysis and also, importantly, reduced the likelihood of incidental findings like BRCA or genes associated with Alzheimer's because although we had the sequence, we wouldn't be looking at it. It would, so to speak, end up on the cutting room floor. Then to figure out whether the variants identified were pathogenic, we had another software program, RUNES, Rapid Understanding of Nucleotide Variant Effect Software. Good, huh? Took these variants and assigned them to existing American College of Medical Genetics categories for pathogenicity. It did this by running each of the variants through dozens of existing databases which are used to categorize things according to this framework that's probably familiar to many of you. Category one previously reported to cause the disorder novel or expected to cause the disorder novel may or may not be causative or sometimes called a variant of unknown significance novel unlikely to cause the disease and the docs would get complicated reports like this where at some point on the report it put it into that category. This is a category one likely to cause the disease. And then Viking would do variant integration and knowledge interpretation which integrates Saga and RUNES, prioritizes variants and generates a report for the doctor in understandable lay language about whether this process led to what was called a molecular diagnosis, a genetic finding likely to explain the baby's symptoms. So taken together, these pipelines suggest whether genetic variants are likely to be associated with the patient's symptoms. Does this without the need for much human thought or interpretation? That is, once the doc filled out the Saga list of signs and symptoms, the rest was pretty much automated. And it led to lots of molecular diagnoses, not just at our place but at most places, people using similarly off-label and proprietary interpretive pipelines that are all a little different report high rates of molecular diagnoses in a variety of different populations. So all these populations are what I would call highly enriched. They're all populations of people thought to have some sort of genetic disease. But these are four recent papers, somebody said 40%, somebody said 57%, that was us, 58%. And that, I would say, is very cool science. Wouldn't you agree? Yeah, yeah. In fact, Stephen Kingsmore got into the Guinness Book of World Records for the fastest interpretation of a human genome. How cool is that? Lots of people thought this was the way of the future. This guy started the Precision Medicine Initiative. Francis Collins, the director of the NIH said, DNA sequencing will lead to a medical landscape in which each baby's genome is sequenced and that information is used to shape a lifetime and personalize strategies for disease prevention, detection, and treatment. So what could go wrong? I think the problem is with this term molecular diagnosis because the question is what does it mean to find a gene that's associated with a pathologic finding, especially because association, as we know, is not causation. Here's how Yang, one of the first papers looking at this, published in JAMA just about three years ago, defined molecular diagnosis. A whole exome sequencing case was classified as molecularly diagnosed if the pathogenic, so the result is in the definition of pathogenic or likely pathogenic variants, were detected in Mendelian disease genes that overlapped with described phenotypes of the patient. So a gene that had been previously reported, if it's a novel variation that overlaps, counts as a molecular diagnosis. But do these molecular diagnoses mean anything? Do they lead to beneficial changes in treatment? It turns out that there are very few reports of beneficial changes in treatment and most of those reports are not in peer-reviewed medical literature. In fact, they appear in places like Time Magazine. Genetic screening saved this baby's life. Time Magazine reported this was actually a case from our place. Six months old, that abnormal development was in the PICU on event, got molecular diagnosis that, according to time, led to beneficial treatment. The variant was associated with a defect in citrate transport. He was started on citrate and reported by time to be doing better. There was no follow-up in a peer-reviewed journal. If this is true, this would be a huge breakthrough and certainly worth reporting. But it's now three years later and there's no peer-reviewed report. So that was part of what this NHGRI proposal was meant to stimulate, to get people to start doing more rigorous studies. And we designed a study to try to figure out whether these tests were actually beneficial, whether they were harmful, or whether they were just irrelevant. Here's what we did. We had neonates who the doctors thought might have a genetic syndrome. We randomized them either to current standard of care, whatever that was, we didn't define it, or current standard of care plus the whole genome. So some of the babies got the genome, the whole genome, some of them didn't. And the main study question was, if you get a molecular diagnosis, the doctors and the parents perceive it as helpful, harmful, or cute, but irrelevant. Here were the preliminary results. It was a level for NICU. We only enrolled babies less than four months of age. We randomized 65 patients, about half got the standard tests, half got rapid genomes. Note that there were five crossovers because even at the beginning doctors had some trouble with equipoise, and if babies were randomized to standard care and the doctors really, really thought they needed a whole genome, they could cross over to the whole genome. What they found was, what we found, with whole genome there were more molecular diagnoses, 41 versus 15%, and they were made quicker, 16 days versus 65 days. This is sort of a stupid result to report. That is, if you're doing a test that's being done faster, the fact that it gets done faster is not a surprising finding. And in 57%, we got a diagnosis. The doctors felt that the diagnosis was clinically actionable in 37%, but here's what I want you to focus on. Of those 37%, about half of them, the action was what we called palliative care guidance, which essentially was a recommendation for palliative care in the published report in Lancet. Genetic diagnosis that confers a dire prognosis can empower early discussions with the infant's parents about palliative care to minimize suffering, and palliative care was given to 29% of infants with a genetic diagnosis, and I had this made up about half of the cases that were reported to be clinically actionable, 17 out of 37%. So let's look at one case just to see what I see as the problem with this sort of thing. This was children's mercy hospital case 545, a baby with bilateral chylous effusions of VSD and unusual facies. He was enrolled in the study and found to have a gene associated with Noonan syndrome. For those who like this genomic language, there are the details, and was reported as having a molecular diagnosis of Noonan's. The diagnosis was made at 69 days of age, palliative care was initiated, and he died at 88 days of age. So what is Noonan's syndrome? Features are short stature, congenital heart defects, developmental delay. A lot of babies have congenital heart disease and some mild intellectual disability, not in every case but in some cases. The heart disease is treatable and usually the developmental disabilities are treated with early intervention and individualized education strategies. There's many genes associated with Noonan's, although again these are associations and causation is difficult to prove, but there are no population studies to show how often these genes occur in asymptomatic or people without clinical features of Noonan's. So it's hard to know whether these genetic findings are true positives or false positives. So it seems there's many genetic variants. Each may or may not be diagnostic. The disease itself is not fatal. The molecular diagnosis may be a false positive, but even if it's a true positive, it doesn't justify withdrawal of life support. So some key questions here. Did they really withdraw life support based on the molecular diagnosis of Noonan's? If so, I'd say that's really bad clinical care. If not, then it's very bad science to report it as a clinically actionable finding and the way it was reported, we just don't know which is which. And this is a common feature of every report you read or that I've read about molecular diagnoses. There are exaggerated claims of clinical utility, no scientific standards in making these claims, little documentation of follow-up or outcomes. And it seems that peer-reviewed journals lower their standards in their awe and wonder over molecular diagnoses, or it may be that the papers are sent to other experts. That would be other genomicists who have an interest in showing that the work that they're doing is valuable. One more cautionary tale because this is one example where, unlike most tests that come back, we actually know what the prevalence of genomic variants is in the general population. Crab A disease is a neurodegenerative disease. On-setting infancy usually leads to death before age two and according to the Mayo Clinic website and many other sources affects about one in 100,000 live births. Remember that one in 100,000 number. So New York State and many other states, including now Illinois, started a statewide screening program in 2006. It was a very rigorous program. They first measured the enzyme activity, which was the traditional test and then they added for specimens with low enzyme genomic sequencing of 17 genes associated with Crab A and infants with low enzyme and a confirmatory molecular diagnosis were then sent for clinical confirmatory testing, which included a detailed history, physical exam by a neurologist, repeat of the enzyme activity and the genotype MRIs, LPs, and nerve conduction studies. So this was an extremely conservative approach to diagnosis. What were their results? This was as of about a year ago. They'd tested 2 million babies, 99.9% were negative, but 620 had low enzyme activity. That's one in about 3,000 babies. Oops. Of those, about half had a known or potentially pathogenic mutation in the gene. That's a molecular diagnosis. That's one in about 6,000 babies. And those were further evaluated. About two-thirds were thought to be at low risk, but about a third were, or no risk, about a third were low, moderate, or high risk. If you put those together, that's one in 14,000, so about six times what was thought to be the population prevalence before this was done. And of the 14 highest risk babies who were sent for confirmatory exams, only five had an exam consistent with Crab A. Four underwent a stem cell transplant, which is the only known treatment. Two survived with severe developmental delays and two died within three months after the transplant. But most importantly here, nine, or about 70%, remain asymptomatic and follow-up is now between one and nine years. So 70% of these tests were false positive and we found that the population prevalence of low enzyme and confirmatory molecular diagnosis is much, much higher than we thought, although most not associated with the development of the disease. So given these numbers, what would happen if somebody did a test for the gene of a sick baby in the NICU and found they had the genomic variant associated with Crab A? How many parents would be counseled to withdraw life support? How many would agree to a bone marrow transplant? And how many of the tests are false positive? In Crab A, we now know 70%. For all the other genomic findings we're getting, we have no idea. Quick real-world example, even though my time's up as part of this study, we also did a bunch of interviews with parents. We talked to one parent who had a set of twins. Both were normal until 11 months, then one started losing developmental skills and at 13 months had seizures. And after an eight-month diagnostic odyssey was diagnosed with Crab A at nearly two years of age. Both twins are still alive at ten years of age and the twin with Crab A seems to have leveled out developmentally, that is still impaired, but not getting worse. And the parents said nothing they read or heard from any doctor, any website, or any support group was accurate as to what the clinical course would be. And the natural history for the kid with Crab A was very similar to the natural history of kids who get bone marrow transplants and there's a claim that the bone marrow transplant was successful. It's illustrated by this study that was published in neurology just a few months ago, developmental outcomes after cord blood transplantation for Crab A, cognitive development below the third percentile and gross motor skills not developing at all. What seems like claims of success mirror the natural history of untreated disease. So, what are future needs here? I think we need standardization of the pipelines for interpreting variants, more careful reports of clinical utility, tougher peer review, and all in the service of learning more about population prevalence of genomic variants, penetrance, expressivity, natural history of disease or non-disease in the full range of patients with particular molecular findings. In spite of this, our doctors love whole genome sequencing. They're no longer an equipoise. In fact, we stopped the study because they're unwilling to enroll people to get them randomized and instead it's a complicated slide that basically shows the total enrolled in the study has dropped to zero in part because we're now offering this test clinically so you don't have to enroll in a study to get it. The key question may be where are we in the hype cycle? It seems that we're now here at the peak of inflated expectations. I think we're headed to here because we'll soon be able to measure just how many babies died because of inaccurate tests or inappropriate decisions or inaccurate but misleading tests. Thanks. That's children's mercy. It looks a little like Disneyland. Isn't that nice? John, this is great work and I think what you're doing is really pointing out what happens with new technologies in general. I don't think that genetics is although it's offering a lot, I think new technologies in general are approached like this. I think that applies to many. Obviously there are lots of conflicts of interest of people who are involved with this. One is just that's their field and they want it to be good. There's also the idea that these tests are sold for money and that's a profit center in hospitals. How do you see us making this a better process based on what you've found? It seems only by a careful study and the integrity of researchers to accurately report their results and report them with more rigor. As you say, it's not a unique story to genomics. Adam Sifu here has the best book on medical reversals which would be situations where we've gone from the peak of inflated expectations, everything from pulmonary artery catheters to antiarrhythmics to postmenopausal hormone therapy, all of which have similar things, but that eventually through careful and courageous research are eventually shown either to be useless or the hope would be after the expose. You do find the situations where it's most useful and then tailor the treatment or the test for those situations. Thank you. I really appreciate your talk. Tell everybody who you are. I'm Ben Brown. I'm one of the current fellows. I'm an OBGYN and a family planning fellow here. I agree with you wholeheartedly about the rush to adopt new technology. I was just having a conversation earlier about robotics in surgery. We could say that about many new technologies and I take your point very much. In particular, the correlation is not causation aspect of it. I want to push back a little bit on the primary ethical argument you make or at least the one that you highlight in your title which is the concern about death and are we having unreasonable deaths, unwarranted deaths for pediatric patients. I actually want to, I don't know if she would agree with me, but I'm going to borrow a little bit from Dr. Ross's framework of pediatric ethics and say while the certain knowledge of uncertainty is a valid part of decision making. And so I think that perhaps our biggest problem is the way that physicians are understanding these tests, the way that they are incorporating them into the clinical picture and the way in which they are counseling, not the fact that this is actually being incorporated potentially into the counseling and in fact that someone might choose to withdraw life support as a result of it as long as it is understood in the proper context of its uncertainty that that to me strikes me as an appropriate respect for the constrained authority of the parent in deciding what is going to be the ongoing course for that child's care in particular because my assumption based on the cases that you've described here is that these tests are being done for patients who do have obvious and significant clinical findings. What are your thoughts on that? It sounds a little like the guns don't kill people kill argument but you know it's not the test which is of course morally neutral. It's how the test is used but if what we learn from doing studies is that it's used in ways like this yeah there would be two solutions one would be use it better that could be either figuring out where it shouldn't be used at all or as you suggest figure out how to better convey both to the doctors and the parents the uncertainties associated with it so that may be an answer to Dan's question I mean to the extent that we could do that we would avoid many of these harms it's difficult to do though Thank you Thank you John Excellent as usual I'd have a comment and a question the comment is as was mentioned earlier this is something that is not new about genetics Jerome Lejeune 50 years ago when he described in trisomy 21 causing Down syndrome predicted just this kind of thing and I would tell you those of us and he was right and those of us who work with disabilities we're still in the trough of disillusionment there's no upward slope of enlightenment Individual with Down syndrome now live a full life and yet we're still screening for it and spending tons of money over it and so the question I have for you is if it's moved to a clinical who's paying for that? Are the citizens of Missouri paying for that if they're on Medicaid? That's a moral outrage when we can't pay for other things and we're paying for this which is causing more death that's terrible I guess the question is if you have reported I mean multiple reports in peer reviewed journals showing that it's useful many of the reports show that it's not just making diagnoses but they're clinically actionable if the doctors who are caring for the babies believe that it's useful and want to order the tests and the parents want the test as well where would you locate the moral outrage? Paying for it but if everybody wants it and we live in a democracy you just pointed out what they're wanting you're just actually wrong I'm not sure how to harness my moral outrage John it's even more complex because we're we're something like Crab Bays was put in newborn screening panels under the assumption that we would find these children and convert a lethal disease which was a rapidly progressive dismyelination with no responsiveness in the first six months of life to a kid who walked and talked and what they did is change it to a slowly progressive complex disorder at the same time to parents who had the condition that was much better than being minimally responsive again to Peter's point motivated by parents who had a kid with Crab Aay to get it done had hope but the individuals who had those genetic variants outnumbered by more than tenfold they were told you can never know when you will become the patient and waiting for the rapidly progressive type and their children and I know that you and I this yesterday Mike is developmental pediatrician who in Illinois has to evaluate all the kids to do that detailed neurologic evaluation so sees all the false positives whose parents are coming back. Let me just tell you one story though that relates to all these points and then I know we have to stop I gave this talk at Vanderbilt or a longer more complex version and at the end of it to your point that it's substituting another disease and these kids are not going to get up and walk one of the people who asked a question was a pediatric neurologist and he said now the problem is we're just not transplanting them soon enough if we transplanted them sooner it would be completely curative and I said well then you're going to have to transplant a lot of kids who never would have developed disease and he said no we're not we suggest you have to transplanting them in utero because many of them are born with the symptoms