 So first of all, thank you for inviting me to do this. This has been interesting for me to kind of pull some of the concepts together to talk about. And I very much have enjoyed going through this and working with you, and I really appreciate the opportunity to talk with you. I'm going to be focusing on the ACE framework, and my scope is going to be with molecular assays. And I spent the last 18 years trying to fit genetic testing into a clinical chemistry laboratory testing laboratory. So AAREP is mainly clinical chemistry, microbiology, immunology, and trying to take those concepts and apply them to molecular genetics has been challenging at times. So the one disclaimer I have is that these are my opinions and that even within the field you will get disagreement. I've been in other, my thinking has evolved, it is still evolving, so A change as we understand more. I also wanted to thank Heather for working with me with the BC, BSA, and the policy. So I was using that as a framework, but I'm also going to be talking a little bit towards the future, even if we may not be there quite yet. The other thing that I wanted to comment on is that my background is in inherited diseases. So I have a few slides on molecular oncology, but it is such a different issue that I've talked with people about bringing people on and presenting just on oncology. So even though I have a few slides with it, I'm going to molecular genetics and inherited diseases. So I'm, but everybody here is aware of the A framework, analytical and clinical validity, clinical utility, and then the ethical, legal, and social implications. And then the purpose of the test, which actually does the Freiburg modifications, is to reduce the morbidity and mortality and to provide information to manage the patient and family members and to assist with reproductive decision making. So the frameworks, the figure of the frameworks is coming from that paper. All of them have basically the same ideas, although there are some other points that I want to make is that in my mind, there's a difference between a biomarker and a mutation. So when I hear the term biomarker, I think of something that is showing some type of an association, and it can give you a relative risk over a general population. These would be clinical trials or GWAS studies to show an association. They may be gene expression patterns. These are more likely to be proprietary, and I will call you, I don't do many of these in our own genetic code. And I would say if you're looking at these types of tests, you need to talk with the laboratory who has developed that, because most of the time they have the information, and they may be proprietary. So I'm really talking about mutations, or in the ACMG's terminology, they've asked us to call them pathogenic variants. So everything is variant, and then these would be pathogenic. So most of the testing I do, the vast majority, we're looking for causative mutations. These are mainly Mendelian disorders looking in germline. Now some of them will then cross over into the oncology, looking at some of the somatic variants, looking for driver mutations, for drug susceptibility or resistance. And just a broad definition for me is does this assay detect what we claim it detects? We do the accuracy, and we do precision studies, and our accuracy determines the analytical sensitivity and specificity. When we look at this, we need to define what regions we're going to look at and interrogate. Are we going to look at common targeted mutations, or are we going to sequence the entire gene? If we do the higher gene, or possibly we may only want to target a few exons, we traditionally look at intron-exon boundaries, as well as all coding regions. And if we know of a deep intronic mutation, or a regular cherry mutation, we can put that in as well. But those are not evaluated all of the time. Some genes have a very high rate of having deletions or duplications associated with them, mainly deletions. But my analytical validity is defined by which regions I interrogate. We know some of the performance characteristics and their interfering substances. Mainly it's heparin, and heparin just interferes with PCR. Now we know how to handle it, so if we get a heparin sample in, we need to dilute the DNA and dilute out the heparin, so it doesn't bind the magnesium and the PCR reaction. But the other ways that the performance is affected is by rare or unknown variants at the primer or probe sites, and some of them may be creating secondary structures. And so we would be getting possibly in a little dropout, meaning that instead of looking at both copies of the gene, we're only looking at one copy of that region of the gene. Or a probe site, if there's a different mutation than what we're looking for, it may show the characteristics of that mutation. And so there's something there, but it may not be the mutation we're looking for. And so that is what affects our analytical validity. And those are things that we cannot completely control, and that will never be 100% sensitive and 100% specific. The other performance characteristics are if we need to look for mosaicism or low mutation levels. Now this is very important in oncology. It's not as important in genetics except for a few genes where we really do need to look for mosaic levels. And at that point we need to establish what our limit of detection is. Some people use this term as also the sensitivity of the assay. I kind of keep the sensitivity, the term sensitivity, I like to keep it to referring to the accuracy and then refer to the detection to tell you how low of a, you know, what percent of mutation am I going to be able to detect. For the most part, the molecular technologies that we use, they all have a very high performance level. It is more in the design of the assay than the technology itself. We all use primer, probe hybridization, sequencing. So these are all very well established and there's a lot of different varieties, but for the most part they pretty much have very, very similar accuracy and analytical characteristics. One other thing about the analytical validity is that even once we bring a test online, so it's not in our, so beyond our validation of the test, once we bring it online we have a continual evaluation through the CLIA program and the College of American Pathologists to do proficiency testing or some way to assess that we are continuing to get the same type of analytical performance. I don't think anything, there could be controversial, but once we start talking about clinical validity, people define this a bit differently. So again, broadly I look at clinical validity, does the test correctly identify affected or unaffected individuals? And my question that I've been talking with people is, is it the analyte or is it the assay that determines clinical validity? In my mind it would be demonstrating for inherited diseases that mutations in this gene do cause this disease. And then whether you identify them by finger sequencing or by a targeted mutation or anything else, that validity that this gene, mutations really do cause this disease is a way to describe clinical validity. And how do we know that mutations in a gene cause a disease? Mainly because of linkage studies with large families, functional analysis, some case control but truthfully most of them have been more of cloning the gene. At first we could clone the gene from a known protein sequence and then we've been able to clone a gene by the genetics, by the family studies as opposed to knowing the protein. So we can identify even genes when we don't know what the protein is. Again, the clinical validity is going to depend on what regions you interrogate as well as how you define the phenotype. And I wanted to use an example for the F8 gene. And so F8 mutations in the F8 gene affect the factor enzyme which then causes hemophilia A. The most common severe mutation is an inversion of a portion of the gene. So you can't pick up the inversion by sequence and a difference was you took that first inversion and then if that is negative then you go on to do sequencing or deletion analysis. Now, if you just look at the inversion, the clinical sensitivity of that, we will only pick up maybe about half of the individuals with severe mutations. So the clinical sensitivity for just the inversion test is going to be approximately 50%. If you do the inversion test followed by sequencing and then followed by deletion duplication, then the clinical sensitivity for hemophilia A becomes very high. So it's not necessarily the method that you test these with, but it is what you test. Do you test for the inversion? Do you sequence the gene or are you looking for deletions? And with that said, we also needed to define this. Are you talking about what is the clinical sensitivity of this assay detecting individuals affected with hemophilia A? And that's the narrow definition of what I believe should be the definition. If you expand that and say what is the clinical sensitivity of this test to detect, to diagnose somebody who has some type of a bleeding disorder, well, it could be factor nine. It could be Von Willebrand. And so therefore the clinical sensitivity decreases if you use a broader definition of the symptoms rather than the specific disease. So whenever possible, I think we need to define our clinical sensitivities by the narrow disease that it is going to be looking for. Along with this, I've heard and I've seen the use of positive persistent values and negative predictive values. And I'm not quite sure where this fits in. Is it really a measure of analytical validity or is it clinical validity? I would say it probably goes into the clinical validity type measure more than an analytical validity. Or does this really measure clinical utility? Now with single gene disorders and the mutations that I'm looking for, looking for causative mutations, truthfully I'm not sure how to use positive predictive values and negative predictive values as being that to me the clinical sensitivity or the clinical specificity is the stronger value. If people have a way how to use this, and I'm really open and I'm trying to learn this of how to use these positive and negative predictive values for single gene disorders. Now there's other times that I do and I understand it. But one of the other questions is that it then becomes dependent on the population. So if I'm using my test to diagnose for affected individuals that's going to have one positive predictive value or negative predictive value. If that test goes into a population screening where the vast majority are going to be negative, the specificity may be high but the negative predictive value may increase but the positive predictive value will go down. And I look at this and say that's the use of the test but not necessarily the test itself. Which is why I am putting the PPV and NPV more into a clinical validity or a clinical utility category. Now there's a lot of complications that happen when we talk about clinical validity. And these concepts that may interfere or may that we need to be aware of when we're talking about this. One is penetrance or expressivity. If we have a highly penetrant male, then if we identify it in this individual, this individual is going to have symptoms of the disease. Now if it's a low penetrance male, then I would see how a positive and a negative predictive value would be more useful for low penetrance mutations. But still most of the things we're looking for are the highly penetrant ones. The expressivity will simply tell you the range of symptoms that may be associated with this mutation or with this gene. There is a clinical overlap between pathogenic variants in multiple genes that cause similar phenotypes. And this is where the laboratory, clinical laboratories and even the clinicians are really embracing the gene panels to look at multiple genes at the same time. Because it is difficult to do one gene at a time. The phenocopy is also difficult and it depends on the disease, on the gene, but that's if there's other reasons and other things that are causing similar symptoms. So an example of this is the BRCA1 in two mutations. If your phenotype is breast cancer, then mutations in these two are going to be a relative small number of individuals with breast cancer that will have mutations. However, if you define it as a hereditary breast or ovarian cancer syndrome, then our clinical sensitivity then increases. And I think I have this point in several different slides. So if you see it again, I apologize. And maybe it's just that I want to drive this home. For me, the same test is used for diagnostic, for predictive and carrier testing. So they have different reasons to perform the test. The analytical performance is the same for all of them. I can test all of this. But clinical sensitivities are going to be different. Two points that I wanted to make is that there are, again, we don't routinely look at deep intronic variants or regulatory variants because most of the time we don't know how to interpret them anyway. So we try to limit ourselves to the regions of the gene where we know that we have a good chance of interpreting them. This is the same thing as if they're a gene that's not well understood. And there are studies, there are publications coming out now that say this gene has been, this mutation has been identified in this gene with this person who has symptoms. That isn't establishing clinical validity for that gene. And so more studies really need to be done before I don't have that gene on and testing for that gene. So when we design a panel, we are very careful to make sure that all genes in that panel currently have established clinical validity. One of the things that has been very challenging as we go to gene panels and looking forward is really knowing what are the right genes to put on, which should be included. And the NIH has funded a project called ClinGen. And I've been on a couple of these working groups and I really like it because they bring the experts all together with the laboratory experts. And you go through the gene and I'm hoping and I guess I have a lot of hope that perhaps this project can come up with better guidelines for clinicians and laboratories and payers that can use in looking to see what is the right thing to do. So as we go into clinical utility then, and I have taken this as the modified ACE by Fryback and Thornberry where they've expanded a bit their diagnosis into the diagnostic thinking efficacy. And with the idea that we do do a number of testing where we're being asked to rule out that there is a differential diagnosis. And what we can give them back is that if we didn't detect the mutation, what is the residual chance that this person is still affected? Well, it can be reduced but we can never take it to zero. One of the important aspects of molecular genetic testing is that at times we can stop the diagnostic odyssey. We can stop, prevent additional testing by identifying the causative mutation and at that point do the appropriate follow-up and the appropriate monitoring of these patients. And obviously there's the other one such as a therapeutic efficacy and what is the drug response. And then they have called what they say the patient outcome efficacy, so patient management. And there's this improved outcomes and that's all of our goals. But also a prognosis, there's a determined aggressiveness of the disease which could also tell you how aggressively you may want to treat. And then with predictive and using this as more of the pre-symptomatic individuals so you know that there is a familial mutation and you can identify them before a patient has symptoms or potentially as carrier testing in the family, et cetera. Then the last on this slide is a societal efficacy and I very much like thinking about this because at this point it's the proper use of the resources either the medical resources or often in genetics we're looking at community resources for individuals such as fragile X syndrome or other types of inherited mental type diseases where they do need additional schooling aids, et cetera. And I put this slide in and I didn't take it off because I really do want to let you know that I do think showing utility is important and not just to get reimbursed. But I really feel like it's important for us and we work as a laboratory to show and help clinicians in ordering the correct test and also how to interpret them and letting them know what this test can and cannot do. And in doing so I think we can demonstrate the value of genomic medicine in a whole. I do believe in it and believe that it has a very strong role in our medicine. Now it's not the end all of everything. There are a lot of other things that are happening other than genomic medicine. But I did want you to know that I want to show utility for reasons other than just reimbursement. Now the definition of clinical utility is going to be different for different people. The narrow one that I have heard is people saying that the only true clinical utility is determined drug and dose and you have to improve your outcomes. Now this is going to be a very, very narrow definition that eliminates the importance of it or minimizes the importance of a diagnosis. And for inherited diseases a diagnosis is very important and that there is inherent utility in establishing the diagnosis because without it you don't know that you're doing the proper treatment or the proper management. And sometimes the diagnosis is such in genetic diseases is that there is nothing we can do about it. There is no treatment. All we can do is potentially help manage the patient. But that's an important piece of information to know as well. It goes beyond this for a family. If you look at a patient in a family, genetics really is about families. Once we identify a mutation in an affected individual, we do go on and get additional family members that also want to be tested. And truthfully I do the best job when there is a known pathogenic mutation in the family and then if the next person doesn't have that mutation, that's much stronger evidence than me actually sequencing the gene again if there has already been a mutation identified. Now for payers they understand that they're looking for treatment and improved outcomes. For regulators they're wanting to look at the analytical and clinical validity and they are expanding into utility and again for society be efficient to use of our healthcare and community resources. So can we get a definition to fit all of the above? One of the challenges with establishing clinical utility is that traditionally it has been randomized prospective control studies or if that's not available retrospective studies with archived samples. These have been very difficult to do in genetics for both inherited and for cancer somatic mutations, mainly because they're rare. And because they're rare disease, we cannot find many individuals with them. Or for example with inherited, often it's a mutation found just in that family or sometimes just in that individual if it's a new mutation. Some of these studies take a long duration and what do you do with individuals in the meantime so is it actually valid to keep things into a study when there is enough evidence that there really needs to be put into routine clinical care. And one of the problems in our genetics background is after all of this that the results are often inconclusive either because they're poorly designed that we realize afterwards or that there are insufficient numbers involved. Now the eGAP has been very useful in being able to pull together a lot of different studies and doing a meta-analysis. But a common conclusion from eGAP is that there is insufficient evidence. And so here is one of the conclusions for the CYP-450 testing for adults with SSRI treatment for non-psychotic depression. And so they say that discourages the use of this for this case until further clinical trials are completed. Unfortunately that's sometimes taken as evidence against and they take a narrow, this was used for SSRI and then blanketing it against others and saying, okay it's not useful for anything. And one of the challenges with this is that it says it's insufficient evidence which means that it needs to be re-evaluated with continuing studies. And truthfully I don't know how often or how much these are really being able to be re-evaluated. There is one example of a 2C19 with the plavix or clopidogrel that the initial studies, at first they looked promising then they didn't look so promising and then we identified another common variant that was actually increasing the activity which was then confounding the other variants which was an explanation why those studies in between weren't able to replicate the initial ones. So now they're going back and by testing more alleles and when we know more about the mutations the studies are coming back actually quite favorably. But it becomes a circular problem if there is lower evidence, it's poorly valued, it's not reimbursed or we can't get funding so we can't do clinical trials so there's a lower evidence. So how do we really break this circle? I was pleased in working with the Blue Cross Blue Shield, a BCBSA in looking at what their parameters were and I think we're very much similar with the testing of symptomatic individuals so diagnostic testing and in my mind I'm looking for something that will explain the clinical system symptoms and if so we can understand better the disease course. The prognosis then it will help us understand the likely disease progression to allow us to potentially do preventive management and then the therapeutic which I would love to do unfortunately in genetics we don't have enough or a lot of them where we determine the most effective therapy or treatment management. Also with an asymptomatic individual so a person who doesn't have symptoms can be done for predictive testing and this is usually done for family history because there's a family history in it. Again it's so much better for us if we had to test an effective family member first so we know what those family mutations are and then we do do a number of these tests for population screening to identify individuals identify newborns and these do have treatment involved and sometimes the treatment is diet so if they can be controlled by diet that is absolutely wonderful. So the testing somatic cancer cells again some of them could be used for diagnostic purposes but more often they are for prognostic or predictive purposes to determine aggressiveness of what the disease should be and therefore the treatment or the therapy or resistance to therapy. Our main difficulty is that the models for clinical studies just are not working very well for us so the fully powered studies they're not feasible so how can we use some underpowered or partial data can we model them to provide useful information so can we think of different ways to gain enough evidence supportive or adequate evidence by looking at chain of evidences looking at biological relationships and pathways once we've identified a mutation that is important in one type of cancer do we need to show it in all types of cancers or can the bar be lower to show different specimen types that it's useful in other situations. For inherited diseases there's approximately 4,600 known medically relevant genes right now it's just overwhelming to me to say how do we show each disease separately in my mind once that that clinical validity has been established that this gene mutations cause this disease it's very hard to not offer that to individuals and try to do a clinical study for it there's another 15,000 or more in the genome many more may be shown to be medically relevant I'm pretty much convinced we have not identified all of the medically relevant genes yet can we show the usefulness of these comparing the non-molecular diagnostic pathway to what we do with the molecular pathway and that may be one of the best ways to show the testing for this has utility and then the other possible way is the diagnostic efficacy and again I've commented the same as I could use for a different purpose I just wanted to show briefly the utility and oncology where we're looking for driver mutations that are essential for tumor progression we may be looking for passenger mutations that may facilitate these are mainly for prognosis and predictive therapy and we did an exercise I was on a working group with the association from electropathology talking about clinical utility and I will say many of these slides are coming from that discussion we went through this and these are some tier 1 CPT codes to say okay, does this test is this useful in the diagnosis or in the management or is it prognosis or predictive and as you can see for oncology a few of them are important for the diagnosis but most of them are for more of the management prognosis or predictive purposes for inherited diseases as I said they're so rare it's not feasible to show utility for each one is there a way that we could aggregate by disease type or potentially by method type and one of the things that our genetic counselor pointed out to me is that there's still a struggle because even though they are rare we may not have strong numbers to show this but they do have a strong clinical validity and identifying a mutation in this may be very important and unfortunately these don't have CPT not all of them have CPT codes with them they're kind of lumped all together into the 84179 so possibly one of the things that I would like to do is take this back and see if we can maybe start looking at CPT codes so they can be more transparent one of the points though together these are very substantive that basically if you look long and hard enough pretty much everybody has something that could be medically relevant in them or a family member from an article in JAMA 100% of individuals have genetic variants that could affect drug response so it is pretty daunting but that just actually affects many people in the population in other words everyone so here are the same type of tier 1 CPT codes mainly for the genetic tests and if you can see most of them are actually for the diagnosis and that is their main purpose however some of them can be specifically used for that management of that patient a good portion of them can be used for prognosis as well but many of them once we can diagnose an affected individual in the family we can use it for a pre-symptomatic and do it as a predictive test so I think we only put one down that it was a limited for hereditary hemochromatosis and that is because we have learned that these mutations are not highly penetrant and so they in this case they would not have a very strong positive or a negative actually this would be a positive predictive value here is an example that I used we are kind of a center of excellence for hereditary hemorrhagic talent dictation it is still considered a rare disease but it's not really that rare probably one in about 20,000 individuals and it has talent dictations around the the mouth and the fingers but the life-threatening symptom is a cerebral or pulmonary arterial venous malformation and so if that is present that becomes really one of the hallmarks of the diagnosis and what needs to be controlled to look for this we need to do a brain MRI with contrast or a contrast echocardiogram and some of those will need about 20% will need a follow-up of a chest CT which then also increases radiation exposure and you need to do this about every five years in affected individuals or in unaffected individuals you need to do it until they're at least age 40 it's only at age 40 that you can completely rule out the disease and these guidelines are available so by having the molecular test available for this family identifying it in a clearly affected individual and then being able to identify the unaffected individuals so they don't need to go through the surveillance is is very useful so I know that a lot of the discussion we are very enthusiastic about some of our gene panels as well and there was a guidelines that came out from the American Society of Human Genetics that I wanted to share with you where they said the scope of genetic testing should be limited to single gene analysis or targeted gene panels based on the clinical presentation of the patient I took out that it said if clinically indicated and so there's they do recognize that there may be times that you want to do an exomer genome but their point which I do agree with is that you use the most focused assay available as appropriate for the clinical symptoms if it is a single gene and it meets the clinical criteria you do the single gene so for example if a child is diagnosed with cystic fibrosis because of a two positive spectrorides just sequence the CFTR gene and they want to start with the targeted gene panel however if there is a small gene panel with a few genes with overlapping phenotype then that can improve the diagnostic yield especially if it's a non-classic one so the HHT that I just showed you the hemorrhagic telangiectasia there's actually two major genes and one minor gene so even that and even though we do it as saying we're sequencing there's still a small gene panel because we do need to look at both of those genes large gene panels these would be looking at more common symptoms and then the exomes would be if you if this really looks like it has a genetic etiology but there's really no other symptoms or no other notes that you can look at one of our gene panels and well I'm sorry we call it aortopathy but overlapping with Marfan, Leistos Dietz, Erlos, Danlos all of these have some symptoms in common one of the things they all have in common is sudden death and a close relative if we do our single gene assay and if an individual has a clinical diagnosis and meets all of the criteria for Marfan disease then sequencing the single gene is completely appropriate we don't have a huge positivity rate because so many times we're getting a suspected pathogenic or a suspected diagnosis of Marfan if you look at this from our positive patients over half are ones that have a clear clinical phenotype less than half then is a suspected diagnosis we're picking up variants of uncertain significance for those with a suspected diagnosis the question then is are these truly you know they're uncertain but they could be or could be maybe milder mutations showing a different phenotype or we may not have picked up what the cause of the symptoms are yet so when we put together the small gene panel each one of them has a separate clinical validity we were able to pretty much double our detection rate and clinic picking up the mutations just looking towards the future the exome if you're looking at one measure is the diagnostic yield and overall it's at 25% which isn't bad considering nobody else knows what is going on with these individuals if you're only looking at severe intellectual disability it drops to 16% so it's not it's not very effective for that criteria however if there are other neurological symptoms involved as well we're getting a very high a 64% which is quite remarkable so just in conclusion the current models may not be able to do everything for us so obviously we do want randomized control studies or retrospective studies when necessary but we need to adapt some of these clinical trials to be more specific for what we're looking at we could potentially evaluate diagnostic yield use observational data our linkage analysis to make sure the gene has validity as well as there are some functional studies that we can show what these mutations are actually doing but we also need to understand the biological relationships and the pathways because we may not have direct evidence and I would propose that we look at the current care versus a molecular diagnostic model to maybe evaluate the utility of it and then one of the other things that we very strongly promote is more professional input of reviewing the data, reviewing the information so that we can better practice guidelines I wanted to thank AMP's committees that I've worked with as well as our own internal genetics genomics group thank you