 Thank you very much for having me. My pleasure to come and talk to you today. Feel free to interrupt me. I'm from a big family. I'm used to being interrupted. It doesn't bother me at all. So I'm going to talk about the CSERC consortium, but we have a big group and there has been 152 papers so far. So I'm not going to capture most of that. So I'm going to focus on a few specific advances. But if you have a broader question, please do ask. The CSERC consortium has the goals of integrating genomic sequencing into medical care responsibly, exploring clinical genomic sequencing, interpretation, communication to physician and patients, providing best practices for how to do this in a clinical setting and evidence base for when it's useful, when it's not, what it costs, and in general just overcoming obstacles. So that's what I'm going to talk about today, a few obstacles in clinical genomics that we're addressing. And I just want to say, you know, genomic medicine is another new technology. We have a lot of experience with new technologies in medicine, and they need to be debugged. And even when you're rolling them out, they need to be debugged. And what we don't want to do is get to, you know, the end of 20 years and find out we've been doing things that were not best practices and, you know, hormone replacement therapy kind of jumps to mind there. So even if we're doing something in clinical medicine, we want to make sure that we're doing it correctly and well and doing what's best and most efficient for our patients. So I'm going to start with a case because clinicians love to start with a case. This is a case from Jim Evans. This is a 36-year-old woman who presented with a neurological disorder, spastic paraplegia, that started when she was six years old. And she had been walking with crutches and using a wheelchair for decades and having really painful muscle spasms. She finally presented to a CSER project at UNC and had this mutation identified. And this is a very rare known disorder, dopamine responsive dystonia. Most dystonia would not be responsible to dopamine. This particular gene mutation causes the disease that is. And so with dopamine, it really is, you know, a healer you can walk. So after a week of dopamine, she walked away after 30 years on crutches. So that doesn't explain my title. CSER goes Cosmo. That's what explains my title. We publish in science. We publish in nature. And now we publish in Cosmo. And Jim is extensively quoted. You can follow this link to the story to hear more details about this case and also get 17 tips for eye makeup. Impact doctor is so much better. Right, exactly. Okay, so the CSER consortium has many sites. We have U awards, which I'm primarily going to talk about today, but also our awards that are more LC driven. All the awards also have an LC project nested within them. It is important to know our Twitter handle is hail CSER. So hopefully easy to remember. And we are really looking within a clinical setting to see if we can do best practices for genomic medicine. One of the things that's different about this program is that there are over 200 clinicians in the program. And that's very important to really solving these clinical problems. Not only are they brilliant, they're good looking. So the CSER populations for the U awards are diverse, and that's quite purposeful to ask different questions in different populations from preconception carriers, screening, pediatric cancers, adult cancers, healthy people, and there's germline and tumor testing. The enrollment to date is over 3,000 people, and about 5,000 people expected by the end of about 18 more months of the program. So it's not the biggest program. We spend a lot of time with each individual in this program. There's a lot here, and this is what we call the hit rate, the diagnostic yield of sequencing across various phenotypes. So this is one of the questions that we have is where is sequencing the most useful? Is it more useful in cancer or developmental delay? And so you can see the hit rates vary. This is pathogenic or likely pathogenic, so I'm going to call that a yes for a hit. 2% in adult cancer, 9% in pediatric cancer, 20% ish for developmental delay, and this would be cardiomapathy and lung QT, et cetera. Very high in hearing loss, 10% neurological, better in eye disease, better in syndromic children. And so we're adding an evidence base. One of the things I did here was I pulled all the cancers together, but since I study cancer, I couldn't let that pass. So I divided it out by a cancer type, and you can see there is some variability in the hit rate across cancer types with non-colon GIs being sort of higher. I didn't put error bars by these, but there is significantly difference between like skin cancer and other GIs, for example. One of the things that we look at when we look at the hit rate, you could say, well, gee, you're not doing very well for colon cancer. Is that really important to continue doing that? But we have to weight the outcome of finding a gene with the impact on the patient and the family. And so for cancer genes in particular, we change care, we do surveillance on other cancer sites. We might look for uterine cancer. We might look for prostate cancer for BRCA people. And also there is more aggressive testing of family members for cancer, so you have to balance that. We do talk a lot about changing outcomes, changing care. I wanted to make the point that doing genomic medicine can save money even when it does not change management. And this is a patient we saw in Seattle. She was in her teens. She had a movement disorder from, you know, seven or eight. And she had seen, by the time she arrived in Seattle, 12 experts from Vancouver to Texas at major medical centers with no diagnosis. She had a lot of tests, a lot of CTs, a lot of muscle studies. And this is what she had, chorioatosis and dystonia of the lambs, mostly at rest. And she had facial twitches and a speech problem. And when she got her exome, she had a de novo, it was trio sequencing, in this known gene actually described by Tom Bird in Seattle, so several years before, for familial dyskinesia and facial myokimia or these facial twitches. And fundamentally, it's not going to change her care, but it is the answer. And it's the answer that will keep her from going to 12 more specialists over the next 10 years trying to find out what she's got and what can be done about it. Okay, so one of the first things that we addressed was the rate of actionable incidental findings, and this came after the American College of Medical Genetics paper saying, we're about the same time actually, saying that there are 56 actionable genes that should be returned in a genomic study to all patients. And so we decided to see how frequent those would be. We used the exome variant server data, and we pulled out everything that HGMD said a variant was disease-causing. So that's one of their categories. And there were 615 of those, and then we also pulled out the truncations and frame shifts that are expected novel pathogenics. Most of you probably know there's about twice as many Europeans as Africans and no Asians in this data set. And the idea was to get an idea of how frequent incidental findings are, also to say, well, these variants, if we're calling them disease-causing and they're in these 6,500 people, maybe we should look at them twice because we're going to see them again probably, and to put the data in ClinVar. So one of the first projects was to make a list of, we have 112 genes that we consider actionable. This list actually went to ACMG. Their list was based upon reviewing several other people's lists, including this. And the yellow highlighted ones are the ones that ACMG also includes. You can see they skip over a lot of recessives. And you know you can argue about some of these, which is a good list overall. We had about 30 people who had to agree unanimously to include a gene, which was sort of fun. Okay, so one thing that's important for those of you who are not clinical is that we classify variants as pathogenic, likely pathogenic, uncertain significant, which is a VUS, likely benign or benign. What we don't do is call a pathogenic thing a mutation. A mutation is a change in the DNA. It is not a bad change by any definition. And so it's sort of sloppy terminology, but we all hear it a lot in clinical talks. Well, we found the mutation in the BRCA gene. It just means a change. It doesn't mean a bad change. So we try and not use that word. Similarly, the quantitative geneticists will appreciate that we understand that something can have an allele frequency more than 1%, which is the definition of polymorphism, and still be bad. So just because it has that allele frequency doesn't mean it's not pathogenic. So we try not to call something a polymorphism when we mean benign. So watch me do that during the talk. All right. I'm not going to spend a lot of time on how we called things pathogenic, likely pathogenic, et cetera for this project, but I want to just point out it was pretty simple. Segregation, unrelated to NOVO, truncation, allele frequency. Pretty simple. We're going to get to a complex scheme in a few minutes. And that was two of those, and then likely was one of those. So not that important how we got there, but it was easy to do. So what we did is we classified those 615 variants and then the disruptives. And fundamentally what we found was that in Europeans about 7% of people had, so this is person level, not variant level, 0.7% of people had at least one known pathogenic actionable gene, lower for African ancestry. These are probably because we don't know what they are. They're not in our databases. Likely pathogenic, again, a little bit higher in European than African, novel disruptives, higher in African ancestry than European. And we would return incidental findings because we don't actually return the likely pathogenics. That's not a recommendation. So we're returning to about 1% or so of people. The caveats here, of course, are that we didn't include copy number variants because they weren't called in EVS and so that will raise the rate. And this cohort had a very small number of Ashkenazi Jews and, of course, for BRCA 1 and 2 alone, you would have 2.4% positives for BRCA 1 and 2 and Ashkenazis. So where were these? BRCA, so colon cancer, breast cancer, cholesterol, cardiomyopathy, malignant hyperthermia, the biggest players there. These were the ACMG genes. For non-ACMG, alpha-1 antitrypsin was the most common thing we found. One of the interesting problems came up, though, when doing QC. And, oh, I just wanted to add this slide. This is what we found for incidental findings across CSER with 2,500 people, about the same. And remember, this would have copy number variants in it, so a couple percent of incidental findings for the ACMG list. But coming back to the QC, I love QC, I love data. And so when we did the QC of the variant calls, we had about 25% of the variants that were called by two people. A little bit discouragingly, about half of those, the two people didn't match. And these were really simple criteria. They often didn't match by one criteria, pathogenic to likely pathogenic or, you know, likely benign to benign. But still, that's a high rate of non-matching. So we then went through, we did find some systematic errors. There weren't all, like, clinical laboratory reviewers. We had some genetics fellows in here, although surprisingly they did as well as some of the long-term professionals. There were a few reviewers. We just had to, like, recall everything they did. But even after that, it was a little disconcerting. So we went back. Everything that we were going to call pathogenic or likely pathogenic, we recalled again. So there were 79 of those. When they were recalled blindly, only 35 matched to the 79. So a little more than half did not match. And when another person looked at those, they agreed that most of them were VUSs and two of them were pathogenics. So they had all been called pathogenic or likely pathogenic. But in reality, 42, more than half were really VUSs. And what this says is there's a bias to over-calling the variants. And we see this on clinical reports from companies as well. There's a human bias to over-interpret these variants that we really have to be watchful of. The happy news about this project is that our final calls matched those of some other big projects. And so we feel like in the end we did get to the right calls, and that can be done, but it takes a little work. One of the other things, this project didn't use any in-silico data or functional data to classify the variants, and those, of course, are important features that you can consider. And so because we hadn't used that data, we were able to look at it. This is a CAD score on this axis. The GRP score on this axis. The known pathogenics are in red. Here they are. The likely pathogenics are in blue. Bigger dispersion. And it does make you think, okay, well, some of these things down here maybe aren't pathogenic, and so maybe these in-silico scores will improve your classification. And in fact, they are in the ACMG system. So why does it matter is because patients and their families make very important decisions based on these variants. If you give someone a false positive, you can have unnecessary surgeries, so mastectomy for BRCA, for example, or lots of screening tests, lots of colonoscopies for colon cancer, and you may miss what is the real cause of their disease. If you have a false negative, then you're missing the opportunity for those preventative actions, okay? And in genomics, if you tell this person the wrong answer, then it percolates through their family. So either you are not testing their family for something because you missed it, or you're testing them for something that really means nothing. So the error is amplified through the family in genomics. So with that in mind, we were very interested in this question of how we classify variants. And a year ago, there was a small bake-off of just six variants across the six first CSER labs. There were six initial sites, and then we rolled in three more sites, but at this time there were just six of us. And for this variant, everyone called it pathogenic, and that's good news, except that it's a truncation, so one would really hope clinical labs would call it pathogenic. But for every other variant, there was some dispute across labs about why, and we were just using the early draft of the ACMG variant classification criteria, which was just published about a month ago, and we actually were able to feed back to the people working on those criteria where we stumbled and why we didn't understand or what we needed that we didn't have. For example, segregation was actually not included in the original criteria, but a lot of labs relied very heavily on cosegregation data, and so that was included in the final recommendations. Unfortunately, for me, the final recommendations look like this. These are the ACMG Richards et al. This is really incredible work that they did, reviewing lots of labs and lots of variants to try and come up with a scheme, and so it's broken up into population data, computational data, functional data, segregation, de novo, allelected as a co-segregate with a known bad allele, for example, and what other databases and other experts say fundamentally, and you can go toward benign and you can go towards pathogenic, and depending on where your evidence is, you get a little code, and I'm going to show you what comes of those codes later, but just notice all of these criteria have these little codes. So one of the things that for me is upsetting about this is I'm a quantitative person, and I don't know what multiple lines of computational evidence means. Give me a cutoff. That's what I want. Same thing, co-segregation. And then well-established functional studies, what's well-established functional studies? So some of these things are a little ill-defined, and so we decided to take another round through the data. Okay, so just when you get those little codes, then you string them together in these ways to come up with pathogenic, likely pathogenic, etc. So it's somewhat complex, and in fact some of the variants that get miscalled are miscalled because people add these up wrong, and there is now soon going to be an app that will do it for you. All right, so what did we do? We did a bake-off, we picked 99 germline variants. Nine of them we classified across all nine sites, but the rest of them were classified by an average of 2.85 sites because a few sites missed theirs. And we tried to do it by the ACMG rules and then by our own laboratory rules, which are different than the ACMG. Okay, so what did we find? So this is ACMG, and this is your own lab, and you're going to need both variants, and the diagonal is your classification would have matched the ACMG classification, and that's true 80% of the time. I don't know where that box came from, but we'll just ignore it. But there were times when ACMG was called less pathogenic and more pathogenic, but when it was called more pathogenic, it was moving from benign or likely benign to a VUS. Okay, so the ACMG rules are more conservative. They're more things to VUS. But what I'm trying to point out with these things, which I don't know what happened to the rest of the box, but if the minor allele frequency is greater than 5% Aha, there you go. If the minor allele frequency is greater than 5%, it gets standalone benign. But if the minor allele frequency is greater than the disease frequency, you still need something else to call it benign, and that's where the labs were having trouble with that benign. If the allele frequency was 4% for something that's a rare disease, they wanted to call that benign, and ACMG didn't let them. So then this is the interlaboratory concordance of 98 variants, and this is if every laboratory who looked at it agreed, and this is like one category of pathogenic to likely pathogenic, etc. And then two categories of three, and then this is as far off as 99 and someone calling it pathogenic. And you can see there's sort of a distribution, and what there's not though is improvement in this distribution by using the ACMG criteria instead of your own lab criteria, which is what we were hoping for when we come up with standard criteria. So why was that? I just picked this is the variant that we did the worst on, that people disagreed the most on. And it is an interesting variant. It's a spastic paraplegia. It's a disease, and so I think that's important because the allele frequency is harder to interpret in that case. And this is what the labs called it under their own category, what they would call it under ACMG, and what little criteria they used. And so one thing you can see is if I wanted to call it pathogenic under in my own rules, I was more likely to call it pathogenic under ACMG rules. So we all kind of come into it with bias and picking which of the ACMG criteria are intuitive feeling about what the variant does. So you can see that here, the two labs that called it least significant were very concerned that the allele frequency was higher than they wanted it to be. That's this minor allele frequency. Segregation was used differently across labs. Functional evidence was used most heavily by the people who wanted to call it pathogenic. And computational, so this would be like your group score, your polyphen, was also entirely the same way across sites. So we're not calling things the same way at the same time. And one of the goals of this project was really to understand why and if the actual criteria can be clarified in a way that makes it more standard across laboratories. That's really our goal. The other thing to note is that the time to classify this one variant was from 25 minutes to 200, I think they stopped counting for one lab. The lab that took the most time was on the sort of heavy side of pathogenic. And the lab that took the least time just said no, the allele frequency is too high. I'm done. So here's one interesting fact that we didn't have in the study that is doing autosomal recessive carrier status, which is Katrina Goddard's study. 30 of the first 50 people in her study had this variant. So that's higher than what we expect from the databases. And this was all Sanger confirmed. So it makes you concern that this probably is a benign variant. And one of the things that someone pointed out is, well, you know, that's higher than 5%. So then are you done when there's all this literature and all these functional studies about this variant? And again, a little complicated by being a recessive. All right. So I'm going to transition off of variant interpretation and talk about outcomes analyses and why are these important? Well, we do research. We provide an evidence base. That evidence is used to write practice guidelines. And frankly, the practice guidelines are what causes insurance coverage for these conditions. And insurance coverage for genomic testing is extremely poor. And I would say in Washington State I've said it's, I think it's poor than other places but people have argued with me about that. And so one of the things that's a really important goal is to provide the evidence that will allow insurance companies to decide that these tests are or are not useful. That will also allow professional societies to write practice guidelines that would become the standard of care that would be covered. In general, we pre-authorize any genomic test. It's very unusual that we don't have to write a letter to the insurance company to say, can we do this test for this patient? And it is very common that they say no even though we think the test is important. This was a comparison of colon cancer panels. For those of you who are not colon cancer experts, these are sort of the most common colorectal cancer genes. They're called the Lynch genes. These are rarer ones. And so the first panel that was compared just had those Lynch genes. The second panel had those plus some autosomal dominant, high penetrance but rarer ones. The third panel added just one autosomal recessive gene. And then the fourth panel added low penetrance dominant genes. And so we're comparing those panels. I'm not going to take you through the whole analysis but I do welcome you to look at the paper which is just recently published. And fundamentally what they found and this was a young investigator Carlos Gallago. And what he found was that as you added these genes, you got to a cost quality adjusted life year of 36,000 and these are called qualities and the quality of 100,000 is considered cost effective. That's kind of like the gold standard for that field. So that this was considered cost effective, actually Lynch genes alone did not meet that $100,000 criteria. And then if you added those low penetrances it doesn't cross a thousand but it's not actually as cost effective as the 11 genes. And so these are the kind of data that professional scientists can look at to write practice guidelines. One of the interesting things about this and again I'm not going to walk you through the data is they did a sensitivity analysis that said well we have a bunch of assumptions in our model how many relatives are there what's the penetrance what is the probability of having cancer genetic versus another cause of cancer what's the cost and all these things are assumptions in the model and so this is a sensitivity analysis that varies these assumptions and says do we still have evidence and the good news is that if you have a lot of mutations wildly you're still under that $100,000 threshold but one of the things I found interesting about this slide is that you gain a lot of economic impact if you can add relatives if you have identified a change in someone that's a colon cancer mutation that's pathogenic and you then track it through the family that dramatically increases cost effectiveness and so I think in clinical genetics we try and get the families involved and better do that and probably engaging the families in a more effective way would make this whole field much more cost effective okay just briefly we've done a lot of work especially Barbara Evans in the space of FDA regulation of genomic tests I assume you're all aware of this Barbara and I wrote a 30 page response to the FDA regulations which was submitted and 20 signatures mostly from CSER co-signing that for us and we're continuing to work on that theme and we actually have a piece coming out in a major medical journal in a couple weeks that's embargoed or I'd tell you about it but they told me not to okay so as I said CSER has I think we're on to like 154 publications today keeps changing I think this is not updated since March so a lot of publications I've thrown in a bunch of slides because I'm hoping that Rudy will share the slide set with you the most highly cited slides but I'm not going to go through these we did it another way because what's highly cited is usually a couple years old so we asked the PIs well what do you think is the most important thing that you did and we put that slide in the American Journal of Human Genetics did this lovely issue that many of you got at the fall genetics meetings that had lots of historical papers but they also had this list of the three most impactful papers in the last three years and two of those were CSER papers at the ACMG meeting there were 20 presentations from CSER not including posters and every site presented at ACMG and so there's really impact coming out of this group to the whole clinical community I am going to take just one minute to talk about some of the working group papers some of the most interesting work is the cross consortium working groups these are some of our working groups action ability, electronic health records, genetic counseling, informed consent, outcomes and pediatrics and we look here for consensus across site on ethical topics or on data analysis and so here is this is lists of what we think are actual genes across the consortium for example how to best represent genomics in the electronic health record a couple of papers there there are nice educational cases for genetic counselors to learn genomic medicine and etc so it's been a very productive consortium in general so in summary the hit rate which the solve rate differs by clinical indication incidental finding rate is low we are working to address obstacles to the implementation of genomic medicine this includes how you classify variants improving the ACMG criteria making that more standardized when you get a test from one lab it would have the same conclusion as from another lab providing an evidence base and I didn't really talk about our LC and regulatory work but there's a lot of that also going on so with that here are the the PIs as well as of course NCI and NHGRI are funding agencies so I'll stop there, thanks do we have burning questions or should we move on to less and take questions at the end?