 I know unfortunately Dr. Curry was not able to be here today to talk about the CDC resources. Maybe I could just get things started for Erin. So you addressed this a little bit of how is it that ClinGen and ClinVar are trying to leverage and capitalize on all of the other more gene specific or disease specific organizations that are out there classifying variants as to function and trying to harmonize the information in ClinGen and ClinVar based on all these competing resources or is that just an impossible task beyond what's feasible? Well it is a very difficult task of course but it's one that we must try at least to do and I think at least in the space of other groups that are already working on variant interpretation or gene specific guidelines things like that we do make a concerted effort to reach out to those groups as we are aware of them and kind of bring them into the fold and encourage them to also become three star expert panel submitters to ClinVar so we do have a couple of examples of that such as the CFTR group or the Insight group you know groups that were already existing outside before we ever came to be that we said hey you know it would be great if you could put that information into ClinVar so everyone could access it you'll get all the credit you'll be listed as a three star submitter so people kind of know which variant interpretations are the ones that were the result of consensus. I mean I know for FarmGKB and CPIC we've been working with ClinVar for a long time trying to get our information more transparently available and it really is a challenge because the system wasn't really built for pharmacovariance and the last thing that you want is to duplicate information from one place to another because the primary source of the information will be updated and change over time so there's so many cancer specific for example I don't want to say uncurated but unconsolidated sources of information on somatically acquired cancer variants and making sure that people get access to the most updated information seems to be almost an overwhelming task I mean something that it seems like we have to tackle but I'm not sure how to do that oh I see the vertical things I'm supposed to pay attention to. Who was the first one? I'm struck by this morning's talk as well as the current talks about we are continually saying you know if the payers would pay for the testing all would be well and what I'm struck by is how that's actually a tiny fraction of what the real cost is of implementing. For example for ClinVar my company has somebody who's essentially full-time job is ClinVar. So that's a whole FTE that's sweat equity that's donated by a company. All the work that I heard this morning about getting the practice guidelines and working with EMRs and working with your local systems that's all labor that is never going to be paid for by third-party payers so I think we need to have a broader view of what the real cost is of doing this and who's paying for all of those costs NIH is certainly supporting some of it as a research into implementation but that's only a small fraction of what all of this is really going to cost. I see Terry and Howard. I think Bruce was. Okay sorry Bruce I didn't see yours. I think the implementation guide that Kristen described really was an impressive sort of approach what I'm wondering is the scalability of it how much time and effort that represented is there a learning curve in terms of when you think about how many of these lie in front of us how are we going to tackle that large task. It's a great question and I think one of the reasons why we approached it systematically was to build kind of a plug-and-play process for it but I still think it's not feasible as far from a scalability perspective. I think that the kind of the farther along we've gotten into it we really see that it's not necessarily the process that we're kind of laying out isn't really necessarily specific to genomic medicine even it's really just implementation and so I'm not really sure how that applies to even other disciplines and so I think that there's a process of building those guides and then I think there's the bigger overarching themes of what do we learn from the process of building this website that builds these guides and so I think that I don't think that specific strategy is really going to meet everybody's need and be able to be scaled to every potential implementation that might exist there. I think though that we can take the things that we've learned from it which is that there are threads we do need to distill this down to kind of what's the clinical question that that person's trying to answer. We need to make this accessible from an interactive guided best practice standpoint. We want to make this so I think learning those things I think helps us to really then inform strategies that maybe we are able to accommodate much much broader and diverse implementations and and then making kind of extending that I think and that's really to me that's at least for me what as we're sitting here talking kind of reinforces that as I hear what Lori's talking about I was bringing to the table is that I think we're we're moving within genomic medicine to say we're thinking about how do we how do we equip people to be able to do this and so I think it's a really important piece of the puzzle. Okay Terry. So maybe just to speak directly to that and to pick up on something that Steven had mentioned earlier this morning you know there there probably are some things that are unique or at least relatively specific to genomic medicine and so if if we can try to identify what those common threads are whether it's family history or pharmacogenetics or somatic variant testing you know that would be a really useful thing to do but it's not easy. Exactly and you're right there are definitely some things in there and I think a lot of it has to do almost with the scientific complexity you know in some cases of helping people to approach and overcome that and so but there are a lot of things that are unique and I think addressing what those are pulling them out I think is an important part of that staff. And then my original question was actually a historical note so I'll let it I'll let it ride until the conversation dies but Bob might have wanted to speak directly to what we're talking about or maybe not. I think it's Howard. So there are centers North Shore and Intermountain are two good examples of community based but academically minded health systems that have invested in infrastructure with them but the majority of health systems rely on consultants and it has been hilarious to see all of the health system consultants that suddenly became precision medicine consultants. Nothing's changed, no additional insights but they're now that person. I wonder whether ClinGen in particular of ClinBar, ClinGen, Duo and the Spark toolbox have engaged with that community whether it's webinars all the way through to them using the guidelines just to try to raise all boats from what is not very high level. So just to give, it's funny because those are some of the things that we thought about in creating the guides and so we have tried to create it on a clinician friendly level and disseminate that through kind of clinician level journals and so for example when we're publishing about this we really try to hit those frontline clinician facing journals. I don't think that in and of itself is going to do it. I think there's really two pieces one is getting the word out to those groups and the other thing is making the tool accessible to be used by those groups and so something we specifically did to reach that community based population because that's really important to me it's that's essential that we really it is the uptake there is so much faster I think than within many of our settings but one of the example that I can give is when we first built the implementation guide we said if you want to know about CPIC, I'm sorry about ClopidogrelCyp2C19 here's the link to FarmGKV and we kind of got feedback from that and we went back in and said we revised all of those to say to actually point to the specific area within FarmGKV so here's the FDA label for CPIC2C19 here's the pathway so we kind of even took a step back further and said what questions does that person need to answer to like what is it that they actually need to know about CPIC2C19 because they don't even know that and then you know and really tried so we tried to create the tool in such a way that somebody didn't even they didn't need to know how to use Clonvar FarmGKV we said here's what you need to ask and we're going to tell you exactly where the answer is to try to help overcome some of those things but I think it's tough to disseminate to those groups and we've done a lot of our implementations at UF within that setting and my experience in that group has just been just kind of doing it and then they get comfortable and then it picks up from there and so I think it's just a matter of getting helping people feel comfortable with it so in any setting. And I think for us I wouldn't say that we've targeted any communities specifically our education thus far has been more broad within Clonvar they do offer a monthly call in which they invite any members of the community to learn more about various Clonvar related topic topics and then within ClinGen you know we've primarily focused our education efforts thus far within the genetics community in general but we're now looking to branch out more into non-genetics providers so the other providers who might be ordering genetics genetic testing that then might be faced with now what do I do I have this result in my hand how do I use resources such as ClinGen or Clonvar to help me figure out what I should tell my patient. Just a comment on that I mean I think having to consolidate resources is immensely important if we're really going to try to bring this information to non-genetic specialists when I go around to my primary care physicians for the education the question is where do I go for help if I'm not available et cetera we're resources so genetic home reference and ClinGen and ClinGen these are all tools that if you can provide a couple of central points that conditions are reassured that there is an active movement to try to address these issues that are being brought up. Okay over here is it Bob? There was a resource that we developed in a future life a couple of years ago about a year ago that's on the NHGRI website that's just a table for primary care clinicians of a lot of different resources so the question I had was or the comment I had was that it occurs to me that so ClinVar variants are represent a snapshot in time of the information that was provided to the laboratory scribbled on a page with little tiny checkboxes and that the laboratory's best effort at that point. There's no information the information that follows that time point which is what happened to the patient when you gave them toxic drugs did they get better did they get worse what happened or did they pass away and you had an autopsy to confirm the diagnosis none of that information gets fed back into the system so I know that that's not an easy thing to do but maybe brainstorm about how to enrich that information because right now it's kind of like taking an exam and never knowing whether you got an answer right so that would be helpful. That is an excellent point and something that we really wish we could do better at we have a few kind of band aid solutions to that at this time so I didn't mention this explicitly but I think I did kind of gloss over it because everyone is encouraged to submit to ClinVar it's not just a lab thing we want clinicians to submit to ClinVar and we also have a mechanism for patients to submit to ClinVar and while they don't yet have specific structured fields to capture that kind of information there are kind of free text areas where we can get at that kind of information so we do have at least a few clinician submitters right now who do things like say you know yes we have that did this testing they have also submitted this variant and this is what they said but here is the additional information that I would like to add maybe some specific information about the patient or the group of patients that have this variant so that is certainly possible. One thing that ClinGen does that I did not get to talk about at all is we also have a patient registry called Genome Connect whereby we encourage patients to actively sign up to submit their genetic and health information to us to transform it into the structured format that ClinVar needs and that allows us to help them put their information out there for people to use as well and they're able to give us much more detailed phenotype information and we also keep in touch with them longitudinally they're asked to update their information annually let us know if any significant changes so we do try to collect that information in that manner the problem is that there's not an optimal way to show that in ClinVar right now especially because it's publicly available so there are issues with consent and things like that but we do have it we do submit that information at ClinVar so if you ever see a Genome Connect submission you know that you can always contact us and we can personally provide you with additional information. Okay is it Dick or yeah. So first of all I think this is a really important topic and Mary if nobody else says it the effort that you and your colleagues put into CPIC in terms of developing those guidelines was absolutely essential what you do and what the Dutch working group does I think is critical for pharmacogenomic information but when the question that I think we're all going to face is if you just look at 2C19 was mentioned here if you look at the ORF and the exome data that's out there the 250 non synonymous variants and most of those are rare we know the genes are important because it has common variants and I think one of the challenges that these groups are all of these groups all of us are going to face is first of all how do we get functional information and there are high throughput methods which we and others are using and if we look at these variants just in our 10,000 patient study half of them are inactivating for the gene. You're not going to see those very often but they're going to be there. What are the thought processes where pharmacogenomics is actually shockingly a mature field or immature field comparatively speaking and it's effective in terms of broad populations. What is the thinking about how we're going to do that and I'm keenly aware since Bob Diazio has his office right down the hall how strongly he feels about trying to not depend totally on clinical information but use functional information and how are all of these groups going to deal with that and I think you deal with that on a daily basis. Right and that's a great example of DPYD of that was one of if you look at the supplemental material that backs up the variant interpretation for that gene it's got an incredible level of tears of functional confidence in the functional assignment to each of those alleles. It specifically came down to based on predictions of the functional consequence of this or that variation in DPYD it be ethical to give the normal dose of 5FU to a patient getting this drug and so that's why it's so important that we've got a group of experts that are oncologists and pharmacologists as well as molecular biologists who try to interpret every one of those variants. Now what we are trying to do for each CPYD guideline is to annotate and add variants as they become known enough that they would be considered actionable and we do update those tables of variants in each CPYD guideline and we update that material on the website so that's the critical step people have to go to is to go to the website to check on variants we're making an API so that companies and others can download that information every night and have constantly updated information but it's a huge task like Dr. Nussbaum was saying it's really a huge responsibility that implementers are taking on when they're being more and more faced with whole exome or whole genome sequencing data as to what it's safe to call non-actionable and what has reached some threshold where it is actionable so given the amount of work and the amount of expertise that's going to be needed to make clinical action ability decisions on every single variant in all of these genes it's so critical that we as the scientific and medical community figure out how to address this most efficiently and how to avoid reinventing the wheel. That's why my first question for Erin was about what are we doing to consolidate existing groups like CPYD that have all this expertise going into variants like all the cancer groups that are annotating somatically acquired variants so that in one place we can have that information available to people who are implementing genomic medicine it's a huge task. Rick. One of the topics of two genome medicine meetings ago was precisely this and how do we not only look at the clinical community but how do we leverage the basic science community who is using C. elegans or Drosophila or whatever model organism and providing really important functional variation information about these kind of functional assays and there was a lot of discussion at genome GM9 I guess about is there some way to create a resource that would actually enable people to say here's a list of variants but here's a list of variants if you're an expert on gene X my guess is that there's a lot of people out there would say oh well I look at that and I can say it's going to kill that enzyme completely so it's really thinking about how to leverage that and I think we haven't really completely figured that out yet. Right. I just mentioned him because he's devoted his career to this he's put together algorithms for looking at hundreds of these variants which Mary knows well and I know because he comes and sits in my office and talks about so we need to get there's a Bob DiAzio for every gene I only pick on him because I know him but there are several Bob DiAzios for every gene I think we need to preemptively begin moving to take advantage of that I'm just agreeing with you Rick. And I would add that ClinVar does accept submissions like that where you're just you know I'm a lab that did functional work on this gene and this is what I did and this is what I say about it so that is something they will accept it's definitely not something that a lot of people realize and take us up on but that is possibility and that's also one of the reasons why in the creation of our expert panels we like to see varied expertise so we don't want them to be just stacked with clinical laboratory professionals or just with clinicians we want a mix of those plus basic researchers who can inform us on the important functional assays that are associated with a given gene or disease areas so that when that group is making these specific recommendations they can incorporate that as you know that's one of the categories in the ECMG AMP guidelines and when one of our three star groups is putting their guidelines together they must specify these are the functional assays that we feel are relevant for this gene these are the values that we think mean pathogenic these are the values we think are undetermined you know this kind of level of detail goes into each one of those guidelines. And I would just add I guess to build on Howard's point earlier another layer of complexity to kind of I guess operationalizing you know a strategy in that area is what that clinicians need is and whether they even identify that as being whether they even know to ask the questions about which of these variants are functional I know from our experience I coordinate the education for all of our clinicians and I've gotten feedback from our attendings before we walk into a group of clinicians or psychiatrists you know they'll talk through like how do we want to build this educational program and they'll say 15 minutes max genetics everything else patient cases like don't get up there and talk about genetics and so it's a they're you know and so that's the challenge I think it's a level it's a layer of complexity of kind of do it you know and what we really seen is in that particular instance the that was not only what the group wanted but that was what influenced uptake of ordering the test and so we went back after that implementation every two weeks and pulled patient cases that they had ordered what the outcome was and we just walked through their own cases with them to kind of disseminate within that own practice and that is actually I think what drove most of all the actual usage of the test clinically after the pilot project was done so it's really there that that clinicians you know focus is bringing these two together these really important concepts of helping to do it in the right way I guess so if I could be rude and just jump on the I am being rude the specific point though that Erin was making and Rex in terms of bringing the basic and the clinical groups together so NHGRI now does have two solicitations out that are specifically for linking variants to function to disease and I would encourage people I know the NIH system is difficult but this would really be a way of moving this forward so I'm sorry Mark okay but let me I think Dr. Nussbaum do you have your verticality is it meaningful because it's been up for a while I wanted to address Erin particularly in general a problem that exists to do with a lack of clarity as to what kind of data sharing is allowed with and without consent and the fact that there are regulations for HIPAA that are not necessarily concordant with regulations from other things like OHRP I've had for example patients who've had testing done at our lab who've called up and said I don't trust anybody with any information so as soon as you deliver this report I want all traces of my test removed we want it wiped out of your databases because you may get hacked and my response to them is under HIPAA we're required to hold these for two years so what do you want us to do and to some extent I don't think that these extreme very highly suspicious outlier people should drive the conversation but I do think that for example there are clinical laboratories who to use a term hide behind patient privacy issues not to share their data and if we all agree that sharing data is extremely important and I should add that getting consent from everyone adds a level of cost and logistics that will make the data sharing possible at the level that ClinVar currently provides it and so I think we need some sort of clarity as to what data can be shared what level of consent is needed and in particular for genomic information how many variants and what their allele frequencies need to be in order for it to become identifiable because I also hear over and over again from people oh you know if you put genetic information and even if it's totally identified that paper that tracked the individual down to a particular person in Utah you can cross-reference all these databases and so there's no such thing as purely idea by data. I disagree with that I think there is totally de-identified genetic information up to a point but we need much more clarity on this and I'm wondering what NIH, NHGRI, DHHS could do to help make it very clear what we can and cannot do without explicit consent. So I'm glad you asked that question as Laura and I actually have a paper about this very topic that just came out a few months ago just really trying to address a very important point that you raised that some laboratories do want to for lack of a better word hide behind this issue of consent in order to not submit to ClinVar. As you may know ClinVar itself is not an arbiter of consent they assume that if you are submitting the data you have the proper permissions to submit the data but in the paper that I'm referring to and you can find it on the Gen website. We do lay out what pieces of information are appropriate to submit to ClinVar without explicit consent. One of the reasons we feel that this is appropriate is because as you point out these things are quote unquote de-identified as far as that can be done given the things that we're able to do now. We do specify you know ClinVar is a variant centric resource so we say in this document please do not submit someone's entire VCF file all linked together we're interested in single variants. We want to know your aggregate experience with that variant so if you've seen it in 20 people you're telling us about your aggregate experience with those 20 people but even if you've only seen it in one person it's still appropriate to submit because you're not giving us any specific information about the person necessarily that's outside of the scope of what you might need to interpret that variant. We do then go on to say if you wanted to submit some more specific information such as the information that we're able to submit through Genome Connect you should get explicit consent which we do through that process and we do have some generic consent materials available on the ClinGen website should people want to do that. Mark. I'm just going to comment briefly on that but then get back to that point I was going to make. I think the other thing people forget about HIPAA is that the HIPAA standard is that a reasonable person could re-identify the individual which means you lay something on your desk and people could look at it and read a name and address and that sort of thing. There are a lot of unreasonable people out there that can re-identify stuff but that's not the HIPAA standard. I also think it would be worthwhile exploring. I really strongly believe that this would fall under the HIPAA exemptions for healthcare management which is exempt if you're using it to improve the management of the patient. You can share any information that you want and quality improvement is also exempted and so I think those would be other things to potentially explore. The point I was going to make just in response to that I'm sorry Mark I'm being rude too is I think there's some ambiguity that people use as to whether ClinVar is a research tool or is a Q quality improvement tool. I think it's the latter that can be used for research but that is an ambiguity that is being misused. I agree with the misuse and you're absolutely correct and in fact if you look at the quality standards ClinVar is referenced in terms of quality improvement activities so I think there's little ambiguity except for those that choose to make it ambiguous. The point that I wanted to make is that this discussion is really enlightening because I think what it reflects is the fact that Ignite, Spark and ClinGen are perfect examples of what implementation can achieve because these are implementation projects. It wasn't something where we sat around and said we've got to get it perfect before we throw it out there and Lincoln will relate to this because at Inner Mountain we have a motto which is every time we launch an improvement project we guarantee it's wrong or your money back and the whole point is that you can't get it right sitting with a bunch of smart people in a room. It's only until you deploy it that you're going to figure out what all is needed and what all could actually be accomplished and that's why the mechanisms to get the feedback and to engage and I would argue that one of the biggest successes particularly of ClinVar and ClinGen have been the number of people that are willing to commit resources to do it because they recognize the inherent value of this and in terms of the responsive, well yeah it costs us a lot of resources to deposit the data but at some point you're going to look and say well now I don't have to do this variant from scratch because this is a four star variant, it's annotated, I can just release it, I can go forward with it. So I view these as really important projects and that every perhaps criticism if you want to call it about something that it can't do or isn't doing or whatever are actually opportunities that we can really take advantage of. Got one last question here I think is that, oh I'm sorry, go ahead. So I hope Eric, I'm going to talk to you a little bit but I think to everyone, one of the points that Bob made is the challenge that we have of longitudinally following somebody with a variant and I think if you ask me what the key next thing we need to do in genomics is figure out and all of us is going to be tackling this a little bit but figure out how we make health information portable such that we can longitudinally follow people with variants and with genomes and so I think when you talk about the technology that is essential to genomics that is going to empower medicine wider, I think fundamentally it's this ability to collect people's genomic data and follow them longitudinally. We know Genome England's doing it and they're doing it moderately well but I think fundamentally one of the questions that we really have to ask over the next five years is how do we make information available in a way that we can follow individuals with variants and see if they really have a disease or don't. We can follow individuals with a rare pharmacogenomic variant and see when they're actually exposed to that drug what their drug levels are, do they respond in an adverse way and this requires us to have ways of sharing data. When we implement things right now we implement pharmacogenomics at my hospital. We don't implement it at a national level and so I think the push that we have to think about is how we do this in a joined up way across a fragmented healthcare system in terms of data and I think that needs technology advances that we don't have right now and so if I was going to be investing in technology it would be that data sharing. If I could just add to that. I think there is some hope about that in the future so I think those who are participants of the All of Us program will in fact be driving the technology that will allow us to do that time will tell it's early days but certainly the goal is that if you're an All of Us participant you will be able to be tracked longitudinally over time and hopefully that will inform the rest of our healthcare system. Is it Lincoln over here or no I'm sorry go ahead. Just to kind of comment further about following the policies and different regulations for genomic sharing yes a very important issue but there's also the practical issue that in the community we face with the patient perceptions and so as these policies mature there needs to be in my opinion a concerted effort. Well how do we address the public about these and what's being done because there will never be 100% full proof solution but again to Mark's comments about a reasonable solution but there has to be community education hopefully through efforts like All of Us cohort that can happen but I think that's an important thing if we really want to move things forward we're tackling as our institution of sort of you know what does our community how do they perceive what's happening and where we're moving in this direction. Mary I'm not you can go a little bit further as long as people are willing to delay their lunch by a few minutes because you know it won't take us an hour. I'm not sure if we've got people with signs up that are holdovers or if we have additional questions. There you go. All right so I guess just to quickly summarize most of our conversations seems like it has focused on that we need resources to help with interpretation of variance and that interpretation has to extend all the way to clinical action ability in a way that's transparent and free and accessible and reviewed and accessible but we also heard about projects such as that ignite toolbox which is sort of taking a step back and saying what are the processes in the healthcare system that have to be addressed in implementing genomic medicine and I think it was interesting we sort of have been focusing on one variant at a time one gene at a time I'm sure at some point somebody's going to try to make an application for that toolbox that says here's your whole exome how do you act on that but I think all of us in this room can realize that it's very difficult to do even when we do this one variant and one gene at a time so we have to be careful what we bite off here and I took a lot of other notes I'm sure you did to Rex is there anything anybody else would like to summarize about the resources section that we've just had alright thank you. One point I might make Mary is that we're very sorry that Moine was unable to come he injured his back and couldn't get on a plane and so he has agreed that we can send out his slides or at least he hasn't disagreed. That means he's agreed. Yeah that's right that's right so Teji maybe you could just mail it to the same list that you mail the booklet. Yes I'll do that this afternoon. And then lunch is as I know many people have been hungrily watching them lay out our lunch out there we're planning to be back at 1.30 please do be prompt at 1.30 and we'll start back up and see you then.