 All right, so what we really want to do now is try to have an open discussion to really think about many of the general themes that we've heard today. With the goal of getting to, at the end of our discussion period, some very well-defined outcomes. And actually, just to remind you, and we'll come back to these in a few minutes, some of the outcomes we had imagined up front were things like a white paper to describe needs in the U.S. that arose from today's discussion, commonalities and interests or opportunities across the agencies, initial use cases, and I think there are a couple we might want to discuss this afternoon, plans for communication and collaboration across the agencies moving forward, and then needs and goals for interaction, and this is related in some ways to our GM 6, which is an international meeting that will be hosted here in October, November, September. So does that sound like a reasonable plan to everybody? So the first thing that we heard, I think, quite consistently from a number of the speakers was the need for standards and IT infrastructure, and this included things like consistent electronic health records data, how do we share data across sites, and I guess maybe what I'd like to do is just take each of these items and see if there are additional comments that should be added to this. So for example, in the area of standards, what process could we use and which agencies here might need to be involved in that standards discussion? So an obvious one who's actually not here today is the Office of the National Coordinator for Health IT. That's clearly an important agency, but I was also struck, there seem some really great opportunities, and we didn't hear much about the plans, maybe it's just too early, but we heard about the creation of this defense health agency. So is part of the agenda there to come to a common IT platform, or is it just too early to know what's going to happen? Yes, sir. I believe that we have a common IT platform and that the Air Force pretty much has the stick on that. The larger issue is trying to have interoperability between the DOD and VA in terms of the EHR. And the VA system reportedly is a good, large health IT system. Are the barriers there just that they're two different governmental agencies? Because it seems like it's a national transition from being in service to being veteran. I'm going to go with that's a political problem, which I don't deal with at my pay grade. They share a common ancestor, the two systems, and so ideally they should be able to, the extracted data at least from the two should be able to be interoperable. There's a blue button option that's being explored. So kind of putting apps on top of the existing EHRs that are communicating with each other. And that I think has had some success so far. So I just wanted to point out that while I think interoperability in that is a huge issue and an important issue to address, I think that as I think about standards in IT infrastructure, I'm thinking about it at several levels before interoperability. In the sense that there are standard terminologies and standard representations of data that go through and achieve a certification status by which any certified EHR would use that as their data standards. So as an example at the present time, we do not have any sort of certified nomenclature for representing genomic variants in the electronic health record. And so it really doesn't matter what level you are interoperable if you don't have a data standard to represent the variant. Even if you can move information back and forth, you don't know that you're necessarily moving the variant information in a reliable way. And so I think in some sense we may be at a more fundamental level amongst the different groups, which is to say, you know, what is it that we need, if we were going to implement genomic medicine, what do we need? We need genomic variants to be represented in a standardized way that we all agree on. We need to communicate those variants in a standardized way which we all agree on. There are groups like HL7 that are beginning to look at some of those standards. But what we don't have is the ability right now to move that through into a formal certification process such that it becomes the standard for all EHRs that are going to use genomic data that this is a standard that will be used. And my contention would be is that if we got all of the players around the table to say, here are the set of critical data elements and communication strategies that are needed, and here are the standards that are needed, and we all sort of marched metaphorically to the office of the ONC, then we might have a shot at actually moving that through some type of a certification for Health IT, CCHID, or something of that nature. And then we have a standard that we can all build off of. Yes, I was going to say about the VA data. So the VA data, even without the DOD data, and we're still working on that. It's also above my pay grade as well. But I do think that it offers a number of things. One is that it is a national data looking at medical record, and I can go in and look at the medical record of someone in another state and use those data if I need to for my research. I do think that what it has taught us is that there are many gaps in these data, and we need not only standard ways of recording things, and that's a huge gap. Another is to understand how clinicians prefer to use the electronic record for these kinds of things. So I think one of the ways that the VA data is valuable right now is to look now at how it is being used to record these events. So it does provide that. It provides a national way to look at variation across the country as well. Teri. So I thought I'd move to where you guys didn't, or I didn't have to strangle ourselves to get to a microphone. But I'm wondering if, given the opportunity of the Defense Health Agency that we're hearing about, is there some way that we can link you, if you want us to, or whatever, with groups that are putting genomic information into medical record in a standardized format, at least across those groups, to at least be able to see what kinds of standards they're proposing and whether you want to use them, or are those links already made? And I should not bother you. First of all, there is keen interest, I think, on the behalf of I shouldn't say all services at once, I can only speak to ours, but I think the Air Force would agree in this concept. But as far as the DHA, it's not even an entity yet. And yet the interest that you're speaking of is something that we can, at the appropriate time, fold in. But we have no idea yet as to what authorities it will have relative to each service's authorities on the activities of that service's research. That's how young it is. It's coming. They've even nominated a head of it. But there's very little specifics at this point that would allow us to plug in the concepts that you're proposing. But we're with you on trying to make something work at the appropriate time line. I didn't hear of you. Go ahead. Well, just to point out, I also serve on one of the shared services work groups for the DHA. And as Admiral Dole pointed out, the current issue has to do with just overcoming some of the statutory roadblocks that Congress built in years gone by in regard to how the services individually would govern themselves. And what that did, although we have a Department of Defense, there have always been concerns about equities and culture that Congress, having been made up in years gone by of individuals who have also served in the armed forces, have sought to permanentize. Now that we're moving forward to bring things together, there's no doubt that there is a bent to do so. However, recognizing the difficulty that Congress is having right now coming to consensus on a number of different items, we envision that it will take some time before some of these statutory barriers, which everyone agrees don't need to be there, but no one agrees as to how to take apart and how to merge evolve. In the meantime, behind the scenes, as has been pointed out, we are working to try to pull this DHA together, but it's going to be an infant when it starts out, and it's going to build to be more robust. And we still don't know which appendages are going to grow first, and which ones will be the laggards. So I would just add, in the meantime, my strategy for our program, since I've been asked to scale it back in the wake of the sequestration and cuts in our FY13 budget, my strategy has been to provide a series of roadmaps for implementation of genomic medicine in our enterprise. So using the Air Force dollars to, instead of figuring out how to do everything, at least research how we might do things and provide suggestions on that so that that material is available. So in that spirit, even before DHA has stood up, I would still like to, I would be happy to hear what IT solutions you may have out there, because we're still voting on the logo for DHA. That's usually the hardest discussion, is the logo. But in an encouraging aspect, there are 10 areas that are being considered for, if you will, some degree of consolidation. One of them is IT. And of the areas, IT is one of the more promising ones, given its relative age compared to how long research has been going on, how long logistics has been going on, et cetera. So my point being that in tying in with the Colonel's comments that that area has the most, what is one of the more promising areas that we could, when we talk about data mining and so on, that we may find ways to bridge across our system with whomever it is we're going to interact with, because there's a mindfulness to make the free services use the same database or software. Mark. I think the other thing that needs to be part of this discussion, since there are research implications of this as well, is the coming back to a point that was made earlier about the use of workarounds like natural language processing as opposed to using structured data. And I think another important thing is to identify those areas where we would say prioritize data and build systems by which the data would be entered as structured data within the clinical encounter, as opposed to those places where it may not be quite as critical, so that we can actually much more easily access data for research questions than we can if we're always dependent on either natural language processing with its inherent problems. And those text blobs that don't lend themselves to NLP at all where we fall back on manual review, which ultimately will stop us dead in our tracks. So that type of prioritization would also be important. I'd like to think about broadening this discussion about IT standards to two different areas. One has to do with clinical decision support, which was mentioned earlier in the day, and make a recommendation that perhaps one can consider a common CDS repository that all systems could use as opposed to many systems creating their own CDS frameworks. And I think this may be something that the CDS consortium that you mentioned earlier is probably working on, but it sounds like that's sunsetting, so the question is where does it sunset to and one potential avenue for it to remain alive and to facilitate a lot of the healthcare delivery systems and the armed forces as well as other stakeholders in the room might be to think about a CDS repository and have that as standard tools. Do you want to comment on that, Mark? Yeah, I think one of the things, and this is true probably with every consortia that's ever been constituted, is that the concept always seems a lot simpler at the beginning than it does when you drill down to the weeds. And so one of the challenges that the CDS has experienced is that when you actually get down into the coding of the clinical decision support that that's when a lot of the interoperability is lost because of the customization that's needed. And so that in some ways is the holy grail in the CDS-C world is can we again identify standards by which we could create CDS artifacts such that they could be much more readily utilized in electronic health records. Now, Cecily also mentioned another potential approach to this which has been presented to the eMERGE group which is I think at Harvard Partners of what Zach Haney's been calling the SMART approach which is where you actually have an app that sits on top of NEHR and where you could have access to cloud-based clinical decision support hosted by anyone where you could import artifacts through this app that would allow them to run moving data back and through this port in a secure way to actually give you the information that you needed. And so those types of solutions beyond building everything into the EHR may be more pragmatic over time, but I think somebody, I know that there's a lot of enthusiasm amongst the groups that were in the CDS-C to continue the work that they're doing. The issue then is could we create a place where we could bring them together, take advantage of what they've learned and continue to play, but maybe with more of a focus on the things that we're particularly interested in. So there's certainly this technology piece of the platform which I admit sounds like it could be challenging, but even something a little bit more pedestrian as a common set of rules that if somebody wanted to implement their own software, at least we have a common set of rules that are underlying clinical decision support. So that was I think also implied in my recommendation. The other area that I think it's related to the second bullet here on evidence generation is oftentimes during the day we talked about outcomes and I think again Mark Eud and Dan talked about the narrowness of some of the ways that we think about outcomes. Can we together somehow expand that platform to have patient centered outcomes systems types of outcomes as we might use if we're managing a health system or managing a practice, or even think about the providers as part of the system that we want to measure so we can think about this framework for outcomes measures that could be more or less uniform perhaps adopted to across a number of the groups here. So CDS and outcomes to me seem like they're important parts of the I, where does the data reside, how do we capture it in order to make the measures as part of real-time, real-world care instead of doing RCTs and generating case report forms and things of that nature. I think for the outcomes it wouldn't necessarily mean that you would always need to do the same thing, but I think what would be very useful there is to try and define the universe of outcomes and to create a repository of here all the possible outcomes that could be considered and here's where you might consider using this outcome versus that outcome. Dan and I were talking offline about the idea of a lot of the things that we default to primarily because they're easier to do is process outcomes and we assume that if you do something that's recommended like check on A1C or do a foot test or something like that that has actually some chain of evidence to the actual health outcome of interest, but in many cases that chain of evidence is if it exists at all is fairly tenuous and wouldn't hold a lot of weight. But if we invest all of our effort in measuring process outcomes because they're technically easier we lose the opportunity to develop methods by which we could perhaps measure true outcomes, health outcomes that are really meaningful and that people in the broader community would pay attention to much more significantly because they directly relate to clinical utility. And so I think some balance of the portfolio of outcomes and looking where investments could really help to improve the ability to utilize certain sets of outcomes that seem to be most important for certain projects. Matching the outcomes to the project would be important. A couple of comments in that regard. First of all I would mention here with very little knowledge but CMS has this meaningful use payment program and whatever IT solution you discuss I would suggest involves those people to make sure that it meets those requirements so that changing systems or implementing new systems helps providers get paid under the meaningful use of the concept. Secondly part of the issue in measuring outcomes as you said is having simple ways to measure those. CMS created and CPT followed codes for the PQRS system for measuring outcomes. Some of those were processed, some of those were clinical outcomes. Blood pressure less than 140 one of those outcomes where CMS initially created what are called G codes and then CPT followed up with actual CPT, no they followed up with F codes. So if there are some outcomes that are developed and you're coordinating with CMS and doing that in their meaningful use quality outcome processes then perhaps codes can be developed that will help measure the outcomes as you do that. I think Steve raised an important point and actually I think this is the first time meaningful use criteria has been mentioned around the table which is unfortunate but I'm not sure particularly as the military starts to develop its system. Probably that's not so terribly relevant to you and yet it's what's driving a lot of the health IT outside and if you can at least have some interface with it it would be tremendously helpful. Just to speak to meaningful use there have been a number of us that have been engaged with that process either directly or indirectly to try and get some of the things we've been talking about represented there. The reality of course is that as with everything else the groups to some degree sitting around this table are also at the informatics leading edge and we're addressing problems that are well beyond the can of most systems that are trying to implement their first EHR or to use it for very basic clinical processes. So in some sense the meaningful use process as it's currently constituted has been focused on those lower level types of things and it's been more difficult to get even things like meaningful use of family history represented in the meaningful use targets. So as an example even though a meaningful use of laboratory information, communication of laboratory information and electronic means from a laboratory information system from electronic health record is part of phase one meaningful use the issue that some of us raised which is well you know genetic tests are laboratory tests that should be explicitly articulated as part of the ability to do that. That was essentially put on the back burner. I understand the pragmatic realities of trying to get that program up and running but and I agree with you that we should try as much as we can to continue to engage and move things through meaningful use because if we can get it there then it's really going to have a major impact. The challenge has been the audience has not been particularly receptive to these types of use cases which are well beyond where most systems are operating. CMS is a huge agency not a lot of conversations it's not uncommon for conversations to never be had between the various sections of that particular agency. I think some of the implementation of meaningful use might be in the areas that you mentioned might be helpful might be improved if there were other people within the agency who are interested in outcomes were involved in providing that input to those particular folks so if we can get the right people discussing it outside of this group with the right people within the agency such as coverage people saying we need data this data needs to be clinically available meaningful use needs to include the ability to collect this data there may be some better outcomes in what has been seen in phase one. We'll take names I have names my name is a no there but we are so maybe we can bring this session to close and not to put you on the spot Deborah but in your discussions at CAP has there been discussion about reporting standards for genomic data as you thought about your framework? Yes, one of the things we wanted was we're going to be advocating for more standardized model report templates for genomic test reporting that will be useful not only to the clinical end users but also reports that potentially could be understandable by patients themselves because the PHR I mean it's one thing that we talk about the healthcare providers not necessarily understanding the results that they're getting but these are going to potentially go into the PHR and patients are going to see them directly so yes it is one of the things that we're advocating for If I can follow up on that Deborah is the recommendation that has been made in some venues to move to a synoptic type of report which is in regular use in anatomic pathology and has been discussed in the context of genetic and genomic test reports is that under discussion as well? Well that's a model that could be considered because the synoptic reports have been very useful for incorporated incorporation into copath and other laboratory information systems and standardizes and make sure that the clinically essential information around cancers is reported in every single surgical pathology report related to that cancer and so the same type of process could be developed around genomic test reporting. Just for the great unwashed of which I'm one, what is a synoptic report? Oh it's basically a report template that says that you know this is a such and such a cancer and then there are specific things that are reported as to the size, the nearness to the margin the number of nodes involved did you see? Yes. And so it provides it in a very structured format that's easily searchable afterward. I think it's still I don't think it's still in separate fields I think it's still within It's still in a text document although there are some systems that it lends itself to creating a structured document and in fact one of the things that CMS is specifically investing in through meaningful use is the use of so-called continuity of care documents which are clinical document architectures that are both machine and human readable so that the critical information is represented both as text that a human can parse but also as structured data that a machine can parse and so this would be I think if CAP is involved in this that would be a model to move to that would I think make it much easier to approach meaningful use and that's something that the HL7 genomics workgroup has been looking into in terms of developing these types of CDAs that can lead to continuity care documents. Right I think maybe we should move on but I like to go out of order because it sort of follows the idea of a curated database so there's been a fair amount of work we had actually Mark and I co-chaired a meeting a year and a half ago now about what we're now calling a clinical no CERVR so so Terry I don't know if you can maybe give a brief description of that so that people who don't know about it are aware of it and maybe comment on how far you think that would go to providing a curated database that some people have referred to? Sure and I might know that there are many curated databases that people have been discussing so one curated database is the very small subset of genetic variance that might have some clinical action that needs to be taken on them has been called actionable variance and that is a term that has various meanings to various people but what the clinically relevant variance is designed and we purposely picked a really long and ugly name so that somebody would come up with something way better but the way better ones that we had were either objectionable to some or they were already taken so but be that as it may what we're trying to do is to get a group a consensus basically of experts and one can define them in any way one wants to tell us and the research community what are the variants that there is really pretty strong evidence that you can do something about that they're important, they have an important impact on risk and that risk is modifiable and so far most of those that have come up have been either the hereditary cancer syndromes where one would give advice for more frequent screening even in the patient but certainly in the patient's relatives who carry that variant or for some of the pharmacogenomic variances as well but there are undoubtedly others and the goal of this group is to survey the literature on an ongoing basis and the available evidence and basically make recommendations as to which are those that should be considered in that realm and so it's something that we've engaged several of the other institutes in there's a lot of interest unfortunately not a lot of resources but a lot of interest in helping us to get that off the ground and we hope to be able to announce awards very very soon hopefully early this summer so that's what that particular database is. There are other databases maybe I might ask David Ledbitter to comment on just looking at all of the variants that are identified in clinical laboratories that are doing sequencing, either targeted sequencing or other kinds of sequencing and when these variants are picked up they're useful both being linked to phenotypic information and not linked to phenotypic information. David can you comment a bit? Just for the group, Liz Mansfield mentioned it briefly in her talk, there was a group that was based on the whole genome copy number variation that was called ISCA that was a voluntary collaboration among clinical cytogenetics testing labs to deposit their whole genome CNV data along with whatever phenotypic data came in with laboratory test requests into DBGAP and then that data would also the calls, the variant calls clinical information into ClinVar. So it's been a collaboration with NCBI through DBGAP and ClinVar for several years and in the last year we've reached out to the molecular diagnostic genetic testing labs to talk about the problems of acquiring genotype and phenotype data as the basis for evidence based review of what's clinically relevant and the problems being the same for sequence variation as they are for structural variation and some of the evidence based review processes that we had developed would be useful to evaluating the level of evidence around genes first and then variants within those genes in terms of what's clinically relevant. In our definitions it never included actionability as opposed to evidence that the gene was associated with disease or phenotype and that the variant was a functionally important variant. So those two communities have come together and applied for a U41 database grant through genome starting a little over a year ago and sent a revised submission still prior to the CRVR RFA. That application included a major partnership and collaboration again with NCBI ClinVar but also with some other database groups and with some international participation. Now I'm still trying to think of the order when the CRVR RFA came out. We then contacted colleagues and were active participants in constructing one of the CRVR applications and there was some degree of overlap in the work around curation because both the database required curation and the CRVR resource required curation. So that was with one of the CRVR applications and we're all now waiting to see what the final proposal is but all of the groups involved are very eager to work together and figure out the most efficient system to come up with standards for evidence review, collect as much data as possible as a byproduct of clinical care generating whole genome data, either structural variant information or sequencing information. So I think this is one where I'm tempted to declare victory and move on. You won't let me get away with that. One of the frustrations that the College of American Pathologists has is that much of the work is focused on inherited disorders and not a lot on cancer so I don't know what the cancer resources are and I don't know if the FDA project is looking at both cancer and inherited and I know with the RFA's that were put out, it included cancer but most of the respondents are likely to focus on the inherited first and yet cancer is one of the areas where the genomics is kind of moving very, very rapidly and definitely is tied to therapeutic implications. And so I wonder why this, I mean maybe it comes from NHGRI having more of a genetic focus than NCIB in cancer but it I think is detrimental to the thinking about curated database or database says by not including the somatic variants related to cancer. We are certainly very interested in somatic variants. One of the reasons why maybe it has not been as big on the radar screen is some of this data is generated and held by pharma companies. The people who are developing the drugs or it's not the same sort of research based enterprise, you know publicly funded research and so on. But we are also looking through, we have a program in CDRH called MDIC and don't ask me what that stands for because I can't remember where it's supposed to be a sort of pre-competitive space where diagnostic companies and other companies can get together and one of our ideas was to actually try to move them into this database idea because it has regulatory benefits for them. If they can point to the database and say see it's there, it's in cancer and it responds to this, we don't need to do a clinical trial for the test or whatever then I could see where it would be beneficial for them but that's still pretty nascent. I think what you're looking at is a completely different kind of enterprise not publicly funded research base but industrial. And one of the things that a lot of this testing is being done in the biology departments and so one of the things that the CAP is looking at is developing an academic consortium of pooling data across cancer clinical cancer testing so that we can see what other people are seeing rather than laboratory by laboratory. Maybe I could ask our NCI colleagues so I know Andrew Friedman was here and probably still is and Moine as well perhaps you could comment a bit on the somatic and the germline interests and both in our resource and in the resources that you all have. So I'll take off my CDC hat put on my NCI hat so we have a definite keen interest in the cancer genome I mean as you know there has been TCGA which is a big NCI initiative now in collaboration with genome What's that name of the institute? Okay, oops. Now okay I was going to go into the database. Not even for a poor institute to spend on one disease. We definitely have keen interest in databases so as part of the CRVR discussion I think we may be heading in that direction as well. I mean it's too early to tell right now but sooner or later and I'm glad that FDA is also interested in this so maybe we can have an offline chat later on about trying to have that resource for the community co-constructed by FDA and NCI and HGRI and whoever else wants to be on it. I don't mean to malign the TCGA effort because it's amazing and doing good things but clinically you look at that information and you have no idea what to do with it. So it's not tied to clinical outcomes or drug responses or anything so it's this huge catalog of information that you don't know how to use clinically. Brad, thank you. You're welcome. I'm the program director. So when is that happening and what's going to happen with that data? Actually TCGA is winding. It's on a trajectory to be over in about two years with its goals of sequencing more than 10,000 tumors from 26 different cancer types. Just this huge discovery effort that will be completed in about two years and we're working with NCI now to develop the next iteration of projects which we recognize has to include treatment outcomes and that sort of data but it's under development. Whether this fits or not there are over the last decade there have been a handful of databases created as a result of CMS coverage applications. A requirement to get paid includes submitting data to these databases and there have been thousands and in some cases millions of patient information submitted to those most in cardiology, plantable defibrillators, carotid stents, intricular assist devices, PET scanning has won. As a result of what I just mentioned briefly in my presentation of this coverage with evidence development concept there's challenges in doing that. There's sometimes dangers in taking coverage decisions to CMS and that they may decide things that you don't want them to decide but at least in your considerations of how to get people to submit data to whatever database you have exploring that option may be beneficial. I just want to throw out as a follow up to the cancer question coming at this from more of a research angle we have projects like the thousand genomes project out there TCGA clinical sequencing that's going on. It would be really good as one of the speakers mentioned earlier that if there could be sort of flow in both directions and we have data formats like VCF files and formats you know that if we could sort of standardize that somehow across the disciplines that should be a benefit. I agree with Brexit this is a success but I think what we do need to look at are the rules of putting anything into a database and I think if we can have some standardization of you know be it informed consent, be it notification, would you allow limitations I think we're all under some different HIPAA rules in terms of sharing across institutions. I think it would be great value added if we could kind of hold hands and really chart out something that would work for a lot of institutions. I know a lot of the DOD has some different regulations as well. So anyway just as a little I think in our effort one of the things we recognize even if a database is curated they're not all curated the same way. There's not a set of rules that everybody's agreed on and a lot of things that are called databases are really repositories and there's a huge difference between a curated database and a repository so you know some kind of like industry standard rule or something like that for what a curated quality database would mean would be really interesting. So I think in this general area of standards IT infrastructure and curated databases we've heard from everybody sitting around or many people sitting around the table that this is really a common theme that crosses all of the agencies that or most of the agencies that are here. So put that aside I think that's something that we should probably think of as we come to what next steps are that there's clearly some work that we could all work together on in the area of standards IT infrastructure and curated databases. Dan? I just wanted to make sure that we sort of say it right that the curated databases we're talking about are not just collections of genomic variants but they're coupled to patient outcomes because I think that's the only way that we're going to figure out what the rare variants do going forward. Not an original idea but I think an important point. Maybe we use that as an example or a basis to move on to the next one one of the other things we heard arise several times in the discussion today was the need for more evidence. More evidence to support that genetic variants actually are whether it's actionable or actually have clinical validity we need to be working together to think about evidence generation so be interested in hearing again just sort of from folks around the table what are the important next steps in terms of evidence generation for example is there an opportunity in the evidence generation to leverage the large amount of data that's in the VA system or that will be in the military healthcare systems or that are obviously in a large number of academic or even non-academic healthcare systems what's the role for CMS in this data that CMS has access to that would help in the evidence generation piece. One of the issues I see in sitting and listening to all the different groups talking is that there seem to be basically different statutory rules by which each group operates that actually prevents coordination across the agencies and groups and closer alignment and coordination of the work being done so how does this tie to evidence. I think each group has a different definition of what the evidence quote evidence is that they need for their particular purpose and so I don't know how you get to alignment of evidence without dealing with that issue but maybe that's too difficult. It may be useful to ask both the folks familiar with newborn screening and those familiar with the EGAP process what evidence was I mean did you have challenges getting everybody to agree on what evidence was enough or what evidence you used and maybe tell us a bit about how we might use that experience to figure out what standards we might want to have here. Well from the newborn screening perspective I think most of us would agree that discussing is consensus based using experts. Now I think that you could make a pretty strong argument that if you look at the initial disorders for newborn screening like PKU and Galactosemia and this sort of thing that you didn't necessarily need to do a randomized placebo controlled double blind study to see that reducing the phenylalanine in the diet had profound impact on children that were identified as having phenylketonuria but if you ask that question a different way which is to say well what's the treatment target of phenylalanine serum phenylalanine that we should be targeting our treatment to we've been treating this disease for 60 years and there's no evidence to suggest that we understand you know what that phenylalanine level should be that gives optimal outcomes relating to cognition and other things up until relatively recently it was really thought appropriate care to discontinue dietary restrictions at age 6 when brain development was quote-unquote complete and I think we now know that that's a very different story. So the point of that in addition to being responsive to your question is that when we talk about evidence there's vast differences in terms of the levels of evidence that can be used and there's vast differences in terms of the outcomes of interest and so if your outcome of interest is let's prevent profound mental retardation and seizures and a child that needs to be institutionalized there's sufficient evidence to say that dietary restriction if your evidence is we want to optimize cognitive outcome and minimize the impact on patients in terms of the dietary restriction that we do not have sufficient evidence and so I think this what that really constitutes is what Deb is saying which is you know we need to be able to agree on you know what are the outcomes that are of interest and what level of evidence do we need to achieve those outcomes and you really can't separate one from the other if you don't understand what the outcomes that you're looking at are you can't decide what the evidence is and but we really don't even have any venues by which we can have these discussions other than the I think the attempts by Muin and others to pull together stakeholder groups including NHGRI to say you know can we talk about this but there's none of these have had a sustainable effort where we've had any sort of tangible output to say here are the considerations to be able to pick this level of evidence for this particular addition what's the prevalence, how many patients are involved, what's the outcomes etc at the very least we should have a framework of deciding how to match outcomes and evidence and even across the agencies it seems that there's different evidence needed by CMS, private payers, FDA, military health care is for I forget what you called it but for the military person's functionality in the field or whatever I mean so there are different criteria and of what's trying to be achieved and so I don't know if you're going to get a consensus about the evidence that's needed because everybody's looking at different parts of the elephant if you will. If we had something like the US Preventive Services Task Force I mean some kind of nationally recognized body that determined what those standards are and gave graded levels of evidence I don't think anybody could argue with that. Right but how much of medical practice is covered by the US PSTF guidelines? Not a whole lot. And so there's a lot out there to drive the use of genomics or not in care and I don't know that that level of guidelines and evidence is going to be possible for everything. You're right when that does exist it does have consensus. And I guess I might I might ask whether we really need to define what evidentiary standards each agency would use or we ask them each to define it for themselves but the evidence generation is likely to be in the same or similar platforms and collecting the same or similar information and however it gets used I mean yes it would be nice to know exactly how a given agency is going to use it and be sure that we collect that information absent that there's probably an awful lot we could probably get 90% of the information I'm just guessing that one would need it at any given agency that might be common across all of them and that may be where we want to start and then perhaps that would drive some thinking as to what is it specific to readiness or that's specific to CMS reimbursement or specific to FDA regulation whatever it might be. I was thinking about how to tackle this one. So if the purpose of the genomic medicine enterprise is to implement genomic tools then we develop those tools you develop those tools and then you subject them to the current requirements by FDA CMS, ARC whatever and at the end of the day you wonder why they're not being implemented because they don't fit with the current evidentiary I mean it's a messed up system as you know I mean the SACGHS looked at the oversight there was an oversight report in 2008 was published which has a very extensive analysis of the oversight trajectory for when the FDA steps in CMS clear and then you have professional organizations it's a patchwork of work but what you have to realize is that if there is no generic exceptionalism and I've heard that mentioned a few times then those new tools no matter how cool they are they have to lead to some measurable outcomes we can debate what those outcomes are and they are very specific depending on what the context is so for diagnostic purposes for a rare disease the end of a diagnostic odyssey may be an outcome that's fine even though there is no treatment but at least you get the diagnosis if you're trying to do screening the whole population or population screening have to be adopted if you're trying to do a companion diagnostic a la FDA where the treatment matches with a test whether it's a genomic or some other test then there are evidentiary requirements so we don't have to reinvent the wheel on this I think the wheel has been invented there are enough I mean I see that shaking your head I think that's positive sometimes you shake your head in a negative way I'm agreeing with you I also think we need to appreciate that probably regulatory and say payer requirements are going to be different because there are going to be different standards for releasing something into commerce versus what a prudent payer who is really a fiduciary of the monies of all of these people who pay insurance premiums is going to have to value to buy I don't think at least in our socio political science that they would they might have harmony but they would not be the same I was going to go and I'll give you the floor in just a minute is that okay we have all these new tools we think they improve outcomes let's test them let's test them out and it doesn't have to be all by NHGRI I mean many of them are disease specific tools like you know heart, lung, blood cancer I mean get the NIH community to buy in get the private sector to buy in and let's start doing some of whenever they're needed some of the trials that need to be done the RCTs and occasionally when they're not needed you can do the observation studies while you build the clinical infrastructure like emerge and electronic health records to supplement clinical trials with observation data so there is really nothing magical about evidence that it has to be collected we can't pretend that we don't need it because at the end of the day we need it one way or another we need the evidence otherwise genomic medicine would not be implemented so let's do our horizon scanning figure out what the applications are and then depending on what the application is and I'm suspecting all the roads will converge to next generation sequencing at some point rather than gene by gene type analysis then I think you know those can be subjected to the already known and established principles of evidence and see what kind of studies need to be supported whether new RFAs are needed and so on and so forth so no new territory needs to happen here I think it's same old. So I think there's a good rule of sort of rule of thumb in the diagnostics literature about observational versus RCT you know most diagnostic literature is observational or that's not fair it's comparative but it's not RCTs but really the area where RCTs become urgently needed is when you're either dividing a new disease or a new spectrum of disease so we need to be maybe thinking in kind of a field with is this something that replaces or improves or maybe doesn't improve a known way of diagnosing a known strategy and interventions that follow it or is actually going into unknown territory where we are really not clear what the disease is or whether we have found a whole you know group of individuals who's been under the radar because they weren't detectable and does that mean they need to be treated or should they be left alone? You know some of the high technology breast imaging has brought up that issue with respect to it's actually increasing the mastectomy rate good, bad, I don't know. Oh I was going to say I don't think we want to sing you know it's kind of like I hate this phrase anymore because everybody uses it for everything but fit for purpose and evidence. You don't want to wait until you have a reimbursement level of evidence necessarily because then nothing's ever getting on the market frankly. So there's a tension there but you have to start somewhere and when I talked about evidence in my talk I was primarily thinking about cancer and targeted drugs where you're actually using a companion diagnostic and now there's I don't know how many companies with these panels and doctors will order them and it says well you have a KRAS mutation in your you know osteosarcoma maybe you know you should be treated with this drug when in fact nobody's ever studied in that population you know it's it's that's the kind of thing that can really derail because everybody's doing their own thing but yeah I think you need to define what the context of your evidence gathering is obviously. I'm from CMS with Dr. Furrow and all of this it's probably a more shameless suggestion than anything else but if this really were something that a group wanted to take on and try to take try to get some traction with I think in what you're talking about with the KRAS test I think the existing CPT codes that CMS is wrestling with right now would potentially be a really good place to start. The information that CMS is given frankly comes from industry, it comes from commercial research, private research and so there's not a lot that we get to see. I would love for a group like this to tear into some of those tests and some of the CPT codes and see what came out of that and I would suspect that there would be some any generalizable points across the tests that are currently marketable that are currently out there that could then be applied to tests that are coming down the pipeline. And so again the shameless part is that it would make our job significantly easier but again I think when you think of low lying fruit I think their potentially would be something there. So one suggestion thinking about this discussion would be as genome and NIH NCI think about doing demonstration projects, outcomes oriented clinical utility studies with genome based technologies. One of the things that's absent from the discussion particularly when pointy headed academics are designing these studies is really engagement of the FDA and CMS and the payers and what are they looking for in terms of the evidence or the outcomes to be measured. It seems like a no brainer to design these studies with the stakeholders that eventually have to make the decisions about coverage and regulatory pathways to be at the beginning rather than you get to the end and then you realize you've done the wrong study. So I'm just suggesting that there may be some ways to engage in some of the efforts that are currently being teed up by NIH to deliver on the country generation component to really engage some of the other stakeholders around the table at the very beginning of designing those studies. So I think Elizabeth has really identified something on the horizon which is critically important so maybe is an opportunity. So often we're you know instead of looking at the horizon chasing it there are now on the market all kinds of proprietary multi-multi-multi biomarker tests that anybody with a cancer has their tumor evaluated by. What a good part is it reflects the understanding of cancer at its molecular basis not so much the convention of a site specific disease. The downside is nobody is really assessing this data and understanding where it's going so it's becoming kind of quite a random occurrence being applied to patient treatment. I think it is critically important and it's out there right now. It's being marketed right now. The college is very aware of all the different cancer gene panels that are being developed and we actually have a work group in the personalized health care committee that is trying to design a quote recommended cancer gene panel that would be pretty inclusive and could be used across all solid tumors. Now payers aren't going to like this but academics are going to like it because right now if we did try to pool data those people doing gene panels it's pretty easy to add some genes or remove some genes that people don't think is useful once you're doing next gen sequencing for cancers. And if there was standardization of the panel it would facilitate combining the studies being done with the outcomes I mean the tests being done at different clinical centers. And so I don't know if this is something NCI wants input on or someone else wants input on but we would very much like this not to be a college specific activity but we're making a start. Mark, I think it's even more complicated because tumors are known to be heterogeneous. So you can't simply say a tumor has mutation X. You can say you can do quantitative allele calls and people are now trying to develop models of clonal evolution and things like that. So I would be careful about building something now that isn't going to be appropriate as we learn more about the evolutionary structure of tumors and how that relates to biomarkers. Do we want to target the major clone or maybe we want to target the minor clone? But when I talk about a gene panel it's actually a very large gene panel. It's not like a 48 gene panel it's more like 500 so that we could do that kind of process of looking at the rarer clones the recurrences when they come back. I mean so you would have clinical data that could help with these kinds of pathways and evolution discovery processes. Well but you need quantitative information as well. You need spatial information. It's very I mean... Yeah pathologists are very aware of that. Yeah and I think the informaticists are struggling with how to even represent the data so it's... as we've talked to Bob Grossman in our place it's like I don't know what to do with this I mean this is all research at this point. This is not ready for prime time from the standpoint of tumor heterogeneity. It's not ready for prime time but there are markers and variants that are ready for prime time use and so you would... you're basically setting we're trying to set up a model that begins to be able to do clinical trials based on the pathways affected rather than the tissue of origin because like you were saying if you've got a K-RAS and an osteosarcoma you don't know what that does but we would be able to begin to look across tissue types and tumor types with the same gene panel. So I just want to push this idea one step further because I think Deborah is saying let's you know let's say I have a sister and I had a way of adapting it but I believe you know the other comments here on the table is so what is the approach that you take in studying this, assessing it, it's very much a living, evolving type of process. Right now clinical decisions are being made based on that. I don't know if they're good or bad for patients and so how's this going to get for you? Yeah it's extraordinarily complex and you know I would applaud the brilliant community of minds who could figure out how to approach this and then push to get it done. Understanding that it'll keep moving. So one thing I might suggest or maybe we could consider it sounded as though somebody around the table or several were making the case for getting advice early on as studies are being designed as to what would be the most useful information to FDA or CMS or payers or other groups. And so NHGRI has this genomic medicine demonstration projects program that is about to get started and we would be delighted to show you the programs that we're planning to do and obviously you can't completely redesign them but if it were possible for y'all to say you know you have to say this is great but after you say this is great you know it'd be even greater if you could add on this little piece or if you collected this piece of information would be tremendously valuable to us. Something that we might not even have thought about. I know when Howard McLeod was here in some of these discussions a couple of meetings ago he said you know there was a payer that wanted to look at some outcome. It wasn't reducing hospitalization it was something even simpler than that. Pardon me? It was adherence. Yeah and they adherence to a scientist is like you know that's soft science but extremely important to the science and so there's probably things that we haven't even thought about. And if there are folks at CMS and FDA who hopefully will not tell us oh no you can't do that because that's what makes people afraid to then consult you it would be I think really great to have you engage. One of the challenges that CMS has a little bit different than FDA. When you go to FDA and FDA says bring me two randomized trials or one randomized trial and three firstborns or something. There's some fairly clear guidance. CMS says bring us evidence that's reasonable and necessary. And they don't define that. And there is no while CMS commonly will meet and likes to meet with trial designers before they begin their trial they will not say if you design the trial this way with these outcomes here's what we'll do. Because they will not pre-define those RNN decisions. The process is also a good bit different in that FDA says do this and then behind close bring us this information behind closed doors except for their advisory committee comes up to the decision and publishes it and it's effective. In the CMS world if you're doing a national decision you have this process that includes public comment on a draft decision. So there are some differences that make that particular issue of saying here's what we if you do this, if you bring us this kind of trial with these kinds of outcomes here's the results you'll get. You can't get that from CMS. They will, this is in the coverage group they will tell you and FDA this is what you needed to do. It would be very helpful to us if you added this particular outcome or if you looked at this age group or if you stratified by 65 plus or some of those things that they commonly ask for and if you read their decisions you could ascertain on your own but they will tell you that but there are no guarantees as to what they'll do with that information. I don't think that at least that wasn't what I was suggesting it was just what would be useful to y'all that we could generate if possible. And unless something has changed in the last week or changes in the next few weeks they're not going to say here's some guidelines for this general classification of trials. They'll not do that. But I think the question is more broadly are CMS and is FDA and some of the other folks sitting around the table interested in being involved in those discussions for how to generate that evidence. It is a coverage issue more than it is a payment issue and I don't wear that hat any longer. My assumption based upon what I've heard the current director say is that he would be happy to do that but we'll have to ask him directly. Because I think one of the goals that we want to do so we talked about standards IT and databases that's clearly one area. Second area is this area of evidence generation. I think what we want to come to in half an hour or a little longer maybe is are there ways that the people sitting around the table should be engaged after we leave this room to move this agenda forward. We've heard how important it is. We've heard how it's not always been easy in the past when people have tried it but is it time to make another run at it? For me one of the other things that I'm struck by is the point that's been made a couple of times today that healthcare reimbursements are likely to change. The way healthcare is provided is likely to be changing. Strikes me as a particularly good time for us to be having these discussions because then we're at the table when the decisions are being made as opposed to having to retrofit everything into what the new world order looks like. So it seems to me 15 years of frustration for you and a side I think now may be a particularly important time for us to be revisiting this and thinking about how do we sustain the conversation and the activities after we leave here. There's someone behind you but there's also... I just had a comment about the different evidentiary thresholds that everyone has for good reasons. FDA, CMS, professional guidelines groups. I mean all of the gatekeepers that we refer to I would imagine too that bringing a group of people that are interested in discussing this to the table can come to some common evidentiary standards that are baseline so that researchers and people in community settings can understand what that is and then can focus on that. And I think this is what Rex is also talking about because then what we haven't discussed but I'm sure we'll be discussing in time is that unless we do that then health disparities and disparities of care based on whether CMS is covering it or whether the institution is going to absorb the costs because it's unethical not to do it and with moving standards of care which we don't really have standards of care in genomic medicine that just feeds into the cycle of if it's not covered then we have disparities in care. So I would imagine that there are some commonalities amongst all of the different gatekeepers that can be shared with the researchers around this table and also with the community environment and the community hospitals so they can strategically plan because I think that's one of that's a really difficult area to plan for especially with mixed you know a large pair of mixes. So one point that might be another bullet point up here several speakers brought up earlier you know is the idea that we have to develop a genomic medicine plan that is effective with everybody in the population and that we know from projects like 1000 Genomes that we've sequenced 1092 people now and have a pretty good handle on how demographic history is different all over the world. So somehow that has to be layered on top of this whole thing. Can I, oh sorry, yeah so I was going to recommend we have a draft guidance out that should be going final soon on clinical studies or clinical trials or something like that that may answer a lot of questions that people have. I mean we've actually, the long guidance talks about how to do studies and so on might be worth looking at. The other thing is you know being really blunt here genomic medicine if you're worried about what FDA wants here you know we don't, we still are practicing enforcement discretion for laboratory developed tests and you may have noticed that we have yet to approve anything in next generation sequencing. We have approved very few genetic tests. So if you think you need FDA input on all of this I mean that's kind of a joke. It's, you don't go straight to market. You can make anything you want. You can sell it to anyone you want. Just call it an LDT. You can take that as a tongue-in-cheek remark or not. But yeah, so I just want to you know don't, yeah I think if I can jump in and I'm sorry Amy but I think what at least some of us are asking is how can we work with you rather than outside of you and there could be some ways in which we can generate information that might be helpful to you or perhaps not. But that's more, I think. Can I respond to that? Yeah, so I think that's really great and we're happy to work with people and I'm not kidding about that anybody, academics we don't care. What we don't want to do is invest I think a tremendous amount of time in advising people what FDA would like to see and then you go off and do whatever you want. You offer us an LDT because then we've just poured a whole bunch of time and money down the rat hole and we still have no idea how the test works so serious inquiries only I think they used to call it in the one ads but yeah we're definitely happy to talk to people who really want our advice are going to follow through on it. I want to go back to the example of these multi biomarker panels for tumors. I actually think it's pretty readily apparent. The outcomes that are of interest not only to payers but I think these are pretty important to patients to physicians and so on. You want to know whether this information improves the result of treatment. You want to know that beyond the markers of tumor response or progression free survival or disease free survival, you really want to know whether there's longer survival or avoidance of toxicity avoidance of drugs that won't work. I think all of those things can be readily defined. I think the real challenge in the situation we're describing given the complexities. Okay then what methodological design one that will be adaptive because you cannot be too static in this situation will actually give you reliable evidence on those outcomes. That's where I think the real challenge is I think the outcomes are not so hard really. Just as a current example of some of the interaction with FDA and CMS and trial design is NIDDK recently brought a group of researchers over to CMS. They had been working to design a trial for artificial kidney. Set with the CMS coverage folks and said CMS coverage folks said well you need to do X, you need to do Y. Here's some other outcomes we think are important. Here's patient population you need to be particularly interested in. And then we think that information would be interested in. That's the kind of interaction that you can get. It's actually two agencies at the same date. But that's an example of like a single specific trial. I guess the question more broadly might be is there a metal level? Is there a higher level where I mean is the only value to those discussions to be on a trial by trial or study by study basis? Or are there some higher level guiding principles that it would be worth having some discussion about? I think it would have to be subdivided into groups that are so similar that the trials would be very similar. So a group of screening tests for pediatrics. I don't know what those groups are. You're smarter than I am for that. But there had to be some kind of segregation of the test into sufficiently similar groups that the study design would be pretty similar. Alright, so clearly there's an area of evidence generation that's important. Everyone's agreed is a key element. I think we still need to think about refining a little bit how we best engage the people sitting around the table to maximize the value of evidence generation for genomic medicine. In our sort of last few minutes can we switch to the areas of privacy data sharing, incidental findings, and then those obviously relate to policy issues. Because the common understanding about privacy and data sharing and incidental findings ought to inform policy discussions that are held. So maybe we can spend a little bit of time talking about the whole issue of privacy, incidental findings. We heard that raised a few times as sort of a scary barrier. One of the reasons we might not want to do let's say go to the extreme and do whole genome sequencing is we'll find other stuff. And when we find that other stuff, what are we obligated to do with it? I think many of us are nervous about a world in which we're so worried about not knowing what to do with stuff that we're not doing stuff. That doesn't seem like a very good place to be. But how much of a barrier is this whole idea of incidental findings and are there ways that the people sitting around the table should be working together to think about it? Bruce? So I think we need to be very careful in our vocabulary about incidental findings and think that there's a big difference between let's call them unexpected findings that are clinically useful and are well demonstrated to be clinically useful, which is a fairly small subset but an important one. Versus unexpected findings that are clinically valid but not necessarily clinically useful. Versus unexpected findings that are variants of unknown significance. And I think when we subsume all of these, which I think we often do, into a single category of incidental findings it creates a lot of confusion because in my mind at least the issue of what you do with variants of unknown significance is a very different question from the issue of what you do with clinically useful unexpected findings. So I think we just need to be very cautious about the vocabulary that we use here. I take your point but one of the concerns that I keep hearing voiced is variants of unknown significance at some point in the future may be better understood. And are we creating liability problems? Are there other kinds of problems that arise from even uncovering those? So I have a question on incidental findings and in the case at least of next-gen sequencing is who are people, I mean what are you doing in order to generate incidental findings? Is next-gen sequencing or is whole genome sequencing being done widely and you're looking at it and saying oh my gosh you have cystic fibrosis? Or are you testing for a reason and you're testing a particular gene and you're finding a mutation you weren't expecting? I mean what are we talking about here? Again we're sort of preempting tomorrow's discussion but maybe a few words on that. So the notion is is that the testing is indicated for the college fields, correct me if I'm saying wrong, it's reasonable to use exome or genome sequencing to diagnose an undiagnosed disease that's likely to be genetic. When other straightforward gene specific modalities have failed. So it's generally done for that indication. So you have an indication for the test, you have certain subset of the genome you're looking at to explain that phenotype and then what do you do if things outside of that set of results arise? And so what the recommendations say is that you should look for selected variants in a selected set of genes irrespective of the indication of the test because those findings are in that first category of what Bruce Korf just listed. Things which it's reasonable for a practitioner to change the management of a patient or family if they find such a variant. But you would only be doing this testing in somebody that you couldn't diagnose otherwise. And my understanding is that the feeling, I think the college's official policy is that it's not appropriate to order a whole genome screening or exome sequencing on healthy people just as we'll see what you can see. As a clinical test. I just think as we have the discussion about incidental findings to be clear, are we talking for clinical testing and not for research? Because I just think the two, I know, but I just think we need to be absolutely clear on that. Yes. Thank you. Was there somebody over here that so wanted to stir the pot a little bit? I mean, I think you were raising the possibility that maybe the time has come that we should be thinking about whole genome sequencing in a different way. Okay. So we'll blame Jim since he's not here. But because if you think about it, there may be, I think you used the number half a percent of individuals may have an undiagnosed Mendelian disorder. If I can remember, my number is actionable. We heard from Dan that something like 2%, but there were also, for people that were being treated with warfarin as an anticoagulant, there is some percentage, I think you used the number 2% there as well, where people would benefit from knowing what their genotype was. And I think we can think about, probably if we all went around the room, another 5 or 10 or 15 examples of where that knowledge would be useful. And so now we're compounding. We're compounding a half percent and we're compounding another a few percent. And we're also generating evidence by looking at two outcomes. You know, is that something we should be thinking about or is that something we should be shying away from? Thinking about, sure. I guess I take the perspective though that the cost effectiveness of doing what Jim was suggesting and looking for screening everybody for the very small proportion that will turn out to have actionable things is a very different question from the one that Les will talk about tomorrow, which is when you've done it for some other reason and stumbled across something. I mean, I think we're a long way, I would guess, from having evidence that this is a cost effective way to practice medicine to screen everybody for an exceedingly rare thing. So one way to think about this is again levels of how we might want to approach this. I think you could make a very compelling case at this point to say of all the people that are having genomes sequenced, whether it's in a clinical setting or a research setting, unless it has done some of this preliminary work with some genomes that he had available, what is the magnitude of the actionability problem? I mean, define your list, whether it's the ACMG list, whether it's the University of Washington list, whether it's just pharmacogenomic variants, whatever. You know, every one of those genomes that's being done, it would be a useful research question to say, what is the magnitude of this problem that we're really talking about? And use real data to quantify it. And then we don't have to argue about the potential. We could really say, look, here's what we found in a thousand genomes. And as I said, Les will probably present some of the work that he's done tomorrow just to try and get a sense of that. But that would be something where if everybody pooled in, we'd learn much faster. And we don't have to necessarily decide what to do with that because that's going to be the purview of the researchers and clinicians that are actually dealing with it. But we would at least have the aggregated data to be able to understand what we're dealing with. So, actually, I was just going to say, I think we're actually much closer to the clinical utility threshold of whole genome sequencing. Now, the data are different for a variety of reasons. But in a setting like the VA where we basically are going to capture somebody for life, it doesn't take very many pharmacogenomic screens to equal the cost of a whole genome sequence. And this isn't really looking at, you know, if you're looking at the Athena panel, one of the more expensive panels of genetic tests, a whole genome sequence is already well less. Now, for the diseases caused by repeat expansions, you can deal with coverage, there's differences in the data. But I'm going to assume that over the next few years we will see the point where it actually crosses. And especially when you're looking at the trajectory of caring for somebody for life, if they have already a little renal failure, a couple of chronic conditions, sequence them, you know, and then say, and then the final thing is we're all talking about the Mendelian actionable items. There's also, you know, you're going to find age or 20 recessive carrier conditions that might affect the way that patient and that patient's family, especially if they're young. And it's not actionable if you have a CFTR mutation, it's a heterozygote, but it might be for that person down the road or for that person's family. I think we're a long way from implementing whole genome sequence in the general population, but the time is right for asking the question on when do we go about doing this and what kind of research needs to happen before we do that. So having Jim Evans' paper, Jim sort of, you know, pushed me into action here because he was thinking along the genome not from the traditional pharmacogenomic traits or the carrier testing or prenatal diagnosis or all the SNPs that we could use, you know, for stratified screening, but purely as a public health effort. And he said we screen newborns. That's rare. I mean, if you want to make a case today for a newborn screening program de novo, you probably cannot make it because we find 10,000 babies in four million births every year. Okay. If you apply those same principles to a whole bunch of conditions, I mean, we have two million people with one of three conditions, BRCA, Lynch and familial hypercholesterolemia. I'm not saying that, I mean, we need to screen the whole population. I think to think about what you can do at the population level driven by the rare would open up if you study it well, if it's cost-effective, if it's, you know, the testing is right, the counseling, you know, all of these issues, the LC issues can be worked out. Then having made the first cut that, hey, we can screen the population, we can find people who need services and it's cost-effective. Now, let's talk about what we do with pharmacogenomic traits because there the associations are weaker, they're not, you know, high lifetime penetrance. I mean, you're dealing with odds ratios from one and a half to three to four and those will, I mean, if you don't have the test, you would probably wouldn't make the case that you should have the test, but if you have the information like what CPIC is doing now with the pharmacogenomic network, you have that information. Why not develop a guideline around dosing and then you can test it out in practice whether or not it works, whether or not the benefits outweigh the harm. So your entry into the genome has to be from two vantage points. The rare, testing people with rare Mendelian mysterious diseases that could have generic component, but that will affect a very small segment of the population or to do a population screen on everyone to find the people who are rare. I can guarantee you, there are many more than what's in the newborn screening panel. Now, can we make the public health case? It probably requires a decade worth of research, if not more, while the technology will improve and the prices will plummet and then we figure out how to deal with the ELSI stuff along the way. So it's a research agenda that will probably expand the next decade. I was just going to respond to Mark's question about what's the magnitude of findings, of incidental findings, and just a brief survey I did with the ICGC, the International Cancer Genome Consortium, about 100 research investigators responded to my questionnaire about incidental findings in return of results, and it's there. About 50% of the respondents said that they did find using a variety of methods. Obviously not everyone was doing this, genome sequencing, that were clinically relevant that they felt was clinical enough to return to the patient and or their physician. So 50% of all of them said they have already found something like that, and about 20% of them, and this is internationally, said that, yes, something has already been returned. So I don't think it's a matter of are we finding them. I mean, we're finding them, and we're finding them more and more as we do sequencing. So it's there. It's just a matter of what do you do with it, and those policies are so variable across the international community and certainly within the United States. And if anything, the perspective of the researchers in the United States was at the time about a year ago, it's really not my responsibility to worry about it. I think that has changed in the international community, there was much more of a sense of it is my responsibility to worry about what I find, regardless of what my original intent was. So I'm not sure we really need to look more at what's the magnitude people are finding incidental in clinically relevant clinical tests that are available in a clear approved lab findings now. Yeah, and I do want to make sure we're staying focused. The discussion here was not about the sort of the broader question of, and I think I misled us, I apologize. Not the broader discussion about should we be doing whole genome sequences, but how much of a barrier is the whole privacy data sharing incidental findings piece to that? Because incidental findings are going to be there, I think. Yeah, and that was part of a question that I asked in that survey and it raised the I'm concerned, but that's not going to limit me from trying to do this. These were all researchers in large academic settings throughout the U.S. So how that will get translated into implementing genomic medicine across the board in terms of our clinicians going to be concerned about, I don't know what to do with this other information that I'm going to get from it and how do I interact with my genetics department or my medical geneticists and what am I going to use to help counsel me, and is this a whole other consultation service that I need I think is still a question, but I don't think the researchers at least that answered the survey, we're not going to do next gen sequencing because of their concern of oh, now I have a whole other headache to deal with. So I'm influenced by the patients too. What do the patients think? And so in the clinical sequencing projects that we have including Jim Evans, we now have some beginning to generate some data on this and so these are mostly whole exome sequencing projects and patients are given choices they can opt out for the return of medically actionable variants and they can opt out of non-medically actionable variants and only a small percentage of patients opt out, they want to know about medically actionable variants only a few opt out from return of medically actionable. But similarly that same small proportion, there's a small proportion that opts out that most people don't care about the non-actionable variants. I mean you have similar results I think. If there's an action people want to know that. So I think we should all be happy that Naomi Aronson isn't here anymore because if I were her I would be terrified at this discussion. So the proposition is that people are going to have an exome or whole genome sequencing done and then they're either a big brother will tell them you have nothing to worry about or they will ask for their sequence data and there will be a proportion of people who will say oh look I have a mutation in Desmaplacan II, a disease gene for arrhythmic genic right ventricular dysplasia and I want an MRI. I don't care what you guys tell me. The only way to know that I don't have that disease is to get an MRI and you do that a couple of dozen times and suddenly the cost calculus changes around completely and I don't see it, this is like a genie you're going to try to keep in the bottle. So I don't see, I understand that most of the variants that are on the ACMG list are variants of certain significance but there's a ton of variants of uncertain significance and I don't know how we're going to handle those because I think they'll be entitled consumers who will say I don't care what you say, this is what I want to have done. Now that might be 5% of the population but it's still going to be a problem. So it comes back to education but that's a long term solution. If we should move to that CMS supports Naomi paying for that before they become Medicare beneficiaries. I'm going to lose you. You have two responses. Number one is I think that is the primary justification that led the ACMG working group to very severely constrain the list. It's exactly the worry and that over interpretation and chasing a lot of ghosts here would be unwise on a clinical basis based on what we know today and that's exactly right and that was an effort which is being criticized on all sides which probably means it's about right to try and constrain that problem while still paying attention to some serious things that you can do something about. Striking that balance is a very, very hard thing to do and where the tipping point is we will probably argue about forever but I think there is one and it's the right thing to do to do that. But the other thing I would add is Rex you framed incidental findings as a problem and I actually don't think they're a problem. I think they're a research opportunity and I think it gets back to what Mark said earlier which is that the marginal cost of the data here is potentially zero and so you can if you were interested in understanding this issue and predictive medicine in a big way you know there are lots of people and in spite of what anybody says about we can't start doing this until we prove it's useful. Sorry folks but thousands of exomes are going to be sequenced done this year for clinical purposes and that's just the way it is and we can sit in our ivory towers forever and scold people for doing that but it's being done and we'd be knuckleheads to throw the rest of those data away and that is a way we can learn to do predictive medicine and our data are essentially free, the raw data so why don't we grab them and do sort of a it's almost analogous to post-marketing surveillance is to do the post-marketing hopefully and grab these and study them because they're pretty unbiased ascertainment for these traits and so this is exactly what we need. I was going to say in the mainstream it's like the genomic version of whole body imaging there's nothing wrong with you but we'll find something so I mean I think maybe people don't view it with the same skepticism but perhaps they should you know I think that this point is very important and it's one that I think about a lot and you know in our whole genome sequencing research project our intent is to basically stay in contact with our patients and families on a minimum of an annual basis essentially in perpetuity so that we can do it disciplines us to do re-annuation but ultimately this is the research agenda that we're interested in doing which is how do people use the information, what information do they want, what do they not want, how many of them are going out and doing a bunch of weird stuff that we wouldn't recommend so as big as the magnitude of that problem is it going to turn out to be like the whole body CT scanning which obviously the business model for that was not sustainable because anything that targets essentially a very small, I think Dan used the word entitled group of the population just as the director consumer genome sequencing or genome, whole genome snip companies have found out, there's not enough people that are really interested in finding out stuff at that level but I think really defining the research agenda so that we can actually answer these questions as opposed to what we usually do which is get a bunch of smart people in the room and argue about what we all believe it really becomes a religious debate as opposed to a discussion about how we could actually design research that could answer some of the questions and help to inform us how best to use this technology going forward I think if you phase it in and you learn how to manage incidental findings for exomes first and then eventually we tackle whole genome which would include mitochondria, promoter areas and those are research topics to really learn how to do that well but it doesn't have to be done now but eventually other people want to weigh in here so I think it does sound like a research opportunity but maybe it's not quite yet a genomic medicine implementation opportunity it's an evidence gathering opportunity I would say that there are implementation opportunities if you turn it on its head and you know assume again you have all the information could you define clinical contexts for a patient whereby you would go back and query the data could you build let's assume that we had actually gotten funded for the newborn sequencing project what we were planning to do was to create clinical scenarios so the child that fails their M chat for autism or the child that shows global developmental delay or a child that presents with later onset hearing loss you can define a number of clinical scenarios that could occur within the first five years of life where you could go back and say we need to assess these specific genes within the genome to look for the approximate causes of these things because we know they're likely to be genetic and that I think is you know if you have that information I think it is a huge wasted opportunity not to kind of think about how you could use this type of information over a lifetime and we plan to beat the hell out of the genomes that we're collecting you know to answer these types of questions because ultimately in my view the only economic argument that makes sense in terms of getting a whole genome is to be able to use it over the course of the patient's entire lifetime and there's a whole it's not only the clinical scenarios but it's how do you store it how do you access it how do you develop the informatics and how do you do it at a sustainable clinical scale I mean those are the types of things that I find the most interesting in terms of how to answer and ultimately if we can figure out how to do that in a sustainable way we'll provide the most value for whatever you get your genome for whatever reason how you can then use it going forward. As we're all eager to implement the whole genome can I just caution us a little bit to approach the tool whole genome sequencing as a tool it has to be evaluated it's a T2 research agenda not in the T3 research agenda in other words at the end of it we may decide it's not worth using it the equipoise is not using the genome now you're all here we're all geneticists and we're all excited about using the genome but in many clinical or population scenarios if you are truly agnostic about the added value of the genome forget the cost but benefits versus harms we should be sort of prepared to the eventuality of possibly coming up with a negative answer are we ready for that I'm just I'm so confused by this you mean I'm confused? No I'm confused you're confused by the skepticism of both the public health and the healthcare community that why should we do a whole genome sequencing on people I'm not confused by that I hear it every day I see no I think it's delusional to think that it won't be useful in the future that's nutty I mean we already know that it is useful for diagnosing rare diseases is that nutty or religion I think it's already useful the point was made that if your phenotype is caused by mutations in more than three genes it's cheaper to do an exome than to do three single gene tests so who is it that wants us to keep ordering single gene tests for these people in the long run I'm not talking about those people I'm talking about people coming in for different healthcare encounters than 95% of people who don't fall in your bucket yeah but I mean I think we're talking about two different things I mean from my perspective I agree well I think what you're saying I mean which is we shouldn't just be going out and randomly acquiring you know genomes I think if you have a clinical indication that warrants doing a genome and I think the based on the preliminary data that's come out of the application of this technology to children with complex undiagnosed diseases where we're including our yield you know improving our yield to you know causal diagnoses you know adding 25-30% then my point is if you do that then you know you've got the genome don't just you know put it away let it gather dust because over the course of that patient's lifetime I would argue given our particularly in this country our propensity to use drugs that there'll be 100% chance that we'll want a pharmacogenomic information to guide drug management at some point in the future so why would we want to exclude the use of that at some point in the future you know now I think the answer that we are seeking is how often would we go back to it how many uses would we have and under what circumstances would you go back and use it and there's a whole bunch of interesting questions related to the sustainability of those types of approaches but I think that you know if you've collected a genome it makes little sense to me to just use it for a single purpose and then just let it molder. So I know this is a very heated discussion but everything's focused on incidental findings and maybe we need to separate that bullet into privacy and data sharing versus and then a separate bullet incidental findings because I think the privacy and data sharing issues are also huge and need to be addressed. We need I reckon genome sequences to like HIV results back in the 90s when they were hidden and you couldn't find them and you'd admit somebody the ED and not know that they're HIV positive because the test results were so hidden and private and I think if Mark wants to be going back and looking at this over time we need to balance the privacy issues with the clinical use issues around genomic information that's done clinically and I think a number of people brought up consent issues that there are consents that are real short and ones that are real long and that's just the form that we aren't even talking about the process. So I think having some discussion around these issues or ways to look at these issues also is very important. That was an area that I highlighted on my list of things that I heard from the various presentations today that the privacy, the data protection that we're operating under a lot of different principles a lot of different rules, a lot of different regulations. So I think even just sorting out what are the different rules that we're all operating under in the different federal entities and is there a way short of some changes in legislation but through rulemaking policy internal ability that we can do some reconciliation around that. I think that would be something that could be a relatively early opportunity. So I think in light of the hour maybe we can just return to where we started the day with what some possible outcomes might be. Especially given the discussion over the last hour and a half now it does seem like there are certainly some common themes that resonate with most of the people sitting around the table. The themes of standards IT infrastructure and curated databases seem to be one. The idea of evidence generation seems to be another and we've actually the whole discussion about genome sequencing also is sort of a part of the evidence generation piece. And then privacy data sharing and I'll take Deborah's suggestion of separating several findings maybe as a separate topic. So then given that sort of agreement around the table where does that leave us? Those commonalities in interest for example create a foundation for just to throw it out there working groups or task forces that would engage the groups around the table here to continue to discuss those three areas. So maybe people can have an opinion about that and weigh in. For the last couple of discussions the folks from the armed forces have been fairly quiet. Maybe because it's sort of of less relevant I don't know but are those topics where continued engagement makes sense. The whole process is very interesting to me. I've heard a lot of very intelligent people make very valid assessments and suggestions in various aspects. I think the metaphor of touching the elephant in a couple of places is appropriate and one would expect that because of the backgrounds that we've all brought to this table. And that contrasts with you might say some of the procedural ways that we work because of the idea that there's an immediacy sometimes to what we have to react to. So while the 100% solution is the best often at least as an ideal situation we often move out with the 80%. Not to get back to Romsfeld you go to the war with the army you have that's not my point at all. Remember that. But it's an interesting evolution of where you go with this and so what I ask is who has the authority to implement the change in order to know where to direct the energies of such a very elite group of experts and that in terms of an immediacy what are your challenges which are financial, they're cultural both in terms of the community that you're directing this at as well as those who are generating this level of appreciation of the entire problem. And with those and then what's the timeliness factor. So within ourselves I sense a degree of urgency but it's countered by some sense of how much do you need by when and what's the value of it in relation to other diagnostics that are currently the standard of care. So with those as I see some of the operating drivers and a working group are very appealing to me as something that barring an emergency it takes and distills what's been discussed here into some and keeping it simple and stupid so to speak where it is something that unifies the community and bring it back. Of course in a way I'm delaying what may be a desire here for a decision but I sense enough expertise and subtle sometimes differences that the authority that needs to go forward with something that can be addressed to whomever the higher decision group is first needs to agree on these simple principles that can be then sold to that group. You know in ours it's our surgeon general and I'm not sure in the other fields who may be directors and so on. But in that scalable way that it reaches up to the highest levels of decision making authority you make the first accomplished easy wins relatively speaking and then come back and revisit with the more complicated or technologically immature aspects of this particular endeavor. I think a working group would be a perfectly reasonable outcome we wouldn't strive for much higher. Others want to VA do you would any of these topics resonate with you enough that you imagine there would be interested in participating in working groups and you know I'm sorry to put you on the spot I don't mean to do that but just in a theoretical way. Well no I came here to be put on the spot in a theoretical way that the VA research already has priorities that include genomics. Broadly speaking absolutely yes I'm on the clinical side and we're you know I'm desperate for data for standards so that we don't lose data I think all of that is very good I guess my take on all of this is that you know so our military colleagues have their regulations and limitations they're interested in managing the people who are actively enrolled often deployed and their dependence in the VA we can't touch dependence and that and family members and that often limits the practice of genetic and genomic medicine the you know our academic colleagues have different kinds of things family units are covered but then they leave the system and at the FDA they don't regulate the tests at CMS they don't pay for them the contractors do and so we all have these little silos and it's really hard to find the big things now I agree we're all interested in these big pictures and the question is rather than touching different parts of the elephant can we all pick up our piece and maybe get it to move but I'm willing So one of the things that we had talked about and this may be way too weird an idea to put forth this light in the day but we at least amongst the planning group thought about the idea of maybe creating a few use cases from our genomic implementation and we've heard some of them today the Lynch syndrome some of the pharmacogenomic things and we could you know basically create a narrative that say you know imagine this clinical scenario here is how you know patient with sensical rectal cancer we do tumor-based screening blah blah blah if we were to create a few of those use cases and send them around to the different constituencies represented here and you were to say here are the pieces that we touch here the pieces that we don't touch here are the issues that we would see that would attend to our being able to deliver this use cases presented that in some ways would allow us to collect information around some of the things that we do think are ready for prime time and could be useful in terms of identifying those areas where there may be either differences of opinion or gaps that could be used to organize what the work group might choose to focus on so I just put that out there as a possibility and whether that would be something that people would be of interest in terms of responding to it would require some work on your parts. Yeah I do I mean Larry just articulated what the problem is everybody's got different purposes in existing and you want to bring them all together and have them aligned but they can't because as the Admiral just said everybody's got a different boss and the bosses aren't agreeing and they down at their level can't make decisions so I mean we kind of know what the problems are and so we're going to create use cases to do what exactly. So the assumption that I heard in your comment was that we can't agree on anything and my argument would be is that there are some things we can in fact agree are important to solve. I would argue that we all want standards and we're all going to have to deal with standards you know that's a no brainer from my perspective so why can't we talk about standards and yeah there may be some implementation aspects that come up at some point but who's not for that and even if we can get you know I'm a big believer in being good enough and not being paralyzed by the perfect. If we can get 85% of the way if we can get everybody on board but the Air Force I'll just pick on my friend here. If we can get everybody on board but one group then haven't we made some progress and doesn't that in fact then resonate up the chain of command and you know for whatever organization we're in I guess that's the point is that we tend to say it's hard and it is hard and yes we've all met and we throw up our hands and we say it's hard but we never get to the point of just sort of taking it from the bottom and say what can we agree on what could we be useful and I would still argue that if all of the groups around the table went to Ankh and said we need standards for this and this is why that we would be much more likely to make progress than my trying to go through somebody on the advisory committee to get it to happen. So I agree with Mark and I'll say two reasons why one I think you know I underwent a change of leadership in the Air Force in terms of the certain general who had a grand vision for implementing genomic medicine and then the subsequent surgeon general who has a much more operational focus so I had to really change kind of the emphasis of our program and I think that by taking it from the abstract of how do we implement genomic medicine to practical use cases where we can get those the low hanging fruit and the easy wins I think that's a much easier way to show where the gaps are and how we can work together and the second reason I think that's a good idea is the thing I'm asked most often in leadership meetings is okay well who's leading who's following and who's watching even if we don't all agree that it's time to move out people who don't think it's time can respectfully watch or follow when they think the time is right and the people who think that it's time to go ahead and implement use cases can then lead the way and we'll learn by doing. So could I just defend myself Mark which is that I think that if you take the health care delivery groups in the room of the agencies I think you might be able to move a lot of things forward but I think if you put the payers and the FDA and CDC potentially not one but you know and but if you start mixing it up then I think that you are going to run into major problems but I think if you do stick to the groups that are trying to deliver the care for different populations then that could be highly successful and that I agree with. I just wanted to bring up one point that I've been noticing in this meeting as well as in several other meetings and that is it seems that we always have a tremendously laudable goal and they're appropriate and we do need to strive for long stretch goals to really get to where we want to go. Not a problem with that but because this is relatively unattainable if we actually go ahead and mesh things together why don't we really look at things where we can actually truly work together like one point that was brought up on one of the lists up there there's really there's quality and there's a standard but there's no quality of data and I'm hearing in this conversation GWAS, SNP data, whole genome exome which one are we talking about for clinical use for research use what are we doing with the phenotypes eMERGE is doing a great job with what they're doing but has it been tested? Who tested it? Are we going to test it more? Are we going to build that? And you've got all these possibilities that we truly together could harmonize and get what is a core baseline and from that at least we got a foundation to go to those stretch goals that we're trying to get to but at least we're working on this level plain of this is the genomic data that we have this is what it's coded as everybody understands what that means because everybody interrelates that data because even currently I mean you go back ten years take a look at how the annotations for some of the sequences may be it may be very different than what we are saying right now too and they probably will be with Human Genome Version 25 when we go in the future so I think that's just a suggestion maybe I'm trying to bring us down too far but I think it's something that we could all work on and at least agree and then we at least have the feeling that yeah we did achieve something we agreed on this one or two or three things Resonate with Ron a little bit can you show us maybe one of the initial slides like what we're trying to do here no just go back like the purpose no I mean okay different because I'm you know I like lofty goals I've do a few of myself at the end of the day you cannot herd the federal agencies together not from NIH not from NHGRI that's a higher level function and that's why before there was the secretary's advisory committee which was a secret you know in HHS level they could herd FDA they could herd CMS of course they don't deal with the VA it didn't work but the times have changed and I think while you're right I mean I'm not well it takes two years to create a new advisory committee so that may not be the answer but I mean I want to ask my NHGRI colleagues what are you trying to achieve at the end of the day and maybe there is a simple answer to a simple question where would you like to be a year from now or two years from now given all these mass histories of missteps and so on and so forth where are you trying to be given the research focus of an NHGRI that's a big question I might ask Eric to comment as well I think what we're trying to do in this forum is to get multiple federal agencies at least moving in the same direction towards genomic medicine implementation and not arguing about what evidence we need or we don't deal with that or that's not our view or this that other thing what are the things that we can agree upon and work on together and it seems to me we've all heard we need evidence we need evidence we need evidence evidence so everybody agrees on that and we have in the room people who are responsible for the healthcare of untold millions of people I mean you know I mean there are large numbers of folks that are represented by the groups here what better place to generate that evidence and in doing so we can address things like standards and IT infrastructure not all of the issues but a couple and privacy and data sharing obviously those issues are going to come up incidental findings are another issue that's going to come up and so those are all things that we can touch on we can't solve them all but at least we can get started on them and it seems to me that if we were to come up with an approach for generating evidence within those healthcare systems and it's great for me to say that because although I am marginally part of one of them I'm not heavily involved in any of them but if that were something that we could agree upon to do I think it would be a major step forward and a year from now I'd be thrilled. Why complicate? No I'm actually going to agree with everything you said and maybe I would come at it with a slightly different way to add and augment what Terry said. Let me be very clear NHRI is not doing this because of some patriotic cause or because we feel we have the duty of the federal government to lead this effort in fact we didn't lead with the federal agencies we led with our colleagues it's a bit of a piper thing we got together people who were actually doing genomic medicine starting a couple years ago at genomic medicine meeting one and two and so forth and along the way we keep hearing you should talk to these people you should talk to these people. In the fall we're going to talk to international groups it would be ironic if we actually were successful at engaging international groups doing genomic medicine and failed to do it at the federal government we were part of the federal government so we will try but if indeed you're correct that we'll get nowhere because we're just some little DITC Institute in the NIH where it should be done at the department level or it should be done at some higher level then that's fine then we'll fail and that's okay but at least we will have tried by convening people to see if you're interested in coming along we have no authority to make you come along we're just convening here we're not trying to do anything more than that and we're not doing it uniquely to other federal agencies we're doing this across the board to other researchers other organizations and other countries. So Eric I don't want you to fail because we I mean there is no need to fail at this point you're probably right in terms of moving the international community easier than moving the U.S. community but given the research focus of NHGRI and the desire to move into evaluation and implementation research I mean that's a laudable goal you can drag the rest of the NIH institutes along with you and you probably don't need much of the FDA or CMS or clear I mean if you focus on the research agenda that says okay this is what we the research community think we need we've heard all the stakeholders we've heard the problems of coverage and reimbursement and oversight and all of these things you know we're going to construct a research agenda for the next decade that essentially at the end of it will make a very strong case for implementation of genomic medicine and that's I think what you're trying to do you're trying to move into that T2, T3 and T4 space which up to this point you've done only the T0 and maybe a little bit of T1 and that I think we can all subscribe to that there is nothing wrong with that it's just trying to herd all the federal agency along with you to say okay at the end of that research oh would you cover us with the CMS I mean forget them I'm sorry if I say that but I mean they're all my friends and colleagues but you know focus on the research agenda well before we offend anyone it is late in the day I mean I think what we're saying is we're as Eric said we're convening we're the Pied Piper we'd love to have people participate with us we think there are things we can do even if others don't participate and so you know it's sort of an open invitation well and I think it's interesting so having you know like several people in the room sat through all of the genomic medicine meetings to date I think we would have been remiss if we didn't have CMS and all of the stakeholders that are in the room today together to have this conversation that doesn't mean that everybody needs to sign up for the long haul but I mean you know I think there are probably lots of opportunities for collaborations across these groups that make the end result of the efforts that we put into it stronger than if we didn't have those folks sitting around the table and so I think that's an important message it's not to make anybody do anything nobody in this room has the ability to do that but it's about creating an opportunity and so we want to make sure that everybody that feels that they have something to bring to the table has an opportunity to do that so I had some remarks at the beginning and I've enjoyed myself all day long just a few observations and that is that all of the organizations that are represented here today are our stakeholders and they have stakeholders who support them and from whom they are influenced but I think all of us and what we have to do in our missions are much more similar or alike than we are unfortunately the human nature is to focus on the reasons why not as opposed to the reasons why and we can very easily fall into those gaps so of course number one might be that we try to put forward our own interest and initiatives and not try to second guess what the others who are here might be hearing or might be agreeing with or might not be agreeing with but I think it is an absolute necessity for all of us from our different organizations and stakeholders to come together, support to come together however I see what we're doing as if I harken back to my high school and college days we're involved in a complex equation, we're trying to solve a complex equation and as the admiral will tell you from time to time he looks at me out the corner of his eye sometimes when I'm giving him a briefing I ask him to take a margin note and of course what that means is I'm focusing on what he's asked me to provide for him but there is something on the edge that we will need to come back to at a later time and I think if we look at it that way as opposed to putting those margin notes directly in the outline we can go move forward and have an opportunity to make progress keeping in mind that to solve the equation we're going to have to deal with the margin notes as well at some point to emphasize what the admiral had to say about the 80 percent and this is a gross example but from our perspective if we are dealing with operational commanders who are looking for the ability to recruit a certain quantity of individuals to go into the field to fight a mission the last thing those people want to hear up front unless we have a solution for them is why they can't just recruit these people take them out in the field and determine how many of them aren't going to function on the basis of how they function as opposed to presupposing that there's a risk that they're not going to function and therefore we screw up their ability to even reach their recruitment goals now that's not to say that that's gross because we should have the capability of providing for them the specific type of individual that they want but unless we include that issue as a margin note as we move forward we will screw them up because we'll come up with all sorts of reasons why as they recruit a thousand people we get 80 out of the group that come forward because we have all of these contingencies and risks and whatever so the complex equation is we move forward we try to define what we can we keep our margin notes and before we solve the equation and turn it into the proctor we are certain that we take care of those margin notes as well in the complex equation for every action we were taught there was an equal and opposite reaction but I'll present to you that there also are tangential reactions that you have to pay attention to and those are the ones that I think we're worried about we can come back and deal with those thank you so we are at the so just to comment about a potential outcome I think if we had a paper in which all the perspectives were captured as to why the eGap recommendations on Lynch syndrome have not been achieved in their organization or from their perspective it would be really powerful you know that's almost five years old everybody agrees that they're good recommendations they're evidence based and yet no organization is really uniformly applying them and that in a sense lays out the research agenda and the needs that we have as a research community when we lay out why we're not doing that so just an idea it's a good example of a use case that we might pursue so we're at the end of the defined time that we had I don't know Eric do you want to add anything Terry do you want to add anything I should mention breakout rooms for the groups that are applying to me so we do have some working groups that have been held over from some of the previous meetings the cancer group planning to meet they have the Potomac room which is on this floor I'm not sure exactly where but it's nearby and the periodontal group has the Severin room we had suggested meeting at 545 but it's up to you as to when you want to so feel free and I'm sure they would welcome others to sit in and listen to what they're doing so I think it's been a really terrific discussion today we've had really wide ranging discussion the Genomic Medicine Working Group I think will have to go away and do a little bit of thinking thinking about how best to capitalize on the energy and the participation of everyone in the group and we'll try to get back to you with some ideas either if not tomorrow then soon but maybe even tomorrow for some discussion in the recap any final comments anybody wants to make otherwise we'll adjourn for today and we reconvene at 8am tomorrow