 recalls of how we do next gen sequencing. But I bet that there's a lot of variability from laboratory to laboratory about what those standards are. And part of that may be based on application. Inherited disease has different standards than cancer testing. But I think it's a major issue, and especially for payers. How do they know the quality that they're getting? And I don't know if Naomi wants to comment, but we're one of the other payers in the room. But it's difficult for payers. Yeah, I would certainly in all items like this separate regulatory from payer communities and not blend those two. For the FDA over the last five years, they've been welcome participants' attendees in our ISCA, now ICCG consortium. And after every meeting, they comment that they're really pleased to see the community working together on a voluntary basis to create standards and to share data. And it makes them feel like they don't need to do anything to regulate the evolving genetic testing landscape in the laboratory developed test mode. So you don't want to convene the FDA. You don't want to ask them for official opinions. You just want them to attend and hear that the community is making a best effort to share information, work together, improve quality, improve standards. And they're very happy to come and listen and participate and make comments, et cetera. But beyond that, they simply don't have the manpower and ability to do very much. And so they're watching the technology as it rapidly emerges and so they're very good participants and discussants. But I would avoid language like convene, which makes it sound official. And I'd separate regulatory from payer issues. Engage. I do think that the FDA are relevant in that they're not only seeking to regulate LDTs, but also sequencing platforms and clinical decision support systems. I'm doing a job for you. So it's not. So I mean, LDTs is certainly probably the primary issue of the moment that generates the most discussion. But some of the things that was told yesterday, like genome in a bottle looking to create reference materials and standards for some platforms and so on. All right. Well, it sounds like we need to go back and rework this particular idea and we'll do that. The other thing we heard yesterday, I think, was that, and here we don't have good ideas about what that should look like, but that we really need to be thinking hard about developing a standards IT project that would make sure that we've got people fully engaged, whether it's the folks generating HL7 messaging approaches or the people thinking about, how do you report back genomic values in electronic health records? And I think that also probably extends to some work where places like Emerge and Page have done some work in the area of phenotype standards. But we probably need to think hard about standards discussions. And the one group that was not able to join us at this meeting was the Office of the National Coordinator. And it seems like engaging with them is an important opportunity here, especially maybe if we had a consortia of groups sitting around the table who felt that this was an important issue for them to tackle and address. So thoughts or comments about that? So having just walked in the room, sorry my bus was late this morning. On the IT standards, formatting that comes off the instrumentation has been maybe Brad, you know something about this? I don't know. It's been VCF files have been the standard format that's been coming off. But I'm not sure everybody's using them or that there is a standardized variant calling file format. And I would think development of things like that that are common across platforms would be really helpful. I guess you'd have to go to industry for that honestly, because they're the ones who are writing the stuff. But it seems like that'll be really important. Yeah, I think we've kind of settled on a basic format of VCF just the way it's a nice file format for computationally analyzing and tracking the variant data. But the actual work that goes under that, the actual mutation callers is still a bit of a wild west with a lot of variability that's still to be settled. Driving towards some common tools, but certainly not there yet. I mean the other thing that underlies the VCF file is it's based off some type of a reference. You call the variance against a reference and we really don't have a good reference at the present time. And so the idea that we're somehow calling something a non-variant when it could in fact be a very important variant is another piece that we have to understand. And so using a different file format like a BAM or something that has everything in it where you could go back and reanitate as we learn more, those are the sorts of discussions I think we need to have. And I would notice the VCF file is not what comes off the machine. Right, right. So for those that haven't attended all of the genomic medicine groups, one of the working groups, the task force groups or working groups, I guess we're calling them, that came out of GM2 or I think was sequencing working group who has thought a lot about this issue of actually how do you compare different laboratories to Deborah's points about standardization across different laboratories or generating sequence and then also this very important issue of even 10 years after the human genome is still not perfect. And so there's areas where there might be improvement in the underlying reference sequence or maybe more likely we need a variety of reference sequences to reflect the ancestry of the individuals that are being sequenced. Yeah, relative to the database, is a lot of the database is also the reference sequence is actually reversed and the variant is the real, the most common is actually the variant. This is pretty common, is there some mechanism? One thing that might help us is to have a mechanism to actually fix all these things. Is there a, so like we're sequencing a lot of people and we see clearly that there's reversal of alleles, is there some mechanism to start fixings? Deanna Church is working on this, isn't she, yeah. Yeah, there's a whole, it's a human reference, human genome reference consortium. And yeah, I think there'll be a new reference coming out this year, HG20. The reference materials. But then they're also doing a lot of work. Thinking about the clinical application of the references is the next phase. Yeah, and the reference materials that we're working with NIST and the genome and a bottle consortium, I think it's gonna be something like eight individuals or something that should be sequenced to at least the completion of the reference sequence. And we're working to make those reference materials so they should be highly characterized. So hopefully a lot of that will get worked out with a lot more sequencing being done. And do those eight individuals reflect different race ethnicity? I don't know, I know some of them are from a family so that you can do Mendelian checking, basically. But I think there is a desire to have people of different ethnicity. But underlying the reference materials is just that sequence is sequence. So it doesn't really matter what ethnicity you are. They're not intended to be references to say that these people are normal representatives of any particular population. I recall there are, I think, three or four trios and they're all from a Utah family. I think it is, I can recall this. My point is just that there's a lot of sequencing going on so we can increase the depth and accuracy of that quite a bit if we had a mechanism for collecting changes. So we do and I encourage you to look online at the genome and a bottle consortium where that's actually being looked at. So from my perspective, I mean, I think that, clearly there's a lot of work that's being done on what might be characterized as the basic science sequencing side. What I'm not as convinced about is that those folks always know the end point that we're ultimately wanting to get to in terms of clinical use and are keeping that in mind as these things get done. So I guess that's what I'm trying to, where I think we could potentially do this relating to standards is just to say if we ultimately want to use this in the clinic, then how do we represent all of this in a way that it would be consistent across any clinic that you're in that is using genomic sequencing that we're calling it the same thing. That's where we really need, I think, a little bit more energy to go into. Well, I think no one had more energy in this group than Howard Jacob who was really quite engaged in trying to address these things. And I think he kind of got the message, well, this is being done. You don't need to worry your little head about it. And I think that's unfortunate. Yeah, so we need to re-engage him. And unfortunately, he wasn't able to come to this meeting, but I think he wanted to work with our sequencing centers and other sequencing groups and sort of told, oh, well, you know, we're still coming to some agreement or whatever. So we need to figure out a way to re-engage. Yeah, we talked about this a lot and I think it was GM2 and I was involved with that sequencing group as well. And I mean, the point is an excellent point that the data sort of coming off the machine then goes through a process where you call variants and there are about five or 10 callers out there and different places use different ones. And, you know, again, part of a consortium like 1,000 Genomes, we see it all coming into one central clearinghouse and then have to merge it all into one dataset. And there are definite differences, especially as you get to indel calling and structural variation. Structural variation is a very unsolved problem. So I think it's still something that needs a lot of attention to come up with sort of a standard, you know, what are we going to use to call variants and interpret them? It's going to be directly relevant to the CRVR because if that's going to be the clinical interface for the annotated database, then we have to have that solved if that's going to be viable. I'll be a little bit of a nihilist. It's one thing to figure out what the genome is to figure out, you know, how they, what the indels and CNVs are. That's one aspect. But if you're surveying ground, what you do is you just put a stake in the ground and then you measure from that. And it doesn't really matter whether we have a reference genome that has a lot of unusual mutations and everything. And everything, all the normals are all called as a variant off of that. It's a point to measure. And to be able to move forward in clinical genomics and to just be able to use it and develop the software, just making a call and establishing a reference has a huge benefit. And then we can go from there. And the people working on the back end, what is normal and what is the normal range of normal, can use that same reference. It's fine. But it's basically the idea of establishing zero. It doesn't matter whether you use Kelvin or Celsius. You just, you can measure a temperature. But sort of a different point is that, you know, if we're gonna portray this as an exact science to the public and then we miss a copy number variant that's relevant to cancer and we don't diagnose it correctly. So there's sort of that space that's well beyond into the future now. All you're doing is establishing that you're going to refer to whatever zero, you know, whatever the copy number is, and you're gonna measure off of that point. And then you can actually, everybody can agree on the terminology. It's just a terminology question. It doesn't establish what normal is. It establishes what the reference is. I think the point may be that not where the stake is, but whether you're using a meter stick or a yard stick to measure from it. So I think, you know, yeah, it doesn't matter in a sense what the reference is, but the idea that you have to have a caller that can actually call what's there is really important and different callers work very differently on different structures. So I think this is a good opportunity for us to re-engage the sequencing working group. And Howard, who is unfortunately not here, but is a member of the genomic medicine working group. And then the final we wanted to talk about, which is actually where we began yesterday was the discussion on a need for national strategy. So as a way to just, and then I realize this is a very complicated slide, but what this is an attempt to sort of catalog a variety of areas where other national organizations or other organizations, such as the IOM or AMA, have weighed in on such topics such as, you know, service delivery infrastructure for requesting receiving genomic results, the provider and patient friendly model, interpretive test reports, informatics infrastructure, data sharing, information, standardized information phenotype, we've just spent a lot of time talking about that, training and development, consent models, et cetera, et cetera. And if you just look at sort of UK, Canada, Italy, the European Science Foundation, CAP, IOM and AMA, you can see there's really pretty good overlap in terms of some of the topic areas that these sort of either national organizations or nations themselves feel is important. Based on the discussions yesterday using the same set of topic areas, this is an attempt, I suspect it may be imperfect and one of the things we thought we would do was circulate this and invite the people who are alleged to have some alignment here to weigh in on whether that's accurate or not, but to capture some of the discussion we heard from Air Force, VA, CMS, FD, et cetera, in terms of what areas they thought they were engaged in or had a role in helping define. And the hope is that by thinking about this matrix and then maybe how it may align or not with the other national matrices on the previous slide, you actually can get to the point where now I'm just using the UK Human Genomic Strategy Group and what their efforts are, you might be able to get to sort of straw man draft white paper that might be able to be generated by a working group from this group to produce a national strategy that again, this is using the UK one, but we could do a search and replace and hopefully more than that, but a search and replace on NHS for the United States and think about what are the issues that are unique to implementing a strategy in the United States and one that's gonna come up immediately, of course, we spent a lot of time talking about yesterday is the fact that we don't have an NHS. We need to deal with an environment where there's a thousand flowers blooming out there, so to speak, and we need to think about how to manage that. So the thinking, so I just wanted to open for discussion, but maybe start by asking the question, does it make sense for us to think about maybe a white paper that could lay out a draft US national strategy for genomic medicine that everybody could participate in drafting to the level that they feel comfortable participating in. So let me just start for opening up discussion, is that a worthwhile activity? It's something we should pursue, and then we can talk about how we might do that, but so thoughts about is it worthwhile? I see a lot of nods, but thank you, other thoughts? I do think that's worthwhile with the caveat that really be socialized and committed to. We've had another initiative in our health system, which we call the transition from healthcare to health, and we've had a lot of discussions about the national prevention strategy, which our colleagues in HHS have been championing for quite some time, but that seems to have fallen on deaf ears, and I would hate to see a strategy like this go to the same route. I think a national strategy would be wonderful if it actually resulted in a national strategy, because the question becomes not, can you put nice words on paper, but what mechanism is going to exist? And the culture here is not one of coordinated effort in changing that culture. It becomes a much harder process than writing a strategy down. Rex. I would pick up on what Bruce just mentioned. I think one aspect that hasn't come up in the conversation so far is that there's a very active marketplace in this area, and that marketplace is out of the gate, and the question is to what extent is sort of a white paper strategy that's coming from here actually gonna have traction in the marketplace, and I think that's something to think about. That's really a reality that we are facing. There are many companies, many startups, et cetera, active in this area, and to what extent a paper will be relevant, that's a question that I wanna put out. It seems like in some of these earlier meetings, and I've heard this before, that there's a big question as to whether genomic medicine will just vastly expand the cost structure of healthcare, or whether it will actually have a benefit in the long term. And so have we really answered that question sufficiently to propose, I thought that's what a lot of these demonstration projects were about, try to figure out, is it just gonna cost us hundreds of thousands more per person per year, and can we afford that, and who's gonna pay for it, or will it actually bend the cost structure down, and it's a good investment? I think it would be an appropriate way for us to move forward. I would once again echo what I mentioned. I guess later last evening, if we could have reasonable stretch goals so we can attain what we hope to attain, but also get some easy victories, I mean, first point is the idea of, okay, BAM files, variant files, which file are we gonna do? Is there a baseline of analytics that we need to have that we can all concur on, and say that this is what we're going at? Anything above and beyond that is proprietary to the given individual's institute or whatever, but this is what we're all agreeing upon. It has to be aerial 12 point font. But if you wanna have sans serifs or serifs of Buddha head carers, I mean, that's your problem. You do whatever you want, but we'll agree that this is the format we're submitting things under. If we can get ourselves aligned in that, I think we already have a good playground to work on. We can also then get the public entities that are trying to push this down our everybody's throats to have them align with what we expect of us as being servants of the public. We need to do that. We need to be those individuals that help the public. And I think this is a mechanism that we can have slow victories and actually people looking at us and trying to see where we're going and try and follow instead of trying to go for laudable goals, which are laudable, but unfortunately, unattainable more often than not. Well, I'll just comment. I think I take both Bruce and Irwin's points, but on the other hand, if we don't have a starting point, if we don't put a stake in the ground somewhere, then we're gonna see chaos emerge as all the commercial organizations do whatever it is that they're gonna do. I was really struck by the comments some of our colleagues from the Navy met yesterday about how they felt almost abused by some of the companies coming to them and saying, you need to use our platform or you need to use our approach. And a white paper is not gonna solve that problem. I clearly get that, but I think it's probably an important starting point for us. And if we don't have that stake in the ground, then it will be chaos. Can I hazard a comment that at the risk of overly, I mean, you can shout me down overly bureaucratizing this, or maybe it's not what this group wants to do, but OSTP, although that might be the place to go for the Office of Science and Technology Policy in the White House to start looking at a national strategy that would also include infrastructure and incentives. Because the grassroots part has often not worked where you try to just have everybody hold hands and sing kumbaya. So that's a place to consider if you wanted to really try to build the policy in the infrastructure. And actually to that point, certainly at least through PCAST, NHGRI is pretty well connected in with that. So that's actually a good idea, a good suggestion. Might be helpful to get some comments from our policy group as to whether you have any thoughts about how best to engage them, or maybe you don't want to address them yet, so. So. Yeah. I think that we should certainly talk to OSTP and see to what extent they're willing to engage. They are very interested in technology issues and big data issues. And so this may be something that there's a willingness to engage them around the technical issues in particular and some of the policy issues that are associated with those. And that would be a broader way to approach it than just through HHS. So it's worth exploring. So would the appropriate way to do that be engage them with an early draft? I think that we should think about and have some preliminary conversations with them first and figure out what is the best way and at what point is it productive for us to plug in to their infrastructure? We have, oh god, I was gonna say we have some good contacts at the staff level as well. So we could certainly run by this conceptually. In terms of the market place, in terms of white paper, in terms of Italian, are all reasons for why we shouldn't have them because they ought to be. One, to use the only solution, not only to apply it, but also to use as their part based on other technologies in the past and in terms of economic development. So I think the grassroots efforts are great, they're part of the project, but I think as part of the national strategy, you need to do two things. One, is the credit innovation where we are, but two, have clearly quick success stories of how to know what can be saved by it. That's where we can pick up the CDC, the VA, the traditionally non-research agencies because people ask me all the time, I cover it on CDC, like the diabetes division and the altitude limit, and say, Moine, wake us up when we get there. When can we use something that in our programs in altitude and cancer and so on and so forth. This is why we've attempted so hard over the last couple of years to specify the application of the Q1, Q2, Q3. Well, here's some Q1, like, let's say that if we have a national coordination strategy implemented here once, we can already say why we encourage innovation and figure out what to do with the original sequence and get the answers normally, but so on and so forth. I think, you know, the fact that NHS, because they can do it in the finance sector, that we can do this, and we expect some of them to come, it's easier said than done, but, you know, we can't not do a comfortable effort on reading sort of the bullets that we have and the right from the beginning. Whatever they're doing in England, but here, encouraging innovation, it's just that the way we do it, very different about playing the system that the federal agencies, and we have, you know, several public health agencies out in the state of, in the UK, I think they're all more coordinated than NHS is a public health system, and everything is much easier to do. So, I think encouraging innovation by using genomics for quick successes and I'm saying, quick, I mean, if we keep going in the next couple of years, we're going to save lives, and that's how we affect the attention of people that come to the U.C. or the U.S. or the U.S. or the VA or the VA, and so on and so forth, just to take as long a research, we're going to lose power. So, will this national strategy be for the government? Because that's what it seems to be targeted at, and there's industry and academia and many other players in genomic medicine, and so how do you engage those individuals in buying into this national strategy, I think would be something very important, unless you are targeting just, not just, but, you know, federal government agencies. Yeah, I think our hope was to at least get started with federal agencies that to some degree hadn't been really engaged, at least not to our knowledge, in much of a concerted way before. So yes, each agency seems to be touching this elephant and working in a particular area, but not in a concerted way, and it doesn't all have to be in lockstep, certainly, but there needs to be at least some understanding and some coordination across them. I think our expectation would be that we can't do anything within the government that doesn't extend outside of it and involve and get input and advice and participation by other groups as well. Not quite clear how to do that when we don't even really have anything, we don't have it yet, so, and I think having everybody, this group around the table is hopefully giving the various agencies the input and the voice from other groups, industry is not heavily represented here. We've been a little bit nervous about engaging them at what stage we need to do that, but obviously it's getting close. Well, and there's also the whole issue of the payer system, which is not all government, and so that's a very important aspect of moving genomic medicine forward, and all the providers and physicians who are out there. So I would just suggest that if this is put out as a federal government strategy for coordination around genomic medicine, it should be labeled as such so that people who aren't in the government aren't offended that they're not being included or their views considered or represented. I think that it's a good point. There's been a little, and Derek I think is gonna report later on some of the discussions we've had with payers that feeds into this, but we also want a practical process by which we could actually think about generating a document, and so part of the deliberations around that draft document would be what the scope is and who gets fully engaged. So just in the, all right, Naomi. Well, I will just mention as I look at the chart up there I'm really disappointed not to see VCBSA tech noted as in the space of evaluating clinical validity and utility because we have been in a leadership role in that regard in a decade. And we would love to add you. I think, be aware that this. We have contributed probably more sympathies than EGAP has at a faster rate and sooner than anybody else who can measure. So I'm a little disappointed. We'll add a call so we can sell it out. Yeah, all right. Let me note that this basically comes from the talks yesterday. And so we should have had you talk yesterday. We're sorry that we didn't. It's okay, I've talked before. Yeah. And this is definitely meant to be an inclusive thing because it's the only way it'll be successful is if it's inclusive. I think the very first slide you showed, I just want to clarity on one. Can you explain to me how bullets three and five are going to interrelate if at all? I'm just sort of imagining if we're, I'm just trying to sort of get a sense for those two completely separate activities. You're going to convene those groups and look at for more longer term goal, blah, blah, blah. But then separate from that, you're going to develop a national. They may see the end, but I think the thinking was, and again, this was thinking that was done but after we finished last night and before this morning. So the idea is I may be maybe not quite fully baked yet, but I think the thinking was bullet three was a very specific set of projects that might inform principles. And that might go along in parallel with the broader discussion in the generation of a white paper. But those principles hopefully would end up informing what the national strategy might look like. Parallel not serious. And just to give some people something to chew on. So there was a discussion about, well, gee, how could we get them engaged in reviewing the research plans of the Institute? Well, that's kind of a big thing, but we have the small thing that are these small projects and that'd be a place to start. Okay, I think we probably need to move on since we're already quite a bit over time, but the discussion was good and I think merited going over time a little bit. We do have everybody's email address. And so unless you send us an email telling us you don't want to hear from us anymore, we will send out some communications and invite your participation going forward in these possible next steps that we've talked about. Any final thoughts? I think I lost less, right? He wants to hear. He's the next one. So I'll have one final thought. Going to Muin's point and Ron's, I think getting success stories early on is good. So for example, I'm trying my best to implement uniform lynch screening within the VA, but I'm keeping track of it. And I think just having a record of a very large healthcare system doing that, we have Ohio, but things like that and the way that we implement and then say, yes, and it works and it saves money and it saves lives. And so if we can all within our own framework, we don't necessarily need to cooperate. I mean, well, it's nice to do, but we don't have to do things like that. And I was just filling in. Something that occurred, well done. And something that occurred to us, I think, was in the NHS strategy where there's this vision of the NHS will be a leader, well, could you substitute the VA will be a leader by 2020 and the VA will do this, that or the other thing, or the Air Force or the Armed Services. So something to think about. All right, well, we had, toward the, actually a couple of times yesterday, but certainly toward the end of the day, we had a little bit of discussion about incidental findings and we thought it would actually be very useful to hear there's been a recent publication from the ACMG on guidelines and we have several people in the audience that participated in those discussions. So Les was gonna give us a presentation about that and that may engender some additional discussion. So Les, the floor is yours. Great.