 I was just wondering to what extent, because a lot of the words and things we've been talking about over the past while are very linked in with the global alliance for genomics and health. A lot of the issues are very much the same. And Jeff was asking me what lines of collaboration could be more formalized or whatever between the US and Canada. We have just launched a new project, it's not very big one, it's about one and a half million, actually it's more than that, it's about three million dollars, to try and get our act together in Canada in terms of data sharing across institutions, across jurisdictions because we have issues in terms of what we're doing in Canada, the accessibility to clinicians or researchers in different jurisdictions and different institutions. And we have to get our act together in order to be credible partners for the global alliance. But I'm just wondering if to what degree we could, and I heard Mike Broodnow's name mentioned several times and he's very much involved in this new project which actually will be driven by Bartha Knoppers who is also very much involved in the global alliance and who many of you know. So I'm just wondering to what extent these things could be aligned and to what extent emerge is already very much linked in to the global alliance? I guess I can take a short version of that, it's an easy answer, we're not especially aligned at this point, that's not to say that we shouldn't be and I think it is something that I hope will be on the agenda going forward. One of the things that worries me a little bit about, I think the work of the global alliance is great but it sort of emphasizes Chris's point yet, is it yet another standard and how do we make sure that we're getting the ones that we have aligned and make sure we use the best parts of each one and not just say oh that one doesn't work so we're going to start all over again and create a new one. So I think that will be an important... So I think that's a great comment and we have certainly been trying to influence the global alliance to not reinvent the wheel, look and see what's available out there, I'm hoping that people like Mike Brodner are sufficiently aware of what you're doing, what other countries are doing because I think they're going to be very much involved in the words interoperability, the APIs and all of this kind of thing. I think they're going to be very involved in doing the detailed stuff on that. So anyway, we can collectively make sure that there's not too much reinvention of the wheel. I think we'd be really good. Mark. Thank you. I thought Chris was going to tell the joke about it. The group gets together and says there are 18 standards. We need to harmonize and then we have 19 standards but I think it's a variation on the same theme. Two comments. First is related to training. I just wanted the group to be aware that there are some discussions that are occurring between the American College of Medical Genetics and Genomics and the American Medical Informatics Association relating to the Clinical Informatics Fellowship Program which is accredited through Preventive Medicine to develop a genomic emphasis for those clinical informaticists that are interested in developing genomics. And we're in the process now beginning to think about what would a curriculum want to look like. So I think this would be an important group to engage in that sort of curricular development and perhaps also then aligning with the NLM training programs to say, okay, can we harmonize across all these different groups. The other thing is that AMIA is also then looking at a non-physician-based fellowship training program that would sort of be the equivalent for PhD or other informaticists that could also get training there. So I think there's a real nice opportunity at a very low investment of time and effort to accomplish some things in this space. And I'm sort of acting as a go-between so anybody that's interested, just let me know. The second thing is you mentioned Genome Connect. I'm glad you did because that's also, Genome Connect has been involved in the ClinGen project. And in addition, in PCORI, one of the PCORnet patient-powered research networks is using Genome Connect to collect data. And one of the things that's really interesting as we talk about engaging patients more in terms of data collection and supplying the phenotype data that is difficult to get out of the healthcare system is that there's a lot of ability to create new forms and templates. And patients seem to be very interested and willing to take the time and effort and they're pretty much experts about their conditions. They are experts about their conditions and certainly there's been plenty of examples where these types of patient-powered networks have really been able to discover things that had previously not been noticed. So it is an innovative area. It's one where I think that there's some real opportunities that are worthy of study and, again, not to necessarily promote one platform. But it is a platform that's beginning to appear in a number of different spaces. And so trying to not, again, have 50 different ways to do this, but having at least some standards around what's being collected and how it ties into things like HPO and that would be good. Just a second. So Adam and then Mike and then Howard. Great. Thanks, Rex. It's a little hard to get recognized back here in the back seats. So I just wanted to comment a little bit on what Chris was talking about, what Pierre was talking about. So it's been commented before the IOM's Digitized Action Collaborative. This is the Genomics Roundtable's EHR work, where we're actually representing genetic information in the electronic health record. We've been building this specifically off of the HL7 standards. We're building this off of 251. We would be going towards FHIR, but FHIR isn't going to do that for meaningful use right now, but we're looking at that as a future iteration for it. The way we've been operating, though, is by trying to make sure that we can actually engage with all of these groups. And we actually have liaisons to almost every NHGRI program we've talked about today, as well as Global Alliance. And in fact, actually after the Global Alliance meeting in Leiden, we're going to start up quarterly conference calls to make sure that we are aligning our work and courting our work together to make sure that we are progressing. We're also working with Moink. We're using standard Moink terminology to build our Moink codes for this. And the end result is that we've already gotten agreements from EpiCerner, Meditech, Allscripts, and Athena Health to actually build these and develop pilots, at least based on two pharmacogenomics use cases, which are our first iterations. So we are looking at making sure that we're using existing standards. We're not trying to build that 20th standard. We're trying to use 1 through 8, 1 through 19, I guess, and whatever's available. But it is something that we've been cognizant of. I'm sure we're talking about the standardizing across different EHRs and using a 95% certainty of caseness. And in thinking about that, as we're evolving the standard, involving the field, that 95% certainty in some instances of specificity might capture 90% of all the cases. In other cases, it might capture 10% of all the cases. And it's a logical starting point. But as Chris said, there's lots of room to store, lots of extra bytes of data. I think Cisco now measures data circling the globe in zettabytes, which is a billion gigabytes. But if we stored the actual probabilities, if they came across, then you can actually do sensitivity analysis for more extreme caseness. Or maybe you don't necessarily throw away all the data of people that are intermediate caseness. So I think that there's potential ways to think about setting the standards that will allow mapping across the EHRs that might be a little bit different. Antithetical to our clinician view of the world, which is either have it or you don't. But from that 10,000 foot vantage point of mining, of being the data janitors that we are, we do assign probabilities and maybe we can use some of that data. And then I have just a brief follow-up comment to what you were saying about. You triggered a question, is it when does a phenotype become actionable? We've just curated something novel and we now know something. We've associated it, so let's say we associate it with mortality. And we're stepping very gingerly in MVP toward reporting to clinicians or to participants and we've actually told them that the rule is that in the research realm, we won't be doing that. But you could certainly conceive of us coming up with a predictor, not just like some of the genetic ones or maybe even much stronger. Where we identify something novel about a patient from a metabolomic profile or from a curation of the phenomic data that is just as valuable clinically to a particular patient or his doctor. But I don't know that we've ever had that dialogue about, are we generating actionable information on that side of the equation? Let me just quickly say one of the ways that I think we've started to think about that and emerge is through looking at what we're calling FIWAS. So looking at across phenotype space and so looking for co-associations there. But I think it's a really ripe area for a potential research project. Of course, FIWAS has done it on ICD level, but don't get me started. You bring up really a deep philosophical point. And that is, what is the boundary between a disease state and a phenotype? I mean, at some level, certainly Mendelian diseases, a single change constitutes a disease by definition. And to what extent does abnormality from the norm, whatever that means, which is what we're talking about when we're talking about phenotyping, characteristics that are not common. Which is why they're recognized as explicit phenotypes. And to what extent do they constitute a disease situation? It really gets into very deep issues of philosophy of disease or how we characterize disease. The reason I fuss about that is because of my ICD-11 role. But I think as we think about genomic association, particularly disease gene association, it begs a more crisp definition of what we really mean by a pathological entity that we deign to call disease, or at least have sufficient pathological effect that it would become an actionable circumstance, or at least a desirably actionable circumstance. And those sort of by their nature are really phenotypes. And so we get very circular in this whole notion of whether phenotypes predict common diseases, okay, that's fine. But phenotypes in and of themselves raise significant boundary issues over what disease is. In the pharmacogenomic space, we've been talking a lot about the fact that these phenotypes that in most cases is what we're really interested in, that it's that metabolizer status or it's that receptor status. The phenotype that derives from the genotype that really drives the decision about how you alter medication use or dose or whatever. And it raises an interesting question, which is, is genotype the way to answer these questions the best way? TPMT is a great example of Mary, has studied, you know, for years. If you could measure the enzyme activity easily and cheaply, wouldn't that be better than deriving the metabolizer status from the genotype? And so I think that we have areas where we're already beginning to explore the role of the phenotype as opposed to the inference of the phenotype from a genotype. Well, we heard about intermediate biochemical phenotypes earlier today as well. I think that that's a very good point that this sort of, that we're developing new concepts of disease. But also, I mean, we're defining old parameters in new ways. I mean, just as an example, a slope, a single value of a slope of urine protein progression over, because you have now the opportunity that never before to look at 100 values of someone over their 10 year life expectancy might have more predictive value than the way the clinician is using it today. Especially with APOL1 genotype. So now, is that information that we should also be reporting back to the patient and his clinician? And if you're having that debate about actionable variants, then we have to begin to have that, perhaps we have to begin to have that same debate about new actionable phenotypic parameters. Doesn't have to be either or, right? I mean, it's just a reminder that we shouldn't build our EHR and our standardized terms that are so specific to genotype that there isn't room to build the same kind of logic into something that's phenotype. So what we do is both phenotype and genotype for a really important gene like TPMT, where patients, you know, health status is at risk. But I wanted to follow up on one of your points, Rex, around the storing of the data. So I think it makes a lot of sense, obviously, not to have the whole genome sitting inside the EHR. But it did beg the question from me as we were talking about in our panel three with reanalysis of data. So how do you see that working in relating to the EHR? Yeah, I think, you know, it gets back to this question that I can't remember who was talking about it earlier, but it's, at what point is it more valuable to redo the test with maybe more sensitivity or with a better approach than it is to store raw data? And that was a topic that came up, you know, a bit earlier. You know, obviously I think what we're imagining in the ancillary genomic system is to store raw data and then process that into a VCF file that can then be matched against whatever knowledge system we have. But, you know, at some point, the question is, which is cheaper, storage or for the original files, BAM files, for example, or to just redo the sequence? And I think that's an economic decision. Storage is almost free. That's not what our institution tells us. Yeah, I don't think that's necessarily true that storage is almost free over here, Chris, other end. So, because the storage depends on whether it's archival storage or whether you want that storage to be close to compute so that you can actually re-analyze. And if you're going to be able to do this re-analysis, especially from first principles, from the raw data, you've got to have all those data close to the compute, and that is not inexpensive. In fact, that's very expensive, depending on the volume of data. And so, I'm not sure that we have good systems in place for shifting data, agilely, from cold storage into these hot zones for analysis. And I think that's one of the areas that we really need to think about. Because going back to the issues earlier today about analyzing variants, well, calling variants is still an evolving art, right? So aligning, the basic process of aligning sequences to a genome is still evolving, and that technology is going to change. And so I think there is going to be a lot of really interesting re-analysis science that we're going to be able to do if we've got the data stored in the proper way, and we can get it to the compute seamlessly. Yeah, I was just going to comment on the phenotype question that, you know, you can think about this in the phenotype as genotype environment interaction, and you can think about the EHR managing that as well. Yeah, so just in the same theme, so clearly we have this opportunity for the first time to examine the phenome, whether we call it deep phenotyping. Up until now, we've been incredibly constrained with simplistic phenotypes, often just binary phenotypes, and we've got marvelous depth of genome sequence data or GWAS data, and suddenly now there's the opportunity for just about the entire portfolio to flip over into rich deep phenotype data at very little incremental cost. So I think that this is one of the most exciting developments that's coming up in the next couple of years is to catch up on the phenome. And I would very much like to see it be interoperable and not be offline, because that, as you have mentioned, enables dynamic mining of data. Because I don't think it's going to be static. We'd really like to do a best effort to have it fully integrated in the medical record, at least all of the non-referenced nucleotide sites, which is about 5 million. That's not a great deal of information and have it be dynamically searchable. So I just want to pick up on this thread because a lot of the discussion about deep phenotyping is implied from what Mike said and others is that it's all in the EHR and the ability to do longitudinal capture. But I think the mobile health and digital health platforms are a form of phenotypes, and for the first time, as Stephen was saying, we now can get 24-7 phenotypic information and translate that into velocities and instead of static measures of particular, you know, well-validated, at least, forms of that phenotype. So in anticipating where the precision medicine seems to be going, it might behoove NHGRI to really think about expanding the kind of integration of genomic information with the EHR and doing something similar with genomic information and digital or mobile health technologies. Just quickly to add, and we all throw around this concept, it's in the EHR. I mean, there is no the EHR, really, because our records, think about where our records are. They're in our doctor's EHR, our specialist's EHR, the hospital we went to in Florida's EHR. We broke our arm, the one we usually go to, CVS, Walgreens, National Death Index, you know, our claims data files. The exposome is the particulate matter that we get from, you know, the meteorologists. So, I mean, the data is everywhere and sometimes not so easy to totally integrate. And that was a fragmentation I was referring to, really, in our decision. But that's why we need personally controlled health records, right? So that the individual has all of those data and is able to share those data. I don't know how to do that. I mean, I couldn't share, think of my financial data. How could I own all of my credit card, everything, and I share it with people? I mean, I just wouldn't know how to do that. I don't know what that means. We own all of our data. I know we all want it to be able to be seen around. I don't want all of my healthcare data. I wouldn't know what to do with it. I just don't know what it means we own it. I heard that over and over again we should all own our data. I want a few pizza. I want the cholesterol level when I want to go to the lab. I want to be able to schedule an appointment. And I want Dr. A to know what was in my record from five years ago. Plus, that model doesn't work for children, for old people, for people who are blind, for people who speak not English as their language. It's not fair to the most vulnerable parts of our community that need us to figure out how to do this. It's great for patients to be able to take control of what they can. But I don't think we should depend upon that as the way to deliver healthcare. Just thinking that, well, I feel that it's very beneficial to retain the sequence data in some form. But if we're talking about data that has not been validated in a CLIA-approved lab and putting it in the EHR, not trying to speak for any federal agencies, but I think there may be some pushback from the FDA in doing that. So just something to keep in mind. Well, I think that raises the whole broad question of what sort of the regulatory pieces that surround this. And one of the problems with the EHRs, at least as most institutions work with them, is there also are legal documents that are subpoenaable in case there's a concern about a mistake being made. So I think we do, there is something that's legally defined EHR as opposed to sort of a collection of health data that may be unrelated. Sharon, did you or? Well, I was just going to echo that even if it was done in a CLIA lab, right now, EHR, I actually did literally have this circumstance where a patient approached me with a hard drive from some cafe that had done. Like, if I walk up to my EHR and say, like, here? Like, I mean, there is no location. But I did just want to echo the other comment, which is that I think some people really do want to own all their data. Most people just want good health care. And I think that there are lots of people who don't have the capability to own their own BAM file. And I do think that EHRs right now are pretty overwhelmed just with this. And just to give one anecdote, my brother was diagnosed with a brain tumor. His wife asked for an MRI and she really pushed them. I want it now. So they actually gave her, like, four DVDs because it turned out it was, like, some very early pre-processed state. And she gave it to her brother-in-law who was a radiologist who was like, I don't know what this is. I mean, I don't think we realize how much processing is done on all of our medicines. So when we say we're getting our MRI, we're getting a very processed, certified version of it and not, like, the raw data. So I think that sort of we've sort of seen the broad complexity of all of these information things related to what's the definition of phenotype all the way through some of these broader questions. But maybe to sort of bring it back to where Howard asked us to start with is what might be research projects that NHGR might think about. So just to throw out a few and see people's reactions to that, you know, one of the ones was, you know, how do we improve phenotype and genotype sharing? I feel like we're really well along the way to, with things like ClinGen and ClinVar to think about sharing of genotypes. I think we're a long way from the sharing of phenotypes and, you know, and I'm guilty for the projects I'm involved with. Most of the time, you put all the genotypes up, but you put the one phenotype up that caused the genotypes to be put up there, and I can go on and I won't at some length about why it is that we do that, but I think it's an area that's worth some discussion. Related to that, I just wanted to say another area where NHGR might be interested to come up again earlier is this interface between what the meaning of phenotype is informed by humans but also informed by a variety of model organisms, and so there's been, and Carol could certainly speak to this, but there's been a lot of activity in the last couple of years in the model organism community to think fundamentally about what the meaning of phenotype is, you know, the human phenotype ontology that we heard about earlier is a good example of some of the principles that were actually learned in model organisms being applied to human phenotypes. And the Monarch system. Right. So, you know, to think about, there might be research opportunities there for bringing together model organism community, things like HPO and Monarch together with the, you know, CSER and Emerge, and to think about how to better capture this phenotype. So I think some focus on phenotype seems to me a really great opportunity. I'll mention three more and then open up for other people's thoughts, but, you know, this whole idea of portable phenotyping algorithms I think is, you know, there's been some work on that done in Emerge. I think there's intention for more work to be done in Emerge, but I think to think about maybe how we do that across IGNITE and CSER and some of the other activities in terms of phenotype would go a long way. Clinical decision support, I think the opportunity for thinking about sharing of at least decision support rules, again, there's, I think, some movement in that direction that came out of GM7, which was one of these genomic medicine, but again, to think about how NHGRI could create some research around how do we better share clinical decision support rules and what do those look like. And then related to that, I think, is the Infobutton project that we've heard so much about. Infobutton is a great infrastructure to be able to collect data outside the EHR with data inside the EHR, but one of the things we need, I think, a lot more of is what that educational material outside the EHR looks like that informs physicians and even patients about what genetic variation means and what some of these conditions mean might be another area for us. And again, there's been some discussion about that in NHGRI space related to the IACC, ISSC? And then, and I'm not gonna remember the name of it, but the project, there is a genomics information education page that's maintained. G2C2. G2C2, right, talk about acronyms, so. They all have Gs and Cs. Yes, I think all of those are things that I would put on the table as lessons from the discussions here that we should talk about. So Mark and then Chris. Yeah, just kind of rank ordering those a bit. Thanks for mentioning the Infobutton project. That's something that in ClinGen we're actually looking at and talked to Wendy earlier. A lot of the resources in genetics for patients and providers in that right now are not Infobutton accessible and certainly not Infobutton compliant and we actually have a plan in place to work with groups like NCBI and others to make their resources Infobutton accessible so that at least those systems that wish to link their electronic health worker to those data sources, they'll be able to do that through that Infobutton standard, which all certified EHRs will need to be able to do as part of meaningful use three. The clinical decision support interoperability is a much harder issue than it probably should be, unfortunately, but I think some of you have heard me say that in HRQ funded seven years of a clinical decision support consortium and at the end of that, they were able to share one rule between two institutions and so that interoperability piece is really, they learned a heck of a lot, but the reality is it's very difficult, but at least at a very low bar in eMERGE, we're creating and we'll have live by the end of this month the clinical decision support rule repository where at least we'll have the written logic or a visual logic of how a decision support rule might be filed and a number of the eMERGE groups are contributing to that so people can at least see the logic pattern and then they'll have to go and try and code it and so it's not interoperable plug and play, but we're at least making some projects, but that would be an area where I think there could be a lot of energy and time and resources invested with probably very little payback because of just the challenges in the EHR space with clinical decision support at present. I'm obviously gonna pick up on your phenotyping theme, but I think the community and not just NHGRI has been very casual about the entire phenotyping exercise. It's time to start treating clinical information and phenotype algorithms as a first rank resource. It's an interesting question of whether NHGRI can and should do it in and of itself because quite frankly the last thing we'd like to see is an NHGRI style of phenotyping and an NCI style of phenotyping and so on and so forth. I mean you get the picture. There's no reason why NHGRI could not take the lead in coordinating a really an OD level project or a common fund project or whatever it might be, but I think phenotyping has reached a tipping point where it is now the most soft and vulnerable part of the whole scientific enterprise as we try to learn association disease outcome and improve health. I'm pleased that IOM is doing this in partnership with the usual suspects on the clinical side. That's delightful. And so maybe extending that kind of collaboration and consortium to a true NIH wide effort to engage in consistent phenotyping or...