 brief slide deck. And I wanted to say thanks to all the organizers for the quick reorganization. I think this has really worked quite well. I've learned a lot about some of the areas of emerge beyond discovery that I was a lot less informed about. I think that there's an enormous amount that the VA can learn. We are developing our big data strategy in the VA which crosses both research and clinical care. You're having built a genomic research infrastructure within the health care system. So a lot of the things that I've heard today are going to be very applicable. We are approaching a 250,000 individuals in MVP and as Neil had mentioned earlier in the day, about 220,000 of those will be genotype using an array very similar to the with some minor modifications similar to the global array or the African-American array that Neil had developed. And then figuring out how to take that knowledge and bring it back to our clinicians. That's always our primary mission in the VA. So I think that that's much what we talked about. It's been very applicable and we have those 250 are consented. It's a trust model that will allow for some evolution and we built in some of the things that we've heard earlier today like the ability to recontact and actually survey individuals and potentially opt in or out of return of information as the field evolves. But there's an awful lot to moving that into the implementation range even within one health care system. So if I can have my next slide, I think that some of the issues that we've heard today are echoing themes and I'm going to be very brief. There's various levels of institutional buy-in and it sounds like you all have gotten institutional buy-in. One thing I've learned about buy-in when we move knowledge from the research side of the health to the clinical side of the health within the VA or within partners, which is the system that I also live in here in Boston, is that there's buy-in at various levels. And I think we have to think about it. We've talked about it today. There's overall buy-in at the top. Then there's sort of clinical leadership buy-in. There's buy-in from the informatists who have to help us with the implementation. And that's one issue that we have struggled with in the VA, particularly because this can be sometimes resource intensive and we may not be able to pay for that implementation aspect. Then there's buy-in at the user end, the clinician end. And so certainly understanding their needs, understanding how they utilize things, understanding their stressors, understanding the fact that they get many reminders, alerts, and other realms of clinical decision support are often in play and we may be competing with other aspects of clinical decision. And then buy-in at the patient level to the extent that these things are accessible in a patient portal. Number two is system heterogeneity. And we've talked a lot about that today. And I don't think we need to go into great detail. I think that certain of the vendors are less amenable to modifying their platform. And so we talked about one solution today, which is to make sure that the institutional leaders that are buying from these vendors speak with a unified voice to try to get, to try to pressure certain vendors, particularly the big ones, that are rolling out now in really cookie-cutter fashion EMRs that are going to be laid out over. And one thing that is being dealt with in partners is that there is a general sense that the modifiability of some of these, what I'm going to say is big EMR vendors is perhaps not what some of the local implementers would like to see. Then how customizable are these and how that could impact the aggregate experience if we want to learn from that in aggregate. If we end up with customized systems where the information, whether it be process outcomes or not, are useful locally. Will that translate across the systems? And how does this compete in terms of appearance and other things with other clinical decision support? The nice thing is that there is, in most institutions, individuals who in the non-genomic world have been involved in this clinical decision support arena for quite some time. I think I might last slide and we'll just finish up and then turn things over to some of the other panelists. So I think that it's important that my only comment here is, and we can discuss in general in the next steps, is moving from process-based outcomes which are easier to measure. You can measure how many times a test was ordered or how many times a report was read, how much information was utilized in a given clinical situation. A little bit harder is then turning toward clinical outcomes. Does this really do what the health system wants to do, make patients healthier? Other system outcomes, such as improving efficiency of care, often put in the context of the health of the individual. And what that means is that we have to phenotype individuals on the back end and phenotype in real time if we want to look at who has less bad outcomes when they've utilized a clinical decision support tool. So I would talk about a lot of this today and there's a lot to think about. And I think it's time to turn to the other panel members. And since they're listed in alphabetical order, I'll suggest we go Erwin, Ken, and Lucila if that's okay. If Erwin is warming up, maybe we can just turn to Ken. I know Ken's on because I see him chatting. Ken sent me a message saying he is muted by the organizer. So, Brandy, could you unmute Ken? Oh, can you hear me? That's it. Great. No, I think everyone's points were really good. But you just have a few comments. So one aspect is, I think, overall, just from a high level view, I think it's important to do, to make CDS for pharmacogenomics and for genomics in general, that's active from an operational standpoint. So my day job now is more operational. I'm in associated CMIO at my institution now. And you sort of see what sort of how the operations and the health system can look at some of these decisions more. And I think there it's really important to think of how can you make this so that the 10 or 20 or 30 second elevator pitch to the hospital admin makes this more attractive. And I think there's probably two components of it. One is to increase the value. So how can this solve a problem that is near and dear and at the top of an organization's operational priorities? And I think we talked about process and needing to go to clinical outcomes. I think also going to cost effectiveness is important. Like, we have a paper that's pending publication in DMC Medical Informatics where we systematically reviewed clinical distance for for them, patient setting, looking for cost effectiveness. And essentially we found there's almost no data. And I think, you know, if you're making a case to clinical administrators why we should invest in pharmacogenomics for distance support in an operational sense in terms of going widely, what you need to do is show that this can be cost-effective in what you're looking at. So I think that's important. And then the other part that folks have been talking about in terms of using standards is that I think that really needs to be decreasing cost. And I think it simply is hard, like the CS Consortium example is a good one, it's hard to do a lot of these things. And if you couple, we don't yet have cost effectiveness data on the impact of these kind of efforts on the bottom line. And also we don't have, and more of the cost of implementation is relatively high and is a relatively big demand on, say, your hospital IT group. I think that's a challenge. So I think sort of from a high level view, looking to show cost effectiveness and looking to come up with approaches, whether it's using standards, services, et cetera, or using available tools in the commercial EHR system without too much need for adjustment, figure out how to decrease the cost of implementation I think would be important. And I think when you have research funded through something like Emerge where you really can, you know, take the time to really figure out how to make it scale, how to, you know, research cost effectiveness, I think that could be a really great outcome if you can, one of the outcomes is we've shown and we have very easily available now information you can provide to your IT group to say to implement this, it only takes these steps and requires 20 hours from this kind of person and you're done. And moreover, we've shown that in this type of a setting, this is going to reduce your cost, whether in, you know, a capitated or an ACO type of setting. I think if you can do that, then it'll sell itself and it'll disseminate widely. Yeah, I would second that that's critical. I mean, as these, you know, big health systems are moving toward, you know, global risk, there are going to be a lot of pressure to use their informaticists to do well proven clinical decision support that's going to improve the health and lower cost improve efficiency of the healthcare system. So we're going to, you know, whatever comes along in this space is going to be competing with that activity. And if you can demonstrate some efficiency, I think that that's a real advantage to getting the attention and resources that you need to get it implemented. So I think that that Erwin was neutered and he's back on now. So Erwin, do you have comments? Yeah, I agree with most all things that have been said. I want to add a few points to that. And one is when it comes to EHR integration and clinical decision support, I think there's broad agreement across the network and beyond on this panel today that clinical decision support is sort of the main vehicle to focus on. And when it comes to clinical decision support, it's a very difficult undertaking, as Mark pointed out in his opening remarks. I think going forward, what we might want to consider more than what we have to date is to actually bring some selected practice leaders, clinical practice leaders into the fold, because one of the key elements to successful clinical decision support, and then certainly also downstream, you know, collecting and measuring meaningful data is that we have a decision support that really integrates exceedingly well with clinical workflows. And so we have now done this in the form of code genetics implementation sphere that was outlined earlier for emergent beyond here at Mount Sinai, but also in with the April one clinical decision support is one of our genomic medicine demonstration projects. And we found that even within one health system, such as Mount Sinai, and having 130 physicians involved in our programs, various programs, these are a number of different practices and practice types that, you know, each group of physicians in the different practice types, whether it's private practice style, faculty practice, or whether it's the clinic, or whether it's a community network clinic, each of those practices has a very different perception of the decision support that is the same text that we presented to them. So I think, you know, going forward, certainly this is sort of probably one of the main vehicles for us to play in implementation. I think what we might want to consider bringing more into this focus is sort of a direct reach into those folks that are sort of opinion leaders, etc., in different clinical practice settings, and that will then obviously be driven from email three perspective from by the point of what what kind of areas we are focusing on or what might want to focus on for implementation. But I think that is something that might be from a very practical perspective going forward. Very helpful in a comprehensive approach to a focus on clinical decision support and having it optimized for clinical adoption and then being able to make meaningful measurements from the implementation of that. Great. Thanks. And Lucila? Yeah, well, I agree with all my colleagues. And I would like also to point out that the modifications to the EHR that clinical decision support needs to make are not that different from the ones that researchers need to make, for example, to implement trials to facilitate recruitment and so on. So there is a lot of need from the research as well as the clinical communities. The clinical community obviously will have much more of an impact in asking for such things. But I think a unified front would be of much influence because when you're selecting eligibility criteria and so on, it's not that different in terms of research or in terms of recommending the right thing to do for a particular patient. One thing that I think is no surprise that pediatrics and oncology have done more in this area, I believe is because of clinicians are more educated about genomic medicine in many cases, as well as the patients being more informed or their parents being more seeking of such opportunities. I think the opportunity to leverage PCORnet, which is newly formed and just had its kickoff meeting this week is tremendous. And the lessons learned from EMARGE could definitely pass on. So PCORnet doesn't make the mistakes or go in routes that you've already shown that won't work. And hopefully vice versa as this clinical and patient power research networks proceed. So I think I'm very pleased to hear that we're moving in the right directions. Great. Thank you. And I think now it's the time for our open forum on this topic. If anyone else has anything to add? This is Eta Berner. I was going to add picking up on what Erwin said is that we tend to talk about clinical decision support or EHR use has to fit into the clinical workflow as if it is the clinical workflow. And in fact, there are many, many different workflows often individual physicians within a given clinic. And we need to take that into account. Right, I think that's important. And I completely agree that having clinician buy in and all clinicians are not the same is critical as the clinical decisions report tools that are being developed. Other thoughts or comments? This is Peggy Pysig from Marshfield Clinic. And I just wanted to comment on something that Ken said. Cost effectiveness is definitely one way of selling this to, you know, operations. But the way that Marshfield has done it in addition, well, we haven't done it by cost effectiveness, but we've taken it actually under patient safety and sold it that way. So that's another approach that may be worthwhile in investigating. Yeah, in addition, hitting quality metrics. I mean, if you're meeting standards, meeting guidelines, it's another, another important consideration. Yeah, this is Mark. I wanted to make two points related to what Peggy said that are, I think that's really important. We have a couple of opportunities within what we've currently we're currently doing to take a patient safety approach. One relates to something that came up briefly on today's webinar, but was really a focal point for the global medicine meeting that many of us were at last week, which is a prevention of Stevens-Johnson syndrome and toxic epidermal necrolysis. While these are relatively rare events, they incur huge cost to the system. And at least for some populations, there's a big opportunity for prevention. So that would be one example of a major problem that could be presented from a patient safety perspective. The other one is if we can figure out the genotyping issues on CYP2D6, hospitals are being beaten about the head and neck on pain control for patients, but also on safe use of opiates. And I think we have something to contribute in that discussion that could be helpful as we sort of ping pong between those two extremes. And that would be something that would be attention getting for the leadership. The other point relating to some of the economic discussions before, you know, when we tend to think about economics, we tend to think about, you know, looking at things from a national perspective and qualities and a very academic approach. What we're really talking about here are economic studies from the perspective of the healthcare delivery system itself, which is a very different methodology and is something that there are very few groups that are really doing, but I think we would have an opportunity to take advantage of what we're doing to look at it from a very different perspective, which would contribute to the conversation about why this should be implemented. Yeah, I completely agree. It will become, you know, as we move to these risk contracts, the economics of the health system that is going to drive, you know, what their EMR looks like and what they're just directing their positions to be doing. So I'd like to respond to Mark talking about the PIP-2D6 and opiates. That situation might be so complicated, and it's so complicated even on the genomic end that an assay that measured how that metabolism happens on a practical level for a patient might be the only way to really get to the bottom of it. It looks, in our superficial way of looking at how complex the genomics are related to how the metabolism actually happens is not that strongly predictive, and hence a more practical approach that's non-genomics might actually be something that would be more useful in terms of the clinical application. And just to say, you know, the genomics is not necessarily the end-all for the way we solve all the clinical problems. This is Mary. I wanted to, well, I guess first of all say, I do think that for PIP-2D6 one can easily identify the 10 percent of the population that are poor metabolizers. I know there's a lot of confusion for some other findings, but the vast majority of 2D6 high-risk patients can be identified with relatively simple tests, and that's allowed our hospital to leave codeine on the formulary for patients that have permissive genotypes, which has really helped with the opiate prescribing problem that you were referring to. But one thing I was thinking as you're talking about using safety instead of cost effectiveness to be the primary driver behind the CDS initiatives, which is also what we've done at our institution, would you emerge institutions be able to summarize what the majority of your interruptive CDS is currently and what the added burden of CDS related to genomic testing will be? I mean we found out as part of our genomic implementation that we needed to do a better job of tracking what all the interruptive CDS is that we have ongoing, but the vast majority of it that's customized is patient safety related and we find that our institution is very willing to support efforts to create that CDS, whether it's genomically based or otherwise. Yeah, Mary, that's a really good point because the whole concept of alert fatigue is one we're well aware of. I think though some of it also relates to not only the current burden within our institutions, but are there other ways to do it rather than an interruptive alert? You know the example for Simvastatin, you know in my view the the you know if you went into a statin prescription and you had the star 5 genotype, Simvastatin wouldn't appear on the pick list and the only time you'd get an alert would be as if you tried to type in Simvastatin and then would say hey there's a reason we didn't show you this on the pick list and so I think there are different ways that we might be able to do this without interrupting workflow and look at that as our only solution. That's true but there's a lot of genomic CDS that will be dose-related not choice of drug related so you're still going to have to have alerts for a lot of genomic findings in pharmacogenomics. Well not necessarily because when we did our warfarin studies at Intermountain what we did was the process for dosing warfarin using clinical features was already in the hands of the pharmacologists and so the clinicians were just seeing a dose and so when we did our randomized control trial it was simple for the pharmacologist to use the genomic dosing algorithm along with the clinical features and again the clinicians just saw the dose so it made absolutely no difference to them the fact that we incorporated genomics in that particular type of a workflow so I think studying workflow design and not just always defaulting to an alert is an important thing. Mark could I ask a different question at Susan? Most of what you all have presented about is CDS and related issues once information is already in the EHR but I wonder if you guys could back up and link more explicitly to the issues just before your panel because you know in genomics and in a merge you've got to worry about this translational process of how do you structure the flow of information into the EHR when is it ready for that move how is that governed I wonder if any of you could reflect on that for a second. Well certainly at Geisinger we dealt with those issues relating to our implementation of IL-28B genotyping for our patients with chronic hepatitis C in terms of treatment regimens and so as we decided clinically that this was desirable and that this information would be usable we had to solve all the different problems which is first of all how do we make sure the genotype gets done because we do not have preemptive genotyping for those individuals and we use a standardized order set that included genotyping to reduce the barriers to trying to remind clinicians to actually order it and then the issue was once it was ordered and we did the genotyping which we were doing in-house we had to develop a manual process to be able to represent the IL-28B genotype as well as the viral genotype as structured data within the EHR so the medication recommendations would be able to run and so we developed a manual solution to that and we now have you know a process that's essentially running in a hundred percent compliance but that's a one off and you know so we have to get over the fact that to solve some of these problems rather than keep building one off. And I wonder how that looks across all the immerged sites it seems to me there's a comparative opportunity there because this is a core question that has to be solved in a more standardized way. So we are looking at that in the that's part of our EHRI outcomes measurements is to be able to look at how people are actually doing some of those things and the hope is that as we aggregate that information that will provide us some opportunities for standardization but as been mentioned the challenge that we have is still ultimately we're dealing with vendors and whether or not they would be amenable to modifying their product to represent what we think it's the best way to do it. So just a reminder we have about one more minute on this topic. This is Ken maybe if I could do a quick comment on that notion of what to do scalable. I think first just one point is vendors actually oftentimes do provide extension points and I know some folks have already exploited those and I think one approach is to just say what's currently supported and let's figure out a way to be able to scale on that and another point is just that this notion of scaling CDS is something that of general interest to the ONC and it's been a priority for for example for development for meaningful use stage 3 proposed criteria. I coordinated one of those efforts called healthy decisions to develop some of these standards and there's still a lot of interest at ONC, CMS, etc. to you know develop these standards to make them scalable to make to solve this problem essentially for all of medicine not just personalized medicine and I think doing whatever is possible to align with that would be a great thing in particular because the idea is then this these would be industry standards maybe part of meaningful use and therefore implemented in all each our systems etc and just aligning forces with these efforts that are really trying to solve the same problem just not necessarily focused on pharmacogenomics. Okay well we'll take this pause in what has been a very good discussion and sorry Justin we're going to move on down down to