 Thank you. I'll hold my comments until the end so we can get right into our two name discussants and if you've got questions after Terry and Karen that would be great. So Terry why don't you start off? Okay so well thank you for letting me participate in this. I appreciate it. I'm gonna take a slightly left turn then perhaps than our current speakers because I really believe that the issues that are problematic aren't how we come up with the clinical informatics system. It's actually the underlying knowledge below that and I think that that's that's where we're having to run into some issues and I think NHGRI has done some remarkable things with multiple networks but the question arises is how are they being integrated across these different networks? So you have Ignite and you have a merge and you have ClinGen and each one has a different website and these these are not being integrated so that's one of the things that and we saw this earlier in the discussion previously with regards to the population and diversities and so for example in the ClinGen activities there's a lot of effort being put towards populations and ancestries and as representative of CPIC where we worry greatly about coming up with guidelines for pharmacogenomics I will tell you right up front the biggest problem we have is how we define those populations and how we define those guidelines and they're not great I'm the first to tell you that and that's because we don't have really strong information or data with regards to these populations and that's really where something like Ignite can really help as we move forward but I think as we move forward we do have to think ahead as to how we're going to unite all of these different initiatives so that they are sharing that information. I agree with Josh about the informatics gap I think that that's a big one. I think there are two other issues that I don't think anyone really raised is how are we going to incentivize our institutions to prioritize the work with regards to genomic medicine. I can tell you that at my own institution that's an issue. It's great in the research domain not so good in the clinical domain. Kelly sort of alluded to it earlier I mean she finally she moved back in the last year so she's familiar with it from the clinic standpoint but we don't really have genomic medicine highly integrated in at Stanford and yet we're one of the leading institutions that do research in genomic medicine. I also think that there's a question that I'm going to throw back to Eric is should we be leaving precision medicine implementation to the vendors you know ours is it really the vendors who I don't know that we have any vendor representation here I know they come in at different places I know Sandy has been involved in them and other people have as well but I think there needs to be better buy-in from them you know is it the idea that ultimately we're going to have one vendor system probably not but I think that there needs to be is it a regulatory issue that there are certain requirements that each of these vendors must have something to do with it so I think that that is also the issue and then lastly I think there's a big issue as to who is going to pay for all this long term and it's not just who's paying for the studies that we are all in this room doing either individually from the research perspective or from the network perspective but how are we going to sustain the knowledge the underlying knowledge as it changes and how are we going to keep that and I know that this is a big issue for NIH it's a big issue for those of us who generate the knowledge that then gets translated into the general practice but I think that we need to come up with mechanisms to deal with that as well. Hi so I am a bioinformatics person with leaning towards genomics and I've been a member of the ESP for Ignite for like two and a half years I think and I want to thank Ebony for bringing me in to this project I don't really see it as service I see it as education for my benefit and it's just been this wonderful opportunity and so from my perspective on this panel I don't have answers to any of the questions that come up I just have a lot more questions and I feel like as a scientist when I look at the projects that have come together through Ignite I feel like the challenges are not so much scientific we're on top of the science I feel like the challenges are actually social and political and that to me is hard I'm used to science I'm not used to dealing with all of these different kinds of people that we need to make implementation happen and so the some of the questions that that I have that I think I think Ignite is is starting to roll with is like how do we how do we engage the EHR vendors they're kind of this black box that they don't they either don't say anything or they tell us how it is but how do we actually engage with them and and make make the relationship work like I find that challenging as a clinical informaticist how how do I guide my institute in where to go with precision medicine and I think Ignite has provided these really amazing demonstration projects that show an improvement in quality that when you use precision medicine in the clinic and I think that we need we need to use these then in this experience to take that back to our managers and really prove that this is something that our hospital should be investing in and so I mean I think there's a lot of social hurdles to get over to do that as well and anyone who who actually knows me through work knows that I'm really interested in standards development and interoperability and I hate free text I'm all about being able to communicate on both verbally and computationally and using using standards to do that so how do how do we guide the standards development committees and these are very diverse groups how do how do we as as like the accumulation of these networks guide those different committees to to focus on the things that we actually need to move forward and then then how on the flip side of that is how do we encourage our systems internally and our physicians to get on board to use these terminologies and ontologies so my own personal experience is is with phenotyping and people really they just want to write free text they don't actually want want to engage in in learning about the phenotype terminologies and use that and that would make my life a lot easier so that's like another challenge I think that one of the things that I haven't heard talked about today as well is is return of results so we've talked about getting things into the EHR and that's great but then how how do we provide the infrastructure to actually take that back to to the clinic and and that costs money and it takes a lot of organization and I don't really see anybody wanting to pay for that so these are just random thoughts that good why don't I ask the speakers to respond to the two discussants but we need to be mindful of our time because I want to give the audience a chance to ask questions well I'll start by addressing there's a lot of great questions there so I I thought we could talk a little bit about engagement of the vendors which I think is very important I mean regardless of what we say here ultimately at this point in them in the commercial market the vendors are are gonna be the ones who would have to implement any kind of actual code changes and so we have engaged the vendors within the CIG group we haven't come give webinars but it's been not a deep bilateral exchange but there are a lot of us have that kind of bilateral exchange in sort of other parts of our professional lives and and so you know I know Mount Sinai and in our our place there's a lot of discussion with the vendors about how to do this and my overall comment about that is the vendors cannot do this on their own they actually come to the table somewhat ignorant of exactly how to show genetic genomic risks and or even what the appropriate standards are what what's the appropriate metadata and so there's a lot for them to learn and they recognize this to their credit and have I think accordingly sought a lot of help around the country so I think there is a lot of ongoing conversation it's not always within groups like this but it the groups like this can help facilitate a certain agree I agree that there were a lot of great questions and issues raised by the discussants to to extend on the vendor question so as somebody within an academic medical center there's some areas where I feel we have great relationships with vendors and there's some areas where they totally drive us nuts and you know and cause concerns I do find that through put in a little bit of a plug for the digitize action collaborative which is a collaborative organized under the genomics roundtable at the National Academies most of the EHR vendors participate in that and we found them to be extremely helpful and engaging in that context and willing to work with us in general relative to engaging with vendors and also this general sort of funding question I mean we are internally outside of grant funding opportunities commercial opportunities opportunities for internal investment it really there is a lot of pressure to look at economics and how you can interject and use genetic techniques in combination with clinical informatics to produce economic benefits to the institution which becomes exciting to the institution and becomes exciting to to the vendors as well lots of you know risk in in going into those areas but I think that there is increasing excitement and increasing views of opportunity there so I'll respond to a couple of the questions also so one of the comments was about using about incentivizing implementation and so what one thing that I think could happen within it within ignite is to define what are outcome measures to I mean there's there's outcome measures in terms of a physician uptake and in patient acceptance and so on but there's also opportunity for outcomes that could be collected on an ongoing basis from the EHR and so reporting measures or metrics that could be looked at and could have a financial implication and so if we're able to show that there's potentially some some cost benefit of doing these kinds of projects that might help with the with incentivizing also there is a point about capturing data as as structured data and and and I guess one of the things that I don't think we brought up NLP too much but national language processing is is potentially important to consider and when trying to find the the data that that we need from clinical records and you know I've been in the the situation where I've wanted to find you know do the phenotyping where you find patients that have complex conditions and see that the the ICD-9 codes or the billing codes aren't sufficient and they're messy and they're missing and there there's all kinds of reasons why that's happening and and so then you have to look into the notes manually and so having automated approaches to do that will be pretty important for being able to find the data that we need to trigger a decision support and so on thank you questions from the audience lawn yeah thanks that was a very informative discussion I have a question for terry so as as I can see you lobbed a few grenades out there and I think I counted three and I think each of them could and arguably should engender some discussion the first one was the communication and sharing amongst NHGRI programs that probably should be taken up at some point second one was the vendor one and the third one I just wanted to draw you out on to see where you were going with it which was who should pay for this and who should pay for what is it the data generation is it the interpretation is it storage is it update where were you heading with that actually all of the above because from my perspective I'm I'm on the data generation side and the knowledge generation and so I worry about that because I know that there's a lot of human involvement in that and as good as NLP techniques are getting I will tell you that a lot of that for example a lot of the knowledge is buried in tables and NLP for example is not good at that we can also talk about mining EMRs that's great except that anyone in this room who is a physician and or a patient knows very well that a physician oftentimes will make a decision to write down a particular code or diagnosis because they believe that their patient will do better on a particular drug but they know that to get that drug they must have a certain diagnosis even if they don't necessarily have that so you know that old phrase garbage in garbage out so there's that piece of it and then there's this there's the storage of that knowledge the upkeep of that knowledge the translation of that knowledge and then the whole idea I mean in terms of that the EMR itself in terms of the healthcare system some of that is built in for the payee system I mean that's the whole health insurance part but that doesn't cover the cost it doesn't cover the cost of building the system the testing of the system the translation of it so I was really throwing a huge bomb I wasn't just throwing one it was I was spreading it across the board about three or four years ago a number of us were on the room published a paper out of EMR it's called Barriers to Implementation and I think as we hear the talk today we could argue that there hasn't been much progress we're doing wonderfully at identifying the barriers but the the so we're we've identified our targets well the challenge that I see is that virtually everything that we identify as a barrier is outside of the control of the groups that we're doing we can't control the vendors we can't control the payers we can't control the healthcare systems we can provide input and we can you know define what we think are best practices but in terms of actually having them move forward we have no ability to really direct that policy discussion so so I have two questions related to the Ignite meeting one is what are the areas within the barriers that would be good targets for research around things that we might be able to actually do something about and then the second thing is the opportunities presented now at the policy level by the movement that seems to indicate that ONC is saying if you want to be a certified electronic health record you have to have some degree of openness for applications that sit outside the EHR so it's not the monolith that we've been dealing with for the last I can see Sandy drooling already because he and I have talked about this a lot so we make what are the opportunities that we would have going forward in Ignite or other things to study how we could get around some of these barriers through the use of applications that sit outside of but interact with electronic health records so so I think to the two different points you know one thing that I do think is important for us to keep in mind is if we build things that if we build things that generate value that demonstrate value economically clinically then that then vendors will go after you know licensing and adopting those things so so that is a mechanism for interacting with vendors and and pushing things into into care I do think that in terms of enabling those things to come into play a huge part of this is establishing as everyone said the interfaces the standardized interfaces that would enable us to get at the specific kinds of interoperability we need to gather the data and and deploy this infrastructure and that again I do think the best way to do that is to demonstrate applications that expose interfaces that provide value to clinicians that provide value to health systems that that then incentivize people to build other interfaces that enable them to operate so Howard then Chris but I have a I want to interject a question here on this thread maybe I'll put lawn on the spot you're representing the entire US Chamber of Commerce in this question why isn't the market working you know typically when these are they're outside this room by and large and there are these opportunities and obstacles we we trust that the market's going to help us fill them but at that you know going back if indeed these are these these problems or challenges or years old where is it are there not market incentives to help fill this void that we need to pay attention to it seems that the by and large that the private sector is not working with us to help fill these voids so many thanks for that small question Eric so I'm I work for GlaxoSmithKline in the drug discovery domain so which is why Eric's probably picking on me on this so that's not a vendor in this context I think but it certainly is a consumer and I guess I would I don't know the big picture answer to that I could say the market is is working in oncology where targeted therapies immuno-oncology things are driving towards a precision medicine and and we don't need to push it I think it's going that way because the treatments are improving it's not the case in my view outside of oncology and rare diseases broadly lack of incentive and I think actually one of the things I wanted to ask later maybe I'll just ask it right now is if you think of if you think of pharma drug discovery before we even start a research program we go ask the payers what they want right and here we're doing all this research and then asking what they want and the risk there is we're going to develop conduct a whole bunch of studies and generate a whole bunch of data that isn't actually relevant to implement I wonder if we should turn that question around and and drive the discussion from there and you get the incentive so that they may not know what they want but but maybe we have the discussion and maybe we have the debate on what good data would look like that would actually move the needle so Howard so I'd like to thank the panel I mean I think you've hit on some of the key issues I mean when we think of informatics we're all often thinking about from the sequence to the clinical report but I actually think what you're touching on is how do you go from the clinical report to actual care is the biggest challenge and I think ignite in round two if there is one has some huge opportunities here to be thinking about it I do not believe that the EHR structure is designed to go forward with what much much of you are talking about much of medicine is it's it's a patient comes in with a phenotype we then do a diagnostic test as we move to doing more and more sequencing and having more information available in advance as was discussed earlier how do you use the pharmacogenomics is already hard in that environment how do you use it proactively how do you have that information and I have no confidence that the EHR structure is going to deliver around that so I think this is a huge opportunity you have data now from version one and you have input from your physicians but how do you move the decision support away from having four million alerts that come up with this variant and I think that this alert fatigue is another really big issue and as we have conversations with physicians about this they don't want more alerts and so I think ignite has got some huge opportunity to really address that critical piece which is going to be the barrier for genomic medicine it's how do we enable the physician to practice medicine in a 15 minute time window and I don't see any structure in place to enable that so I don't have an answer but I think that is a huge opportunity for this group as it goes forward in my opinion go ahead so I was just going to say that I think something that's really important is to have somebody from the health IT side on research teams if at all possible because I've been on like clinical decision support committees where there are other priorities of the institution so they have an allergy alert that's that's going off way too often and so they have a priority of what's what's going to have the largest impact and so these genomic medicine projects may not bubble up to the top necessarily and so have if it's possible to include somebody on the team to actually take ownership on some of some of the the implementation piece I think that that's important and there's apis within epic that I'm learning about and within these vendors that are that make data accessible for you know developing software so I think that there's there's a it's promising Chris last question from the audience thank you uh I just want to riff off your your point Howard and I I think there's there is a widespread expectation that and Sandy you alluded to this the genomic information will be in a sort of the moral equivalent of a PACS system uh and that in silico evaluation or if a clinician would order an in silico test effectively of something that exists in that repository but my question really is if this is going to go forward and and Terry your point about how do we coordinate this across NIH and frankly with with digitize and and other efforts I accept the premise that the core requirement is really information transfer transmission and interpretation that in my little brain translates to standards and as usual the nice thing about standards is there's so many to choose from uh in the context of this problem uh to what extent can ignite again in partnership with other NHGRI and for that matter with other NIH and digitize you know National Academy uh efforts really begin the winnowing process and the refinement process and the promotion process it is a socio-political problem I accept that uh but I think there's enough mass momentum and gravitas across the groups that are that are assembled on this problem to begin saying okay folks you know here's a rational way forward and in the kind of vendors are willing to follow the lead of clear and and and rational recommendations at least in my experience this this whole notion of having proprietary divergence for the sake of proprietary divergence and if if a separate industry starts to emerge we want to get ahead of that industry making up its own darn standards as industry tends to do what can we do to promote the harmonization and consolidation of information transfer representations so in my opinion there's only one way to really make this happen which is what we need is we need a killer app that provides value to clinicians that requires genetic data to be sent into it in a structured way and if we can do that we have to I think we have to lead with providing the value to the clinicians through a piece of infrastructure if then the interface is into that piece of infrastructure that people will want to implement into their systems will what will truly drive standardization I think driving from the standards towards the infrastructure I think it's extremely hard to make that work Terry you want to add to that yeah I just want to add two points one is when we talk about standards we're assuming we can define a standard and that's part of the problem so in pharmacogenomics for example not all clinical labs will define something the same way and they will not test it the same way unlike other sort of clinical tests if someone is measuring creatinine we know sort of the values so there's an inherent problem and it's not just in pgx it's in disease risk and and other things as well but that is a huge thing and so when we talk about standards we have to be a little careful there the second thing is actually I'm just going to build on what Sandy said and it sort of builds on what everyone else has said you have 15 minutes with with a doc and I'm glad you have 15 minutes because many with my docs it might be six minutes but the idea of an app if we have if if groups can come up with a way to convey the important information or knowledge that affects that specific patient and we usually say we talk about it in 30 seconds or less then in fact you will get buy-in you'll get buy-in from the informatics community you'll get buy-in from the vendors you'll get buy-in from the physicians who right now are not being trained in genomics but ultimately will be if you make it easy for your for the participants it will happen so is it going to happen in the next year or two probably not but if we're doing if if NHGRI is funding research and networks to do things that are effective now they're way too late they are they're trying to do it for five years or ten years down the road so some of the ideas that are being talked about here in fact are even though they are still barriers we are coming up with better ways to address it so I think that that's part of the issue. So I'm going to take the moderators prerogative and have the last words that you know this session in my opinion it all boiled down to sort of working toward you know clinical decision support and I'm going to add for the long term into that that sentence and I really think I want to thank the speakers the the the discussants and the audience everyone it was a fantastic session. The two I think take-home messages that I bring out for Ignite is first the focus on interfaces at many different levels you know there was an there were that there was the technical side the interface between the physician and the screen there was the interface among data components the case he brought up there was the personal interfaces of of multidisciplinary teams I think importantly towards the end the interface between the research community and the private sector and I'm going to come back to that in a minute and then maybe building a bridge to the previous session is the interface the social interfaces I think are important. Then the final point is I think for Ignite is the importance of engaging very early non-traditional parts of this community. There was some nice banter I think Mark and Howard about the importance of you know sort of engaging the payers very early in this process and then I think towards the end of this session we can extend that to the need to engage the EHR vendors again very early in the process and we we shift our focus of success from being counting publications to making sure we have a path to implementation so the the research that we're doing really is defining a path to implementation so I think if we can keep all those things in mind not only will Ignite be successful but frankly the entire community will be more successful so I want to thank everyone for their their engagement and for the ideas that came out and I believe the next I'll turn it over to Ebony but I believe the next agenda item is lunch. Yes and I want to remind people that Colette is going to email you the poll for testing for us to use later on this afternoon she's also going to display the questions for the poll on the screen so when you have time I know we're doing a working lunch so please just grab your lunch try to eat fast especially the ones that are going to speak in the next session if you could try to be the first ones or allow them to be the first ones to grab their lunch so they can grab a quick bite to start the next session and then we can be eating while they're presenting thank you very much we'll meet back here at one o'clock