 Thank you. Okay. We'll open it up broadly now So a really almost trivial question though is the ignite example has been brought up a couple of times and I was just having a side conversation with Jeff Because I've I've been under the impression that ignite is a bunch of different projects all focused on different aspects of a similar problem, so The idea of creating you know common surveys or is a really pretty laudable one So the very specific question is was that in the budget or is that something was that an unfunded mandate added? afterwards So I'll have to defer to Jeff would you can you I'm happy to make a comment and Erwin and Julie you should as well I think So like all the other networks we we came to the table with specific projects in mind and began discussions about where the cross-cutting themes would be across you know amongst the projects in the consortium and Now we've arrived at a fairly sizable number probably more than we can truly manage So I haven't answered your question yet, but but I but I think it's implied that You know, there's the project funding. There's the working groups that I think Most most of us I think all of us PIs and investigators feel are quite compelling and necessary for the success of the network I don't think there's a specific line item save that the very there's a high expectation as there are in all these programs that the coordinating Center will You know Reduce the additional burdens and logistics of that as much as possible, but I think All of the groups right now, which are the common measures group. You've mentioned a Group associate associated with dissemination and outcomes and economic sustainability and some emerging groups that are You know thinking about the scientific focus on pharmacogenetics the interactions with the payers as we've discussed and also what the provider needs Are for adoption. Those are some of the main themes right now And I think just another Just sort of caveat the ignite network is only two years old, which may sound like a long time But in the context of what we're trying to do here It's insufficient to draw significant conclusions at this point So let me let me I'll stop there Part of your question and maybe Erwin and so we would like to let me just let me let me react Let me re-ask the question and that is sort of or not re-ask the question was let me back off from the phrase unfunded mandate because I think that that sort of carries a Tone of criticism that I don't wish to imply I think I think you've done a terrific job And I and I think maybe one of the recommendations that we can make to NHGRI leadership is that is that when we put Efforts like this together, even if they seem disparate at the beginning There needs to be some mechanism to make the whole greater than some of the parts in emergence Easy because we're all sort of focused on the same kinds of issues in in Caesar. It's it's it's It's I can see how Many sites could could have commonalities in ignite it was less clear to me But I think you've managed to show us that it can be done and to some value presumably So so I guess that's the sort of unspoken or maybe clear to everybody in this room except me of my question Yeah, no Dan That's it's a very good point because I can say from our experience that initially There was sort of some sort of tension between each project being funded and wanting to get their own project out of the gates as quickly as possible and through Coordinating center and other in the sections. There was at the same time a Mandate to come up with common measures, etc. Etc. Which made sense But you know wasn't exactly what we had in our proposal and therefore we needed to make up effort to address that and I think it is an important aspect I think in in In hindsight if it had been sort of explicitly stated as part of the RFA or something That would be helpful and I think going forward if you or one networks have these kind of mandates then You know, this is also part of the budgetary issues, etc. Etc. I should be part of that That is a very clear point. I think Let's let Julie get a shot in here too if she wants to make a comment Yeah, so I mean I would I would agree with what Jeff and Irwin said I mean, I think that None of us had budget to do some of these things that were asked especially around these common measures and And I think everybody wanted to play ball if you will because we recognize that as a network We have to accomplish things as a network. I think we're getting a bit challenged because as Jeff said You know, we had some early places where we said we had to start and now And the networks really in some ways only one year old or less than one year old because the last three groups joined officially about nine months ago and that's really when I think we begin to establish some momentum and So so I think we're at the point now that we have a lot of opportunities that we see for common work across the groups, but probably not the bandwidth and so You know, it's a challenge of in the budget We have in our individual projects that we have to also be successful in and then thinking about how we prioritize those group efforts So, I mean I would agree with Irwin's comment that it might make sense in the future to have And probably in the RFA you say X number of dollars that would be just sort of set aside for common Common network-wide projects and I think some of these challenges for these Common elements would be less so I think there's two things that I take away from that one is is that One can envision even though we have a relatively small time exposure that there are probably Outcomes common outcomes across dignite that you say, you know, these really make sense Where you could bring that to the initial steering committee meeting of Emerge 3 and say You guys use these process outcomes unless you can develop, you know, a good argument about why you shouldn't use these process Outcomes and I think you could take it the next step Which PCORI has done which is to say if you're putting in a PCORI proposal We want to see Methodologies that have been developed and vetted by PCORI as part of it or we want to see something in the project That says here's where the deficiencies are and we think there's a better way to do it And I think that that would be something that could be systematically applied to funding opportunities in the space going forward So Sharon, I think you were next And I would just comment from the Caesar experience Where I think the diseases and the ages are more Wide-spread than some of the other consortia. It's actually really important to build this into the RFA So we have, you know, some groups are being ascertained for hearing loss. We have adults with cancer We have everything in between. It's actually been a major challenge and to find constant measures So I would just say that building that into the RFA this say you're including phenotyping is very important So Jeff had a brief comment and then Mary and then Howard I'll be brief. I First of all, I thank Mark and everybody for Highlighting some of the things that ignite could bring to the table and I think I speak on behalf of the network and saying we would be honored to have more programs adopt some of the measures and strategies for implementation and share in this what we called in our first panel the Implementation commons and met possibly some of the evidence Data databases that have common measures. So that's just a out there for NHGRI and leadership to to embrace or reject But I think that that would be an outcome that we would like to see Great Mary. I was just gonna say I mean it does seem to me like for any of these programs It is helpful to have a coordinating center try to help facilitate some of these efforts But I would imagine that you need money at the sites in order to generate data that is shareable Unless I'm not understanding what you're doing. We had a similar thing imposed on the PGRN and it was very very difficult to try to In the translational pharmacogenetics project get people After they were years into funding to generate data in a completely different way than they were set up to do So I would just imagine it sounds great to me to try to come up with common outcome measures across as many NHGRI Programs as possible, but I wouldn't think a coordinating center could handle most of that Yeah, maybe I should be more clear within the RFA means money For the the grants know to put that into their budgets that they're gonna have to report those phenotypes Yeah, so it'd be it'd be similar to the expectation that you build data sharing Data deposition as part of the funding That is absolutely right. I think that is exactly the point Mary that you know from our experience And we had talked about this that on the local project side everything is focused and geared to execute the project and yet we have all these important conversations about what should be the Common measures and then develop these and even the coordinating center Coordinates perfectly well the data and the input still needs to come from the local sites And I think that's an important consideration. So I'm hearing us in violent agreement on this point So unless there's strong objection or somebody that says there's a point That's not been brought up relating to the common measures in the RFA and Chris Do you have something that that's specific to this that's okay? So after Chris if there's anybody that and Terry apparently so Thank You Mark You know this issue about being in the RFA is important But I would quibble as to the relative influence of coordinating centers Having been part of PGRN and having been part of a merge those experiences were night and day And I think what made a merge more cohesive Particularly in the first round was when the coordinating center Proactively Contributed toward the development of shared standards and representation paradigms that Made emerge as a cohesive data resource come together that never happened in PGRN And whether it was the coordinating center or not or whether it was the intention of the group or not We could have long and spirited debates But I do think somebody it needs to be somebody's day job To worry about how all these pieces within the context of a consortium actually do come together And unless I can be informed otherwise it would seem that the best way to do that is with an empowered Coordinating center and thank you Chris for not bringing up integration of informatics data. Appreciate that This is extraordinarily helpful and we really appreciate the input and recognizing I think Dan You know you're coming about an unfunded mandate. It's probably not totally unfair You know I think that if there were resources put in into it They were very limited and and they were really you know We were focused on the on the projects as a whole way I think though the thing that came up that that sort of struck me was you know We're hearing that there are challenges in getting people to work within the networks because you have your own program And you're responsible for the success of that program And then we're also hearing wouldn't it be great if you know all of these programs could interact and at some point You know the band with fails and and so We could use some real help on what's the best way to facilitate interactions I mean some of my colleagues sitting behind me do a yeoman's job and bringing together specific working groups And I think the EHR working group has been quite successful in that we have our reporting results We have a couple of others there But but you know there's there's variable uptake of those and so do we let that just sort of happen organically as Investigators are interested we do we try to you know put a feeding trough out there and get people to come to it So we could use your help on that great and I saw Ken I think has a comment also related to this one of the colleagues behind her No, it's really more of a question as you were covering the informatics and the tools that are being developed Does this group feel that our goal as far as trying to make these tools widely disperse While it applicable to multiple and diverse platforms Do you feel that this that what we are developing what we're developing in the future? Well, it's gonna be fulfilled and what I mean by that is say for example with a merge We have algorithms that is on there verified by different systems that use different EHR systems What I'm trying to get that is do we feel that this group that the tools that we are developing Would they adequately be developed to cover that kind of diverse platform? And is there an art is there a need to actually try to encourage and endorse development of plate of tools that actually Can cover diverse platforms. That's what I want. That's the question. I wanted to ask So that that is defines a different subject area for me So that'll be next panel. So so can we put that in the parking lot for the present time? Are there any other comments specifically related to the Point that I think Terry really nicely summed up. Yes, so so the only other thing I would suggest in the RFA is To ask groups in their proposal to think about what are the things that might be those common Measures and is a generic term Because I think when a network comes together at least everybody's thought about it And you could hopefully you would begin to see cut some common ground And so I think it would get that ball rolling more quickly if that was also part of the RFA It's not just there that there's money for these common efforts, but that everybody's Put in you know a paragraph or two about what are the things that they think would make sense as common measures Maybe I could just share a little bit of an easiness about Describing how RFA should be written if you can tell us in much broader terms, but but we probably shouldn't have you Specifying what might or might not go in a solicitation just for more for your protection than anything else So that you're not in conflict then with the solicitation that comes out with that suggestion good So I'm gonna again take moderator prerogative here to just say that you know the my experience with these is that you know with 55 60 70 people in the room you can infinitely Drill down on topics and so I saw Jonathan's hand up and and I guess the question that I would ask to anybody that's thinking It contributed to this is that is there something that you think is completely novel that we've missed or is this more of an incremental? Comment and a refinement that we might be able to pick up later because I know Howard has a different subject That you'd like to launch and I want to make sure that we get a chance to cover the sphere So Jonathan, I don't want to shut you down. But no, I don't know how if I have something novel to say or anything, but The I would just say that you know I think there's a great deal of benefit to the data sharing and having some Standardization across the sites But one of the things that's been really nice about Caesar and end site is that the projects are very diverse And I think that there's some danger to proscribing too closely exactly what the The you know groups have to be studying and it sort of eliminates some of that You know creativity that comes out in the grant writing process. So just a little site balance with standardization and innovation Okay, great. I wanted to just make sure that your bullet to here is reflected in this discussion which we know so developing whether common measures of outcomes in the absence of some input from The payer community or the regulatory community would seem like it has an opportunity a risk of failure to achieve the desired goal So I'm not gonna say how or where or but I think as we've talked about through several of these discussions If there's going to be future research In this area from NHGRI somehow upfront in designing the Project goals and aspirations that community needs to be part right so I'll just again point out that Panel nine is going to specifically represent and be focused on the patient perspective And so I would think that that would be a nice opportunity to discuss this there Regarding the payers. I think you know those of us that have been involved in the pair meetings We've had is that the payers have been extremely difficult to engage and in particular even when we bluntly asked in the question But what sort of evidence do you want? It's sort of the Potter Stephens thing. Well, we'll know when we see it. They've not been particularly Valuable and again, this may be an opportunity where if we think about Healthcare system research networks that are you know engaged and embedded with payers that we might get more substantive answers So I am going to move on to Howard and a different topic now Well, not that different because Jeff kind of transitioned over to a different aspect of the discussion and that is around What are the metrics of success? And I think I'd love to hear some discussion around Intermediate in points any immediate metrics So we're seeing an oncology right now that many of the payers are interested in is that somatic sequence going to change a medical decision They don't know how to capture whether that changes for the better or for the worse but the fact that there was change is a step towards paying for it and then you know at some point in time We'll look for changes for the better. However, those are measured and at some point time look at survival But even these intermediate which are maybe intellectually less interesting But it is you know does change actually occur those endpoints are often ignored by NIH type funding because they're just not exciting enough but I think they you know that comes back to Jeff's point about the the patients the delivery systems the payers often those are the kinds of decisions that are driving their changes and You know their bar is really low in some ways because they don't usually want survival. They're happy for these intermediate endpoints Yeah, and certainly for those of us that write letters of medical necessity I mean that's the question that we're specifically asked to respond to how is this going to change your management decision It's not show me the evidence that it's you know ten years from now this patient is going to be better off So so that I think is a very reasonable point so others that have comments about Sort of this intermediate outcome or maybe what might be characterized as this space decision space or when yeah I thank you for starting this conversation Howard I think as I reflect on the programs that I'm witnessing and ignite and emerge and How I can optimize our own program at Mount Sinai It is increasingly clear that the very first Intermediate outcome if you will or intermediate metric has to deal with the medical decision-maker the provider Because there's so much to learn about where they are and there's so much to study How can be optimized sort of the comfort level of the provider and decision-maker with Incorporating genomics and then actually acting changing a practice as a result of genomic information and I was intrigued by surveys of our providers and as part of the ignite project where 68% two-thirds are enthusiastic about genomic medicine and this is a survey of almost 300 providers Whereas less than a third so around 20 or so really are comfortable in any way shape or form So there's a tremendous gap there is enthusiasm But we still need to learn and that's I think an area that deserves much more study how to optimize the process of provider adoption of Genomic decision support. I think that is an important point. I want to stress and that is and that is an intermediate Outcome in in my view you can measure this as you know How many CDS were fired and the fraction that was leading to a change in in in orders Etc. Etc. So there's a number of ways you can readily measure that Thank you Other comments related to that Robert. Yeah, I mean I just building on Erwin's comment I think that's a really good point I worry a little bit about this iterative nature of the implementation evidence evidence implementation So are you actually saying that the that the That the frontline physicians Behavior becomes one of the outcomes we study or are you saying we generate evidence? Including penetrance including penetrance of intermediate phenotypes to set precedent to set standards That then they can adopt because I think the second alternative is more true to the way they may be enthusiastic But I don't think they're going to adopt rational actions without that first evidence Based generation Yeah, no, I'm not Proposing this to be sequential. I think this ought to be a parallel and I think that you know the interpretation of of outcomes In a situation where the delivery was not optimal was not optimized is difficult It is potentially biased to suboptimal processes. So I think you know the knowledge Closing the knowledge gap the evidence gap is extremely important, but also optimizing the delivery and So then then you can really with good conscience look at studies that do both and draw conclusions on outcomes Racks and then Sharon So in the in the context of thinking about research projects going forward Our experience when we tried to roll out the our pharmacogenomics project in a merge Was that we went to them with these really carefully considered CPIC guidelines that had been developed by experts where they thought there was real Evidence to say if you have this very you should do the following And it was very interesting when we're in our in our group that we rolled it out to was a general internal medicine clinic and They at the end of the day had a the ability to say we'll do these ones but not those ones and They actually chose to do only a couple of the ones that were approved as CPIC guidelines And so it occurs to me and we've been trying to do a little bit of this But really understanding what the barrier is there and why it is that they thought that What a group of experts in pharmacogenomics thought was good evidence to create a guideline but yet was inadequate for less Deeply trained individuals with expertise in that area I think would be a really important set of things to better understand So that might be a good research project to think about going forward. Yeah, it's it's interesting because then it gets into the issue of you know evidence versus Cultural change which ultimately we all know that culture eats strategy and evidence for lunch Mention the role for RV use. Yeah Sharon Yeah, just very briefly. I I do think that we have to we really need better research on physicians So in an NHG or I LC project that I carried out now a couple of years ago We were looking at very an interpretation But in fact the main discovery out of that project was that most and this was primary care physicians in the state of Texas Completely unselected did not know the concept of a familial mutation And so when presented a familial mutation and a cancer patient and asked what tests would you order? They overwhelmingly selected full gene sequencing So it wasn't so much that the variant threw them off, which was our hypothesis. They actually were reasonable about that They had no clue you could order a $400 test instead of at the time of $3,600 test So I do think we need a lot more such information about what non Specialists in genetics know and what their clinical practice is and guidelines really clear guidelines Yeah, I mean there's there's a whole as putting my informatics hat on there's a whole range of Implementation questions, which is you know a lot of times we default to the idea of its education and all this sort of stuff Whereas ultimately the the the decision and this is a again a studyable hypothesis is Well, what if you just build the process so that the clinician doesn't get to choose that if that familial mutation is there That's what's ordered and that's the end of it and you don't have to you know Then you avoid, you know some of the other issues now again there that's You know, that's a that's more of a cultural And systematic change approach as opposed to an individual change approach But there are clearly going to be some instances where you have to do this at the individual provider level though I mean be others where systematic approaches would be more effective and you could study Under what circumstances would one approach versus the other be more be more efficient. So Howard. I think I had you next So I think the other side of this equation is is also looking at the patient We know that as you were just saying, you know implement this is we're talking about 21st century medicine We're we don't look at patients anymore, right? But that's the whole point right is that adoption is really a hard part and what we've learned is Direct to consumer Advertising I'm not saying direct to consumer testing advertising is a driver And I think there should be some look as to how could this be done actually in a way that people feels responsible In educating the patient because we do know that if you keep having patients showing up asking for something And they're going down the street that is probably going to adopt Much faster than almost anything I can think of and and that's what the pharmaceutical industry is actually learned around this So I want to be careful that we don't call and we don't move to direct to consumer testing, but Enlightenment and asking for this, you know, get your genome sequence ask your doctor might be a little bit too bold But something in that direction. Yeah, and again a big advertising budget. Yeah There's that which I'm sure in each year. I'd be happy to come forward with so I think Julie we had you again Yeah, I'd like to come back to Something that Rex was talking about which is you know when do physicians choose and and I think it really goes back to the very first Discussion about evidence I can guess some of the ones that they chose not to and if you ask them It would be because there's not an RCT That documents the benefit and then you do an RCT and you get back to What I think Jeff brought up which is an RCT is usually an efficacy study not an effectiveness study So there's this very I think maybe more for pharmacogenetics than the other examples, but there's this rather circular Challenge I think we are in in some ways and convincing Certain sectors of the provider community. Well, I think it's also just the point that Mike brought up earlier Which is the idea that had that come out of the cardiology that guideline come out of cardiology That would be something the cardiologists would probably be more likely adopt and oppose this group that they may never have heard of before and And and that of course then creates a real conundrum because it's probably Unrealistic to expect professional societies to come up with guidelines around all of these different instances But there's usually no process Although I know ISCC has been you know taking a little bit of a look at this Which is how might we be able to have a cross-cutting process where we could you know Have joint development but in most cases and certainly Having been on the board of the Merrick College medical genetics and genomics Trying to develop joint guidelines with professional societies that we're aligned with like NSGC or AAP or that It's a real nightmare to deal with the administrative hurdles of doing that what the by-law say you can do so You know that that process and whether that's something that you know We know that NIH has in general been not playing in in the guideline Business, although I think the ISCC is at least talking about it theoretically Whether that's a worthy topic for research or not. I don't I don't know so Mike did you want to say something because I think you you had had Yeah, well I mean think that there are going to be and a lot of this the guidelines evolve Organically and we can we could have a whole day About that that whole process, but they are that there will be uptake of certain You know actionable Variants or profiles that that will find their way and there's always are you know in cardiology There's always already discussion about You know whether or not risk prediction we heard the example and in cardiomyopathy So I mean there is going to be uptake it's going to be spotty and hit or miss and I think that there There is some utility Even though the politics of working between societies can be sometimes quite challenging and can be cumbersome I really think that that it's going to be important for The ACMG to at least explore the possibility for engaging various specialty Societies and and and if if for one reason just dissemination and education the other is that I you know I you know I Envision the possibility of recommendations around the same variant being different society to society So, you know, so I'm giving clinical context So we're we're getting a little bit close to time here I've been given a little a few extra minutes because the discussion has been going really well So I've got Dan and I've got Bill and I've got Steven, but before we move off of that I just wanted to mention something that I found out is that we were developing in ClinGen a resource landing page And that is that NCBI actually maintained. I think it's NCBI. I don't think it's NHGRI Maintains a site of all genetic and genomic guidelines And so there's an aggregation site that I wasn't aware of and it seems to me that that's something We plot that on to our ClinGen resource page But that the idea of having a place where we can Centralize and use that as a dissemination vehicle would be something to probably leverage that would be a relatively low cost so Yes, but the problem with the National Guidelines Clearinghouse is that every guideline is in there And while you can search them for genetic or genomic this one is focused solely on genetic and genomic guidelines Which in this space I thought was incredibly useful No, nope, it's still going still going. Yes having formerly worked there. Who was who's that disembodied voice? Okay Can I Can I add to that just that in terms of collating all of the genetics genomics oriented guidelines? That's something that Medgen and the genetic testing registry do at NCBI and then we've been working with moving Currie's group and Sherry Shelley and Dave Dotson to actually do an analysis of that about what's happened over the past few years and Etc. Yeah, that's that's what I remembered seeing so thanks Wendy for clarifying that since I couldn't perfectly remember Dan Just a very brief comment that that's the business of you know Who writes the guidelines and then who pays attention to the guidelines so the clop integral guidelines from CPIC? I wanted Mary to say this but she didn't want to say it is you know include Clinical pharmacology people clinical pharmacogenomics people and clinical cardiology people as well And I think that's the way to do it you engage the multiple communities to to write the guidelines in that way them The guidelines or the document will have credibility across multiple Multiple constituencies the the the problem the Rex has heard me say this before the problem with implementing something like CYP2C 19 and clop integral in your own environment is that it becomes very important I think to engage the interventional cardiology community And if you don't have a champion within that community at your own institution Then it's and it's a hard slog And if you do then it's a much easier slog and just one last point and that is the point that Mike just made We've looked very very very hard for an effect of genotype on clop integral response in cerebral vascular disease And maybe it's the small numbers. Maybe it's a different path of physiology, but it's we can't find one yet So it really is The the evidence is best for people with clots inside their coronary stents and after that the evidence is less good I think that's an interesting observations bill As a general internist, I will say and this is reflecting also some of the comments that we've just had I would argue for actually engaging the Clinician various clinicians societies up front before you start thinking towards Planning out your next research program as opposed to making it An objective to look at the barriers as part of the research program because Having been on the side of having people try to dictate to me What I should be doing in clinic. I'm often looking for different things than necessarily what the recommending Organizations are looking for so actually having the conversation up front is probably a useful Start as opposed to sort of saving it as a research question Yeah, I think that that's certainly been a lesson learned from the inter-society coordinating committee meetings that have been ongoing as That's been an attempt to really get at that and one can argue about how effective that's been But at least we're trying to work in that space Stephen you get the last word in the discussion Then we'll sum up boy you're giving an Irishman the last word dangerous So I think we need two types of things for those of us who are on the inside who are believers in Genomic medicine and pioneers Guidelines are phenomenally useful because they allow us to understand what the community is thinking about and to adhere But for those who are non converts non genomic medicine pioneers It's exactly what we've just heard. They're intrusive. They're somewhat obnoxious. They're yet more work What the vast majority of medicine I believe needs Takes me back to when I was an intern and that was the Timmy and Tammy trials So we had a situation there where the treatment of an acute myocardial infarction was a piece of art And over a period about 15 studies in a decade it became completely parameterized down to dosing Timing the the set of Measures that would be implemented in in series and that's really what we need to develop I believe as a community for the specific use cases that we believe precision medicine or genomic medicine Can deliver on today? Yeah, I think that's a brilliant point And of course anybody that's tried to parameterize guidelines immediately runs up to the fact that The people who write the guidelines aren't really that interested in being adequately explicit that we like to use weasel words and now that you may consider or You know, perhaps should think about which doesn't you know lend itself to Developing in an algorithm and so that's a constant battle that I see Bob shaking. He said that we've that we've been through But the point is well taken that if we can create guidelines that can in fact be computable and implement them where there's Strong evidence that this is the right thing to do that can be very useful and no Wendy I'm sorry I'm not gonna give you your last chance to go in because we need or nope Terry's giving you special Dispensation so all right then and then I bow to Terry so you lucked out quick Well because you talked about computable and being explicit in guidelines I just want to support that because we we spoke with CAP for example about well so in in the What we do when we find guidelines is we try to attach them to conditions to which they're relevant genes to which they're relevant And then that way they show up on tests and conditions and so forth So there's some judgment involved in saying okay, what kind of lung cancer is this for and so we spoke with CAP about okay, you know, I know cancer genetics Am I really right about what I've picked out and does it correspond to these specific? Condition unique IDs and can we move to a place where the people that write guidelines can actually select the Unique cuis that they belong to or genes gene symbols. So they were very open to that You know, I think that this is a very new idea for people that write guidelines to make them You know machine readable, but it's it's very doable just with some effort Right, so Terry is being very generous here and has allowed Jeff now to get in so so sorry Steve And we didn't give an Irishman the last well where I thought the Irishman Jeff is Irish and I'm not aware of it Where I thought the Irishman was going Was was also the Timmy Tammy and various trials like that have really strong industry Sponsorship in order to do the kinds of studies you're talking about Require resources that probably are beyond what NHG or I can afford So it just is a plug for us to really give serious consideration about where the opportunity is to engage industry to develop the evidence along with us Great. Thank you. So I'm gonna turn it over to because of the way I moderate I asked if Terry would mind taking Summary notes and so Terry's going to do a summary and then I'm gonna quick go around to the panel members So I've also asked to Pay attention again if Terry mentioned something Don't reiterate it But if you there's something that you heard that isn't in Terry summary then we'll each have a chance to kind of fill in the gaps Hopefully this will will display so I did share my screen read in it's I assume it's coming up like really soon But even if you if you can't see it one of the things that we had had are hoping for from this session is not only impact of Sort of the the implementation itself, but the impact of our research programs So one of the things we need to do to in order to justify a research program is to actually show that you know That it's made a difference and so how we're going to do that would be very helpful to to know Rita Chambers Will stop sharing content as soon as you start sharing. Okay, so I hit share. Oh my god. It's working Great. Yay. Wow. Okay So so yeah, so those are those are the kinds of things that we'd really like to hear I'm not sure why it's quite so ugly, but we can go with it this way So so we heard a lot about similar methods and common elements as as word was done in ignite and about the challenge of providing support for that and Really that that NHGRI should look to ways to support and expect that common measures and other kinds of program-wide efforts would be Implemented it was pointed out that this is more challenging the more the more diverse the programs are I would argue that it's also the more fruitful the more diverse they are because you really what you know The genome is we like to think of ourselves as being everything And and so if we can find ways that our work is relevant all across the NIH Institutes as well as all across disease and organ systems in that that's that's a plus for us, so We were also encouraged to include plans to produce program-wide data and common efforts and the value of Coordinating centers or somebody having that as their day job. I think was was Emphasized and and it's not always the case the insight program for example does not have a coordinating center Maybe that's something that we should be we should be looking toward to try to do Actually emerged started with just kind of an administrative coordinating center And they were very proactive as Chris pointed out and kind of grew by themselves with some help from us But but showed the value of that and and really expanded into a much more active kind of group We were also encouraged to try to integrate with the HCS RN Which used to be HMO RN and I don't remember what it stands for but that's alright And particularly because that group on unlike many of ours has payers at the table and has them involved in Designing the kinds of research that they're doing as I understood it And so that that is something that we ought to to look toward on the other hand I think we need to be careful You know the the healthcare industry as as much as we complain about how much it costs It is a tremendously profit-making industry here And so I'm not I'm not sure that we want NHGRI research to support more profit-making Really what we what we want is to make medical care better and and I think that's what everybody wants even even the payers But we do need to draw so you know try to find line there in terms of what is the research that we're actually What's the goal of the research that we're trying to do We are encouraged to measure outcomes of value to patients payers healthcare delivery systems And even regulators and and I think we're we're not asking as much as we probably could from the regulatory systems Okay, what is the kind of evidence and information that you want and they may not know? And it may be up to us to sort of say wouldn't it be useful to you to have this kind of information? In order to change the way that you do regulation But we need to think about better ways of getting that those messages across and I was intrigued by Mark's reminding us of Reed Tuxin's comments back in May of 2012 I think it was that we're they're looking to us to transform the way we care for patients Which is probably asking a little bit much on the other hand there are some very proactive ways I mean if you if you look at some of the work that's been done in that the Ceaser program and that the Quincy project here at NIH and we really are going from you know Redefining what disease is based on some of the genetic findings that you then go back and you find oh my goodness This person actually has been suffering from a condition for years if not decades and just sort of assumed That's the way it always was especially with Mendelian variants where there are other people in the family that that's the way It always was and so that's just the way we are So so something to keep in mind But we are in a different era and particularly now we can't compare ourselves to cat scans and pet scans and those kinds Of things that the standards are different now The the question was asked in and I think is one that's well worth exploring maybe in collaboration with the ILM roundtable Can we design systems that will help to guide the clinician to the specific tests? They should do so that when really an individual gene test or even an individual Variance is the test to be done is there a way to kind of you know prompt them to do that rather than Jumping immediately to a whole genome sequence some systems and we could talk about this at the discussion session at the end of today Some systems actually have a gateway or a gatekeeper for whole genome sequence and they would ask many of these questions I think so something else to consider in terms of the research we could do The point was made when well taken that each profession looks toward its own societies for guidelines and promoting joint development is really really hard It's a little bit like promoting collaboration amongst the NIH institutes in that it's you know hard enough to do things within your own group But then trying to do them across groups especially where every group says unless we're involved from day one You know we're not gonna we're not gonna sign on that can be a major problem Also engaging studies and societies and study designs to find out what information is useful in their guidelines That kind of harkens back to what we said about having payers and other stakeholders at the table And then this this question of could guidelines be computable? Wendy assures us that that's not a problem as long as they're specific enough and we rely on on Groups like CPIC and others to give us specific guidelines Great, so quick around the panel a very brief or one. Yeah, I mean it's great summary Terry I would perhaps request to add to Now it's moved on the previous slide Yeah Mesh's outcome of value health care delivery systems and providers are not always aligned and I would add here provider as a separate entity to deal with in the value proposition Thank You Irwin Warwick Measure outcomes ask them first what they think might be made might be useful outcomes Thank you and Ruth Okay, great the only thing I heard a concept that Jeff Used that I thought was great. I'm not sure exactly What all would it be entail it but I love the idea of an implementation commons? And I just think that that really sounds it sounds great, and if we can actually Create something that's as good as it sounds. I think they'd be really cool But that was that was something that really was sticky for me So with that, thank you all very much for I think a very productive discussion and thanks to my panel members for all their help