 Thanks, Gary. Thanks, Howard. And also, thank you, KG and Rita, for your fantastic job of putting this meeting together. So, being number one, I guess, is, I guess, we will set the pace for the panel discussions. I would like to say that this area of evidence of what I think about is that what we're really seeking are the paths or the pathways to truth, ground truth as much as we can define it about the value that genomic information has in the care of our patients. And speaking of patients, I think patients with a C is something that we probably all probably need to have in this arena. I know that everybody in this room wants to see the impact of the work that they've been doing in their lifetime. But I think this is a long journey, and I think we're learning about the length of that journey through the work that this very important work that all of these groups are doing. So, I think it's an exercise of patience to make sure that we actually, excuse me, get it right. Because I think that's the most important thing. I think if we screw up, the consequences will be enormous. Spacebar, right? Arrow? Arrow? No arrow. Any other keys that I should be pressing? I tried that once, Eric? Ah! Try eject. Oh, okay, we're getting closer. Another spacebar. Arrow. Down arrow. Right arrow. Left arrow. Up. There we go. Okay. I think that last one, it was a spacebar, but it had to learn my touch, I think, or something like that. So, I think it's terrific to have a panel that we have today. The names of my colleagues are here. So, Jonathan Berg from University of North Carolina who runs the clinical cancer and adult genetics clinics there and has involved in several NHGRI projects. We're really honored to have Pierre Moulin here at who's the president and CEO of Genome Canada. And also grateful for whoever needs Rondeau from ARC sharing his experience and evidence generation. I think actually one thing this panel highlights is the fact that despite the myths that Duke and UNC are rivals with one another, this panel highlights the fact that we can actually collaborate. And there's actually a number of collaborations that take place across the great divide in North Carolina academically. But I feel like I need to remind you that we did win the NCAA basketball championships this year. So, in sports, no such thing. So, I don't know if any of you were reading the New York Times yesterday, but in the Sunday review section, I caught this article how the theoretical physics community is actually in its own evidence debate. And the question is how much empirical evidence is needed to prove theories that have been in existence for decades or even perhaps a century or more. But I thought this quote in the op-ed piece was actually highly relevant to what we're talking about today. Our most ambitious science can seem at odds with the empirical methodology that has historically given the field its credibility. So, there's a true balance. I think we need to be cognizant of as we think about the question of evidence. So, why do we need evidence for the impact, for genomic medicine to proceed and achieve its impact? Well, I think it's simply because we practice evidence-based medicine. And I have a picture here of David Sackett, who is known as the father of evidence-based medicine. He passed away within the last month and was revered for his contributions to creating this paradigm by which medicine around the world is currently practiced. But of course, the kinds of evidence we need, we have to get into the details of clinical validity, clinical utility that inform eventually the development of guidelines that are the benchmark by which clinicians practice medicine every day and that the adoption of those, of the practice of genomic medicine will require some infiltration of the evidence that we're talking about here into those guidelines. We want patients to advocate for the kinds of genomic medicine technologies and innovations we're talking about. So, the evidence from their perspective is quite important. And as we'll discuss, I think at greater length, the payers and the regulators are going to play critical roles in the evidence discussions and how regulatory approval and reimbursement is achieved. And then a notion that we'll come to in a few moments is the notion of pre-implementation evidence as a gateway to the broader evidence generation that will ultimately impact dissemination. So, I think it was the CDC about 10 years ago, or at least I first became aware of this evidentiary framework from the CDC about 10 years ago, probably was associated with eGap, people associated with eGap would know better than I. That's not really projecting very well, I apologize. But the word analytic validity should be projected on the second circle in it in the lower right-hand quadrant. But it's clear that different groups have taken ownership, different stakeholders have taken ownership of the evidentiary framework. We have groups like CMS and CLIA focused on the laboratories and the ability to achieve analytic validity that the test is reproducible and precise and robust. And the FDA has taken some ownership of clinical validity along with analytic validity. That is the test actually correlates well and is associated with a clinical characteristic or phenotype. And certainly the payers and the provider and to some extent the patient community are embracing clinical utility, which is another way of saying does performing this test lead to an action that actually improves healthcare outcomes? And while there are a number of elements of this framework, I guess the question that we will be asking today is how do we get alignment of what the actual goals for evidence are across this continuum from analytic validity, clinical validity, and clinical utility, and can we engage this community in this important discussion? So I mentioned this earlier, and just again a question that we discussed in our group prior to this meeting has to do with the fact that there has to be a threshold in which you make a decision to actually test whether a genetic or genomic test is going to actually be something useful to generate more evidence about. So what is that pre-implementation evidence or preliminary evidence needed to put this into implementation science research? And then what is the evidentiary threshold that is targeted for widespread dissemination and perhaps adoption? So there is not just one kind of evidence we would posit. There are several kinds, and this community I think has to make some decisions of not completely sure who decides which evidence, whether evidence has reached a pre-implementation phase. I would argue that an evidence by GM1 in which a lot of local groups were making decisions to implement in the absence of any type of national consensus or global consensus amongst the expertise in this room for sure. And when we discussed the evidence questions within our group before the meeting, we also thought that not only that evidence also needs to be contextualized for the decisions that it's being used to inform. And as again I've alluded to this already, but how do we align the expectations between the scientific community that's generating the evidence, the individuals that are actually evaluating what that evidence means from the payer provider and patient perspective and gather that community in a way that achieves the alignment to create the most efficient processes for evidence generation? I think a concept that we also want to touch on at some point is personal utility because that's what really is what the patients might be seeing most from the evidentiary frameworks that emerge. So these are just some examples that our group came up with for just giving a sense of the context. And I know this is debatable and we can have a discussion around this. It's not the point, but if you're going to put a genomic screening test for population and public health into action, you want to ensure that the characteristics of the test are almost immutable, that the sensitivity is high and that its impact following action is really going to have significant impacts on public health. So the bar for evidence for genomic screening could arguably be very high. Same could be said for selecting, using genomic testing as the basis for selecting not only an expensive, potentially expensive 100K per year type of molecular targeted therapy, but also one that might be critically important for a single decision that a patient has around their mortality from a disease. So the evidentiary threshold might be quite high there as well, whereas in other situations where the risks associated with making the wrong decision could arguably be tolerable and less impactful, the bar could be lower, so around the use of genetic risk testing to modify behavior or to maybe optimize pain control for patients at the end of life. Maybe that's a medium bar, I don't know, but I think the point is just to kind of contextualize these decisions. This is an evidence matrix that we've been actually working with at Duke and as part of our IGNITE project, which actually looks at many dimensions of evidence from the patient, provider, and even the health system point of view and also expands the dimensionality as you can see vertically into not only clinical evidence, but molecular behavioral, emotional, and financial. I guess I would argue that the vast majority of clinical studies mainly use the upper left-hand box as endpoints, that we focus mainly on the changes in clinical characteristics, which is our comfort zone perhaps, and maybe where we have the greatest tools. Maybe there are some in the lower right-hand corner when we do cost-effective or cost-utility analyses, but I think our goal in creating this matrix and also as part of this discussion is to really think a little bit outside of those boxes into ways that we can derive evidence that clearly has useful and that arguably even the provider community has been largely omitted from the endpoints that we design in our clinical studies. We also recognized in our discussions, and I'm sure you have, the infrastructure that's necessary for evidence generation is really a large part of it could be a leverage from the clinical community with the advent of a number of health IT solutions to bring genetic and genomic information into EMRs at least in a small, at least in an initial way. The use of a variety of technologies, which are allowing patients to report their outcomes, is becoming increasingly important. And again, finding that sort of sweet spot in the Venn diagram of clinical care and clinical research to leverage the activities of a health system as a way to also be an evidence generation engine. So who is addressing this topic across the spectrum of NHGRI programs? I've listed some of them here. I'm not going to go through them. A lot of them Terry mentioned in her opening remarks. And I really applaud the thinking behind developing this matrix, but as we discussed it amongst our group, I got a lot of pushback by trying to identify which of the NHGRI communities were actually contributing to evidence because I think there's a lot of discussion and debate about which of these activities are actually evidence-generating activities, among other things. But I would say a vision for where we want to be at the end of this meeting is to think about how do we organize all the rich talent and data that is being generated across the multitude of programs represented by all of you into a consortium of consortiums or something that we, by some mechanism that allows us to really pool the kinds of evidence that we're generating and leverage the strengths into synergies that will be evidence promoting. So the next couple of slides are gaps and barriers and I don't want to make it too depressing because there are many. And I'm not going to go through them in any great detail. I hope this is the richness of our discussion that will happen after I'm done speaking. But the first one is a simple conundrum or paradox is that to implement you need evidence and to generate evidence you need implementation. So there's a problem there or a challenge there we need to solve and I highlighted it earlier with the pre-implementation evidentiary threshold idea. I think many of us would agree that RCTs and traditional clinical trials are efficacy studies but generate, seldom generate the kind of evidence for real world implementation and I'm really pleased that a number of the programs that are represented here in the room are actually generating real world data in healthcare delivery systems and not doing traditional RCTs. We need to be cognizant of the particularly in high dimensional data sets and the kinds of multi marker panels that some of us are involved in using of the powers and design of the studies to really achieve the notions of clinical validity and clinical utility. I'm sure we'll talk in the discussion about how particularly in the U.S. we're highly fragmented not only as healthcare delivery systems, as scientific organizations, institutes, even as programs in this room we're pretty fragmented until today and that we need to really think about how to de-silo the systems in order to achieve the optimal results particularly with regard to evidence generation. I mentioned the HIT tools and infrastructure and I'm sure we'll talk about the need to augment what we're already doing in terms of professional education to get us to where we want to be in terms of evidence generation. The misalignment of the payers, opinion leaders and I think patients is important to include in this alignment strategy or alignment gap that we also are probably not doing adequate health technology assessments particularly on the value that genomic information provides to the system. Actually few studies I would be interested in hearing more about this during the discussion where the economic analyses that are really going to convince payers or health systems to really adopt these technologies. Where are those being done in our programs and should we be doing more of them going forward? I've talked about the need for data infrastructure in clinical care and also arguably in clinical trials integration of genomics into healthcare and electronic health records. And an important gap I think is that a lot of health systems do quality improvement initiatives. These are often done because you don't need to consent patients for them. They're meant to be sort of internal evidence-generating activities but they never get published. I think that's a real loss I think for our field particularly if any of us are doing QI initiatives around genetic and genomic testing. So I sit on the Institute of Medicine's round table for translating genomic research to health and I wanted to just make the point because industry is not represented in our discussions here that industry suffers from the lack of evidence too. And obviously the extrapolation of that is they could be an important partner for us in this area. So we have a working group in this IOM round table that is focused on generating a slightly different kind of evidence that enables drug discovery. So how do you move from a gene signal to full understanding of mechanism that actually supports a drug development hypothesis? I would say the leaders of pharmaceutical companies are trying to pull the trigger on multimillion-dollar, hundred-million-dollar programs on a very limited set of evidentiary of mechanistic data that supports their hypothesis. So I just wanted to make sure that we capture that important stakeholder community in our thinking about evidence and there are a number of other thoughts here on this slide I won't go through. So I've highlighted here some potential synergies across the programs, the ability to generate a common measures platform across the programs that are making these measures to create an implementation commons that really gives us all the lessons learned and allows the next generation of studies to clearly benefit from what has been done in the past. Thinking about evidence databases, which I think we've sort of thought about conceptually, but maybe they need to actually really exist. Thinking about joint publications again across the program has a theme that I'm obviously going to make over and over again and bringing in the broader stakeholder community like we're doing today. In terms of training opportunities, I don't know if there's an actual place to go to get skills and evidence generation, but I would say our genomic medicine trainees, particularly in the translational space, need to really understand the analytic validity, clinical validity, clinical utility issues, and a lot of the barriers that we've highlighted not just in this meeting but in the past and really have their research agendas hopefully focused on addressing some of those challenges, which I think leads to the possibility of linking some type of fellowship to these programs. And I'm just naming a couple here, but I can imagine that genomic medicine fellows that actually have a very specific role in helping the expanding research agendas that at least I see from Ignite and I'm sure are happening in the other programs would really be a huge asset for us and a tremendous opportunity for them arguably. It's great to have Pierre here and think about other ways that forward-thinking programs like Genome Canada and this one can really collaborate and then perhaps even having the educational tools that are needed, not just for the trainees but also for the physicians in practice and even for myself I would learn a lot from many of you that I haven't yet. So that leads me to the discussion part of this. We teed up a few questions, the first one about really the alignment question around the regulators and payers and the other stakeholders. The second one is around getting all of you to help us think through about how to create the right partnerships. I'm not sure that bullet number three is one that will arrive at a conclusion for today but it would be nice to articulate some pathway to getting to thinking about how to create a framework underneath the framework of evidence that guides implementation. And then the last one which may be tied to the first which is ultimately achieving the incentives that align the patients, the providers, the health systems, the researchers, the payers and the regulators which is a daunting task but something that inevitably has to be done. So I'm going to stop there and what I thought I would do is, so we're now entering the discussion phase which means that all of you who are looking at your computers need to look up a little higher. And I mean the best part of this meeting is the things that you bring to it, certainly not me. So this part of the meeting is really where we want to hear from anybody and everybody whether you're a PI, an investigator or from one of the other ICs or other organizations. This is a really open engagement. But we have three remarkable panelists that have thought about this a lot and I should say they contributed enormously to the thoughts that I shared with you in the last 20 minutes or so. But I thought I'd start with the first question which actually was a permutation of a statement that Pierre has made several times in our discussions over the last few weeks and I thought I'd ask him if he went mind to just lead off, you know, maybe highlighting the experience in genome Canada or beyond that, the floor is yours. Well thank you very much Jeff and thanks again for inviting us to this really important meeting. We had a great discussion, a series of discussions while we were generating some of the thoughts that Jeff has described. And so I'll be talking from obviously a Canadian perspective here. And our issue was that we have a, in Canada we have a terrible track record in implementing any kind of new technologies into the healthcare system. And so when we were designing the genomics and personalized health competition that we did in partnership with the Canadian Institutes of Health Research, we very much thought of this idea of end-to-end integration. So bringing all the stakeholders together within one consortium per project to try and understand all of the potential roadblocks and actually on the slides that Jeff showed three A and B, we have all of those issues and I think any country in the world will have the same, exactly the same list, totally overlapping. So we designed the program saying, okay yes, we know the project teams have great research, fabulous clinical science going on, but please involve health economists in your deliberations. Please involve those who are going to look at the social, economic and legal aspects of what you're doing because there might be regulatory jurisdictional hurdles that you're going to come across. We want to know those up front. So we were lucky in the program that we were able to raise $150 million to run this program. It involves 17 projects that are ongoing right now, we're about halfway through. And we just decided that we would build, and I know some of you have been directly involved in this program and I'm looking at Terry and Dan and Eric, Howard, I learned today that Jonathan was a reviewer and there are probably many others who have been directly involved in this program, so thank you. But really to try and get that end-to-end kind of integration of thinking and we're just about to launch a network, so a network of these mini consortia to try and learn from each other in terms of what the roadblocks have been or are. So we have, in Canada, the health delivery is a provincial mandate and currently some of the provinces are spending about 50% of their budget on healthcare delivery and they're all saying, we can't go anymore, we cannot increase that percentile anymore. So help us. And so we're saying, well, technology in some instances could be one solution but obviously we have to make sure that we get all the evidentiary stuff that really attracts the payer to pull this technology into the system and not so much that the pushers, the scientists and clinicians are pushing something that the healthcare system say, oh, it's new technology? That's just going to be an add-on cost to our bill, no thank you. So I think that's the thing we're struggling with. I think from our perspective there are some really key projects that are already actually a lot of collaboration going on, for example in the rare disease space and Heidi I know has a good few collaborations with Canada on this. But that's a topic whereby it's a great model for what's going on in personalized medicine or precision medicine in general, right? And the issue there is we're already bringing value to patients and families with rare diseases through just stopping their diagnostic odyssey and we've done that with over 200 families in Canada now but all of that impact on patients has been through research dollars and health care delivery dollars. And we need to understand that interface much more clearly going forward. So there's a few comments from me, I hope I didn't go on too much and thank you again, Geoff for inviting us. No, thanks Pierre. I don't know if Jonathan or wherever you need, either of you want to make copies do. Okay, go for it. So I'll just add a couple of things I completely agree with what Pierre has said from some of the experience that ARC has had in this area. I think a couple of things that we should look at is how do we improve our research efficiency? Although NIH at least from ARC's viewpoint has a tremendous budget but it really can't do everything itself as Eric pointed out to us earlier. We need to make sure that the research we fund is not always thought of as one of research studies that the research funding agencies have to shoulder the burden all the time. So the question becomes how do we leverage the ongoing transformation of health care delivery and see how it benefits the research enterprise? One of the things that we learned is clinicians are not researched as one can imagine and are not interested in research per se but if they are getting information from the data systems to improve the quality of care they don't mind it being used for research given proper safeguards. So the whole bidirectionality of information proved to be critical and that also goes into the value proposition. The other thing we should think about is sustainability and engaging maybe pairs early on. I had some experience with the CMS coverage of the evidence development but I think there are several limitations here so can we go and improve on this notion so that when we design the research it's done with the end user in mind? I think those would be important points. The other thing that I would just add in terms of thinking about the way that we frame all the discussions at the meeting is to, you know, going back to that ACCE framework the central circle in the middle specified disorder and setting and I think that all of the evidence is really needing to be thought about in the context of what is the question that we're asking. You know, I would ask whether genomic medicine necessarily always means whole genome sequencing medicine or whether genomic medicine is many different tests that we incorporate and so thinking about the context in which we would use those tests and what is the purpose of that testing, whether it's from a prenatal, newborn, diagnostic, predictive, pharmacogenomic risk prediction. All of those are very different settings. They require different types of evidence. They have different stakes in terms of how we use that information clinically and so I think I would like to think about framing a lot of the questions about evidence and genomic medicine really around the context.