 Thank you very much. I'm appreciative of the opportunity to comment. I also appreciate Maureen and Wendy having put their slides together well in advance. I put my couple of slides together this afternoon as we were listening to them. And so these are just a couple of the points that I wanted to emphasize. And I think that we all know in our own minds what LC refers to. There's actually been relatively little work in Emerge on the legal issues. Very little comment about it today. Some of us may appreciate that, but others probably see major opportunities going forward. And so I would just highlight the fact that that has been a relative desert. I also have always felt that economics belongs in LC, not just because it begins with E, but that it is a science, albeit the dismal science. But nonetheless, there is room for much formal investigation as to how economic issues impact virtually everything that we've been discussing and will discuss today. We need to involve the stakeholders, obviously. Payers and administrators have been mentioned. But even beyond that, I think they will respond best to accurate, valid data that the Emerge network itself could begin to generate formally. And so that would be one important, I think, recommendation going forward. There was an RFA not too long ago that began to address some of these issues, but there's clearly much more that could be done. As Eric Green emphasized this morning, there are a number of consortia that are addressing a number of aspects of genomic medicine. And most of them involve an LC component by necessity. Certainly point to the CSER projects and Ignite. And of course, there are the LC CSERS that have been in existence for going on a decade now. Many of us have been working in these areas independent of Emerge. And I think it would be very useful if there were some mechanism for bringing those of us working on LC issues together to discuss what we've all been finding. Whether this is the annual gathering of the CSERS or some other forum, it might be useful to ensure that we're not reinventing the wheel, as it were, to have dedicated discussions, opportunities for discussion around some of these issues. I noted in both Maureen's and Wendy's comments and in the title of this particular session that the notion of education and governance were subsumed by LC. I think that's fine. We've certainly heard some ideas about who to educate and about what. IRBs are the low-hanging fruit I would maintain. And certainly providers are important. And here the ISCC effort, a number of people on this webinar are involved in ISCC, is another forum that is actively engaged. And certainly anything that's being done through Emerge should make use of what ISCC and other groups are contemplating and testing in the realm of education. And I think NHGRI generally needs to be applauded for their work in public education, in particular the Smithsonian exhibition. But there's clearly more that needs to be done in this regard. Can we have the final slide, please? So one issue that hasn't been discussed but I think falls into the general realm of LC-related activities is the generation of policy. And we have to ask ourselves how important policy is for effective implementation of any of the recommendations coming out of Emerge or any of the other genomic medicine consortia that are actively engaged at this point in time. I would suggest that policy can be very helpful. It can also be a flash point as the ACMG recommendations on return, mandatory return of incidental findings emphasizes. But nonetheless, I think we need to consider whether developing or at least recommending policy is an important activity that could be done by the Emerge consortium. A particular area and I'll end on this note of interest to me is a whole notion of the duty to re-contact when we're smarter. Clearly, interpretations change. Heidi has presented an example of how from the laboratory perspective this can be implemented. But I'm also concerned from a number of other perspectives, particularly the clinicians perspective of what are our responsibilities to get back to patients when we do change the interpretation of something we counseled them about at some point in the past. So those are my thoughts. I'm very interested in what our other panelists, Laura Besko, Ingrid Holm, Kathy McCarty and Tracy Trotter, most of whom have identified themselves as being online have to contribute. So I'll first ask Laura to comment. Great. This is Laura Besko at Duke. Hopefully you all can hear me. I'm an associate professor and the Duke Clinical Research Institute, and I have had the pleasure of actually working as sort of an external person with a couple of different of the LC projects in Emerge and have found those both fun and beneficial. You know, a combination that I very much appreciate. So it's been great and a lot of good work being done that I look forward to continuing. I'm also a member of the Regulatory and Ethics Corps of the NIH Healthcare Systems Research Collaboratory, and I was quite struck this morning with the discussion about EHR phenotyping and, you know, moving into implementation and sort of the area between quality improvement and research and so on. Just the real synergies between Emerge and the NIH Collaboratory that I'm really wondering if there are ways to more explicitly collaborate in some way. The NIH Collaboratory already has relationships, for example, with PCORI, certain ways that we interact with CTSA on some projects. And so it really, I think there could be some really good interactions with Emerge as well. Many of the comments I had an opportunity to see the slides, and so many of the comments I had were already incorporated, and so I'll just make a few additional comments. I think that the sort of LC component of Emerge has really done such a fantastic job in terms of incorporating an ethics perspective throughout a lot of its different teams and has done a lot of really good work on soliciting stakeholder input, whether that's through descriptive sort of research, community consultation, and so on. And so those are very important foundational efforts. And I think in Emerge III, what I would personally love to see is very focused effort on building on that sort of exploratory descriptive work to actually develop and test interventions designed to address some of these ethical issues. So for example, in the area of consent, obviously a lot of work in the model language, and that's continuing, it sounds like with the pediatric language, and I think that's fantastic language. But I think it would also be interesting to actually then go ahead and test that language, try to find out about the uptake with that language, see how participants react to it, find ways to assess comprehension, and improve comprehension. So building on that initial work to actually test and see what sort of an effect in ways that things can be improved. On the regulatory front, my ears perked up this morning with the talk of the Emerge sites coming together on the data sharing agreement. And I really wondered if there was a way that Emerge could also demonstrate that sort of interaction on the IRB front and find ways, whether it's a central IRB model or some collaborative IRB model, to really demonstrate sort of a streamlined way that different institutions could work together and sort of get through some of those IRB issues. That would be great to see. With the EHR phenotyping, I think that that presents a lot of very interesting ethical issues. One that I think was alluded to this morning a bit was the idea of if you find folks with a particular genotype of interest, possibly a rare gene variant, that the idea of being able to contact them and do more in-depth phenotyping. That is a topic that I personally have done some work on. David Goldstein and I had a paper a couple of years ago where we talked about what happened at Duke when we tried that, erased some interesting issues. And so to me, that's an example of where some descriptive work has been done. And so finding out, exploring that in more robust detail, including coming up with interventions on good ways to do that sort of thing and being able to really make use of the combination of EHR data as well as genomic data. So I'll just conclude by saying I really would hope that eMERGE could become a showcase of robust empirical bioethics, similar to the way that eMERGE has been really a great resource in terms of creating large sample sizes across the different sites. I think that same benefit could apply to bioethics research. I think some of the limitations in some work today has been things like small sample sizes and so on. And so with the expertise that eMERGE has as well as being able to work collaboratively across all the sites, I think that really gathering data to address some of these ethical issues is an important contribution that eMERGE could make. So I'll stop there. Thank you very much. Is Ingrid Holm on the conference? Physically. Yes. So yeah, this is Ingrid. So my comment I think part of it is that Maureen and I were working together and kind of put some of this also talk together. And I really appreciate everybody's input, particularly that Reid and Laura were perspective on it. I think from my perspective I feel like that what our group can do at this point kind of going forward in eMERGE is probably on the education piece of it. And I think the idea of educating IRBs in particular is crucial. And just as an example, Maureen and I are now dealing with developing the survey across all the sites and we've had IRB issues already across the sites and it's been rather challenging. So I think and I sit on my IRB and I'm always educated. So I think that one of the things that we can do as a group is try to educate IRBs. I also think that eMERGE too is kind of all of has been really bad kind of genomic medicine and integrating genomics into the EHR into clinical care. And I think to me like going the next step in terms of how physicians being involved in how that information is used and particularly from kind of a patient perspective and using patient portal and just I guess kind of in general how and this is one of the comments that Maureen made too is that how people are taking this information, how they use it and kind of how they feel about it. It seems to me that that's kind of going the next step beyond getting the medical record and looking at real kind of clinical support or decision making in terms of PGX going kind of far further on with that. And a number of people have mentioned the fact that I think these patient portals may give us the opportunity to kind of do that. And I really like what Laura was saying about this idea of not just kind of developing policies and developing consensus really testing them. So for example testing consent forms and seeing how kind of getting feedback from participants in terms of using things like consent forms. So I think as we are in right now in our third group getting feedback from participants about how they feel about broad consent, we can do I think an emergency get more information back from participants about how they view genomic data that they're getting, how they view the consent forms that we're talking. So really trying to get out in the community and looking at the kind of patient stakeholders or participant stakeholders I think would be a kind of a good next step from our perspective for eMERGE. Thank you very much. Kathy McCarty? Yes, I'm here as well and some of our previous speakers I had sent my comments to so many of them have already been addressed as well. So Laura mentioned potential collaborations with the Collaboratory and how the Collaboratory has ensured collaborations with the Corey. And it strikes me that so much of what we've done already is very well aligned with what PCORI is doing, their initiatives and the methodology around including and engaging our patients as truly as partners. And what can we do to more formally work with PCORI? I think it would be something to explore. We've talked in the past about how do we engage payers. That's very important. We're doing an AMD, make the degeneration genomic medicine pilot here at Ascensia. And one of our questions as we're following up with participants is to see whether or not they would be willing to pay for genetic costs out of pocket. And that's been interesting. Some of them yes and varying costs but almost all of them would want to talk to their payers about whether or not they would cover genetic tests before having them. So how do we try to actively engage them? And I know for myself I've shown that challenging when I was at Marshall Clinic in Wisconsin. It contacted the statewide organization of HMOs and just found it challenging for them to even identify somebody to come and sit on a panel with us. So we're just concerned about doing that. How can we try to engage them as part of the community to engage? IRB education is also really important. So I have a new hat here at Ascensia. I've taken on the role of the IRB chair so I can see some little bit on that behalf. And none of the sites within the eMERGE network are part of the HMO research network. And there's currently a document going around about feeding and having essential IRBs in the HMO and how much it's supposed to be. And I think by the end of this month it will be interesting to see what we get because we have almost always have some changes to that document every time it goes through any of the legal departments. And so we need to remember to engage our legal counsel early on in the process. I don't want to look at that. And then Barbara Keenig has been our voice all along and I want to bring her voice back again. When we're educating the IRB, how do we educate them about the importance of the engagement of the community and incorporating their feedback into our IRB applications? Really important. So that we can develop, potentially have some best practices for IRBs. I think it's an absolutely great contribution to the wider community. Thank you very much. And finally, Tracy Trotter. Great. Thank you. I think about a month ago I was traded to the panel 7 pediatric team for an expert to be named later. But I saw a couple of comments. Because you called on me. One is the overall comment as a pediatrician is all the LC and consent issues of course become more complex when dealing with pediatric patients and possibly even more complex when we get into newborns and sequencing and things like that. So I look forward to the model language. I think it's important that we get as many people on the same train as we can on this thing. And I look forward to somebody coming up with good answers about updating new information and updating information as it changes over the lifespan because when your patient's four months old, when the information's gathered, you've got a lot of time to think about that and how to do it correctly and how to do it well. And I'll save the rest for my own panel. Thank you. Great. So I think it's time now to open it up to general discussion. Mark Williams again. So I just wanted to come back to the E that you posed, Reid. For those of you that were paying attention to the sidebar discussion earlier, there was a fair amount of back and forth about the role of adding a component of Emerge 3 to look at economic or value-based propositions and proposals and how to study that. Clearly that would be something that is of interest related to implementation. And we do have precedent in that the newborn sequencing RFA that was referenced did explicitly allow for an economic study to be a component of Subproject 3, which was the LC. And to my knowledge, I was the first one that specifically called that out. Clearly, there would be methodologic issues that would arise much as had been encountered related to variability in IRB, variability in phenotyping, et cetera. But in some sense, if we could develop a model across the network of doing this type of economic analysis, not only would that be valuable for the implementation of public medicine, but I think it would provide some methods for shared economic research that would be broadly applicable. So I would endorse a consideration of an economic arm to Emerge 3. Can I pipe in also on reason for judging at Susan Wolf of a more robust legal component? I think that's really critical because successful integration of genomics into clinical care requires supportive law. And conversely, if institutional leaders perceive or misperceive a legal barrier, that may stop the train in its tracks. So if you were thinking of a short list of high-impact legal flow size, it would include the clear issue we've already discussed, the HIPAA implications, because as soon as you talk about reaching out to family and even posthumous, use of posthumous information, HIPAA doesn't die when a person dies, and so that's a big set of problems. The recontact issue that was already raised, return of results itself raises a lot of legal issues, including rights to refuse, and whether you do need consent for incorporation to the electronic health record. I loved Heidi's idea. I think it was Heidi about mapping responsibilities within the EHR environment that has a lot of legal pieces to it, including from the biorepository. And lastly, you know, the advance notice of proposed rulemaking is a proposed regulatory change. So it's both ethics and law. So there's a great and really important list, I think, on the legal front. Thanks, Alan. I guess I'll echo both what Mark and Alan both said, that I think both the economics and the legal pieces are important, and I just expand on the economic piece that people have proposed population genetics and genetic screening for BRCA mutations and for limb syndrome mutations. Israel is already doing population screening for BRCA mutations, and since we have mostly population-based samples, we could really speak to the economics of screening and down-the-road costs. The other interesting aspect of an economic study is that genomic data is different from other health care data in that it is persistent. So if you think about an initial investment in genotyping, that information can then be reused over an extensive period of time based on different clinical contexts, and that's a very different type of economic assessment of an investment. I'm just not aware of anything else in health care that has that type of persistence, which I think would be very interesting to study. On the IRB front, one interesting connection is that among the NIH BD2K workshops was one that I was able to co-chair in September on the research uses of clinical data, and for the entire workshop, Jerry Manikoff, who was the director of OHRP, was present and engaged, and one of the observations he made was that delegation models are very familiar, and IRB uses a central IRB or delegates to some other member of a consortium, and his personal view was that resulted in no education. The IRB was no better for the next project they reviewed if they delegated, and he was very interested in OHRP actually supporting and engaging in some experiments of IRB consultation and education with one another that might improve the performance of the network in a persistent way. So that was OHRP inviting, I thought, organizations like eMERGE to say, let's do some interesting experiments of IRB that are not just like business as usual of delegation or central IRBs. One related aspect is the rather, I think, overly legalistic format of most consent documents, and from my point of view, they impede the consent process, and I know Mike Gaziano is out there someplace, but a project that he did before the MVP at the VA was instituted or was to pilot a brochure with information about the study versus the formal consent document that had the same information, and participants strongly preferred the brochure, which probably none of us would be surprised at, but I'm unaware of many studies of that nature, and so I think as part of educating the IRB, I don't think we're going to change CFR 36 quickly, but we might work towards a better, getting a document that was actually more user-friendly and actually communicate with what we wanted. Yeah, we should just... I wanted to make a couple of quick comments here. In large measure to support much of what was said by the panel, but people have called for a formal evaluation of the efficacy and impact of the recommendations that are coming out of the work groups, and I very much support that notion, but those types of studies, of course, are fairly expensive, so I'm wondering whether the Emerge Network can support that type of outcome study or whether this is something we need to look to with NCRI for an RFA targeted to a particular domain in that respect. And then let me ask the second question quickly, and then I'll get off. There's the people who talked, and they very much support the notion of IRBs as key stakeholders for the network here, so I think that's just a wonderful idea. So has there been any collaboration between Emerge Institution, IRBs, and support of the network today? I mean, there's been a lot of consultation at the individual level, and also a large study across IRBs in the country that several of the sites collaborated on with the GRIP consortia, so while you're suggesting the incorporation of IRBs along with some of our studies, I don't think that has been done, but certainly there has been consultation and work with IRBs in the network at most of the sites individually. I think the other elephants in the room in this area is the CTSA consortium. We are all affiliated with CTSAs, and one of the three must-dos this year for the CTSA consortium is widespread IRB reliance agreements. So that is going to completely change the IRB landscape across the CTSA for the next 12 months. Jeff, I wonder, this is Harry. I wasn't quite sure you said that would Emerge be able to support that kind of outcome study in terms of formal evaluations of many of these recommendations. It sounds like a lot of these recommendations are things like you can study the impact or the outcome or the attitudes or that sort of thing. It's not like a huge outcome study to me, but maybe I misunderstood what you were suggesting. No, I think that's exactly what I'm suggesting and as sorts come forward, but what they think would be more appropriate consent language and formats for conducting this kind of research. I think those sorts of recommendations are quite welcome, but that's different than actually evaluating their impact in terms of people's understanding of key elements of the consent process and their response to the information, et cetera. It may well be that that sort of thing, just in the flow of research participants through Emerge, you can look at some of those outcomes relatively inexpensively. What I know is we try to set up studies to formally evaluate different approaches to consent. It turns out to be complicated in a relatively expensive process in certain circumstances. So this question was, can you adequately build on the Emerge network with some supplemental funds to get that kind of work done or do you really need a separate allocation of funds to do that in a rigorous way? Right, and I guess the how we go about this is maybe a little bit down the road. We're kind of trying to define the what at the meeting today, but I think just to address the how or the who, one might also say not all of these questions are uniquely addressable in Emerge and so there might be other consortia groups that we could look at to answer them. But I think rather than getting into that, let's try and focus on the what and I think we do have to move on. Yeah, we're right on time. But Terry, I think the how is something we should be thinking about and hence my suggestion that all these different groups that are thinking about the same things need to get together and chat. Excellent point. Okay, and an excellent discussion on a topic which segues directly to actually we have another presenter in the room here in Bethesda, so Istakar will be the Emerge presenter for the topic of return of results.