 So the next section is admittedly experimental. I haven't really tried to do this in a workshop before. But it is the time we've heard a lot of great ideas, very expansive. But now we have to try to start making sense of it. By we, I mean we program folk at NHGRI. And I want to bring you into our world a little bit. We, these are, these set of discussions is designed to bring out some of the issues that I'm anticipating will come up as we internally discuss how to follow up from this workshop, how to take it and take the results and the recommendations and make them into programs. So at the beginning of the workshop, I de-mapped our programs from the topics. And now we have topics with some changes and nuance and recommendations. We're going to try to remap them. But we're not asking you here to converge on specific programs. I'm trying here to tease out some important elements of programs, some decision points for each of the main topics that we've discussed today. And to try to get some pros and cons. Because as I said at the beginning, the pros and cons are many instances more useful to us than you should do it this way, kind of recommendations. Because at the end of the day, we're going to hear lots of conflicting stuff about you should do it this way versus this way, or if you go this way, then you won't be able to do this. With the rationales, then we can try to plot a path of what we need to encourage and what we should try to avoid. And that's going to ultimately be more helpful. And again, I don't know what's going to work if it's going to work. So here is the idealized scheme for this. You sort of start with roughly for each of these topics, roughly what would be the right organization. You could start with the current organization as a default. You can start with a generality based on what you heard earlier this morning. And then list a set of topics or features that could be alternative. So centralized versus dispersed. Or as Bill brought up, siloed versus unified. And there actually are reasons we silo things, organizationally. Sometimes we have to do that. And again, these things are maybe a lot of them are false dichotomies, as was raised a number of times during this. But it's the features that we want to keep from each of those that we should focus on here. And I'm going to do the first one with Len. So maybe you get a better idea. They won't be exactly the same for each of these, for each of the major headings. And at the end of this, we'll try to wrap it up. Rex and I will be up here trying to wrap up across all the different topics. And by the end of that discussion, I hope we have a good feeling of what people want and don't want to see in programs. And more importantly, why? And again, I hope the first example that Len and I will go through will give people enough information to suggest some of these instead of us just coming up with them on the fly. Because these are coming up on the fly. I haven't given my colleagues any more briefing than I just gave you. So, Len, do you want to come on up, or would you rather circulate in the audience? And if I could ask Lou to keep time and let us know when it's time to wrap up after about 20 minutes or something like that. So, yeah, for the whole thing, how much time did I give on the agenda for each element? Half hour. So let me know after about 20 minutes of discussion. So element one is the general architecture of disease topic. It has to encompass Common and Mendelian and all the other things that we're talked about. And I think what we heard this morning, vastly oversimplifying it from the breakout run report, was there's still a need for discovery at scale. We're just figuring out how to do this. Keep going with paradigmatic examples. That's a two sentence summary, three sentence summary. And here are some, just launch right into the topics. You said, Eric, that we need to consider everything across the spectrum. But I heard David Altshuler say last night that the current organization we have. We actually have a current organization. We have big centers for common disease and other stuff. And then we have very pretty focused Mendelian centers. And I heard David say, well, don't do it that way, because it doesn't go with the way the science is going. And on the other side, I'm worried. I'm very worried that if there isn't some way to focus on them, some organizational way to focus on the Mendelians, that we'll actually lose focus on those. So I'd like to hear some. So that's one potential pro and con. And I'm going to ask David to start off with why did you make that remark? And how would you suggest addressing it? Well, the main point, the only comment I want to make about what you said is I wasn't making an organizational point as much as a scientific point. So I wasn't necessarily saying what the form should be of grants or centers not specializing as much as making the point that scientifically in terms of integrating data, in terms of having the thought process, we don't want to make a fetish out of what the prior assumption was about the architecture. So I haven't been deeply involved enough in the Mendelian centers or there might be groups of particular expertise who choose to focus on that, and that might be a very good thing. As long as we don't end up promoting that they should be held separate, as long as we don't silo the data, as long as we don't make arguments on the assumption that they're different as opposed to the data drive it. So I don't mean to undermine your provocative statement, but. No, it's okay. That's right, Bob. Just to follow up on that, it seems to me that for the Mendelian in particular, that involves a set of particular, I don't know, expertise and links reaching out to the community and so forth, and so to pile that in with everything else might be a mistake. You need some of that particular expertise. Yeah, I also want to support that. I was maybe dubious about the idea of creating large scale centers, Mendelian centers, clinical projects when that first started. I've come to think this is a really good idea. The thing that supports, the thing that NHGRI does very well is it invests in creating the infrastructure that's really excellent around the problem. And sometimes that's the infrastructure about large scale data production or analytical design or project design or analysis. Seeing the Mendelian centers really focus on a problem, it's very clear that's a different problem in certain respects and the technical under the hood respects about how to manage that. I agree with David Altru, there's got to be a lot of free flow of data. There are going to be Mendelian things cropping up in common and the Mendelians will be focusing on common traits in overlapping ways. But actually developing deep excellence is the one thing that doesn't get done either in the commercial sector or gets done in lots of little grants. Critical mass has benefits and I think we're seeing it in the Mendelian centers, we're seeing it in the clinical and so I think what distinguishes NHGRI is the ability to get critical mass and excellence at things that most other people, most other institutes, most other companies kind of ignore as requiring of excellence. So I kind of think that that's actually an important element. Evan. I mean I guess one comment regarding this Mendelian versus common, I would suggest or like to argue that maybe the definition of Mendelian should be expanded a little bit. I mean there's a lot of success being achieved in things like autism and autism doesn't necessarily fit nicely into one of these bins in terms of Mendelian or common. And we've talked to people that run Mendelian centers and they have a lot of very strict rules in terms of what types of families can be put into them. And I think there's a lot of bang for the buck that could be leveraged if focused efforts in specific diseases where rare variants as well as common variants are playing an important role. I was curious, is there a need for better integration between the common architecture and Mendelian or right now it's pretty siloed. So is there? I think integration would be a really good thing and there is this overlap around things but just munging the distinction wouldn't be a good thing. So one way to follow up on Eric's comment and Bob and I were just talking here is I think the essence of Eric's comment which I do agree with is specialization, expertise, critical mass, form following function are very good things. Whether an HGRI should dictate that the one such split and I'm not saying it was the wrong decision past is Mendelian not Mendelian. It could be there are other forms, there's some groups, the kind of stuff Evan and Jim talk about so passionately about structural variants. Maybe there's some specialized thing around genome assembly and structural variants doesn't matter whether it's a common or rare disease. There could be multiple types of specializations. So the idea that you want to integrate and not silo or predetermine that specialization doesn't mean everybody should be a generalist. Yeah. With that coming on the specific organizational structure I want to say that over the last three or four years one of the most satisfying and productive things I've witnessed is the contact between genomics expertise and the clinical expertise and bridging the full span there has been enormously productive. So my hope is that whatever the organizational structure is that that is a central focus. Thanks. I want to add to that which is our comment on that which is that's actually the area within the current program that seems to be seems to have the most separate needs and administratively tends to be more and more separate. And I'm I always worry about how to get the best cross-fertilization. So if you if people have ideas yeah. Just to comment specifically on that I think you know one of the goals in the current iteration was to try to get more disseminated activity around the country in hopes that that would nucleate additional activities. In our case certainly the Mendelian Center has catalyzed clinical sequencing. We did 500 clinical cases last year based on the scale that was achieved through the Mendelian Center without NHGRI putting in any additional funds for development of the clinical work. That's the kind of leveraging that I think you can achieve by getting some level of dissemination of high quality work being done. Yes, Ewan. I mean I'm just wondering whether you shouldn't let this be a bottom up process with some framework for how for the criteria but you just let people try and choose the area the kind of scope that they want to fit into and then think more about the criteria about how to judge and rank them. I think there's something incredibly I mean everybody's commented on this but there's something incredibly artificial about trying to draw the boundary lines here and I just feel that investigators will know better where they want to draw the boundary lines and it will fit better to their local context. I know that puts that kind of puns this problem onto the review committee but at least the review committee there will see concrete proposals in front of them. So I think we've been seniorly challenged by the idea of sending vague things to review committees. I like, you know, in theory I like it in practice. For example, we've sent up centers without any work plans. We've been unclear whether the center is responsible for coming up with the work plan or whether it's gonna be given a work plan. I think actually the heart of this is if we're looking for centers that are defining the work plans and we're coming with these ideas, these samples, whatever, that's one thing. If we're saying they're a central resource and NHGRI is defining the projects and allocating and some of that is coming up along here and some of the questions, that's another but I think punting everything to a review committee that meets for an afternoon and isn't deeply invested in constructing a program that hasn't even spent as much time as we've spent in two days has its problems also. So it might be a good thing for NHGRI. Now, that's not to say it's gotta fit in fixed boxes. Those boxes can be more flexible but I think leaving all degrees of freedom to the randomness of a review committee has challenges too. Don't get me wrong, I'm not suggesting that. And I think this is the art of crafting good RFAs and doing this right. Which bits do you constrain and which bits do you leave open? And the thing that I think is a mistake to constrain here is this place in this spectrum between Mendelian through to Common. Just from the conversation, it seems like that's a mistake. Now, I might be wrong. I mean, I'm not saying I've got incredible insight into this thing. But from the conversation so far, that seems to be like a place where the investigators understand better than trying to set it out ahead of time. But of course, a challenge then is you toss it to review committee and the review committee likes all grants of type B and doesn't like the grants of type A and NHGRI doesn't end up with, say, Mendelian's covered. So there is some art to figuring out how you get the portfolio you'd like to get out also. I think that can be constructed in my understanding. We're just putting the design criteria out there. The portfolio is rounded. I mean, that's the criteria is that you have a rounded portfolio. Good. Fair enough. And I think that leads naturally, as Eric said, to this project selection question, which I have to say is an area that I don't think we've got completely right. I don't think I worry about, I worry about how to bring the community more into some of these decisions about doing this. I worry about fairness and rigor and review issues. All these comments are comments that have been brought to my attention. So I just put this as a sort of three kinds of elements. There's investigator initiated versus NHGRI initiated versus community initiated. How do we balance that? What's the right thing to do? Or even those of you who are familiar with the current way we bring things on, where have your, especially outside the centers, where have you noticed things that you didn't think were right and what kind of things would you consider to do them better? David? Well, one thing that I think has been very productive just to share the example in type two diabetes that I think is a good model. We talked in Eric Green talked about funding partnerships is NIDDK started a type two diabetes genetics initiative and put up money. But in the end, then there was a partnership with NHGRI and actually not only two different centers if I understand correctly, different work went on. In fact, some of the work was outsourced to complete genomics. But it ended up being that coming together, there was a project that had funding from multiple sources and was a better project. And now there's also a lot of energy that's gone into aggregating the data and making it useful. And now pharma has come in through the accelerated medicines partnership. That's an example where NHGRI's funding was leveraged but clearly played a catalytic role in important moments. I'd like personally to see not that NHGRI do this alone which is a failure, I think will be a failure, nor that all the institutes go off and do it and companies go off and do it in a silent way but somehow that kind of model where there's also diversity of designs and many people looking at the data as opposed to monolithic sort of coordinated or not even coordinated but there's this idea that I learned about David Housler taught me this phrase define the interface compete on implementation is a sort of software thing. We want to do is make sure we have multiple people contributing data and resources but we make sure it will come together in some way or be useful in some way together as opposed to being siloed. I think the trick of that is something that NHGRI has done well, maybe in cancer, maybe in diabetes, maybe there's four or five others I don't know about but this should be modeled. Dave. What's the rough breakdown right now in these categories in terms of projects funded by NHGRI? Yeah, I would say that about, well between 70, I don't know exactly, I would have to guess between 70 and 90% of the funds are actually going to big projects that are partnerships of the kind that David just mentioned over the last four years that includes TCGA and more recently ADSP, type 2D genes and a couple of others. Yeah, you win. So I appreciate the headache and the problem here and just to make sure there's a lot of practicalities of sample flow, sample quality, consents, if you add recall as being a desirable thing as well that complicates it even more and therefore the practical business and then if you want to add in this scale that it's got to be done at high scale, I think you probably come down to quite a small list already, I may be wrong and I think the experience has been again and again that the system that's going to consume those samples need to have a very good engagement process with the system that has those samples and it has to be a two-way process and one can't dictate kind of forced marriages in this, it's not a productive process. So that I think leads to the idea that there has to be quite a managed process that involves the investigators that are working at scale that work with the people that have the samples and one has to probably give those investigators who work at scale quite a lot of freedom to make sure that they will deliver on that whilst also holding their feet to the far to deliver on that scale but I think this is a very, very complicated area to execute and there's quite a lot of experience about how to handle this, so I think I would be interested in other people's views about this because I just don't think this is an easy thing to get balance on compared to say those model organism white papers which were once you decided scientifically that this was a good thing to do you just had to find one person with one animal but sometimes a few animals. It was a completely different problem in that era to this sample scale out era. So I really think we should explore this because this is going to be a big problem at working at scale, keeping the communities alongside, keeping the other ICs involved but also delivering on it. I mean that matrix is what it feels like and one of those things has got to go. You'd like all of them but I feel like one of them's got to go. Eric. But I guess maybe I'm naive but I think it's a place that we've done well the last several years of integrating ourselves very well into these disease oriented communities. It's no longer, it's the clinical community and the sequencing community and we send emails back and forth. In the case of diabetes that was already mentioned heart disease, the Mendelian centers, I think the genomics community is very much integrated inside that biomedical slash clinical domain and I think if we're going to achieve the goals that you and talked about we need to continue that. At the end in order to make sure that the large numbers of appropriately consented high quality samples are available and that information will be made available largely in a very broad context. I think having that complete integration and not just keeping the community alongside but actually that full integration I think is the way to go and I think we're achieving that one step at a time. Lou, how much time do I have left? Okay, we've got time. No, I just didn't, I didn't know. Okay. I wanna spend a little bit of time on the last two because they've come up directly and this is almost a prioritization or a timing or a staging. And we heard both about the merits of discovery sort of bottom up and we heard about the virtuous cycle and the miracle database that's gonna capture all the variant information from all the clinical efforts all over the world and feed into discovery. What's the, we're gonna have to make some choices. We don't have infinite resources. Even though we may have really great partners in this what's kind of the right allocation? Is there any way to talk about the right allocation or is it only, can we only say yes we have to do both of these? Is there some way to get a better refinement on where and sort of staging how we need to invest more and especially in the second one of these? Cause I'm... Well if we have a finite amount of funds and we can invest it in the discovery in the mode that we've been doing now or we can invest it in trying to recapture clinical data so that we can get enough to make discoveries or have a resource for discovery 10 years from now how do we decide between those? Or should we decide between those? I think the point was made well before that we shouldn't be paying for clinical service and the act of integrating into clinical practice and completing the virtuous cycle and having prepaid research subjects is really what we're trying to achieve. So maybe it's not such a dichotomous choice and that the integration into the clinical activity is not going to drain the budget. Okay. I think if we don't make discoveries we're gonna lose the support of the public and the rest of the NIH community. If we make discoveries, past year there's been a number of interesting discoveries of loss of function variants that are protective and the drug companies are all excited and they're on them. If there's a steady flow of discoveries the rest will follow including funding and other people following us. If we say sit tight, we're gonna assemble resources that over time will let us discover. The argument I fear will wear thin. So that doesn't say the portfolio involves none of that but I think we better deliver on discoveries. And if the next three or four or five years have a lot of the architecture and pathways of common diseases and hundreds of more Mendelian diseases coming out will be fine. And then the whole clinical world that's doing sequencing will help shape that and that will drive the generation after. But we're gonna be held as I think we've been held in every period of genomic work to having fruitful products in the here and now. Howard. So I think they're not split. So I think we still need for discovery we have to have large scale. If we're gonna be looking at these disease issues we're gonna need to have large scale activities still in place. That's for sure. I mean just the sample sizes alone are gonna require some large scale activities. But if we're gonna grow the clinical side as was discussed before I think there's a limited number of things that really have to be done to help drive value into this. As values driven more into it other people are gonna pay for this. The hospital systems, the insurers are gonna pay around that. So what we have to do is build the base that helps that happen but not at the expense of discovery. I think if we go away from discovery we're in trouble but if we completely ignore and hope somebody else is gonna figure out how to do this clinically we will also slow down the process of developing large sample sets in the clinical environment. So I think CSER and these other programs are at the beginning but I think what the clinical group was talking about although I share Eric's concern our job isn't to implement across the board but I think we are in a position to help establish baselines of how you do it and then that enables others to then get on board in the clinical arena. Yes. Yeah, along the lines of public support I think a relatively small investment in education and in making the public aware of such discoveries which I agree do need to happen might go a long ways for sustainability and for further investment. So I just don't see that in the bullets there but I think it's an important portion. Yeah and there the contrast would be with building the foundational part the community resource part versus making the discoveries. I think that's another kind of balance we've got a strike. So yeah. Yes. Back to Eric's point. I can see over the next few years discovery is gonna be still a fairly narrow channel and the NHGRI can have a leading role in figuring out how to make those discoveries but in a few years I would suspect that there's gonna be a torrent of sequence coming from a variety of places and hopefully and the NH just because of the budget of the NHGRI they're not going to be a major driver anymore. And one way to continue to be a major driver in that is this integration of the data because all of these sources are not going to talk to one another well and NHGRI can play a major role in making sure that that happens and make it a much more powerful resource for everybody and lead in the discovery in that sense. So NHGRI has to its credit and I think to a lot of productivity been a fairly top down center or institute and I think that harnessing the community maybe by shifting that just a little bit will take some of the difficulty of making these top down decisions which are at some level kind of impossible and perhaps some consideration to increased grantees, right? Increased individual investigator type driven projects not in an extreme sense because again NHGRI has been very successful with the top down approach but that might allow harnessing, things are mature enough in many ways where one could imagine effectively again harnessing that thinking of the community in ways that we haven't been able to as well. I just want to echo Bob's point because if you just think a little bit about the projections that Illumina has for the number of sequencers they expect to sell and it's somewhere on the order of dozens next year in the mid dozens maybe 40, 50, 60 of the X10s and you just sort of put that into context with what capacity we already have. It's clear that NIH is gonna get dwarfed in terms of sequencing capacity if it isn't already and so that aspect of figuring out how to get the people who've bought those machines to share the data is actually probably the single most important thing that and where the global alliance and others can come in. I think the key thing here and I think I know that Eric and others are thinking this way and Phil born at B2K but we just have to be careful. This room is an NHGRI meeting and it might be attractive to imagine that NHGRI can dictate this answer but certainly every discussion I've been in makes clear that as I said this in our breakout group I'll say it again here in the international discussions and sort of discussions with different stakeholders, clinical companies, et cetera one group that they clearly they're not sure who should do this but the one group they're clear should not do it is the United States government. Okay and so we just have to realize that like a top-down imposed and I think one of the things you hear in these discussions is many parties will say yeah yeah we're setting the standard and others will use it and I think the only way to actually do this and there are models from other fields and that's what the global alliance is modeling itself on the worldwide web consortium which has been highly successful is and I sort of threw this in out of context that was wrong time to say it but focus on the interface. You know what I'm saying don't try and determine the outcome don't tell everyone what data they should collect or how are they going to store it or implement it. This is where you get the mistake as you try and build the big database in the sky or tell everyone what technology to use figure out how the data will flow get the data flowing and we do have to think about you know there are technical aspects of that there are regulatory aspects of that and there are incentive issues and I think that when Eric Lander pointed out you know earlier we don't know what the business models are I do think that that not that any sure I are going to set them but everyone in this room or people in this room are all leaders like if there's no incentive the reason things happen often in the world and certainly in medicine is because someone reimburses them or somebody profits from them and we can be very idealistic and say the data should flow because flow is good maybe we need to create a social pressure to do that that's another incentive but we can't be naive that if we you know we have to build the pipes and we have to think about why will people actually send things through them because otherwise they'll sit idle right and the reason people share okay yeah yeah Mark Mark I I yeah I think we we've David you made that point strongly and I think a couple of others have we've yeah it's a good point Mark I just want to amplify what Bob and Carlos were saying and that I I really do think that with so much sequencing in the future you know any sure I really needs to think about things beyond sequencing and things that are going to enable the use of all that genomic sequencing better you know sort of foundational resources that you know can make all the genomic sequencing quite useful whether it's you know characterizing all the variants or interpreting them so so I want to ask in the in the last 30 seconds a specific question about especially coming from this point of view this this first breakout's point of view the bottom up balance of bottom up versus top down sort of functional analysis and for example if you had a program much like the current program now and you want it to start exploring functional validation how how much would you how much would you let them do or large centers do I'm not talking about any specific people but in a large center how useful is would it be to team it with some kind of downstream functional validation exercise and is that is that too much or is that an amount that's that's just right something I'm asked I hope you have a feel for the question so Adam so what one idea there might be that there there are some assays that have been developed already a few years ago and that could now be done at scale for example for many promoters are many transcription factors some of the things that Jay illustrated but I think in parallel you do want diversity so you want people to be developing the next round of assays and maybe the balance is that you know trying to figure out bar assays that you can do at scale for you know for hundreds of KB or megabase of sequence what our ass what their new ass is that you want to develop to add that portfolio over time is that from the perspective of the Mendelian centers this has been a key issue because so many of our collaborators we make the discovery from the genomic work and they of course then want to pursue uh... prosecute the functional validation and the pursuit of that that's quite a complicated interface for NHGRI to be intimately involved in and it's anti-leveraging right you don't know at the outset what the assays are going to be uh... and how you're what's going to be necessary to solve them it seems like a very good interface for some kind of partnership with the specific institutes that are the domains in which the discoveries are made but you would like to have a more rapid uh... turnaround with the other institutes where you say we've got this new discovery of uh... you know we can name two hundred and sixty eight uh... genes per following uh... the discussion yesterday uh... where we now know that uh... this gene is implicated in this disease and we would like to do the next steps how to manage that interface i think it's a key uh... consideration but i think it'd be very complicated for NHGRI to be in the middle of actually trying to fund each of those thanks i think that's i i want to wrap it up there because i can i just quick quick comment on that i think given the diversity of genes uh... and you don't know ahead of time what you're going to get it's just impossible to know where the boundary is going to be and how much it's going to cost it's uh... it's a decision that has to be made when you find it so this is a good time to turn it over to the two