 Okay, so the next presentation is I should probably give a little history in that I would tell you that I almost from the first moment I became director two and a half years ago or whatever consistently I get asked when I especially when I go around and travel lots and lots of questions about sort of the needs and training in genomics and that's very broad broadly talking everything from more clinical to more basic bioinformatics the whole gambit and different entry paths into this and graduate student level postdoc level clinical and so forth and so one of the things I asked to sort of take a fresh look at and we want to wait till after we were done with the strategic plan getting that out was to sort of take a fresh look at sort of the big picture view of our training portfolio and so there is a subgroup of council that we've interacted with a little bit by phone and Betty has been sort of trying to capture this and so this is this is really a presentation introduced to this and also to have some discussion about what our training portfolio looks like now and what we might want to do in thinking about it moving forward. Okay, so this presentation will involve four basic topics a summary of the small group of the council discussion a review of our training program a review of what NIH is doing and what we have identified as gap areas and future directions which is open at this point. Here we go, let's do this. So the members of the small group included Mike Banky, Carlos Bustamante, Rex Chisholm, Ross Hardison, Amy McGuire, Howard McLeod and Deidre Meldrum. The purpose of the discussion was to respond to the community's need for more training programs in light of budget constraints and Eric just told you about why he's so when he's out giving information about NHGRI the question of training also comes up and I probably at least once a month will get an inquiry for someone who wants to start a new training program to review NHGRI's current training programs and assess how they align with the strategic plan to determine what are the gap areas in NHGRI training programs and what other institutes and centers at NIH are doing in these areas and then a more open discussion to determine how to shape the future of NHGRI's training programs and how to accomplish this. So by way of background you are all familiar now with the 2011 strategic plan in which it really expands the research mission of NHGRI and as a result of that it's important to look at our training and career development mission to see how it aligns with the strategic plan. So we had a small group of council members have a teleconference in February. We identified a path forward by identifying gap areas in NHGRI's portfolio and they recommended a fact-finding effort to help refine the issues and specifically what they had asked us to do was to go out to all of the T32 program directors to ask them what are their goals and how do they see their program aligning with the strategic plan. They also wanted some information about the strengths and improvements of the program from the perspective of trainees. So the discussion with the small group they identified or they said there was expertise lacking in the workforce to take advantage of new knowledge and some of the areas they identified were statistics, bioinformatics, translation of basic genomics to the clinic, development and exploration of new technologies for early detection of disease through discovery of new diagnostics that would more effectively translate discoveries to the clinic, clinical science, healthcare economics and they also had another area because this was a small group and they thought that maybe council members might have additional areas they wanted to add. We also had some discussions about what are the appropriate training vehicles whether they are T's training grants or small efforts or whether they were career development activities. How long should the training be? Are we looking at short term, long term? Who should be trained? Are we looking at basic biologists versus clinical clinicians? There was a lot of discussion about genomic medicine and since this is an entirely new area for us especially for training, what should be the critical elements of a training program? And the big question is should NHGRI partner with the disease institutes for training in genomic medicine? There was a discussion of the continual areas of need so the idea there is where we may try to move to the right, we still need to be cognizant that there are areas that we still need attention in. Some of those ideas were that all students must be trained in bioinformatics and statistics, that basic scientists must be prepared to conduct research on the right hand side of the density plot that is the side that is moving more to the clinical applications and that individuals need to be trained to develop methods for analyzing and interpreting large data sets, that more individuals need to be trained in bioinformatics and there was a big note here that bioinformatics is an NIH-wide problem not one that is just specific for NHGRI. So now we come to the review of the NHGRI training program. This presentation will include a list of the training grants, the goals for each training grant and the trainees' comments on the programs that they participated in, strengths and suggested improvements. So you can see here we have 12 training grants and they are listed here from the ones that were originally made when we were center down to the more recent ones. They include the University of Michigan, the University of Washington, Stanford, Penn, Washington University in St. Louis, University of California Berkeley and Los Angeles, Harvard MIT which is moving to Harvard, University of Wisconsin, Yale, Princeton and an MIT Whitehead Broad Combined Group. You can also see that the number of slots varies tremendously and not all programs are including both pre-docs and post-docs. We have a total of 129 pre-docs, 32 post-docs, seven short-term training and our investment in this is about seven and a half million dollars. So now we'll get into some very detailed information about the goals of each program and for this discussion I more or less included verbatim what the programs presented except that when we come to the trainees' comments I removed some of the attributes to particular individuals not because they were negative they were all positive but I thought that where as it was important I don't think it was relevant to what we are considering here. So for the University of Michigan their program goal is to provide training at the interface between genetics, genomics and the mathematical sciences with emphasis on training statistical human geneticists and human molecular geneticists with a strong grounding in statistics and University of Michigan feels that its current program is very well aligned with the current strategic plan. The University of Washington their goal is to prepare pre and post graduates for a research career in genomics, proteomics and computational biology. They talk about the courses that they have initiated that the trainees are continually exposed to the latest in genomics research and translation of genomics to the clinic that trainees and mentors are clearly innovative and leading the use of genomics and medicine and that program needs development is in bioethics and human subjects as related to genomics. So for them it was mostly trying to focus more now on the on the ethics part of their program. Stanford University they want to train students and postdocs to be innovative leaders in academia and industry by utilizing our diverse array of faculty advisors who are experts in technology development, functional genomics and clinical genomics. Our graduate students and postdocs have the opportunity to work with both basic and clinical research scientists who carry out projects relevant to the strategic plan. And here they say that they are already heavily aligned with the strategic plan. The vast majority of the labs in the training grant focus on one of the three basic science goals. We also have clinical mentors in the program. There is evidence that the university is moving more toward personalized medicine and that they are aligning with that as well and that they are recruiting in the areas of computational biology. Penn, their goal is to produce PhD scientists who have a solid background in genomics, modern biologists and computer science who have depth in one of these areas and who are strong researchers. So they feel that their training program is aligned with the strategic plan and they also say that outside of our training program they are also aligning themselves with a clinical component at Penn for which they will not be requesting funding from NHGRI. WashU, they want to train genomic scientists to address significant questions in fundamental biology and translational science by combining cutting edge computational and molecular methods. Most of our trainees come from the mathematical in the computational background area. We require them to take rigorous courses in the fundamental theory of genomic analysis and applied computational genomics. We train them in a dynamic, integrated and multidisciplinary research environment. The program is already aligned with the strategic plan and the most important features of their alignment are a strong computational mathematical component and emphasis on technology development and exposure to medical applications of genomics. University of California, Berkeley, they train the next generation of leaders in the generation, analysis and interpretation of genome scale data sets. They say that this program is relatively agnostic as to which organism of species is studied, focusing instead on the mathematical and analytical commonalities, especially in computational genomics. We will use the next several years leading up to our next competing renewal to examine our training program in the light of the strategic plan and add additional components as appropriate. University of California, Los Angeles, their goal is to teach students how to design and implement new methods for identifying genes that influence disease risk as well as normal trait variation. Our goals and training focus continues to fit well with the goals articulated in the strategic plan, particularly those that relate to bioinformatics and computational biology. University of Wisconsin, their program embraces cross-disciplinary studies that span across the basic sciences with special emphasis to the recruitment of engineering trainees and trainers. They feel that their plan is aligned as well, and they indicate that they are adding faculty with expertise that integrates genomic and clinical practice, and that's where they're heading. Yale, their goal is to train graduate students in modern genomics and proteomics and prepare them for a career in biomedical research and teaching. The program will increase emphasis in three areas, biology of disease, the human microbiome, and professional development in education and teaching. I think Joe Handelsman is sort of very strong in education, so I think you can see where the emphasis here is on her move not only to train researchers but also individuals in teaching. Princeton University, their goal is to facilitate graduate education at the interface of biology and the more quantitative sciences and computation. They have a relatively young program already focused around genomics and quantitative computational biology, and they feel that their program is already well aligned with the strategic plan. And then lastly, the MIT Whitehead Broad, they plan to produce a new breed of interdisciplinary scientists who can create fundamentally new computational and mathematical approaches that enable significantly forward progress on biological and health-related problems using computational approaches to genetics. And they feel that in general, their plan is aligned with the NIH, NHGRI goals, but that they are taking further steps to improve the training program by increasing our coverage of quantitative topics in genomic medicine. So that sort of is a broad sweep of what these programs are about. Now we get to the trainees' impressions, and what happened here is that we ask every program director to go out to their trainees and sort of taking a cross-section of those that had already completed the program to those who have newly completed the program, and to ask them what were the strengths of the program and what would they recommend for improvements. So for the University of Michigan, the strengths of the program were the multidisciplinary design of the program. It was excellent, and it provided biostatisticians with a superb comprehensive training in genetics and epidemiology. Another comment, a valuable framework from which to complete interdisciplinary training while the University has essentially unparalleled resources as a doctoral student was difficult to prioritize training and research opportunities that match specific goals. The training program provided that organizational medium that that student needed to make that program successful. Then the opportunity to learn more biology while completing my degree in statistics through a structured curriculum as well as various talks and seminars. The opportunity to fully focus on my dissertation research without worrying about how I will be paid. And the program enabled me to network with students, postdocs, and faculty across the University who had similar research interests. They talk about the annual retreats being very useful and to hear the perspective of multiple faculty on these various topics was extremely useful. Some of the suggestions for improvement. One person said that they had no suggestions that the program was stellar. Another one, I cannot imagine anything that could be improved for my days on the training grant. Another one said interacting with students and faculty from other departments was one of the most useful and enjoyable parts of the program. And that individual suggested increasing that kind of interaction among the groups. Another one said to create more opportunities for students to present their work to other trainees as well as affiliated faculty members. The University of Washington, the strengths, the training program and symposium was valuable. Giving research reports and journal club on a yearly basis. These requirements really shaped my scientific communication skills as well as my ability to comb through the and understand the literature. Another comment, learning molecular biology and organic chemistry for the first time alongside of students who were learning probability and computational programming for the first time. Another comment, through the training grant and the encouragement of the department, I made several connections with the faculty members outside of the training program that might not have otherwise happened. Some suggestions for improvement, focus the ethics related training on emerging topics of related clinical applications, and then incorporation of medical testing issues of informed consent. So here this individual is saying they want more information about ethics and how it applies to modern genomics. Stanford University, it created opportunities for interacting both with other trainees, all of which were a high caliber and many of the faculty and clinicians involved who are all at the top of their fields. I had the opportunity to work very closely with clinicians researching Mendelian disorders through whole genome sequencing, whole exome sequencing. This cultured my interest in clinical and personalized genomics, and I went on to join a personalized genomics company called Personalists to continue in that line of work. Another most valuable experience was the inaugural course in genomics. This person is now an assistant professor at Yale, and he indicates that his experience at Yale was very seminal and helping him get his first NIH grant, and he is collaborating with a lot of other people supported by NIH. Another individual said the monthly meetings allowed trainees to present their research and regular contact with students and faculty with genome oriented research. These individuals who replied did not indicate any things that could be changed, so I did call up Mike Snyder, and I said, you know, all of the comments are positive here. There were no negative comments, so he said, well, he couldn't explain that except that perhaps maybe some of the students were so far out of their training they felt that commenting on what is currently happening in the training program would be inappropriate, but that was just his take on that. Yes, some of them had finished, you know, like 2005, 2003, so we wanted to get a cross section of those who had finished early versus those who had finished more recently, but it was up to the PI to make the determination. Yes. So, Betty, if I understand you right, all this feedback has been collected and filtered through the program directors? No, it wasn't filtered through. What the program directors did is they sent me directly the comments from their students, so they went out to them and say, could you provide your comments on the strengths of the program and improvements, so they wrote back and said, dear doctors so and so, these are my comments, and so they just sent me the files for what was sent to them. Is there any reason you wouldn't just go and want to go and get the comments directly from the people who have been in the program? Well, it would be hard to know who to contact. I mean, they would probably give me, people from 17 years ago would have given me probably 40, 50 names, so I, you know, I think what you will see at the end of this is that there are some common themes that are strengths and needs for improvement in all of these, and I think that's one way to calibrate people's thoughts, the nine people at a time. Can you hire somebody to do the survey, though? Because, I mean, Mark, I didn't realize it was done that way. Mark, this is not going to be very scientific if you're... Yeah, I don't think this was meant to be a scientific... I think it was just sort of to capture some of the thoughts about the program itself. This was not an evaluation of the program. U Penn said that the facilities and faculty are fantastic. There are lots of support available. One person said the seminar series and the opportunities to meet and talk with visiting speakers was a good thing. Multi-disciplinary program gave access to faculty from many different disciplines, and the other one said that the chalk talks and funds for research were strengths of the program. In terms of suggestions for improvement, to hold occasional programming events where past and current trainees could get together and share their work, set up biweekly or monthly meetings for the trainees, invite faculty to come in and for students to talk about their research, more opportunities to present our research in post and talk form, and one person had a quibble with the supply funds, and he didn't get clear instructions on how he could use it, so he had some concerns there. WashU said the positioning of the program is in a unique multi-disciplinary environment where experimental biologists, genomicists, and computer scientists mingle, that was a positive for them. The program and field that it provided an important foundation for my postdoctoral work. The training program enabled me to more efficiently conduct my research that merges computational and experimental approaches to address important questions in basic biomedical research. And another one said one aspect is that I found particularly advantageous was the availability of travel funds so that they then get to meet other people in their field. Suggestions for improvement, I imagine that I would benefit more by interacting more closely with other trainees. I think I could have benefited from additional feedback on my research program from a variety of faculty members. Berkeley, I dreaded the ethics course, but after taking that I definitely had a feel for broad understanding of how to approach ethics questions. Funding for stipend and travel were great. As a postdoc the individual research training and mentoring was good. The course work was indicated as a strength as well as the lectures and series that were supervised by the faculty members but given by students. And one student made the comment that while he was a graduate student in this particular program because everybody was sort of like each other in terms of their background and whatever, they were not considered special. But he took a position with Pablo and worked on the Neanderthal genome project and he apparently arrived there at a very opportune moment so that his expertise was very critical to the publication that came out and he said he never realized how important his skills were until he was set into an environment where that type of expertise was lacking. So some of the suggestions for improvement, no real central organization of the training program, I do not remember meeting any of the other genomics trainees. There was never a clear explanation of what educational resources were available to postdocs. University of California Los Angeles, they talked about the courses, the class, the core two-quarter class sequence in the program gave them a broad introduction to the key topics in genomics, the breadth of the coursework and the mentors. Some of the suggestions for improvement is some consideration should be given to formalizing instruction and two approaches of scientific investigation, hypothesis-driven research and discovery-driven research. The seminar series, they suggested that more could be done there and they also talked about to familiarize trainees with issues of managing and leveraging biological big data should be offered. And I think the next comment was along the same lines. So the Harvard MIT was the ability to take coursework and do research across the Harvard MIT system, the strong faculty and fellow students, the clinical training and the strong integration with the medical school, including exposure to translational research and clinical practice. Some of the suggestions for improvement was initially not knowing how this training program differed from others. More opportunities for the students to directly interact with one another and have some common experiences, having a central core of resources that students can use to ensure that there is an adequate progress toward their thesis work. An intensive, rigorous core, bioinformatics, biostatistics course for all students to lay a common foundation and base qualifying exams on research proposals rather than generic bioinformatics curriculum. The University of Wisconsin, each person has a valuable opportunity to present and get feedback, the interdisciplinary nature of the program. The training was particularly effective in giving students a broad perspective on the statistical, biological and computational challenges that are inherent to large data sets. The ability to unite people from different fields to work on important medical problems and bringing together people from diverse fields. So this program had a lot of multidisciplinary, interactive things. Suggestions for improvement, it would be beneficial for the trainees to have an opportunity to write a proposal preferentially based on new ideas that is collaborative among the trainees and the faculty. The program should incorporate some type of journal club that will allow for discussion of research that may not be going on at the University itself. And the course requirements do not align with the requirements of my department. This individual had a problem in that not only did he have to satisfy the requirements of the training program, but he also had to satisfy the requirements of the department. So that was a little tension in that. Yale, the strengths of the program, the diversity of the science being conducted. This person, after completing the degree, went to law school and is now working in the area of intellectual property and feels that the background that he had in science was very useful for him in dealing with the scientists that he has to deal with when it comes to patent issues. The training grant was very useful. I subsequently went to Stanford to do a post-doc, and I'm now assistant professor at Uppsula University. It allowed me to perform exciting genomic research in graduate school and gave me the educational background to pursue a career developing cutting-edge genomic technology used around the world. And again, this was another program that didn't have any suggestions for improvement. Princeton is relatively new. One person said the resources, the constant stream of lectures, colloquia and seminars, the quality of the faculty, and the exceptional mentoring were the things that were all strengths of the program. Suggestions for improvement, more structural interaction with the students and programs, and a little more structure in my academic program. We are moving now toward large-scale sequencing data generation. It would be good to beef up some of the infrastructure pipelines. My experience was so positive that it's hard to imagine what could be improved and more organized formal events to encourage interactions with the other students. So what I thought I could do to sort of bring this all together is sort of look at what were some of the recurring themes that dealt with strengths and improvements. For strengths, it was the multidisciplinary nature of the program and the opportunity to cross-train in another discipline. The opportunity to network with students, postdoc, and faculty. The financial support for tuition, travel, and supplies. The opportunity to present the research would sharpen your written verbal and organizational and analytical skills. The curriculum, the integration of the basic training with exposure to translational research and clinical practice. The quality of the faculty, learning how to communicate across the computational biology divide, and the rotations which allowed students to select the projects that they would be interested in rather than just going directly into someone's lab and not having the opportunity to see what all is available to them. And learning how to manage large data sets. Some of the suggestions for improvement are increased opportunities for students to interact with other students and that seemed to have been throughout most of these programs. To create more opportunities for oral presentation. To add LC topics to the curriculum. To provide more feedback on research progress from a variety of faculty members. To provide more instructions on how to manage and leverage big data sets. To require all students to take a bioinformatics by logistics course to a common foundation. To provide opportunities to interact with industry. And to align training program course requirements with the departmental course requirements so that students do not take essentially duplicate courses. And provide students with an opportunity to write grant applications. So one of the other charges from the small group was to look at what else NIH is doing in these gap areas. And also to look at what training is going on in the clinical and translational science awards. So for this particular exercise the directors, the director NIH has a set of working groups that are looking at the NIH workforce. And they had compiled a group, a lot of information about the training that is ongoing at NIH. So that that document includes all areas within NIH that we have interest in training. But what I did was just to pull out those that are relevant to this discussion which includes informatics, bioinformatics, biostatistics, computational biology, and molecular medicine which is asked to here so you'll see that. So in terms of informatics most of this is supported by NLM and cancer. Now the NLM awards are kind of interesting because they are T-15s and usually we look at T-15s as being awards for courses. But because NLM does not participate in the national research service award program, this T-15 is used basically as a training program. And most of these awards are like $900,000 each. So you can see here that their investment in informatics is almost $12 million. Cancer and nursing each have an EB. There are three of them and they have about a half a million dollars invested in that. With bioinformatics there's one in I, one in environmental health, one in HG6 and GM, and that is close to the level of $3 million. Biostatistics there are a total of 33 awards at $8.4 million. Computational biology, a total number of awards, $25 for $4.97. And then what I call molecular medicine there are five and the investment there is $1.2 million. So there are other institutes that are doing research in these gap areas. It doesn't tell you how many trainees there are, but at least you see that the investment is somewhere around 20 million dollars or something like that. Then looking at the clinical programs out of the old NCRR, their 2010 annual report lists all of their scholars and trainees in the various areas. And as I parsed through this list I didn't see anything that really popped out as being having a focus in the gap areas. So further down here you can see just statistics, research, methods, informatics, there are five trainees in that area. So I don't think we can look to that group as being a place where it's a focal point for training in the gap areas. So I think right now I will just stop and ask if there are any questions about what I've discussed so far. Alright, so thanks for putting this all together in your earlier slides before you started going through all those charts and kind of follow on to some of things from the teleconference and some email discussions. There's just, there's listing of gap areas which I think are important. But then I think what's missing, it shows up in what we see in what these different training programs are doing. But I think it's key to include also that it's not just gap areas and these disciplines in isolation, but there's that. And then if you go forward a couple more slides. Back down here or up? Forward. Continuous areas of need. I still didn't see in there students that are, they need to be strong in some discipline, but also in environments that are highly interdisciplinary where they're working together with the different sciences as well as the say clinician or something their laboratory training. So there's that whole interdisciplinary part I think didn't get put in here so much. So this is what came out of the discussion. So is this what you're saying does not include that? That one and then the slide where you had bioinformatics, which recognizes the need, but then tying it all together to produce these interdisciplinary scientists that are able to work together with all these different disciplines including the clinician. And it was something that I had sent in an email and then you said that was in a science article, but it didn't get represented in here. I don't think and I do think it's important that it's included and I see it coming up in some of those training centers and the things that they're doing right like Broad and some other ones had it. So if we could. Do you want to include that? I said I did send a bullet about it in March and I don't see it represented here. I will go back. Yeah well it's not just a listing of disciplines that students need to be trained in but it's going a step further and saying that students that are trained in certain areas, different sciences, engineering, statistics, you name it, but that they also have experience working together in laboratories or environments that are highly interdisciplinary so that they're able to more readily translate their discoveries to the application in a clinic or whatever it may be. So it was getting that extra step of the training environment for these students, not just it's not just simply a listing of we need more people trained in this area. So you're saying cross training so that people can talk to each other? Yeah so that they're truly trained in a multidisciplinary environment. Not in isolation and that includes working with the physicians and whoever that is, those teams that are important so that they can more readily, it helps in the discovery phase but also is important for being able to more readily translate the new discoveries to the application of the clinic. Okay. Yes. Is there any attention given to the criteria by which students are selected for these training programs at this level? Was that done? No that's done on the locally by the program director but that training grant is reviewed so how he or she plans to select students or recruit students that's also looked at and if you look at a training grant it will always have information about the background of these students you know their their scores and things like that how they make their decisions but that's a decision that's made at the level of the of the T32 program director. Maybe Mike can. Sure I think I think it really does vary training grant but training grant I mean ours for example has an executive committee that solicits information from the different departments, asks faculty and departments and students to nominate or self nominate and brings together applications and goes over them. I think that's a fairly typical kind of process. The comment I wanted to make actually was was was getting back to Dee Dee's comment and I think all of these programs are at least all the ones I'm aware of and there are actually quite a few that I wasn't aware of really do have a focus on interdisciplinary training. I think at the same time we have to be pretty careful we we can't try to train super people who do literally all of these different things. I think we want people who work in more than a single discipline but if we if we try to have them you know trained across all these different topics that we might be interested in they'll be in training for the rest of their lives. So I think the key thing is to have them be able to have a core discipline in which they're really expert and to be knowledgeable and able to talk to people across other relevant disciplines but but realistically it's probably two or three rather than six or eight or something like that. I agree they need to be strong in a discipline but I'm saying some immersion in some environment where they're working together with other disciplines. Rick do you want to have have any comments since you were at Stanford for a while? Yeah I'm happy to see that grant in its 15th year or 16th year years is a couple years older I think. Hold on just a minute you chose me right when I was chewing something. From the very beginning and it really wasn't just the training programs from the very beginning these were interdisciplinary I mean we overuse that word and the deans overuse the words especially et cetera but the truth if somebody's in bioinformatics they're automatically are biostatistics are automatically interdisciplinary because I have to know these two areas that are that at least originally were very different they may not know engineering but they know at least those two and in that training program and we had people from all sorts of departments and I think they still do clearly they do they're making it even broader it looks like. I will say that what struck me so much about the way this whole training area evolved since really the early 90s even the late 80s when we were first talking about this is that every graduate student who comes through that program sorry the genetics program and one many of the ones that are related to it end up because of the courses that are taught end up knowing how to handle large data sets so the the wet biology people who would probably never ever even think about this really do get training in that and what happens afterwards is that many of them end up becoming I mean it really has made a huge difference for them populating the the the field many of these are now professors or at least assistant associate professors at various places so I have always thought this program overall is really really important I think that we want to want to stick to it and I guess you could always argue how many people are retraining well we're trying to train the best in this sort of still emerging areas I don't think we should worry if we're over we're getting too many genome scientists with broad interdisciplinary you know sort of training skills because I think we're going to need even more just out of curiosity do you have a breakdown of given the emphasis on on quantitative training training quantitative skills like bioinformatics statistics do you have a breakdown of how many people start these programs with a strong background in say math and statistics and then learn biology versus the other way around there say they're biologists and then they acquire some quantitative skills you mean completely switch so that if they're informatics they're only doing wet lab work or no no I'm just asking what are their pre-existing backgrounds when they come into these programs yeah they all have these are graduate we're talking about graduate students so they all come with a background in genetics or informatics or bioinformatics because like Wash U all of their people come in with degrees in math or bioinformatics or informatics statistics and then they get exposed to biology so that's probably that program along with Penn it's probably the purest where you have you know the the informatics computational biology types coming in and getting exposed to biology so so I'm just asking what that breakdown is perhaps by program and across the whole set of programs how many start with say a degree in math versus a degree in biology that I don't have Rick I bet you it well might might be able to do better very I'm sure it varies from one to the other the Stanford one which is a very large one as you could see from the numbers the whole time it was probably 75 percent you know biologists trained who came in and either had a little you know may have had a little quantitative skills but became much more quantitative and 25 percent who had very little biology who then became biology as well as bringing in obviously they're already strong and fully developing their biostatistics or bio or bioinformatics I think that's David you probably know this as well that's something numbers something like that because most of them are biologists who come into that and a lot even though they're there's a great computer science department and statistics et cetera that they do recruit we did recruit some of them to it so it really it really does vary by program ours is sort of the opposite direction is probably two-thirds quantitative a third genetics I'm just asking it would be good to have is the government allowed to collect those statistics well I I guess the other thing is that if you look at all of the training programs they usually have five ten fifteen departments that are collaborating and and not all of these are just little narrowly focused programs the pool of students that they draw from probably comes from a much larger group so it's kind of I'm sure we could get those numbers but is it part of the reason I ask is that at least in my experience I I think for dealing with these very large types of data sets very complex analysis questions it's it's much harder for somebody who doesn't have perhaps that initial training in in some kind of say say math or statistics discipline to to acquire those skills it's easier to take a mathematician and teach them the biology than the other way around so I'm just wondering is that okay well just my two cents worth but so I'm just wondering if if basically that's something that you're taking into account maybe other people don't think you should for for the evolution of this program if in fact you want to train more people in quantitative aspect ready I don't know if you still do this there was a was at least for many years a strong push to get more what you call bioinformatics people and in fact I think we even got extra slots a couple of years if we if we got more and so I think that was the direction it didn't mean to disagree so violently but but but but but but just having watched a lot of pure math and pure computer folks who had no background as undergraduates come into the graduate programs and do this it was it was it was quite hard it was the hybrids the ones who came from undergraduate who had a little bit or maybe even a lot of both who who did really really well um but but I think I think that your hope the whole point in the training programs were get to get those two types of groups together and at least it we saw lots of benefit and I think it's fair to say that all of our training programs will have some of those types in there they won't be all biology types we that we don't have that's that's not in hgri and as someone said we've always emphasized interdisciplinary and I think going through some of these comments it was very clear that many of the individuals said the opportunity to listen and work with other people who had you know training in the opposite discipline was was a very sort of synergistic type of an activity for them rick well I have a question yeah no I mean my you know my take on all this and I actually was in a statistics department for eight years and precisely tried to take statisticians and turn them on to interesting problems in biology and it turns out that the goals of graduate school and statistics and the goals of graduate school and biology are pretty different and even the way they're funded and what they prioritize and even when you sit on a search committee and what you need to succeed and get a job in statistics is really different right like we would hire people who'd never published anything but because their advisor had said that the chapter of their thesis was really excellent they had proved something really hard then that would be a real bonus as opposed to having multiple authored papers where there were second or third author so I actually think we really really need to focus on creating the mechanisms by which we can train people who really interested in developing algorithms and applying them to data even though they may not be solving interesting problems in computation or in statistics but rather really working on the interface and that can be done but it has to start exactly just saying either by taking people who've had some math and statistics background and then they go and do a PhD in genetics and statistics focused on these problems. I thought you were saying Mike. Yeah different kind of Mike. You know I've had experience in both directions and I somewhat agree with Mark but at maybe a different level that is if I look at and I would say it a little differently that if I look at people who already have PhDs in biology it can often be a little bit more difficult to move them into the more sophisticated algorithmic realm than actually I don't I think mathematicians I'm of course a notable exception I actually think physicists do better and maybe then statisticians and computational scientists supplied mathematicians than pure mathematicians but that's irrelevant. I mean I to me the major attribute I've seen the major phenotype I've seen in people who go on to be successful computational biologists are those who become totally obsessed with and I think Carlos you just said it this way solving the biological problem that they are interested in developing new approaches new algorithms in order to address some driving biological problem and I think that you can't generalize there are some people with statistics backgrounds who are completely captivated by the biology and there and there's some that aren't. In terms of undergraduates I see more and more people coming into the MIT program that have the combination of both as undergraduates they have joint majors in math and biology or computer science and biology or statistics and biology and they they do very very well. I think there's always a leap when you have to go from a single discipline to the interdisciplinary thing and you have to be really driven by the other side in order to be successful there but I don't think we can sort of make a general statement about it but I do think that successful computational biology or bioinformatics PhD programs stress that a successful thesis is one that solves an important biological problem and also makes some advance on the analytic computational side. I think the only thing I would add to that is that aside from training and developing people who will be pushing the boundaries in terms of algorithms and methods development there's also the whole really really important issue of training biologists to think quantitatively and be able to analyze large-scale datasets. I agree with you 100% and that's actually I mean I just think it's like it is the same way that they have to understand chemistry and and the way their reactions are working they have to understand. I agree 100% and that can often be done by specifically tailored courses to those people because they may need a different kind of approach. And it's actually I mean what I found is that it really you know there's a sort of old guard that you find that says well you know I never took statistics when I was in graduate school and if you needed to do statistics and you did the wrong experiment it either lights up on a gel or it doesn't and they're the ones that tend to be you know the most reticent to this but you know I think that is is crackling down now as more and more students just come in and say look I've done this huge experiment and I have no idea how to analyze the data so you know what am I supposed to do do I always need to have somebody that I'm going to take my data to and I think that's the big frustration of people who are training today that they feel that they're sort of falling in between the cracks where they're taking kits they're doing experiments and then at the end they're going to flood a data and they don't know what to do with it. Betty how many of these courses or these programs make their material available online you hear more and more about people who are now making this available to a broader audience. I'm not aware of any but that's we haven't required them to do that either that is it is are their training but what is it that you're asking? The training broadly I mean multiple types of things can be made of it. How they do it you know the course itself I think I think most of us who have these training programs don't create them de novo I think I think it's you know taking existing things putting them together in novel ways encouraging people to do a broader range than they might otherwise do and so if if I were given the requirement to put my materials on the web that would be difficult I would do my best but I would have to be talking to the instructors for all the relevant courses who are not all of them members of the training program but the notion of trying to get some relevant information online and I think I think one really useful thing is is the sharing of best practices across some of these for example some of the sort of issues that that Betty identified as being difficult for one or another of the training programs having the training programs talk about you know what do you do to address this issue do you feel good about what you're doing there could you have information that could be helpful to someone else who's having some difficulty there I think it'd be quite useful what do you think about just perhaps requiring these programs to do the work to in order to make the at least key aspects of the training broadly available to anyone wants to access them I think the cost would be not trivial and I think it wouldn't be complete but but one could get one could get some of it and you know for for larger training programs like the one at Stanford where it's not entirely but largely built within the genetics program of the genetics program extending beyond to different groups maybe there's more centrality there than there is or not even there because we're dealing with you know eight different departments and one or two faculty in this one and lots in that one it's it's a little complicated I wonder almost if part of the issue here isn't that we need to start a little earlier in the pipeline you know my experience has been you know in the phd students that I've had in my own group there's been a strong selection in biology for people who were non quantitative you know you'd sit with the folks in your lab and you'd just ask them to walk work through the first thing you do is you look at data you work through some numbers start getting into the numbers and you know I could do those numbers in my head and they weren't even thinking that way about the problem so you know maybe something that we need to be thinking about is how to make sure that people that have some quantitative training and a quantitative approach just sort of naturally or through their training as an undergraduate realize that there are some really terrific opportunities out there for people to bring those skills skills into biology I think it's happening but if there was something we could do to just sort of encourage people to say you know if you get really great if you're sort of quantitatively inclined get great training in biology and continue your developing your quantitative skills you know you're going to have lots of opportunities David Botstein's course is and the whole thing that he's set up at Princeton is really based on that open but just with my university president had on him and I think that the one of the arguments about the values of liberal arts education is that we develop critical thinking and critical analysis skills but the whole quantitative skills and the amount of information that data that all graduates will need to deal with in the future is very haphazard some departments offer it as a course many don't many biology departments don't offer it and I think it needs a fundamental rethink in terms of entire curriculum review of whether there should be a broad base like what David Botstein has done at Princeton in terms of developing quantitative skills as one of the things that you do get from a liberal arts education not just you know what you can do in critical thinking because it is really haphazard at least a tough you know from what I've seen and I'm sure it's true at other universities it's very much dependent on the department and their ethos and approach you know so students will self-select for statistics or maths and other areas but in general most students walk out of university without any any skill set in how to deal with large data sets or the amount of information that they'll be needing to deal with in their jobs no matter which avenue they choose I always found a key very interesting discriminator was whether people had taken physical chemistry as an undergraduate if they'd taken physical chemistry as an undergraduate they probably had the skills to go this direction if they didn't have the skills to go this direction they would run screaming from physical chemistry that's all I ended up in my job I never took physical chemistry I think just just to make sure we have on the table I think we're talking about the biologist versus a mathematician computational person since we're talking genomics there's also the clinician side and our biggest problem is epi biologists who don't know DNA comes from a person it comes from a commercial company that sends you spickets of blood so I think you know in some of the comments up here where I you know I was so nervous taking the ethics course well you know they should have that's been the first one anyway so also that interface kind of want to change the direction as Congress and I have one more question too yeah no I just wanted to return to the pipeline issue how many persons from underrepresented minority backgrounds have been trained over the years and what are we doing to increase the number since this is an area where they are grossly underrepresented so we we have about 10% are a little bit more of the people are underrepresented minorities we also in the past and I am hoping we can continue this in the future if a program director has already feel their number of slots with students and have been kind of good citizens and appointing some minorities that if they find other minorities that they can come into us with a request and we will support that person we we're also moving to a point where we are we have allowed training grants to have extra activities for underrepresented minorities and these will be moving to an r25 program announcement they're in that process now so that those are the issues that those are the things that we're doing to sort of deal with that issue Rick and then I want to say something I had was going to actually ask about the map programs but but before that I just wanted to comment it's sort of going back to something Jill said when we first started doing a lot of this stuff we I think we thought we needed to teach the quantitative people about how we do experiments and what became clear is that we needed to teach them to learn about what a promoter is or what you know how sales work and that's when mark when my reaction you partly was that that that learning biology when you're for the first time when you're you know way past your training it's hard to get all that because of the complexity of it I think it's not a it's not a boilerplate or a single thing and that that's where the the passion for the problems come from because they understand that this is where the questions are not you know the sequence I mean they need to know where the data come from it's not that they don't but but I don't think it's important a lot of them almost all of them wanted to come in and do some genotyping so they they could you know say that they had done an experiment and we would do that it was we learned that that was less important than them understanding the the depth of what the biological questions are so I just wanted to say that and then the other thing is changing the subject entirely Betty it's related to what you were just talking about I wanted to hear if how you feel that the map these are the what does it stands for action plan my no gap diversity diversity action plan how those have been going do you think that it's helped is it working well have you had problems with it are there are there improvements well I think a lot of effort into this I I think it has helped but like everything else it's going to take a long time to get there but we do know that some of these individuals have have gone beyond the DAP to go into graduate programs and other places some of them have gone to medical school but you know that's we can't control we can't control that but I think by and large that's been helpful yeah to continue on that there's a map are now adapt for each of the t-32s as well and that has really given us resources to actively work at this and it's still hard but to actively work to really try to make progress and it's something that has to be a constant point of attention and I think it's particularly true in the case of mathematics where it's a particular deficit just in general and for particular groups as well we just do a terrible job as a society encouraging people to study mathematics yeah that if we really wanted to talk about the most serious issue from my perspective that's it and all the all the kids we lose in about middle school to early on in high school who you know before that think math is cool and then just get totally turned off so so let me shift the discussion a little we've been talking more about how we are training and pipeline issues related to interdisciplinary training and so forth one of the things I was hoping to come out of this discussion was a little bit more emphasis on how many and there is a certain amount of our portfolio I mean and council is getting increasingly familiar with our what our portfolio looks like and distribution of whatever increasingly scarce resources but we have x number of training programs of x demographics in terms of distribution of postdocs and graduate students and so forth and one of the thing and I get lobbied all the time for desire to train more train more train more we have ideas on how to train and different but it would be training more I guess I was hoping I we don't have a whole discussion here but one of the things I think I would like to see council start to do in the coming months to years is to give a little bit of guidance about you know is this the right amount of training we as NHRA should be doing should we be investing more if we invest more will the money come from what should we invest less should these be phased some of these phased out and we'd be using that money for other activities I have had a difficult time getting the sense of why is it that we have x number of training programs and why not 50% more 50% less so so any I hear I've heard lots of opinions from people who talk to me you know at an ad hack basis but I'd be curious what any of you think now that you've at least seen a snapshot view of what we do in the in terms of training programs and what you hear about or what you think is needed or not needed so wreck I think this is a really important conversation for the NIH in general to be having not just NHGRI you know there's lots of questions about how many under employed PhDs there out are out there do we continue to just train ad lib or do we need to try to you know be more honest about what people's prospects are so I think there's two pieces of that I think there's the broader question about how many PhDs we should train and it's a complicated argument right there's a lot of people who say well it's it's great training and you learn how to solve problems and you you know you you can become a great cab driver by doing that but I think there really is a question of what the biomedical research community needs and I know there's an NIH workforce group to try to address that question report will be coming to the June meeting of the advisory committee to the director for and I forgot exactly who's coach here on that besides surely Tillman and someone else I forgot Sally Rocky Sally Rocky and Shirley Tillman so we will know more but I think you're right but that we no matter what we do in reality we will be a small pebble and a large quarry of training going on in NIH and so I'm thinking mostly around genomics and what we need in that full spectrum of basic to clinical genomics so the second point that my guess is at the end of the day what we're going to see is that we need to keep it either around the same or even maybe let it shrink a little bit that's just my guess as to what will happen but I think what that says is that we all agree that there need to be more people trained with quantitative skills in biology so that means if you need to take some slots away from other areas and put them into quantitative training of biologists you know that'll drive movement of people into into that field so so one thing we could do and this shouldn't be the only story because I don't think it's just about going out and getting jobs but we could you know look at people who are five years post training or more three years post training or more or something like that and ask you know how many of these people have jobs that we're happy with you know and you know we have to make a decision of what that corresponds to but how many people are doing genomics in a way that required a phd or required a postdoc in the kind of training that we've provided and if it's 98 percent of them then I disagree I think maybe we need to stay the same or get a little bigger if it's 50 percent well then I agree we need to make it a little smaller but but that kind of data could could actually be useful and it might start out with what mark was asking earlier about where do these people start from what do they get to in their training and then what is their employment experience after that in terms of outcomes that we would view as successes for our trainees power so we're looking at the portfolio I was struck at the high quality that was there but it's incredibly narrow and focused and if you're if you go outside statistics bioinformatics and you know maybe half a standard deviation beyond that there is really not much representation at all and especially in genomics no one else is picking up that that piece and you know we've talked about it on the phone the phone corridor where you you don't have a lot of people with expertise in genomics in the clinical disciplines outside of medical genetics whereas that's their thing and there's no place for them to train and I think a t32 in that area is not necessarily the right answer I think that most of the time we don't want them to be I want them to learn beyond the point where they're dangerous so the point where they're they're just not dangerous and often that means knowing they need to collaborate or knowing they need to work with others but going and getting a phd you're almost repurposing as opposed to augmenting and I think we need to be more creative about the types of training programs we have for adding skills to someone who already has a day job as opposed to repurposing someone's day job I think you also need to think about the international competitiveness of this space because you've had the bioeconomy report saying that we're in trouble and the sequencing is happening in China the question is is the analysis happening in China I suspect they are increasing their programs and I think in thinking about this problem you need to look and see what other countries are doing in terms of the training of individuals in this area I don't know if I want to overblow genomics but just going back I like what Mike said about Rex's comment that let's get the data and we can kind of figure this out but I just remember I think I brought it up here or maybe Eric brought it up this Betel report from last year that talked about the economic impact of the human genome project and and that's an opportunity and it's huge it's 141 or something fold you know I mean it's just it's 171 but it's some huge yeah yeah but the point is it's because the field has been so revolutionary and it's not that we want to turn every biologist into a genomicist or a bioinformaticist but but and given that and then the other report you showed today with the potential job not jobs potential markets for these not to mention the fact that we really haven't even started applying this clinically I think we need to be a little careful about being too austere about holding back I mean we need more and more of these people and I think the hybrids I just want to re-emphasize that the hybrids people who come at it if they know that this is exciting and important you will get people more and more kids coming up where they will actually kind of learn both at a young enough age where when they hit graduate school they it's not painful for them so Ross and then I think Betty will turn back to you yeah continuing on that line you know well in your comments and we're all well aware that these are a difficult budgetary times and I think Rex was getting at that you know how many people really are going to have jobs and and are we overtraining what we're talking about now though is training the people who will be looking for grants in another five seven years it really is I am hopeful that the situation we're in now will be a nadir that we will be rising up from that and with all the terrific potential that that genomics has and and all the concerns that that people were mentioning you know that the report about there are losing leadership and and uh genomics and medical research is uh this is that if there's any way to move in a positive way to increase the the training funds we should could I just have one quick thing that goes with that is that I mean it's much broader than even that because in engineering and all the different technologies it's also we're in a dire situation where we're not able to train enough students for the for what the job market has and also if you think about engineers and and businesses and so on and them doing developing new genome technologies that's another whole part of this as well so so um just responding to the issue about how much we should be training I I looked at what we were doing about a couple years ago and I don't think the picture has changed since then we had about two percent of our extramural funds and training and on average NIH was spending about three point three percent in training so in a sense we are kind of on the lower edge of what other institutes are doing in terms of training so that's just something to think about but um I I think you know I I think it's very difficult to make decisions in a in a forum like this and I I'm wondering do you have Betty but that several of us here were saying that that comparison is surprising to us and maybe we've heard this before and have just forgotten but if it were my bias coming into this it's that opportunities and genomics are stronger than most of the disciplines across the NIH and so I would have thought that genome might be higher but but but there's economy of scales issues I mean that's I think what's working me that yeah that I don't know but I take gm out is if you take gm out of that nigms then what is the average of all the other institutions I think I did look at all the other institutes and I'll see if I can get that figure you know okay but I think what we really need to do is perhaps once again have a committee a little small committee that might help us shape this because I think one of the concerns is most of the discussion has been focused on you know training on the on the left hand side of the the density plot and I think not that we're going to move all the way to the right hand side but I think we need to perhaps broaden the the skill sets of the people training in in genomics proteomics genomic medicine so I think we need to really think seriously about how how to do that so I think maybe Eric we can probably get together and invite some people to be part of a small working group another area we haven't touched upon is how how would we like to handle training and career development in which it's the pyramid is getting even steeper and do we need to train people in those areas especially if we're looking for individuals trained in clinical sciences to do genomics proteomics research so I think this is just the opening discussion and we will be back at some time soon to to review these issues again and I guess just in closing this what I think what we're trying to get at is I personally don't feel we have a strategic a renewed strategic vision for what we want to do in training I think we sort of see a successful set of programs the the the settings are changed the landscape is changing and budgetary constraints are real and we are approached by people who want to know should we bring you good ideas and I think at the moment we're a little flat-footed with respect to making a clear statement about what we see strategically our role should be in training the scale of those training opportunities so forth so if we could call on some of you to continue and I you know I actually I'm Tony I don't think you were involved in that working group but maybe we could get you I think the perspective you would bring from the university or university leadership perspective would be very helpful as one example maybe others as well who are already participating but I think yours would be useful so I think we'll continue this discussion again I think for an institute that has you know reasonably strong views of strategically what we should be doing I think we could update ours in this arena okay so we're going to take a break now mark I think I think I think all everybody's due for a break why don't we reconvene really properly at 3 30 if you're about 20 minutes we have then left in the open session one two three more presentations I don't think there is lengthy three more presentations and then any additional discussion okay so let's reconvene at 3 30