 We're going to start the afternoon off with a talk by Mike Lauer who's in building one and he's going to give us an update on the next generation researchers initiative and the clinical trial reforms and Eric will introduce him. Yeah I'll keep my comments brief but we're delighted our good friend and colleague Mike Lauer is here to tell us a couple of updates related to things that this council is very interested in some of which we spoke about at recent council meeting some I've kept you updated about. For those who don't know Mike, Mike really was well known and well regarded in his 14 years at the Cleveland Clinic as a professor of medicine and active clinical researcher well known epidemiologist but then was recruited to National Heartland and Blood Institute where he served as a division director from 2007 until 2015 and then Francis Collins convinced him to become a deputy director and he's the deputy director for extramural research or he is the director of the office of extramural research and many of you are probably familiar with him because of his blog I guess it's called the open Mike which is you know well read and well utilized I think as a means to communicate information to the extramural world about policies and developments related to extramural grants and policies and so forth at NIH and so I Mike's terrific colleague we deal with him a lot at many levels and it's really kind of him to come spend some time talking to us and I think he's going to give us updates one on the next generation researchers initiative and second on some of the clinical trials reforms that are ongoing and being discussed at many councils so I know as well as our council so Mike I'll let you take over here. Great Eric and Betty thank you very much so I was asked to present some updates about what's going on in building one and the major update for today is that we're open and Friday morning it wasn't so clear there we weren't quite sure whether we were supposed to show up to work or not or when we were supposed to show up to work but nicely today things are okay. Alright so I've been asked to talk about two topics first I'm going to talk about the next generation researchers initiative some of you may have seen this this was a series of articles that appeared in nature a little over a year ago about what's going on with young researchers this headline kind of said it all young talented and fed up scientists starting labs say they are under historically high pressure to publish secure funding earned permanent positions leaving precious little time for active research and this problem that young researchers are facing in the increasing pressures that they're under has made it into the mainstream media as well here's a photograph of a famous scientist and this article was written by Gina Colada so many researchers and yet so few faculty positions and in this article she wrote this article appeared me about a year and a half ago the average age in which the lucky few actually get a grant steadily increased it is now 42 up from 35 in 1980 which means that biomedical scientists in academia are essentially apprentices until middle age and the tendency is for grants to go to scientists who already have them making it harder and harder to break into the system. So now what we'll do is look at a little bit of data to see if this is really true that the tendency of the tendency for grants is to go to those scientists who are established and who already have them. Here's some data in which we look at the success rates for attaining funding for two kinds of R01 grants. One are de novo grants or what we call type one grants where you're submitting your idea for the first time and then the second is for renewal grants or what we call type two grants. So the x-axis shows time 1995 through this is 2015 and then the y-axis shows the success rate. We can see that during the time of the NIH doubling if you are submitting a renewal for a grant you had over a 50% chance of getting that grant funded the success rate was 54% and even after the doubling ended and we went through what's called the pay line crash that's from 2003 to 2006 there was still about a 37% chance that a renewal grant would get funded. On the other hand for type one or de novo grants the success rates have always been much lower and now runs around 15% last year it ran about 16 or 17%. So this should seem obvious but nonetheless it does explain a lot of why it's much easier for established investigators to get funding than new investigators it's much easier to get a grant renewed and then to get it funded in the first place. Okay now here's a second way of looking at this here what what we did was we looked at age as a proxy for career stage. If we look at people who are in the early stages of their career which we define as an age of 45 or less the x-axis here shows time this is going from 1990 into 2015 and the y-axis here shows the percent of funded researchers who are within that particular group. So back in 1990 about half of the researchers that we funded were age 45 or younger. This number steadily went down until around 2005-2006 when it started to stabilize. We think the reason why it started to stabilize was at that time we instituted a number of policies to make it easier for younger investigators to get funded. Now the blue line refers to investigators who are age 60 or older so we might say that they are in a later phase of their careers. These people made up about 5% or maybe a little bit more of researchers back in 1990 and then we've seen a steadily increase and now they're about 20% or a little bit more. And then the green line refers to people who are in the middle stages of their career. These are people between the ages of 45 and 60. They were representing an increasing proportion of our investigators until the end of the doubling and since that time they have been making up somewhat less. And there's been some literature that has been written about this it's been suggested that folks who are in the middle stages of their career are being squeezed. On the one hand younger investigators are getting certain advantages because of NIH funding policy and investigators who are in later stages of their career are more established. They have more renewal grants. They have more resilience and so therefore these folks are getting squeezed. There's some literature on that. Michael Levitt from Stanford wrote an article in PNAS which he published last May called the future fundamental discovery in U.S. biomedical research and he asked the question what caused the drop in the number of young scientists. He says older grantees are getting money at the expense of younger grantees and that study sections have a form of age bias. Now it's not clear that that's really true and in fact we've got some data that suggests that study sections really don't have an age bias or if there is an age bias it's fairly weak and there may be other factors at play and that's something that we could talk about. Nonetheless it is clear that the system is under a great deal of stress and even with the recent budget increases which of course we greatly welcome the stresses are very real. This was a piece that came from the University of Wisconsin. It was a series of workshops that Judith Kimball led on rescuing biomedical research in the U.S. and she and her co-authors argued about there being two core problems that be set biomedical research. One is that there are too many researchers vying for too few dollars and there are too many postdocs vying for too few positions and everything else such as the stress that peer review is under can be viewed as symptoms. Some of you may have seen this before but we've now updated these data and there's some actual interesting trends that are going on. So this looks at people where what we're doing here is we're counting people. We're counting people who are PIs on an NIH research project grant or people who want to be PIs on an NIH research project grant. So the the green line here refers to unique awardees who are PIs. This is 2003 which is the end of the NIH doubling and this is 2017. What we see here is that the number has increased a little bit. It's gone from about 25,000 to about 29,000 and over the last few years with the budget increase has increased a bit more. Now the the red line here shows the number of people who wanted to be funded in that given year. This includes people who applied for funding that year as well as for people who applied for funding the year before and the year before that and year before that. Back in 2003 at the end of the doubling there were about 60,000 unique people who wanted to be funded that that year and that number has increased to close to 90,000 but there's there's actually an interesting trend which is going on which is that over the last three years the number of applicants has plateaued. This this steady increase that we have been observing which had it continued you know we might have seen something like this. We might have hit 100,000 by now. Hasn't happened and there's some interesting trends that are going on. There's been a decrease in the number of postdocs and maybe we're starting to see that. The concern is is that the hypercompetition even if it's starting to get a little bit less severe is primarily affecting younger scientists. This goes back to that series in nature. They say here in the United States funding success rates for all age brackets are less than half of what they were in 1980. I remember hearing that there was a time that the success rate for applicants at NIH was 60 percent, six zero percent. So you could imagine if the success rate is 60 percent all the stuff that we're talking about peer review and so forth would be completely irrelevant but we're in a totally different age now. So researchers have to spend a lot more time seeking funds. That burden falls most heavily on new faculty members which makes them conservative rather than ambitious. Congress is now coming to the act. About a year ago Congress passed by an overwhelming bipartisan majority the 21st Century Cures Act and in there there is this provision on investing in next generation the next generation of researchers and enshrined in law there is this statement the director of the national institutes of health shall develop, modify or prioritize policies as needed to promote opportunities for new researchers and earlier research independence such as policies to increase opportunities for new researchers to receive funding and enhance workforce diversity. And essentially this is a work in progress we're now in the middle of this of trying to figure out ways in which we can answer this law and make it possible for more new researchers to get funding to get funding in a state that we can say that they have earlier research independence and at the same time enhance the diversity of the research workforce. Francis Collins, Larry Tabeck and I posted a piece in PNAS this is a came out last November in which we talked about some preliminary steps that we were taking. We now have assembled a working group of the advisory committee for the director they're working this is co-chaired by Larry Tabeck and Jose Flores and they're working with us to help develop a policy so that we can we can take this going forward and hopefully reverse some of these concerning trends. Now like many many laws that we see from Congress or good ideas that come from Congress the bill will say something like you should do so-and-so because it's a really good thing to do and it's a really great idea but there are no there are no monies that go along with it and this is one of the concerns and challenges that goes with the next generation researchers initiative is that provision did not say and you know we are giving you x amount of money that you're going to put into this particular bank account over here and you're going to use that money to support this so one big question is is where will the money come from if we are going to fund more early-stage investigators then we have been you know obviously that that costs some money we have to figure out where where it does come from now some of it may come from the budget increases and and we've been pleased over the last few years we have seen steady budget increases so some of the budget increases could potentially go into supporting more early-stage investigators and there are a variety of other potential mechanisms as well and then another key part to this is that as we take steps to be able to fund more new researchers we have to also assure that as congress asks for that we are seeing what impact does this have on the on the scientific workforce is the at the very least we would like to see that diversity is being enhanced we would like to see that the that the squeeze that investigators are seeing may be attenuated a bit and that scientific excellence and outcome are if anything improved but but certainly not in any way adversely affected these are some of the things we have to think about okay now this is one of my favorites I couldn't figure out whether or not for those of you or how many of you are familiar with this okay good so this is John Cleese, Monty Python and what he would say is a now for something completely different so the typical scene is that the desk is up on a hill with lots of sheep walking around and but I found this one where he's by the beach and you can see the ocean water coming in I thought this was good so now I'm going to talk about something which is a bit different this is another another issue that we're currently dealing with in building one so this goes back I'm going to start this one this is a story that goes back to 2012 when we were notified that a paper had appeared in the British Medical Journal that suggested a major problem with NIH funded clinical trials this article came from Yale Debra Zarring from the National Library of Medicine was also a co-author on this and what that the folks these folks did was they looked at 635 clinical trials that had been funded by the NIH that had been registered and that as best as they could tell have been completed they tried to find out when the main results of those trials were published so on the on the right here there's a plot the x-axis shows the time from the completion of the study so a time of zero means that the trial is finished follow-up is done theoretically you can publish the results tomorrow and then this is 20 months 40 months out to 100 months then the y-axis shows the percentage of trials that had their main results published and what you see here if we look at 30 months which is a two and a half years a little over 40 percent of the trials had their main results published over 50 percent of these NIH funded trials did not have their main results published within two and a half years of completion so the authors were really quite concerned about there's there had been a law that had been passed that affected FDA-regulated trials which try to to deal with the problem of publication bias non-reporting of results and the authors say here that there are many NIH trials that are not covered by the FDA a law such as trials of behavioral interventions and surgical procedures then they go on to say no policies exist to make sure that the public has access to results from NIH funded research that is not published in other words if the NIH funds people to scientists to do experiments on people they do their experiments and then they don't report out the results the public has no recourse so this study comes out in 2012 and got a lot of bad press it was in the newspapers it was on the radio we got a letter from congress and I got some questions from my former colleagues at Cleveland Clinic who were wondering what was going on and that we must be asleep at the wheel and and our reaction to it was that this was all wrong there was absolutely no way that this could possibly be true for one thing you might understand why a drug company might not publish the result of a trial because maybe its drug didn't look good but there's no reason why an academic wouldn't publish the result of a trial that NIH funded after all publication is the currency of academia so this is when I was at NHLBI and working with some colleagues of mine led by David Gordon we decided that we were going to show just how wrong this was and how everything was just fine the way we were going to do it was we identified 244 consecutive cardiovascular trials that have been funded by NHLBI and we used brute force for every single one we looked up the paper that showed the main results and if we couldn't find the paper we called up the PI and said where is it and please send it to us so we we did this and we got a result and the result was we published that result in the New England Journal of Medicine in November of 2013 and so let me show you what we found so what we did was we divided our trials into two different types one type would be trials that focus on hard clinical endpoints like death or heart attack or stroke or an unplanned hospitalization and the other were trials that focused on on non-clinical endpoints or or surrogate endpoints so the the plot on the right is very similar to the plot I just showed you before the x-axis shows the number of months after the study have been completed so here's zero here's 10 there's 20 60 months five years and the y-axis shows the proportion of studies in which the main results have been reported out so these are the trials that focus on clinical endpoints and after a year about two-thirds of these studies had published and after two years all of them had published and this was completely consistent with our view of the world that everything was absolutely fine now the problem however was that those trials represented a minority of the trials that we funded less than 20 percent of the trials that we funded focus on hard clinical endpoints most of the trials that we fund focus on surrogate endpoints and here we saw a very different picture after about a year maybe 10 of these studies had published and made results after two years maybe it was about 40 and so because these surrogate endpoint trials were overwhelmingly the the trials that we funded in the course of attempting to refute the results of the Yale paper we replicated that and so this was obviously quite concerning in our discussion we wrote a number of parties share responsibility including funders investigators academic medical centers universities clinical research organizations and of course journals so i put funders in bold because that's obviously who we are and we'll talk about that in a bit but let's take a look at academic medical centers now in 2016 the folks at Yale looked at a different group of trials they looked at about 4,000 trials that have been completed between 2007 and 2010 these trials were mixed some of them were funded by industry some by NIH some by foundations and they look to see what percentage of those trials had their main results posted in one way or another within two years either they were published in a major journal in a peer review journal or they were posted on clinicaltrials.gov then what they did was they divided up the trials according to the main institution where they were centered they were primarily interested in 51 institutions which largely corresponded with the top 50 institutions that NIH funds so here are the results so the the x-axis these are the names of institutions so each of those long string of letters is the name of an institution these are all very famous places and the the y-axis is the percentage in which the main results of the trial have been posted within two years and the winner all the way over here on the left with the highest number of trials published within two years was the University of Minnesota and i don't know whether any of you are from the University of Minnesota but their their rate was 55 percent 55 percent of their trials had been posted or published within two years most of these institutions were in the 30s and 40s so the authors here say that despite the ethical mandate and expressed values of academic institutions there is poor performance and noticeable variation in the dissemination of clinical trial results across leading academic medical centers now the lead author of this paper is Harlan Krumholtz Harlan is a cardiologist and academic at Yale University and at the time that this article was published he posted a column on the NPR website in which he expressed his real opinions so what he says here is he says academic medical centers get an F in sharing research results we have a bottleneck at our nation's bastions of research excellence too many times study results are neither reported on the government website nor published in a journal the failure to share results is so pervasive that it seems inappropriate to blame individuals instead it is a systemic problem and we would agree with that this is a systems issue this is not an issue of individual people misbehaving and then he goes on to say not reporting results violates the basic principle of the scientific method it hurts patients society and science it dishonors the people who gave their consent and bore the risk of participating the holding back of the results impedes progress towards scientific breakthroughs corrupts the medical literature and waste research funding now waste research funding was something of course that was brought to our attention when the original paper was posted back in 2012 if over half of the trials that were funding are not posting their results either at all or in a timely way that arguably represents money that has been lost that could have been given to another investigator to do something else so this is a problem that results of trials that we are funding are not being published are posted either at all or in a timely way now we have another problem and this was pointed out by the GAO the government accounting office the the government accounting office asked us well you folks in OD you folks in building one how well do you know about your clinical trials that you're paying for so we said well you know we're the national institutes of health notice the middle word it's institutes notice the last letter it's s and each institute oversees their trials and they said yeah that's fine but how do you know what's going on across the entire agency how do you know for example whether or not a trial that's being funded by one institute doesn't overlap substantially with the trial being funded at another institute how do you know that if there's a problem with the trial in one institute that you can't potentially deal with similar problems that may be occurring with trials in another institute so they basically dinged us and said that we have not we're not collecting the data that we need in order to be able to function as effective stewards of of clinical trials particularly when clinical trials represent about 10 percent of our overall of our overall budget so effectively we have got two problems one is that we don't know what's coming out we fund the trials they get done but then the results don't get published at all or in a timely manner and then secondly the GAO says we don't even know what's going in because we don't collect the data that we need in order to monitor what's going on across the across the spectrum of trials so hundreds of people have been working on this for many many years and this so this goes back and it starts in 2012 Kathy Hudson and Kerry Wallin at that time in building one played a major role in pushing this forward back in October of 2014 we put out a revised definition of a clinical trial which was actually done in coordination with the development of the new common rule my predecessor Sally Rocky posted a blog in which she urged people to look at a proposed rule the proposed rule would essentially say that if the NIH funds a trial whether whether it deals with an FDA regulated product or not if the NIH is funding a trial the results have to be posted and the results have to be posted within a year of completion and then in order to know whether or not the the results are there the trial also has to be registered and the registration has to occur within 21 days of first patient enrollment at the same time Francis and Kathy posted an article in JAMA in which they talked about the concerns that they had that results of clinical trials were not being reported out in a timely manner so we put out a this proposed rule and we requested comments this was out for public comment from November of 2014 through March of 2015 we received 240 comments these comments came from individuals, institutions, professional societies and then after going through these comments and largely under the leadership of Carrie Walinetz in the Office of Science Policy the policy was issued this policy was issued in September of 2016 and was posted in a number of places in there it is stated that a fundamental principle of all NIH funded research is that the results must be disseminated and this is particularly true for our research involving human beings because they are scientists have an ethical obligation to ensure that the burden and risk that volunteers assume comes to something at the very least by ensuring that others are aware of the study and that its findings contribute to the body of general knowledge so ensuring that others are aware of the study that means registration and that the findings contribute to generalizable knowledge means that when the study is completed the results get reported out and that they get reported out in a timely way and there was a lot of publicity that went along with this Francis Kathy and I posted a piece in JAMA there was another piece that appeared in the New England Journal I believe on the same day Carrie Walinetz and again I have to say Carrie to a large extent really led this effort and I posted we posted a series of blogs about building better clinical trials through stewardship and transparency and then we also posted the the actual policy itself within the NIH guide so the policy actually became effective in January of 2017 we're in the midst of a of a continuing rollout the main point of the policy is that we need to know what clinical trials were actually funding and by knowing what clinical trials were actually funding we're then able to make sure that they're registered by knowing that they're registered we're able to assure that the results get posted and in this way we can go from a state of existence where less than 50 percent of our trials have their main results reported to hopefully we're 100 percent of our trials have their results reported we have posted on our website a lot of information about the the new clinical trials policies and this is just the cover pages there's a lot in here there are there are their references to various articles their videos their decision trees to help people out as well as contact information frequently asked questions and I'm going to just end this by showing you two newspaper headlines I think I started with a newspaper headline I'm going to end with two newspaper headlines so the headline on the right this is an NIH news release and the headline says HHS and NIH takes steps to enhance the transparency of clinical trial results the headline on the left which comes from a mainstream media outlet says a surprising amount of medical research isn't made public that's dangerous now what's interesting about these two headlines are the dates when they appeared so the the one on the right about HHS and NIH taking steps to enhance transparency that appeared in November of 2014 so three and a half years ago whereas the the one on the left about a surprising amount of medical research not being made public that appeared in august of 2017 that appeared just a few months ago now to be fair the the people who wrote the article on the left said the NIH is actually trying to do something about this the NIH now has a new policy in place at least for clinical trials to assure that the that the results are made public although whether or not the that that is going to work is something that only time will tell but ultimately the the goal of this is that we need to be accountable to the public and and at the very very least the public should know that the results of the studies that we fund will yield results that become public they may not be true or they may not be the best but at least they'll be there so that so the public knows what what it's paying for so i hope you found that i've talked about two topics i hope you found them interesting i'm happy to talk about anything i mean there's a lot of stuff going on right now so one of which being that the government is open so i'm you know i'm happy to answer questions on this but if there are other questions i'd be happy to take those as well thank you mike i'm sure there's lots of things people want to talk about if there's like you said you covered two topics both of which we've discussed at this council recently but there might be other things you want to talk about as well so carol so thank you very much so one of the you know something that's really key for transparency is actually findability of information and and one of the challenges to looking for things in clinical trials dot gov is that people aren't required for example to conform to standardized gene nomenclature so you'll have people publish studies about her too which is not the official gene symbol and er bb2 and you unless you know at at your fingertips all the synonyms for those genes it's really really difficult to search through this compendium of clinical trials and find the relevant ones so i'm wondering if that that aspect of making these studies transparent that is the metadata standards that allow people to retrieve information in a reliable way are being addressed in these guidelines yeah so that's a really great point and one part of this which i didn't mention is to modernize clinical trials dot gov and clearly you know having data which is findable and comparable is an important part of this in order to make this this data resource as useful as possible so building one is putting money into clinical trials dot gov because there's a recognition that it needs to be modernized it needs to be modernized in order for us to implement these policies but it also needs to be modernized i that that's a really good point thank you well i'm sure you've probably been asked this question but it seems like the issue is there were a lot of clinical trials where the data wasn't reporting and now you reported and now you're going to have like a lot more clinical trials in clinical trials dot gov so is there really like is there a staff and an office now that a is going to help all these new investigators who never used it before and is going to really try to um like it seemed to me it would have been better to make sure the ones that were in there were actually reporting the results as an investigator most of us just think like okay now we're going to have to register trial register grants we didn't have to register before but how is that really going to help get the results um so i guess there are a couple of parts to that so one is we did go through a period of time before when the number of trials that were registered in clinical trials got got dramatically increased it starts in 2005 when the the medical editors made their policy statement then in 2007 with the with the FDA law so we've gone through periods before when there has been a substantial increase and yes this is a major priority for NLM and it's a major priority for NIH and it's recognized of course this takes time and people then the second part has to do with the compliance of the of the reporting of the of the results and that's something that we are working on now the good news is we have a little bit of time because the policy is primarily affecting the primarily affecting trials that are being funded around now and it's going to be a few years before those trials have their results but there are there are a lot of people who are working on this because you're absolutely right this in it's one thing to say it but what we ultimately what are going to look for in the ultimate metric will be what proportion of trials actually do have their main results reported out and we want it to be well above 45 percent two questions for the second half of the talk do you have data on what fraction of NIH grants results and publications so presumably there's is also going to be a large fraction that don't and then the second question is going to be for the first part of the talk what fraction how much of that those trends are driven by changes in the demographics of the PI eligible workforce if any okay so the first question is what proportion of NIH grants yield publications and it's over 90 percent yield at least one publication there is a great deal of worry obviously when a grant yields zero publications of course the first concern is maybe we missed them you know maybe there actually is a publication somewhere and we didn't see it but there there is we are seeing evidence that there are some grants that yield no publications at all including a publication that could say well we didn't find anything so that's that's an interesting question that's an interesting metric that we're looking at yeah so it seems that now publications from grants in general doing actually much better than clinical trials do you have a sense of why why clinical trials are doing particularly badly in the past have been doing yeah so that's a great question and you know if we had that study to do all over again we would have included some systematic questions about why did it take so long to get your results published or why didn't you get your results published because i have to say we were or i must confess we were so convinced that everything was going to be fine that you know we didn't bother with that now we did we did hear a number of anecdotes but certain anecdotes that we heard over and over again one was that it took after two or three journals and long periods of time and we and the studies were rejected we just decided not to bother anymore but we heard some more concerning ones about 10 of the studies the the authors never analyzed their results now the reason why we know that is because we actually asked each author what were the results of your trial because we wanted to see whether or not there was a relationship between a positive result a negative result and the rate of publication and so for the the trials that had not published we asked the authors could you tell us what you what you found and 10 percent told us that they actually didn't know okay now for your your first one about the general demographics that's a very interesting question probably the person who's done the most amount of work on this is Bruce Weinberg who was an economist at Ohio State University Bruce was actually funded by the NIA and his area of interest is what happens to all of us as we get older he published a paper last year in which he looked at the demographics of the general scientific workforce and he compared that to the demographics of the country as a whole and the general labor force and what he finds is that the scientific workforce is aging much more rapidly than the than the general general labor workforce there are a whole host of explanations about why that might be going on and he did some very sophisticated modeling it appears that perhaps the most important contributor was the end of mandatory retirement in in the mid-1990s and so that plus a marked reduction in retirement rates among our current scientists maybe the reason why there is such a marked increase in the aging of the scientific workforce compared to the workforce in general so we have we have one question from a gale who's attending remote lead gale or do you have a question okay and really her question had to do with the expansion of the definition yeah so we we've heard this and the the definition really has not expanded and if you look at what has happened you know what the definition was in 2014 I think what perhaps what what what people are reacting to is the same thing we reacted to which is that when we thought about clinical trials when we did our study back in 2013 we were thinking about the trials that have hard clinical endpoints like death or heart attack or stroke most of the trials that are funded by NIH over 80 percent of the trials that are funded by NIH focus on endpoints that aren't clinical so a change in ejection fraction or a change in a lung function or a change in brain blood flow these are not clinical endpoints there there are certainly of great interest scientifically and they may be of interest to clinicians who think about this kind of thing but we included those in that in the definition back in 2014 and in fact in in our study that that we did we found that over 80 of the trials that we looked at which were already registered by by their scientists were not trials that focused on on clinical endpoints Wendy and then Jeff and then so just to continue on Jonathan's question a little bit more about the demographics if you look by gender or race ethnicity beyond just the age is there anything and then a second question which maybe is provocative maybe not if the goal is to actually inform participants about the results of these is there any movement to actually do this in a more meaningful way to them so in other words to put it in patient participant friendly language or present video summary results or anything along those lines yeah that that's a really great question and perhaps Eric I don't know if you've talked about this but what's happening with all of us the precision medicine they're they're really working on this they're they're trying to take this to a whole new level some of the CTSA work is also focusing on that on bringing people and having to be more engaged with their research I'm also involved with PCORI and this is something which for PCORI this is a very big deal is engagement on a very selfish level part of the ideas is that if people are more engaged actively in in the research that they're involved with then they might show up and they might they might be more likely that they'll be retained and stay with they won't be dropouts to follow up and also maybe they'll tear their friends and neighbors you know this is a great research is great and you should become part of it now your first question deals with some other very interesting issues there is a great deal of concern about underrepresented minorities in science a number of years ago there was a provocative paper that was published by Donna Ginther that suggested well showed that African-American applicants are less likely to get funded than white applicants everything else being considered equal part of the reason why we developed an office of scientific workforce diversity within Building One that Hannah Valentine is leading is in a reaction to that is a concern that the that the workforces is not as diverse as it could be we see evidence for example and you probably know this that women now make up the majority of PhD students and yet they certainly don't make up the majority far from it of full professors or associate professors and there's clearly more going on than than just timeline or pipelines so there is a there is a great deal of concern about this and this is something which a number of people are working on one of the things that you know we have to make sure of is that any effort that we make to expand the scientific workforce or expand any particular part of the scientific workforce is is not having unintended consequences on on the on the makeup of the scientific workforce Jeff and you may have already addressed this sufficiently the question was really how many of these studies are failed clinical trials does the NIH have any understanding about how many failed to achieve their recruitment goals yeah to achieve any meaningful results to be reported yeah so that that's a very good question and now these studies were studies in which the trials were actually completed which means that the real picture may actually be worse because we know that a certain set of trials don't get completed they failed to meet their enrollment targets I think that the best study of that that has been done was a look at the National Cancer Institute trials that just goes back five six years that suggested as many as 30 to 40 percent of their trials did not meet enrollment targets they're not even come close to meeting their enrollment targets so many of the reforms that they have undertaken have been to help deal with that there's been some very interesting systems engineering work that was done by David Diltz and others that showed that in many cases you can predict the trials that are most likely to fail by what happens within the first six months if within six months they have not reached certain milestones the likelihood that they're going to somehow recover and make it to to full enrollment becomes very very low and so a number of the institutes now are using this kind of information to studies off early there there are some institutes that are using something called a U3 U3 U4 UH UG UH UH2 UH3 I should know this so the the idea of this is is it's like a two-phase grant the first grant the first part of the grant typically lasts a year to a year and a half and the idea is is that this is a time when you get your trial set up and then the second part of the grant is where you actually do the do the enrollment and do the study um what we what we then look for is at the end of the first phase of the grant is how successful have you been do you have your sites up you have all your IRB approvals is your protocol done do you have your staff trained how about your your initial your initial pilot screenings have they worked well and this gives us a good sense as to whether or not this study is likely to be successful and if we see that that things aren't going well we can then stop the grant at that point without there being a a big to-do about it um and then we have money available then to fund something something else mark uh your next generation research initiative focuses on early career investigators do you have plans to do something for the mid career ones who are getting squeezed yeah so that is that is questions absolutely spot on and so um one of the points that uh our working group has made is that we need to pay attention to people who are in the who are established investigators typically mid-career investigators and who are at any given time one grant away from losing all of their funding about 60 to 70 percent of our researchers are living on one grant um and uh and that means that if they're if they fail their next competition they might lose everything somebody's got two or three grants if they fail a competition it's it's not a good event but but there there's still some resiliency so one of the things that we're we're looking at and this is still very much a work in progress is can we highlight investigators who are at risk for losing all funding and yet are doing great work um this is what we're particularly worried about our I guess you could call this an involuntary loss somebody who is doing good work their grants are productive they submit either a renewal or they submit a new grant which gets a good score a score which in the old days would easily have been funded and uh and yet you know they're they're at risk for being pushed out so we're we're looking at ways to do that now that's a really important any other questions thank you Mike thank you very much we appreciate that update yes