30 Comments
We all know that NIH has seen a large increase in applications over the past decade, but how much of this is due to scientists writing more applications and how much is a result of a larger number of scientists doing biomedical research? I decided to take a closer look at this question, particularly at competing applications for investigator-initiated research project grants (RPGs), i.e., those that are not submitted in response to a specific request for applications.
First, we can see in figure 1 that the amount of money (in direct costs) requested in such applications has increased from $4.4 billion in fiscal year 1998 to just over $13 billion in fiscal year 2011, while the amount of money awarded to competing applications during that time increased from $1 billion to around $2 billion.
Looking at these data as a ratio (figure 2), the requests for dollars rose from 3.6 times the supply in 1998 to 6.5 times the supply in 2011. This ratio has been between 5 and 6 for the past 5 years.
What contributes to the increase in the amount requested? First, the total number of these applications has almost doubled—from 25,000 in 1998 to almost 50,000 in 2011 (see figure 3). The average number of applications per applicant has also risen slightly (from 1.3 to 1.5) and that contributes somewhat to the total increase in demand, depicted by the red portion of the bars in figure 3. However, the major contributor to the increased demand is a large growth in the number of applicants—from about 19,000 in 1998 to approximately 32,000 in 2011. This contribution is depicted by the blue portion of the bars.
These results are not unexpected, but it is interesting to see the actual numbers, and this information helps define the biomedical research enterprise that interacts with NIH. We’ll be delving a bit more into this in future posts. I won’t give away the content but will just say: stay tuned!
What proportion of the new applicants are soft-money applicants? How does this compare to 1998?
I’m wondering why the number of applicants has increased since 2009. I know there has been some improvement in the economy–but university funding and federal research funding have been worse than flat. What would explain this sudden turnaround?
Wait–that’s the effect of ARRA money, isn’t it? If so, it seems likely that ARRA actually made success rates WORSE. That would be interesting, if it’s true.
To clarify, ARRA money was excluded from this analysis.
But it is still very likely an effect of ARRA since the perception that there is more funding available for research drives the application process.
I’m wondering why the number of applicants has increased since 2009
I would bet:
1) Traditional major-research institutions permitting / encouraging more less-than-TT-faculty to apply in their own right. Think staff scientists, research scientists, etc. Especially as the noises about too-well-funded PIs grew louder, the motivation to disperse a single (functional) lab’s grants across many “PIs” has increased. There is also the growing appreciation from postdocs that they need to have funding in hand to be successful on the job market, thereby increasing the pressure for them to seek the permission to submit grants before going on the job market.
2) Institutions with historically less interest in NIH funded activities which are under budget pressure from their traditional sources (state governments, usually) are ramping up their efforts to shake money out of the NIH.
3) The overall economic picture shutting down nonNIH sources of money- endowment returns, philanthropy, state research initiatives, smaller foundations and industry collaborations. All of these have come under pressure as well. So perhaps people who were previously getting by on nonNIH sources of funding are now being forced to turn to what appears to be the only remaining game in town.
It would be very helpful if you would post the longitudinal trends in number of applicants per year. Preferably broken down by never-funded, currently-funded and previously-funded categories. Maybe even break out the “never applied before” folks within the never-funded category.
Something is odd about your analysis, otherwise. There is a net increase in 13,000 applicants but this doesn’t tell us much about how many new people (mouths to feed) are coming into the system each year (and how many are leaving). If there are at least 13,000 applications *per year* being submitted starting around 2004. Does this mean all of these extra bodies entered the system across 1-2 fiscal years and then this pool was static for the next 8?
The red band is a bit funny as well. It obscures whether existing, well funded PIs are increasing their submissions or whether the newcomers are a larger percentage and they find themselves needing to write many more applications per year than the established folks do. Applications-per-PI broken down by years of participation in the NIH process would be fascinating.
I’m guessing that we are seeing the wave of post-docs from the 1998-2003 NIH doubling period applying for their own grants possibly in non-tenure track pseudo-independent faculty posts in the 2004-2011 period. I say pseudo-independent because of the way collaborative research projects and grant support are commonly arranged for non-tenure track research faculty. It is the soft money position ‘safety net’. Where else could the increase come from given that there has not been comparable increases in tenure-track independent positions during the 2004-2011 period? Whether this increase in applicants is good or bad for the science productivity and system quality is the real question. Decreased originality (independent and creative thinking) is probably the core problem with what I think we are seeing.
Making some ballpark assumptions about the average overhead rate, your numbers would seem to suggest only about 10-11% of the NIH budget goes to investigator-initiated RPGs in the past few years. In 1998, prior to the doubling, it was probably on the order of 15%. Fair estimate?
A few percent of the overall budget seems unworthy of mention in the larger budget context… until you consider that this is a 25-33% slashing of the investigator-initiated RPGs. A significant change in the focus / portfolio balance of the NIH extramural outlay.
Do you have the numbers that track the percentage of the NIH budget that was devoted to investigator-initiated RPGs over this timeframe more precisely? Those would be interesting to review.
I mention this in the context of the usual protestations from the NIH that they prioritize investigator-initiated Rmechs above all else and do everything they can to dismiss suggestions that translational initiatives, big science initiatives and the like have affected the core mission.
The percentage of the NIH budget devoted to investigator-initiated RPGs is shown in this chart in the NIH Data Book (as well as in a table under the Data tab.)
OK, so maybe I shifted a year in the overall budget numbers. I’m assuming an average 55% overhead rate to come up with around 12% of the budget to investigator initiated in 97-98 and a highwater mark of about 15% for 99-01. Thereafter there was a decline to 11.8% in 05 and 05 and then what looks like a recovery trend to the 13.5% in 2011.
Clearly the high of 15% was anomalous and a 12-13.5% range has dominated since 97.
thanks for linking to the data that could get us there, even if that graph doesn’t actually show the unsolicited percentages (it gives the “targeted” percentages, direct cost only it would appear?).
Interesting analysis. I would be interested to see the trends in how many RPGs each investigator had over that time as well. However, I would include inflation in any discussion of these trends. Since personnel is likely the major cost on most RPGs, that’s a simple place to start. In 2001 when I started my postdoc the salary rate for 0 yr postdocs was ~20K, and now it’s ~40K. Over the same time (’98-’12) the graduate student salary has gone up ~30-40% and tuition increased as well.
As we try and arrive at solutions to support biomedical research in this country, we need analyses like these to clearly identify what the status quo is. I read the above as an indication that the awarded dollars has (barely) kept with inflation. In effect, funding has been static since 1998, or earlier, for RPGs. In that light, the 2-fold increase in the ratio of requested to available dollars is simply explained by the doubling of applications.
I agree a more thorough analysis of the demographics of applicants needs to be done. It would appear that the applicants have not become discouraged by a lack of success, rather perhaps motivated to write more and more proposals. I do not envy those who need to make difficult decisions on the future of investigator initiated research in this country.
We need to try to attain state of sustainability with transparent feedback mechanisms– like these analyses allow. We can no longer afford to have institute directors and university administrations making all the decisions because those decisions clearly affect us all. For example, expanding non-tenure track faculty as a means for an institute to compete for grants leads to the current situation. It’s not in the best interest of the overall enterprise because obviously investigators are going to get squeezed out, and it might be some excellent scientists who have smaller, highly original, but ‘non-competitive’ labs. One simple way for the NIH to guarantee this is by paying no more than a total of 50% salary to PI applicants on grants and cutting back on overhead.
One trend that has really irked me (in the last decade or so) is the tendency of ‘big’ labs to ‘shark attack’ small labs utilizing young and ‘desparate’ investigators trying to advance in the system (eg. post-docs, non-tenured faculty). Obviously, this can not be controlled on the global scale, but a closed-system needs to have rules of scientific behavior, and training credentials. Getting rid of the desparation factor for example by immediately demanding hard money and firmer committments to investigators from institutes is a very simple solution. It also punishes those institutes who have been abusing the system. Though it may be too late.
Some US institutes borrow money to build new buildings, if NIH demand hard money from them to support PI and cut F&A, they will bankrupt. They will do what so ever to stop this from happening, even it means destroy the whole system.
Is this source data publicly available? It would be very helpful if others could analyze and write about their analyses, as well. Is there any reason not to make it public, if it’s public funding?
Perhaps what needs to be considered is to move the bottleneck back to the number of graduate degrees and post-docs being funded by Big Science and RPG’s. Less garbage in, less garbage out.
Maybe, but I defy you to figure out which scientists should be retained by admission in a Ph.D. program and those never given the chance by getting rejected. There is little correlation between long term success in the profession and application credentials once you get beyond the minimum required for success in graduate school. Creativity and passion for academic research are not measurable as part of the application process.
Also, your comment does not take into account the fact that most biomedically trained Ph.D.s will never work in academic science once their training period is done, most of the job market for such folks is in industry and government (I believe the number of biomedical Ph.Ds ever becoming a PI in the funding rat race is well below 20%). Cutting training of new biomedical Ph.D.s too much will stifle our industrial competitiveness as a country. The real problem is that mentors do not set postdocs down and give them a serious reality check about what their credentials for future success in academics. Mentors are also not honest enough about what a tough life it is with younger folks, particularly in light of the current competitive climate. The snobish culture in which academics feel that theirs is the only honorable profession for a Ph.D. is changing, but not fast enough which sends too many postdocs into unrealistic job paths. There is also not enough encouragement/supplemental training for folks interested in pursuing industrial instead of academic careers in most programs.
I’m really surprised that those making the decisions about the number of scientists to train and employ do not understand or recognize the myriad problems associated with too many scientists and limited funding. Leaders in science need to have read the classic work of John B. Calhoun (1917-1995), a 33 yr veteran of the NIH, on overcrowding of rodent cages and its application to human populations including biomedical researchers. Overcrowding causes system breakdown forces arising from psychological, physical stress and bad behaviors. Those who manage to survive dissociate themselves from the madness he called ‘beautiful ones’–but these are essentially infertile. Get it together!
-a beautiful one-
The primary problem is the quality of the peer review process. Most of us have enough integrity to recognize valid criticism and either modify or scrap the particular idea. However, if I receive unbelievable nonsense in my summary statements (“your idea is bad, because 2+2=7”, hair-rising examples upon request) then of course I resubmit. Over and over, others do the same, and your system is flooded with regurgitated applications that have, in fact, NEVER received fair and meritorious reviews.
You want to deal with the flood of such applications? Fine, here is the solution:
1. Get rid of triage, and at the same time get rid of -A1.
2. Ruthlessly and zealously treat nonsense in the critiques as scientific misconduct. To do so, make the appeal process external, or at least autonomous within NIH.
Limiting the enrollment of PhDs nationally could be a part of the sustainability equation. The number of M.D. graduates from medical schools was held at 18,000 from 1980 to 2000, and is carefully controlled. If graduate students were only funded through nationally competitive mechanisms, such as the NRSA or NSF fellowships, this would allow better regulation of the work force. Obviously the number of these national fellowships would have to increase, but this could be accomplished by shifting the dollars currently used from R01’s that fund graduate students.
coldhot3’s comment from August 15th brings up an important point that is rarely discussed in this context. The continued expansion in the number of scientists at the trough requires not only new positions but new laboratories. It was a specific policy change during the Reagan-era that allowed universities to pass through all of the costs of building new buildings into their negotiated overhead rates (before this the practice was unheard of). With this change, universities, medical schools and free-standing institutes were now at liberty to add indefinitely to the scientific workforce at almost no risk to themselves (think of it as soft salaries plus soft buildings!). Much of the instability in the scientific labor force can be traced to this critical policy change. Those who wish to re-stabilize that labor force may find it more politically tenable to go after this policy, rather than going after soft salaries. It will take a much longer time to have an effect, but it will not strand those who are now trapped in the soft-salary merry-go-round.
As a new investigator, fresh out of a T32 training program and in my first month of a tenure-track faculty position at a Big Ten university, I just need to say … I am terrified. It seems like this may be the worst moment in history to be in my position. I read articles like this and know all too well that there is too much competition and too few resources. I am also aware of the “politics,” even as a lowly junior scientist … that the system is controlled by the seniors who are growing older and fussier, less likely to retire, and enforcing a system of rules that would have killed their careers when they were in my position. For example, my new department chair told me that if I do not bring in R01 or the equivalent by my second year review, I am “history.” When I checked the records for his own funding history, he did not bring in a DIME of funding (NIH or other) until a decade after he received tenure. Even now, he has never been funded above R03. And he went through the system at a time when funding flowed like wine in Caanan. I am sorry to turn this into a therapy session but, seriously NIH, is it even worth it for me to try?!?!? I feel like I have a higher likelihood of being promoted at Dunkin Donuts than making it in this broken system.
Your department chair sounds like the devil incarnate. I suggest you ask around discretely to find out whether he’s really serious or just blowing off steam (if the latter, he’s merely irresponsible).
I’m guessing you are on a mostly soft salary too? While this career may not be as impossible as you fear, the job you took may be. Even if you get your R01 in two years, will you be stuck merely “dancing as fast as you can” for the rest of your career? Kind of hard to be creative, generous and collegial under those circumstances.
For all the posts we see about how the NIH ought to disallow payment for faculty salaries, the fact of the matter is that universities and medical schools are able to get away with soft money positions because there’s a large pool of folks willing and eager to take them. Those of us who train students and postdocs need to encourage them to think twice before running headlong onto this treadmill.
I agree. Get your RO1 and immediately start applying for hard money jobs in basic science departments and get off of the soft money, particularly in a medical school clinical department. A junior investigator who has funding is very employable outside of soft money positions as long as you are also willing to do some teaching. The so called prestige of working in a big ten school is not worth it in the long run.
Several factors contribute to the soaring number of grants. First, there is increasing pressure on faculty members to “bring in their salary” on NIH grants- hence slightly more applications per individual, and many more individuals competing to get a grant in order to survive. Teaching and service simply do not cut it any more in the more grandiose universities and medical centers. Second, the elite universities have put in place a “strategic plan” that calls for “more, more, and more”, raising their stature in the most funded list for NIH grants. Of course, this is not a strategic plan at all. The one change NIH could make to greatly reduce this problem is to remove ALL faculty salaries from investigator-initiated requests. Salaries should be available for career awards, training awards, and intramural laboratories. Removing faculty salaries from NIH grants would require universities to actually make REAL strategic decisions about their mission, goals, emphases, and areas of expertise- not just “more.” Unfortunately what we now have is faculty members wasting an inordinate amount of time spinning their grant writing wheels (often with negative results), feeling more and more pressure from the administrations that seek to rise in the prestige NIH rankings, and a virtual abandonment of serious teaching efforts by some of these individuals who realize that the ONLY thing that counts is salary money and overhead.
I’m very skeptical of the statistic that says that applications per applicant have risen only slightly, from 1.3 to 1.5 as it seems to fly in the face of the personal experience of many of us. It also flies in the face of the fact that the amounts paid per grant (in real, science-inflation-adjusted dollars), has gone down so much but the personnel sizes of laboratories have generally not.
The problem, I believe, lies first in the fact that, in calculating these ratios, the denominator is not a constant, i.e. in 2011 the denominator is much larger than in 1999. However, even this correction is not enough, because the “new” PIs that have come in since 1999 are far more likely to be single-application PIs (precisely because they are new, and therefore more likely to be young, and not having many projects going yet). This effect should greatly skew the distribution of applications per PI toward smaller numbers (in other words, the artifact arises because the system is not at steady state at a time when new PIs are being added faster than old ones are being removed).
The way to correct for this major artifact is to identify the specific cohort of applicants from the 1999 period and determine the aggregate number of applications those individuals submitted in each subsequent year. Other things being equal, this number might trend upwards due to labs getting more mature but that should be balanced by the downward trend of PIs retiring. So the null hypothesis is that this line should be flat, and any deviation from flatness will reveal the real pressure to apply for more grants.
Another variation on this analysis would be to use just the cohort of individuals who submitted successful applications in 1999.
My prediction is that this sort of analysis will show the big increase in applications per PI that most of us see anecdotally.
A lot of you have asked about the tables behind these graphs: we’ve posted these on the Special Reports and Current Issues page on RePORT: http://report.nih.gov/special_reports_and_current_issues/index.aspx, in the section titled “Report Archive”, and will be adding more in the future.