24 Comments
Application and award summary data for fiscal year 2014 are now available in the NIH Data Book. These data are of particular interest for all of us this year, considering the historic low of the success rate last year, and the reduction of NIH’s budget in fiscal year 2013, due to sequestration. For this reason, in the table below, we include both FY 2013 and FY 2012 data for comparison purposes.
We received 51,073 competing RPG applications at NIH in fiscal year 2014, an increase compared to last year, but still below the highest number of applications received by NIH in a fiscal year (51,313 applications in FY 2012). We’ll continue to monitor this closely, to see if there is a true downward trend in incoming RPG applications, and to observe the effects of the new resubmission policy announced last April.
Looking at data across both competing and non-competing awards, the average size of RPGs increased to a historical high ($472,827), but in constant dollars (normalized to 1999 value of a dollar as per the biomedical research price index) it is similar to 2012 ($293,463 in FY14 versus $293,141 in FY12). In FY14, the average size of R01-equivalent awards was the highest in history ($427,083), but constant dollars it is the smallest award size since 1999 ($265,072) except for during the FY13 sequestration ($254,719). The large increase in the average size of RPG awards (up 7% compared to FY13, and up 4% compared to FY12) can be explained by the number of large collaborative infectious disease research projects that were issued. Many of these particular projects were up for competing renewals. Also, the average size of R01s, the largest component of the RPG pool, went up 6% compared to 2013, and up 3% compared to 2012.
Interestingly, there was a large increase in the percentage of RPG funding that went towards targeted research programs. Remember that in 2014 NIH launched a large number of initiatives in cancer, diabetes, Alzheimer’s, emerging infectious diseases, and the BRAIN, BD2K, and BUILD projects. It is important to note that the success rates for targeted and investigator-initiated RPG awards are primarily the same, with investigator-initiated success rates slightly higher than targeted research success rates, showing that these targeted programs are as competitive as our investigator initiated programs. (The percentage of new R01-equivalent grant awards and funding to targeted research remained stable compared to recent history.)
I’ve included a highlight of some additional numbers below, and provided the last fiscal year as well the prior fiscal year, since the 2013 sequestration was an unusual circumstance. We’ll take a closer look at more of this data later in 2015. Have fun data diving!
| 2012 | 2013 | 2014 | |
| Number of research project grant (RPG) applications: | 51,313 | 49, 581 | 51,073 |
| Number of RPG awards: | 9,032 | 8,310 | 9,241 |
| Success rate of RPGs applications: | 17.6% | 16.8% | 18.1% |
| Average size of RPGs: | $454,588 | $441,404 | $472,827 |
| Average size of RPG awards in constant (1999) dollars: | $293,141 | $279,291 | $293,463 |
| Total amount of NIH funding that went to RPGs: | $15,923,746,065 | $14,917,675,859 | $15,635,912,476 |
| Number of R01-equivalent grant applications: | 29,627 | 28,044 | 27,502 |
| Number of R01-equivalent awards: | 5,437 | 4,902 | 5,163 |
| Success rates for R01-equivalent applications: | 18.4% | 17.5% | 18.8% |
| Average size of R01-equivalent awards: | $415,445 | $402,569 | $427,083 |
| Average size of R01-equivalent awards in constant (1999) dollars: | $267,900 | $254,719 | $265,072 |
| Total amount of NIH funding that went to R01-equivalents: | $10,898,788,608 | $10,075,486,256 | $10,238,888,890 |
Does the grant size listed include also indirect costs? If so, that is not the value that we grant applicants care about.
Also, the ~18% success rates listed are far above the the payline that we see, for some odd bookkeeping reason. The actual payline-based success rate is what the grant applicant cares about.
Success rate is determined for each grant. Once a grant has obtained a number from the NIH it is counted no matter how many times it is submitted. Thus this is a measure grants that are ultimately funded (including lucky grants with single submission or revised grants after multiple submissions). So it is not the percentile funding line, which is much lower (now below 10 percentile for most ICs). Nonetheless 18% is way too low for a sustainable biomedical community. It should be 30%.
As noted in the footnotes of the NIH Data Book slides linked from the blog, average award size is calculated as total costs per year.
You might find these prior blogs helpful to clarify the relationship between success rates and paylines, and understand how success rates are calculated: Revisiting the Relationship Between Paylines and Success Rates and Comparing Success Rates, Award Rates, and Funding Rates.
Additional details on the success rate calculation and success rate data are available on the Success Rates page of RePORT.
Is the average $ amount of RPG for direct and indirect, or direct costs only?
What other RPG activity codes are included in “RO1-equivalent grant applications” ? Are numbers available for specific activity codes?
As noted in the NIH Data Book slides: R01-equivalent awards include R01, R23, R29, and R37 activity codes. Not all of these activities may be in use by NIH every year.
Visit the success rates page and NIH Data Book on RePORT for additional data by activity code.
What do you do when you have a tight budget? Well in my family, we re-evaluate our priorities and cut down on things that are less important and re-direct funds to higher priorities.
The NIH budget is roughly $30 billion. The total amount of the budget awarded to R01’s is about $10 billion (about 1/3) and to all RPG’s is $15 billion (about 1/2).
How can we (including Congress) restore the success rate to 25%? It will take $1.66 billion. (i.e. 51,000 grants x 0.25 = 12,750 grant. 12750 grant – 9241 currently funded = 3509 more grants. 3509 grants x $472,827/average grant = $1659149943 or $1.66 billion).
So NIH has a shortfall of $1.66 billion. There are many possible solutions. You may have a good ideas that would help. Here are my ideas. The first is re-directing existing funds to RPG’s (better yet R01’s). The second is to use any increase in the budget mainly for funding RPG’s until the success rate returns to a reasonable level (i.e., 30%).
However to solve a problem we need to agree one exists and we need leadership to be committed to improvements.
George, unless I have missed your basic point, you want to raise the success rate by about 1/3 (18% -> 25%). That success rate will apply pretty much regardless of the mechanism, because it’s the biomedical researchers (and research) that matters, not the accounting. So that requires a budget increase of about 1/3, which is much more than $1.66B.
Having said that, I completely agree that a hit rate ~ 25% or more is essential if we expect people to dedicate their careers to NIH’s mission, biomedical research. (And pointing unsuccessful NIH applicants to the Gates and other Foundations doesn’t really help: They already have plenty of people competing for their budgets. Their part of the pie is not being ignored.)
This number isn’t arbitrary: Something like this rate is what it takes to fund a career. Otherwise, we have a large number of struggling new PhDs who can’t pay their student loans or their mortgages (I work with some of them), and can’t sustain a focus on one long-term research topic because the funding routinely runs out. They become semi-pro statisticians or technicians, and semi-pro biomedical researchers.
It isn’t about whether to fund only the “best” science: I really DON’T want only Isaac Newtons and Louis Pasteurs to be competitive, and to be able to spend their careers on this research. That’s because I don’t want to wait 200 years for all that “great” science to trickle through society. Fund lots and lots of very good science, and cure heart disease in 40 years, instead!
Either (A) Congress agrees that this pie is significantly too small for what the US needs, or (B) it collectively concludes that we have significantly too many biomedical-research careers compared to other national financial priorities. The evidence points to (B). No one there will voice that position, but it really is the observable behavior of that creature.
George, Most direct costs go to salaries for personnel. Large grants have lots of personnel. Often the personnel to fill those spots are not available so we look international-and are not too picky–we put out (LCA) visa requests by the hundreds. The current estimate is about 65% of the workforce. If we (had) limited that, the size of the grants would have (had) to diminish, and there would be a requirement for greater cooperation or ingenuity. It would (have) require(d) a different style of science-perhaps better focussed. The way it has been is for salaries to go to mega labs to fund these salaries/grants over small labs that have far more accomplished scientists working on significant problems with far greater impact per scientist. Unfortunately, administrators/managers favor big science over small science because the administrators like to be generals of an army fighting a big war that they can never win (that way). Look at your own institution, and you will see this.
I agree with Walter — something is amiss in these calculations when we observe that the success rate for truly investigator-initiated proposals (that is not including those that may be responding to for instance an RFA which tends to be only for a narrow pool of applicants). Perhaps you could provide us with non-RFA, non-PA, non-supplement to already funded research data, to allow us to really assess the situation.
And does “size of RPG” mean the amount for one year or for the whole grant including all years?
It is total costs per year (including direct and indirect). Remember RPGs include all research (not training) grants, so the denominator includes a lot of small R21 and a few large PPGs.
I think the success rate for R01 equivalents (18.8%) includes multiple resubmissions. Thus, whether a grant is funded after one submission or three is counted equally. Yet it makes a huge difference for PIs if it takes 3 years to get a single grant funded. Given that the paylines for most institutes are well below 18% and can be as low as 6-7% for NCI or NICHD, these numbers are misleading. A more accurate representation would be the actual paylines. Can we get a chart of the funding paylines for each institute as a function of time?
I agree. However if the success rate goes up then more grants will be funded on first submission.
Also of interests is the percent of the NIH budget that goes to investigator funded R01s. This seems to be decreasing as more money goes to defined projects, contractual agreements, and other initiatives. There ought to be more debate on whether this is a good strategy.
Abut $10 billion of the $30 billion budget goes to R01’s or about 1/3.
With respect to the other commenters, I think that a success rate goal of 30% is arbitrary. How on Earth is an optimal success rate calculated? There has to be some high degree of failure built into the system in order to choose the best science to be funded. But the amount of work that goes into applying for grants, and the lost productively in funding lapses is a bigger problem. I think the PhD training needs to be re-evaluated so that PhD’s graduating do not increase the pool of NIH RPG applicants exponentially like it has, but instead cultivates a knowledgable workforce in non-NIH-funded (public and private) areas. This is long-term cultural shift that NIH hasn’t a whole lot of a control over. Secondly, the questions these other commenters are asking could be answered by spending a couple hours on NIH RePORTER.
Please look at the link above on Number of RPG submissions which includes historical success rate. You can check it out at the NIHReporter
http://report.nih.gov/NIHDatabook/Charts/Default.aspx?showm=Y&chartId=124&catId=13
The success rate of 30% is historical and was the rate from 1998 to 2003.
I agree that only the best grants should be funded.
However, are study sections powered to discern which are the best 5-10%?
Also check out slide 20 of this ppt for a longer view on success rates, particularly note the rates for experienced investigators.
http://grants.nih.gov/grants/new_investigators/Workforce_Info09072007.ppt
E-Rook, the easiest way to fix this is to cut back on NIH funding of faculty salary. Universities have been the prime drivers inflating the application pool.
Grad students get a good deal and a useful education at most schools, and experience that is generally welcome in a diverse set of jobs. They also provide full time labor for many smaller universities. Cutting off their training would just hurt the base of the broader scientific workforce.
You could put the choke point at the postdoc, via restricting fellowships and implementing training programs that could control the levels. Seems like that would require a lot more careful shifting of personnel, though.
I wonder whether “targeted” means targeted by its research topic, or also include grants that are open to select applicants? E.g. various DP-type grants. I would imagine that these awards are relatively large and have fewer applicants. Would be great to see breakdown of the data by award type: a simple table, showing activity code, number of applications submitted, number funded, average award size.
As described in the footnotes on the graph: Targeted Research is research funded as a result of an Institute’s set aside of dollars for a specific scientific area. Institutes solicit applications using research initiatives (RFAs for grants, RFPs for contracts).
The footnotes on the NIH Data Book graphs linked from the table include this and other information you may find helpful, such as the activity codes included in the analysis.
In reply to the questions about how average award size is calculated: As another commenter noted, average award size is calculated as total costs per year. The footnotes on the NIH Data Book graphs linked from the table include this and other information you may find helpful.
Agreed that paylines and success rates are too low. Reviewers also need to really think about how they assign impact scores. Too often applications seek to keep a successful research enterprise going by parsing off increasingly specific takes on the established work, that don’t appear to lead to substantive improvements in human health. The review committee’s eyes, glazed over by the long list of prior NIH grants supporting the work tend to question the value of this new application less than they should. And so it goes.