29 Comments
The numbers for fiscal year (FY) 2012 are in. Here are some facts about applications and awards in FY2012, compared to FY2011:
| 2011 | 2012 | |
| The overall success rate for research project grants (RPGs) stayed the same compared to 2011. | 18% | 18% |
| The average size of RPGs increased. | $449,644 | $454,588 |
| In 2012, there was an increase in the total amount of funding that went to RPGs. | $15,815,319,592 | $15,923,746,065 |
| NIH received more R01 grant applications. | 28,656 | 29,515 |
| Success rates for research using the R01 mechanism remained the same | 18% | 18% |
| The number of R01 awards increased. | 5,264 | 5,340 |
| NIH received more R21 grant applications. | 13,145 | 13,743 |
| Success rates for the R21 mechanism increased. | 13% | 14% |
| NIH awards for the R21 mechanism significantly increased and reached the highest number of awards ever. | 1,694 | 1,932 |
| The success rate for center grant applications decreased. | 37% | 33% |
| The average size of a center grant increased. | $1,863,037 | $1,914,070 |
| Success rates for SBIR grants increased (Phase I success rates shown here) | 11% | 16% |
| The number of research grant applications received by NIH increased and reached the highest level ever. | 62,267 | 63,524 |
This 2012 data, and data from past years, can be found in the NIH Data Book. This is the first place to look for summary statistics on NIH awards — data and charts are exportable for easy incorporation into reports, presentations, or your own blog posts.
Looking back on these data, the first thought that comes to my mind is, “We made it.” Despite a flat budget and complex fiscal times, we maintained last year’s success rate and slightly increased the amount of award dollars that went to research project grants. We continue to strive to maintain a diverse portfolio of biomedical research, and keep this important work moving along quickly.
Even though the current fiscal year remains uncertain, we know that above all, it is critical to support your continued work on innovative science. Stay tuned for more data.
This is confusing. The data shows that 18% success rate for RO1s in 2012. But, the payline percentlie of the funded proposals was much lower. For e.g., NINDS payline was 14th percentile and NCI payline was <10th percentile. How the 18% success rate translates to funding percentile?
I believe the success rate is per application, regardless of how many times it has been submitted. For example, a resubmitted application only counts once in these statistics. The relatively new metric of success rate used by NIH actually does not correlate to payline or success rate per submission. So even if each submission has less than 10 % chance of funding an overall success rate of 18 % could still be achieved.
Another thing to consider is that R01s as part of RFAs probably get funded at a higher rate and bring up the average percent funded when combined with R01s that represent open calls.
Well, I know that many institutes are using one year bridge awards to grants outside of the payline to keep labs open in the current disastrous environment (I know first hand, I got one and so have many of my colleagues). Since the system counts that as a successful application, that would at least in part explain the pay line versus percentile difference, and also resets the number of submissions needed for funding as well.
The only way I see to reconcile an 18% success rate with a 10% payline is to assume that the remaining 8% represent grants that are funded outside of the payline….the grants “brought up” by program staff. If true, this would suggest that over 40% of the funded R01 pool represents “outside the payline grants”. Perhaps Dr. Rockey could clarify this.
Dear Dr. Rocky, I read the numbers you use to describe funding with great interest as you might imagine. However, i do not understand how you calculate that RO1s are being funded at a rate of 18% when everyone is saying that if your percentile score is not in the single digests right now, you will not get funded. This dichotomy between study section rankings/funding and the reports form the NIH administration of funding levels has always been a mystery to me and to many of my colleagues. Could you please clarify once and for all why there is this difference and the significance of the different values between what you use and the rest of us use.
Thank you once again for all of your help that you so generously give all of us.
Dan Rifkin
You may find the blog post on Paylines, Percentils and Success Rates to be helpful as well.
Perhaps you could provide some numbers showing how success rates are determined? I read the 2011 post but agreed with the commentators there that the reasons given simply didn’t seem to be significant enough to reach the discrepancy between paylines and success rates. Maybe offering some statistics could help us understand.
I would appreciate seeing the success rate for individual investigators. What percent of PIs who submitted one or more grant applications in the past year were successful in receiving an award?
I am not sure the statement “we made it” accurately reflects the view of scientists trying to maintain their research efforts.
“We made it” ?
The fact is that there is virtually no change from 2011 and funding remains at an all-time low.
Considering the discrepancy in positivism between the article writer and the posters it appears that a happy administrator does not necessarily translate into happy workers.
I would like to echo the comments above, and request that Dr. Rockey explain the apparent contradiction between a Success Rate of 18% for R01 grants and the single digit pay lines.
Perhaps this webpage will help those interested:
http://nexus.od.nih.gov/all/2011/02/15/paylines-percentiles-success-rates/
I think the blog is helpful – the difference is funding of revision in the same year is a success, a mix of higher %iles in R01s in response to RFAs and for Early Stage and Young Investigators and institutes funding somewhat above the automatic payline – not just the latter.
What shocked me in this analysis is the high payline and high cost of Centers – I understand that the science has to be good to warrant the huge amount of work in putting together a Center, but Centers largely are established investigators, and often long-term, potentially high impact research but if there is a move to more innovation, more cutting of centers would be helpful – in terms of publications per $$$ spent (not per PI or per grant) centers may be worse than many R01s.
If I have read and understood the relationship among percentile scores, paylines and success rate correctly, then one of the major factors that cause the discrepancy between the payline and success rate are the RFAs that do not have a permanent study section and thus are not subjected to percentile scores. With these applications figured into the overall success rate, then naturally the success rate for permanent study section applications would go down proportionately. Perhaps a breakdown of the success rate for regular study section RO1’s versus those for special emphasis or new RFAs would be more meaningful and provide a more realistic view for those submitting RO1’s to permanent study sections.
The Paylines are indeed around 6-8%, depending on section – terrible. Why NIH insists on soft-pedaling and massaging the numbers is unclear. NIH uses an “aggregate success rate,” meaning that a grant’s success is calculated after 2 (or 3) submissions. In reality we all know that the current probability of grant success at any given instant is 6-8%. The fact is that research in the US is being crushed. NIH would be better off talking to us about what their plans are to fix this.
It would be very helpful to separate unsolicited (percentiled) from solicited applications in your statements on success rates. Perhaps a quick comment about the numbers of applications received and funded under both mechanisms would also be helpful. Aternately, you might separate applications (leaving aside the SBIR and similar mechanisms) reviewed by CSR from all others.
Thanks
To answer the questions about success rates for solicited and unsolicited applications: the NIH Data Book “Success Rates and Funding Rates” section has this information for new RPGs (see: http://report.nih.gov/NIHDatabook/Charts/Default.aspx?showm=Y&chartId=157&catId=2).
Bottom line the system is broken and the stats are not represented correctly.
It is ridiculous to present this information with such optimism knowing that research in the United States is being destroyed due to short sited interests. Excellent US research labs are collapsing because they can not obtain funds to do their basic science research. In 10 years, we will look back and realize we have no new discoveries to make drugs because all of the small basic science labs were unable to obtain funding.
R01s need to be funded at 20% payline for the system to work. We are at 8-12%.
Invest in our future and fund basic research.
Dr. Rocky and her NIH administrative colleagues may have “made it”, but many of my colleagues have not. I’d like to hear about some innovative, creative ways to fix our basic research budget crisis rather than what seem to be fairly meaningless statistics. I’ve talked to several mid-career colleagues recently who are ready to throw in the towel and look to do something else if their latest RO1 resubmission doesn’t get funded. What is the NIH planning to do to keep the US competitive, and not lose a generation of mid-career scientists? I’d like to hear more discussions about these topics…forget about the statistics.
I concur, 4-5 years ago we thought it could not get worse, but it did. Many of my colleagues were denied tenure or had to shut their labs and take positions at NIH or industry due to the poor NIH grant outlook. It seems, more and more, that NIH has circled the wagons around it’s own intramural faculty and a few hand picked extramural programs. More than a few scientists I know have suggested that we may be headed into a post-NIH era for research in the US.
“We made it”
Morale at an all time low among the ranks of scientists, bordering on outright despair. You need to get out of Bethesda more often…
This disparity in numbers, scores, rankings, words, and meanings reminds me of Orwell. It is unacceptable to permit bosses, administrators, beaurocrats, or policitians to cook the numbers to serve their own spin. However, they are always good at it.
I don’t see how anyone can claim “we made it” with such funding numbers. I know I made it: right on out of research and into the private sector. With acrimony, yes, but I am much better off now. I feel a profound sense of anxiety for my colleagues. These are PhD scientists for crying out loud, and hovering on the brink of unemployment. A factor of 2 or 3 more R01’s funded per year (given constant numbers of applications) is what it is going to take to sustain the careers of those now employed. Any less and we will see enormous attrition. I’m just on the crest of the tsunami, and we won’t be able to say “we made it”, unless we made it out of there.
Unfortunately, proclaiming 18% funding rate weakens the argumentation that must be successfully conduced in Congress to increase the financial resources available to the NIH for competitive award to investigators. Especially in economically challenging times such as we experience now, and with the potential of significant cuts in important federal programs, projecting ‘success’ makes NIH makes themselves a target for additional cuts so that all units can share the pain of budget reductions. We should, rather, be presenting the dire truth on single digit success rates and the number of very fine, productive investigators ‘on the ropes’ or out of the system due to insufficient resources. NIH-funded scientists are on the unemployment line, too!
It would be nice if you would address what is going on with FY 2013 on a blog post. I had a proposal that got a possibly fundable score in June 2012. I should have gotten a decision in October 2012 but because of the continuing resolution and the possible sequester there is still no decision, a year after I submitted the proposal. I know some top scoring grants have gotten funded but those of us closer to the line are still waiting. It is really frustrating. I skipped the Feb 5 deadline because I don’t want to go through this again next October-January.
“We made it” Talk about rose colored glasses! Hundreds of labs have shut down and dozens of my colleagues are considering leaving science altogether. Every lab I know has lost half or more of their funding. We have not made it! We are dying and NIH and congress have done nothing to save the research labs. The mega labs are surviving, but not the small labs. What has worked in the past is not working now and with sequestration, many more scientists will be done. Scientists are leaving science in droves after the American tax payer has already paid for their training. What a total waste of millions of dollars. What is NIH doing now to save more labs?
I have noted that Dr. Rockey is more than willing to jump in and provide commentary regarding how NIH success rate is calculated in comparison to payline calculations. However, when it comes to the overall negative sentiment and deep despair that make up the majority of the above comments- nothing, not a word, just the sound of crickets. Perhaps apologizing for making the comment “we made it” would be a good start. I have observed several entire labs that have shut down this past year due to a lack of funding. I believe that most scientists do not place the bulk of the blame on NIH, but mostly they place it at the feet of a dysfunctional congress. What would be appreciated more, in my opinion, would be for NIH staff and administrators to come out and say “the situation is bad and we are mad as %^&&!” It wouldn’t change things, it would just be a much more honest response to a terrible situation. Even the lay press has a better understanding and outlook of what is happening to U.S. science than what was presented in the above Nexus post (http://www.huffingtonpost.com/2014/01/14/budget-deal-science_n_4592746.html).