In December 2018, the NIH Advisory Committee to the Director (ACD) offered a number of recommendations to NIH on the “Next Generation Researchers Initiative.” Among those: The Committee recommended “special funding consideration for “at-risk” investigators. These are researchers who developed meritorious applications who would not have significant NIH research funding if the application under consideration is not awarded. We plan to draw more attention this year, both inside and outside NIH, to outcomes for at-risk investigators, to ensure those submitting meritorious ideas to NIH are not lost to the system.
We agree with the ACD on the need to support these researchers and are identifying strategies to call attention to both early-stage investigators and at-risk investigators who are designated on a meritorious grant application that did not get funded.” Here’, we present some data on our progress towards achieving this goal.”
To answer this question, we will focus on the “funding rate.” This is a person-based metric, as described before, representing the proportion of those scientists who are successful among those applying for an NIH grant.
Funding Rate = (Number of Unique Awardees in Fiscal Year)/(Number of Unique Applicants in Fiscal Year)
We analyzed outcomes for applicants who submitted Type 1 (“de novo”) R01-Equivalent applications in fiscal years 2016, 2017, 2018, and 2019. R01-equivalents follow the expanded definition (2019) which includes activity codes R01, R23, R29, R37, R56, RF1, DP2, DP1, DP5, RL1, and U01, OR activity code R35 with RFA/PA Number GM16-003, GM17-004, PAR17-094, PAR17-190, or HG18-006. Not all activities are in use each fiscal year (FY).
Table 1 shows the number of unique applicants who submitted Type 1 R01-Equivalent applications according to career stage. The career stage categories are early stage investigators (ESIs), other new investigators, at-risk investigators, and established investigators. These categories are meant to be exclusive of each other – in other words, all applicants are classified as being in one, and only one, of these career stage groups.
Table 1: Number of unique Type 1 R01 applicants according to fiscal year and career stage
|Fiscal Year||Early-Stage||Other New||At-Risk||Fully Established|
Career stages are defined as:
- Early-Stage means no prior support as a Principal Investigator on substantial independent research award and within 10 years of a terminal research degree or end of post-graduate clinical training; A substantial research award is a research grant award excluding smaller grants that maintain ESI status (see here);
- Other New means no prior NIH support as a Principal Investigator on a substantial independent research award, but not considered Early-Stage;
- At-risk means prior support as a Principal Investigator on a substantial independent research award and unless successful in securing a substantial research grant award in the current fiscal year, will have no substantial research grant funding in the following fiscal year; and
- Fully Established (or later just “Established”) means current substantial research support with at least one future year of support irrespective of the results of the current year’s competition(s).
Figure 1 shows our main findings – the funding rate according to career stage. The numbers beneath the X-axis reflect the same data shown by the height of the bars. In 2016 and 2017, the funding rates for at-risk investigators were lower than for early-stage investigators (22% or 21% vs 24%) and much lower than for established investigators (22% or 21% vs 29%). The picture changes in 2018 and 2019. In 2019, the funding rate for at-risk investigators had increased to 27% (almost equivalent to that for early-stage investigators), which was only slightly lower than the funding rate for established investigators (at 31%).
Throughout fiscal years 2016-2019 though, at-risk investigators, for whom the stakes are arguably greater, had lower funding rates than established investigators. Why? Three possible explanations include:
- At-risk investigators submit fewer applications (effectively getting fewer “shots on goal”);
- At-risk investigators are less likely to get an application to the discussion stage in peer review; and/or
- At-risk investigators receive worse priority scores for those applications that make it to discussion in peer review.
Table 2 shows the number of applications submitted by career stage in FY2019. The difference is modest: among at-risk investigators 84% submitted 1 or 2 applications (meaning 16% submitted more than 2), whereas among established investigators 81% submitted 1 or 2 applications (meaning 19% submitted more than 2).
Table 2: Number of applications submitted by career stage in FY 2019
|Number||Early-Stage||Other New||At-Risk||Fully Established|
|4 or more||3%||2%||7%||8%|
Figure 2 shows the scientist discussion rate which is defined as the percentage of applicants who have at least one application discussed in study section. In every fiscal year, there is at least a 10% difference between at-risk investigators and established investigators. In 2018 and 2019, 59% and 60% of at-risk investigators saw at least one of their applications make it to peer-review discussion compared to 71% of established investigators.
Figure 3 shows box plots of the distributions of best priority scores for those applicants who saw at least one of their applications make it to the discussion phase of peer review. The priority scores for at-risk investigators are only slightly worse than those of established investigators.
In summary, like with early-stage investigators, the picture for at-risk investigators has improved somewhat in 2018 and 2019. Still their funding rates are lower than those for established investigators who are assured of continuing funding for at least one more fiscal year. One possible explanation is that the peer review system is more likely to send applications of established investigators to the discussion phase.
This is an ongoing analysis – there is more to come. But we are seeing both positive and concerning signals regarding this critically important group of the Next Generation Researchers’ Initiative. One promising signal we have heard about that may help this group is bridge funding opportunities being offered by an investigator’s home institution that provide a year of funding if an at risk applicant gets a score that is close to the payline while they continue to submit applications. Through programs such as these, bridge funding from NIH Institutes and Centers (see this post), tracking trends of funding for investigators at all career stages, and monitoring for unintended consequences, we will continue to pay close attention to the types of investigators we are supporting.
I am most grateful to my colleagues in the Division of Statistical Analysis and Reporting for their work on these analyses.