36 Comments
Last year, Kimble et al. published the findings of a workshop held at the University of Wisconsin on proposed strategies to rescue biomedical research in the US. The workshop organizers brought to together a number of stakeholders, who concluded that the research community faces two chief problems: “Too many researchers vying for too few dollars; too many postdocs competing for too few faculty positions.” These conclusions raises at least two questions: How many scientists, or more specifically how many principal investigators, does NIH fund? And how many scientists (more specifically aspiring principal investigators) want to receive NIH funding?
Today I’d like to discuss two ways to examine these questions, by looking at the number of principal investigators awarded funding, not just on a yearly basis, but also in a way that captures a broader view of NIH-supported scientists over a window of time. NIH leadership and other groups external to NIH have looked at extramural support of the broader biomedical research workforce, and this remains an active area of interest to us. In this post however, I’d like to focus on an analysis of those individuals named as principal investigators, or individuals seeking major, independent research awards.
We first looked at all research project grants (RPGs) funded between 2003 and 2015. For each year, we identified unique principal investigators who were named on at least one RPG award in that year. Figure 1 shows that the number of NIH-supported investigators has increased only slightly, and has remained fairly constant at about 27,500 over the past thirteen years.
To determine how many unique researchers want to be funded, we identified unique applicants over 5-year windows. We chose to look at a multi-year window for two reasons: most research grants last for more than one year and most applicants submit applications over a period of time measured in years, not just 12 months, that may overlap with their periods of funding, if they are funded. Figure 2 shows our findings for applicants as well as awardees: the number of unique applicants has increased substantially, from about 60,000 investigators who had applied during the period from 1999 to 2003 to slightly less than 90,000 in who had applied during the period from 2011 to 2015.
Using these two variables we can calculate a “cumulative investigator rate,” that is: the likelihood that unique investigators are funded over a 5 year window. Figure 3 adds this cumulative investigator funding rate and shows a decline from 43% to 31%.
We were curious to see these data for R01s and R21s, the two predominant activity codes that make up NIH’s RPG portfolio. Figure 4 shows analogous data for R01 grants: the red line shows the number of unique researchers holding at least one competing or non-competing R01, while the green line shows the number of unique researchers seeking to obtain funding over a 5-year window. The number of unique R01 awardees has actually declined over time, specifically by about 5% between 2011 and 2015. Meanwhile, the number of unique R01 applicants has substantially increased. The cumulative R01 investigator rate has declined from 45% to 34% between 2003 and 2015.
In Figure 5, we show analogous data for R21 grants, which compared to R01 grants offer less money over less time. The number of unique awardees has increased over time, but so have the number of unique applicants, yielding cumulative investigator rates that are actually lower than those seen with R01s.
Finally, in Figure 6 we show findings for P01 grants: the number of applicants has increased slightly, while the number of awards and the cumulative investigator rates have declined.
These findings, when put together, support Kimble et al.’s conclusion, and the observations of many other voices in the research community: that a core problem faced by the biomedical research enterprise is an increasing number of researchers vying for limited dollars. We find that the overall number of unique awardees has remained largely constant, while the number of unique applicants has markedly increased. Furthermore, we are supporting fewer unique awardees of investigator-initiated “long-term” grants like R01s and P01s, but are supporting more unique awardees of “short-term” grants like R21s. The number of awardees receiving cooperative agreements, which are often institute-initiated, has increased.
The question is what NIH and the biomedical research community should do? Kimble et al. offer a number of recommendations, including limits on salary support, use of alternative investigator-oriented funding mechanisms (like the R35), limits on the maximum amount any one laboratory can receive, and decreases in the proportion of NIH funding dedicated to RFAs, R & D contracts, and intramural research. Pickett et al. summarized recommendations in the literature, categorizing them according to whether they are targeted towards NIH, institutions, or the scientific community in general. These recommendations focus on overall levels of funding, research regulation, compensation for postdocs, length of and funding for training, and use of staff scientists. NIH leadership is currently engaged in efforts to explore which policies or policy options best assure efficient and sustainable funding given the current hypercompetitive environment. These efforts include funding opportunity announcements for R35 awards which focus on programs, rather than highly specific projects; new models for training graduate students and postdoctoral fellows; establishment of an office of workforce diversity; and even what we are doing here, namely drawing attention to numbers of unique investigators and applicants. We realize the difficulties and challenges brought on by the current hypercompetitive NIH funding environment, and look forward to working with all stakeholders to determine how best to meet them.
I would like to thank the OER Statistical Analysis and Reporting Branch for their work with me on this analysis.
I have seen many analyses of NIH grant funding patterns over the past several decades. This is the FIRST ONE that seems to me to ring true to what I observe for myself and a large number of colleagues in multiple chemical/biochemical fields. Thanks for providing this analysis. I was on a study section in the late 1980s when GM first introduced the concept of limiting the amount of funding to any one PI/lab. I supported it then, and support it now. I also support decreassing substantiall the amount going to RFAs and contracts (and U-grants), but not necessarily decreasing support for intramural NIH researchers.
Agreed! I would like to see another line added to the graph of PI’s representing the number of PI’s with more than one R01/R21 each year. I bet this line could account, at least in part, for the disparity between applicants and awardees.
I agree with your comments. Next we should ask NIH to give us true success rates based on total applications per cycle and not after subtracting the triaged grant apps. That would be another sobering number…
Perhaps the size of awards should be evaluated; the $250k modular budget, so often ignored, may be excessive in many cases and could be reduced.
The difficult issue of indirect cost rates should be evaluated in the context of their overall Impact on the volume of biomedical research.
“Sliding funding rates” – I.e., a relation between impact score and the magnitude of funding (amount x duration) should probably be considered.
And most importantly, the goal of the effort should, be to maximize the number of investigators supported by NIH, even at the expense of providing smaller grants.
Please use colorblind friendly images. Those with red-green colorblindness will have difficulty interpreting the graphs. There is no reason why these graphs can’t be color blind friendly – use alternate colors, or use different shapes for each line.
From the Section 508 Requirements & Standards:
“1194.21 (i) Color coding shall not be used as the only means of conveying information, indicating an action, prompting a response, or distinguishing a visual element.”
Thanks for your comment. We’ve updated the markers on these graphs. In addition, source data for these graphs are available on the RePORT website, on the bottom of the “Other Special Reports” page: https://report.nih.gov/special_reports_and_current_issues/index.aspx.
Thank you for this important analysis. The current funding situation is extremely discouraging. If this urgent issue is not properly addressed, it really discourages young people from getting into the biomedical research field. We will lose a generation of brightest biomedical researchers. In addition to the potential solutions you discussed, one suggestion is to substantially increase the number and funding rate of smaller grants. In this way, a fixed amount of money will support a much larger number of research grants, keeping small labs alive. This is very important for keeping a broad and strong foundation for biomedical research. As history has taught us many times, it is highly unpredictable where the next breakthrough will emerge from. Quite often breakthroughs happen in small labs through the efforts of creative individuals. So please don’t forget to have funding mechanisms to support small labs.
This is a curious analysis that focuses on the needs of investigators seeking and obtaining awards rather than the national needs for research answers to important questions. How many researchers do we need and how important is the work they are doing?
This report is sobering and reflects the increasingly stressful conditions that are negatively impacting the academic medical community. We are victims of our own success and of an unenlightened biomedical research funding policy, and political gridlock of the federal govt. Given the importance and potential ROI of biomedical research, a greater portion of the 4 trillion dollar federal budget should be allocated to the NIH.
Thanks Mike for this excellent summary. I recently heard your ppt presentation at the Advisory council meeting of the ORWH, and was impressed by these analyses. I was wondering if you would be willing to share the entire slide set you presented there.
Thanks, Emeran Mayer,
You mentioned in one sentence that “The number of awardees receiving cooperative agreements, which are often institute-initiated, has increased.” Could you provide data for these awards, including the actual amount of funding through these mechanisms? I suspect each of such awards (U19 mechanisms) amount to more than 50 R01 awards (at the rate of $250,000 direct cost per year). Having these data would be illuminating for this debate.
The analysis makes clear the problem facing US biomedical science. We can all see the decrease in the number of qualified students entering science as they see the cost to their advisors and the promise of a perpetual fight to become functional.
The analysis tends to emphasize the role of the winners, whereas the costs to the much larger population of losers is ignored. An unbiased sampling of the cost of NIH should reflect the lost effort of talented trained PhD applicants who are not funded; their time required to ever produce useful research, the time and energy allotment of the endless hours spent preparing unfunded applications. The emotional cost to those denied funding. It is universally accepted that tenure requires having grants, so the NIH is now in charge of maintaining university teaching loads. That was not the sense with which NIH was built.
Clearly life would be easier with more funds, but more central is how to distribute the existing funds. Why should the NIH pay overhead costs on the PI’s salary? What is the role of the hosting institution? Let alone, why the NIH should pay PI’s salary anyway? It is time to cut back on university welfare and return to doing science.
YES!!!
It is difficult to view this analysis and not see (perhaps I missed) a listing of a strong contributing factor to the problem as the continuing production of a prodigious number of Ph.D.’s and therefore postdocs, driven by the pressures for productivity derived from investigator funding which in turn relies on the maximum number of inexpensive workers, i.e. graduate students and postdoctorals. There is no clear and convincing evidence that the scale of Ph.D. training in biomedicine is driven by any other factor. Couple this with the slow progress in recognizing that “alternative careers” are still viewed by many trainers as a “failure”….not on their part of course but that of their students and postdoctorals….and the problem will continue to persist and fester.
Actions taken by the federal agencies as suggested will appropriately transfer the problem to the academic institutions who will need to decide if lower salary recovery and the added cost of supporting the enterprise is cost effective. This seems to be the only way in which a Lake Woebegon setting of all institutions being above average can be addressed.
I believe the increase in the number of applicants for grants comes from the increase in the number of soft money positions at many institutions. You can have a faculty position only if you can fully fund it yourself with grants. The result is that NIH money pays for salaries instead of research and the investigator leads a stress-filled life focused on grant dollars instead of science. If these institutions value research, they should dedicate salary lines to it. Putting limits on cumulative salary support or requiring a percent salary support from institutions would solve this problem.
Decreasing the pipeline of researchers-in-training would probably be most effectively handled by reducing the number of visas given for graduate study and postgraduate training in the U.S. Of course the quality of global education and international ties would suffer.
This discussion begs the question of whether the research enterprise *should* shrink to fit the current funding level. If our society valued research the way it should, both government and institutional support would increase.
Scientists are caught in the same loop as are politicians: the job of getting money to keep their jobs. Scientists endlessly write grants, just as politicians endlessly raise campaign funds. No one has time to actually work. My university employer is and always has been teaching-intensive but, to help make ends meet, the admin is preparing to ramp up grant-seeking. Simultaneously the state government is actively weakening tenure, such that we could all be on soft money eventually. It’s unsupportable.
Very true and a sad situation overall.
The quickest change to increase availability of funds to support more investigators is to cap indirect rates, with a recommended maximum rate of 50%. Researchers have been “tightening our budgetary belts” for a long time. However, there does not appear to be any effort to similarly contain research administrative costs. Institutions that receive grants should have to live within their budgets just like the rest of us.
I had a conversation with my program officer and he indicated that if the NIH receives a budget boost, the schools around the country will simply be expanding (e.g. building more research buildings, hiring more administrators, etc.) because these “activities” will increase the overhead rate. This is one of the most critical problems we face. If there is political resolve to cap overhead rate at say 40%, that will free up more funds to support real research. I also echo the comment above regarding institutional salary support. You will not ask an English Professor to raise his/her own salary support. Why should we do that for faculties in biomedicine?
If you wish to make what an English Professor does, be my guest.
This is fantastic that you are finally focusing on the people/careers reality behind the grant applicant stress.
Can you next turn to analyzing the entry/exit stats? Turnover from year to year is a key measure. Of those 27,500 PIs, how many come in and out of funding? What are the distributions of those funded for 5, 10, 15… years? These analyses should help with setting policy to stabilize careers on the one hand while still allowing fresh new ideas into the system.
(And of course you should be doing these analyses by IC, btw.)
I saw a presentation by Toni Scarpa (former director of the Center for Scientific Review at NIH) about 10 years ago. At that time, among PIs who had received NIH funding of some kind in their careers, the average TOTAL duration of NIH funding was just over 7 years.
I do not know what the average number of “funded years” is now, but I suspect it has not increased.
Both the NIH budget and the direct costs ($250k/year) have been flat. Adjusted for inflation, the purchasing power has been lower than 10 years ago. However, indirect/overheads levels have been steadily increasing (to nearly 100% of direct and greater)! Much greater scientific benefit will be reaped with a revaluation of indirect costs.
During the period 1990-2016, the US real GDP increased nearly three-fold. During the same time, the real NIH budget increased by about 50%. If the US government would commit to a constant proportion of GDP being devoted to civilian research and development, the problem would largely be alleviated.
Not that I’m holding my breath for that to happen.
You are really describing frequencies not rates. However the message is still the same. Our system can not survive if only 15% of the approved applications are funded. To make sure that iexisting labs in important research areas don’t completely disappear consider proportional funding to keep these labs “alive” to try again
The ICs already use R56 and exception funding to keep some labs alive to try again. With a stable number of funded PIs, as per the graphs posted, this just prevents newcomers from entering and bringing novel ideas and approaches to the table.
Thanks for performing this study, confirms what those of us in the trenches all know already, but always good to have the data to prove it.
Easy fix. Limit the number of R01’s a lab can have to 2. Those with 0 compete against those with 0. Those with 1 compete against those with 1. Those with 2 can stop writing R01’s, go in their labs, actually do some experiments and mentor some people, improve the quality of their teaching, run their labs with economic efficiency, or look for funding in the private sector or from other non-government sources. Also, divert more money into R21’s – we need to get back to exploring.
Yes, “easy”. Especially when the response will be to push those two to $499K direct. Resulting in few additional dollars available.
Do you not recall Rockey’s posts on the number of grants per PI? This has edged up as purchasing power of the modular cap has eroded. Additional grants simply restore that purchasing power that was established with the start of modular budgeting.
Several individuals have commented on this issue —- let me add to the growing concern.The issue of overhead rates must be addressed, and equally important, how universities actually spend those F&A dollars. The objective should be to free up more funds to support real research, instead of allowing public research universities (and I’m sure private too, but my experience is with a public research institution) to use federal dollars to support activities other than bolstering the research enterprise — for example salary to support fiscal staff that should be covered by F&A but is often not, subsidizing graduate education that should be supported by central and state funds but is often not, etc.
Very interesting statistical analysis. Can these data be projected on the SBIR/STTR grants? Do you have specific data on the SBIR/STTR research grants?
Thank you.
Lost in this is why the indirect costs have increased. An NIH grant application now has more than 300 pages of instructions, for a 12 pages of science grant. This is mostly due to the many new requirements and rules for accountability, and bogus “training” programs from ethics to hazardous chemicals that have been accumulation faster and faster in the the last 10 years. Always added, never taken away. All of these require more and more costly personnel for administration and over site of that administration Ad infinitum. How much more money is spent on this now? Has there been any cost/benefit ratio examination of any of this. I cannot imagine that HIPAA and increase over site of IACUCs as only two examples of many, many others could ever be worth 1% of the multi billions they cost.
It would be better not to have overhead. My argument is that it is not the PI’s responsibility to build the infrastructure. The administrators should take the responsibility. There are advantages to do this way. Because the administrators can only get money (or easier for them to get money) for infrastructure if PIs are productive, they would be more care about PIs’ productivity and use money more carefully/wisely. My impression is that Universities benefits from research grants too much, while there is virtually not much money for PIs to get things done in the lab. Because the benefits Universities get from grants, it is natural for them to hire more and more researchers, which makes low funding rates and bring misery to PIs. I believe that a lot of talents and life are wasted in the current system. It needs intelligent, responsible and hard-working people to make the system better (I do not imply the current people are not doing a good job, it is just a so difficult problem).
The findings and the discussion is overdue. Clearly, indirect rates (mostly profit) is low hanging fruit for significant pruning. The institutional drive for research dollars (via salary, or promotion and tenure) leaves clinical data sets often under-analyzed in pursuit of the next grant. This issue may motivate the suggestions regarding capping award numbers per investigator or lab, and perhaps limiting total federal support for an individual’s S&F, as well as expanding support for secondary analyses of existing datasets. Funding business research projects, and academic ones that become “marketable” may need to be reconsidered as a form of venture capital having a return to the NIH (the taxpayers).
No doubt from large policy changes to small technical funding and administrative detail alterations, discovery and productivity can be supported more efficiently and effectively.
Very interesting analysis. The other companion question is what kind of value NIH and taxpayers are getting for these investments, and is there a reasonable way to calculate ROI? Given the range of different types of studies funded across NIH, there is probably no one-size-fits-all assessment of the impact of a given research project, as some projects focus way upstream on basic biological mechanisms, some on clinical efficacy, and some on clinical effectiveness. But we to move beyond number of journal publications or post-docs as our key performance indicators and figure out whether our research is having a more substantial impact at the point of care.
This is welcome and excellent. Thank you. If you plan to do follow-up studies, may I put my vote in for a line of analysis? Many colleagues and I have the impression that there is a “rich getting richer” element to federal grant funding as the agencies’ budgets shrink. With less money and more “scrutiny” from the Jeff Flakes of Congress, the agencies can’t afford to take as many chances on potential negative findings, so there might be a bias toward proven investigators, proven institutions, and proven research models. The first two might be leading to an increasingly smaller group of researchers from a less diverse group of institutions (i.e., “R1” schools) getting an increasingly larger proportion of federal funding.
In colleagues’ rejections for federal grants I have repeatedly seen reference to the fact that the applicant has not previously held a grant like this (or otherwise has a “weak CV” for grant applications), the institution’s support is not clear, the co-PI or consultants are not experienced enough, or targeted enough, etc. Which makes sense, of course. I’ve also seen comments that proposed collaborators were too far away to be effective (e.g., in other cities or states), or were too occupied with other work. Again, this all makes sense for a funding agency doing its best to hold on to its limited funding in a somewhat hostile political climate.
The result, however, might be that “fundability” is determined from one’s academic birth–from one’s acceptance to a PhD program with a recent track record of federal funding and one’s success in obtaining an adviser who can provide training, opportunities, social/professional connections, a postdoc position, letters of recommendation for jobs at other research-intensive schools, and so forth. How else would one ever become co-PI on a federal grant, almost certainly at the same school or at least in the same city as the PI? How else would one learn how to pursue and obtain grants successfully? Workshops and speakers teaching one how to get grants tend to be focused almost exclusively on the “R1” case, as I’ve seen repeatedly; their recommendations are often disconnected from reality at any other type of institution.
I know there are new investigator grants, etc.; however, I keep hearing from other faculty that those grants are increasingly awarded to “new” faculty who come directly from the research-intensive pedigree nearly required for R01 grants.
I don’t actually know what can be done about this, because if it is happening as my colleagues and I have often perceived, then it is because of rational responses of the granting institutions to financial and political pressures placed on them.
I guess I am just hoping that these possibilities might be explored in some data analysis in a blog post like this, so my perceptions can be either confirmed or denied. Alternatively, you can just send me your dataset; I’d be happy to take a crack at it.
NIH is hiring maximum foreign MD/ Ph.Ds for their intramural research. After completing their postdoc tenure at the NIH, they are looking for grants to secure a position in US Universities and it is very hard for them to get an extramural NIH grant with a foreign MD/Ph.D degree (a statistical analysis is important). Therefore, there is no point in telling that too many researchers in the US.