Rock Talk

Helping connect you with the NIH perspective

What’s Behind the 2011 Success Rates?

I’m going to discuss the fiscal year 2011 research project grant (RPG) success rates, and in doing so, I’m showing you a few screen shots from the NIH Funding Facts tool on RePORT, so you see how easy it is to use the site yourself to find some nifty information.

Delving further into what’s behind the fiscal year 2011 RPG success rates, we queried:

  • Topic: number of applications and awards, and success rates
  • Funding mechanism: Non-SBIR/STTR research project grants
  • Type: All competing applications
  • Fiscal Year: 2010 and 2011

screen shot of Funding Facts showing the selections outlined above

It is easy to see in the resulting hit list the reason for the decline in RPG success rates from 2010 to 2011. There were 3,609 additional applications, and 656 fewer awards in fiscal year 2011.    

screen shot of hit list showing the changes in number of awards and applications and success rates

Many More R21 Applications

Focusing on the success rates for two of the most-frequently used grant activities in the RPG category (R01s and R21s) provides a more complex picture.

screen shot of R01 and R21 applications and awards

The number of competing R21 applications increased by 17% (1,904), accounting for more than one-half of the total increase in the number of RPG applications. The increase in the number of R21 applications alone accounts for nearly all of the decrease in R21 success rates from 15% to 13%. 

screen shot of R01 and R21 success rates

R01 Success Rate Is a Result of Several Factors

The R01 picture is more complicated; the increase in R01 applications accounts for only 1-point of the 4-point drop in the R01 success rate. The remainder of the decrease is accounted for by a combination of several factors.

The NIH budget was reduced by 1%

As you recall, the fiscal year 2011 budget was reduced by 1%, resulting in a reduction in funding for R01s of $117 million. This reduction affected competing awards disproportionately because in any given year, the proportion of the NIH budget for R01s that is tied up in commitments (funding for non-competing continuations), can range from approximately 70 to 80 percent. This brings us to the next factor—funding ongoing research.  

NIH funds grants incrementally

NIH funds most RPGs on an incremental basis—approved awards are funded for an initial budget period (typically one year), and subsequent budget periods are funded as non-competing continuation awards out of the current year’s appropriation. The commitments to the non-competing continuation awards in any given year can be considered a fixed cost, and any reduction (or increase, e.g., during the NIH doubling from 1999-2003) in the NIH budget would disproportionately affect our ability to make competing awards. Even with a cut to the non-competing continuations by 1% (or 3% in the case of NCI), the amount committed to ongoing research in fiscal year 2011 increased by $189 million. This demonstrates how carefully we need to manage our funds since funding decisions in any one year have implications for the out years. 

graph of noncompeting commitments as % of total R01 funding showing and increase over time 1998 to 2011

These two factors, the reduction in the budget and the increased commitments, accounted for the majority (2.5 points) of the 4-point drop.

The average cost of R01 awards increased

An increase in the size of an award contributed the remaining 0.5-point drop in R01 success rates. While not keeping up with the costs of doing research (as indexed by the Biomedical Research and Development Price Index), the cost of R01 awards did increase by 2% in fiscal year 2011. 

graph showing the increasing total cost of competing R01s from 1998 to 2011

So what’s the story?

To summarize, the 4-point drop in R01 success rates (the biggest driver of overall RPG success rates) can be traced to:

An increase in R01 applications: 1.0%
NIH budget reduction: 1.0%
Increase in commitments: 1.5%
Size of award: 0.5%
Total decrease: 4.0%

While the success rate number is often cited as the indicator of the state of funding at NIH, it is very complex, and certainly, there is not a simple relationship between the success and the quantity of science supported by us. Success rate may be more closely equated with an indicator of competition, at least within some mechanisms, but even then, it is not straightforward. Hope you find these data as interesting as I did.

48 thoughts on “What’s Behind the 2011 Success Rates?

  1. I assume NIH understands that when they raise the NRSA pay scale, the vast majority of academic institutions raise their minimum post-doc salaries accordingly, thus increasing post-doc salaries supported on R01s. You keep kicking standard-size R01 budgets into the gutter by the various mechanisms employed–like eliminating annual increases for non-modular budgets and other schemes–yet inexorably raising the NRSA pay-scale by several percentage points *every year*.

    Eventually, we are going to be paying post-docs to sit around and play angry birds all day, because there will be no money left in already-committed R01 budgets to do anything other than pay salaries. And how can you continue to leave the maximum modular budget at $250,000 dollars, where it has been for over *TEN YEARS*, while increasing the NRSA pay scale MASSIVELY over that same time frame?

    What is the desired end-game here??

    • what percentage of departmental pay scales are forced to match postdoc pay to NRSA? Anecdotally, most of the ones i know of dont.

    • I hope you’re not suggesting the answer is to pay postdocs less. $40k/year is hardly a living wage in high cost-of-living areas, especially with a family. Technicians make twice that much, and we should pay postdocs twice that much — or better yet, hire technicians to do the lab work, instead of exploiting postdocs and grad students for cheap labor. Postdocs (and grad students) take the pay cut in the false hopes of getting a professor job someday — jobs that for most, are not panning out anymore, but the pretense is being kept up — why? If we need to revamp the whole system (and reduce the number of total labs) to stop the pretense and exploitation, then I think we should do that.

      • On what planet do technicians make $80K? I mean sure, *some* do, but it is hardly the mean, median or mode.

        On what basis do you conclude postdocs should be paid $80K? Just because you would like to make that salary? I can invent some numbers that I’d like to make too. Starting with the “not very much” amount, thanks.

        What is a “living wage” anyway? When these discussion come up, it always turns out that the person advancing the argument is really talking about the fabulous upper middle class outcome that they “deserve”. Well, look around you! Who else is getting those bennies?

        Right now, postdoc is not supposed to be a career job, period. I suggest the better approach is to revamp job classification, career prospects, etc.

      • Personal analysis of oddities or fallacies in this post, which seems to presume that doctoral students or post-doctoral trainees are unaware of the fact that only a minority (about the most successful 10-15% – 1500 virginal, New PIs/yr divided by intake of US-minted PhDs + imports) can make it into a research-intensive faculty position:
        (a) Research fellowships should be training positions. Yes, it should be tight (and needs to be tight) on the $$$. Hard to get by on $40,000 per year? – that’s better than the median salary nationwide and for a training that, if the person is successful, means they go on to much better-paid job.
        (b) Technicians getting $80,000 ? Come on; get real. [I do not doubt that there are a small % who add that much value to a research team or discovery program in NIH grantee institutions, but it would be a very small %. More broadly, a fundamental point the NIH (and Federal gov't) ultimately need to face up to is, how much will the taxpayer base want to continue subsidizing marginally better research at high-cost grantees (in ultra high-cost parts of the country) vs. stretching scarce research $$$ further with a progressive 'market-driven' shift towards regions where $40,000 / yr with health coverage is pretty solidly middle-class?
        (c) yes, it is getting to where the average post-doc is not cost-effective compared to a research assistant.
        (d) when did student or post-doc stipends get cut? Never happened.
        (e) Informal samplings (and I think formal surveys) show that a minority of students or post-docs are thinking about a research-intensive faculty position for their job after the training phases. Also, which law is it that requires someone to be a doctoral student or a research fellow? Ostensibly, people are doing the work because it's what excites them.
        (f) NRSA training pretty much requires reinforcement of the reality check about the research-intensive jobs situation. [how could someone be a valid Ph.D. level education and not know the job situation?]

        • (a) Training positions for what jobs? The professor jobs are disappearing, yet the training positions keep increasing. It is a pyramid scheme. We need to reduce the number of training positions. I think it’s fine to pay the trainees a low wage if they are really being trained for a future job as a professor. But they are not being trained for a job as a professor, and paying them that low wage under false pretenses is wrong.

          $40k higher than US median? Nope.
          http://en.wikipedia.org/wiki/Household_income_in_the_United_States#Median_income
          Please try raising a family on $40k in a high cost-of-living area. (See also b)

          (b) I’m in a high-cost-of-living area. I looked up salaries
          http://www.genomeweb.com/sites/default/files/newspics/genometechnology/Median%20Salary%20by%20Region.jpg
          http://www.genomeweb.com/sites/default/files/newspics/genometechnology/Median%20Salary%20by%20Gender.jpg
          Ok you’re right, I was wrong, $80k is high for a tech. But not for a researcher/scientist. So I edit my argument to say that we should not be hiring postdocs for cheap labor, we should be hiring scientists. How’s that?

          Interesting idea that the government would only give grants to institutions in low cost-of-living areas. I never thought of that. Crazy, but I sort of like it. Never would happen though. And how do you know that the research done in the high-cost-of-living areas is only marginally better?

          (c) good

          (d) I didn’t mean that pays got cut. I meant that young people (college, and entering grad school) are pursing science careers in the false hopes that they will get a professor job. They won’t, because of the pyramid scheme. Those young people could have entered finance or engineering in college had a higher pay (>$200k at Google, Wall St, etc). That was the “pay cut” I was thinking about.

          (e) I have never seen any such surveys and do not believe they exist. In my experience, grad students don’t realize the dismal job situation until at least 2nd year of grad school. People sometimes do switch careers at that point; many don’t. They should have been informed while in college that there are hardly any good professor jobs, and they shouldn’t waste 10 years of their life being exploited on low salary in order to get one. Sure they can get a scientist job in a big corporation, but that is quite a different job than many have in mind when deciding to enter academia for their PhD.

          I do agree no one is holding a gun to these people’s heads to follow this career path; that is your strongest argument against me. But I still feel they get on the career path because of false pretenses and a pyramid scheme by bosses who want cheap labor, and I think that is just wrong.

          (f) See (e)

          • One more thought on the point of “who is forcing these people to be a PhD student or postdoc?” Again I feel the false pretenses are to blame. You mentioned at the beginning that only the most successful 10-15% PhDs get a faculty job. What defines “successful”? Largely publication record; especially getting high-profile papers. When does a student/postdoc know that he is successful by this metric? Not until the papers are published — i.e., at the end of the PhD or postdoc. During their training, these people hold out hope that they are in that top 10-15%, until their papers get rejected from the high-profile journals and eventually published elsewhere. Then the job prospects dim, reality sets in, and they’ve invested 5-10 years in training for a job they can no longer get.

          • (a) Research Fellow = training for a position that puts what the PhD ostensibly is as a degree to use in a position of greater independent responsibility and leadership, which could be research-intensive faculty, team leader or something like it in industry, education/training-intensive faculty position, etc. Since a central word in all of the above text is ‘independent’; part of the training process is to know and apply “the rules” (an old one, “publish or perish” now has expanded to “publish well or perish”) and take responsibility for one’s performance in delivering on acceptance of risk at a level one can manage. [The NAS put out a great book on the whole post-doctoral experience, including how to do a good job identifying a good training lab.] Highly recommended for people who, late in the ir doctoral work, apply the Ph.D.-type skills to thinking about and selecting the right ‘next step’ in their life / career.
            (b) you are confusing household with individual income (didn’t actually read the Wiki-post you cite? “Household income is not to be confused with family or personal income. Household income is often the combination of two income earners pooling the resources and should therefore not be confused with an individual’s earnings.” This is the link to personal income: http://en.wikipedia.org/wiki/Personal_income_in_the_United_States. Bear in mind that incomes went down in the time since the stats in the Wikipost.
            (c) response seems self-explanatory (and ties to my earlier comment about aligning choices with cost-of-living).
            (d) Interesting world view. To take a pertinent extension, then, NIH / taxpayers should take it upon themselves to pay folks who trained as physicians in cardiology $400,000 or $500,000 per year because that is what they could earn if they chose to be full-time invasive cardiologists instead of falling in love with doing research and doing a bit of teaching and training.
            (e) There’s a fellow at Harvard (at least was there a few years ago when I heard him give a talk with his survey data) who has pioneered the anthropology & sociology of scientists, trainees, etc. True confession, as a limitation to scholarship in my post, I’ll leave it at that and agree I don’t have the citations at my fingertips (though I do have the anecdotal evidence of most PI’s I talk with or hear from, even from top places like UCSD, and certainly my own fine research-intensive institution. Only a minority of doctoral candidates are thinking about trying to become independent leaders of a scientific team.
            (e) Are they asking the right questions? [often a good quality in a scientist]. In the age of search engines and the internet, the ease of access to data about supply of PhDs and demand in different setting are pretty easy to come by. In the older (pre-internet) model, I was told in unambiguous terms “don’t try just to be a biological scientist” (ditto, “don’t try to go into math unless it is statistics since there always are jobs in the insurance industry”).

      • It’s never made sense to me that post-doctoral pay scales are not graduated based on local cost of living. At current wages one can’t live on a postdoc’s salary in NYC, but you can buy a house near Duke or Wash U. It would be trivial to lay hands on tax, census or actuarial tables and create a sliding scale that is more equitable. I literally went into the red as a postdoc in a major city, but as faculty in the middle of the country I see grad students buying condos, and postdocs buying houses.

        • Yes, it could be done (in theory), but in practice, why should it be done? In other words, why should taxpayers (with NIH as the intermediary) actively move to expand a perverse incentive that raises the costs, when instead there are terrific research and training environments (no doubt such as your lab :) that can produce terrific successes in science and in training outcomes, where the cost of living is lower? The central question is about preserving best science in the US in an era where flat funding is a “good outcome” (i.e., dodging the strong possibility of a 7.8% nominal $ cut in the funding of NIH in the next federal fiscal year, 2013).

    • What is really a MASSIVE increase in pay scale- $38,496 for some one with a terminal degree and expertise to work for you so that you can get papers and R01′s ???

  2. Pingback: FY 11 Success Rates & Other Stats from the NIH « Medical Writing, Editing & Grantsmanship

  3. Adding to CPP, when modular budgets are cut across the board to 188K/year, the actual purchasing power of what a modular looks like now is a much vaster decrease in what it was 10 years ago.

  4. Kudos to OER for this information. Very informative if adding to the depression and amplifying the sense that current consultative role in reviewing grant applications is the scientific equivalent of Sophie’s Choice. I had a few questions for Dr. Rockey and the OER team.
    (1) in the screen shot of analysis showing a (depressing) drop in numbers of R01 awards FY2010 vs FY2011 (7% decrease, 656 less awards – disaster!), is any of the FY2010 base number used there (9,432 awards) inclusive of ARRA awards. [Did some R01's get awarded with ARRA $$ in FY2010?]
    (2) Of the 0.5% increase in average size of total award, how much is driven by progressive increases in the F&A components after our Age of Excess driven by the Doubling?
    As noted in a response above and highlighted by another post about regional differences in cost-of-living, part of what needs to happen to preserve the research leadership of USA is to manage a gradual shift of public monies toward lower-cost areas (or have some of the disproportionate costs cost-shared by the institutions or benefactors).

    • RePORT’s Funding Facts tool that we used for this analysis does not include ARRA awards. So no, the 9,432 RPG awards made in 2010 does not include ARRA-funded RPGs.

  5. Well as a new and early stage investigator (but not really young), I am out. After trying unsuccessfully to obtain an R01, no funding to generate new data, and being one of the only ones who study a certain process, I am out. NIH refused to review my latest submission due to it being too similar to previous submissions. With the payline so low and researchers losing their jobs, this is what the NIH is using resources for–to police grants for similarities?? Grants that have never been funded, may contain important science (yes I have had good papers from previous smaller grants), and are from new investigators are being withdrawn. What does it matter if there are similarities–the payline is so low, grants with great scores are not being funded. So, then you have to change the whole grant to resubmit? This does not cure disease. In another heartbreaker, my score on an R21 was a 29, then a 22 on a resubmission. Guess what? Not good enough. Can’t resbumit the same science. So I am out–done with this game. Good luck to all of you.

    • Neglected Disease Researcher. I am deeply saddened to read your post. I hope that your situation is not yet “out” but if so, I think it a travesty. The NIH has no right to tell researchers what science to do. Their use of plagiarism software to screen new submissions to determine similarities to a previous A1 is draconian and must stop immediately. I challenge the new acting director Dr. Richard Nakamura to direct that this no longer be done. I suggest that the acting director reach out to the science community and immediately explain that the practice will no longer be performed. The NIH may no longer allow A2s but I say, good science (22) should be allowed to come back for another go!
      It in my opinion that we are witnessing the death throws of a good system that cannot effectively operate in a fiscal climate that limits grant to an 8%tile pay line. Human nature (reviewer proclivities) dictate now that non-science factors can influence a review. Do you know the lab or the department; your impression of the reputation of the school or the investigators; the conflict of interest inherent in peer review when money is so tight that going in to the meeting you know that maybe one or no grant will be funded so you pick a favorite and this impacts your judgement of other proposals. Note that despite the 9 scale, we are still clustering our reviews and creating what is little more than a lottery.
      New R01 Review Suggestion: five pages; why it is important; what you can do about it; how you will approach it and; what will the field look like when your done. NO NAMES, NO INSTITUTION; CITATIONS BUT NO LANGUAGE THAT ALLOWS REVIEWERS TO LINK TO LAB OR INVESTIGATOR. All the vetting can then be done after the science review. This will get the bias out of the scientific review!
      IMHO

      • Bravo Iain this is my thoughts exactly and i could not have said it better, you are 100% right!!

        New R01 Review Suggestion: five pages; why it is important; what you can do about it; how you will approach it and; what will the field look like when your done. NO NAMES, NO INSTITUTION; CITATIONS BUT NO LANGUAGE THAT ALLOWS REVIEWERS TO LINK TO LAB OR INVESTIGATOR. All the vetting can then be done after the science review. This will get the bias out of the scientific review!

    • “Neglected Disease researcher” is right on target. NIH should seriously consider throwing “similarity check” for resubmission to bins. To add to Points raised by the above author- When you get a score in 20′s and realize there is not much criticism how do you rewrite your proposals- commonsense dictates ” don’t fix if there is no problem”. Instead mow you are forced to throw the proposal . As a young investigator , you cannot reinvent a new project by somehow generating preliminary results and under go the same fate all over again. In the end, only BIG labs with many people can turn in “different proposals” to be in the game and everybody else cleared from the field.

      • “….and everybody else cleared from the field.”

        beginning to believe that is the point–especially when some are wondering why we are bothering to train a next generation.

  6. How about offering some of the highly meritorious but unfunded applications to other countries (China?) to fund? On a bond-like basis? If the work gets performed and published as promised, NIH will later pay off the bond with some small interest. A good part (and an increasing part) of what the government does now is with money we don’t have. Why not research?

  7. Many starting assistant professors don’t make $80K a year! This is totally dependent on cost of living and where. If you keep raising postdoc salaries and cutting RO1′s, there will be less jobs for postdocs. Keep in mind.

    But there needs to be birth control in academia. Each professor should only graduate ONE – repeat – ONE PhD student who is intended to go on to get a job, or else, basic math, it becomes exponential growth that is unsustainable. This ONE is over the entire career of the PI. So the idea that you have a lab full of grad students each year for 40 years means you are breeding numerous offspring certain to starve to death.

    Simple math. Radical restructuring needed.

    • “If you keep raising postdoc salaries and cutting RO1′s, there will be less jobs for postdocs.”

      Wouldn’t this actually help the glut of people in the market? Post-docs would be more highly respected for the training they already have (and more able to afford their most likely years to raise a family), their status would be competitive with the private market (where you can get $80,000+ straight out of grad school in some fields), and there would be a more reasonably competition for positions afterward. If there are no post-doc positions, it would be hard to justify putting so many grad students on grants, so it may push the market down.

      Yes, the PIs will have fewer worker bees, but that is the price for a well-trained HUMAN BEING you are employing. If all you wanted was an extra un-trained pair of hands, get an undergrad to volunteer. If you feel that the extra Eppendorf tubes are worth more than an extra brain doing science, then by all means, don’t hire an extra post-doc!

    • Be careful with this one. In our graduate program, almost none of our Ph.D. students plan on getting faculty jobs, even coming into the program. I have graduated about 10 Ph.D. students over the past 15 years. One is in a faculty job in a foreign country while the rest work in the USA in various industries, one has even started their own biotech company.

      As far as I know, none of them have even aspired to NIH funded research faculty jobs and clearly the job market has a place for them (some of them make a great deal more money than I do). I think folks forget that there is a lot of well paid, permanent employment available for folks outside of the NIH rat race. One just needs to get out of the mindset that “research faculty” is the only honorable profession for a Ph.D. level biomedical scientist.

      This is a BIG reason I am not sympathetic to perpetual postdocs in academia. There are jobs out there off of the rat race. In my experience those who say there are not apply to 2-3 jobs and give up. A real job search will take 50+ applications in many cases.

      • with the ever shrinking pipeline and big pharma starting to slash R&D spending, I do not see good prospect for PhD degree holder in US biomed industry. (Or you considered getting fired at 45 as good prospect?) Just read the quarterly hiring data on Nature Biotech. 2011Q4, four years after economical recession, net firing for noticable pharma and biotech companies is still more than 1000. Plus many hiring occured in China and India, not western countries. With a few friends working in Chinese CROs I discovered that they are massively hiring bachlors instead of PhDs to do basic research tasks. That is their magic to lower cost. You know, in the industry, they do not need too many costly brains to think, just hands to do basic research tasks.

  8. Not to throw wood to the blaze, but I was prompted recently by colleagues to go to NIH project reporter (http://projectreporter.nih.gov/reporter.cfm) and plug in names of colleagues who are NIH investigators. This database now contains the funding levels for intramural investigators and I was STUNNED to see levels of funding that no one – I repeat – NO ONE- in the extramural world can even dream of (the equivalent of 10-15 R01′s per PI; or HHMI super investigator). And at least for the ones I looked at, the output in no way reflects the input of resources. If there’s massive restructuring needed to cut excessive waste, NIH should start looking intramurally. From the looks of this intramural budgets can be cut by 50-90% without losing much science.

    • The intramural waste in the NIH is legendary. When you have essentially free money, it inevitably leads to waste. Anyone who has been anywhere near NIH will tell you many super-expensitve equipment sitting unused and unmaintained, many expensive experiments sitting unanalyzed. Reducing intramural waste in the NIH would be very easy, given how much of it there is, if there was a real willingness. The extramural R01 funding rate can be bumped up probably by 10% just by small reduction in intramural waste.

  9. My R01 modular budget was cut 17% the first year, 10% the second year and an additional 10% this year. The aims that I am supposed to do with $62,000 less than I budgeted (this year) haven’t changed. I’m cutting staff in order to have a little money to do the work. Next time-I’m not going modular!

  10. Just a reality check:

    According to the AAMC the average salary of an Assistant Professor for the southern region is 77.5 k. Comrade PhysioProf, is right regarding the fact that the increase of post doc salaries, and the increasing necessity to cover big chunks of the PI salary, dont leave much money left for doing Science. Nobody want to cut salaries, so the solution should be to increase the value of the modular grants right? However, doing this probably will reduce the success rates since you will have bigger grants for the same total NIH budget. It is a very complex problem.

  11. Why not require the Universities pay the salaries of their professors? And let the money go to the research (including postdocs).

    • On this topic, I always did wonder how universities got away with “hiring” faculty on the condition (i.e. tenure hangs on it) that they pay their own salaries (from public funds). In return, the institution “owns” the money that researchers get and also receive the credit for the work (or why else would we have to add affiliations to papers?). This seems like an unethical scheme to get public taxpayer money to pay for the prestige of private institutions, all while leaving the “employee” highly vulnerable to institutional abuse. Anyone else see that and/or can explain to me why I’m off base in being confused?

      • I am glad to see this message since I have been wondering the same. The Unite States seems to be only one practicing such an “unethical” scheme in the developed world as far as I know. Such business scheme reminds me of a mafia in that as long as you can bring cash you are accepted and allowed to purchase “privileges” but cash flow dries you are out. I am also disturbed by “administrative” job holders who are somehow paid by institutions and hospitals (don’t know in universities) even though they don’t seem to be generating fund (I could be wrong since I never seen any financial spread sheet). These administrative positions are really few in other countries where employers pay salaries for faculties, postdocs and technicians. It’s time to clean up mess and introduce a functional infrastructure to maximize our work, research. However, I have no idea what and how can be done.

  12. And another thought.

    Using the RePORTER tool (http://report.nih.gov/rcdc/categories/#bpopup) one also can see that out of the roughly 30billion budget, fully 3.3 billion go directly to clinical trials (and an additional 10.9bn to “clinical research” broadly defined). Now NIH’s mission is to further human health, so that clinical trials/research ought to be the pinnacle of our efforts, but here is the problem: clinical research in general and trials in particular are of terrible quality, at least according to Dr Deborah Zarin who happens to be the Director of the clinicaltrials.gov database (interview here: http://www.sciencemag.org/content/333/6039/145 in case you didn’t catch it, and snippets below).
    ——————–
    Zarin describes her immersion into the world of clinical trials in this way:

    “I call it my introduction to the sausage factory. It appears that there are a number of practices in the world of clinical trials that I hadn’t been aware of; it surprised a lot of people. For example, researchers might say, this is a trial of 400 subjects, 200 in each arm, and when they came to report results, they would be talking about 600 people. We would ask them to explain. They would say, “We are including 200 people from this other study because we had always intended to do that.” … There were a lot of—what would I call it?—nonrigorous practices.”

    And in answer to a question about how some trials are conducted, Zarin replies:

    “We are finding that in some cases, investigators cannot explain their trial, cannot explain their data. Many of them rely on the biostatistician, but some biostatisticians can’t explain the trial design. So there is a disturbing sense of some trials being done with no clear intellectual leader. That may be too strong a statement, but that’s the feeling we are left with.”
    ———————
    So I am left with the suspicion that, if the rigors applied to standard basic science R01s were also applied to clinical trials, the NIH budget would “save” a good chunk of that 3.9billion.

    I guess the overall issue I’m raising (and which we ALL need to raise, vociferously) is that there’s plenty of money at NIH to fund good science, but it is mis-allocated (to intramural programs; to poorly designed “clinical trials” and who knows what else — mostly because it’s allocated according to models which are no longer reflective of the reality on the ground).

    So if you want to lobby, don’t lobby for “more” money for NIH; lobby the NIH instead, for better use of what in the end is OUR (taxpayer) money. And lobby for PARITY in QUALITY and output between intramural programs (or clinical trials) and the outside world (or basic/translational science).

  13. While many factors may contribute, an elephant in the room is the bloated size of an average R01. Some 430k$, heh? I guess this pays full PI salary as well? That is the key problem, right there – NIH should never have agreed to fund PI salaries. If one is ~100% funded by NIH, they should probably spent 100% of their time doing research. Why are they paid 2-3x more than a postdoc who then does essentially the same? Or 4-5x more than a grad student? NIH funding should not be intended to fund extramural research institutes where people do nothing but research. NIH extramural funding should fund laboratories led by professors who teach students (that is what faculty is for) and thus need someone to help them with actual bench work. If most of PI’s salary is covered by home institution (as it should be and still is in many places), then a postdoc + a grad student + equipment/materials would account to something ~120k$/year. With administrative costs at ~50% (and not the ridiculous 100% and more in various scripps/salk/whatever), this is ~2-3x less than what NIH pays now. So NIH could fund 2-3x more labs, right?

    Ask yourself why this simple consideration is not followed. It’s convergence of interests. Bigger grants mean more overhead money for university bureaucracy. Fewer grants means less work to do for NIH bureaucracy (sorry). Bigger grants mean more money/power for big shot funded investigators who advise the policy making. Losers are young investigators of all stripes, and yes, taxpayers, but they have no say in policy decisions.

    What happened to NIH funded research community is the same thing that happened to US overall. Competition for the grant money is controlled by funded investigators, and they collectively install policies that favor them (no matter how much they whine themselves during hallway conversations). It is not to say that the research they do is not meritorious, simply that more labs can be funded with the same amount of money.

    It’s not too late to make changes, and you have to start doing so at some point. Cutting existing awards is not an option, but simple things can be done.

    a) It should be next to impossible to obtain a second R01. A third should be out of the question.

    b) Create the system of smaller “life support” grants for young investigators/smaller labs. Sure, it’s great if someone can study “toddler feeding styles” or “sources of change and stability in low-income black marriages” at a tune of ~1M$/year, but if the same amount of money can be spent supporting on a small scale some 15-20 laboratories, perhaps more research will be done.

    c) Preference should be given to the labs that need less than, say, 15% salary support for the PI. This would provide incentive for the universities to move away from the hiring new faculty who are expected to bring full salary support, not teach.

    d) Universities that charge less overhead should also be favored. Even better, private institutes should no be allowed to charge 100% administrative costs. Surely, someone has to pay for the infrastructure etc., but all the research-only labs should be confined to intramural labs at NIH, for the sake of cost reduction.

    e) Create, promote and ultimately shift to 3-year grants (similar to AREA). This would increase competition and make for a more focused research plans. Yes, the PIs would have to submit renewal applications more often, but they are doing nothing but writing grants these days anyway.

    f) Not-yet-funded researchers should be included in study panels at certain percentage. I know they would deny it, but slashing every new investigator grant is the obvious survival strategy for “established” folk. If there is one thing we learn from Darwin, it is that winning group strategy will always appear under increased selective pressure.

    g) And yes, repeal those ridiculous new rules that prevent the resubmission of the grants that in two attempts didn’t make the payline. The only argument that makes any sense in support of this rule is that it may prevent people from clogging the system with obviously uncompetitive proposals. Well, that is easy to fix by allowing panels to provide such determination. Killing highly meritorious research that is at top 15% but barely missed the payline is not a good solution. We were told that abolishing A2 will somehow help good research to be funded faster. Really? As a side note, another rule that has to go is that one cannot submit additional info. It’s 2012, a lot of things can happen in 2-3 months. Surely one page addition a month before panel meeting is not much of a burden. If one is worried about abuse of this in order to get around size limits, well, it’s already 12 pages not 25.

    NIH funding was doubled. What we got in exchange are big-scale projects like structural genomics that have drastically diminished returns but are easier to manage and bloated faculty ranks as medical schools and newly created private institutions launched the arms race. The problem of funding shortage is self-inflicted by poor policy decisions. It won’t be fixed, of course, but ranting does make me feel better.

    Good luck to everyone.

  14. It would be interesting for your next issue to compare inputs and outputs of intra-mural to extra-mural research!! And to see the distribution of resources devoted to intra-mural investigators.

  15. I am less curious about the % of applications funded than the % of PEOPLE. If someone applies for 5 R01s over her first three years and gets one, maybe 20% is ok — her career is still on track. If she doesn’t get one at all, maybe she is misses tenure. The report doesn’t reflect this at all. Is there any way to know how many grants per investigators INCLUDING investigators that get no funding? Also I think we need to look across 3 or 5 years on the same principle… if she got an R01 last year it is not a big deal to have a 0% success rate this year. But if she has no funding at all, she may very much need to get funded this year.

  16. The NIH should carefully monitor out-year expenses so that they do not impinge on current FY funding. Budgets should not have escalating costs in out-years. By limiting the increase in out-year costs, the success rate and number of grants funded would have been higher. The NIH cannot control the number of applications or the budget. But they can do a better job at ensuring that the budget is fairly divided between those grant recipients who received awards several years ago and equally deserving grant-writers who are applying in the current worsening environment.

  17. This whole discussion and recent developments in the NIH review/funding processes deeply sadden me. I have a faculty member for almost 33 years and have been mostly funded during this period. I just went on to an unfunded extension. I had decided to do so because, under the current circumstances, I did not know if I “had another NIH grant in me”. After reflecting on the situation and reading the responses, I am convinced that I have made the correct decision to retire in August. it’s bee a great ride – I wish it looked better for scientists younger than I.

    • Wayne you are rare in your thinking, most people in your situation would not want to hang it up so to speak and would be even more aggressive! These times are bad and slanted against funding new investigators. It’s become a mafia so to speak, an unholy alliance between the policy makers and the most well funded labs. Look at the RFAs you have to wonder who writes them???

      As a post-doc I saw my boss get privileged treatment from the NIH institute where he served on panel. So much so that at the end of any one year he was offered excess budgetary funds from the institute. All he had to do was draft a letter with quotes and the request to secure the funding. This is completely wrong no matter how you see it!! First, he received $68,000, the following year he received $224,000 dollars. With this money he purchased new capital equipment. I was shocked, as much as I was happy to see the new equipment in the lab. He was given these funds without any review or formal application or oversight. C’mon I thought this is tax payer money given without any oversight, what a pile of crap, meaning those with money get more money and those without stay without!!! these funds could have been used to fund grants that were just below the payline!

      • Per NIH rules, collaboration administrative supplements are given only if ALL the collaborators are currently NIH funded. Thus, if the NIH-funded investigator wants to extend his research into a new area and needs some expertise her own lab does not have, the funding opportunity is closed if the prospective collaborator is a new not-yet-funded investigator. I’d like to see NIH explain the reasoning behind that.

        What happens in the reality in these situations is that funding either goes to people who already have the money or the young investigator gets exploited as a co-I, which means zilch for tenure purposes. Yes, I deal with a nice person here who pays part of my salary and doesn’t mind shouldering some of the costs, but a more fair outcome would be for part of that money be treated as direct costs to my lab.

        I do like the NIH administration to explain right here in this blog the reasoning behind the rule. I understand that the purpose of this funding opportunity is to provide extra support for research that have already been funded, but why all the collaborators should be card-carrying R01 club members? And how is it that NIH has spare funds to do this in the first place?

        • In general, the NIH does provide investigators with the ability to ask for additional funds for an active grant due to unforeseen circumstances through requests for an administrative supplement. These requests are carefully reviewed by NIH staff and must be within the scope of the original project, or the request must go back through the peer review process. It sounds like you may be referring to a specific supplement program from one of our ICs rather than NIH policy. Without knowing the specific program or circumstances, I can’t provide any additional details.

  18. 1. There should be a cap at 2 RO1s

    2. A2s should be reinstated and the “substantial similarities” ban re-examined

    3. Step back from Zerhouni’s disastrous re-allocation of resources to huge publicity-grabbing projects

    4. Scale back intramural research that does not scale with extramural competition

  19. This article neglects an important point. I imagine that there is an assumption here that success rates and ‘competitiveness’ are well correlated. In other words, you score 1′s and 2′s on your grants, you get the grant, if you score 4′s and 5′s, you don’t.

    NOT TRUE!!!

    NIH hides the data they generate from study section from the public. Interesting how NIH pays to have reviewers look at grants, but then does not disclose the data associated with each grant. This is because it would cause outrage and show bias towards particular investigators. I have scored many grants well (1s and 2s) in addition to the other reviewers and these were not funded (at least years after the scores), and i and the section have scored a grant 5-7 range and I see them pop up on NIH reporter a few months later. How many section members check up on their grants they reviewed??

    Do PIs go around bragging that their grant that scored a 55 was funded?? No. Do PIs go around saying they scored a 22 and were not funded? All the time. Why is this? Corruption NASA circa 1990′s style. I do feel bad for SROs and other Chiefs in a broken culture, they are pawns, but it is only a matter of time before NIH goes the way of NASA. Hey, the party couldn’t last forever, right, I mean we don’t make anyone any money 99% of the time, and people only see short term benefit, it is human nature.

    Oh well, we’ll see where it goes! Hopefully i am wrong, but NIH is slowly killing itself with policy.

  20. This whole NIH grant funding system is fatally flawed! Basically, grants are eaten up by salaries, with little or no money left to do the work that the grant was originally funded for!!! Look at all the institutions with faculty on soft money, they need at minimum 2 grants that are over the modular budget of 250K to cover their salary and the salaries of their staff. The NIH is basically functioning as the employer and the institutions are benefiting with no commitment at all (all UCs, all Ivy league institutions and all private institutes operate this way!) In addition, I am not against making money but the NIH needs to check on the numbers of faculty that are double dipping their salaries, for example, they have a VA appointment and an academic appointment at the neighboring university. Their VA grant is administered through the VA and their RO1s are administered through the academic institution. If you add up effort, it is well over 100% sometimes as high as 200% in some cases. They are basically drawing two separate salaries for the same effort! No one is keeping track and they get away with this by design.

    I feel that the NIH should consider getting rid of funding PI salaries, I know this doesn’t sound attractive at all, and would be a death knell to all soft money positions, but come on, like the real estate bubble, this system is not sustainable at all! Too many PIs abuse the system and have overly inflated salaries with the help of NIH funds. The NIH needs to examine the bang for the buck so to speak. How many publications and what progress was actually made on the previous grant, etc.. Study sections are not very reliable, they generally protect their friends and help scratch each other’s back so to speak. The current review system is an absolute abomination and designed to benefit those connected and already established. What needs to happen is that grant review must also change, grants also need to be looked at for the science alone, and remove the intangibles, meaning, perhaps blinded with no consideration of the PI in the proposal only the proposed science. Leave the decisions about the budget, the institution, the investigator, etc. to an internal review committee etc., not to study section. More often than not on panel, you see discussions about grants based on perceptions of the investigator and the environment, and not on the science being proposed, in my opinion this is utter BS!!!! These are intangibles and purely opinion in my mind and cloud the perception of what is being proposed scientifically (this translates to an established scientist proposing work out of their expertise, but given a pass based on their perception, reputation, connections, etc, and a new investigator that is isn’t established, being shot down because they lack expertise for proposing similar work !!!)
    Also, I agree the funding levels for intramural investigators are overly inflated and a major joke! Wake up NIH!!! Pay attention the house is being robbed and you are letting it happen!!!!

  21. The problems stem from there being too many scientists developed during the NIH budget expansion of the Bush administration. It was a good thing, no doubt. Unfortunately, NIH budget cutbacks, NIH bureaucratic creep and elevating administrative costs, and university greed [evidenced by their ever-expanding indirect requirements] has fueled the attack on the great promise of the Bush initiative. Universities are churning out tons of new scientists every year, but with fewer dollars available there is now great harm being reaped upon the scientific endeavor — young physician scientists fleeing to clinical jobs and non-physician scientists getting bumped to teaching jobs or worse. More established scientists aren’t faring any better–it is only tenure in non-medical school environments that is saving some fields in the system to this point. The idea of a physician-scientist is an ephemeral notion these days. The granting process is not much more than a craps game, where you have to be a card carrying member of the group to even get in the door. Obviously the fiscal dilemma that our country is undergoing will have great consequences for ensuring scientific progress, especially in the basic sciences.

  22. Pingback: Estimating the Purchasing Power of the NIH Grant | DrugMonkey

  23. Pingback: Fiscal Year 2011 R01 Funding Outcomes and Estimates for Fiscal Year 2012 – NIGMS Feedback Loop Blog - National Institute of General Medical Sciences

Leave a Reply

Your email address will not be published. Required fields are marked *


− 5 = three

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>