Moving Forward with Special Council Review

Posted

It’s been three months since I discussed how, during May Advisory Council meetings, NIH would pilot a new Special Council Review process for particularly well-funded applicants. Since that time, we ran the pilot and carefully considered all the great feedback we received from Advisory Council members and staff. We are now ready to implement the final policy on Special Council Review.

The new special review process will be in effect for the 2013 fiscal year, beginning with September 2012 Council meetings. The new policy will provide additional consideration of new and renewal applications from well-supported investigators who currently receive more than $1 million or more in direct costs. I want to remind you that this policy does not cap the total amount of funds an investigator may receive from NIH, but rather is a special review to complement existing NIH policies that require monitoring all investigators’ activities for overlapping support, and determining whether additional funds should be awarded to well-supported investigators.

You’ll note that NIH has lowered the threshold for the Special Council Review process from $1.5 million to $1 million, and now only considers direct costs. This change stems from widespread concerns expressed by you and during the Councils that using total costs as the trigger for this review process does not reflect differences in indirect cost rates among institutions. We examined how this change would affect the number of applications subject to special review, and found that it will have a minimal effect on the number of grants that undergo additional review. Approximately 89 applications for the upcoming council round will undergo special review (only 19 more than using the $1.5 million threshold).

I like that the Special Council Review process provides a consistent prompt for the Advisory Council to consider the amount of funds a principle investigator receives, yet allows each NIH institute and center (IC) to retain enough flexibility to fund the best science in support of their respective missions. It also responds to a growing need to address how to manage the funding of scientific research in fiscally challenging times, and to research community feedback, some of which took place right here on the blog.

Applications that will be given special review include:

  • New and renewal applications from investigators who currently receive $1 million or more in direct costs for research project grants (RPGs), excluding no cost extensions

Applications excluded from special review include:

  • Pending applications received in response to requests for applications (RFAs)
  • P01s and other multi-component RPGs, unless all the investigators exceed the $1 million threshold
  • Multi-PI applications, unless all the investigators exceed the $1 million threshold
  • Administrative supplements
  • Subprojects within complex applications (this may be revisited by NIH once we begin to accept complex applications through eRA Commons)

The full SCR policy is published in the NIH Guide (Notice number NOT-OD-12-140). We realize that refinements might be necessary as we dive into the implementation stage. However, we’re confident that this policy, piloted at each IC and modeled after a long-standing review process already existing at NIGMS, provides an effective yet adaptable process to responsibly fund the best science and support the biomedical research community.

18 Comments

  1. Will NIH conduct an ongoing evaluation of this policy? Will NIH regularly report the number of applications not funded as a result of this enhanced review?

  2. How many applications underwent special review in the Pilot study and how many were denied funding?

    Of those that were reviewed and not denied funding, were the justifications across ICs similar (suggesting a universal NIH standard could be established)?

  3. We will indeed be evaluating the policy as it rolls out. NIH does not make factors that go into funding decisions (scores, priority pays, etc) public, however, so we do not expect to report on the number of applications that are not funded as a result of this policy.

    1. How is this different than the funding outcome data posted by NIGMS for years? If you just take the 200-300 apps per year across the NIH and indicate how many are denied, that shouldn’t get anyone back to specific applications in any way that violates reasonable expectations of privacy.

  4. Has the NIH considered special review considerations for intramural labs that receive more than $1M in support, tallying up also salary support or free services not denoted in a lab’s awarded budget? Would be fair I think. Let’s see what we are getting for that nvestment too.

  5. Based on my experiences at NIGMS, this policy is not necessarily going to lead to simple fund/don’t fund decisions. In some cases, proposals from well-funded investigators may be funded but at a reduced level. It would be useful to plot the ratio of funds awarded to funds requested as a function of percentile score for applications subjected to the extra scrutiny and to prepare a similar plot for an appropriate control group of applications. This will allow the impact of the new policy to be examined. I do not see why such plots cannot be shared with the extramural community as they need not reveal any sensitive information.

    1. What you are trying to set up is a communist system with a supreme capitalist (single PI) at the top. What’s the point of putting all the money in the hands of one PI? I doubt the work or ideas in the grant application are fully attributable to that PI.

  6. In order for the Councils to make educated decisions, they will need good information about the nature of the funds already in the hands of a well-funded PI. For example, the Council may view an investigator who has $1 million in his or her lab with no subcontracts different from an investigator leading a large, collaborative project who only has a small fraction of the $1 million in his or her lab and subcontracts out the majority of it. Those may be explicit subcontracts to other universities (which NIH can more easily track) or they may be internal to the university for sharing among multiple investigators at the same university (which NIH can not so easily track using current mechanisms). Clearly, there can be other special circumstances regarding how funds are budgeted as well.

    In order for Councils to have information about these budgetary issues, will there be a Just-in-Time or other mechanism for well-funded PIs to provide information about the nature of their funding to inform the Council? I think this would be critical to the Councils making educated decisions.

  7. Is the 1 million dollar threshold total funds (across all years) or per year? If it is total funds, then it would be hard to ever get a 2nd R01 without this increased scrutiny.

    Also, what about expended funds – if your new proposal reaches Council when you are in the last year of a grant exceeding the 1 MIL threshold (total or per year, whatever the case may be), I’d hope that the new grant would not be scrutinized, as this could lead to gaps in funding and lab layoffs.

  8. NIH should stop allowing multi-awarded PIs to double count publications on multiple grants. This leads to inflation in productivity, and is one reason well funded can fool study sections into thinking they are more productive than they really are. If a publication is counted on multiple grants, its impact on each should be fractionally decreased so that its total impact is the same as the one publication on a single grant PI. Unless council does this, they will not have an accurate indication of whether the increased funding is actually leading to commensurate increase in productivity.

  9. Does the 1 million dollar cap include leadership as PI of P50 and P60 and T32 grants? (This would seem to penalize leadership roles).

  10. Seeking some clarifications:
    1) how is “per year” defined? Project period? Calendar year? Federal fiscal year?
    2) is there a URL to a report of affected investigators?
    3) What source will the NIH use to pull this information for reviewers?

    1. The year is defined as the federal government fiscal year, and we will not publish a report of affected investigators. To determine which PIs fall under the Special Council Review criteria we will use information from our internal databases as well as pending applications (including just in time submissions) before council meetings begin. Hope this answers your questions.

  11. This is definitely a step in the right direction to correct one of the most disturbing (but far from the only) injustices of the Study Section system (in which I have participated actively since 2004). Study sections I have been on have, for example given very high scores for people’s 6th concurrent R01. Events such as these simply kill new investigators.

    However, the major issue that NIH has to deal with is definitely indirect costs, which in some private research institutions can top 100%. If the object is to fund more proposals, this is the only way to achieve it.

    Regarding the exceptions to the policy, I disagree strongly that multi-PI proposals should be exempt, since it is simply telling people with >$1M to invite less well funded individuals to Co-PI with them. And trust me, that is exactly what will happen. A more reasonable question would be whether the funds requested in a given proposal would take any investigator over the $1M (or some other) limit.

    Notwithstanding these criticisms however, I think this is an important first step to reduce the concentration of funding in certain hands and it will definitely stop PIs from making application under their own name when it would be more appropriate that their junior faculty were applying as opposed to being supporting staff.

  12. I agree with PI that the process by which a Council will evaluate these proposals is not clear – or reasssuring. There’s an argument to be made for setting out a set ofNIH-wide standards that every Council should follow in its evaluation process. It’s also only fair to allow each applicant whose proposal has been selected for Special Council Review to submit additional “just in time” information that may help to clarify the nature of her/his “other funds” and allow the applicant to argue why the new request should be evaluated on its own merits indpendently from such other funds. From my experience with the NCAB (1884-91), the Councils will rely heavily on the Institute’s recommendation that will accompany each proposal – on what other basis could they credibly make these decisions? And that’s why NIH-wide standards would help to avoid competely idiosyncratic, or perhaps worse, capricious outcomes.

Before submitting your comment, please review our blog comment policies.

Leave a Reply

Your email address will not be published. Required fields are marked *