Preparing for Changes in Peer Review

Posted

The first stages of implementation of changes in NIH peer review will begin in January and February 2009. These changes will be timed to apply to all three standard review/award cycles in each fiscal year. Please see the chart below showing the precise timeline by review/award cycle. While many details continue to be worked through, the first set of changes are highlighted below:

Scoring
To improve the transparency of the scoring process:

  • Before a review meeting, assigned reviewers and discussants will score applications on the five review criteria using a scale greater than 1-5.
  • Each assigned reviewer and discussant will also provide a preliminary overall score using the new scale.
  • At the meeting, discussed applications will receive an overall score from each eligible (i.e., without conflicts of interest) panel member and these scores will be averaged, multiplied by 10, and called the priority score.
  • The priority scores then will be percentiled against the appropriate base. The new scoring system will necessitate the establishment of new percentile bases. Percentiles will be reported in whole numbers.

Critiques
To improve the quality of the critiques and to focus reviewer attention on the review criteria:

  • Before the review meeting, in addition to preliminary scoring, assigned reviewers will provide written critiques of the application through an electronic template that will prompt for strengths and weaknesses for each criterion.
  • After the meeting, the critiques will be compiled into a summary statement that will be shorter and more focused than current summary statements due to standardized organization and reporting of strengths and weaknesses.
  • Discussed applications also will have a summary of the panel’s discussion at the meeting.
  • Feedback for Streamlined Applications. Currently, applications not considered to be in the more meritorious half are “streamlined.” Streamlined applications are not discussed by the full review committee and are not assigned numerical priority scores, but the applicants do receive the reviewers’ critiques.
  • In 2009, streamlined applications will receive scores in addition to the reviewers’ critiques to help applicants assess whether or not they should submit an amended application.

Balanced and Fair Reviews Across Career Stages and Scientific Fields

  • The New Investigator Policy was modified to identify Early Stage Investigators (ESIs), see http://grants.nih.gov/grants/guide/notice-files/not-od-08-121.html, and becomes effective beginning with traditional R01 applications received for the February 2009 receipt dates.
  • Another notice containing specific guidance for ESIs will be released on October 31, 2008.
  • Where possible, NIH will cluster new investigator applications (including ESIs) for review.
  • The same clustering approach will be considered for clinical research applications.

Amended Applications:
To speed the funding of meritorious science and minimize reviewer burden:

  • NIH has issued a policy announcement decreasing the number of resubmissions (i.e., amendments) permitted from two to one. Beginning with applications intended for the January 25, 2009 due date, all original new applications (i.e., never submitted) and competing renewal applications will be permitted only a single amendment (A1).
  • To facilitate this transition, original new and competing renewal applications submitted for FY 2009 or earlier, i.e. prior to January 25, 2009, will be permitted two resubmissions.

Enhanced Review Criteria

  • The current five scoring criteria will have clearer descriptions, with questions for reviewers to consider in evaluating each criterion.
  • The application instructions will be updated to inform applicants of these revised questions.
Funding Round Non-AIDS Receipt Dates AIDS Receipt Date Review Date Council Date Award Date
FY 2010
Cycle 1
Jan/Feb 2009 May 7, 2009 May/June 2009 Sept/Oct 2009 Jan/March 2010
FY 2010
Cycle 2
May/June 2009 September 7, 2009 Oct/Nov 2009 Jan/Feb 2010 March/May 2010
FY 2010
Cycle 3
Sept/Oct 2009 January 7, 2010 Feb/March 2010 May/June 2010 July/Sept 2010
FY 2011
Cycle 1
Jan/Feb 2010 May 7, 2010 May/June 2010 Sept/Oct 2010 Jan/March 2011
FY 2011
Cycle 2
May/June 2010 September 7, 2010 Oct/Nov 2010 Jan/Feb 2011 March/May 2011
FY 2011
Cycle 3
Sept/Oct 2010 January 7, 2011 Feb/March 2011 May/June 2011 July/Sept 2011

The full set of application submission dates for each cycle can be found at:
http://grants.nih.gov/grants/funding/submissionschedule.htm

For more information on Enhancing Peer Review and the NIH Guide Notices referenced in this document go to:
http://enhancing-peer-review.nih.gov

Keep abreast of the latest in the Enhancing Peer Review Implementation effort by joining a new Enhancing Peer Review listserv.