A Reminder of Your Roles as Applicants and Reviewers in Maintaining the Confidentiality of Peer Review

Posted on April 7, 2017 by Mike Lauer and Richard Nakamura

Imagine this: you’re a reviewer on an NIH study section, and receive a greeting card from the Principal Investigator (PI) on an application you are reviewing. A note written inside the card asks that your look favorably upon the application, and in return, the PI would put in a good word with his friend serving on your promotion committee. Do you accept the offer, or just ignore it? Or, do you report it? …. Or maybe several days after the initial peer review of your application, you receive a phone call from a colleague you haven’t spoken to in quite a while. The colleague is excited about a new technique you developed and wishes to collaborate. You realize the only place you’ve disclosed this new technique is in your recently reviewed NIH grant application. What do you do? …. Continue reading →

Characteristics and Outcomes of R01 Competing Renewal Applications (“Type 2s”)

Posted on December 22, 2016 by Mike Lauer

An investigator’s long-term success depends not only on securing funding, but on maintaining a stable funding stream. One way to assure continued funding is to submit a competing renewal application. However, as we noted earlier this year, while new investigators were almost as successful as experienced investigators in obtaining new (type 1) R01s, the difference between new investigator and experienced investigator success rates widens when looking at competing renewals (type 2s), and success rates of new investigators’ first renewals were lower than those of experienced investigators. In addition, we know that since the end of NIH’s budget doubling in 2003, success rates for competing renewals of research project grants overall have decreased. To further understand trends in success rate for R01 competing renewals (“type 2s”) I’d like to share some additional analyses where we look at characteristics of type 2 R01 applications, and the association of their criterion scores with overall impact score and funding outcomes. Continue reading →

Are You On the Fence About Whether to Resubmit?

Posted on October 28, 2016 by Mike Lauer

When applicants receive their summary statement resulting from the review of an application that was assigned a score outside of the ICs funding range, there are important decisions to be made that, ideally, should be based upon evidence. What is the likelihood that an application like this one will be funded? If I resubmit the application, what changes might improve the chances for a successful resubmission?

Recall that in 2014, NIH relaxed its resubmission policy (OD-14-074) to allow applicants to submit a new (A0) application following an unsuccessful resubmission application. Also, we recently posted a piece showing that review outcomes for new applications submitted following an unsuccessful resubmission had about the same funding success as other new applications. But some applicants may wonder, what is the funding success for a resubmission application? …. Continue reading →

The Predictive Nature of Criterion Scores on Impact Score and Funding Outcomes

Posted on July 22, 2016 by Mike Lauer

In order to develop and implement data-driven policy, we need to carefully analyze our data to understand the “stories behind our metrics.” Without analyzing our data to know what’s going on, we’re essentially flying blind! A group of authors from the NIH Office of Extramural Research sought to investigate the stories behind peer review scoring and why some grant applications are more likely to be funded than others. They extended analyses previously reported by NIH’s Office of Extramural Research and National Institute of General Medical Studies. Last month, they published their analysis of over 123,000 competing R01 applications and described the correlations of individual component peer review scores – significance, investigator(s), innovation, approach, and environment – with subsequent overall impact score, and funding outcome. …. Continue reading →

Perspectives on Peer Review at the NIH

Posted on November 12, 2015 by Mike Lauer

In today’s New England Journal of Medicine, Richard Nakamura, the director of NIH’s Center for Scientific Review (CSR), and I published an essay titled “Reviewing Peer Review at the NIH.” As the competition for NIH research grants has become increasingly stiff, review scores often are pointed to as the reason for failure to obtain funding. Indeed, over the past few years, peer review has come under increasing scrutiny. Critics have argued that peer review fails in its primary mission – to help funding agencies make the best decisions about which projects and which investigators to support. …. Continue reading →

Understanding the Capacity of NIH’s Peer Review System

Posted on July 30, 2015 by Sally Rockey

My office devotes a lot of effort to evaluating its grants policies and practices. Since the introduction of the Enhancing Peer Review changes, NIH has sponsored an ongoing evaluation of peer review, involving formal surveys about the peer review process, as well as other types of analyses. We are keenly interested in your responses to our surveys, so if you receive an invitation to take a peer review survey, I hope you can find the time to respond. The importance of participating in these surveys is nicely illustrated …. Continue reading →

Reminder About Your Responsibilities in Maintaining the Integrity of NIH Peer Review

Posted on June 18, 2015 by Richard Nakamura and Sally Rockey

Today we’d like to start with a little scenario:

Kevin is a principal investigator whose application is undergoing peer review. At a national research symposium, he bumps into one of the reviewers on the study section that is evaluating his application. What should they do or say? The answer: …. Continue reading →

Connecting the What, When, Why, and How of NIH Peer Review

Posted on April 15, 2015 by Sally Rockey

As I’ve written before here on Rock Talk, peer review is the keystone of the scientific process. I’m excited to call your attention to a website update that connects the what, when, why, and how of NIH’s peer review process. …. Continue reading →

Supporting the Call to Peer Review Service

Posted on February 20, 2015 by Richard Nakamura and Sally Rockey

Last year we highlighted a new section of the NIH Data Book that provides statistics about peer review across NIH, including peer review organized by the Center of Scientific Review as well as peer review organized by NIH institutes and centers. Then, and now, these data truly demonstrate how you, our peer reviewers, are the lifeline of the scientific process. During 2014 nearly 24,000 reviewers participated in more than 2,500 peer review meetings to assess the scientific and technical merit of NIH applications, and many of those reviewers participated in multiple meetings. Your service to NIH expands our capacity to fund the best science, and provides an essential breadth of perspectives on the research projects and research training that NIH supports. In the lab and in the study section, we’re grateful for the myriad ways you work with us to advance human health and benefit the health of our nation, and the world. …. Continue reading →

New Efforts to Maximize Fairness in NIH Peer Review

Posted on May 29, 2014 by Richard Nakamura and Sally Rockey

We want you to know NIH is working on multiple fronts to get to the bottom of unexplained racial disparities in R01 grant funding and to maximize fairness in NIH peer review. Since the problems and the solutions are bigger than NIH, we have reached out to the scientific community and other concerned citizens for help. Now armed with a team of experts and a set of new initiatives, we’d like to tell you about our efforts to address this important issues –- particularly an exciting opportunity for you to submit your input. …. Continue reading →

NIH Extramural Nexus