Dr. Larry Tabak is the Principal Deputy Director of NIH.
Nothing could be more important to our enterprise than research rigor, assuring that the results of our work are reproducible. Our conversation with you on this topic began early last year with a commentary in Nature by Francis Collins and today’s guest blogger, Larry Tabak, on the importance of reproducibility and how NIH plans to enhance it. As described in a follow-up Rock Talk post, the topic of reproducibility is not new. Evidence has shown that too many biomedical-research publications are irreproducible. Thus this topic demanded our community’s immediate attention and we have had continued dialog with and participation by you over the course of the last 18 months to describe the issue, request information, launch pilots, and craft a way forward to enhance reproducibility.
Since that January 2014 Nature commentary, NIH has begun to address reproducibility from a number of different angles. In 2014, NIH worked alongside journal editors to develop a set of common principles to guide how research results are reported. In 2015, NIH published a series of videos as a resource intended to stimulate conversation in courses on experimental design. In addition to these efforts, NIH’s Office of Research on Women’s Health has led the discussion of the consideration of sex as an important biological variable that should be considered in designing experiments and reporting results. They have posted a variety of resources on their website, including outcomes of an October 2014 workshop where research experts discussed ways to strengthen scientific design by incorporating sex as a biological variable. The problem of misidentified cell lines also has an impact on research reproducibility, and a December 2014 article by NIH leadership focused on ways NIH can help catalyze improvements in cell line authenticity. There are a number of NIH institutes and offices that have completed or are embarking on other projects that contribute to the goal of improving rigor in the design and methods used in research (see PAR-13-383, RFA-DE-15-004, PAR-14-267 NOT-NS-11-023, NOT-MH-14-004, NOT-DA-14-007). Many other NIH efforts also support the goals of reproducibility, including data sharing, public access, PubMed Commons, and more.
All of these activities are culminating in changes to the grant application instructions and peer review criteria that we are planning to put in place for applications submitted in January 2016 and beyond. As announced in the NIH Guide today (NOT-OD-15-103), we are clarifying our long-standing expectations regarding the importance of rigor in scientific research. In this notice, we outline our expectations for the scientific community about describing the rigor of the research proposed in grant applications, with additions to the review criteria used to evaluate proposals. These changes will prompt applicants and reviewers to consider issues, which – if ignored – may impede the transparency needed to reproduce key results and thereby slow scientific progress. Included in our clarifications about rigor is our expectation for scientists to address sex among other biological variables, in order to improve the transparency of this fundamental aspect of science in shaping biological processes and outcomes critical to health (see NOT-OD-15-102). And finally, we ask you to tell us how key biological and chemical resources are authenticated to ensure their identity and validity.
These new additions to application and review practices represent aspects at the beginning of the research process which will promote greater transparency of the work we do and the research NIH funds. These additions complement the journal requirements for publication at the other end of the process. It is this strong foundation upon which research going forward relies, and we are confident that these changes will be embraced as an important step in lifting the entire research enterprise to even greater heights.