2 Comments
The role of preprints — complete and public draft manuscripts which have not gone through the formal peer review, editing, or journal publishing process – continues to be a hot topic in the biological and medical sciences. In January, three major biomedical research funders – HHMI, the MRC, and the Wellcome Trust, changed their policies to allow preprints to be cited in their progress reports and applications.
Thinking about preprints also raises questions about the broader class of interim research products, and the role they should play in NIH processes. Other interim products include products like preregistration of protocols or research methods, to publicly declare key elements of a research project in advance. While, under current policy, NIH does not restrict items cited in the research plan of an application, applicants cannot claim preprints in biosketches or progress reports.
So, in October, we issued a call for comments to get a fuller understanding of how the NIH-supported research community uses and thinks about interim research products. Today I’d like to follow up with what we’ve learned from your input, and the policy changes this feedback suggests.
We received 351 responses, the majority (79%) submitted by scientists/authors. Twenty-two professional societies representing groups of scientists also submitted responses. Of the 351 respondents who commented on how use of preprints & interim research products might impact the advancement of science, the majority were supportive, and some predicted or noted specific benefits, such as improving scientific rigor, increasing collaboration, and accelerating the dissemination of research findings. (See Fig. 1)
When asked about the peer review impact of citing interim products in NIH applications, the majority of respondents predict positive impacts. Some specific benefits noted include speeding the dissemination of science, helping junior investigators, and providing authors with the chance to incorporate feedback into their drafts and even form new collaborations.
We also received some concerns about these materials not being peer-reviewed, and whether any potential benefit they may offer to the review process was offset by potential burden to reviewers and applicants. However, the overall response about review was favorable. Respondents felt reviewers should be able to tell the difference between a final and interim product and could draw their own conclusions about the validity of the information. Again, it’s worth noting that these findings inform a potential increase in the use of interim products in review; we already have no restrictions in what can be cited in the reference section of a research plan.
Based on this general feedback and many other thoughtful suggestions, we developed guidance on how NIH applicants will have the option, for applications submitted for due dates of May 25 and beyond, to cite interim research products in applications. As described in the NIH Guide Notice issued Friday (NOT-OD-17-050), citations of interim research products in biosketches should follow citation formats that include citation of the object type (e.g. preprint), a digital object identifier (DOI) in the citation, and information about the document version. This guidance is also incorporated into NIH application instructions, which were just updated last week. We also offer FAQs.
|
The Guide Notice also outlines NIH’s expectations for what qualifies as a preprint, and suggests best practices to the many preprint repositories, including: open metadata; machine accessibility; transparent policies about plagiarism and other integrity issues; and an archival plan for content, versions and links to the published version.
For renewal applications submitted for the May 25, 2017 due date and thereafter, awardees can also claim these products on the progress report publication list (an attachment required specifically in renewal applications.) Awardees can also report these products on their research performance progress reports (RPPRs) as of May 25, 2017, and link them to their award in their My Bibliography account.
On behalf of NIH, I’d like to thank all of you who took the time to submit comments and share insightful and thoughtful viewpoints and experiences with us. There is a growing recognition that interim research products could speed the dissemination of science and enhance rigor
We see preprints and other interim products complementing the peer-reviewed literature. Our goal with this Guide notice is to offer clear guidance and suggested standards for those in the research community who are already using, or considering use of preprints and interim research products. Some scientific research communities may be more ready than others to use preprints –for example, there continue to be discussions and concerns specific to clinical research. We appreciate that different biomedical research disciplines are likely to adopt interim research products at varying paces; at the same time, with our new guidelines, we aim to make this option as viable as possible for all members of our community.
My concern is that reviewers might consider that some of the work proposed has been done, thereby not justifying the full amount of funds asked for to accomplish the work.
while I understand the positive attitude toward citing preprints, I think they make the review process for both applications and publication more complex and difficult. The reviewer’s task becomes more difficult and less transparent. Having served on both the reviewing side and application/manuscript submitter for many years, I believe that the proposed changes will make the peer review system on which science depends less effective.