Continuing Steps to Ensuring Credibility of NIH Research: Selecting Journals with Credible Practices

Posted

The scientific community is paying increasing attention to the quality practices of journals and publishers. NIH recently released a Guide notice (NOT-OD-18-011) to encourage authors to publish in journals that do not undermine the credibility, impact, and accuracy of their research findings. This notice aims to raise awareness about practices like changing publication fees without notice, lacking transparency in publication procedures, misrepresenting editorial boards, and/or using suspicious peer review.

This may not be a big problem for NIH-funded publications now; our colleagues Jennifer Marill, Kathryn Funk, and Jerry Sheehan from the National Library of Medicine note that more than 90% of the 815,000 publicly available journal articles reporting on NIH-funded research are published in MEDLINE indexed journals. Nonetheless, we do know that a problem exists – there are articles reporting NIH-funded research appearing in journals that engage in questionable practices. Ensuring the credibility of NIH funded research is important to maintaining public trust in research.

NIH has taken—and continues to take—many steps to ensure the credibility of the research it supports. From enhancing rigor and reproducibility, to encouraging sharing of data and protocols, to promoting pre-prints, and to requiring timely registration and reporting of clinical trial results, NIH establishes policies to make our funded research as credible, transparent, rigorous, and full of impact as possible.

But what can we do?

Simply put, publish where you cite. If you are not familiar with a particular journal, then consider speaking with your local academic librarian as well as consulting resources from the publishing community (e.g. Think Check Submit) and the federal government (e.g. Federal Trade Commission).

In addition, there are other ways you can enhance the credibility of your research and publications, including: using rigorous practices, such as authenticating cell lines; clearly documenting methodology so others can replicate your work; sharing data; preregistering protocols; and issuing preprints to collect community feedback prior to publication.

All in all, to help convey the credibility of your work, be careful where you publish. We hope that our community publishes only in journals that do what they say they will do. If the rigor of your work is clearly conveyed in writing, and published in journals that maintain high quality standards, then your work will be viewed with respect. By taking these approaches, we can continue ensuring the credibility and trustworthiness of the biomedical and behavioral research findings resulting from public support.

9 Comments

  1. Thank you for addressing this important topic. What are investigators to do about publishing in reputable journals in their field that have page charges? Many investigators, especially junior investigators, do not have enough grant money or institutional support to do this? Can NIH do anything to help with this scientifically biased practice?

    1. NIH appreciates the financial constraints that may be associated with publishing NIH-funded research. Though we cannot mandate journals change their fee structures, ublication costs, including author fees, may be charged to NIH grants and contracts—as noted on the FAQs page for the NIH Public Access policy. This may be done if (1) such costs incurred are actual, allowable, and reasonable to advance the objectives of the award; (2) costs are charged consistently regardless of the source of support; and (3) all other applicable rules on allowability of costs are met. In situations where your grant does not have sufficient funds to cover publication costs, or the grant has expired, we recommend you consult with your institutional official for advice and options.

  2. This is a very valid discussion but neglects to point out that predatory journals are different from new journals that are credible but not yet indexed on Medline. This article might cause confusion amongst NIH funded authors in his regard. It isn’t always possible to publish where you cite.
    I am Editor in Chief of Research and Practice in Thrombosis and Haemostasis, a new open access journal that is an official journal of the International Society on Thrombosis and Haemostasis. We provide rigorous peer review and are following practices that we anticipate will lead to Medline indexing. We welcome scientists to join other NIH funded investigators that are submitting their work to us. We have written about this in our latest issue available online.

  3. One possible way to improve credibility would be for journals to share some burden of reporting scientific malpractice. Typically, if a reviewer spots an ethical problem in a manuscript (such as apparent data faking by the authors), or if a journal itself notices a problem when running image analysis software, the journal rejects the manuscript but is unlikely to pursue the concern with the authors’ institution. But then, the manuscript will just end up published somewhere else. If journals were obligated to bring concerns about unethical scientific practice to the authors’ institution, then there would be a better chance to root out these problems, rather than passing them along.

    NIH could help in this endeavor by insisting that Pubmed-indexed journals take this kind of reporting step when they identify potential data fakery.

    1. In terms of preserving the credibility of the scientific enterprise, concerns about data falsification and/or “selective reporting” (especially with a profit motive) dwarf all other considerations. From personal experience on more than one occasion, I would say that:

      – relatively few reviewers pick up on the clues that suggest falsification
      – @Bagnall is absolutely right in saying that current practice seems to be to pass the buck down to the next journal (even in cases where everyone involved in the review is unanimous in thinking that the submission likely represents a case of falsification)
      – chances to catch falsification diminish with each successive journal because the authors get another chance to hide their tracks, thus benefiting from issues raised during the previous round of review.

      If you wanted to design a system that coached cheaters into how to effectively game the system, you could not do much better than the system we have now.

      A fundamental problem is that prosecuting cases of falsification is an exercise that surely benefits from experience. Hoping that individual faculty members will become vessels that hold and pass on this experience seems crazy. To me it seems that the proper place to accumulate this kind of expertise is with the journals themselves. If the NIH can nudge them in this direction, it could have huge impact.

  4. Every time I have a new paper indexed on PUBMED, I am immediately besieged by, literally, 50 or more predatory publishers of the type you’re describing. Virtually all appear to have software that trolls PUBMED for new articles, inserts the title into a form letter that’s obviously provided with the software (format is virtually identical, regardless of the publisher), and sends a request to publish to any authors who have an e-mail address listed in the PUBMED file under “Author Information”. It does no good to junk-mail these requests because they just make a minor change in their address and re-send (e.g., editor@predatorypublisher.com becomes editor2@predatorypublisher.com or even editor@predatorypublisher.org). I am convinced that this is being done by some commercially-available trolling software because the request letters are so much alike. Is there some way PUBMED can stop such trolling?

  5. As long as mediocre and cronies are sitting in the Study Section, there will be no improvement in the NIH funding system for researchers who are not part of the ‘gang’. These reviewers comment more on PIs or with generic statements than scientific contents of the application. Their hate and prejudice are also reflected in the Summary Statements. This destroys ideas, genuine research grant….Yet, PO and division Directors are deaf and blind. There is no punishment for such reviewers who continue to destroy careers of some PIs but continue to sit in the Study Sections. Thus, NIH Funding System will remain CORRUPT. How much Citation for papers contributes to scoring 1 through 9. It does not matter much. The Hypothetical Scores between 1 and 3 (depends on the DESIRE of the reviewer) makes the difference. The two initial reviewers decide whether the application will be forwarded for discussion or be prematurely killed before anybody knows…you have to be Crony of these two or at least one!!! I am humbly drawing attention of Dr. Michael Lauer to this real problem.

  6. It would great if NIH could publish a “whitelist” of acceptable journals or publishers that we can use, in addition to the guide notice (NOT-OD-18-011). There are plenty of “potentially” predatory journals and publishers that are listed on PubMed. In the past, most of us used to look up Beall’s List of “potentially” predatory journals. Unfortunately, that site had been taken down. A “whitelist” will prevent those journals from threatening lawsuits against NIH if such a list is released, plus it would avoid guesswork on our part in selecting journals. Such a list/webpage would be helpful, similar to the webpage that NIH released recently to help us select study sections when submitting our applications.

  7. I believe much of the fault lies with the journals. And not just the predatory ones but also the ones that we all publish in, up to and including the society journals. They are under constant pressure today to get more content/submissions and to be more selective in accepting articles at the same time. Thus, they are unwilling to enforce rigor, appropriate statistics, cell line or antibody verification, etc. A few years back I proposed some minimal standards in writing and contacted several journals who agreed to those rules but none have followed through. My request was simply to insist that epitope tagged proteins be correctly identified and labeled in text and figures. It is not accurate to put into Methods that you are using a tagged, expressed protein in all your studies and then refer throughout to the native protein, with no mention of that tag. Even this simple issue could be addressed by journals but is not.

Before submitting your comment, please review our blog comment policies.

Leave a Reply

Your email address will not be published. Required fields are marked *