8 Comments
The fourth and final segment in our series on rigor and transparency in research grant and career development award applications focuses on authentication of key biological and/or chemical resources.
Research performed with unreliable or misidentified resources can negate years of hard work and eliminate any chance for a study to be reproduced or expanded upon. For this reason, it is imperative that researchers regularly authenticate key resources used in their research.
Updated application instructions under “Additional Attachments” ask the applicant to:
Briefly describe methods to ensure the identity and validity of key biological and/or chemical resources used in the proposed studies.
- Key biological and/or chemical resources may or may not be generated with NIH funds and: 1) may differ from laboratory to laboratory or over time; 2) may have qualities and/or qualifications that could influence the research data; and 3) are integral to the proposed research. These include, but are not limited to, cell lines, specialty chemicals, antibodies, and other biologics
- Standard laboratory reagents that are not expected to vary do not need to be included in the plan. Examples are buffers and other common biologicals or chemicals.
The authentication plan should be included as an additional attachment (not as part of the research strategy), and it should should state, in one page or less, how you will authenticate key resources, including the frequency, as needed for your proposed research. The resources that require authentication will vary depending on the reagent/resource and the experimental context in which it will be used. You do not need to provide authentication data itself in this one page attachment; reviewers will be asked to assess the adequacy of the plans you propose for authenticating key resources.
Purchased or established resources may have been authenticated prior to receipt, and the vendor may have included a specification sheet with the product. If the authentication data provided by the vendor meets your needs in terms of how the product will be used, this may be mentioned this in the plan, but you should also include a plan to independently verify the identity and activity of the product before use. If the product will be used long-term, consider the stability of the product and how the validity of the product will be assessed over time.
Key resources developed in-house should also be regularly authenticated and plans to do so should be provided in this section.
The methods used for authentication will depend on the key resource type, and methods may vary by research field. For instance, key cell lines might be authenticated by chromosomal analysis or short tandem repeat (STR) profiling. Key antibodies might be validated by Western blot, ELISA, immunoprecipitation, immunofluorescence, or flow cytometry using knockdown cells and positive and negative controls, depending on the assay proposed. Key chemicals might be validated by liquid or gas chromatography or mass spectrometry. Authentication plans should be based on accepted practices in the applicable field of science.
NIH encourages the scientific community to establish guidelines for the authentication of various types of key resources. Proper attention to all areas of rigor and transparency will help us ensure that science moves forward in the most efficient manner.
These new rules are well intentioned and rational, but they will not fix the problem of poor reproducibility in acacemic research labs!
You are correct that one such policy will not fix the larger problem. But it is clearly a step in the right direction. A challenge is clearly to address the “real” issues without layering on yet more administrative burden. I suggest that the focus be on journals and editors to enforce a standard, which should be openly displayed and then adhered to. Currently, journals have abdicated almost all gatekeeper responsibilities yet they remain the gold currency for productivity.
Are RRIDs not the right solution here? These are standard identifiers taken care of by the community data repositories, which do not make reproducibility problems disappear, but they are the answer to the first question, which biological resource did you use and which other papers used the same resource (antibody, cell line etc)?
The UCSD Library has prepared an example document for this, which includes standard language and the appropriate authorities such as ICLAC for cell lines.
Thank you for the example. It is very helpful. However, the instructions state: “The authentication plan should be included as an additional attachment (not as part of the research strategy), and it should should state, in one page or less”. Certainly on can use what you wrote as a guide. It is a big service. Thank you!
This is partially a consequence of reduced length in NIH proposals. An increasing number of grants are reviewed and yes funded! with inadequate Methodology. RRIDs should be mandatory for peer review. Which brings us to the second source of the problem: proliferation of Open Access Journals, rife with inadequate review, plagiarism, and a pay for play mentality.
You are not alone. In Denmark we have almost the same rules in laboratories. But anyway it is a guide and we have to stick to it.