22 Comments
In August and September we released case studies and FAQs to help those of you doing human subjects research to determine whether your research study meets the NIH definition of a clinical trial. Correctly making this determination is important to ensure you are following the initiatives we have been implementing to improve the transparency of clinical trials, including the need to pick clinical trial -specific funding opportunity announcements for due dates of January 25, 2018 and beyond.
We have made no changes to the NIH definition of a clinical trial, or how the definition is interpreted. What we have done is revise existing case studies and add a few new ones to help clarify how the definition of clinical trial does or does not apply to: studies of delivery of standard clinical care, device studies, natural experiments, preliminary studies for study procedures, and studies that are primarily focused on the nature or quality of measurements as opposed to biomedical or behavioral outcomes..
As a reminder, the case studies illustrate how to apply the four questions researchers involved in human studies need to ask, and answer, to determine if their study meets the NIH definition of a clinical trial. These questions are:
- Does the study involve human participants?
- Are the participants prospectively assigned to an intervention?
- Is the study designed to evaluate the effect of the intervention on the participants?
- Is the effect that will be evaluated a health-related biomedical or behavioral outcome?
If the answer to all four questions is yes, then we consider your research a clinical trial.
Note that If the answers to the 4 questions are yes, your study meets the NIH definition of a clinical trial, even if…
- You are studying healthy participants
- Your study does not have a comparison group (e.g., placebo or control)
- Your study is only designed to assess the pharmacokinetics, safety, and/or maximum tolerated dose of an investigational drug
- Your study is utilizing a behavioral intervention
Studies intended solely to refine measures are not considered clinical trials.
The adjustments to the case studies include the following:
- #7a, #8a, #24, #31a: Clarified whether it meets definition of intervention
- #18c: Replaced with a more illustrative case study
- #18d, 24, and 33: Clarified whether study was designed to assess the nature or quality of a measurement, as opposed to the effect of an intervention on a behavioral or biomedical outcome.
- #18g: New case study about testing procedures
- #36 a-b: New case studies about standard clinical care
- #37: New case study about Phase 1 device studies
- #38: New case study about natural experiments.
- #39: Proposed case study about preliminary tests for study procedures.
- New case studies specific to select NIH Institutes and Centers
We recognize that sometimes in an attempt to be helpful we end up providing a lot of material to look through. So to help you quickly find the case studies that are most relevant to your research we have added the ability to filter the case studies by keyword.
We also added two new FAQs on standard clinical care and Phase 1 devices.
Thank you for your continuing dialog on this topic. We look forward to continuing to work with you as we move towards higher levels of trust and transparency with our clinical trials.
Update: Some of these case studies have been revised since this publication.
“We have made no changes to the NIH definition of a clinical trial, or how the definition is interpreted.”
Heaven forbid the NIH responds to investigators’ concerns.
Hi,
I appreciate the fact that the NIH is working to refine it’s case studies, which have been largely helpful. However, I would like to point out what I think is not so helpful in your revision of 18c, and compare it to 18a. In the former, the case study states that feedback to subjects of winning or losing in a gambling task would be an intervention qualifying as a clinical trial. But in 18a, which could involve a working memory task, is not a clinical trial. However, what happens if after each memory trial, a subject were to receive correct/incorrect feedback? This would affect some of the same brain areas as the win/lose feedback in a gambling task. The implication of this is that studying working memory with feedback is a clinical trial, but without feedback, it’s not a clinical trial? Why is it that studying reward processes makes it a clinical trial, but studying memory processes are not? It becomes very hard to discern the rules that you are using as to what constitutes an intervention, and what does not.
Steve Taylor
The distinction between 18A (an fMRI study that is not a clinical trial) and 18C (an fMRI study that is a clinical trial) is not at all clearcut. Please consider:
– In case 18A, fMRI is used (“Participants are administered … brain scans (e.g., fMRI)”), and the determination is made that the research project is not a clinical trial.
– However, in case 18C, fMRI is used (“The investigators will measure the comparative effects of ‘ wins’ and ‘losses’ on brain function (fMRI in striatal regions) during the gambling task.”), and the determination is made that the research project is a clinical trial.
This naturally raises the question: Would the findings of case 18C apply if the (unspecified) fMRI task in case 18A involved ‘wins’ and ‘losses’? If so, then how shall fMRI studies be assessed? Do all tasks designed to probe (say) working memory and early visual processing fall under case 18A, while tasks designed to probe the reward system are to be considered clinical trials, per Case 18C?
How did we get here?
The real issue is that such nit-picking arguments should be unnecessary, given the clear divide between basic research and clinical trials.
In biology, the idea of a “manipulation” is fundamental to the definition of experimental (vs. observational) science. Basic experimental science is about seeing how manipulations affect measurements, while clinical trials are about seeing how interventions affect health or biomedical-related outcomes. Historically, there has been little confusion regarding these distinctions, but NIH’s published “Cases” avoid this historical understanding and instead manifest expansive readings of “intervention” and “health outcomes” so as to create a vast enlargement of the category of clinical trials, in a manner that diverges from NIH’s original statement of purpose, as the 2014 definition stated that new definition “is not intended to expand the scope of the category of clinical trials”.
The (now twice-revised) “Cases” continue to disregard historical understanding, remain inconsistent with NIH’s 2014 statement of purpose, and create unnecessary and harmful confusion.
James J. Pekar, Ph.D.
Professor of Radiology,
Johns Hopkins University
Amen- I completely agree with Dr Pekar’s statements. Clinical Trials designation should be reserved for “long term’ intervention (i.e. in the case of behavioral or a drug administration clinical trial should encompass continuous and long-term administration, not single discrete administration and measurement) at least in my opinion.
In my opinion, to say that anything involving human subjects except observational or epi studies are clinical trials is a gross change in definition of clinical trials as well as accepted experimental procedure. Also, I don’t think all these changes will accomplish the underlying problem of clinical trial results not being published and publicaly available. NIH could accomplish that by just making final grant report abstracts available on the REPORTER system
Jennifer Nasser, PhD, RD
Drexel University
The comment by James Pekar above is completely on the mark.
Cases 18a, 18b, and 18c reveal the confusion about what is ‘modifying’ vs. ‘measuring’ behavior. The problem is that *every* experiment (say a brain scanning fMRI experiment) is designed to ‘modify’ the behavior in some sense.
18b says that if you administer standard cognitive tasks then it is not a clinical trial, because ‘The standard cognitive tasks and the fMRI are being performed to measure and describe brain activity, but not to modify it.’ But of course any task modifies the brain activity: that is the whole point of the task. You would not do a task if it had no effect on brain activity or behavior. If you administer a standard sustained attention task, you will induce activation in attention related areas of the brain, which is a modification from the normal state. You then measure and describe it.
18c says that if you assign subjects to ‘win’ and ‘loss’ conditions it is a clinical trial because you are modifying brain activity. Again, every task modifies brain activity. How is win/loss different from say attend/not attend, read words/read nonwords, see faces/see houses?
If you see faces, that includes activity in face-processing areas of brain. If you win, that induces activity in reward related areas. Why exactly is the latter a clinical trial and the former not (I assume)?
Vastly expanding the definition of clinical trial is clearly harmful and burdensome.
What are the requirements for the NIH definition for FDA clinical trials that involve a medical device. For example, evaluation of a rapid HIV test on prospectively collected samples from subjects, as compared with results from a reference test?
It is deeply disappointing to see the NIH callously disregard and dismiss such overwhelming concerns. Personally, I have yet to meet a single practicing researcher who agrees with the NIH’s approach or responses.
The APA and FABBS sent letters last July on this matter, yet their concerns were apparently not seriously considered. Was there a public, point-by-point response? This isn’t an issue of clarifying boundary conditions or edge-cases, but of wholesale, fundamental issues with the structure of this policy.
The NIH leadership needs to do some serious self-reflection about the negative impacts their conflation of basic, translational and clinical science. This goes far beyond administrative bloat and bureaucratic burden.
What will happen if NIH and/or reviewers disagree with an investigator’s assessment of whether their research is a clinical trial or not? That is, what if NIH and/or the reviewers think the application was submitted under the wrong PA?
Four comments
1) How all this effort will solve the stated problem of non-publication is manifestly unclear.
2) When a policy requires complex explanations illustrated by debatable cases there is a problem.
3) We all want to do the right thing but there must be a better way to do it. The FTEs involved in all this effort could be better spent.
4) NIDDK case #3 is classified as not a clinical trial because “a brief bout of exercise” is not regarded as an intervention. What if it was “a bout of exercise” or “a bout of prolonged exercise” or “running 2 miles” or “an exercise program” – which of these would these be interventions? (In the previous NIDDK case #2 fasting is regarded as an intervention).
Mike. Dude. Please for the love of all holy consult with some more human cognition researchers before continuing to refine these case studies in response to our collective outrage and confusion. As many of us pointed out last round, the case studies are inconsistent with one another. Now, #18c just makes no sense compared to the other examples under #18. There is no way to “assess brain activity under standard laboratory conditions” without having some kind of manipulation involved. The “wins vs. losses” in #18c is just one of the many ways that brain activity is assessed under standard laboratory conditions. These examples continue to fail to capture basic aspects of the experimental logic of fMRI (and related methods). I am still not sure how my next grant will be categorized and who at NIH I will have to argue with when trying to figure out if my “standard laboratory conditions” do or do not count as a “manipulation.”
While I appreciate the efforts that NIH has made to make the case studies more useful for applicants, I believe there are still improvements that can be made and that will minimize the questions that program officers and other staff will need to address under these new policies.
Since the revised (September 2017) version of the case studies was released, NIH has started to divide clinical trials into two categories: mechanistic and clinical efficacy. It would be very useful if the case studies could reflect this distinction, since some FOAs are specifically designated for one or the other, since it would add depth to the definitions and draw distinctions between the two.
More generally, I’d like to see the logic behind some of the new and revised case studies expanded on and clarified. Looking at some of the subcases in Case 18 (particularly 18c and 18e), it appears that the underlying rule for whether a study is a clinical trial is whether there is any brain activity that can be detected as part of the study procedure. This is an interesting standard, as any task (including passively being in the MRI) will result in brain activity, which is considered a biomedical outcome under this standard.
The logic here seems more curious in contrast with Case 24, where asking someone about the legibility of a text is not a clinical trial, but Case 26, which asks about comprehension, rather than legibility, is. Overall, the current standard – that brain activity of any sort, resulting from any experimental manipulation – requires compliance with clinical trial registration and reporting requirements seems to be used somewhat unevenly.
I do appreciate seeing Case 18g and Case 39, which provide guidance on pilot studies, which are essential for developing good experiments, and I appreciate that pilot studies are exempted from the need to register.
I want to end by saying that I believe that requiring registration of studies, as both a matter of diligence in government-funded research, and out of respect for our subjects, is essential. That said, I believe that NIH could achieve this goal with respect to what are now being labeled “mechanistic clinical trials” by moving away from the clinical trials language, and simply calling those experiments “mechanistic trials” to reflect their basic, non-clinical nature.
I’m particularly concerned here that, to people outside the research community, the phrase “clinical trial” has justifiable weight, and that by including these basic science studies under that umbrella, we are collectively opening ourselves to problems down the line. In general, drugs and treatments tested with clinical trials are considered, by the public, to be efficacious, and I am concerned that “mechanistic clinical trials” will be used to misrepresent basic research to the public, and potentially to Congress. I fully support NIH’s push for preregistration of basic science studies, under the designation of “mechanistic trials”, but I believe they should be separated from clinical trials, to both reflect the different goals of clinical and basic research, and to prevent confusion.
Benjamin Wolfe, PhD
CSAIL, MIT
I appreciate NIH’s continued efforts to refine the case examples. I concur with the concerns that continue to be expressed about Case 18 and its variants. I believe that Case 9, which is a clinical trial simply because the experimenters assign participants to various sleep durations and measure changes in stress hormone levels, also merits further consideration. For Case 18 variants and Case 9, there seems to be no room for distinguishing between a biomedical consequence that is measured as part of a basic research design aiming to test a theory about mechanisms of action or involved systems, as compared to an outcome measure specifically targeted for change, due to its potential for therapeutic benefit.
Regarding the ever-broadening NIH clinical trial definition case studies:
As a behavioral scientist, I¹m worried that Cases 24 & 26 contradict each
other, and neither should be a clinical trial. The substantive difference
appears to be that #24 involves opinions (so not a trial) and #26 involves
memory, which is suddenly a health-related outcome (yes a trial). All
memory, really? No opinions, really? What if #24 were about the
readability of consent forms; would it suddenly be a clinical trial? What
if #26 were about memory for announcements; would it no longer be a
clinical trial? These examples confound modality (opinion/memory) with
content (announcement/consent), so the examples are ambiguous as to which
dimension is decisive.
Quoting from the webpage:
Case 24
The study involves evaluating different types of printed announcements to
identify the best designs for ensuring comprehension and retention of
information in adults. Visitors to public libraries will be selected at
random and asked to read one of the two announcements and then to take a
short survey to elicit their perspectives about readability.
* Does the study involve human participants? Yes, the visitors to the
library are human participants.
* Are the participants prospectively assigned to an intervention? Yes, the
participants will be prospectively assigned to an intervention, reading
printed materials.
* Is the study designed to evaluate the effect of the intervention on the
participants? No, the study is designed to learn about participants¹
opinions. It is not designed to evaluate the effect of the printed
materials on the participants health-related biomedical or behavioral
outcomes. There is no change to the participant as a result of providing
an opinion about readability of the printed materials.
Case 26
The study involves randomizing individuals to different processes for
informed consent. It is designed to assess the effectiveness of
interactive and multimedia components in enhancing participants¹
understanding of the study¹s purpose and procedures.
* Does the study involve human participants? Yes, the individuals assigned
to the different consent processes are human participants.
* Are the participants prospectively assigned to an intervention? Yes, the
participants are prospectively assigned to an intervention, different
consent processes.
* Is the study designed to evaluate the effect of the intervention on the
participants? Yes, the study is designed to evaluate the effect of
different informed consent processes on understanding the study.
* Is the effect being evaluated a health-related biomedical or behavioral
outcome? Yes, enhanced comprehension of information is a health-related
behavioral outcome.
I agree with the other comments made above. It is deeply disappointing and disheartening that NIH is unable to address concerns from the scientific community in order to modify the clinical trial definition with logic and humility.
Regarding case 18 (in particular 18c), we’ve heard your constructive feedback regarding inadequate clarity, engaged in follow-up discussion with outside stakeholders involved in this field of research, and, with their help revised case #18(a-g).
Mike, can you please do a blog that addresses what constitutes a “study” and “study record” in the new PHS HS/CT Form. I’m an administrator, and I’m getting this question quite a bit from faculty. The instructions state to add a separate study record for each protocol involving human subjects, but what constitutes a protocol? Is it each separate hypothesis? Each aim may have several hypotheses, does a study record have to be completed for each one? Is it per aim or per hypothesis? I understand that every study is unique and may require different approaches. Or, the specific FOA may even dictate what to do! Can I direct researchers directly to NIH program staff for further clarification for specific studies? Any clarification would help. Thank you!
Mike addressed a similar question recently in an Nature Human Behavior Interview. In that interview he provides the following guidance. “Your question gets to the tension between lumping and splitting. We would encourage lumping. That is, to the extent possible, describe a series of experiments as a set of variants of a single design. Then one registration can cover all of them. The FORMS-E grant proposal will work in a similar way; you could describe several studies as variations of one basic experiment.
Thus it is certainly possible, indeed likely, that one protocol will cover multiple hypotheses. It’s certainly conceivable that one study record could cover multiple aims. Think lumping, not splitting.
Do I understand correctly that a single-session study with repeated measures would NOT count as a clinical trial, because it does not meet criterion #2? Thanks.
As we are not familiar with your specific study design, we recommend you speak with program officials at your prospective funding NIH Institute or Center to determine if it would be considered a clinical trial.
Has NIH issued any updates regarding the interpretation of the NIH clinical trial definition and registration and reporting requirements since the omnibus budget passed last month?
“The 2018 Omnibus is referencing the September 21, 2016 (not 2017) NIH policy on registering and reporting clinical trial results in ClinicalTrials.gov. NIH is working with Congressional members and professional societies to address specific concerns about reporting basic science trials to ClinicalTrials.gov as we move forward. We hope to have more to share after these discussions.”
It would be helpful if all the case answered all 4 questions, not just the cases that are clinical trials. Additionally, it would be helpful if the NEI could provide some example cases to add to this list. Thanks.