4 Comments
Guest post by Noni Byrnes, Director of the NIH Center for Scientific Review (CSR), originally released on the Review Matters blog
On March 1, NIH Director Francis Collins announced NIH’s broad-based initiative, UNITE, to end structural racism and racial inequities in biomedical science. This is a recognition of the need for urgent, sustained effort on many fronts across the research enterprise, including in all parts of the NIH’s extramural processes, to change culture. While the NIH Institutes and Centers will examine their programmatic priorities and discretionary funding practices, here at CSR, we are committed to pushing ahead with efforts to protect the peer review process from the systemic biases that exist in all areas of the scientific community.
In the June 2020 Review Matters blog, I wrote about some of the steps that CSR is taking to address individual and systemic biases in peer review. Following that, in July 2020, we held three community listening sessions, in which we heard the rightful anger and the call for urgent and specific action around the persistent funding disparity for Black investigators. I shared the report and recommendations from those forums with NIH leadership, with the UNITE E group that is focused on extramural changes, as well as with our own CSR Advisory Council. Since then, I have held a number of individual and small group conversations with investigators, who shared their personal experiences of bias as an applicant or reviewer, which has helped us further refine the strategies we were already pursuing, as well as develop some new approaches. Below are a few of the actions we are taking:
- Reporting: Many of you asked for a way to report concerns regarding bias in the peer review process directly to CSR management. Our Associate Director for Diversity & Workforce Development, Dr. Gabriel Fosu, will serve as a reporting avenue for any concerns around fairness in review. Dr. Fosu reports directly to me and I will see all reports. Beginning on March 15, all CSR scientific review officers (SROs) and staff will provide this information in their email signature lines. To report concerns around fairness in review: G.Fosu_AssocDir@csr.nih.gov.
- “Bias awareness in review” training for SROs, Reviewers, Chairs: Despite a brief interruption due to an executive order that has since been rescinded, we are forging ahead with the development of an interactive training module on bias. It will include a range of nuanced case studies to raise awareness of potential biases and mitigation strategies and tools for bystanders. We plan to launch the training for all CSR reviewers, chairs and SROs in August 2021.
- Diversifying and broadening the pool of reviewers: Through repeated multi-pronged communication efforts, CSR leadership has made our expectation and commitment to improving demographic diversity in special emphasis and chartered panels clear to staff. We have posted data on reviewer demographics and those data will be updated regularly. CSR is working hard to expand the pool of well-qualified reviewers and to build tools to help our SROs identify new scientists to bring to NIH review.
- Decoupling the science from the investigator/environment: The systemic advantages of reputation, network and pedigree are deeply entrenched in the culture of the biomedical research enterprise, and rarely benefit minorities and women, who are less likely to be part of these networks. The peer review system is certainly not immune from this systemic bias. In an effort to address the effect of these positive biases on the evaluation of scientific merit, CSR is investigating the use of blinded review processes. In collaboration with the NIH Common Fund, CSR is now conducting a pilot of a multi-stage, partial double-blinded review tR01 applications (April 2021 review). In addition, a Working Group of the CSR Advisory Council is spearheading efforts to simplify peer review criteria, recommending a decoupling of the science from the investigator/environment criteria, opening the door for a first-stage review of the science without knowledge of the investigator or institution.
Culture change is never easy, and I expect that there will be resistance from some who have benefited from the status quo. Together with all of CSR’s senior management, I stand with the NIH Director and leadership as we commit to making the changes that are long overdue.
Thank you for collecting these data. The reviewer demographics tracking does not include individuals with disabilities and individuals from disadvantaged backgrounds. Under represented minorities are defined as racial and ethnic minorities in the data release while the notice that the release cites (NOT OD-20-031) also includes disability and disadvantaged background. Will these latter indicators ever be tracked in NIH data collection?
The only consideration of any consequence is that of ability. Any other consideration is doing a disservice to the entire population of the US. Having raised over $160 million in NIH funding over the past 10 years and having been involved in the grant review process since 1993, I say without a doubt that the quality of the review process has taken a serious dive over the past 5 or so years. There is a simple way to correct this and it is not by engaging reviewers on the basis of criteria other than ability. It is odd that CSR desires to separate the science from the investigator/institution, so that the reviewer can focus on the science itself, yet CSR wants to engage reviewers based on things other than their scientific ability and accomplishments. How ironic. If you let politics and sloppy thinking run science, then the science will suffer. This is quite obvious, but politics seems to be running science now. If the desire is the improve the review process, then one needs to give reviewers feedback on their reviews, especially when score driving factual errors occur…which is far more often now than it was before, at least in my experience. The reviewers need feedback on their entire review, since so many reviews contain review points that are not relevant or incorrectly applied, along with the factual errors. The SRO needs to write these reviews and send them to the reviewers and go over the review with the reviewers on a one on one basis. When score driving factual errors occur in a review, then the application needs to be re-reviewed. Right now, only an admin error will allow for a re-review. This is unfair to the entire country since the best science is not getting funded or is being delayed by such errors and other errors. If CSR wanted to improve the review process, then CSR would take a root cause analytical approach. Instead, they are trying to improve the review quality by hiring people based on non-scientific qualities. Let me know how that works out for you.
My grant was not discussed because my marks as investigator/institutional support were very low because I was competing with more established faculties. As a consequence of not getting my grant scored, despite receiving high marks in innovation and science, I now have to leave science because I ran out of funding. Another minority bites the dust.
I’m a minority faculty member who was hired as a diversity candidate at a top research institution. I was given a smaller startup package and less resources to support my work, lab space that didn’t make any sense for my work, and little support from my senior colleagues. I was unable to get NIH funding at this institution, so I left without tenure. I was able to find a new faculty position in a more supportive environment with better medical connections at another top research institute; however, again I received less startup funding then other hires and my contract specifically says that I must have R01 support to receive tenure and to be promoted to associate professor. I have submitted multiple NIH grants since moving to this other university. Reviewers are very critical of my applications because of my status as an assistant professor and because I have not had previous NIH funding. My summary statements often have comments like this one (“The applicant has apparently been Assistant Professor for over 10 years and has not obtained an R01”) and my scores don’t match the strengths and weaknesses presented by the reviewers. I just wish the panels could judge my applications based on their scientific merit. I know several minority faculty members who have fallen into this same trap and most of them just disappear from academia. At this point, I guess this will likely happen to me as well in spite of my persistence.