30 Comments
In 2011, Ginther et al. first demonstrated that African American and Black applicants to the National Institutes of Health received grant awards at a lower rate than their white counterparts (Ginther 2011). Since then, multiple studies have reproduced and extended this finding (Ginther 2011; Ginther 2016; Hoppe 2019; Erosheva 2020). Recently we reported that African American and Black (AAB) PIs are more likely to propose research on topics that are less likely to be funded (Hoppe 2019). We found that topic choice has little or no effect on whether an application is chosen for discussion, but after considering a number of confounders, it accounts for over 20% of the gap in funding success for applications that are discussed (Hoppe 2019).
Why are applications linked to certain topics less likely to be funded? Review is not the only determinant that considers whether any given application will be funded. At the same time that applications are assigned to a study section for review, they are also independently assigned to a funding Institute or Center (IC), based in large part on the topic of the work. Figure 1 shows that ICs have widely varying award rates (the ratio of funded applications to all applications). These marked variations (from 9.1% to 26.9%) may explain funding differences, a possibility that we did not consider in Hoppe 2019.
Figure 1. IC award rates for R01 applications, FY11-FY15. MD, Minority Health and Health Disparities; AT, Center for Complimentary and Integrative Health; HD, Child Health and Human Development; NR, Nursing Research; CA, Cancer; EB, Biomedical Imaging and Bioengineering; ES, Environmental Health Sciences; AG, Aging; TW, Fogarty Center; AI, Allergy and Infectious Diseases; AR, Arthritis and Musculoskeletal and Skin Diseases; HL, Heart Lung and Blood; AA, Alcohol Abuse; LM, Library of Medicine; DA, Drug Abuse; DK, Diabetes and Digestive and Kidney Diseases; NS, Neurological Disorders and Stroke; DE, Dental; MH, Mental Health; GM, General Medical Sciences; DC, Deafness and Communication; HG, Human Genome Research; EY, Eye. N = 157,405 competing (Type 1 and Type 2) applications.
Notably, five of the six ICs that received a higher than average proportion of applications from African American and Black PIs – Minority Health Disparities, Nursing Research, Child Health and Human Development, Environmental Health Sciences, and Allergy and Infectious Diseases – had an R01 award rate that is below the NIH average (Table 1).
| IC | % AA/B | R01 award rate |
| MD | 16.1 | *9.1 |
| TW | 9.6 | 14.8 |
| NR | 4.3 | *11.8 |
| HD | 2.7 | *11.6 |
| ES | 2.0 | *13.6 |
| AI | 2.0 | *15.1 |
Table 1. Institutes that received a higher than average proportion of applications from African American and Black PIs. IC, Institute or Center. %AA/B, percent of R01 applications FY11-FY15 that NIH received from African American and Black PIs. Overall, 1.5% of R01 applications to NIH were from AA/B PIs. Award rates are determined by dividing the number of competing applications funded by the number of competing applications reviewed; if the same project is submitted more than once in the same fiscal year, the two submissions are both counted as independent attempts to secure funding. R01 award rate is given for each IC. MD, Minority Health and Health Disparities; TW, Fogarty Center; NR, Nursing Research; HD, Child Health and Human Development; ES, Environmental and Health Sciences; AI, Allergy and Infectious Disease. Rates significantly below the NIH average of 16.3% are marked with a ‘*’; in all cases, p < 0.0001, Fisher’s exact test. The Fogarty Center (TW) does not meet the criteria for statistical significance (p = 0.8) due to the relatively small size of their portfolio.
These six ICs that received a higher than average proportion of applications from African American and Black (AAB) PIs (“ICs Higher AAB PIs”) received 19% of all R01 applications for FY2011-2015, but they accounted for 35% (796 out of 2274) of applications from AAB PIs (Table 2). Applications submitted to these ICs had similar rates of discussion and, for those applications that were discussed, similar median and mean priority scores and percentile rankings; award rates were 24% lower (Table 2).
| IC Characteristic or Outcome | ICs Higher AAB PIs (N applications = 29,285) | All Other ICs (N applications = 128,120) |
| PI AAB | 3% (796) | 1% (1478) |
| Discussed | 55% (15,980) | 55% (70,369) |
| Priority Score Median (25th-75th percentile) | 36 (26-45) | 36 (26-45) |
| Score Mean (SD) | 36 (13) | 36 (13) |
| Percentile Rank Median (25th-75th percentile) | 27 (14-41) | 27 (14-40) |
| Percentile Rank Mean (SD) | 28 (16) | 27 (16) |
| Funded | 13% (3950) | 17% (21,554) |
| Funded if discussed (N=86,349) | 25% | 31% |
Table 2: Comparison of applications submitted to six ICs that received a higher than average proportion of applications from African American and Black (AAB) PIs with applications submitted to all other ICs.
Consistent with Hoppe, 17 topics (out of 148), representing 40,307 R01 applications, accounted for 50% of the submissions from African American and Black (AAB) PIs. We refer to these topics as “AAB disproportionate” as these are topics to which AAB PIs disproportionately apply. ICs that received a higher proportion of applications in these 17 AAB disproportionate topics had lower empirical award rates (Figure 2).
Figure 2. Proportion of applications in selected AAB disproportionate topics relative to IC award rates. 17 topics, representing 40,307 R01 applications, account for 50% of the submissions from African American and Black PIs. R = -0.45, correlation between IC award rate and proportion of applications on these 17 AAB disproportionate topics.
Applications submitted on these AAB disproportionate topics had similar rates of discussion and, for those applications that were discussed, similar median and mean priority scores and percentile rankings; award rates were 12% lower (Table 3).
| Application Topic Characteristic or Outcome | AAB Disproportionate (N=40,307) | All Others (N=117,098) |
| PI AAB | 3% (1143) | 1% (1131) |
| Discussed | 54% (21,950) | 55% (64,399) |
| Priority Score Median (25th-75th percentile) | 37 (27-45) | 36 (26-45) |
| Priority Score Mean (SD) | 36 (13) | 36 (13) |
| Percentile Rank Median (25th-75th percentile) | 28 (15-42) | 27 (14-40) |
| Percentile Rank Mean (SD) | 28 (16) | 27 (16) |
| Funded | 15% (5847) | 17% (19,657) |
| Funded if discussed (N=86349) | 27% | 31% |
Table 3: Comparison of applications submitted on 17 AAB disproportionate topics with applications submitted on all other topics.
In Hoppe we constructed a probit regression model of funding success and found that after considering a number of confounders, topic choice accounted for 21% of the racial gap in funding success for applications that are discussed (Hoppe 2019). We constructed a model using the assigned IC instead of topic choice and found that similarly, assigned IC accounts for 23% of the gap.
These new analyses demonstrate that AAB PI’s sent more applications to IC’s with lower award rates (Table 1), that differential award rates (Figure 1), rather than decisions made by peer reviewers (Tables 2 and 3) as indicated in Hoppe, were critical drivers of differences in funding outcomes for applications linked to different topics, and that IC’s which received a greater proportion of applications in topics to which AAB PIs disproportionately apply had lower award rates (Figure 2).
I am grateful to my colleagues in the OER Division of Statistical Analysis and Reporting (DSAR) for their help with these analyses.
“On adjusting the funding disparity in NIH grant awards.” Drug Monkey Blog June 16, 2020.
This analysis goes right to the question of proposals that are “picked up” for funding by the IC, despite falling below the percentile funding line. If pick ups for AAB applicants (currently negligible) were the same rate as for other applicants, it would have an outsized impact on AAB funding rates and minimal impact on the funding rates of others.
Congratulations, once again, on your work. This shows how a careful scientific analysis of an issue can disclose causes and associations that aren’t obvious on at first glance. How the scientific community responds to this is still difficult, but it is reassuring to see similar impact scores between the two groups. I look forward to reading suggestions for addressing this! One obvious one, which would go against my self interest (I am NIDDK), is to balance funding based on applications across the NIH. Wonder what the positives and negatives of that would be.
Thank you for analyzing the data showing us clearly that racism is systemic in science as well. I am emeritus and trying to support younger women of color to apply for NIH grants.
Hi Dr. Lauer: Please tell me how I can obtain a copy of the research report.
Aside from the original publication in Science, there are no additional data or reports to provide. Please see this NIH Open Mike post for a discussion of the original paper: https://nexus.od.nih.gov/all/2019/10/10/delving-further-into-the-funding-gap-between-white-and-black-researchers/
Thank you for presenting these important data. But the conclusion of the post was underwhelming. Let me give it a shot:
These data suggest that racial funding disparities are not exclusively the fault of reviewers. But they do place the blame on Congress and NIH program officials.
First, applicants do not choose an IC. They suggest one, but it is up to NIH Program to find the right match. If the NIMHD and NICHD have such low paylines, then it means other ICs need to pick up some of those strong applications at the margin. The onus cannot be on applicants to efficiently allocate NIH resources.
Second, clearly there is more supply than demand for disciplines like minority health disparities. Is the problem on the supply side (too many PIs)? I can’t imagine so. Considering how much the US spends on health care, efforts to explore predictors, outcomes, and efficiencies would seem quite prudent. (This is not my field so apologies if I am not using the correct jargon). Congress should fund NIMHD and NICHD more and NIH staff should lead on lobbying for that.
FYI: federal employees are not allowed to lobby Congress, but this data can be used by anyone else to do so.
I am emeritus and trying to support younger women of color to apply for NIH grants.
Thanks for sharing the data which clearly shows us that discrimination is still structural in science.
Discrimination in the dictionary means selection among options. Choices need to be made. The fact that applications from African Americans or women or Muslims or blonds might compete in an inferior mode in competition does NOT mean that the discrimination is nefarious. I am amazed that the scientific community emblemized by these comments jumps on one among many hypotheses. Let us be fair. If the scientific community believes that special consideration should be given to AA or female applicants, then let us say so and state the reasons. If not, intellectual and scientific merit should remain the standard for discrimination.
For me, the most salient lines from the Hoppe et al (2019) paper were these: “We found that 2.4% of reviewers were AA/B scientists (table S10), which is very similar to the percentage of applicants who are AA/B (2.1%; Fig. 1). While not underrepresented relative to applicants, the absolute number of AA/B reviewers is still quite small, and it is conceivable that a more demographically diverse group of reviewers might have different opinions on the significance of some grant applications.” What in the world is NIH doing to correct this under-representation in AA/B reviewers? Also, I very much agree with “Grumpy” above.
The discrimination could never be a good thing and that’s why the women should get support and apply for NIH grants.
As a young minority researcher with rejected grants in the past, this confirms to me that the odds are and will remain stacked against me.
I have not followed this from the start. I remember some other report where NIH had other studies before on PI identity and success rate. Has NIH and other funding agencies tried to reduct PI identifying features on proposals for reviewing purpose?
IMO science should be color blind. picking a grant to be funded based on a persons ethnicity is not good for science; as long as the proposal furthers our understanding of a subject. is it possible that blacks (or Indians, or chinese, etc.) just dont write good proposals. no one really believes that the thought process of a reviewer follows “well its a good grant but the person is [insert ethnicity here]” ; give me a break.
i propose ALL proposals be stripped of the authors identity and ethnicity before review. reviewer would only review the science. that would solve this (and other) issues of bias
I don’t think it’s necessarily that a non-African-American reviewer says, “This applicant is an investigator from X racial group, and I don’t like research by those investigators, but rather, “This application focuses on police violence in African-American communities and its association with PTSD. Yes, they have preliminary data, but I have a hard time believing that this actually happens. I’ve never even heard of police killing people. In my neighborhood, the police are great.” ?
NIMHD has a lower payline than other institutes because they have a smaller budget. Applicants may disproportionately apply to NIMHD, but that is not always the case. I personally know of at least two instances where a Black PI requested a specific IC, but was reassigned to NIMHD. Yet NIMHD’s entire budget is smaller than the health disparities budget at NCI, NHLBI, etc…
Why would a topic be considered ‘disproportionate’ if an application is responsive to the FOA, has supporting preliminary data, and minor grantsmanship issues? The NIH encourages innovation and funds novel research. As stated in the comments already, perhaps the review panel should be more diverse and/or instructed to consider different topics for funding. The SRO should also be trained to inform the reviewers that the ICs have interest in neoteric topics. How does the OER plan to address the findings of the study?
This article is very informative, but lacks a conclusion. I appreciate the data and analyses, but I would like to know your recommendations to NIH ICs, program officials, and even Congress. Are there structural changes needed? If the science of Black scientists is just as strong as of others (which seems true from these data), why do they fail to get the grants despite similar scores? Please dig deeper. Do you have a call for action?
Hi Karen! They don’t provide a call for action, but Guy et al (in press, JAH) do:
“Now is the time for the NIH to address systemic racism by including at least 13% AA/B scientists as reviewers in every SRG, prioritizing topics of relevance to AA/B scientists, and committing to ongoing analysis of factors contributing to systemic racism in academic science. To do otherwise ensures the continuation of an intolerable status quo and the unacceptable disparities in research funding that adversely affect the health of our nation.
Completely agree. Yes topic choice is a thing. But is it the only thing????????
Having served on Study Sections for many years, my suspicion is that the problem lies in the fact that (a) the Reviewers are not interested in the problems that AAB scientists wish to study, such as health disparities endemic to their communities; (b) they don’t know and don’t have an interest in supporting the careers of the applicants, who are not part of their circle of colleagues.
These are very interesting and thought-provoking data. Curious to know if other award programs such as R15 or K awards follow the same trends?
The low rates of success at NIMHD are eye popping. This brings to mind issues of mentorship for African American or Black scientists. Early in my career, I was encouraged to leverage my “Blackness” by applying for grants focused on cancer health disparities. The same type of leveraging is encouraged for faculty from HBCUs, especially when partnering with majority or R1 institutions. I think there is an assumption that minority applicants have some intangible edge in applying for grants related to health disparities. However if the IC has modest funding, any real or perceived advantage would be eliminated anyway.
It would interesting to see the relative number of PhDs that are achieved in given fields (nursing, Child Development) that may also lend to this disproportionate level of application by African American scientists.
This is an important study. Thank you. In this analysis, more factors need to be considered. Scientific review is more than examining the significance of proposed topics. For example, PI’s previous scientific productivity is a key factor in a scientific review. If you normalize PI’s H-index by the number of years since undergraduate degree and control for this factor (interaction term), what do you see in the analysis? I think most reviewers do not consider PI’s race while assessing proposals but they do look at PI’s credentials and track record carefully. Thanks
is there any objective evidence that the ethnicity, gender or sexual orientation of the reviewer negatively influences the evaluation of a grant proposal? It is totally impractical to insist that every study section and review committee be exactly balanced by these factors. We are talking about science. There are alternate hypotheses to explain lower acceptance rates for proposals from certain subgroups. Let us be as objective and unemotional as we can be.
I have a very simple question: how do reviewers know the race of the submitter?
If submitters are trying to get an advantage via their race, and there is race-identifying information attached to/embedded in the proposal (which should be illegal/disqualifying), then they should not complain when it backfires when they try to game the system.
What happened to the awards based on merit?
And, don’t you dare call me a racist, because I’m definitely not.
So the issue is that is very easy in many cases, to decipher a persons ethnicity without any explicit statement to that effect. If I work at and HBCU (address bias affects most smaller schools), If I have certain name, if I trained at a certain college, or even from certain areas of the country.
Besides, think is more about implicit bias. Reviewers and program managers may not wake up in the morning to cause harm to Black scientists. But if at any point the knowledge of my background gives a slight disadvantage to the PI, we end up int his space.
Again, this article suggests that the subjects Black scientists study, despite being equally scored, don’t get funded.
The Director’s Transformative Award’s two (or more) stage review strategy should at least partially solve the problem. In the first stage, PI’s identity is not revealed until the review of scientific merit is done in the later stages. Unfortunately, the funding rate of that program is too low. At the end, the PI’s identity will most likely make a big difference. If other programs adopt the strategy, that would be great. Hope the strategy will serve the Transformative Award Program well. If there are problems in the pilot phase, hope NIH will not abandon the strategy. Instead, make a lot of efforts to solve the problems and optimize the strategy.
“These new analyses demonstrate that AAB PI’s sent more applications to IC’s with lower award rates (Table 1), that differential award rates (Figure 1), rather than decisions made by peer reviewers (Tables 2 and 3) as indicated in Hoppe, were critical drivers of differences in funding outcomes for applications linked to different topics, and that IC’s which received a greater proportion of applications in topics to which AAB PIs disproportionately apply had lower award rates (Figure 2).”
Hoppe was authored by the NIH Office of Portfolio Analysis. Was the analysis you present here conducted by the same group? If so, is there a plan to retract the Hoppe paper or issue a formal correction through Science Advances? Publishing a correction to the Hoppe analysis in a blog post does not seem appropriate given the importance of this issue and the widespread citation of Hoppe in this critical moment.