126 Comments
Update 5/13: The second graph and the table below are incorrect and have been updated in a new post.
Meetings have taken me out of the office and made me a bit negligent in my blogging. Let’s catch back up by discussing more about the distribution of NIH funding.
There are many urban legends about NIH funding, only some of which have some basis in fact. For example, we often hear that there are many successful PIs who continuously add to their bounty by piling on more and more NIH awards. Is this true?
Take a look at the graph below to see, on average, how many NIH research project grants a PI holds in any given year.
Average Number of Competing and Non Competing RPG Awards per Contact PI per Year
In fact, on average, PIs only have between one and two awards at a time. It differs among PIs at different institution types but only slightly. However, if we take a look at the investigators who receive the most funding (the top 20% of the total pool), we see they averaged 2.2 grants in fiscal year 2009; although, a significant portion have three grants ongoing at any given time.
It is important to point out that NIH makes funding decisions based on our vigorous review process and the alignment of proposed research with IC priorities. Thus the success of any applicant, and the number of grants she or he holds, is based on a competitive process that I believe these data well reflect. Having said that, these data point to the fact that NIH awards are generally distributed broadly among PIs and not concentrated among a few.
Updated 4/8/11
Here are the raw numbers used to create figure 2 above.
| Number of RPGs Held | Number of PIs (Top 20%) | |||
| FY 1986 | FY 1998 | FY 2004 | FY 2009 | |
| 1 | 1,203 | 1,010 | 975 | 1,168 |
| 2 | 1,832 | 2,345 | 2,725 | 2,868 |
| 3 | 580 | 1,138 | 1,841 | 1,637 |
| 4 | 176 | 407 | 735 | 636 |
| 5 | 43 | 123 | 265 | 193 |
| 6 | 10 | 26 | 91 | 67 |
| 7 | 2 | 13 | 24 | 14 |
| 8 | 1 | 5 | 14 | 9 |
| 9 | 0 | 1 | 1 | 2 |
| 10+ | 1 | 0 | 2 | 0 |
| Total | 3,848 | 5,068 | 6,673 | 6,594 |
Can I just not do math, or is the Y axis of the top 20% graph labeled as %/100?
It is perfectly possible that I cannot do math. But I am trying to figure out how, for example, 0.15, 0.45, 0.25 etc. add up to 100%…
Thanks for the question Arlenna. We updated the Y axis. Hope that makes more sense.
Yay, thanks! I wasn’t just being disingenuously sarcastic, I really am arithmetically challenged sometimes so that revision is very helpful, haha. 🙂
Looking at the mean data is too simplistic an approach. How many awards are represented by the third, fourth, and fifth grant awards? The impact of a single individual having 3, 4, or 5 grant awards would best be appreciated by knowing how many total grants this represents. Restricting the #of awards to 2 per individual would undoubtedly allow hundreds of additional labs to maintain funding. I know grant awards are based on the best science . . . but there is a measurable value to having a diversity of ideas and approaches in each field. This situation where the rich get richer is forcing some of us out of research.
The April 8 posting is very helpful and gets at the question above, and I agree that what NIH policy-makers appear reluctant to do is to open the can of worms of exploring the following issue:
NIH “frees up” 700 awards if one models all 4-5-6-7-8-9-RPG PIs as dropping to only 3 RPGs.
That would prevent the loss of [x] programs that were being productive on one RPG and got out-competed. So, what diversity of research avenues and ‘different / not-yet-in-favor’ ideas, mechanisms, molecules, cells etc is being lost from those [x – probably approaches 500, probably not 700] awards going to less [skillful; opportunistic; connected; big-reputation] applicants whose awards nonetheless were at least rated better than 20th or 25th percentile but worse than “pay line”? And how much marginal increment of real field-moving, game-changing findings is provided by providing the more [skillful; opportunistic; connected; big-reputation] with their 4th, 5th, etc RPG? The latter point, unfortunately, is hard enough to determine that it is understandable that some might wish to avoid the fire-fight.
And of course, the scary thing is that the data don’t even count support to well-funded labs from industry and NGOs (ACS, LLS, JDRF, Nat MS Soc, etc)!
So, what about productivity? For instance, those researchers with 7-8 major grants in a given year (few as they may be), are they really more productive than 4 laboratories with a couple of R01s together?
I understand that productivity is subjective, but you should have data for publications, etc.
But how do you define productivity? Are we talking papers? If so what types of papers do we count (cut offs for IF used in productivity metrics)? Its a bit of a slippery slope. And who knows maybe labs with 2 major awards are more productive than two labs each with a single award?
And, GR, perhaps the single R01 of the young guns are skewing the numbers because those are the hard charging vigorous types that are on their way to being a multi-grant investigator.
Perhaps when you look at mid career, the productivity relationship changes because the single grant labs are those that are just kind of…sleepy. Or “comfortable” might be another way to put it…
Fine. All I am saying is let’s get the numbers in all possible ways and decide consequently.
If holding 9 major grants in a single year is good for the system, then let’s promote it.
Or perhaps the data will indicate that the 92 researchers holding more than 6 major grants in 2009 could have been capped at 3 R01s and 300 additional R01 could have been issued for other researchers. Rather than cutting 17-20% of the award of a new investigator.
I do not know. I do not have the data but, based on what I see around, my hypothesis is that holding 9 grants is a waste of public money.
Amen. In an environment where often the payline is in the single digits, it is extremely difficult to compete against established labs which have been collecting data for decades. The is no argument for some labs to have 5+ R01 while many talented new investigators are voluntarily choosing to leave academia or being forced out following denial of tenure due to the inability to secure an R01 in their initial years. It is now certainly an old boys/girls network, where the PIs of established labs watch out for their friends. These ‘innovative new investigator’ type programs which tout support of new PIs is disingenuous…thousands of grants received with a success rate again in the single digits. 30 years of training to land a tenure track position (and beating tremendous odds in this age to get one), and NIH could care less to protect the future of the scientists that who were suppose to lead the next generation of discoveries. Here we come 1980s again…another (larger) lost generation.
Amen! I strongly agree with this statement: “92 researchers holding more than 6 major grants in 2009 could have been capped at 3 R01s and 300 additional R01 could have been issued for other researchers. Rather than cutting 17-20% of the award of a new investigator.” This all comes from the same budget. Where is hope for new scientists whose careers depend on getting their first grant?
Or well connected is my interpretation.
Jeremy Berg of NIGMS had a post a while back TRYING to do an analysis of this question of papers per grant as a function of number of grants or total $$$ in budget; not a great analysis, but a start, and he was reasonable about noting some of the caveats.
But I’d have to disagree some with some of what “Broken System” says. I’ve not been struck, on the whole, that the new investigators’ applications going through study section are noticeably different or more innovative, and if anything the set quotas of new investigator awards are posing a real problematic challenge [less the “split payline”, more the mandate of equal success rates].
What is the definition of “contact p.i”? Does this mean every unique applicant? Or every unique recipient? Both numbers would be interesting but each provides different information. For instance, if the denominator equals applicants, than the apparent drop in awards/investigator would reflect the decline in the per cent of applicants funded. On the other hand, if the denominator equals investigators funded, than the recent drop in awards/investigator would mean that there has been a broader distribution of awards. Without a better definition of terms it is difficult to understand the meaning of the graph provided.
For multiple-PI grants, one PI is designated the “contact PI.” This means that for a grant with two or more assigned investigators, the grant is counted only once—for the contact PI—and is not included in the calculation for the other investigators.
Counting multi-PI grants only once skews how many grants a single PI has, doesn’t it? If anyone to look at the labs that have multi-million budgets, she almost won’t see R01 but multi-PI grants and contracts.
And counting should be done by $$$, not the number of grants. Think, for example, what PI can do with a small $200K R01 or even a smaller R21 (which are included in the graphs!) vs. $2M contract which involves a lot of help and managing from NIH.
According to the NIH Databook, in 2009 NIH funded 35,600 RPG. [See http://report.nih.gov/NIHDatabook/Charts/Default.aspx?showm=Y&chartId=207&catId=2%5D
The number of grants does not, of course, equal number of investigators, but lets round it down to 30,000 investigators. The top 20% of investigators would then number 6000.
In 2009, one quarter of them had 3 awards = 1500 investigators; 10% had 4 awards = 600 investigators, 3% had 5 awards = 180 investigators, and about 1% had 6 awards =60. That totals 2340 investigators with 3 or more RPGs. Anything wrong with my calculations?
Phoebe, your numbers are pretty close, but we posted the actual numbers above.
One will have to have one new award within 1.5 or less years in order to keep three or more awards. This means that the top 20% will “occupy” the system.
One bit of technical arcana that has some impact, especially now, is that the numbers may include RPGs that in NIH system are viewed as “active” but in fact are on no-cost extension; that can distort numbers of awards a bit to the higher side (which many on study sections seem not to grasp).
What is the n for the Top 20% and the total number of funded contact PIs (a useful statistic for interpeting the first graph as well)? And why not just use n (vs % of top 20%) on the y axis for the second figure? The numbers cannot be that large, and the impact of PIs with multiple awards on the larger extramural community would be easier to assess.
As others have noted, the issue is whether having 3 R01s produces 3X the productivity. Assessment of productivity is a contentious issue, and no one measure will satisfy everybody. But if you look at publication number or impact, be sure that some sort of correction is applied for the common practice of acknowledging multiple R01s on each paper that comes out of the lab. This inflates the perception of productivity for multi-R01 labs when renewal time comes around.
@CD0, Beaker: see, e.g.,
https://loop.nigms.nih.gov/index.php/2010/09/27/measuring-the-scientific-output-and-impact-of-nigms-grants/
and
https://loop.nigms.nih.gov/index.php/2010/11/22/another-look-at-measuring-the-scientific-output-and-impact-of-nigms-grants/
But of course, since “scientific output” is ill-defined, so too is its relationship to funding. Thanks Rock Talk for the informative post.
Yes Paul those are the best data out there and I am grateful that NIGMS put them out there. It shows that 3 R01s do NOT produce 3 times the output. And I don’t think those data were corrected for the multiple-attributions practice I mention above. If so, then the more-is-not-better argument holds even more. The alternative of 3 investigators receiving one R01 each would seem to give NIH and taxpayers more bang for their buck.
Dear Dr. Rockey,
With all due respect, I do not agree that this is a compelling argument that the NIH system is not biased to reward those who already have more money than they need.
Analysis of the numbers reveals that for every 1000 grants awarded in FY2009 to the “top 20 percent of researchers” (as defined by the NIH system), 583 of those grants went to researchers who already had at least 1 grant. Thus, consistent with what is common knowledge among P.I.’s, those with NIH money are more likely to procure more NIH funding than the “bottom 80 percent of researchers” who probably receive 0.3 – 1.3 NIH awards per year if one averages over time.
As far as the bell curve of numbers, plugging in the NIH-provided numbers from this blog reveals that if 72 of the “top 20 percent” of P.I.s had a single NIH award, then 183 P.I.s would have two NIH awards, and 162 P.I.s would have 3 or more NIH awards. In these times, when many P.I.s have no awards and are in the process of laying off highly trained personnel, are we expected to applaud the fact that 16% (162 per 1000) of NIH grants to the top 20 percent of researchers are going to P.I.s who already have 2 NIH awards?
The reality is that it does not cost that much money to perform high-quality research. However, it does require critical analysis of how nature works in reality and more willingness to cry “B.S.” when the top 20 percent of NIH-funded investigators are spinning new yarns in hope of getting yet more NIH funding. This is not a hypothetical complaint, but is quite real. If the big labs in my field of study performed more rigorous science, then I would not have to contend with the fact that 90% of NIH-funded, peer-reviewed research is either (1) trivial in nature or (2) just plain wrong.
What the NIH peer-review process fails to take into consideration is that the forward progress of science has never (for the past 400 years) been dependent on putting vast sums of money in the hands of a few people. This is a perfect recipe for large labs to synthesize “sexy, new flavor-of-the-day stories” to justify their large budgets. There is simply no historical precedent for the concept of “big science” (and big budgets) being a good thing. More to the point, if one studies the famous quotes of the great scientists, then it is quite clear that the real advances in science almost always issue from the humble beginnings of a junior investigator who has the audacity to point out the flaws in current thought and models in their field of study. The current NIH system empowers those with funding to crush any dissention from junior investigators, and hence the differential awarding of NIH grants to the “haves” and “have-nots” is quite counterproductive to everything history teaches us about how science really advances.
At the root of it all, the forward progress of science is contingent upon maintaining a diversity of opinions such that we may provide each other with the checks and balances essential to differentiate sexy, new stories from new and important facts that advance our understanding of how the natural world really operates.
Politics is unquestionably important to the scientific process, but at the end of the day, agencies such as the NIH have to fund some people who are actually in the business of being critical, honest, and restricting their “discoveries” to findings that are real and will stand the test of time. In my opinion, the NIH is doing an abysmal job of differentiating real science from story telling, and all of the recent changes to the grant application process only further compound the difficulty of distinguishing between “stories” and “science.”
Hopefully Asia, Australia, and Europe will keep science moving forward while the NIH continues to justify to American scientists why it is OK to focus on how well the NIH is funding the top 20 percent of researchers while treating the “bottom 80 percent of researchers” as expendable refuse whose contribution to science is equally trivial in nature.
Sincerely yours,
Bill Halford
Bill, I agree with almost entirely. Living very near the Wright Bros. legacy, I think of the contribution of them and other individuals often. If you’ve heard their story, they ‘got off the ground’ with about $300 (1898-1903) whereas S. P. Langley was awarded $25,000 by the US government. They requested government assistance, but gave up on that until they had a flying airplane to actually sell. Their story (as nicely recounted in ‘The Bishop’s Boys’) and experience parallels what happens in biomedical research in many ways. It appears to be human nature acting in the field of innovation and discovery in certain repetitive patterns. However, if you were to talk about going to the moon or sequencing the human genome–that is a different thing. That is a task that involves many people and innovation, but in itself, is not innovation. So we all work together, but it really is a sum of individual parts that makes it all happen. Creating efficiency and productivity is the art of the managers. I think that is where we need much improvement. It makes little sense to give more grants to an individual unless there is a comparable payback as well as no harm to the overall group effort.
Paul,
I think you have hit the nail on the head and provided some fine examples that bring focus to the discussion. In my opinion, the NIH’s first and highest priority should be to provide relatively modest sums of moneys to many young “Wright Brothers” as many of these bold new ideas will not pan out, but it only takes one successful new idea to lay the groundwork for a system of global air transportation. Unfortunately, in the current NIH scheme, I think an application from the biomedical equivalent of the Wright Brothers would be viewed as far too fringy. I can hear reviewers talking among themselves now, “What kind of environment do these people have?” “Is this idea really innovative? I know 5 groups that tried similar ideas over the last 30 years, and they all failed.” “These guys are not in the MIT engineering department? How can these two hillbillies from North Carolina possibly compete with the best and the brightest of our elite institutions.” The fact that the Wright Brothers succeeded on their own does indeed confirm that only small amounts of money are sufficient to give life and creedence to bold new ideas when the investigators have a large amount of passion for their work.
I think history teaches that useful discoveries often issue from the most unlikely places, and the NIH process dismisses this historical lesson. You would think that these panels of 20 people who review grants are omnipotent and have a track record of identifying the best and the brightest science. How many tens of millions have NIH study sections invested in siRNA and miRNA research in the past decade? How’s that working out so far? All I hear at meetings are concession speeches from well-funded labs saying that “their microRNAs research projects revealed that microRNA have only modest effects and induce only 20 – 30% reductions in expression of the target gene, which is followed by speculation about the potential importance of fine-tuning in gene regulation.” I recently published a gene expression paper that showed that herpes simplex virus’s major transcriptional regulator, ICP4, represses an important viral regulatory gene by >3000% (>30-fold reduction). My grant application to request funding to further explore the implications of this phenomenon on the establishment of latent HSV infections was triaged. Perhaps if the repressor molecule was something more fashionable (i.e., a micro RNA instead of a boring regulatory protein), then the outcome would have been different.
Paul, you bring up the excellent point that some endeavors in science must by definition be group efforts such as putting a man on the moon or sequencing the human genome. The fundamental question here is, “What is the NIH’s role in funding big versus little science?” There is no room to doubt that the sequencing of the human genome or space flight were important to the evolution of science. However, it is important to remember the the initial DISCOVERIES that brought to light the original, critical idea (hypothesis) had relatively humble origins. Only through the wide-spread acceptance of these ideas over a period of decades was it later possible for a large group of scientists to work together to APPLY THE IDEA ON A LARGE-SCALE such that a man could be placed on the moon or the human genome could be sequenced.
At least one essential component of our ability to put a man on the moon was Einstein’s theory of relativity that he presented in ~1905. In today’s NIH funding environment, I get the sense that the NIH is so worried about applications such as “putting a man on the moon” that they lose sight of the fact that you must first truly understand nature before you may seek to apply (control) that understanding to achieve a practical goal. The NIH’s priority should be on funding DISCOVERY of knowledge, and once real discoveries are made (not stories), then practical application will logically follow without much of a push from the NIH.
Case in point. I have been working on herpes simplex virus 2 (HSV-2) vaccines for 5 years and the biology of herpes simplex viruses for 18 years. During that time, the NIH has spent hundreds of millions of dollars on the idea that injection of a human being with one or a few proteins derived from the HSV-2 virus will yield an effective HSV-2 vaccine, and will break the transmission cycle of genital herpes (affects ~50 million people worldwide). Many permutations of a similar “subunit vaccine” approach has been repeatedly tried with HIV (AIDS) vaccines over the past 20 years, and have met with larger numbers of even more spectacular failures.
This is a perfect example of the NIH “taking research from the bench to the bedside” but not having the wherewithal within their bureaucracy to recognize that there is not a shred of hard, scientific evidence that injection of a person with one or a few viral proteins (derived from HSV-2 or HIV) should be sufficient to prevent such an aggressive and persistent virus from invading a recipient who received a lame “vaccine” that was about as potent as an injection of water.
A billion dollars later (a lot more money has been put into AIDS vaccines), I see no willingness from within the NIH to admit that attempts to APPLY THE SUBUNIT VACCINE DISCOVERY TO CURE HERPES AND AIDS may be futile, because the NIH did not adequately vette the underlying assumption that a subunit HSV-2 or HIV vaccine should actually work as well as most of our successful viral vaccines (based on live-attenuated variants of the natural virus). If anyone cares, I provide a link that expands upon this point further in the context of HSV-2 vaccines (http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0017748).
The point is not to plug my own work, but rather to highlight a dichotomy in logic that underscores why the NIH’s current mantra of funding big science and “moving discoveries from the bench to the bedside” is flawed.
The highest funding priority should be to dedicate small sums of monies to many investigators to create the fodder for turning ideas into proto-discoveries that are supported by data and published studies.
The second highest funding priority should be to assure that there are enough experts in a field of study that we can meaningfully vette each others ideas and arrive at a consensus about which discoveries at the bench may really translate to the bedside. In the current funding environment, who dares to criticize the work of a senior investigator (even if it is complete rubbish) because this is political suicide and will only further shrink your slim odds of ever receiving another dollar from the NIH. When scientists are afraid to openly and honestly speak their minds, then science is for all intents and purposes dead……a vacuum of discourse allows inaccurate hypotheses and bad ideas to linger for years longer then they should, and science move backwards as result (i.e., trainees fill their heads reading series of manuscripts based upon a flawed premise).
The NIH should be in the business of ensuring that the flow of new, important, and ACCURATE ideas about how the natural world works may continue to emanate from those young and foolish enough to think differently than their colleagues who have had an additional 10 – 50 years in a field of study to become comfortable with the current status of knowledge. By definition, the new investigator brings a fresh pair of eyes and are most likely to catch the incongruities in logic. Good ideas that are mirrored in the natural world will stand the test of time, and will eventually be converted into useful applications regardless of whether or not the NIH sets asides funds for APPLIED RESEARCH.. In contrast, if no strategy exists to recruit and retain brilliant young minds in science, then the flow of new and accurate (USEFUL) discoveries will slow and there is no private investment firm that will intercede, as good science makes for lousy business investments….the timetable from discovery to application to profitability is too unpredictable.
I don’t know where all of the answers lie to the NIH’s financial woes, but I believe that some areas that warrant consideration are (1) smaller budgets for everyone, (2) a higher priority on creating incentives for young people (the future of science) to stay in the game, and (3) a rational discussion about the appropriate retirement age for individuals in academic science. Just because an individual chooses to work in the lab through their 70s, 80s, and 90s, I am not certain that large amounts of NIH funding to support large numbers of graduate students and postdocs are required. I by no means wish to discourage the idea of the “Professor Emeritus” as such individuals have been invaluable to my training and mentoring over the years. However, it concerns me to see so many senior scientists well beyond the retirement age with multiple NIH grants while promising junior investigators leave science in search of a career where they may support their families.
I do not hear nearly enough recognition from within the NIH that they, like all large companies, should have a “human resources plan” in place that ensures that the money they invest in training new, specialized workers is logical and is balanced by the long-term needs of the institute in terms of having a sufficient amount of money to support the fraction of prospective workers who are likely to make valuable contributions to the institute’s mission. I seriously doubt that Microsoft accepts a class of 120 incoming recruits into which it invests 8 – 10 years of money (graduate and postdoctoral stipends) with the idea that it will accept 10 of those people into potentially permanent positions (faculty positions), but will withhold making a final decision on who the “top 20 percent” are that will be permanently supported by the institute in Years 20 and beyond. Hey NIH, there are lots of human resources consulting firms. Perhaps, it’s time you hired one.
– Bill Halford
Bill and others,
I strongly recommend that you read ‘The Bishop’s Boys’. I guarantee that you will see the funding situation in a different light. The Wright brothers story of discovery innovation is more interesting than you would ever expect especially since the personal accounts are included. It tells of the many difficulties successful pioneers actually face in addition to funding. A rewriting and obfuscation of history is just one thing that can and does happen. For example, the Wright Bros. lived and worked in Dayton, Ohio (not hillbillies from NC or Ohio for that matter) which in the late 1890’s and early 1900’s was at the leading edge of technology in the U.S. with the highest per capita of patents. Aviation was developed in Dayton, while the plane and first flight occurred at Kitty Hawk, NC. The early winter ocean winds were suitable there, nothing else was. Wilbur and Orville only stayed there for periods of weeks over a couple of years, and it was fairly miserable other than the high winds and sand dunes. This eventually led to the fight between Ohio and NC over their discovery of powered flight. NC is now credited with the first flight, and Ohio is credited with the development of aviation.
There appear to be reasons for the concentration of funding in a few hands or institutes, but they do not appear to have been a part of early American principles, or a distinguishing characteristic that made America great. James Fenimore Cooper stated this nicely in his book “Notions of the Americans” written in the 1820s when he compared the growing U.S. dominance of maritime commerce to that of other less prosperous nations. We are perhaps acting more like France did at that time.
“The talents, and enterprise of France, have hitherto been mainly pressed into the employment of the government. In whatever they have arrived at perfection, they have been concentrated in order to consolidate the power of the state, instead of being dispersed to effect that vast accumulation of individual prosperity which constitutes the real wealth of nations”.
He goes on to say something that aptly applies to government (NIH) management of resources:
“The secret of all enterprise and energy exists in the principal of individuality (eg. in this context small R01s). Wealth (eg. mega-R01 or multiple R01s) does not more infallibly beget wealth (eg. publications, discoveries), than the right to the exercise of our faculties begets the desire to use them. The slave is every where indolent, vicious, and abject; the freeman (eg. small R01 investigator), active, moral, and bold. It would seem that is the best and safest, and consequently, the wisest government, which is content rather to protect than direct the national prosperity (eg. mega-funded centers), since the latter system never fails to impede the efforts of that individuality which makes men industrious and enterprising”.
The Wright Bros. were clearly acting as individuals when they developed aviation. I think the list of individuals goes on…. Best wishes for further success with your work on vaccines to important viruses. The approach you describe reminds me of Jenner’s original cow pox vaccine.
Dear Paul,
Thanks for the perspective and for the book title….The Bishop’s Boys. I will give it a read. I think such discussions remind me that discovery has never been particularly easy for anyone (including the Wright Bros.), and that it is a mistake on my part to place too much blame on the NIH funding situation. Getting funding to move excellent science forward takes a lot more time than we would all like, and that’s just the reality of where we are. The plus side is that this gives me more time to hone the questions I wish to ask.
Will read more about Orville and Wilber, and will try to remember that I am just one of hundreds (thousands?) of scientists who need to adjust my expectations for how fast science can move forward when financial resources are rate-limiting.
With best regards,
Bill
Bill,
Your comment seems to argue that giving PIs multiple grants is a bad thing. I disagree. Some Ph.D.s are not getting grants at all, and may wind up without research support. Is that due to system unfairness or lack of aptitude and a flair for discovery? Some folks, regardless of academic achievement, are just not going to be leaders in their field. Although supporting the research cadre is necessary, it is not sufficient to generate breakthrough knowledge. How many medical researchers does the United States need, and is it NIH’s responsibility to support them all?
All of the multiple grant recipients in my portfolio support other researchers as Co-I’s and RAs as well as students. Those PIs get more grants because a) they are highly efficient and effective scientists, and b) they are more practiced in writing applications that communicate clearly to reviewers. Neither of these two criteria can be influenced by NIH policy. I support the ESI program because it gives talent a chance to blossom. It would be unfortunate if NIH were to begin denying support of its leaders merely to support unproven people who want to be researchers for a living without regard to their talents. NIH’s system is competitive, albeit imperfect. It is up to POs to help NIH get the best science for the taxpayer’s dollar.
Hi Tom,
I think we have a fundamental difference in opinion in how we define effectiveness in science. Based on your post, you define an effective scientist as “Those PIs [who] get more grants because a) they are highly efficient and effective scientists, and b) they are more practiced in writing applications that communicate clearly to reviewers.”
. I don’t think Albert Einstein ever received a single NIH grant, but rumor has it that he figured a couple of useful things out that we are still using today. Although Einstein does not satisfy your definition of an effective scientist, I would characterize him as OK because he peered into nature, and reported back in plain and accessible terms a new and useful relationship between energy and matter that was not previously appreciated.
In my book, the best scientists are the ones who teach us (reveal) the most about how nature operates…..period….end of story. The NIH would do well to learn the difference between a sales pitch (a grant) and a new and real idea about how nature operates. As hard as it is to believe, the two things do not always coincide. Moreover, many of the multiple NIH-grant-funded PIs that I have known over my career have a bad habit of requesting NIH funds for a story that is “on the brink of cracking wide open” and 5 years later they have abandoned the original story in lieu of a new story for which they are now seeking new monies. Looking in your portfolio, such investigators would appear to be very successful as they maintain a steady stream of funding into their laboratories at all times. However, Tom, how often do you or other NIH Program Officers go back and double-check that the funds being awarded to such PIs were used to actually gain an important (real) piece of scientific knowledge?
If your definition of being an effective scientist is to extract as much money out of the NIH as possible, then I will be the first to admit that I have not been a very effective scientist in my career. However, I think that dollar-for-dollar, my lab has done about as well as any other lab in the world in advancing knowledge of a key regulator of herpes simplex virus’s latency-replication balance and explaining the implications that this knowledge has for a new type of HSV vaccine that could be used to end genital herpes. However, as I read your post, I cannot help but feel that any new requests I make for NIH funding will be further eroding the NIH’s ability to support the true leaders in my field (i.e., those who are already well funded) as I am just an unproven PI who wants to be a researcher for a living without regard to the limitations of my talents.
Tom, I accept the fact that there is a very large disconnect between the size of the NIH budget and the number of requests for NIH funding; that is an irrevocable fact. However, I think that there are a quite few unfunded PIs in the U.S. scientific work force whose contributions over their careers deserve a lot more respect than what you offer in your post.
– Bill H.
I’m sorry Tom, but you are fooling yourself. Funding rates are low…period. It is impossible to tell the difference between a grant in the 10% percentile and one in the 20% percentile, and yet likely only the former will get funded. Are you saying that the latter applicant does not have the aptitude for good, even breakthrough scientific discovery? The argument of ‘the cream always rises to the top’ is a convenient way for the NIH to be able to sleep at night.
Amen. Thank you for spelling this out
I’m not totally convinced that this has busted the myth for me–for one, I don’t know how relevant it is to lump R01s in with R21s and R03s (at the smaller end) and big P and U grants (at the higher end). How do all these different types of grants correlate, and how do the actual DOLLARS per investigator map out to different grant types?
By promoting/allowing empire-building and discouraging young investigators from doing good science [by not supporting them] – never so evident before – the current NIH ways of funding science is a disservice to this country and the global health. Innovation is, and always was, higher in smaller/younger labs – and the “sweet” numbers given in the chart do not change the fact that frustration and confusion among the scientific community in this country reached its historic peaks. Put it simple: young scientists are leaving US science, either finding alternative careers, or establishing their productive labs elsewhere. Fig. 2 explains the reason.
Well, I am one of them. I opted for a decent pay outside the ever devaluating dollar and a better life/work balance. Ciao.
I am one of them as well. With limited support from my supervisor, I never knew how to pursue funding or even that was something I should be doing. I was a great student and scientist but the well-funded PI’s always needed me to do their work to push their research agenda and had no quarter for mentoring a rising junior star. So, I started my academic career struggling to get a grant with no mentor, no data, and no support but lots of pressure. I am thinking of leaving science altogether to get a good paying job with a work life balance and at least see some return on my 7 year investment of graduate education. By the way that same PI who is well regarded in his field said I had potential to become a strong investigator, so I know I was doing a great job, but no one cultivated that potential. Its sad to see us younger than 30 leaving and putting science and academia as a whole without talent when the current scientists will be on their way out. Where does this leave us?
Do remember that some “four R01 labs” are not lasting ” empires”. They may be on the peak of the pendulum. Calibrate your perceptions of “all those multi grant labs” with this and the data from OER may look more intuitive.
Also remember that direct costs vary between R01s…
Bit of a huge gap between universities (excluding medical schools) and everything else.
‘Having said that, these data point to the fact that NIH awards are generally distributed broadly among PIs and not concentrated among a few.’
HUH?
Looking at your own raw numbers I note that the number of 1 grant PIs has stayed roughly the same comparing 1986 with 2009. BUT Between 86 and 09 the number of :
2 grant PIs has increased 1.5x
3 grant PIs has increased 2.8x
4 grant PIs has increased 3.6x
5 grant PIs has increased 4.4x
6 grant PIs has increased 6.7x
7 grant PIs has increased 7x
and
9 grant PIs has increased 9x.
The doubling of the number of grants awarded is not then, because we are awarding more grants to more PIs, but instead because we are awarding more grants to individual PIs.
The raw numbers you show are in direct contradiction with your post.
Doubledoc, the shift after 86 from 1-2 to 3-4 awards for these top funded investigators really begs the question about relative grant *size* doesn’t it though? What with BRDPI and the introduction of modular grants isn’t it possible that the single big grant (in relative numbers) has simply evolved into two smaller (again, relative) grants?
I was going to make the same general comment.
If we accept that funding has been getting tighter from 1996 to 2009, how is it possible that 3/4/5/6 grant holding investigators has generally trended up?
This is increasingly difficult considering that the number of applicants has generally been rising during that same time frame.
Anyways, I’ve decided this whole blog process is pretty silly. It’s not a way to actually exchange ideas between NIH and investigators, but appears to be a way for the NIH to simply defend what it is doing.
It was a nice idea Dr. Rockey, but it’s clear that the format is not meant for exchange but rather for cover.
Would you consider that it is more difficult for a “regular” applicant to get grants precisely because the grants tend to accumulate in the same hands no matter what the competition is?
Also, we are convincingly told (and there are research article published on this topic) that the current review system cannot reliably tell which applications in the top tier (top 25% or even more) are better than the others, so the grant award becomes a lottery. Yet someone is able to win this lottery eight or even nine times! Truly inspiring!
I noticed that immediately. The divergence of the premise with the raw # trends for RPGs held clearly indicates a trend toward a higher multiple of grants. Whether that trend restricted those without an RPG is less clear.
While NIH funding doubled from 1999 to 2003, the number of funded grants increased by only 31.7%, suggesting that a rapid trend toward more expensive grants. The rate of increasing expense logically occurred at over twice the rate of adding new grants. However, if internal NIH funding increased at a more rapid rate than extramural funding (a trend since the 1980s), this could also be a mechanism by which the # of RPGs lagged behind.
Could you post the full numbers – not just top 20% — on this issue? What I would really like to see, for each year, for each # of grants, is (1) # of investigators with the given # of grants and (2) average $$/investigator for those investigators. If you would publish those numbers — or point us to where they can be found — that provides a basis for really evaluating the distribution of NIH resources. (If you post this, could you email me, as I probably won’t keep checking the comments thread. Thank you.)
I just did a quick calculation for the numbers in your table. For FY 2009, among the top 20% of researchers in funding, about 60% (actually 59%) of grants are the 2nd or higher grant of an investigator, and about 25% (actually 24.47%) are the 3rd or higher grant of an investigator.
To say it another way: this group included 6,594 investigators but 15,914 grants. If grants had been limited to 2 per investigator, then .2447*15,914=3894 additional grants would have been available, i.e. almost 4000 unfunded investigators could have been funded. (And if limited to 1 per investigator, 9300 unfunded investigators could be funded.) Given that a total of 5*6594=32980 investigators are funded at present, it’s clear just from this top 20% data that there’s room to significantly increase the number of funded investigators.
Given the number of excellent investigators that are finding themselves without funding, I think the question of the distribution of NIH grants needs a more careful and less congratulatory analysis.
Right on, Ken, although like I commented above, since Ps, Us and R21s/R03s are in this pile, too, then it’s harder to elucidate the real meaning of this.
Amen to that!
People say this over and over and over, but without the slightest bit of evidence.
The problem with the system is not the number of grants given to large investigators. The problem is stability for that first grant. The problem is that because it is no longer (*) true that good work on an R01 implies you’ll get it renewed, one-grant labs are unstable. Losing that one grant means the destruction of a lab due to the loss of expert technicians, students, and postdocs. (One can’t let a technician go for two years while you try to get your grant renewed.) This means that people are forced to try and fight for multiple grants. Many many multiple-grant faculty I know would happily trade those multiple grants for one stable, predictably-funded R01.
The key here is that reducing the number of multiple grant-labs is only going to change that stability indirectly, but we are still faced with that uncertainty. What we need is a system where if you’ve done good science over the last cycle, then you can KNOW that you’ve done a good-enough job over the last 5 years and that you’re going to be renewed.
* if it ever was. It certainly has not been true in my scientific lifetime of grant applying.
I agree that stability is important, but the award process needs to stay competitive. NIH is not (and in my taxpayer opinion should never become) a welfare system for scientists.
The problem is — and has been for the past several years — too many applicants for the same pot of money. There are exactly two ways to fix the ‘problem’ and raise funding rates: 1) More money, and/or 2) Fewer applicants.
What can you do?
If you want more money, lobby congress and convince them to shift financial priorities. Or convince the public to do so. NIH can only give out money that congress gives them.
Alternatively, help reduce the number of applicants. How? Labs need to stop overtraining workers for jobs that don’t exist, and institutions need to stop hiring investigators (especially those on soft money). Trouble is, there is no incentive in the system for anyone to do either. Big labs produce more data and publish more, making them more competitive. Institutions risk little and potentially gain much (thanks to high overhead) when they hire professional grant application writers — er, I meant ‘soft money scientists’.
Two other problems with these data:
1) Let’s say you have some labs that are struggling to get a single grant, such that they have, on average, 0.5 NIH grants at any given time. If this year is one that they have the grant, they will be included in these data – if it’s an off year they aren’t included. In order to understand the effect of giving multiple grants to single PI’s, it’s essential to count those labs that would otherwise have been funded were there a grants/PI cap.
2) You need to include some measure of variance in your mean grants/PI figure for this data to be at all informative!
An excellent point from jojo. I would be very interested in the number of (established) investigators who go for a year or more without funding and then manage to secure subsequent grants. Also some sort of tracking of latency to first award from a (New) investigator’s first submission to NIH. Oh and perhaps for those PIs who never secure a grant, track how many proposals have been submitted.
Are people being washed out because they aren’t really trying (fewer than 5 attempts, say) or because their grants are really poor (20 attempts? 30? Draw your own line on that one…) or because they can’t remain viable for the time it takes or…?
20 or 30 attempts? You get mobbed to death by your Department chair after 5 unsuccessful trials. See that with some former colleagues.
Also: Does nobody see the hypocrisy of this system? We claim to want to combat disease. But spend at least 70-80% of our time writing grants, generating preliminary data for grants or reviewing grants. The NIH has created a system where most people run in circles like mice in a cage.
Do we ever think of patients anymore? How many diseases could be treated better if scientifically productive people just had some stable funding? If I was a patient I would be mad as hell.
Given the number of excellent investigators that are finding themselves without funding, I think the question of the distribution of NIH grants needs a more careful and less congratulatory analysis.
By definition, investigators that find themselves without funding are not “excellent” investigators, or they’d be able to get fundable scores on their applications. The NIH has the best peer review system in the world, as Dr. Rockey is happy to repeatedly point out, so thankfully the best science is the science that gets funded.
It is simply not true that “investigators that find themselves without funding are not “excellent” investigators” because there is noise in the system. Even if it were true that “NIH has the best peer review system in the world” (*), there would be noise in the system (as there must) and luck plays a big role. If there is no difference in real quality among the top 25% of applications, then which grant ends up in the fundable 10% is due to luck, not quality. (**)
* I realize you were probably being facetious here, but I’m not. I actually think NIH has one of the best peer review systems in the world. Having served on my share of study sections and seen the internals of the review process first-hand, I don’t think the problem is in the review process, but rather is in the more fundamental mechanisms of the grant application process itself. Rather than assessing small changes to the application system (you can only have N grants), we need to consider much larger changes (for example, renewal of one grant is easy).
** Actually, I think 25% is way too low. In fact, I think that there is little real difference in the quality of the science (let alone the predictable “impact” of that science) that would come out of the top 50%.
“By definition, investigators that find themselves without funding are not “excellent” investigators, or they’d be able to get fundable scores on their applications.”
That’s supposed to be sarcasm, right? You can’t actually believe that. Are you really trying to define the problem away?
Rock Talk team
Given the data from NIGMS (Jeremy’s blog) that the lab productivity plateaus after 750K in direct costs (= 3 R01s) , do you think NIH will come up with hard policy to restrict PIs from having more than 3 R01s ?
As Sally mentioned in an earlier post, the Advisory Committee to the NIH Director is looking at ways to address the needs of the biomedical workforce. We expect they will examine this question and many others.
Note that this “productivity” is limited to numbers of pubs and does not account for the steep increase in cost per pub for GlamourMag articles. I am decidedly not a fan of the GlamourMagification of science but it is disingenuous to ignore this when discussing Berg’s dataset.
Picking up from drdrA:
The temporal trend in grant distribution appears to be nonlinear. From 1986 through 2005 or so, concentration (that is, the median number of grants per PI and indeed the entire curve) moved steadily upward–among the ‘top 20%’, the one-RPG PI went from more than 30% of all funded PI’s at the beginning of the time series to less than 15% of all grantees as of 2004–just before the trend plateaued. This seems very unlikely to have been due to a movement toward taking a larger number of relatively smaller grants, since a RePORT data search tells me that the mean grant size for all RPGs–and individually for R01s, R03s, R21’s, and U01’s, which is most of the pot–increased quite dramatically during this period, even taking the R&D CPI into acccunt. (Someone with access to the microdata could determine definitively the average size per grant over time for PI’s with each number of grants.)
But this trend turned around five or six years ago. Not that NIH fully reversed the two-decade movement toward greater concentration, but clearly the sign of the first differential changed. The critical questions would seem to be: (a) why/how did this extended trend reverse? and (b) has this reversal continued since FY09–including ARRA funding?
The data shows that *output* plateaus after 750K. Productivity is output divided by input. Productivity is already declining at the very left hand of the graph, below $200K. That graph is strong evidence that funds per investigator should be capped below $200K/yr.
Actually, there IS a skew suggesting that funding begets more funding. What is not shown on the graphs or added into the statistics are the vast majority of PIs (applicants) that have ZERO grants. If you include these people, there would be a huge peak in the histogram at zero, then an anomalous DIP in the histogram at 1 R01, and another peak at 2 R01s. In other words, there are disproportionately few people with a single R01.
I’m not making any judgments regarding NIH policy. It makes perfect sense that having some NIH money already would bolster one’s chances of getting more NIH money. After all — it’s pretty tough to get competitive preliminary data with no funding.
I just wanted to point out that the graphs and statistics leave out a LOT of people. Don’t forget that a lab not funded by NIH is still a lab! Therefore, the average number of grants per PI is somewhere around 0.2. Which makes 2 R01s look like a lot in comparison.
Again – I’m not making judgments. The system has to be competitive. But let’s discuss ALL the data.
I want to raise the issue that medical schools, even public ones, are increasingly forcing faculty to obtain as much salary as possible from grants and thereby providing incentive to obtain more grants than one needs for the research. In this way, the number of faculty and grant dollars will increase along with the ranking of the medical school. The problem is that there is no reason for medical schools to pay their faculty anything when salary can be transferred to the NIH grants. To promote increased salaries on grants, some (many?) medical schools have introduced incentive plans (the more salary you bring in, the higher you are paid) and base salary cuts for those who are not bringing in sufficient salary. This sound like the scientific equivalent of leverage and sets up a rewards system grants rather than successful research. I know of many distinguished, senior PIs who would be satisfied with 1 or 2 grants to support their research program; however, their institutions increasingly discourage this.
Finally, back to the productivity issue. Some labs with many grants cite many of their grants in each publication, thus making their productivity on that grant look better than it is. I was on a study section and saw that one applicant had 27 papers on a $220,000 grant with 1.75FTE personnel. I raised the issue that this is impossible and was immediately silenced by staff saying that the issue should not be raised and that I was disqualifying myself. Who is going to raise this issue?
Simple, why NIH allows a very few labs (no matter how few, one is too many when ten labs could benefit instead of one) to be exuberantly “rich” and keep getting richer to the point of “needing” to spend their money at the end of the year not to “loose” it? Specially disgusting is most of the time those labs have a pathetic per capita productivity rate (of course this is usually hidden by the analysis of productivity that does not consider the wealth of the lab or the amount of non-last, non-first author publications that inflate the productivity).
…and you know what? Those labs will keep getting the money and wasting a lot of it…that is unacceptable.
Let us see – – looking at your numbers for 2009 in the last figure, there were 2868 people with 2 grants each (which means NIH could have funded an additional 2868 people under a one-grant/PI policy); 1637 with three grants each (i.e., NIH could have funded 3274 more under a one-grant/PI policy); 636 with four grants (i.e., NIH could have funded 1908 more people under the one-grant/PI policy); 193 with 5 grants each (i.e., 772 more investigators could have been funded). This means, under a one-grant-per-PI policy, NIH could have funded 8,822 more investigators ! This is good enough reason to implement a one-grant/PI policy – – under the current harsh climate with the NIH funding, NIH needs to figure out better ways to distribute its dollars to more people, instead of keep rewarding multiple grant holders again and again. The NSF already has an unofficial policy along the same lines (i.e., if you have a NSF grant already, the NSF is generally very reluctant to fund a second one, unofficially ofcourse. They prefer to distribute their dollars, as far as I know). Hope this helps.
Sometimes I think that the Rock Talk team hates the current funding system just as much as the rest of us and puts the data to provoke an emotional response from the scientific community and, hopefully, a push for change.
As others have noted, the data disproves the notion that the grants are NOT hoarded by the bigger labs: almost 60% of the grants are held by the labs that have 3 or more of them. The data is incomplete, as the bigger labs tend to attract additional fellowship money, as well as the grants awarded to senior postdocs (Research Assistant Professors, etc.) as nominal PIs. So, the real picture is even worse.
There are many possible explanations of why grant hoarding happens, from outright corruption, to the advantage of using a single result to report progress on multiple grants, to the added safety of surviving when one of the grants is accidentally lost, etc. Neither explanation would characterize the existing system as either fair or cost-efficient. In fact, is this distribution inconsistent with an outlandish scenario that a small number of cliques took charge of generously funding their own members?
Some of the data is plain ridiculous: a sizable number of grants are held by people who hold seven, eight or, even, nine of them. Since PIs, usually, are faculty with many administrative, academic and other obligations, it is unimaginable that a person in such a position could intellectually lead nine completely independent projects. In fact, just preparing these applications (presumably, not every single one of them is funded), as well as the corresponding yearly reports, safety protocols, etc., looks like a full-time job. Possible explanations that i. the nominal PIs take credit (and money) due those who really intellectually lead the work; or ii. the projects are highly redundant and people shamelessly double-dip, are not very flattering to the NIH funding system.
I find this discussion so very interesting because so many fundamental points are being missed, and so many myths are being propagated without critical examination. First is the concept that creativity and innovation belong to the junior investigators. That is just completely wrong. Many of the most creative investigators I have known in my career were in their 60s, 70s, or even 80s. Moreover, these people have a wealth of experience that certainly must count for something. We should toss these people out in favor of unproved junior investigators? Yes, we need to preserve a flow of investigators of all ages, but that does not mean to me that older investigators should be put out to pasture just to fund junior investigators.
Second, no one has criticized the system of “hypothesis driven research” in which depth is favored over breadth, and surety in nailing down a small, incremental step is rewarded over bold thinking toward a breakthrough. I have sat on many study sections in my career, and most of the innovative ideas I have seen (which incidentally generally came from more senior investigators rather than junior ones) were triaged or were nitpicked to death in favor of small, incremental steps that were solidly designed to leave no doubt. As a result, 1/3 of the human genome is completely unannotated, half has no more than one associated paper, often an omnibus paper mentioning thousands of genes. The proteins we are investigating are those that were discovered 20 years ago. Is it any wonder that cancer survival has not changed in the last 40 years?
I also saw the myth that we must fully understand the basic science in order to make breakthroughs. This is simply not true. We now have the possibility of doing brute-force top-down science to find breakthroughs if we are willing to fund the effort. Is this any less sure than spending decades learning how every molecule in a complex interacts with every other molecule? Perhaps we should focus more on filling in the blanks rather than learning more about the same old proteins.
Finally, nothing has been said of a system that depends upon the cheap labor of graduate students who will have little prospects of ever finding a job. We keep churning out PhDs not because we need them but because we need the cheap labor.
Bruce Alberts wrote an editorial in “Science” last year on the unsustainability of our system. He was correct. Indirect costs are too high and encourage expansion on soft money. Remember when investigator salaries were a small fraction of research grants? How many more grants could we have if investigator salaries were capped at 50%? Institutions should have an investment in their people. Not to do so enables the exploiting system we have now.
The very fact that the x axis on the grants-per-investigator histogram goes up to 9, is itself confirmation that this “myth” is real! As detailed in the responses, and similar to the post regarding cessation of A2s, these types of post rarely have the intended effect on researchers they are intended to pacify.
Many of the comments assume that reviewers are able to discern among the 4-8 grants they see at a time and thereby award scores with some reliability with respect to scientific quality. Perhaps, but the stochastic (“bad reviewing”) aspects and skewing (“rich get richer”) effects are magnified by the current tight funding lines that require priority scores of 23 or better. In turn, score compression (triage lines at 3.0) is another factor further degrading the process.
I would argue that we need to deal with the current crisis, which has existential significance for far too many colleagues, many in mid-career and recently considered promising newcomers or leaders in their field. Some hard adjustments need to be made to accommodate the budget crunch and keep these otherwise productive scientists from being shut down. If an when we can go back to funding 30% of proposals, I think we can return to arguing about the best way to support innovation, and if some highly innovative scientists deserve their nth grant, so be it…
Steve
If you examine NIH funding over the past 10 years, you will find that the top 65 ranked U’s are receiving about the same relative funding regardless of “efficacy output”. It is as if the funding is designed to maintain a “research capability”for the country in order to permit a trial and error approach to research(“so many monkeys at typewriters will eventually produce a novel”). The creativeness required to move the science forward is part of a very select and extremely small group. It appears that the funding should go to creativity for the breakthroughs first then followed by applied research funding in the application. If the peer review group could recognize this, there would be no problem, but they do not have the creativity to do so. Predictability has never been part of breakthroughs, but the unexpected and unanticipated has.
No amount of funding can force the creativity necessary to solve the nations health problems. So we retain the well established and politically acceptable trial-and-error funding method to research. This approach cannot stand the test of time and eventually the amount of money spent on this kind of research with little to show in health breakthroughs and results will become an obvious waste of time and resources with over due consequences that stop it!!
There real problem that exists today is that the funding for science and medicine in this country is woefully inadequate at a time of greatest discovery. NIH and congress should be made aware of that fact. Our future growth in technology is running out, many educators are losing their jobs due to lack of funding, and our children and education systems is at risk in the fallout.
That is the overarching problem, and it needs to be fixed. However, this problem should not be confused with funding the best science. To make funding a numbers game is to further diminish quality and amplifies the first problem. The plain fact is that all PIs are not equal, and funding incremental science just to fulfill a quota is wrong. It is reasonable to maintain a democratic system in which each grant is scored on merit.
If we decide to deny funding to those who have been judged meritorious, then it make it necessary for us to choose from those who were judged as less meritorious, who should be funded. That process is more unfair than the first.
Cynthia McMurray, PhD
Wow. What a lot of talk over something that is somewhat easy to divine.
1. Some people are good at writing grant applications. They get more grants. They are not necessarily better scientists. To me it constitutes greed. And they are certainly not more productive, it is long established that small labs are more productive.
2. We have too many good scientists vying for too little money. This industry (which is what it has become) should limit new investigators, not promote them, like other industries do when money runs short. New scientists getting incentives? How about keeping productive established scientists from losing their labs, from having one or two typographical errors in their applications? As one of the latter, and after serving on 25-30 study sections, I resent the incentives.
The point is not the number of grants per investigator, although 8 grants to a single individual mean that some resesarchers are getting funded under the wings and name of someone else. These researchers may not be better than the ones actually applying for funding on their own.
A more important point is: how do you plan on a long term career (given that science takes time, as some of you mentioned)?
Let’s calculate the probability of success, assuming that all applicants are equally smart and have the same chance of success.
Probability of having one grant funded: 0.1 or 10%
Probability of having two grant funded: 0.1*0.1 = 0.01 or 1%
Probability of having three grants funded: zero
An ESI would need three or more grants to survive (given that the first ones may be short R03-R21). Should an ESI plan a career in science and discovery based on zero chance?
The likelihood is that all ideas, good or bad, will die before they will ever be tested. We need to be realistic: none of us will survive the zero percent fate in the long run.
How well would NIH do if every year all the NIH employees were fired due to lack of funding? If it is unconceivable to think of a 1-year turnover in the NIH system. That is why the number of grants per PI is rising. NIH know that only a few will survive and they are betting more and more on just those few who are most likely to survive the zero chance.
The issue is not really “numbers of grants” , it is costs awarded per grant. I hold two modular R01s whose combined costs after the recent post award budget slashing are less than most of the recent “single” R01 applications I have reviewed for NIH.
In the 10 years I have been reviewing grants I have watched the direct costs awarded to investigators skyrocket with no real justification except that PI salaries in soft money institutions are very high (often more than equivalent levels in industry) and that many investigators are paying lab staff more than assistant professors make at my institution. But we are told that we as reviewers can not cut these budgets since that is the cost for those PIs.
One can do great science on a modular budget (and yes my work is very mouse intensive) if one is frugal. Also, I agree with others that productivity needs to be assessed based on the budget of the prior grant period and there needs to be more accounting of the productivity double dipping that folks do. Papers per dollar is the true measure here, not “number of R01s”
As someone with a modular budget (not even the max’ 250k allowed), and a lot of mouse work, I agree 100%. At competing renewal time, I plan to emphasize this fact in my progress report… productivity per dollar is an excellent way to express this. Thanks for the idea!
The raw data supplied is useful, but can we get that for the whole 100% of PIs rather than just the top 20%?
Or better yet, can we get percentiles for how many dollars of annual funding PIs have? That is, for what values of X1, X2, …, X99 is it true that the number of PIs with annual funding of $X1 or less is 1% of all PIs, the number of PIs with annual funding of $X2 or less is 2% of all PIs, etc.?
Agree with many of the comments being posted especially those talking about the difficulties in young faculty making it to the “next level” and the “stability” of established labs.
One thing I haven’t seen posted yet is the push from institutes to acquire grants for the benefits they bring in indirect costs (F&A). From what I see many institutes have become “addicted to the indirects” and constantly push investigators, junior and senior, to keep getting more and more grants when the reality is that both sets of investigators would probably be better served to focus on spending the money they received on the project they described so, being competitive for the renewal 5 years later instead of constantly looking for new angles with which to try and get that second grant.
I know the Department Chairs will cite the costs of running departments, equipment etc, but it seems to me that pressure is being applied from the administration above. The flip side of that argument is that I don’t see many of my colleagues in some of the other, non-biomedical research departments (both liberal arts and physical sciences) being constantly pressured to achieve multi-year, multi-million dollar grants. However, they teach no fewer classes than we do, generally have less graduate students than we do, rarely have post-doctoral fellows, and as far as I know they have electricity, water, gas, heat, etc in their buildings, have admin support, use the same university systems of accounting, finance, admin, etc. I have no doubt this is an over simplistic view, but I do feel the “commercialization” of academics in the past 20 years has pushed biological and medical science research to the forefront of money generation for many colleges and institutes, and this is a significant factor in driving the pressures on investigators to “push the boundaries” for multiple grants to be funded. Similarly, the pressure to maintain funding, leads investigators (particularly junior investigators) to push for a second grant to make sure they don’t gap in funding. The reality is focusing on the 1st award is more likely to guarantee success in renewal but, the problem is that becomes a huge risk to put all the “eggs in one basket”.
Alas, the NIH had the opportunity to address this a couple of years ago with a major overhaul of the funding mechanisms. Despite what seemed an age of town hall meetings and opinion gathering, surveys etc, it seems to me that nothing really changed except the length of the grants got cut in half because reviewers were (rightly) overwhelmed by the number of applications they were being asked to review. This was a real opportunity to address and change the funding levels for junior mid-level and senior investigators to provide stability while retaining competitiveness. Alas, despite a lot of “shuffling the deck”, I believe the reality is that not much actually changed; The junior investigators still have to live on a knife edge of success and failure, mid-level faculty find it increasingly difficult to compete with well established senior personnel and “super labs”, and the senior investigators are constantly being harassed that being PI on two multi-million dollar grants is not sufficient to guarantee their positions.
Finally, maybe it’s just me getting old, but since finishing my PhD (15 years ago) I seem to see a lot more administrators and “coordinators” at different levels that didn’t seem to be there when I started? It also seems that we are letting faculty close labs while those teaching are seeing class sizes increase to unmanageable levels. Strangely, I haven’t seen any reduction in the number of administrator positions despite the downturn in funding rate over the last decade.
I fear that without significant change, we are going to see in science what is happening in medicine; more people interested in entering the world of managing labs/departments/hospitals (2 year MBA in health care management anyone?) as opposed to doing the hands on bench work/idea generation/patient care (or a 5-6+ year PhD/MD & residency). Don’t forget, many of the senior level investigators are of the baby boomer generation……
Ah yes… “admin creep” as I like to call it. The situation wherein an institution’s secretarial, administrative, and facilities support become a hindrance to the performance of research. It is very hard to fire such individuals, as they are often fully benefitted employees with contracts and unions. Much easier to simply squeeze the researchers for more grants.
All sorts of incidental expenses, covered by our indirects, “support the local economy” but siphon money away from research….
– Why does it often take me 2 weeks to get a PO# issued for common purchases of reagents and supplies.
– Why, if I want something from one supplier, but another supplier is “preferred”, do I have to fill out additional paperwork and it takes up to a month to get a PO# issued?
– How often does a department really need a new photocopier?
– Does every secratary need a new computer every couple of years?
– Do all grants have to really be submitted through an opaque system of software that is only understood by a few individuals, and signed-off by numerous officials?
– How many deans and vice deans do we really need?
– When my post-doc’ engages in some teaching, is it absolutely necessary to fill out a mountain of paperwork so that a nominal tiny percentage of his salary can be paid from a non-research account?
– How come when we blow a circuit breaker in my lab, it has to be reset by a “facilities operative” who has a higher hourly wage than me?
– Why do the parking officers on campus drive around in new hybrid SUVs?
– Who paid for the new flower bed out front?
– Who voted to go with a particular software package for tracking lab chemicals, then changed it 2 years later, requiring all labs to re-enter all their chemicals into the new software?
– Why, when I order a piece of equipment for research use, does a PO# get issued with all kinds of riders about it being certified for human clinical use, causing the supplier to refuse to ship the item?
– Why, as a basic researcher, who never encounters patients, do I have to spend 2 hours screaming at a mandatory on-line exam for all employees, with questions about how to handle unruly patients, and how to interpret various overhead emergency codes?
– Why, when the animal use committee switched to an online system, could my existing protocols not be automatically transferred? Instead, I had to re-enter them all manually into a buggy java/web-based program.
These are just some examples of the things we have to put up with at a medium sized university medical center. For want of a better phrase – this s*** is bleeding the research process dry. As researchers, we have to fight “admin creep”, and the inevitable call for “more grants, more grants” that comes with it.
It is not clear to me how you reach the conclusion that “these data point to the fact that NIH awards are generally distributed broadly among PIs and not concentrated among a few”.
I just ran a quick calculation on the data provided herein and my calculations indicate that 39% of PIs were awarded 3 or more R01 grants in FY 2009. This implies that NIH awards are not distributed broadly but rather concentrated among a small elite group of scientists. Given that so many talented scientists out there are struggling to land even one R01, these NIHtistics are absolutely shocking.
Besides, there is little or no correlation between the number of grants a PI holds and her level of scientific productivity. In fact, more R01 a PI holds, the lesser science he contributes to: It is the PIs with one or two R01s that actually spend time in their labs doing science and mentoring future generation of scientists, while a majority of those with three or more R01s have probably forgotten the direction to their labs and spend more time pursuing politics than science.
In my opinion, NIH should altogether do away with story telling and lies (grants) and simply award scientists on the basis of their scientific productivity (papers) in a more objective manner.
Science needs diversity. NIH is doing exactly opposite of that. God Save America!
Just some thoughts!
“Stories” and questionable adminsitration, evaluation, and communication of science could be the barrier of our times –
I am a mid-career scientist in my late 40s who has been enthusiastic throughout my career about the science ongoing in my own lab, my institution and in those around the nation. I’m also of the opinion that science, throughout history, always has been a difficult task and only those most dedicated scientists were able to overcome barriers specific to their own historical timeframe.
Today, I see a real barrier that may be as dangerous as any in past times, which is “story telling” as one blogger put it earlier. Such practices are often rewarded at the expense of true discovery. This, in my opinion, has to do with publication practices of “high-impact” journals, political networking (i.e. versus scientific networks), and short grant applications (12 pg limits) that all rob scientists of venues to discuss data or fully explain experiments completed or ideas that needs to be explored. Incentives (i.e. grants/pubs) are often placed on popularity and “percieved impact” and not on seemingly “dull” and “incremental discoveries”; however, I contend that such real data is indeed our real backbone as scientists and should be collected and valued at all costs. No one can argue that “breakthroughs and innovation” and “impact science” are not important, but the ideas that lead to such discoveries are built upon well-established data sets that are vented and confirmed by many labs (big and small!!). The focus on “stories”, which often gets grants funded and “high-impact” pubs are too often shown to be just that (i.e. a fictional stories creatively written from incomplete data sets). Such story telling, while inviting, gives none of us the real credibility we as scientists need, and will ultimately give society less enthusiasm to fund our endeavors in these fiscally difficult times.
A lot of good comments posted here but do NIH administers really read, listen, and consider these comments and suggestions?
Hi Lin, we do actually read all of the comments posted here. And while we might like to, unfortunately, we/Sally can’t respond to all of the comments and questions. When we can contribute to the conversation, we do. But we are listening, so we hope you will continue to comment and question.
nice to know!
(if nothing else, I need to howl at the moon anywya)
The comments and analyses of Bill Halford are spot on, in my opinion (although I know nothing about vaccines!). Perhaps a little competition between viewpoints would be useful? Could NIH publicize a few examples of PIs with 5-7 RO1s whom they consider well worth the investment? And perhaps Bill could point to a few examples of a particularly ‘successful’ lifetime single RO1 recipient? Such specifics might be extremely informative.
Some excellent NIH data here posted by Blue Ridge Institute for Medical Research for all you data hounds:
http://www.brimr.org/NIH_Awards/2010/NIH_Awards_2010.htm
Particularly interesting is this file:
http://www.brimr.org/NIH_Awards/2010/AllPIs_2010.xls
, which is a ranked list of principal investigators by funds awarded.
Ignoring issues of apples and oranges, running sums on these data suggests that 20% of NIH funding is going to the top ~2% of investigators and that 50% of NIH funding is going to the top ~15% , at about the $1M per PI line.
well, I ought to be reviewing grant applications [ack!] and can’t plow through the data, but without yet having pored through the analysis, think one has to be pretty careful and be fair (i.e., NOT ignore issues of apples & oranges, and yes, compare apples to apples, not apples (Newton) to sputniks (20th century).
It is pretty vital that NIH fund various clinical trials and other such entities, and also to recognize that although we need lots of basic yeast, worm, etc research, we do also need progressively more expensive types of research as well: mouse-intensive AND human studies research + clinical trials (going up the cost/award size scale. I don’t do the human side research but for sure there are certain ways in which PI’s with great teams (nurses, data coordinators, medical center ties) are going to be better able to do these things and thereby appear to haul down BIG shares of NIH $$$.
Lin, it is not the NIH administrators who sink my grant applications, but it is (so I surmise) my turf-guarding R01-funded scientific peers. Until the self-preservation and nepotism of NIH grant reviewing is eliminated (and it never will be), new investigators will always have a tough go at receiving funding.
This is all about perspective. I am 2 years out from my K23 and just received a JIT for my 2nd R01. I am a PhD clinical outcomes investigator at a major academic institution. I am all soft money and expected to cover 80% of my salary–but will probably get talked to if I drop below 90%. Because I do human subjects research in the community my grants are staff heavy=expensive. I can’t budget for more than 20-25% of my effort and stay under the 500K cap.
As my first R01 enters year 4, I have to submit another grant because it could take 2 years to get funded. Because of the research I do, few foundations are interested or can afford the project. Industry is willing to give money, but unfortunately through their marketing department– which is prohibited by my institution– and with too many strings attached for my comfort. So that leaves NIH. My lab is very skilled at writing grants so sometimes we get “lucky” and get funded on a first submission. That means for 1 year I could have 3 grants– one in it’s last year, one mid-way, and one starting.
Does that make me greedy? Do 3 grants overlaping for a single year mean I’m not doing innovative research? If an RFA is released that perfectly suits my research (as was the case with my 2nd R01) should I not apply because I already have a grant?
In just 3 years I went from being told that I needed to add senior investigators to my grant to one reviewer negatively critiquing me on my 2nd R01 for having too much funding. Senior investigators at my institution are complaining that all the recent changes (e.g., removing the A2 submission) puts them at a disadvantage. My feeling is we all compete for the same money with the same rules. The system isn’t perfect and never will be. The pie is just too small so we’re all making excuses for why we aren’t getting a bigger piece.
These comments are arguing many technicalities related to multiple grants. No one is observing the 800 pound gorilla in the room- that of NIH awards providing funding of investigator salaries. This consumes enormous amounts of available funding, and allows the elite universities to virtually set up an NIH branch on their campus. To my thinking, NIH payment of salaries should be ONLY for government employees working in the intramural system, or for time-limited training awards. If a university wants to promote a certain type of research, it should pony up for the faculty salary. This might actually force the university/medical center to do some serious strategic planning, as opposed to “go get MORE, MORE, MORE” of any type of grant you can get. This numbers game is skewing the stats, and as a result, sending many outstanding research investigators out of the research arena.
I have sat on many study sections in many institutes. The peer-review process is basically a good ol’ boy system of connected researchers with the “correct” pedigrees, from elite universities, assuring that the elite are taken care of, while the rest of the researchers are “damned with faint praise.” I have watched unstated quota systems in which topical areas representing new and innovative fields are restricted, while the same old same old from “established” investigators are perpetuated.
The issue of numbers of grants is also a game. For a SERIOUS RO1 or any other major project, there is no way an investigator can do it justice with less than 25%+ effort. I watch researchers manipulating their percent efforts to numbers as ridiculous as 5% or 2%, so that they can continue with their chairmanship, their clinical activities, their administrative responsibilities, and all of the other percentages, so that they can continue to pile up the PI stats. But in actuality, their postdocs or their “research professors” (soft money researchers) carry out the work while the big name professor receives the PI accolades.
The best system for actual productivity would be: (1) no longer pay ANY salaries for faculty other than time-limited programmatic training awards; (2) restrict overhead to a fixed percent, rather than rewarding inefficiency with greater indirect costs; (3) limit the number of awards any one individual can hold at one time to 2; and (4) limit the direct cost budget to a modest amount. For those who want a bigger budget, have them compete for a limited pool of funds with all other investigators who feel they need mega-funding. This might actually slow down the “numbers game” that universities play in order to climb up the scale of total federal dollars. It is a game similar to “my endowment is bigger than your endowment.” Let’s put some sanity back in the system. Lest you think this is disgruntled musings of an unsuccessful investigators, I have been highly successful at all levels. And I have come to realize that 6 grants do not result in 6 times the productivity of 1 grant.
David,
Thank you for your comments. I believe that your comments (and those of others) on indirect costs, PI salary, and percent effort nicely cut to the heart of the financial matter.
I would add two fundamental questions that the NIH should be asking from a financial / human resources point of view that would further coincide with / reinforce your proposal that n=2 federal grants is adequate to keep science moving forward in most research laboratories.
1. On average, what is the optimal size of a research laboratory?, and
2. On average, how many trainees should each PI in the system be training at any given point in time?
The answer to these questions dictates whether (1) the NIH blithely continues to operate a pyramid scheme in which each junior scientist is encouraged to go forth and pursue the “lofty goal” of obtaining 3 to 6 federal grants while supporting 10 – 20 trainees in their labs (who in 10 years will be further straining the NIH budget) or (2) the NIH acknowledges that the post-World War II economic boom has been over for several decades, and now it is time to think through the details of how one achieves a “sustainable academic research industry.” The basic principles of human resources dictate that the rate of trainees entering science should match the rate at which senior investigators are retiring, and I think the principle variable that one controls to achieve that goal is lab size (or federal funding to support a lab of X individuals per lab). If one talks with their colleagues in the U.K., Europe, and Australia, relatively small research laboratories have been the norm in many countries for decades.
In my experience, I would suggest that 4 – 8 total personnel per laboratory is a manageable (including PIs, techs, dishwashers, etc) number of individuals where the trainees have the full benefit of the PI’s time and attention, and there are enough trainees in the lab that 1 or 2 unsuccessful students and/or postdocs will not kill the momentum of the lab. While I have observed many labs that are considerably larger, I have not generally seen a corresponding increase in scientific productivity because (as someone pointed out above) the P.I. becomes a slave to their grant-writing obligations to keep enough money flowing to retain all of their lab staff. I have seen many such large labs in my career, and have noted that much of the innovation in big labs often comes from students and trainees…..because they actually have time to think about the experiments. What’s the point of having an experienced P.I. whose intellectual contribution is limited to asking students, “Did the data prove our original hypothesis?” If I ran my lab this way, I would miss 90% of the most interesting (accidental) observations that we have made.
With financial times being what they are, I would suggest that 3 – 5 total personnel is adequate to keep science moving forward in any lab with a motivated / engaged PI. The reality of science is that the kernels of new ideas need time to grow, and thus it is important to not only give a P.I. adequate financial resources to support personnel who may generate new data but it is also important to allow the P.I. enough time to actually think through the ramifications of the data being generated in the lab. If the NIH gave PIs more time to think through their data sets (insteading of pushing for more pubs), the average publication would contain less fiction and a lot more fact.
Unlike all of my long-winded posts, I believe that David has nicely summarized (1) the essence of the financial problem and (2) proposed some viable / implementable possibilities. So, the real question is, “Does the leadership of the NIH have the motivation and/or courage to address the real issues that explain why biomedical science in the U.S. is in its current state of crisis?”
If the NIH were to seriously consider changing their policies, I would suggest that these changes should focus less on the bureaucratic window-dressing of grant forms, application lengths, etc. and focus on the substance of what scientists need to do their jobs….a relatively modest amount of funding that may reasonably be sustained over a period of 15 – 30 years (without having to write 4 new grant applications per year to sustain that funding). The ratio of trainees to PIs is a critical issue, and is highly inter-related to the pragmatic suggestions of financial reform suggested by David. Change is possible, but the NIH leadership will need to grow a pair and start talking about the hard choices that need to be made if the NIH is to regain the respect of a majority of American scientists.
As with most scientists, I have no problem dedicating my life to the pursuit of knowledge. However, if the NIH wants the respect of American scientists, then you should listen to people like David who have been in the trenches and have been telling you for years what the problems are with the current NIH system, and how they may be fixed.
– Bill Halford
Just one thing to remember: Most trainees do not continue in academic science and “further strain the system”. I have seven Ph.D. students in my lab right now and only one aspires to my career path as an NIH funded PI. The rest want to either work in industry or teach at a small liberal arts college. All of the Ph.D.s I have trained over the years have done well in their careers, however, none of them even wanted to pursue an academic research path. There are plenty of opportunities out there for Ph.Ds and the US economy needs them. However, we should not expect all of our trainees to be “like us”. As pointed out above, the system can not support an exponential increase in young NIH investigators, but lets be careful about “birth control” of the new Ph.D. pool, industry does need all of these folks we are training to keep biomedical research moving. Unemployment among biomedically trained Ph.D.s is between 1/5 and 1/10 that of the general population.
David,
Thank you for some wonderful insights. Te saludo — I salute you!
You raised TWO beautiful points that ought to be part of a fair and just NIH system:
(1) NIH should have a requirement of no less than 25% effort for a PI on an R01 grant — only a liar and a con artist would get away with less than 25% effort. NIH is fully aware of this but, as we all know, NIH is run by these con artists and story tellers!
(2) NIH should allow 0% salary for extramural PIs — under the current system, an R01 basically pays the PI salary with little or no money left to do any serious science. Besides, the fact that NIH pays PI salary, the money that could go toward PI salary in extramural institutions ends up in the pockets of administrators such as deans.
Or at the very least lower the ~$199,000 salary cap to a more modest level. Does anyone really need to make that much money?
Two questions about this data:
1) Can people who have multiple NIH grants hold still more grants from other agencies, such as NSF and Dept of Education?
2) Where do private companies that are not primarily “research institutes” appear in graph 1? The labels are “Research Institutes,” “Higher Education,” “Schools” and “Hospitals.”
Yes, the investigators could have also held grants from other federal agencies or private funding sources, but those grants would not have been reflected in our analysis. For-profit companies were not included in graph 1.
Some of the arguments for limiting the number of grants per PI end up looking like “the NIH should change its policy so that more grants like mine are funded.” This is not very persuasive. If your grant is funded, someone else’s will not be and they will have an equally good argument about why their grant should rather be funded.
There are better reasons to have some type of (hard or soft) cap on the dollars awarded to a single PI. This has to do with the concept of “best science.” The thought is that you want to fund the best science, regardless of who is doing it. If one person can come up with many great ideas, why should they be punished in favor of funding a worse grant?
While this is fine in principle, in reality the boundary between “best science” and the rest is very fuzzy. Anyone with any experience on study sections will tell you that you can separate the best grants from the rest till about 2o to 25%. Within that top group, it is impossible to tell who is in 5%, 10%, or 15%. Whether your grants get 10% (funded) or 15% (unfunded) is crap shoot. It is a matter of luck, literally. You hear this over and over, it is not something dreamed up by some disgruntled individuals. So first, we should stop pretending that we have a clear way of separating the best science from not-so-best science.
Once you acknowledge that there is no objective way to tell apart top grants, it is clear that it is essentially a tie at the top. In this case it would make sense to fund people who have no funding over those who have substantial funding. Recognize that you would still be funding the best science, not second rate science at all.
There are several ways to implement a scheme that favors unfunded investigators with excellent grants. One is to put a hard limit on dollars per PI, e.g., no more than 1 million per year in direct costs per PI. Another way is to use a more graded, soft cap. For example, add 1 percentile point to your score for every 100K in direct costs you have form the NIH. Someone with one 200K R01 will get 2% added to their score for a second R01. Someone with 1M in grants will will have 10% added. They can get yet another grant only if their score is in the top 1 to 2%.
A lot of the real-world systems have a built in rich-get-richer dynamics. It is much easier for wealthy individuals or countries to get richer. People with high IQ will provide their kids with more stimulating environments, leading to high IQ. Corruption leads a spiral of corruption. Sports leagues tend to have dynasties unless there are corrective measures in place. That’s why the most successful league, the NFL, has the Superbowl champions pick last in the draft. It is not because they want to see second-rate football. It is because increasing balance and competition promotes better football.
Actually, I’m not convinced that even the “top 25%” number people always toss out (as in “anon”‘s post) is valid; for sure, the top 50% of applications are terrific and these days – especially with CSR-mandated quotas of “not discussing” (streamlining; “triaging”) – even a lot of applications not discussed would have a good chance of making a bigger contribution than some things that hit all the right buttons, are beautifully presented, PI accumulated and skillfully marshals vast trove of preliminary data (not easy in the 13-page R01 era).
My pet peeve is that since people recognize the lack of resolving power within the range that is forced by combination of shrinking real-$$ NIH extramural budget & immense numbers of applications, real actions are not being planned on prioritizing awards based on lower indirect cost rates, or total award size (when adjusted reasonably to compare apples to apples within ICs’ portfolios {not a human studies R01 to a yeast biochemistry R01}.
One thing not clear from the helpful data posted by Dr. Rockey and her team is how the spectrum [eg. MEDIAN (not mean) +/- 1 SD] of award size compares for the data tranche {4th – 9th RPG} vs the median +/- 1 SD for the tranche 1-3 RPG.
Agree that even grants that get below top 25% are often excellent. The 20-25% estimate is anecdotal, based on opinions of more than one individuals with considerable experience on study sections.
About the award sizes, I believe the logic of NIH (and that of other funding agencies in Europe like the Wellcome Trust) is that it is better to fund some individuals well so they can do what they really want, instead of funding a lot of people a little bit so that their hands are always tied. They are not looking for the best bargain but the best science. That’s why the ability to do things at a lower cost does not impress them much. They don’t worry about higher indirects if the science is better. I don’t agree with this completely, but that is their belief.
This can of course be questioned and reevaluated. Maybe the ability to do great science at a lower cost should be preferred over the ability to do great science in a costly and inefficient manner. One must not automatically assume that low cost = ordinary, incremental science and high cost = high impact science.
The indirect costs for some universities is too high and does not reflect their actual expense. Perhaps the indirects should be reduced based on the total number of grants held by the institution (every additional grant gets a smaller % of indirects) because it does not cost proportionally more and more once you have some infrastructure in place. The cost of electricity, phone, and trash collection is a joke compared to how much money they get. The 200K salary limit is also high. Nothing wrong with the salary itself, but let their medical schools pay for it rather than taxpayers.
This may be slightly off topic, but certainly related. A couple years ago, I received my first R01 and later also received a list of other new grantees from the same institute. Looking through the list I was quite surprised at the number of R01 grants awarded to foreign institutions. Since then, I have also spoken with colleagues on study sections and heard of numerous instances of foreign institutions receiving R01’s. Are there any published statistics on NIH funds that are granted to PIs located outside of the US? A quick scan of NIH RePORT confirmed that unsolicited R01s (i.e. PA-10-067) are certainly being funded to foreign entities. It is surprising to me that in the current funding climate this is even allowed. Although I am not an expert here, it seems to me that all other countries require research money to be granted to PIs located domestically. For example, my colleagues that are not US citizens are still ineligible to receive grants from their home countries because they are faculty within the US. This makes sense given that research is funded by taxes. I have discussed this with other faculty and have yet to receive a clear answer why the US is different in this regard. Is this a unique US policy or am I mistaken? Given our current federal fiscal situation should this policy change?
David,
Your suggested system for creating actual productivity out of NIH grants is nearly identical to one I’ve been pushing for years. Too bad the relevant decisions are made by university administrators and NIH administrators with the “advice” of powerful “winners” in the current system. As a result, significant change is not likely. And like you, I’ve been very successful at winning NIH grants my whole career, so this is not sour grapes.
And Greg also is correct. No change in the structure or rules of the peer-review system will be effective in funding better science if the changes can somehow be circumvented by the self-serving cabals that constitute most study sections.
Over the years I came to realize this is “an old boy’s club”, a facade to a meaningless road full of empy, PR-type people with sales experience.
Let’s face it. Grants are a sales pitch. It’s all in the script and your buddies. If there was ANY consideration for science, the pressure would be on QUALITY of papers published (thoroughness of studies done), not quantity of papers!… It is conceptually wrong to have a quota of publications in order to keep your job or be respected. Anyone can publish junk.
Science should be left to passionate people, with no/little financial interests but visionary when it comes to discoveries. I despise a system that rewards those with more publications. What does it tell about an investigator with 300 publications but no clear original work? If any, it should go against him (for waisting resources)… If I spend $10 mil. for building a palace and it comes out a highly-visible shiny shack, am I a good builder?
I decided to leave science.
Not that anyone wants to hear this, but the most effective way to make significant advances in medical research and application is to provide greater funding for SBIR and STTR grants. It’s a pretty simple concept; about $200,000 for proof of principle to prove your idea has merit and then about $2,000,000 to move it to the IND enabling stage. Success is dependent on actually proving something to be of value. No mind numbing, incremental pieces of data that are given credibility, no matter how mundane or pedestrian, by writing a paper and having it published. No academic administrators to support with overhead, only those necessary to manage a small company. No ivory tower sense of entitlement that “my research is important so it should be funded.” After 25 years in academia consistently funded with RO1s, I am happy to leave it. Either my company’s drug development work is novel and shows promise as a medical benefit as defined by certain criteria, or I’m out of business. Right now SBIR finding makes up 1% of any NIH agencies budget. It should be 5% at least. And some of the money can come from those 3-6 grant laboratories. If an investigator has 3 or more grants for any length of time and can’t come up with something that can be developed as a potential commercial product, then the money should go to another lab with fresh ideas. And no whining about basic research being the backbone for medical advances. Of course it is, but way too much is irrelevant. If you are spending your time complaining about NIH funding, go join or start a company.
yr facts are a bit off –
there is a Congressionally mandated “set-aside” for SBIR/STTR way higher than 1%, and most ICs appear to have trouble getting enough non-junk applications (even setting aside that some companies seem to exist largely as SBIR mills)
Re: Data links provided by Steve Kron above.
50% of all NIH funding goes to just 20 institutions. Does anyone believe these institutions are pushing for change in the NIH funding system?
@ Bill Halford
“Many permutations of a similar “subunit vaccine” approach has been repeatedly tried with HIV (AIDS) vaccines over the past 20 years, and have met with larger numbers of even more spectacular failures.”
Perhaps relevant to the #1 winner in the NIH funding competition?
Thanks again to the data links provided by Steve Kron.
Bottom line: The myth is NOT BUSTED. The raw data clearly show the real story. The graphs are simplistic and misleading. Moreover, the raw data show that the percent of grants going to investigators with 3+ grants has more than doubled since 1986.
Bill Halford is brave to point out the ‘story telling’ aspect of science coming out of big labs. Most of us eschewed a career in sales because of the underlying lack of truthfulness inherent in the job, only to find that it is the salesmen who survive in the highly competitive science funding arena.
I also agree with Bill’s other points about diversity and the need to fund those who will challenge fad science. This kind of insight into the reality of the NIH funding machine is lacking in our leadership.
NIH should generate the data to truly measure productivity in the way of publications, and their impact factors, per grant. I believe this was done some time ago (10-15 years earlier) and the NIGM data are an update, but using direct amounts funded as the basis rather number of grants/lab may be misleading – in either direction. Of course there will be caveats in any analysis; the impact factor is greatly skewed in the case of review articles, which of course are generated more often by the big labs because they have the personnel to do so. Let’s get the data and then talk again.
Bill and David –
It appears that most of the damage in wasted time and money is identified too late in the study or after the study when it is found that the results are ‘stories’ rather than real science. Is a system that manages the study in real time for compliance and patient safety, has an audit function that time and date stamps all events, and correlates study data with research outcomes and enables continuous management in real-time, at the patient level, available to all management levels be of use in addressing this inscrutable situation being called clinical research??????
Hi Kenneth,
Once a grant is awarded, I believe that P.I.s need to be given the opportunity to complete the proposed project with minimal oversight. Therefore the issue of “minimizing waste” is best considered over the career of a P.I. rather than over the period of a single grant.
A productive career in science generally spans 20 – 40 years, which gives us an awful lot of time to figure out what motivates and drives our colleagues who are requesting NIH funding. My questions of waste and inefficiency relate to the disconnect I sometimes see between investigators who are consistently well funded, but who rarely contribute observations that have any real permanence or impact on the body of scientific knowledge under investigation.
I want to see P.I.s getting funded who have sufficient focus and attention-span to tackle at least one problem for real. However, the current NIH system encourages P.I.’s to submit applications in a more opportunistic fashion, requesting funding for topics that (1) often lie outside of their primary expertise / interests, and which (2) seem to be the most pressing problem in the world based on the dollar amounts being invested. Anyone ever heard of Q fever or tularemia? These are areas of microbiology that were really obscure in 2000, but had the NIH throwing up Biosafety Level 3 and 4 research facilities all over the U.S. in the mid-2000s in the wake of 9/11 and the anthrax scare. How much do you suppose all of those independent facilities cost? How much has U.S. health improved as a result? This is but one example of NIH funding fads that have come and gone, and I am sure that more lie ahead.
A primary source of waste I see lies in (1) the NIH’s attempts to make specific areas of research important by virtue of pouring money into said areas, and then (2) not recognizing that throwing money at a specific health issue / scientific issue usually does not make a difficult problem suddenly tractable / prone to be solved in a few years.
Good scientists and their new, individual insights have always been the backbone of the scientific enterprise, and always will be. Good scientists always do good science and publish good papers that address important topics at the front of everyone’s mind. It is in every fiber of their being. In my field, there are certain names that appear on papers in PubMed that cause me to drop everything I am doing, and immediately read their papers to find out the latest insight that they have gained. We all have such people in our fields of study that are a consistent source of insight, and who challenge us to re-think what we thought we already knew. Often, these same people are those who do not disappoint when they present at scientific meetings (be it a poster or a talk). Unfortunately, when funding lines tighten to less than 10%, there are simply not enough slices of pie left for the NIH Peer Review process to necessarily reward or encourage those scientists to continue pushing the envelope.
I do not know how to solve this problem, but I do feel that others have offered some viable options that merit consideration. Whatever the mechanism, I believe that there are two keys if the NIH wishes to create a sustainable system for funding quality research. First, it is imperative that we learn to (a) place a higher priority on funding scientists whose primary drive stems from a desire to solve scientific and/or human health problems versus (b) funding individuals whose desire to extract money from the NIH supersedes their interest in the topics of their grant proposals. Second, we need a system for funding scientific research that rewards good scientists not with financial wealth nor lavish labs, but with the reasonable chance that 3 years of solid scientific productivity might be rewarded with sufficient funds to continue the process of scientific inquiry for another 3 years.
– Bill H.
P.S. While I have greatly enjoyed “howling at the moon” as one blogger commented, it’s time for me to get back to my real job. Best of luck to all of you with your grant applications.
Having a limit of 2 grants per investigator is a must-to-do at this point, especially for investigators who have many other duties and titles (department chair, dean, NIH council members……). Otherwise, many people will be forced out of the research.
We have been attempting to replicate your data on the number of grants for the top 20% of PIs and cannot. Here is what we have come up with. Our total, using each of two assumptions about how you counted grants, falls short of your total number of PIs by more than one quarter. Can you explain how PIs were counted in your data in FY2009?
We apologize for the confusion. The incorrect data was posted. You can find updated numbers in our correction post.
I can’t find it in my database right now, but wasn’t there a “Science” article a few years ago listing 200 NIH grants held by 20 PIs? I wish I could find it…does anyone have this reference???
Published online 19 March 2008 | Nature452, 258-259 (2008) | doi:10.1038/452258a
News
222 NIH grants: 22 researchers
Thank you Saundra.
In my department, 2 PI’s 65+ have 9 RO1’s between them, studying the same molecules they have been studying for the last 25+ years. The Associate/Assistant Professors have a total of 2 RO1’s, and spend most of their time writing grants, rather then doing science. The Generals can justify all they want, us Privates down here in the trenches know what is really going on.
Want to make an immediate impact on the system?
Simple. No more then 2 RO1’s per lab. This will:
1) Allow PI’s to go back in the lab. Most can’t, too much time spent writing grants. Once good PI has 2 RO1’s, chair will stop nagging to get more funding to increase indirect cost $$$ for Big State U. PI can now go back in lab. With PI back in lab, no need to hire extra staff or take extra graduate students as cheap labor.
2) Allow PI to spend more time training students, rather then army of Post-Doc’s, non-TT faculty and technicians training students in Mega Labs. In my department, graduate students are having less options as smaller labs are shrinking, so they flock to the 4-5 Mega Labs. However, they take 6-8 years to graduate, get limited one-on-one with their PI, and learn more about gaming the system than doing actual science.
3) Allow PI time to dream about science. No time for that these days, too scared about not having grant $$$, so we write grants 24/7, search for the safe scientific ground that will get funded, and ignore the exciting chance to make a new discovery, as this leads to “project is unlikely to be successful”, “project is time consuming” stock NIH review comments (unless you are politically connected). By limiting PI’s to 2 RO1’s, PI’s can have bread and butter projects continue along and NIH could create a *true* pilot project system for risky ideas (R21’s are judged like RO1’s, we all know it). Think about it, a 50k injection for a risky idea that could actually make a difference. Even if 1/100 succeeds, it could lead to a major change in society, create opportunities for biotech creation, jobs, tax revenues etc etc etc…
4) Eliminate the age war that is flaring up. Old PI’s gripe that Young PI’s are getting favors in the system. Young PI’s gripe that Old PI’s hoard the cash and get funding because of connections and past discoveries, not current talent. Anyone who denies this is happening is either ignorant or a liar. Limit labs to 2 RO1’s, don’t give new PI’s favors, don’t force old PI’s to retire until they are ready. What is happening between old vs young is *not* good for anyone, and will probably get worse as funding becomes more scarce.
Seriously, if your science is that frickin’ spectacular, you will have no problems getting funding from private sources to create your Mega Lab. If no private funding agencies are interested in your science to fund it beyond 2 RO1’s, why should the trusting, uneducated taxpayer be interested to fund it beyond 2 RO1’s?
Unless you are neck deep in this mess (like me), you probably don’t understand just how scary this time is for new PI’s. Once the economy turns back around, feel free to let cunning PI’s be as greedy as they want to be … but for now, we need to fund as many good PI’s as possible.
Dear “Not Expecting Anything,”
I agree that 2 R01s is a sensible amount of public money per investigator.
I also agree that the NIH system is prone to squandering an excess of money on P.I.s who are for one reason or the other (1) particularly well tied to the right study sections or are (2) particularly adept at telling stories that fit the expected shape and size of a NIH grant application.
I agree that your level of expectation of change from within the NIH is realistic….it is not going to change until there is no other choice. Luckily for the rest of us, life goes on with or without support from the NIH.
Finally, I think terms like honesty / integrity versus greed / cunning are not completely irrelevant to this discussion. While it does not surprise me that some individuals will have no problem bleeding the NIH for all they can, it does surprise me that apparently the NIH does not have the wherewithal to identify those P.I.’s who chronically squander NIH money obtaining data that serves no particular purpose, nor addresses a single, salient question.
The story of “The Emperor’s New Clothes” comes to mind…………when did we collectively lose our backbones and our ability to differentiate good science from drivel?
The purpose of science is to answer well-defined questions that are formulated in a manner where the empirical measurements point us in the direction of which of two or more possibilities is wrong.
Unfortunately, I see far too much of NIH’s money invested into gee-whiz science that is driven by technophiles who love their new toys, but who have not developed a clear and compelling scientific question that needs to be addressed. Questions and common sense should take precedence in science over the ever-escalating one-upsmanship to prove to your colleagues that your lab is in possession of the fanciest methods on Earth. In high school, kids like this used to get beaten up on a regular basis. Maybe that wasn’t such a bad system after all.
A 19th century logician / mathematician, Charles Pierce, has a nice quote that cuts to the heart of the matter: “There is one thing even more vital to science than intelligent methods; and that is, the sincere desire to find out the truth, whatever it may be. ”
Just think how much better the NIH’s funds would be spent if a larger fraction of P.I.’s wrote one well-crafted grant application per year that was truly motivated by a desire to expose an unrecognized truth about how nature operates. I would assume that most visitors to this blog would agree that what the NIH system is encouraging P.I.’s to do is write 4 to 8 “NIH lottery tickets” per year with the idea that one should, by random chance, be funded every 2 years. Under this latter system, the more detached a P.I. can be from each grant application, the better. I seriously doubt that this is a healthy approach to encouraging scientific innovation.
In short, I think that the NIH funding system does not reward “honest and thoughtful” applicants who strive to align their scientific writings to the reality of (1) what exists in nature and (2) which raises questions that represent the most tractable and most pressing matters in a field of study. In contrast applicants who are “cunning and/or greedy” generally treat the truth as a more malleable substance that can be bent and reconfigured to meet the needs of their next NIH grant application. As a postdoc, I assumed that the NIH system would necessarily distinguish between these two pools of applicants, and make the correct choice 90% of the time in favor of the honest and thoughtful P.I.s. However, nothing in my own experience of the past dozen years has led me to believe that this is indeed the case. To the contrary, the “cunning and/or greedy” seem to be doing just fine.
– Bill Halford
Comment removed at the request of the submitter 7/21/11 — Rock Talk Blog Team
I’d like to cite this. What is the source of the data?
The NIH Office of Extramural Research. Be sure to use the data from the correction post.
Does the NIH have a new rule coming down the pipeline that will restrict the number of grants that any one individual can hold as PI at the same time. I have heard rumors that a maximum number of concurrent grants for any one PI will be either 1 or 2.
In an earlier discussion on the blog, (How Do You Think We Should Manage Science in Fiscally Challenging Times?), this was among the proposed funding scenarios, but no policy change has been issued.