29 Comments
Last week I was in Weston, Florida participating in our second and final NIH Regional Seminar of this year. Our seminars draw approximately 500 individuals who, with the help of approximately 35 NIH and HHS staff members, learn how to apply for and manage NIH grants. This year we began tweeting live from the seminars our favorite tips, and some of you followed along. For those of you who missed the tweets, here are some of my favorites.
- RFAs usually don’t come out until we have a NIH budget. So pay attention to NIH guide.
- Subscribe to the NIH Guide. Best way to find out about programs and policies. You’ll get it every Friday. http://t.co/NYzQvyf
- Success rates are at 20% but pay lines at an institute may be lower. Understand the difference.
- Thinking about applying for a R21? Think about applying for a R01 instead if you are ready. R01 has higher success rate.
- Some ICs have paylines for early stage investigators and assistance for new investigators at first renewal stage See IC funding strategies.
- Office of Laboratory Animal Welfare has many outreach efforts. See http://t.co/6g1wsop
- Loan repayment is available to help pay back advanced educational debt in biomedical sciences. www.lrp.nih.gov
Like what you see and want more? Find me on Twitter @RockTalking. You don’t have to join to read my tweets.
Which ICs extend breaks to first renewals and what on earth is the purpose of that?
The first renewal is one of the spots where investigators drop out of the pipeline, so some ICs extend breaks to new investigators at this point. You can see IC funding strategies at http://t.co/uaegDkH.
Can you give us a hint? My eyes are glazing over after trying to sort through a half dozen of my closest ICs and not finding any mention of this policy…
NHLBI is one that we know of that states it in their funding strategy. You might also want to contact your favorite ICs directly to find out more about whether they offer something similar.
Comment removed at the request of the submitter 7/21/11 — Rock Talk Blog Team
Hard to argue the point about study sections. I see it from both sides: as a study section participant as well as from perspective of grantee. There is an element of randomness in every grant review, just have to get lucky that the reviewers latch onto the idea for whatever reason instead of using it as an opportunity to look smart in front of colleagues by zeroing in on methodological issues that are more a matter of opinion than true rigor. Or of course the lack of reviewer accountability on a conscientious and thorough review. Or the “how fast can we get through this meeting so we can all go home early” mentality of peer reviewers and SROs alike. Why are we speeding through meetings?
I would think I was just being a defensive grant writer but I see it in action in every single study section I have been on. The unfortunate deaths of good ideas. Peer review is a horribly flawed system with no oversight or checks and balances whatsoever.
I 100% agree that we need guidance on how to navigate the peer review system, especially if these problems are not going to be fixed.
Comment removed at the request of the submitter 7/21/11 — Rock Talk Blog Team
I agree with micromike123 not on just a single point but on all seven points. I would say that this is just a short list, which probably should be extended.
I currently see many of the top researchers loosing their NIH grants, most of them were competitive renewals. I have personally seen researchers with publication lists that include a series of Cell and Nature family publications. I know several of these people personally. Most of them have many years of experience in writing NIH grants. The consequences are dire. Some of the best students and postdocs, thus the future of science and the countries competitiveness, either loose their jobs or they decide that if despite the success of their PIs, or their own work, grants do not get renewed, they do not want to follow this career path. The NIH lists ~50% success rates for renewals – I can just not believe that this is true, given the large number of top scientists I see struggling. What is common to all of the cases I know is that they do basic or strategic science. The only explanation I have is that there is a shift away from basic science – if thats the case, this is a disaster. Big pharma has given up basic research or support for basic research many years ago. If funding agencies do the same, this country is on its way to follow Rome’s destiny.
“I currently see many of the top researchers loosing their NIH grants, most of them were competitive renewals.”
I’ve seen a lot of this, too. I’ve also heard from reviewers that the bar is higher for competing renewals because reviewers are looking for the new twist. Drilling down into your system isn’t considered compelling work, regardless of productivity, and it’s even harder to keep a cohort study going without at least one exciting new aim. I’ve heard from reviewers (and those reviewers said a few program officers agreed) that if the next grant submission is new enough, just make it a new application because you won’t have to use any of the 12 pages of your Research Strategy section on a progress report.
I asked about this in Weston, and the NIH representatives there disagreed. I was told that so far, OER’s statistics still show that competing renewals succeed in making the payline more than new submissions. Still, it’s interesting to see you make the same observation that I hear in other places. Is it perception (paylines are down in general), reality, or more senior investigators not adapting well to the shortened format? Hard to say.
I would agree with micrmike- one of the biggest frustrations is when the Program Officers do not communicate and do not return emails or phone calls- yet they are supposed to help you !
The fundamental issue keeping paylines low is that every time the NIH budget expands, med schools hire more soft-money faculty. Real reform needs to include serious caps on salary recovery.
In the meantime, as a study section member, my advice is:
1) Learn to write!
– Make sure your introduction succinctly lays out the information a reviewer will need to understand your project rather than simply listing random facts. Make sure it is obvious how these facts relate to the work you will actually be doing.
– Each paragraph should start with a topic sentence that lets the reader know what to expect and why you’re telling them what you’re telling them. Your old English composition teacher was completely right about this.
– Don’t leave anything important “between the lines”. Your readers are probably pooped. Just tell us what exactly what questions remain unanswered in the field and exactly how your work will address those, and point out what is innovative.
2) Read the relevant literature.
3) “Innovative” is great, but “well worth doing” and “feasible” are probably more important in the end.
Comment removed at the request of the submitter 7/21/11 — Rock Talk Blog Team
I do not know if this possible to do – let the reviewers rank the research proposal portion of the grants first without knowing who submitted the grant. Once this is done then they can look over the rest of the grant to give a final score. This way I think there is no room to think that proposal reviews are biased based on ‘good-old-boys club’ and ‘I scratch your back and you scratch mine’ mentality.
In at least ~10-20% of cases, I would be able to tell whose grant I was reviewing anyway just by knowing the field and/or skimming some background literature. Given that each grant gets 3 readers (or more), I don’t think real anonymity is possible.
Comment removed at the request of the submitter 7/21/11 — Rock Talk Blog Team
One of the issues facing every grant writer is the need to compress an incredible amount of information into a few pages. This leaves little room for formatting, separation of topics, etc., that would aid reviewers in the review process. We should have enough of the new format applications that average word counts could be generated. My suggestion is to change the text limitation to number of words, but allow more pages. This would allow the grant writer to use different font sizes to headline sections, extra lines between topics, and other ways of producing clearer presentations. The overall workload on the rviewer would not increase, and in fact, such an approach would likely lighten the load.
I’ve been on study sections for years, and this is the very best suggestion I have ever seen. Sally, please take note. What a simple way to make our life easier.
I couldn’t agree more with the suggestion to limit words, not pages. Encouraging applicants to use effective formatting techniques would ease the burden of reviewing proposals
Quoting the late Ephraim Racker:
“The peer review system works; the problem is that you are not always reviewed by your peers.”
Comment removed at the request of the submitter 7/21/11 — Rock Talk Blog Team
@micromike: There is no sophistry there. Ef meant clearly that “peers” are those with experience in the same field. In other words, those who can actually REVIEW the application. I have served in several review panels, both as an ad-hoc (because of my own expertise) and as a regular member. It is appalling to see that some applications are reviewed by people who do not understand the field and who make relatively little effort to do so, especially when the application involves novel, specialized technologies. I have witnessed applications being scored primarily on the name of the applicant, and others being subjected to standards so stringent as to prevent fundable scoring (like requiring crystallization of complexes to prove that such complexes exist!). Likewise, I have seen extremely well thought-out reviews hat addressed the strengths and weaknesses of the application with such great detail and insight that, whatever the outcome, a revised application could not help but be better. We should strive for the latter, and prevent the former as much as possible.
Comment removed at the request of the submitter 7/21/11 — Rock Talk Blog Team
It’s frustrating all right, for most of those points above. The whole thing’s almost a crap-shoot right now, and I’m not sure the study sections have standardized the scoring at all just yet. For a bunch of scientists always talking about rigor and statistical strength, you’d think some internal consistency in the scoring system would develop over time, but it doesn’t sound like that’ll be happening any time soon.
I have to agree with BVM too. It seems to me a better system would be to anonymize the initial review. The system should be set up so that the scientific merit and feasibility are the only parts that matter at first. If the idea and the methods are sound enough to warrant a high score, THEN go back and check the qualifications of the individuals and institution. If they’re a little sketchy, then limit funding to the first year, maybe two, and reduce the budget so that the PI has a chance to collect some data and publish it – to demonstrate that he/they can do the work despite the questions. My grant submissions have gotten high scores for science/innovation, but environment/PI they range all over the place (but usually only middling). So basically, the study section is punishing me for having a postdoc that didn’t work out, without allowing for the possibility that I might not be the main reason for it not working out. And punishing me for my institution not having the reputation of a Salk or Johns Hopkins. If the NIH wants new and good ideas, they should only review the grants based on those ideas first.
At this point, I could change my name to Ouroboros – I can’t get funding because of my biosketch/environment, but I can’t improve those because I can’t get funding to do the work (or to get a job at a higher-profile institution).
Remember the weighting of those individual numbers in assigning a final score is up to the reviewer.
When I review grants, I give a number to the investigator and institution because I have to fill in the box. However, unless there are exceptional circumstances (e.g. the PI has no actual lab space or appears to know nothing about the techniques to be used), it doesn’t really contribute to my overall score.
Comment removed at the request of the submitter 7/21/11 — Rock Talk Blog Team
Comment removed at the request of the submitter 7/21/11 — Rock Talk Blog Team
Comment removed at the request of the submitter 7/21/11 — Rock Talk Blog Team