59 Comments
Dr. Richard Nakamura is director of the NIH Center for Scientific Review.
Last year we highlighted a new section of the NIH Data Book that provides statistics about peer review across NIH, including peer review organized by the Center of Scientific Review as well as peer review organized by NIH institutes and centers. Then, and now, these data truly demonstrate how you, our peer reviewers, are the lifeline of the scientific process. During 2014 nearly 24,000 reviewers participated in more than 2,500 peer review meetings to assess the scientific and technical merit of NIH applications, and many of those reviewers participated in multiple meetings. Your service to NIH expands our capacity to fund the best science, and provides an essential breadth of perspectives on the research projects and research training that NIH supports. In the lab and in the study section, we’re grateful for the myriad ways you work with us to advance human health and benefit the health of our nation, and the world.
As you can imagine this enormous peer review endeavor is highly dependent on your willingness to serve. We know that pay lines are low and that your professional obligations are high, both of which have impacted and continue to impact your decision to review when we ask. However, we believe that the professional obligation to serve as a peer reviewer is extremely important to keep this great enterprise going, particularly for those of you who currently have NIH support. NIH examined the peer review service records of scientists with current research project grant (RPG) support and who have received a total of $1M or more in total costs from NIH in the last 5 years. There are more than 25,500 scientists in this pool, and of these, 83% have provided at least one day of committee service in 5 years. Looking closer at the data, if one day of peer review service per year would be considered a reasonable expectation for service, then currently fewer than half of these funded scientists (45%) achieve that level of service. It is important to note that not all funded PIs are asked to serve each year, and those individuals are also included in this figure. In addition, there is an increase in service with more funded R01 grants. The data revealed that 42% of PIs with one R01 served at least one time per year, 59% of those with two R01s served at least one time per year, and 72% of those with three R01s or served at least one time per year.
So with service in mind, today we published an NIH Guide Notice reiterating our expectation that NIH-supported PIs will serve as reviewers when asked and calling on awardee institutions to encourage their researchers in accepting invitations to serve on peer review and advisory groups. There are many venues for institutions to promote review service among their NIH-funded investigators, for example, at faculty and employee workshops, orientations, or training sessions. Peer review service is part of a researcher’s overall contribution to the scientific enterprise, and workplace incentives and evaluations are opportunities to acknowledge individuals who serve when asked.
NIH works as full partners with our grantee institutions in advancing the scientific enterprise. By encouraging NIH-funded investigators to serve on peer review and advisory groups, together we can promote a culture of sharing and service that maximizes the potential and promise of biomedical research.
Try making service easier. Reinstate free coffee/beverages etc so we don’t have to wander the hotel/local area looking for a reasonably priced drink. Stop putting the per diems together with the honorarium on the 1099 so we don’t have to file schedule C or whatever that form is prevent having to pay tax on it. Give us some flexibility on travel arrangements. Give us fewer grants to review (I had 9 last time!). Drop the new biosketch idea, which will add 10mins per grant to read (x9 grants = 90 more minutes for me per cycle) to say nothing of the waste of time by the applicant. In the balance between time available to apply for grants vs time to review them, low funding means more time required to write grants, and less to review them. Its not surprising that more people are turning down NIH service.
Agin PI is exactly correct- All expect the work load- A long time ago in a galaxy far away the work loads were all 12-14. Agree with no online reviews. The new biosketch is a waste of everyone’s time.
Agree that new biosketch is a waste of time for the individual PI’s submitting grants as well as those reading them. Who comes up with these ideas and implements them without getting the opinions of the PI’s?
As minor as it sounds, I agree about the coffee, and would add back the danishes and fruit we used to get back in the age of falconry (a few years ago). At any given study section, about half of the panel is jet lagged and we are all crabby. I suppose this goes to some rule that came about because some other branch of government went to Las Vegas. But rules are made by people, they aren’t laws of physics–surely this can be changed.
I also agree about the online reviews–the quality of the review is simply lower. They have a place, but it’s a specialized place.
I agree totally with aging PI. I don’t drink coffee but find it amazing for all the time and work spent they can’t even provide it for those who do. The new Biosketch is just another waste of time and should be cancelled. Online reviews is a bad idea, there is a big difference when grants are discussed face to face. There should be Assistant Professors in the study sections, but their numbers should be limited and the number of grants they review should be limited. Being on study section is a great learning process, it was for me and my SROs controlled the number of applications I reviewed then.
In discussions of this NIH Notice online, there has been a discussion of the suggestion that NIH should post an aggregated list of reviewers who serve over the course of a year, ideally with an indication of their review load. This step toward more transparency involves aggregating review committee rosters which are already publicly available. This would provide the basis for analyses of which members of the scientific community are contributing to this essential activity and would help individuals and institutions allocate credit for this service.
I would second all the comments above. The new guidelines for the biosketch are a huge waste of time both for the applicant and the reviewer.
I agree with Dr. Berg–an aggregated list of reviewers and their service would be illuminating. I suspect that the service distribution is highly skewed, and that there are investigators who are giving far more than they would normally owe–e.g., the many HHMI investigators who serve on panels–and many more who are serving less than they might well be willing to. My sense is that the panels need more service from established investigators, who have an institutional memory of the process, and (oddly) are less likely to get distracted by minor flaws, and (often) more supportive of innovation and so ESIs.
NIH has to discipline the whole review process and increase reviewers’ responsibility for their written work. I reject as false the popular claim that scientific review should be as simple and short as possible. Review writing is one of the major professional functions of scientists that corroborate their best qualities – competence and vision. To enhance the responsibility of the referees, CSR should conduct random internal reviews of written critiques and make conclusions public on its web site.
What is your current stance on Assistant Professors on study section? Your predecessor was very keen to eliminate them with no real rationale or data offered to explain why.
We carefully and systematically work to enrich and develop our pool of reviewers by training and engaging assistant professors to serve on our peer review groups. Our Early Career Reviewer Program provides emerging researchers with unique and real review opportunities that help us train them to be tomorrow’s reviewers and give them career-boosting insights into the application and review processes.
As assistant professors move further along in their careers, we regularly try them out with a full review load as temporary reviewers and bring them back if they do well. When recruiting chartered reviewers, we seek seasoned researchers and leaders in the field, who are usually full professors (60%) and associate professors (35%). However, we will also recruit otherwise highly qualified assistant professors (5%) who are clearly on a tenure track.
How do those numbers compare with the proportion of applications submitted by PIs of each rank?
There are often assistant professors on the SBIR review panels on which I also participate. Many of them are trying very hard to show how good they are and grand stand to the detriment of the proposals that they are allocated for review. My preference would be that they not be involved until more seasoned and promoted or better review instructions be provided.
According to the CSR’s own analysis (PLoS ONE 5 (11), e13526), Study sections are not very effective review tools. The sooner the better, they should be replaced by a more elaborate and less costly online reviews by a large peer group. Respectively, CSR may need to reorganize the staff dealing with assignment and receipt of reviews.
Different sciences of the XXI century constantly crisscross. Something that people in a given field have barely heard of can revolutionize it, forces the field to learn, to acquire, to adapt, to change its very mindset, let alone its technical knowledge. CSR has to take it into consideration and permit the targeted review of such single Specific Aims in interdisciplinary projects to be conducted on request by the relevant invited experts. Respectively, referees should be instructed to abstain from the reviewing those specific aims that do not conform to their expertise. To meet this challenge, outsorsing of peer review to private consulting companies should be piloted now, before the new Administration orders it.
I agree with NMS on both points. I think Internet assisted review is much more efficient when more aggressive reviewers do not overpower quieter ones, and reviewers have much more time to read about applications where they are not primary reviewers. Additionally, it would save millions of $$ that are now waster for travel and lodging. This way the reviewers could be better compensated for their time.
You do understand that “one day of service” as counted by an actual study section meeting is actually many days of work on the part of a reviewer, correct?
If not, this sort of easy disregard for the task you are asking us to complete may be part of your apparent problem in recruiting reviewers.
Exactly right Drugmonkey.
Well, NIH review is a public service although I myself like to review for the DOD who pays more.
What % of R01 funded investigators were asked to serve on study section? I know several that have had R01s, that were never asked to serve. Perhaps you could elaborate on how reviewers are chosen – with special emphasis on the parts of this process that might influence the low participation rate by funded PIs.
DM is spot on about this. Thoughtful and helpful review, whether of manuscripts or grants, takes time and often more than one reading. Along these lines, incorporating J. Berg’s suggestion for transparency recognizes the significant commitment review requires (at the very least it can be shown to home departments for P&T and compared to other colleagues’ service within and outside of the dept.) and someday may help distribute the load more evenly (at least one may hope).
Make sure all study sections are in person where conversation and debate are encouraged.
While I understand that there are tremendous cost savings associated with online web-forum type reviews, and undoubtedly some individuals who otherwise wouldn’t volunteer to review can be recruited, these are a serious mistake in my view. The panel engagement with applications for which there is significant disparity of pre-discussion scoring is just not as good as it is in a face to face discussion.
I and other colleagues I’ve talked to are becoming quite weary of the nitpicking that goes on in panel discussion to try to spread out scores. When only the top few grants in a large pile will get funded anyway, the process of peer review starts to seem dangerously close to futile.
I agree with the BugDoc. Having worked on study section for a few years, I found that the process has become more than futile. Seemingly random committee scoring no longer rewards the best science, or useful approaches/drugs to treat diseases. After dedicating my time and service to NIH, I found that my own applications were not being judged fairly by “peers”, but by inferior committees and I was forced to remove myself from the review process. Years have now passed and at this point I would volunteer and try a new committee to see how the process has changed.
I agree. I prefer the review process that the DOD has when the 3 primary reviewers discuss the application among themselves over the Internet until they come (or not come) to a common vision. Then during the meeting discuss only applications that receive disparate scores and request participation of all experts on the panel. Based on my experience, all study section chairs, without exception, requested a minimal discussion for the applications with really close scores, so there will be no difference here.
Also, at this level of funding, 3 reviewers is not enough as one unfair review could sway the entire score into non-fundable range.
Suggestions:
1) As stated elsewhere, roll back the ill-conceived expanded biosketch decision;
2) Institute a meaningful pre-screening to eliminate a large percentage of grants that are not competitive, similar to a HHMI process. Reviewers would review this one cycle before the full proposal, which would be invited on the basis of score of the preproposal;
3) Limit the number of full proposals per reviewer;
• Reinstate coffee/beverage service as caffeine levels do make a difference in reviewer weariness
• Go back to separating per diems from honorarium on the 1099 so taxes are more straightforward
• Go back to the recent biosketch idea, which nearly all members of my current study section view as a waste of time by the applicant and reviewer
• Decrease unrealistic pressures to spread scores beyond what is reasonable based on the science
Question: I’m interested in serving as an “ad hoc” on a study section that reviews proposals in my area of expertise. I’ve served other Study Sections before, but not that one. Can I contact the SRO to volunteer?
CSR welcomes researchers who volunteer to serve on our peer review groups. There are a couple of ways you can do this:
1. Check out our Early Career Reviewer Program if you’re an emerging researcher. This program is designed to train reviewers and jump start their research careers.
2. Send your CV to csrvolunteer@mail.nih.gov if you’re an established researcher and you’d like us to explore pairing you with an appropriate review group.
3. Send your CV to a CSR Scientific Review Officer you know from having your applications reviewed or from having served as a reviewer in the past.
If an SRO finds an appropriate opportunity for a volunteer to serve, the SRO would then contact the volunteer directly.
Many thanks to all of you who are responding to our public appeal.
Despite overlapping the change in grant format and scoring systems, my experiences on study section have been generally quite positive. I think that the process generally works better than I initially expected, at least in my limited experience.
The various little cost-cutting measures (providing coffee, breakfast, snacks; making the distinction being reimbursement for out-of-pocket expenses and the honorarium distinct in the eyes of the IRB; allowing reasonable flexibility on travel arrangements) add up. These changes have definitely affected the experience negatively and likely also decreased the willingness of some to participate. The funds saved by eliminating these small “perks” would seem trivial compared to those incurred if the system of making decisions that impact NIH funding were allowed to deteriorate. Most scientists know what a disproportionately large impact a box of doughnuts can have on the climate of a meeting.
I think it is important to select the grant reviewers based on their current research activities and not just full professor, because many full professors are not active researchers and many have abandoned research many years ago but still serves as NIH study section reviewers.
I wanted to participate and sent my CV to the unit that is responsible for recruiting early career investigators, but was turned down. I got an email saying that I was not independent, although I am and have an active R01. Don’t know what happened and why I got that type of email.
Could you hold more grant review meetings on the west coast? Flying to DC three times a year gets old after a while, especially when the amount of money I have to pay a babysitter is more than the amount I make as a reviewer.
I have usually jumped in to help funding agencies and journals with reviewer service. But the less than reasonable reviews conducted on my own applications and a generally low level of support I have received from the NIH, combined with unenthusiastic SRAs and POs, I am really not inclined to undertake any reviews. Its like this, why do I want to review for a journal where I never publish. I have the same view to reviewing grants, why would I want to review for a system which has been unsupportive? Serving on a panel is not really an ego trip for me either. I am of the belief that many mid career PIs may be of this opinion as well. That is bad news for the reviewer database for the NIH.
You mentioned the abysmally low pay lines. I get called to provide peer-review all the time. Back in the “good old days” those who served on numerous study sections, upon submission at least had their grants discussed even though funding lines were relatively low. Senior faculty with skills necessary for fair peer-review are unlikely to volunteer their time because so many of us need three research project grants to run a good research program. Aa long as funding is so difficult to attain, and for such a long period, it’s hard to persuade federally funded scientists to review grants for all the reasons previously mentioned. One more point, since most grants are now only four years, even capable reviewers will say “no” because as soon as one grant is funded, another renewal or new grant needs rewritten.
The Early Career Reviewer program is a great idea. Experience on study section not only provides a service to the scientific community, but valuable experience for one’s own grant writing. I agree with the other points made though. Funding is so poor right now that one feels anxious about giving up grant writing time for study section. In addition, it’s very depressing to review excellent grants that won’t get funded and watch careers end before your eyes. The new biosketch format is really salt in the wound. I have not seen a single positive comment on any of the blogs, and we all agree that we don’t think the extra time is worth it for reviewer or applicant. If the NIH would give the scientific community the respect of listening to us, I suspect a lot of us would feel better about serving on study section.
I think there are more important questions to ask and a need for substantial disruption in the NIH model. For, example why are reviewers unpaid, NIH staff on salary, and PIs submitting grants that are only funded at 15% per round? Why does a grant review process take 8 months from submission to Council meeting? Why does all the NIH money go almost all to more basic science research and not to creative applied research to translate the enormous amount of knowledge we already have into practical solutions ? And why does the NIH only get $30B in funding when we spend $3Trillion annually on healtcare, 85% of chronic disease management largely for preventable complications. Why is everyone so happy with system?
I have never been asked to be on a study sectionr and yet have had several NIH grants. No idea how reviewers are chosen
I have received 3 Phase I STTRs and although I have volunteered to review (at the advice of one of my program officers) I have not been selected. I found it surprising that such a large percentage of STTR/SBIR reviewers are academics rather than small businesses. It would be wonderful to see more transparency around how reviewers are selected.
I have evolved recently from being at NIH funded clinical researcher at a hospital system to a start up company founder based on my work. Interesting to note that the SBIR/STTR grant mechanism is considered in the entrepreneur community as an extremely slow process with very low pay lines that would be nice to have but totally unrealistic for a fast moving healthcare start up company that needs fast money from Angels or Accelerator programs to build the company and execute a practical business model based on real customers. Small start ups don’t have 2-3 years to work through a couple of NIH grant review cycles and invest a huge amount of time for a 15% probability of success each round. As noted in the earlier post, if there were business people in the SBIR/STTR leadership and committees these issues would be obvious. We need a two month turn around for SBIR/STTRs and balancing of academic and business people at the NIH end. How can academic research leadership with no business experience manage and run SBIR/STTR programs?.
Please include more reviewers from small and large biotech or biopharma companies for related applications. Eliminate the PIs (both academic and companies) who are perpetual grant writers and some how convince the reviewers to fund them esp in area where we are trying to find treatments for urgent problems
Encourage new PIs to apply and discourage those who got 5 grants and have nothing to show in terms of development
I reviewed grants where PI are manipulating system by using data obtained with one set of bacteria and apply to another species (RFP was for this species) and call it NOVEL. Some of them received more than $10 million dollars in past 5 years and have not shown any progress to developing anything from these funds.
There are many new PIs who may no have stellar resume, but have great new ideas, we should fund such projects esp if these are in translational research area.
The term for each regular member of study section is 4-5 years, which is too long. I suggest that 1-2 years is better for each term. Let every NIH-funded PIs make turns. Do not always call those senior people for reviewing grant proposal. Give junior faculty a chance! In addition, I found that some study section members are not active and productive in science. I do not know what criteria NIH uses for selecting study section members, based on politics, social circle, or ?
I agree 4-5 years is a long time for reviewers. it promotes nepotism and you fund me and I fund you” mentality. Additionally, if a set of reviewers don’t like your idea or are biased you do not have much of a choice. I have a number of grants assigned to study sections not of my liking.
No one is asking NIH staff to work for free; why should staff ask the same of grantees? Other agencies (e.g., IES) manage just fine by actually paying reviewers an amount that comes close to compensating them for their time. “Requiring” grantees to serve as reviewers is really just a hidden tax (on reviewers’ time, or on their universities) to make the review section of the NIH budget look more trim. If you really want to require this time, there is NO reason not to state this explicitly, AND build in extra time and money into grantees budgets to account for this. Making already overburdened grantees feel guilty for not volunteering so as to make the line for ‘review’ in theNIH budget look trimmer reflects a kind of entitlement that tends to seep into the attitudes of those charged with distributing others’ money who come to believe they are doing a ‘favor’ to recipients by doing so.
Make sure and categorize some reviewer whose score is often OUTSHINE to others in same study section [rev. 1 and 3 average 2 and 2/3 scores while # 2 is 7 and 6]. Such issue will not lead to a fundable score for a excellent application.
I volunteered a while ago to do grant reviews and never heard back. I was an R01-funded PI at the time. It might be educational to me to find out why study sections keep giving me unreasonable reviews, and spread the love, so to speak. Even not being a funded PI, I’m a funded team scientist, and last time I checked, that was a big focus of translational science, so I might actually have something to offer.
I agree with:
– limiting # of applications: It takes me 3-6 hours to read and review and application. Multiple that by an average of 6 applications and you are talking one full week with travel and review.
-translation: We need to screen our really low quality applications and perhaps those that appear to have limited translation. I have read many apps that either lacked any direct path to intervention or were yet another proposal from a well-funded researcher whose work over a decade or more yielded no tangible effects on human health or the healthy system.
You are right. One of the problems is that NIH does not require to define the task of the application. In part this explains the inability of peer review to discriminate between innovation and novelty: innovation creates new value, while novelty creates only amusement. To create new value, innovation has to be in tune with strategic realities, which are political, social (e.g., increase of longevity), but above all – economical (bioeconomy, health care). Moreover, NIH should distinguish between continuation and breakthrough research programs. Both are important, but the former ones bear on predictable improvements while the latter ones court risks and therefore have to be managed differently, beginning with the initial merit review. They may require panel reviews, while online reviews are sufficient for the “continuation” grants.
This is absolutely correct!
All the comments above are valid. The compensation for time (1 day a year? how is that even possible?), the biosketch (let’s put that to rest!), the old friends favoring the old friends, the inability to move forward careers because in that study session that your grant went Dr. Big Guy and Dr. Infalible have their RO1 up for the 5th renewal and there will be only an average of 2RO1 funded/study session anyway… All of that is and will continue to be true…
Yet, who is monitoring that the (not so new anymore) review criteria is observed? when are we going to hold reviewers responsible for writing, for instance: “Environment: Strengths: Excellent, no concerns. Weaknesses: None… Final score for environment: 9!” That is in writing and can be monitored… and should be punished! Those people are serving in the study session over and over again and keep doing this over and over again. With total impunity. Nobody is held responsible. No the SRO, not the Chair of the study session, not the reviewers themselves. The process is infinitely skewed and flawed. And don’t be a minority or work at a minority institution or have a name that sounds as that of a woman… now, on top, we add the waste of additional time with the biosketch to the forms to fill ad infinitum and… science?… where is the science?… where is the good science?… who has even time for science any more?
Pressure on investigators is increasing daily. Academic institutions face significant budget challenges and require more service of faculty members to meet academic and clinical missions. Release time for service outside of the institution is difficult to obtain when there is pressure to earn your salary at home.
The NIH needs to identify creative ways to support the peer-review process. The NIH notice NOT-OD-15-035 does not address the climate that biomedical researchers are facing or acknowledge the difficulties PIs may have in fulfilling the NIH expectation for service.
The NIH could request input from PIs on how to lessen the burdens of peer-review. It is also important to acknowledge that one day of service is really much more than that. Some ideas to lessen the burden of peer-revue include: 1) more opportunities to review on the west coast. For those of us in western states, one day of NIH service equals 3 days of travel which can disrupt an entire work week; 2) NIH could create methods of recognizing PI service that could be used for promotion and tenure at home institutions; 3) The NIH could avoid placing undue tax burden on those who review; 4) The NIH could explore a processes that could reduce the number of applications that go for full review; 5) The NIH could build institutional peer-review obligations into the indirect cost calculations. I am sure there are many other ideas that could be generated by the PI community.
The NIH grant review system is largely ineffective in its present form. Dr. Scarpa’s “reform” in the Center of Scientific Review has failed not because, in his words, “changing an organization that resists change is quite difficult” – it is well known how to overcome resistance to change since late 1990ies. Rather it was based on unrealistic expectations because emphasis was made on reorganization and change in scripting that was confused with action. This is a common trap for any management: Typically when a process no longer produces results and should be abandoned or changed radically, management “reorganizes”. But by itself reorganization is no substitute for action.
Today the NIH review system is in the hands of scientific laity, this is, experts in other areas of research. The major flaw of the peer review of grants and contracts is the widespread belief that being “bright” is a substitute for knowing and “bright” people may adequately review almost anything. However, this is absolutely untrue, and for the sake of scientific integrity we have to abandon this false belief. The mean of production for the scientists is their knowledge, not fantasies. The feedback analysis shows that the main reason for poor performance of NIH reviewers and advisors is the result of simply not knowing enough, or the result of being contemptuous of knowledge outside their own specialty. This feeds intellectual arrogance that causes disabling ignorance, negatively affects NIH performance and all too often condemns NIH awards to mediocre research programs.
The quality and responsibility of advisers and reviewers for their work should be in the center of CSR’s concern through the implementation of anti-bias policies and rigorous and continuous feedback analysis. The review process should be carried out under strict policy that only the proven experts in the area of the grant’s research are involved. Any problem with lack of objectivity are made up by the volume of responses. In appraising the performance of a reviewer or advisor, the question of how many reviews s/he can write is quite secondary to the question of how many reviews are valid and reliable.
Dr. Nakamura,
Thank you for this post. I currently sit on a standing study section and for the most part, enjoy my service. Several suggestions come to mind to make peer review easier:
1) Dump the conference/web reviews. They are unfair to applicants and stifle conversation. In-person review is optimal and fair.
2) Move human subjects/responsible conduct/resource sharing, etc. out of peer review and let that be addressed pre-award. I should focus on science.
3) I agree with the other recommendations re: refreshments. It’s very challenging to do this work without basic refreshments.
Kind Regards, Chris Friese, University of Michigan
I have done a tremendous amount of reviewing. It takes me at least a full work day (8 hours) to review a grant. If I get 12 grants to review, that is 12 work days. Then there are three days to fly to Washington from LA and back for the committee meeting. That’s 15 work days gone. If i do that three times a year, that’s 45 work days or a month and a half each year for reviewing grants. And this is honestly a minimal estimate, its probably more.
My conclusion? You will get more reviewers if you reduce the burden.
Grant reviews must be done in person with face to face conversations. Internet reviews are a disservice to both the reviewer and the applicant. The grant does not get adequate discussion, and the reviewer does not get the advantage of face to face interaction with peers and networking advantage.
I am serving as an NIH reviewer for about 15 years sitting on at least 5 review panels per year, and I am not happy with the quality of reviews. In fact, NIH reviewers is the only group of US workers who have NO quality controls of their performance. In my experience about 5-10% of reviews are of poor quality containing either factual errors, or demonstrating lack of reviewer’s expertise in the scientific field of application. One such unfair review may put an excellent application in non-fundable range, so the overall impact is more than 5-10%. Presently, there is no scoring system for reviewers. I suggest creating such system by expanding the existing NIH reviewers’ database to include some kind of scores from (i) program directors; (ii) SROs, (iii) co-reviewers, i.e. two other reviewers of the same application. This will help SROs to select the best scored reviewers to their panels and also further train and instruct reviewers with poor scores.
I personally prefer internet discussions because they give you time to look at discussed applications where I am not a primary reviewer, look at publications, etc. I cannot do this within limited time during face-to-face review session.
For those of us with continuous submission, it appears that you have changed the rules. It used to be that the grants had to be reviewed within 120 days. Many of us planned on this and timed our applications accordingly.
For this year, it appears that the 120 day rule has been abolished. My recent grant went over 150 days and a colleague of mine is close to 7 months. There is no notification of this in any of the voluminous NIH literature that I can find.
Perhaps it is not the best policy to change the rules on people who are doing the bulk of reviewing for you?
We last updated our continuous submission policy in 2014. The policy notice reminds the community that the policy applies to R01, R21 and R34 applications, and it provides a table showing application submission date ranges along with the council round to which the applications will be assigned. The notice also notes that following initial peer review, if timing permits, applications may be considered in an earlier council meeting. If you are concerned that your submission was not handled appropriately, we encourage you to reach out to the scientific review officer assigned to your grant application (PIs can find that information in their eRA Commons account.)
In that policy announcement, there is no mention that the 120 day rule no longer applies. This is in contrast to NOT-OD-11-093 which clearly states: “Applications submitted under the continuous submission option must include a cover letter that states the eligibility of a PD/PI for continuous submission. This cover letter may also include a request for an institute/center assignment. These applications will be reviewed no later than 120 days after receipt. Applications will be initially assigned to the most appropriate Council meeting following review. Applications may be moved to an earlier Council if timing permits.”
In none of the followup policy changes that I can find is it every stated that the 120 day rule no longer applies.