A Grant Submission New Year’s Resolution

Posted

On December 29, 2014, two days before New Year’s Day, Reuters posted a headline, “New Year’s resolutions may be more procrastination than motivation.” Dorene Internicola, who wrote the accompanying story, cited a study from the University of Scranton which found that “While about 45 percent of Americans make New Year’s resolutions, only about 8 percent succeed.” Among the most common American New Year’s resolutions is “Getting Organized,” which by some interpretations might mean avoiding procrastination. One recent posting suggests that “to stop procrastinating” is “the only New Year Resolution you should set.” Procrastination is highly prevalent. In the words of Scarlett O’Hara, “I can’t think about that right now. If I do, I’ll go crazy. I’ll think about that tomorrow.”

Procrastination is actually related to task avoidance, which is a symptom of anxiety and depression. Since grant applications are anxiety producing, it is only natural that people would avoid finishing them. But hard facts about how applications fare in peer review when they are submitted closer to the receipt date may give applicants the cognitive tools they need to overcome their inclination to delay.

I particularly like the abstract from this paper:

Strong and consistent predictors of procrastination were task aversiveness, task delay, self-efficacy, and impulsiveness, as well as conscientiousness and its facets of self-control, distractibility, organization, and achievement motivation. These effects prove consistent with temporal motivation theory, an integrative hybrid of expectancy theory and hyperbolic discounting. Continued research into procrastination should not be delayed, especially because its prevalence appears to be growing.

One might think that scientists and the institutions might be different, particularly when it comes to submitting grant applications. The stakes are high – should a submission with errors come in too late, it may be too late to fix. Months of work are lost. Scientists, being highly organized people who spend countless hours designing experiments and study designs, surely must be the kind of people who would not put off submitting grants until the last possible day. So we might hypothesize…

Let’s look at the data.

Graph showing the increase of R01 applications received each day leading up to the 10/06/2014 application date
Figure 1: Number of R01 applications received by NIH each day leading up to the Oct. 6, 2014 due date
Graph showing the increase of SBIR applications received each day leading up to the 12/05/2014 application date
Figure 2: Number of Small Business Innovation Research (SBIR) applications received by NIH each day leading up to the Dec. 5, 2014 due date

Figure 1 shows the association of date with number of R01 applications received by NIH for consideration in the May 2015 council round, the latest round for which complete award data are available. The primary R01 receipt date for new, unrevised applications was October 6, 2014. NIH received 2,788 applications through October 5, the day before the deadline; an additional 2068, or 43% of the total, came in on the October 6 due date. Even among the applications received before the deadline, the vast majority came in the week before. It may be worth noting that it is an institutional official, usually in the office of sponsored programs that is responsible for submit the applications. So PIs may have completed their application earlier, but may not have built in enough time for the authorized organizational official to process and submit the application earlier.

Perhaps submission patterns are more timely for small business applications. Figure 2 shows corresponding data for Small Business Innovation Research (SBIR) applications being considered for the May 2015 Council Round. The primary due date was December 5. NIH received 794 applications through December 4; an additional 843, or 51% of the total, came in on the due date of December 5. And again, NIH saw little submission activity until ~1 week prior to the due date.

Is there any advantage to waiting? Do the extra time, and the stress imposed by pressing deadlines lead to higher quality work. Do applications submitted on the last possible day fare better?

Table 1: R01 outcomes according to date of submission

When ReceivedNumberDiscussedAwarded
Through October 527881319 (47%)240 (9%)
On October 62068880 (43%)186 (9%)

Table 2: SBIR outcomes according to date of submission

When Received Number Discussed Awarded
Through December 4 794 436 (55%) 113 (14%)
On December 5 843 433 (51%) 123 (15%)

Table 1 shows the results for the R01 applications. The later applications – that is those received on October 6 – were less likely to be discussed (odds ratio 0.26, 95% CI 0.23 to 0.30, P < 0.0001 [Correction] odds ratio 0.83, 95% CI 0.74 to 0.93, P = 0.001) and similarly likely to be awarded (odds ratio 1.05, 95% CI 0.86 to 1.28, P = 0.6). These findings seem to suggest no clear advantage for waiting until the last day; if anything, at least for R01 applicants, there may be a disadvantage in that the last-day submissions were less likely to make it to discussion. Table 2 shows the results for SBIR applications. The later applications were similarly likely to be discussed (odds ratio 0.86, 95% CI 0.71 to 1.05, P=0.15) and to be awarded (odds ratio 1.03, 95% I 0.78 to 1.36, P=0.8).

But – there are many possible confounders.  What about revised applications, which have later receipt dates?  What about new investigators?  Does it matter whether the institution submits applications using a system-to-system interface?  How about the local requirements of institutional business offices?  Basic science and applied science?  These considerations, and many others, may be worth further analysis … but I’ll think about that tomorrow.

Happy New Year! And special thanks to the eRA team for their help in putting this data together.

9 Comments

  1. The idea is to get on a study section so as to be able to submit at any time. The list is quite impressive.

  2. The odds ratio reported for RO1 awards being discussed (43% vs 47%) based on the data in Table 1 appears to be miscalculated. You report an odds ratio 0.26, 95% CI 0.23 to 0.30, P < 0.0001. I calculate odds ratio of 0.825, 95% CI 0.7357 – 0.9251). Otherwise, thank you for an interesting article.

    1. Thank you very much for noting the error! We confirm that your number is correct and the blog has been updated to reflect this. Thank you!

  3. Your asymptotic chart is dramatic — but not surprising.
    I would love to see it broken down by hour on the last day.
    …and by minute on the last hour!

  4. As the “institutional official, usually in the office of sponsored programs that is responsible for submit the applications,” I would kiss the ground if more of my PIs “completed their application earlier.” It’s quite stressful when multiple proposals are being submitted on the last two days. My pet peeve is people who revise their bio sketches at the 11th hour.

  5. It is important to consider the role of sponsored project offices (SPOs) in creating the observed distribution. PIs do not submit grants to NIH, SPOs do, and many SPOs have adopted a just-in-time model. In this model, regardless of when a PI submits a grant to the SPO, the SPO does not submit it to the funding agency until just before the deadline. This may contribute as much to the sharp peak at deadline as hypothetical “procrastinating PIs.” Imagine plotting when passengers board airline flights—all the passengers will fall in a short window, regardless of how early they arrived at the airport, due to the fact that airlines do not begin the boarding process until a half-hour before departure.

  6. I have to respectfully disagree that it is the SPO’s fault. As a scientist myself, I have been on the side of procrastination, revising and revising. In my role now as a research administrator, I have that unique understanding. However, where I come from, it is always the PI that gets the proposal to me at the very last possible minute. It takes a few “not accepteds” for some PIs to get the message that the cost of last minute submissions is not worth the risk of wasting everyone’s time and effort. And, of course, we must often deal with more than one last minute submissions at the same time! Luckily, I thrive on the adrenaline rush but can’t always work miracles. I appreciate the article, although it certainly will not disway a PI from waiting until the last minute…

  7. As a grants administrator, I’ve had PIs submit in accordance with University policy 3-day or 5-day rule, only to have the SPO wait until the day of. But, as anyone who works in grants knows, it is really most often the PI revising and revising and revising and revising one more time on the deadline day. We alllllll know that!

  8. By splitting into due date vs. everything before, that may wash out the effect of all the better preparers who are mixed in with the huge number of people who submitted 1 day before the deadline. Consider a plot of the funding rate vs. the decile chronologically. With ~470 R01s in each decile, it should be possible to show significance of differences and the trend (if any).

Before submitting your comment, please review our blog comment policies.

Leave a Reply

Your email address will not be published. Required fields are marked *