82 Comments
We certainly heard a lot of input on the blog on the issue of the new biosketch format. I really appreciate the dialog. Even when the input is critical it is so important to hear what you think. Remember that the blog is just one of many points of input when we make policy decisions, however. In this case NIH continues to believe that the new biosketch format will allow researchers to paint a more complete picture of their scholarly work which may help move peer review beyond an enumeration of reports in prestigious journals. We did adjust the policy to address the issue of timing in response to your input. We heard how challenging it might be to convert to the new format by January deadlines, especially for those large multi-component and training grant applications due January 25. Accordingly, we will allow researchers an additional four months to make the transition to the new format. Until due dates of May 25, researchers are encouraged to use the new format but will be allowed to submit applications using either the old or the new biosketch format as described at NOT-OD-15-032. Applications coming in for due dates on or after May 25 will require the new format. This timeline change also allows our National Library of Medicine to make all the adjustments necessary to the Science Experts Network Curriculum Vitae (SciENcv) tool to gracefully accommodate the new biosketch format and address a few technical issues by the end of December. Thanks again to all of you for the lively comments.
What the new biosketch format will precisely promote is the trumpeting of reports in glamorous journals, along with other subjective forms of bragging and self promotion.
You will just make the biosketch useless in the eyes of responsible reviewers. Dismaying, really.
A serious question for NIH: has anyone checked in with Hannah Valantine, M.D., NIH’s first Chief Officer for Scientific Workforce Diversity? I believe there are research studies addressing whether certain groups (women, under-represented ethnicities) are equally adept at self-promoting narratives, which the new format encourages (at least lists of papers are objective). I think Dr. Valantine should sign off on the changes before they are implemented – has this occurred?
Lists of papers are a “hard” metric, meaning it is something that has established weight and can be counted. Are the papers in good/bad journals, are they long/short, do the titles reveal incremental progress or significant advancement? The more hard metrics that are used in an evaluation, the stronger the evaluation of a woman or minority candidate. Personal statements are a “soft” metric, more easily biased and of lesser value. Numerous studies have shown sexual and racial bias in evaluating soft metrics. The same paper with a man’s name at the top is reviewed more highly than when a woman’s name is at the top. (Unfortunately, it happens with a list of hard metrics too, but to less of a degree.)
There are many people for whom I don’t read the biosketch because I know their work, good or bad, and its impact in the field. I hope that my reviewers are chosen well enough that they know my field and don’t have to rely on my biosketch. I feel sorry for new investigators and people who are more likely to be marginalized. I fear this will be one more small strike against their success.
Completely dismaying.
A poor answer from NIH. Delaying implementation is not a response to the criticism leveled here. As stated in the above post from CDO, this will just lead to people writing puff pieces about their publications in high impact journals. As a reviewer, this will be of no help, as 50% of reviewers noted in the NIH survey. It took 4 years for NIH to reverse the 2 resubmission limit rule. Do we have to suffer this nonsense for the same time period?
Indeed. A poor answer from an imperial NIH administration that has lost contact not only with research but also with researchers.
Reviewers already give little weight to the bloviating in the current Biosketch personal statement section, and tend to just use PubMed and Project Reporter to draw their own conclusions about the qualifications of the PI and the influence those qualifications have on the predicted impact of the proposed studies. Including even more self-interested bloviating will simply reinforce the uselessness of the Biosketch as a tool for reviewer assessment of PI qualifications.
Correct. And obvious to any active researcher, grant writer, or grant reviewer.
But apparently not obvious to clueless, unifmormed, non-responsive research administrators
There are a few lingering questions that I hope the NIH and Dr. Rockey can address. This post reports an interest in the “dialog” which I think requires input from both directions. So, I will ask for a response to the substantive critiques that were leveled in the previous comments section. Specifically:
1. What are the data regarding the impact on gender bias in investigator (and overall) scoring using the new format (h/t Sponsored Programs)?
2. Was the direct and relevant question posed in the pilot, namely “To what extent is the new format an improvement over the original format (h/t CD0/Aging PI/others)?” If not, will the ongoing pilot incorporate this question?
3. What are the data the NIH is using to bolster its continued belief in this value of this change?
4. How does the NIH find that the stated goal above of moving “peer review beyond an enumeration of reports in prestigious journals” is achieved using a new format that fosters relating the “magnitude and significance of their scientific contributions (including publications)”?
5. Will the NIH commit to rigorous collection of comprehensive data on investigator and overall scores using the new form, specifically categorized by gender, race, New/ESI, established investigators, and IDEA-state investigators? This data, along with the corresponding control data from applications using the previous biosketch format should be made accessible to the scientific public.
I know I will have missed some valuable and substantive unresolved questions, but this is a start.
I was an applicant for NIH grants starting in 2008 as a fellow and later as a faculty in a university until earlier this year. The application process is being tweaked almost on continuous basis from twenty five-page application to twelve-page application and other associated changes. I understand that it is important to evolve in this digital age and make the process as simple as possible. However, it seems it is getting complicated every year with additional information needed at the time of application such as budget, statistical analysis, vertebrate animal section and etc. I believe applications should be judged based on merit and feasibility. Additional information can be collected if an application gets fundable score. It is possible to generate an application required for review using simple software such as MS office without the need for generating several documents and trying to combine into one document using the online system. I failed to understand the logic behind it. I was made to re-take a scientific ethics course as part of career development plan supported a NIH. It is interesting to observe how the whole process is changing.
When I sent the notice of the new requirement on to my colleagues as an FYI, there was collective outrage to the tune of “they are changing it AGAIN??”. This change is not making anything as simple as possible.
What are the other “many points of input”? What was the nature of this “input”?
I’m so discouraged by the NIH response. At no point has anyone at the NIH indicated how/why this extra “investment” of time and energy will substantively help the grant review process. In contrast, the VAST majority of stakeholders have indicated why this is a bad modification to an already difficult process. This policy change is so unbelievably stupid and misguided, I cannot believe it is coming from a scientific agency. It feels, however, on par with expectations for an agency of bean counters and middle managers – well done, NIH.
Complete and utter rubbish. No one asked for a delay, everyone clearly asked for this disastrous plan to be aborted, full stop. I have no more words.
I don’t understand the need for this change. The current format gives investigators ample space “to describe the magnitude and significance of their scientific contributions”. The new format adds an additional burden on grant writers, and it adds another burden on grant reviewers who will now need to read a longer biosketch with extended commentary and still look up the entire publication list of the applicant anyway. Further, I believe, it will encourage people to oversell their accomplishments, which could backfire during the application review process. .
“it adds another burden on grant reviewers who will now need to read a longer biosketch with extended commentary and still look up the entire publication list of the applicant anyway”
Reviewers will simply stop reading the Biosketch at all, and will rely solely on Pubmed and Project Reporter to assess PI qualifications.
Speaking as a reviewer: Exactly right!
Exactly correct
In the new Biosketch format, a PI is given 5 major contributions to describe. For me, each will represent about 10 full years of my life’s focussed effort, and I will not have five. Being a scientist, each will be one of my ‘selective internal syntheses’ to understand a phenomena or solve an important problem. Indeed, my syntheses were never done in a vacuum, but they were original syntheses and there were competitors, detractors, collaborators, followers and the expected tribulations for a major contribution or new field. Original thought is unique to an individual’s background experience, suppositions, inspiration and serendipity. That original thought contradicted the current understanding-there were detractors. Often, there was an observation that elicited the thought. Other times, a discovery was anticipated by the the thought/syntheses. Each of my reported syntheses will be the ‘whole salami’ not slices, like an ‘enumeration of publications’ is. I will start with my PhD thesis, like an ESI, or anyone should. Your thesis is your scientific foundation which by definition should represent a worthwhile and lasting contribution. For an MD, it would be the results of early mentored training- on understanding and treating a disease. Any major grant I had, or have, will hopefully be in there. Otherwise, how would I justify my past funding? Of course, the whole original idea may have been a flop. Too many or too big and expensive–that will, and should be, a future grant killer regardless of an efficient salami-slicing paper mill. Then, of course, there are the service contributions that are not that original, but nonetheless, keep science moving slowly forward or in circles, as is often the case–till the skillful breakthrough by the original scientist.
It seems to me that this view is opposed to the principle that has successfully supported the research enterprise in the US for decades; namely that we fund novel and relevant ideas based on the current capacity of scientists to develop them.
We are not used to support individuals based on what they did 50 years ago. This is the prevailing system in Europe, where old timers and endless sagas of relatives prevent the access of fresh blood into the system.
This is precisely the problem that I have with the new format and I thank you for stating it in a very eloquent manner.
If the Biosketch were the only criteria for funding, I would agree with you. However, there are so many elements to the grants including the proposed experiments. Time is the best judge of the impact and reliability of work and an investigator. Most investigators produce their best work at a younger age and continue with derivatives, however, some continue to produce significant work and solve important problems–and inspire younger scientists. The latter are the great scientists, and I think the NIH needs them. Solving an important problem every 10 years should be the goal.
CD0, I now see where we starkly differ. It is in your statement “that we fund novel and relevant ideas based on the current capacity of scientists to develop them.” You have used the plural ‘scientists’ rather than the singular ‘scientist’. While this decade old system may profit a field, society, and competitive scientists in the short term, it has probably killed the careers of many original young scientists in the increasingly hyper-competitive environment. As it is now, having the factory and equipment for getting work done and publishing papers is more important for judging the ‘Biosketch’ than the originality or proven skill of the investigator in solving important problems. Remember that those who have one good idea will have another–but they may have no funding or career in the current system. Hyper-competitiveness kills the goose that is able to lay the future golden eggs.
Citin-scientist,
Emphasizing the contributions that somebody made 30-40 years earlier, even if the interested part makes an effort of objectivity, does little in my opinion for these new researchers you are talking about. In that respect, we certainly have a major disagreement.
I do not know how many researchers have been productive for decades, but I can see that even if this is the group of people that these changes aim to protect, there will be many more investigators with potentially very high productivity that would suffer from this.
As I commented above, and based on my experience as a re viewer, what will will happen is that the NIH has effectively killed the bio sketch as a a valuable document to evaluate applicants. Nobody will care anymore.
In any case, thanks for a respectful interchange
Please reconsider implementation of this. This is a terrible idea.
If NIH were truly concerned about reducing the emphasis on publications, reviewers would be blinded to the identities of the investigators during review of the science. If NIH doesn’t care about the feedback from investigators and reviewers (i.e., I haven’t read a single comment in support of this change), whose opinions make up the “many points of input” on which they are basing this decision? This doesn’t add up to me.
“We certainly heard a lot of input on the blog on the issue of the new biosketch format. I really appreciate the dialog. Even when the input is critical it is so important to hear what you think.” – Dr. Rockey
Based on the feedback generated here, I think NIH has an obligation to revisit this decision. All the comments on this blog argue for the current format being sufficient and preferable to the new version.
CD0, I appreciate this meeting of minds also. What I support and value is new and durable knowledge obtained by young, old, all inclusive. In my estimation, the new Biosketch will provide a valuable tool to bolster the most valuable research. There is truly infinite knowledge with potential for millions of fields of specialty. 30 years ago a well-known scientist predicted that the growth of knowledge with its estimated doubling every 17 years since Newton will choke itself off. Do the math in terms of funding requirements and you will see why. Also, some knowledge is redundant, easily surpassed or worse, as you are clearly aware. A Biosketch that focusses on significant achievements and captures the essence of an investigator and his/her past investigations is of great value. For an ESI, it motivates and rewards significant goals and achievements rather than numbers of publications in the current ‘publish or perish model’ or ‘enumerating and measuring impact model’. Of course, publication #s, public talks, private talks, training of students etc. are important to think about, but it is the essence of the work and contribution of the investigator that is most important to evaluate. For the NIH, it can achieve and sustain what the public asks for at minimal expenditure. I agree, it will be difficult, or perhaps impossible, for many competitive scientists to adapt. It is a different style of science – not a lot of fat, but definitely worth embracing and teaching.
Huh?
NIH is not listening to the scientists because it’s not run by scientists, it’s run by administrators who left science a long time ago. Look at the peer review system: you go into a study section and you are allowed to vote a score on a grant you have not read. The best science is buried in the noise. These bureaucrats don’t know how to make the best use of the little money there is, and then they blame the lack of money. When there was abundance, nobody noticed they were incompetents.
This new biosketch requirement could discourage interdisciplinary research–and favor the inclusion of more senior collaborators. When I submit grants with others from fields such as engineering, computer science, etc who have little experience with NIH, I often am the one pulling together these biosketches, as the engineers, etc don’t have the time/patience for this and just hand me their CV. Pulling together detailed biosketches in this new format for co-investigators in fields I do not fully understand (which is why the co-investigator in engineering, etc is being added to the grant) is going to be very challenging and time-consuming. It also means that I will be more likely to include senior engineers/computer scientists, etc. who might already have these biosketches prepared, or work repeatedly with people for whom I have prepared biosketches in the past, thus saving me hours of work. There is nothing wrong with the current biosketch format. The goal should be to simplify the administrative workload so that team selection is based on choosing the best scientific collaborators, rather than collaborators with pre-prepared biosketches.
I agree with all the negative comments. IF reviewers pay any attention, it will only benefit those who are skilled at self-promotion or have track records that are already evident. It seems just another bureaucratic change we must comply with, rather than allowing us to focus on the important stuff – science.
I am a regular NIH reviewer. I will now pay virtually no attention to the biosketch and be forced to look elsewhere for information about the applicant. The extreme self-promotion makes the biosketch useless. I will have to see for myself what they have published. This is an extra burden on me as a reviewer. It is a waste of time for me as an applicant. This is a misguided change. I can’t believe any behavioral scientists were consulted about this change.
There must be a backstory to this that the NIH is not telling us. If the review branch were doing its job, people reviewing the grant would know the topic well enough to understand the contributions of the applicant without having to depend on personal hyperbole. Something about the signal and the noise.
This process is an absolute disaster and the NIH should be ashamed of itself. It took me 4 hours on New Years Day to put together a new Biosketch (which I opened by criticizing the need for it, and emphasizing how obnoxious the requirement was – bite me reviewers!). Now, I find that the Online CV “tool” is horrendously complicated and will take MANY, MANY hours of effort to navigate and make functional. It’s TOTALLY wasted time. Heads should roll here. And at quite a high level…
I wish there was somewhere to vote on this. As both an experienced reviewer
and grant awardee the new format is silly. It makes reviewing grants more cumbersome, not less and I can’t help but feel that reviewers aren’t going to read it, and submitters are going to spend the amount of time on it it should get – not much..
I would bet 90% plus of the NIH external community would be opposed…
Over the years I have seen ever increasing burden of paperwork and additional requirements coming our of NIH CSR. The new requirement for CV is just another example. Why other federal agencies like NSF and DOE are still requiring 2 page CVs? Why not make the CV format simple and the same across the agencies or at least leave it the same as it was before? I do strongly believe that the applications should be judged based on merit and feasibility.
The new format is more appropriate for a self-nomination for a prize and would simply destruct me from producing the best possible application or review an NIH grant.
With current funding rates, NIH grant IS a prize.
Part of my job entails compiling all the administrative parts of the grants, especially program project or multi-investigator grants. My PIs don’t care about the biosketch. They just view it as burdensome admin paperwork. I have trouble getting them to select their top 15 publications for the application. Now I need to give them drafts of their 5 biggest contributions to science?
The only positive outcome of this new policy that I can see is that my grant writing skills may earn me a higher paycheck, as scientific communication and marketing becomes a valuable commodity.
Are we promoting dishonesty or blindness with this new requirement – How many of these blurbs will read accurately
My contribution: Well, not much – have maybe 2 papers I’m really proud of, the other 470 are really just what we are expected to do.
Maybe this will help young investigators feel they have a chance by enhancing their contributions – but contributions are a long term evaluation – are they important in the long run to science! Touting that you’ve had a NEJM article is a short term outcome – and that is visible in your bibliography. Saying you are a good mentor is not evaluable as a reviewer without breaching the confidential nature of reviews and truthfully that is for the beneficiaries of mentoring to judge – not self assessment, which confused process with outcome. Thus, this, like the personal statement is a very costly addition to the submission process when added to all the other requirements which have evolved, sucking the dollars out of our research system – all with good intentions, but without the cost/effectiveness justifications many other of our medical processes now require. NIH doesn’t have to pay for these requirements directly so these issues play a back seat to doing something that is visible. But alas, we all pay in the long run.
Had to look this one up:
“Literature is a luxury; fiction is a necessity.”
― G.K. Chesterton
As a frequent reviewer on study sections I refuse to consider this new format. I will only consider the following in the bio sketch: education, number of publications, quality of listed publications. In other words I will spend approximately 1 minute on the bio sketch and I am quite confident that this will give an accurate reflection for the score of the “Investigator”.
“In this case NIH continues to believe that…”
In other words, non-research-active administrators know more about research than researchers.
The proposed format is another “waste of time” effort for researchers who should justify using their time generating new ideas and spending time to think between publishing papers on the directions of their research. How is it possible for one scientist to list and justify their contribution among many authors (as is the case these days for high impact publications) and for reviewers to actually know what the contribution of the applicant really is in that manuscript? How is it possible for a communicating author to claim that the credit for that high impact paper which on most instances was thought, conceived and executed by postdocs? This funding system is flawed and doesn’t appear to have a future.
Since you obviously aren’t influenced by feedback, I shall not waste my time again.
I am an RO1 funded investigator of nearly >30 years, and currently serve on a study section, and I can find no redeeming value in the change to the Biosketch. The current format is quite adequate for the reviewer, and this change in the Biosketch is an unfunded mandate, which is simply wasting valuable time for PIs and administrative staff. Why don’t you simply call off this bad idea? Where are the data that this will improve anything? Who in the research community or study sections requested this change? It looks like someone at the NH has too much free time on their hands.
I agree it a terrible idea that will be a burden to applicants and reviewers. I do not understand the NIH need to change grant format so frequently. It is harder and harder to get applications out since PI have to do more and more of the grunt work.
Do not delay the implementation. STOP IT
My biggest concern about the new changes is that it will be much harder for early- or mid-career PIs to enlist busy senior colleagues as advisory collaborators to demonstrate a team with the kind of breadth and experience that reviewers appear to want. It used to be simply a matter of getting their advice, approval, and biosketch; then it was necessary for the senior colleague or their administrative assistant to put in a project-relevant personal statement and select project-relevant publications; and now it will be necessary for the senior colleague (and not their assistant) to spend a few hours tailoring their biosketch for the project. I imagine some will not have time to do that. And if the senior colleague has not had to prepare an NIH biosketch in the last year (e.g., they are in the middle of their three R01s), all but the most saintly ones will decline to be listed as collaborators if it means constructing this style of biosketch for the first time, even if they are happy to informally advise the PI, and the investigator score will suffer. Preparing it is too large of a burden for someone who isn’t heavily invested in the outcome of the application. (It took me hours to make my new biosketch, for one of several projects in the lab.)
Perhaps at the very least, this format could be only required of the grant application’s PI, and third party biosketches could have the option of being in the current or previous format. That would obviate a large part of the problem.
(Although I do think that most reviewers won’t bother to read the new format, as mentioned by other commentators.)
So, basically, you appreciate the input but you will ignore it. You heard “a lot” of input? I’ll say! It was overwhelmingly negative as well. Yet, you persist. That neither inspires confidence nor engenders respect for your processes.
I have yet to talk to anyone that believes this new policy to be a good idea – it seems like there is already an opportunity to address this issue in the Personal Statement. While I do agree that bean counting total papers published is a poor indication of one’s direct intellectual contribution to these papers, and that some reviewers may not be aware of the high quality of journals with which they are unfamiliar, all this does is make us try to become self-promoting science journalists – personally, I am uncomfortable with self aggrandizement
I fully agree with those criticizing the implementation of the new biosketch format. Why, exactly, is such a change needed? The arguments advanced by the NIH administration are, frankly, weak and unconvincing.
The primary purpose of the biosketch is to give grant reviewers background on the PI and key personnel. Why is a brief recap of one’s interests and accomplishments, together with a list of awards, publications, etc. (as is currently implemented) now deemed insufficient for this task when it has work so well for so long? This change smacks of rearranging the deck chairs, not fundamentally improving the system.
The new format is bloated and will likely be skimmed or ignored by a majority of reviewers. At the same time, it puts an onerous burden on those who have craft (and constantly update) their five different narratives and is patently unfair to young investigators who lack the multi-year track record necessary to “pad their resume”. When I want to judge whether the publications listed by a PI have made a significant contribution to the field (as a reviewer), I’ll simply go look them up. A danger with this system is that it will reward those PIs who excel at “dressing up” their findings in tidy sound-bites.
The grant writing process is already onerous enough. Ideally, the new format should be scapped and the biosketch layout left as is. However, given that the NIH almost never reverses itself on these decisions, why not adopt a compromise: if a PI feels that the new format better serves his/her interests, then they can switch to it. If not, allow them to stick with the old format instead.
From reading the comments after every article from Dr. Rockey about this issue, and talking to colleagues, collaborators, and study section members, it is clear that the vast majority of investigators think the new Biosketch will be burden on both the PIs and reviewers. Since CSR-NIH cannot force anyone to fill all 5 pages with the recommended/suggested self-promoting description of own’s achievements, I plan to use the new Biosketch form but fill it with the same content I used to use in the old form. In the section where my trumpeting self-promotion should be, I will merely list the papers I’m more proud of, or perhaps copy-paste the Abstract below the bibliographic citation. That way, I will save reviewers the time to go and check for themselves in PubMed. And since all reviewers commenting here mention they would do that, I expect they will not be upset from my take on the new Biosketch.
“I really appreciate the dialog.” – Dr. Rockey
Who said it is a dialog? Hundreds unanimously yell “No” to the ridiculous change, but nobody wants to hear. A question to NIH: What should we do to finally be heard?
The new Biosketch will provide another bias towards more senior and experienced investigators and will not serve more junior investigators. As someone who has been in the field for a long time, it is easy for me to name a number of substantive contributions I have now made. When I was just starting, that would not have been possible. If the NIH wishes to fund more senior investigators on past accomplishments, they should put in place a mechanism for that explicitly. I actually think the old, old biosketch which asked for “all papers published in the past 3 years and relevant earlier papers” was ideal. It gives the reviewer a chance to quickly see what the investigator has been up to….
Since this policy became news, I did not read single positive comment by anyone in the scientific community. So why does the NIH think this is still a good idea? It shows a complete disregard for the community that the NIH is supposed to be serving.
OK, it is constantly one of my “New Year’s resolutions” to stop posting in the Comments sections, because it requires some investment of time and thought, and then is ignored (at best) or deleted by moderators (pretty often). Still, I cannot help responding to this new development. I just cannot believe that this decision was done by people having any (at least distant) relation to science, as well as several other recent NIH policy changes. I remember the times when decisions concerning any field were done by professionals in that field; not anymore. It is also a sad testimony that marketing starts to trump everything else, even in such traditionally non-commercial fields as science, education and health care.
I actually liked the suggestion in the comment above. I would also say, let’s adopt a semi-free format: 1) allot the applicant 3-4 pages for the biosketch, 2) make it mandatory to list full name, education and the most recent 10-15 publications, and 3) allow to fill the rest with any information/reflections/whatever the applicant finds appropriate. This way, the reviewers will also have an easy way to tell serious people from fiction writers.
I have not looked in detail yet at the new Biosketch guidelines, but I am willing to give it a try and believe it might be a good idea.
Science is not farming, but is sure is treated as such in most study section panels in which progress is usually described as X papers in Journal A, Y papers in Journal B, etc., without any meaningful description of the work itself or the impact of this work on a field.
I believe that I am known in my field for only a small fraction of the total number of papers that I have published. These papers usually are highly cited. The remainder of my papers tend to be incremental steps leading to the major advances that come later.
Most of these important advances for which I am known, however, took years to accomplish through experiments that typically did not yield publishable results for 4-6 years. The current grant review process does not readily support such long-term research, but instead heavily emphasizes short-term, incremental work along an established line of investigation. To be honest, this led to a huge shift in research emphasis in my laboratory towards generating “papers” to provide funding, with the new research fueling my major advances typically being funded as side-projects. As I look back on my research career, I can’t help but wonder what I might have accomplished if this balance of research emphasis could somehow have been flipped around by changes to the NIH funding mechanisms.
I believe the current intent of the new Biosketch format is to recognize the most important work and effort that an investigator has achieved and to explain to reviewers that might be in different fields the importance of these achievements. I believe this is a worthy goal, as it may tip the balance towards rewarding long-term research of high impact.
Ultimately, it is up to the reviewers on study sections to make this shift in appraising grant applications. As a reviewer, I look forward to having this additional information to help in my understanding and appreciation of research outside of my immediate field. I am not so worried, as others seem to be, about how different groups of people might be better or worse at self-promotion. Sometimes the simplest summary- for instance the number of times a paper has been cited- might be the most effective, and I hope study section reviewers will be able to separate true impact from self-promotion.
In my last ten study section meetings I have not seen a single application with a fundable score that proposed incremental work. I think that the new reality is that the level of competition is so high that only truly transformative ideas with a “wow factor” get funded. That’s the problem of old-timers that succeeded in the doubling time, but it may be my field…
I would certainly agree with you that nowadays proposals in a fundable range are all quite impressive and strong. But both incremental and wow are still relative terms. Let me rephrase. I have seen “Super-wow” proposals get downgraded and scored below the “wow” proposals based on “productivity issues”. In previous eras, this did not usually matter as it was the difference between two fundable scores. Now, with the funding levels so tight, these finer distinctions matter.
On only one occasion can I remember a study section going beyond paper counting and discussing the actual impact of the publications of one PI versus the number and journal quality. In this case, the discussion essentially rescued the proposal of a PI with only two publications in the pervious cycle. The primary reviewer had described these two publications each as having had a major impact on their field, but then still initially gave a non-fundable score based entirely on the low paper count. It was only after the study section Chair pointed out that she had not heard such positive comments about paper impact from PIs of other proposals scoring higher that the conversation changed.
I agree with you on this point, but I do not think that the new biosketch will change that.
The new format appears to be designed to protect individuals who were productive in the past but are not necessarily productive anymore and are at risk of losing their long-term grants. Let’s remember that the system started at NCI…
Having tons of papers of course does not equate to making an impact, but I see in my study section service that many of these individuals are not creative or propose cutting-edge research anymore, after years of free lunch. I think that this is the reason why many of them are getting left behind, and rightly so.
I understand that the American system is brutal, but respectful of a principle that was not respected for me when I worked in Europe: Equal opportunity.
Congratulations on pushing forward with this plan, one that has as close to universal opposition as possible!! I am glad to see that NIH is so responsive (sarcasm) to the scientific community and the people it was created to serve. It is great to know that we can put all this effort into reformatting a biosketch that will have no impact on our potential for obtaining funding. Scientists are being pressured by their home institutions to cover significant percentages of their salary and face increasing and ever-changing grant and bureaucratic burdens from NIH. How about we judge a proposal for its scientific merit and limit the role that self-aggrandizement plays in the process. Press releases and overstating the importance of any one publication is the job of a university’s public relations office. Simply amazing that NIH would move forward with something so reviled by the scientific community.
As a reviewer, I can attest that only something really outrageous about an applicant would make me argue against a good application. Of course, a clever applicant would not be highlighting his problems anyway. Conversely, no prior achievement will convince me to give a good score to a bad application. Indeed, according to an NIH study into the matter, the “investigator” (as well as the “environment”) score has no meaningful correlation with the final score and, presumably, the funding decision. So, this whole aspect of grant preparation and review is a veritable big waste of time and money.
To those that think there is no logic in this (and many other “improvements” that the NIH has pushed in the recent years): you are looking at it from a wrong angle. This makes sense from the point of NIH staff: they can use the long list of their “innovations” and other busywork, as an indication of their activity and to justify the very existence of their jobs in times when there are not enough funds for actual research. In addition, the whole series of changes in recent years served to ensure that a cohort of well-connected “big lab” investigators remain well-funded as well.
This said, I do want to commend Dr. Rockey, who courageously opens up this blog to comments, which, predictably, are overwhelmingly negative. Sometimes I think that the people who operate this blog hate the current NIH system as much as the rest of us do. That is why they put the most outrageous follies of their organization up for public scolding.
An earlier comment mentioned that there must be something going on behind the scenes at NIH to drive this change, despite the vocal opposition. Another comment mentioned that the application has been continually changing for years now.
So, what will the next change be? This new Biosketch must facilitate it somehow… maybe personal information (name, university, etc.) will be removed from NIH applications?
Knowing who an applicant is and where they work is known to introduce bias in the review process. Without personal information, reviewers would be forced to rely on this new Biosketch to learn of the applicant’s previous accomplishments. This new, potential format would still be biased toward those who are better at self promotion, but perhaps anonymity would encourage everyone to toot their own horn?
Clearly, we can expect further changes to NIH grant applications. I’m interested in seeing how the new Biosketch will support those changes.
Is anyone in favor of this policy? If so, he/she should speak up. As it stands, I think that the support for the change is close to zero!
I entirely agree with the predominantly negative comments here regarding the new biosketch format. I do find it hard to imagine how such changes ever see the light of day.
As an applicant, spending even MORE time on non-science-related drivel just makes an already onerous application process more difficult and wasteful. As a reviewer, I can’t imagine how I would use this information. If applicants have published meaningful work in decent journals, this productivity speaks for itself. If applicants haven’t done that, no amount of self-promotion and spin is going to convince me that they are outstanding scientists, no matter how desperately important they think their contributions have been to whatever endeavors they care to cite. This is another useless, meaningless time-waster. (And, speaking as a reviewer, I don’t even find the current introductory personal statement to be of much value at all — and I don’t recall ever hearing another reviewer comment on this information). Enough. Stop it.
As a well-funded investigator and study section reviewer (now chair), I see no reason for this new biosketch format unless it is to add even more burden on already overworked scientists. This will not improve review, just slow down the pace of science.
I see several positive things about the new format. Ultimately I think it provides a better introduction of the investigators to the reviewers. Yes – it does take time to convert to the new format (took me several hours) and the myncbi site takes time to set up the list of publications, but there are advantages. For people with common names (especially if no middle initial) it is very difficult to figure out the publication record on PubMed. This new format will eliminate this problem. That is a huge advantage. Some commentators on the blog have said the new format will emphasize publications in high impact journals. I don’t think it will be any worse than the current format with a limit of 15 publications. In setting up my biosketch in the new format, I selected papers that were most highly cited, and not the ones in journals with higher impact factors. Finally, I think this new format will help reduce the problem of the “old boys club” where grant applications are much more likely to get a great score if they go to a study section where people know you and respect your work. I’ve had good success when my grant apps went to places where people knew me, and poor success when they went to places where they didn’t. Hopefully this new format will help counter this trend (but of course, only if the reviewers bother to look over the longer biosketch). One last point – in an earlier post, Dr. Rockey said this had been tested in study sections and was liked by applicants and reviewers. How large was this testing? How enthusiastic were the comments? If tested in a large enough group, and the results were highly encouraging, then don’t pay much attention to the mainly negative comments on this blog – go ahead with the roll out as planned. But if only small groups were tested, then I recommend further testing before full-scale roll out in May.
Lloyd,
Here is the link to the “data”:
http://nexus.od.nih.gov/all/2014/11/26/implementing-the-modified-nih-biosketch-format/
The numbers were very small, and as many of us might state as paper reviewers: “the results do not justify the conclusions”. None of the reviewers really thought the new format was improved, because this question wasn’t asked. Just “helpful” or “suitable”. The maximum % for any opt these was 55%, hardly a resounding endorsement from the 29 reviewers included.
I agree with this. If we look at actual reviewer numbers (n=29) rather than percentages (which are misleading in this case), 55% of 29 is ~16 reviewers, which means that ~13 reviewers either were neutral or negative. Two reviewers changing their minds to neutral or negative would completely change the conclusion here. This wouldn’t be publishable, so why is it being used to force us to make significant changes to the review of thousands of grants? Additionally, it’s impossible to get 50% from 29 reviewers, and rounding up or down with these low numbers drastically changes the apparent conclusions (52% = 15/29, 48% = 14/29). Therefore it seems that there has been some number massaging to get the data to look favorable.
Scientists that I interact with clearly think this change is detrimental. I personally don’t understand the purpose of the new format. It has long been my sense that NIH staff do not understand the perspectives of and the burdens on academic scientists. Every time they lay on a new layer of bureaucracy on to researchers, it’s an unfunded mandate. Each new issue may be small, but they do add up. I run several extramural NIH programs. In every case I’ve had to explain to NIH personnel the ramifications of each procedural or programmatic change that they place on the investigators.
NIH is under strict guidelines to be fair and equitable (a good thing). But NIH cannot accurately know the best approach to fairness and equity because they are not the ones in the trenches actually engaged in the time consuming “art” of peer review.
I planned not to comment again, since my previous comments and those of scores of fellow scientists were completely ignored. However, I can’t resist pointing out again that the NIH is failing the science community miserably on this issue. If you refuse consider, respond to, or even acknowledge our feedback, what is the purpose of the blog? Very disappointing.
As a full member study section reviewer, and full professor, I find the new biosketch format policies unhelpful and misguided. It discourages the early stage scientist PI who may not have made more than one or so “big” contributions, and results in a self promotion of the expeirenced scientist that i think is not nescessarily appropriate. It strongly discourages the careful disection of minutae of scientific investigation (often which turns out to be important) and promotes the lucky big-bang-thank-you-maam type of studies that we all hope for, but few acheive. A solid stepwise scientific analyses should be promoted. I think the new biosketch format is a grave mistake that makes the very difficult process of getting the funding for doing the science all the more harder. It is unnescessary and certainly does not appear to serve to make grant reviewing easier. I will likely pay less attention to biosketches and do the “whats been recently published” on pubmed searches. Thats what reveiwers of my grant applications seem to be doing.
I enocurage you to strongly reconsider this poorly justified decision.
Even Coke acknowledged they made a mistake with New Coke. Will the NIH do the same? Come on, bring back the Classic Biosketch! At the very least, make the new form OPTIONAL. Delaying by a council round will really not accomplish much. The NIH is changing way too many variables at once with review and submissions. How will we ever tell what works and what doesn’t with so many variables changing at once?
Obviously someone at the top thinks this is just a marvelous idea and the decision was made without much consultation with the true end users (reviewers and applicants). I would love to see the market research for this concept during the requirements generation phase. Who am I kidding, I doubt they even gave it a second thought. Left unsaid is the other party that might find this useful: the NIH staff themselves. Obviously someone thinks it will be useful, just not the applicants or the reviewers.
I’m on study section. I have NIH grants (R01) and I’m co-director of a training grant. The new biosketch is a miserable waste of time for both reviewers and applicants. No benefit.
Plus….
Imagine trying to get your collaborators/other significant contributors to spend 6 hours updating their biosketches for your grant (when they won’t need to update their biosketch until their own renewals are submitted two years from now)? All you need is a reagent and a letter of support/biosketch as proof, and now you need to convince them to waste 6 hours updating their biosketch early? Won’t happen.
And what about training grants? It’s already an administrative nightmare to collect biosketches/data from 30-40 training faculty– now they all need to provide updated biosketches ASAP? Won’t happen.
WILL SOMEONE (ANYONE!) AT NIH PLEASE LISTEN TO US AND STOP THIS MADNESS?
Slogan at the bottom of this page is “NIH… Turning Discovery Into Health”
It should read “NIH… Replacing Discovery with Needless Administrative Burden”
OK, I get it.
Someone in the bowels of NIH administration thinks this terribly misguided and wasteful initiative is a good idea. Maybe that person(s) really believes it is a good idea, or maybe that person(s) is just looking to implement some initiative – any initiative – to justify their job. I don’t know which.
But it is clear that NIH is not interested in listening to the nearly unanimous chorus of complaints from those who will be most affected by this change – the investigators and reviewers that will have to cope with this on a regular basis.
Dr. Rockey, Dr. Collins, ANYONE at NIH – if you continue to insist on going forward with this initiative despite vehement objection from the real stakeholders here, then PLEASE at least make it OPTIONAL. That way the INVESTIGATORS can choose what format they feel is the best for them. Who knows, maybe there are some investigators out there that will choose the new format. But give the investigators the choice. I’m quite certain that reviewers will be able to manage, even if they get a stack of applications in which the biosketches aren’t all in the same format.
I implore you – make this new biosketch format optional.
It is not too late to fix this.
I think the optional idea is a great compromise.
The new format is really promoting self trumpeting and unnecessary floritures and elaboration. Publications speak for themselves. If a proof of dedication and contributions to science are needed these things are best accomplished by letters of recommendations and letters of support. I am a new researcher, I have still relatively few publications and my contributions to science are ongoing. Yet I’d rather let the publication list speak for myself than have the awkward task of tooting my own horn.
The tone of this blog sounds more like dictatorial than democratic. I always wonder who is driving changes at NIH particularly in the application process. I tend to believe businesses that provide services to NIH such as Information Technology provider are the force. Keep tweaking the application process just to keep businesses happy without substantial benefit to either research or research community does not make sense. I think the proposed changes to biosketch is one of such changes done in recent times.
I am not at all shocked by the fact that I have not read one single word of praise or support for the new biosketch format. It does not sound as if NIH gives a hoot about the opinions of its most valuable resource – its investigators. This is so sad.
As a regular reviewer of NIH grants I agree wholeheartedly that this new biosketch format is a waste of researchers’ and grant reviewers’ valuable time and effort. I cannot for the life of me see how this extended format will have any impact whatsoever on how I will assess grants in the future – except perhaps by wasting my time reading through all the drivel that researchers will feel compelled to write.
The current format works. It is informative and concise. There is ample space to make your case in the personal statement. In any case, I am much more interested in the science that is being presented in the grant, than the person writing it. The best science will be funded if we focus on the science.
Everyone makes mistakes. The leadership at the NIH would only win the respect of the community if they admitted their error in this case, rescinded the decision, and left well alone a mechanism that is working well.
I wanted to express enthusiastic support for the new biosketch format. In a world where there are many ways to contribute to science, NOT ALL OF WHICH INVOLVE FIRST/LAST AUTHOR PUBLICATIONS IN SINGLE-WORD-TITLED JOURNALS, it is extremely refreshing to see NIH provide the capability for one to indicate our achievements for those of us that are not regularly in such a position. This includes many key roles in the scientific landscape – data curators, informaticians, statisticians, software developers, core facility personnel, etc.
I have recently created my new biosketch where I was for the first time able to tell the story of what exactly my contributions to science have been. Some of these things are standards – which are used by NCBI, Elsevier, and many other publishers. These have a large impact on science, yet don’t necessarily have many publications and therefore impact factors associated with them. If we want to promote team science and less competition to make the most of our limited resources, then we need a mechanism to value these non-traditional contributions and promote credit to and collaboration with those that create them.
As a woman, I also found that the instructions for describing lapses/changes to one’s career path empowering as well. For the first time, I felt comfortable telling the real truth about why I left the bench – having a baby and not wanting to work with the worlds most toxic carcinogens any longer. Why not just be honest about such things? We don’t live in a world like our parents did where we get one career and 50 years later retire from it. We can and do and should change careers all the time – it can make us smarter, more informed, and potentially more impactful in our science.
Is the new format more self promoting? Not really, it is just organized differently. The new format provides an opportunity to classify and explain the threads of where we come from and what we can bring to the table for any given proposal that the old format did not. If you are a scientist whose primary products are publications only, then the new format has you explain the threads of your research a little more specifically. However, if you have scholarly contributions that are not traditional publications, it is absolutely key to highlighting these types of contributions. Just because the products of such less traditional activities don’t have publisher-invented metrics (yet), doesn’t mean they are any less valuable. As scientists, we ourselves should dig a little deeper and try a little harder to understand the impact of non-publication scholarly products – that so many of use and are impacted by every day in our scientific work.
I see a lot of negative comments above, asking to bring back the old biosketch. I certainly understand that change is always hard and no one wants to do extra work (and it did take me a day to redo my biosketch), but I don’t really see many comments where the benefits of the old approach are clearly delineated. Comments like this one: “If applicants have published meaningful work in decent journals, this productivity speaks for itself” don’t understand how much of the scientific landscape they work in is not published within peer-reviewed journals. Think about that, the next time you go to a public database and search for data, or use some software tool. There is all kinds of public peer review on such things and often few or no papers. If anything, the transparency and peer review on software in particular, is often much more extensive than a publication.
If you want those of us that build tools, standardize data, etc. to keep doing so, we need a way to talk about those things and we need traditional scientists to value them.
Dr. Haendel’s comments seem right on to me. She raises exactly the sort of issues that NIH ought to be addressing in getting beyond the old boys club. Her comments about how the new biosketch support non-traditional but very important contributions to science is thought-provoking and makes me much more positive about the changes (I was fairly neutral until reading this).
As far as I can tell, the NIH is NOT listening to the voices of the scientists who are affected by the changes that they have implemented in the biosketch. Who makes these decisions and why??? I WANT TO MAKE 2 POINTS – THE NEW BIOSKETCH IS BAD FOR REVIEWERS AND BAD FOR SCIENTISTS.
I am currently reviewing grants for the NIH some of which have the new biosketch format – I cannot figure out what the person has published that is relevant to this proposal because of the new organization – it is USELESS and CONFUSING, and a waste of time. I will not be reading the biosketches that come with the grants anymore.
On the other side of the coin, I have already spent a significant amount of time trying to fit my work into the new biosketch format and it is extremely difficult – we cannot always package our accomplishments into 4-5 neat packages of a few papers – scientific research is a continuous process, not packaged in topics.
I realize that I am just one of MANY voices stating the obvious problems with the new biosketch format. BUT, WHY IS THE NIH INSISTING ON MAKING THIS CHANGE DESPITE THESE MANY LEGITIMATE AND IMPORTANT CONCERNS????
Melissa likes the new biosketch. I hate it. This female PI fails to see how this is new format takes down the old boys club, and the old format already gives opportunities for non traditional citations. If, for example, you develop software and share it in the public domain (akin to publishing a manuscript) on GitHub you can obtain a DOI and cite it.
This new format just takes too much time to prepare, and if you are a Co-I on multiple proposals as well as writing multiple applications of your own (a necessity with paylines at the 8th percentile in some institutes) this is a huge burden.
This is because to do a good job you have to tailor each one to the application. Good luck getting a busy physician co-investigator to do this at all!
So as I start on my fourth one for this review cycle I’m begging you could we at least make it voluntary? PLEASE??