Tag: Relative Citation Ratio (RCR)

New NIH Resource to Analyze Biomedical Research Citations: The Open Citation Collection

October 16, 2019
By

My colleagues within the NIH Office of Portfolio Analysis sought to answer this call. Drs. Ian Hutchins and George Santangelo embarked on a hefty bibliometric endeavor over the past several years to curate biomedical citation data. They aggregated over 420 million citation links from sources like Medline, PubMed Central, Entrez, CrossRef, and other unrestricted, open-access datasets. With this information in hand, we can now take a better glimpse into relationships between basic and applied research, into how a researchers’ works are cited, and into ways to make large-scale analyses of citation metrics easier and free.

, , ,

5 Comments

Impact of Teams Receiving NIH Funding

April 4, 2018
By

Almost 11 years ago, Stefan Duchy, Benjamin Jones, and Brian Uzzi (all of Northwestern University) published an article in Science on “The Increasing Dominance of Team in Production of Knowledge.” They analyzed nearly 20 million papers published over 5 decades and 2.1 million patents and found that across all fields the number of authors per paper (or patent) steadily increased, that teams were coming to dominate individual efforts, and that teams produced more highly cited research.

, , , ,

32 Comments

What Can We Learn from the Early Outcomes from the NIH Director’s New Innovator Awards?

October 27, 2017
By

In earlier posts, like this one, we discussed the importance of moving towards “evidence-based funding.”. NIH seeks to apply data-driven strategies to conceptualize, develop, implement, and evaluate policies, such as those that will affect the NIH-supported biomedical research workforce. Today, we’d like to spotlight a recently published analysis of an award program directed to investigators early in their careers – a population that has received much attention at NIH and beyond in recent years.

, , ,

7 Comments

Patents and the Relative Citation Ratio: Correlations to Assess NIH Impact

September 18, 2017
By

We previously referenced Ioannidis’ and Khoury’s “PQRST” mnemonic for describing research impact: “P” is productivity, “Q” is quality, “R” is reproducibility, “S” is sharing, and “T” is translation.  We wrote several blogs about “P,” productivity, focusing on publications, citations, and more recently the relative citation ratio.  Now we’ll focus on a different kind of “P” for productivity, namely patents (which arguably are also related to “T” for translation).  …. Do NIH-supported papers that are cited by patents have a higher Relative Citation Ratio than those that are not cited by patents? As a refresher, the Relative Citation Ratio uses citation rates to measure the influence of a publication at the article level…. We identified 119,674 unique NIH grants that were funded between 1995 and 2007 and that generated at least one publication….

, , , ,

4 Comments

Applications, Resubmissions, and the Relative Citation Ratio

April 25, 2017
By

Measuring the impact of NIH grants is an important input in our stewardship of research funding. One metric we can use to look at impact, discussed previously on this blog, is the relative citation ratio (or RCR). This measure – which NIH has made freely available through the iCite tool – aims to go further than just raw numbers of published research findings or citations, by quantifying the impact and influence of a research article both within the context of its research field and benchmarked against publications resulting from NIH R01 awards.

In light of our more recent posts on applications and resubmissions, we’d like to go a step further by looking at long-term bibliometric outcomes as a function of submission number. In other words, are there any observable trends in the impact of publications resulting from an NIH grant funded as an A0, versus those funded as an A1 or A2? And does that answer change when we take into account how much funding each grant received? ….

, , , , , , , , ,

14 Comments

Following up on the Research Commitment Index as a Tool to Describe Grant Support

February 15, 2017
By

Many thanks for your terrific questions and comments to last month’s post, Research Commitment Index: A New Tool for Describing Grant Support. I’d like to use this opportunity to address a couple of key points brought up by a number of commenters; in later blogs, we’ll focus on other suggestions.

The two points I’d like to address here are: 1) why use log-transformed values when plotting output (annual weighted relative citation ratio, or annual RCR) against input (annual research commitment index, or annual RCI), and 2) what is meant by diminishing returns. ….

, , , ,

5 Comments

Research Commitment Index: A New Tool for Describing Grant Support

January 26, 2017
By

On this blog we previously discussed ways to measure the value returned from research funding. Several of my colleagues and I, led by NIGMS director Jon Lorsch – chair of an NIH Working Group on Policies for Efficient and Stable Funding – conceived of a “Research Commitment Index,” or “RCI.” We focus on the grant activity code (R01, R21, P01, etc) and ask ourselves about the kind of personal commitment it entails for the investigator(s). We start with the most common type of award, the R01, and assign it an RCI value of 7 points. And then, in consultation with our NIH colleagues, we assigned RCI values to other activity codes: fewer points for R03 and R21 grants, more points P01 grants.

, , , ,

66 Comments

Applying the Relative Citation Ratio as a Measure of Grant Productivity

October 21, 2016
By

Last April we posted a blog on the measurement of citation metrics as a function of grant funding. We focused on a group of R01 grants and described the association of a “citation percentile” measure with funding. We noted evidence of “diminishing returns” – that is increased levels of funding were associated with decreasing increments of productivity – an observation that has been noted by others as well.

We were gratified by the many comments we received, through the blog and elsewhere. Furthermore, as I noted in a blog last month, our Office of Portfolio Analysis has released data on the “Relative Citation Ratio,” (or RCR) a robust field-normalized measure of citation influence of a single grant (and as I mentioned, a measure that is available to you for free).

In the follow-up analysis I’d like to share with you today, we focus on a cohort of 60,447 P01 and R01-equivalent grants (R01, R29, and R37) which were first funded between 1995 and 2009. Through the end of 2014, these grants yielded at least 654,607 papers. We calculated a “weighted RCR” value for each grant, ….

, , ,

12 Comments