When a Country is Open, Do They Have Strong Science Too?


As no scientist is an island, the overall scientific enterprise grows stronger when people work together. But, an interesting question emerges from this concept for us to explore: how can we quantify the effect of collaboration on productivity and impact on science?

In the October 5 issue of Nature, Caroline Wagner, Ph.D. of the Ohio State University and Koen Jonker’s, Ph.D. of the European Commission Joint Research Center

published an interesting analysis of the association of a country’s “openness” and its scientific productivity. The authors assembled data from the Scopus database—a wealth of information on citations from peer-reviewed scientific journals—and data on workforce mobility from the Organization for Economic Cooperation and Development.

Drs. Wagner and Jonkers developed an “openness” measure to look at international engagement. This measure is a composite of “numbers of scientists emigrating from, immigrating to, and returning to a country, plus international co-authorships.” These data encompassed activities in 33 countries and included 3-year citation data for 2.5 million calendar year 2013 publications across all scholarly fields.

Fortunately, because the authors shared their data freely, we could easily download it and dive into the numbers. Figure 1 is a reproduction of the figure the authors published in their paper. It shows a reasonably strong association between a country’s openness and the citation impact of its scientific work; the author’s used a field-normalized citation impact measure here because that allowed them to compare different-sized countries and subjects. In a regression model, a country’s openness “explains” 59% of the variance of citation impact (P<0.0001).

Figure 1 shows a grid of a country’s openness (on the X axis) and its citation impact (on the Y axis). A country is represented by a dot on the graph, where the small, medium, and large dot size refers to either 250,000, 500,000, or 750,000 publications, respectively. Bottom left are countries with low openness and low citation impact, while countries in the upper right have high openness and high citation impact.

Most countries are either “low-openness and low-impact” – in the lower left quadrant (e.g. China and Japan) – or “high-openness and high-impact” – in the upper right quadrant (e.g. Singapore and Switzerland). The authors note that the United States is one of only 4 countries in the upper left quadrant – with relatively low openness but high citation impact.  They suggest this may be “because of the magnitude of its scientific enterprise and its geographic distance from possible collaborators.”

In their essay, the authors note a strong correlation between levels of government funding for research and development and number of publications. Figure 2 shows a log-log plot demonstrating a near-log-linear association (with funding explaining 89% of the variance, P<0.0001).  We use a log-log plot because both funding and number of publications follow skewed distributions; this also follows standards illustrated, for example, by Geoffrey West in his recently published book Scale. The slope of the log-log line is 0.77, a value less than one, consistent with diminishing marginal returns; that is to say, an increase of funding by 10% is associated with a less-than 10% increase in number of publications.

Figure 2 shows a log-log graph of a country’s investment in research and development in 2011 (in millions of U.S. dollars on the X axis) and the number of publications in 2013 (on the Y axis). Countries on the left have low spending and fewer publications, while those on the right have a larger investment and number of publications.

The authors also note that, even though there is a correlation between government expenditures and the number of publications, there is little correlation between funding and citation impact. Figure 3 is a scatter plot which shows the absence of association (in a regression model, funding explains only 5% of the variance, P>0.10).

Figure 3 shows a plot of government spending on research and development (on the X axis) and citation impact (on the Y Axis). A country is represented by a dot on the graph, where the small, medium, and large dot size refers to either 250,000, 500,000, or 750,000 publications, respectively.

These findings offer intriguing insights into research inputs and outputs on an international scale. Taken together, they suggest that funding is strongly associated with quantitative output (i.e. number of publications), but that other factors may play a stronger role in citation output – which the authors describe as a signal of “engagement and recognition .”

We congratulate the authors for their analyses and publication. And, separately, we commend them as well for their willingness to share their data with the wider research community. By facilitating this access, information can eventually lead to ground-breaking findings that will improve the health and lives of all Americans and those around the world too.


  1. A quick glance suggests that a country’s citation impact is roughly correlated with it’s “world happiness ranking” See Gallup story entitled, “The Happiest and Unhappiest Countries in the World.”

  2. I agree with Dr Lauer that is is commendable that the authors shared thier data, but this should not be an unusual act. Since these are US and European authors, it is very possible that they were not just being generous, but that they are abiding by the rules set out for government sponsored research. The US (I don’t know the various policies in Europe) requires researchers funded through non-classified research activities to make thier data open. The fact that this is not a clear norm within the US despite the Obama Administration express policy is upsetting. The US needs to do better and EXPECT data to be available once the primary publication is completed, rather than commending people for following the rules.

Leave a Reply

Your email address will not be published. Required fields are marked *