How to take charge of science policy: making research more visible

Governments can get lost in counting; in the worst case, they lose sight of the policy goals that some measurement is supposed to facilitate. In Norway, our system for rewarding universities for scientific publications is under review. As politicians begin processing the recommendations for change, perhaps the most important step they can take is to renew the policy objectives of the system. What exactly should the incentive system lead to and how might it be constructed to achieve these goals?

Promoting internationalization and open access would be easy with today’s system. Here’s how.

In the first two parts of this three-part series, I have identified four incentives that — if changed — would lead to a system with greater integrity and rationality. (Part 1: Do you make these 6 mistakes? A funding scheme that turns professors into typing monkeys. Part 2: 3 simple distinctions your government should eliminate from its research financing system)

In this third part, I want to highlight some policy goals and show how the incentive system for publishing could easily be made into an important tool for achieving them.

Increase internationalization

The Norwegian government has a clearly articulated goal to increase the international visibility of the research it funds.

To further enhance the quality of Norwegian research and higher education, international cooperation must become an integral part of the average workday of more Norwegian researchers. Long-term perspectives – knowledge provides opportunity

With one very simple modification, the publication incentive system can play an important role in encouraging Norwegian researchers to be even more international.

Here’s how to do it: The current system awards points as a function of the number of authors on an article. You receive half the number of points if you have one co-author, a third if you have two, and, well, you get the pattern.

If the system were modified so that the points awarded were a fraction of the number of authors who are in the Norwegian system, that would give a strong, clear, and rational push towards doing more work with international co-authors.

If I am writing an article with another colleague in Norway, and have a sensible opportunity to draw in two North American authors in ways that will increase the quality of the research and the paper, the current incentive system actually gives a disincentive that must be overcome.

If international co-authors were not included in the equation for assigning points, that disincentive would disappear. Easy. Effective. And directly connected to a clearly articulated goal. Who will make this happen?

Reward open access publishing

Many governments and research funding organizations have a policy that publicly financed research should be freely available. The incentive system could easily lead to major progress here, too.

Open access publishing comes in many forms and this blog post is not the place for elaborate discussions of their nuances. However, there can be no question that the best open access model is the so-called gold model, whereby the contents of a journal are freely available upon publication.

Given that the government wants the research it funds to be published with open access and given that gold open access is the most sensible variant we have, the government should use the publication incentive system to encourage publishing in such journals.

One easy way to do that would be to use a multiplier for the points if the article appears in an open access journal or to add a fraction of a point for each gold OA article when calculating the annual total for each university. If the point value of OA articles were increased by 25%, universities would push harder for their publishers to use such fora and the policy goal would get a boost.

Looking ahead to a better system

In this series of three postings, I’ve argued for six modifications of the Norwegian government’s current system of rewarding universities for the publications their employees: (i) eliminate the current two-level system, (ii) treat anthologies and journals alike, (iii) bring textbooks into the system, (iv) count introductory works of scholarship in anthologies, (v) eliminate international co-authors from the calculation of points, and (vi) introduce a multiplier for articles appearing in gold open access journals.

A system that pushes for increased quantity of publications has to emerge from clear policy goals. It may sound odd to suggest that the government should not have the goal of getting people to publish more, but that in and of itself really cannot be what a publicly financed research system is about.

Instead, incentives offered to the research sector should be about pushing for better research, for more discovery and for new applications that increase our quality of life. And, sure, the system should also be about engaging in an international research community through publications, to the extent that they feed the other goals here.

Beyond increased internationalization and open access publishing, an example of a policy goal might be that the work of Norwegian researchers is more visible.

I hear conservative voices telling me already that increased visibility is exactly the goal of giving extra rewards to publications in the best journals, whether that is determined by the current two-level system or by the internationally more familiar (but also subjective) impact factor.

Increased visibility of research

Pursuing a policy of increased visibility by sorting journals is a strategy from yesteryear.

Visibility today is about search engine optimization, it’s about Twitter, and it’s about the ability of digital journals or repositories to position themselves on the web. (Check out Melissa Terras’ clever demonstration of the effect of tweeting on the visibility of research papers.)

New ways to measure visibility are constantly emerging. In addition to traditional citation rates for individual articles, we can also track downloads. And if we use that measure, then it’s an especially good idea to support open access publishing, since those papers are downloaded more.

Does counting downloads as a measure of visibility miss something important that could be captured by counting citations of individual articles? Probably not. In fact, we should be cautious if we believe that citation is a measure of quality, given that only about 20% of those who cite a paper have actually read it.

It’s time to update our measures of visibility and alternative metrics must be part of any modern system for quantifying research production.

The possibility that the government articulate goals and incentives about visibility is especially relevant for Norway since Norwegian research is less visible than research done in other countries we like to compare ourselves with, such as Finland, Sweden, Denmark, the Netherlands and Austria.

But whether increased visibility or something else is the government’s goal, improving our current incentive system surely must start with a reinvigorated and compelling statement of what our politicians are trying to accomplish.

Part 1: Do you make these 6 mistakes? A funding scheme that turns professors into typing monkeys

Part 2: 3 simple distinctions your government should eliminate from its research financing system