What Science — and the Gonzo Scientist — got wrong: open access will make research better

Science is arguably the most important magazine in the world of research. Recently, Science published a blistering critique of the most sacred cow of scientific research, namely the peer review quality system.

Unfortunately, Science doesn’t seem to have understood what it has done. They say they’ve run a sting operation — written by the Gonzo Scientist, John Bohannon — revealing the weaknesses of a relatively new publishing model called open access. In fact, the Science article shows exactly the opposite of what they intend, namely that we need an even wider use of open access than what we currently have.

What is Open Access publishing?

At its core, open access is about making the results of research freely available to anyone. Obviously, the internet is at the heart of this revolution. But it’s also driven by idealism and principle.

Consider the flow of money for research at a publicly funded institution. A national government or a research council gives funds to a university that ultimately passes these monies along to a researcher. The researcher makes a discovery and writes an article about it. The article is then submitted to a journal. The journal is responsible for rigorously studying the reported work to make sure it is reliable. This is the heart of quality control: peers — experts working in the same field — anonymously review the work; they challenge it, critique it, ask for new perspectives to be considered, and may even suggest changes in the analysis and presentation.

What are the journals’ costs? There of course is variation, but the simplest model is this. Journals pay the authors of an article nothing. They pay the editor of a journal nothing. They pay the three or more reviewers of articles nothing. Some journals incur expenses associated with typesetting and related activities. And of course those that publish on paper have costs associated with printing and distribution.

Publishers then sell the journals to — you guessed it! — universities. They sell them to the very institutions that have given so much to the publisher already: research papers and the costs behind them as the time of researchers, editors and reviewers. All this — and the copyright to the article — are freely donated to the publisher.

Even though the costs are limited, the price of journals has exploded. Harvard University — the richest university in the world — says it can’t keep up anymore and has started cutting subscriptions. This is happening everywhere. The publishers respond by packaging journals in massive bundles and no longer allowing universities to take only the ones they want. It’s become ugly.

One of the very biggest publishers, Elsevier, consistently reports hundreds of millions of dollars in profit. Much of these profits come from public expenditures to buy access to articles that have been produced with public funds. This is the context that recently led to a worldwide boycott of Elsevier by researchers around the globe.

And it’s the context in which an entire industry — scholarly publishing — has become ripe for innovation. Open access is the result.

When bad science gets published

The peer review system is taken very seriously by scholars. But most researchers struggle to meet the expectations that their institutions and fields place on them in the course of a normal work week. And sometimes, bad work slips by and gets published.

The spectacular case of Diederik Stapel showed that a researcher could fabricate data for years without being caught. A massive review concluded that Stapel published 55 articles based on fraudulent data in serious journals — even in Science! (Science naturally followed the standards of good practice and retracted the article.)

Is it reasonable to expect that the peer review process would catch fraud like Stapel’s? The committee reviewing the case found it “inconceivable that … reviewers of the international ‘leading journals’ … could have failed to see that [Stapel’s experiments] would have been almost infeasible in practice, [and that they] did not notice the reporting of impossible statistical results … Virtually nothing of all the impossibilities, peculiarities and sloppiness mentioned in this report was observed by all these local, national and international members of the field, and no suspicion of fraud whatsoever arose.”

Bad work gets published. This is a crisis for science and it’s a crisis for societies that want to make the lives of its citizens better through the results of science. (See also my Guardian piece, Science research: three problems that point to a communications crisis.)

This is the crisis that Science shines a sharp light on this week; but Science misread the label on that crisis. It’s not a crisis caused by a decision to make the results of research freely available — open access; it’s a crisis caused by the meltdown of the peer review system. We need change. It’s the digital age that allows that change, and it’s the very best OA journals that are leading the development of new approaches to peer review.

Corruption

When research is made freely available through open access publishing, there may still be legitimate expenses that a publisher must recoup. One strategy for this is to take a payment from an author when an article is accepted for publication. It is now common that research councils allow grant money to be used to pay these so-called author fees.

This is a model that invites corruption. Set up a journal, accept some articles, charge high author fees, and publish the articles on your website. This corruption is fed, of course, by the fact that researchers feel incredible pressure to publish more and more and more, fed by a system that uses quantity as a proxy for quality.

The author payment model has in fact led to corruption, and that is what Science does an excellent job of revealing this week. But it is a mistake to equate open access and author payment.

The Science article refers to a database called the Directory of open access journals. 65% of the journals listed there do in fact not charge authors to publish. They are funded instead by universities directly or by research councils or professional societies.

In Norway, where I work, there is a national committee that sorts out bad journals — open access and traditional alike. And, yes, because of author payment, that committee sees a proliferation of illegitimate open access journals. But, again, that is because of author payment, not because the results of research are being made freely available.

Quality control and peer review

The real problem for science today is quality control. Peer review has been at the heart of this, but there are too many failures — in both open access and traditional journals — to simply plod ahead with the same system. We need new approaches and many people and organizations are working on those, such as the open evaluation project.

The creative potential offered by digital communication of results, where open access journals are in the forefront, is exactly what we need to focus on. And if we do so, we will solve the problem of a broken peer review system — and that is the problem that Science and the Gonzo Scientist have uncovered.

An edited version of this piece was published at The Guardian on October 4, 2013, under the title Open access publishing hoax: what Science magazine got wrong.

Additional discussion of the Science article:

Science magazine rejects data, publishes anecdote, by Björn Brembs
John Bohannon’s peer review sting against Science, by Mike Taylor
New “sting” of weak open access journals, by Peter Suber
I confess, I wrote the arsenic DNA paper to expose flaws in peer-review at subscription based journals, by Michael Eisen
Science reporter spoofs hundreds of open access journals with fake papers, at the wonderful Retraction Watch, by Ivan Oransky
OASPA’s response to the recent article in Science, entitled “Whose afraid of Peer Review?” Press Release
What Science‘s “sting operation” reveals: open access fiasco or peer review hellhole? by Kausik Datta
Who’s afraid of open access? by Ernesto Priego
Science Mag sting of OA journals: is it about open access or about peer review, by Jeroen Bosman

and they keep on coming:
The open access hoax and other failures of peer review, at Language Log, by Mark Liberman
A publishing sting, but what was stung? at Open Science Collaboration, by Åse Innes-Ker
Flawed sting operation singles out open access journals, at The Conversation, by Martin Eve

If I missed yours, mention it below and I’ll add it to the list.