Preprint Policies Sow Confusion

By Brian Jacobsmeyer

For the vast majority of his scientific papers, Terry Rudolph, a quantum theorist at Imperial College, London, had no qualms about posting a preprint on the popular arXiv server. But this one was different.

This research would soon be widely considered one of the most important papers on the foundations of quantum mechanics in recent years. Rudolph submitted his team’s paper to Nature and, with some reluctance, posted a preprint of it to the arXiv.

This paper never appeared in Nature, however. In a highly unusual case, Nature rejected the paper at a late editorial stage after Rudolph and a co-author, Jonathan Barrett of the University of London, posted a surprising follow-up article to the arXiv. In a post on the popular Cosmic Variance physics blog, Rudolph publicly contested Nature’s decision, blaming it in part on the scientific “buzz” surrounding his preprints.

Rudolph’s post echoed the fears of many physicists who avoid or limit posting to the arXiv during the submission process at premier academic journals, despite explicit journal policies that allow such practices. Emails obtained by APS News and conversations with Nature editors have revealed a muddled case that centers on the authors’ follow-up preprint that initially surprised the journal’s editors and some outside experts.

Adapting to Change

Since its creation 20 years ago, the arXiv preprint server (arXiv.org) has emerged as a primary source for many physicists seeking papers on cutting edge research in their field. Although arXiv preprints aren’t truly peer-reviewed, researchers in some sub-disciplines frequently depend on the arXiv more than academic journals.

“I think putting things on the arXiv is much more important,” said Matt Leifer, a quantum theorist at University College London. “More important than getting in a journal ever.”

As the arXiv has become more influential, journals have adapted. Physical Review Letters and other APS journals don’t have embargo policies, and APS permits authors to post preprints of submitted articles (see this month’s Back Page for a comprehensive explication of APS policies). Embargo policies at Nature and Science explicitly allow authors to post preprints of submitted articles without penalty, provided the authors don’t actively pursue press coverage.

“[The arXiv] caught us a little bit by surprise at the start,” said Karl Ziemelis, Chief Physical Sciences Editor at Nature. “Quite rapidly, we came to appreciate that the preprint server provided a very valuable service.”

Nature had no formal policy on preprints at first, but they ran a series of editorials over a decade ago that outlined their policy allowing the use of preprints, according to Ziemelis. Now, Nature lists this policy on several areas of its website.

Authors of papers in both Nature and Science may talk to members of the press one week before publication, however. Registered journalists are given access to upcoming papers and contact information for authors during this week under the condition that they don’t publish news articles until the press embargo has passed.

Although both journals have clear policies in principle, confusion and apprehension persist among physicists. Some physicists worry that posting preprint articles or giving conference talks without “embargoing” the information for attending journalists may affect their chances of publication.

“More often than not, it’s a fear that scientists have rather than the reality,” said Ivan Oransky, a science journalist who runs the popular Embargo Watch blog.

A Tale of Two Preprints

Other researchers had told Rudolph that posting to the arXiv may not have been a good idea, but he thought those fears were largely unfounded. Furthermore, another pressing issue had arisen: Rudolph’s colleagues had warned him that other groups may be posting similar research soon, effectively “scooping” their results. In November, he decided to post a preprint of the research conducted with his student, Matthew Pusey, and Barrett, under the title, “The Quantum State Cannot Be Interpreted Statistically.”

“Ultimately, the research priority is determined by when it goes on the arXiv,” said Rudolph.

Quickly, journalists picked up on the preprint. Rudolph, Barrett and Pusey declined to talk to the press per Nature’s embargo policy. Nonetheless, Nature News published a popular science article covering the research in which several respected experts praised the new results. Nature News belongs to the same company as the journal Nature, but the two publications maintain an editorial firewall.

Pusey, Barrett, and Rudolph had shown that the quantum state is, in fact, physically real. In other words, the authors’ argument suggested that there was no deeper theory underlying quantum mechanics. This result conflicted with “epistemic” or “statistical” interpretations of quantum mechanics that suggest wave functions merely reflect an observer’s knowledge about a system. These “statistical” interpretations have enjoyed resurging popularity among some philosophers and physicists in the past couple of decades.

“It was certainly the biggest result in the foundations of quantum theory in the last 5 years,” said Leifer. “That might be underestimating it.”

Approximately three weeks after posting his preprint, Rudolph heard promising news from Nature: The referee reports were mostly positive, and the editors were accepting Rudolph’s paper, in principle. A large majority of papers accepted in principle are eventually published. “In principle” acceptance, however, is a stage removed from formal acceptance.

The three authors then revised their article and sent it back to Nature’s editors. Meanwhile, Barrett and Rudolph posted another arXiv article with a cheeky, conflicting title: “The Quantum State Can Be Interpreted Statistically.”

Although this second paper’s title suggested a contradiction with the original results, the authors’ intention was to explore a key assumption of the first paper –preparation independence. This principle asserts that it’s possible to set up two experiments independently, say in different geographic locations at different labs, without any underlying super-correlations that would affect the results of both experiments.

“That assumption is essentially a bedrock assumption of science,” said Rudolph. “Otherwise, there could always be the leprechauns tricking us hiding under our experiments.”

The second paper aimed to strengthen the first paper by carefully examining this assumption mathematically. In the paper, the authors presented a technical counterexample under which this assumption was false. But they were not suggesting that this could be a plausible feature of the world, said Rudolph.

Rudolph wanted to post this follow-up article, which included authors who were not involved with the first paper, as soon as possible. In his field, priority is often given to arXiv posting dates, not publication dates in journals. Nature’s policies require authors to notify them of any submitted research that may have a bearing on a submitted article, which Rudolph did not do.

In a matter of days, one referee alerted Nature to the second paper and expressed some concerns. When asked to comment, another referee expressed surprise but still supported publication of the original paper. Within a week of the second preprint’s publication, Nature rejected the first paper without sharing these referees’ comments on the second paper with the authors.

Rudolph appealed the decision in an email the next day. During the next two weeks, Karen Howell, Nature’s corresponding physical sciences editor for this manuscript, consulted again with the referee who alerted her to the paper. This referee had provided the “most detailed and thoughtful comments,” throughout the review process, according to a written statement provided by Howell.

“The advice that we received from referee 3 was that the later work considerably weakened the findings of the original paper,” Howell wrote in her statement.

In a follow-up rejection email, Nature reiterated their position, adding that one referee felt there was no justification for the preparation independence assumption. At the end of the email, Howell added an explanation on why publishing the paper “would not necessarily serve the wider interests of the community,” according to her statement. This explanation was stated as an aside after the reasons for rejection were given, she said.

She added: “In this regard, I pointed out that the timeliness and impact of the Nature submission had been diminished following wide debate.”

In light of this debate among scientists, the “take home” message of the original paper was becoming increasingly unclear, according to both Howell and Ziemelis. The evolving scientific debate facilitated by the authors’ second preprint appeared to weaken the original paper’s impact in the minds of Nature’s editors.

Rudolph believes that Nature should have consulted more expert opinion to resolve any confusion, and that he and his colleagues should have been allowed to respond directly to the referees’ comments on the second paper.

“I don’t believe that referee 3 ever had opinions that the [original] paper should not be published because of paper two,” said Rudolph. “And I have not seen any information to the contrary.”

Rudolph admits that the second paper may have made some experts “uncomfortable” in the early days after publication, as intended. At subsequent conferences, however, no one has called the preparation independence assumption into question when considering serious physical theories, Rudolph added. Instead, discussions have addressed the way this assumption fits into other key assumptions in the philosophy of physics, he said.

In the end, the first paper was published in the less prominent Nature Physics. Physical Review Letters published the second paper.

Ingelfinger’s Legacy

The Ingelfinger Rule, which stipulates that authors cannot publish the same paper in two different outlets, has led many scientists to be extra cautious. Originating in the 1960s, the rule’s intention was to prevent publication of the same research in multiple scientific journals as a way to maintain originality.

The arXiv has complicated interpretations of this rule. Preprint servers have significantly accelerated publishing in physics, and top journals have tried to adapt.

In this particular case, Nature’s editors concluded that the message of a submitted paper had changed in light of a new preprint. Nature’s rejection did not appear to violate its own pre-publication embargo policy. Nonetheless, the case reveals the confusion and miscommunication that can arise amongst editors and physicists who use two different publishing platforms.

Many physicists still approach preprints differently than normal when submitting to high-end journals. Although this case won’t affect how Rudolph uses the arXiv, he still sees this sentiment amongst many of his peers.

“They feel that they have to [act differently],” said Rudolph.

Oransky, the founder of the Embargo Watch blog, believes that these fears are typically unfounded, but the specter of the Ingelfinger Rule still exerts a strong influence on scientific publishing.

“Scientists often fear the so-called Ingelfinger Rule more than they have to, but it has a real chilling effect on the flow of scientific information,” Oransky said in an email. “I’d love to see more open discussion of these issues, so if this sparks that in the future, I think that’s a good thing.”