The Back Page
The Future of Science: Building a Better Collective Memory
By Michael A. Nielsen
Many great scientists of the age, including Leonardo, Galileo and Huygens, used anagrams or ciphers for similar purposes. The Newton-Leibniz controversy over who invented calculus occurred because Newton claimed to have invented calculus in the 1660s and 1670s, but didn’t publish until 1693. In the meantime, Leibniz developed and published his own version of calculus.
Such secrecy was natural in a society in which there was often little personal gain in sharing discoveries. This secrecy faded because the great scientific advances in the time of Hooke and Newton motivated wealthy patrons such as the government to begin subsidizing science as a profession. Because the public benefit delivered by scientific discovery was strongest if discoveries were shared, the result was a scientific culture that to this day rewards the sharing of discoveries. Today, when a scientist applies for a job, the most important part of the application is often their published scientific papers.
The adoption and growth of the scientific journal system has created a body of shared knowledge for our civilization, a collective long-term memory that is the basis for much of human progress. This system has changed surprisingly little in the last 300 years. The Internet offers us the first major opportunity to improve this collective long-term memory, and to create a collective short-term working memory, a conversational commons for the rapid collaborative development of ideas.
One way of viewing online tools is as a way of expanding the range of scientific knowledge that can be shared with the world. A successful example is the physics preprint arXiv, which lets physicists share preprints of their papers without the months-long delay typical of a conventional journal. More radically, the internet can also change the process and scale of creative collaboration, using social software such as wikis, online forums, and similar tools. I believe that such tools and their descendants will change scientific collaboration more over the next 20 years than it has changed in the past 300 years. Yet, with the exception of email, scientists currently appear puzzlingly slow to adopt many online tools. This is a consequence of some major barriers deeply embedded within the culture of science.
Two Failures of Science Online
Inspired by the success of Amazon.com’s review system and similar sites, many organizations have created comment sites where scientists can share their opinions of scientific papers. Perhaps the best-known was Nature’s 2006 failed trial of open commentary on papers being peer reviewed at Nature. To date, none of the sites have succeeded.
The problem is that while thoughtful commentary on scientific papers is useful for other scientists, there are few incentives for people to write such comments. Why write a comment when you could be doing something more “useful,” like writing a paper or a grant? Furthermore, if you publicly criticize someone’s paper, there’s a chance that person may be an anonymous referee in a position to scuttle your next paper or grant application.
Contrast this with the approximately 1500 reviews of Pokemon you’ll find at Amazon.com. We have a ludicrous situation where popular culture is open enough that people feel comfortable writing Pokemon reviews, yet scientific culture is so closed that people will not publicly share their opinions of scientific papers. Some people find this curious or amusing; I believe it signifies something seriously amiss with science that needs to change.
Wikipedia is a second example where scientists have missed an opportunity to innovate online. Wikipedia has a vision statement to warm a scientist’s heart: “Imagine a world in which every single human being can freely share in the sum of all knowledge. That’s our commitment.” You might guess Wikipedia was started by scientists eager to collect all of human knowledge into a single source. In fact, Wikipedia’s founder, Jimmy Wales, had a background in finance and as a web developer. In the early days few established scientists were involved. To contribute would arouse suspicion from colleagues that you were wasting time that could be spent writing papers and grants.
Some scientists will object that contributing to Wikipedia isn’t really science. It’s not if you take it for granted that science is only about publishing in specialized scientific journals. But if you believe science is about discovering how the world works, and sharing that understanding with the rest of humanity, then the lack of early scientific support for Wikipedia looks like an opportunity lost. Nowadays, Wikipedia’s success has to some extent legitimized contribution within the scientific community. But how strange that the modern day Library of Alexandria had to come from outside academia.
An Open Scientific Culture
The value of openness was understood centuries ago by many of the founders of modern science; indeed, the journal system is perhaps the most open system for the transmission of knowledge that could be built with 17th century media. The adoption of the journal system was achieved by subsidizing scientists who published their discoveries in journals. This same subsidy now inhibits the adoption of more effective technologies.
We should aim to create an open scientific culture where as much information as possible is moved out of people’s heads and labs, onto the network, and into tools that can help us structure and filter the information: data, scientific opinions, questions, ideas, folk knowledge, workflows, and everything else. Information not on the network can’t do any good.
One way to achieve cultural change is via the top-down strategy that’s been successfully used by the open access (OA) movement. The goal of OA is to make scientific research freely available online to everyone in the world. In April 2008 the National Institutes of Health mandated that every paper written with the support of their grants must eventually be made open access. This is the scientific equivalent of successfully storming the Bastille.
The second strategy is bottom-up. It requires that the people building the new online tools also develop and boldly evangelize ways of measuring the contributions made with the tools. As an example, since 1991 physicists have been uploading their papers to the physics preprint arXiv, often at about the same time as they submit to a journal. The arXiv is not refereed, although a quick check is done by arXiv moderators to remove crank submissions. In many fields, most papers appear on arXiv first, and many physicists start their day by seeing what’s appeared on the arXiv overnight.
After the arXiv began, a service for particle physics called SPIRES-HEP extended their citation tracking to include both arXiv papers and conventional journal articles. As a result, it’s now possible to search on a particle physicist’s name, and see how frequently all their papers, including arXiv preprints, have been cited by other physicists.
SPIRES-HEP has been run since 1974 by the Stanford Linear Accelerator Center (SLAC). SLAC’s metrics of citation impact are both credible and widely used by the particle physics community. When physics hiring committees meet to evaluate candidates in particle physics, people often have their laptops out, examining and comparing the SPIRES-HEP citation records of candidates. The result is a small but genuine cultural change towards more openness in science, achieved using the bottom-up strategy.
The Problem of Collaboration
When doing research, subproblems constantly arise in unexpected areas. No one can be expert in all those areas. Most of us instead stumble along, picking up the skills necessary to make progress towards our larger goals. We have a small group of trusted collaborators with whom we exchange questions and ideas when we are stuck. They may point us in the right direction, but rarely do they have exactly the expertise we need. Might it be possible to use online tools to scale up this conversational model, and build an online collaboration market to exchange questions and ideas, a sort of collective working memory for the scientific community?
To see how much is lost due to inefficiencies in the current system of collaboration, imagine a scientist named Alice. Many of Alice’s research projects spontaneously give rise to problems in areas in which she isn’t expert. Suppose that for a particular problem, Alice estimates that it would take her four to five weeks to acquire the required expertise and solve the problem. So the problem is on the backburner. Unbeknownst to Alice, though, there is another scientist in another part of the world, Bob, who has just the skills to solve the problem in less than a day.
Unfortunately, nine times out of ten they never even meet, or if they do, they just exchange small talk. It’s an opportunity lost for a mutually beneficial trade, a loss that may cost weeks of work for Alice. It’s also a great loss for the society that bears the cost of doing science. Expert attention, the ultimate scarce resource in science, is very inefficiently allocated under existing practices for collaboration.
An efficient collaboration market would enable Alice and Bob to find this common interest, and exchange their know-how, in much the same way eBay and craigslist enable people to exchange goods and services. However, in order for this to be possible, a great deal of mutual trust is required. Without such trust, there’s no way Alice will be willing to advertise her questions to the entire community.
Let’s compare this situation to the apparently very different problem of buying shoes. Alice walks into a shoe store, with some money. Alice wants shoes more than she wants to keep her money; Bob the shoe store owner wants the money more than he wants the shoes. As a result, Bob hands over the shoes, Alice hands over the money, and everyone walks away happier after just ten minutes. This rapid transaction takes place because there is a trust infrastructure of laws and enforcement in place that ensures that if either party cheats, they are likely to be caught and punished.
If shoe stores operated like scientists trading ideas, first Alice and Bob would need to get to know one another, maybe go for a few beers in a nearby bar. Only then would Alice say, “You know, I’m looking for some shoes.” After a pause, and a few more beers, Bob would say “You know what, I just happen to have some shoes I’m looking to sell.” Every working scientist recognizes this dance; I know scientists who worry less about selling their house than they do about exchanging scientific information.
In economics, it’s been understood for hundreds of years that wealth is created when we lower barriers to trade, provided there is a trust infrastructure of laws and enforcement to prevent cheating and ensure trade is uncoerced. The basic idea, which goes back to David Ricardo in 1817, is to concentrate on areas where we have a comparative advantage, and to avoid areas where we have a comparative disadvantage.
Ricardo’s analysis works equally well for trade in ideas. Indeed, even were Alice to be far more competent than Bob, both Alice and Bob benefit if Alice concentrates on areas where she has the greatest comparative advantage, and Bob on areas where he has less comparative disadvantage. Unfortunately, science currently lacks the trust infrastructure and incentives necessary for such free, unrestricted trade of questions and ideas.
An ideal collaboration market will enable just such an exchange of questions and ideas. It will bake in metrics of contribution so participants can demonstrate the impact their work is having. Contributions will be archived, timestamped, and signed, so it’s clear who said what, and when. Combined with high quality filtering and search tools, the result will be an open culture of trust that gives scientists a real incentive to outsource problems, and contribute in areas where they have a great comparative advantage, fundamentally changing how science is done.
Michael Nielsen is a writer working on a book about the future of science. For information about the book, see Michael Nielsen's Blog. In a past life he helped pioneer the field of quantum computation, and was the author of more than 50 scientific papers. The above article is adapted from an essay appearing on his blog, based on his keynote talk at the New Communication Channels for Biology workshop, San Diego, June 26 and 27, 2008. Full version of keynote talk.