APS News


Perspectives on Ethics and Validity in Science

By R. Stephen Berry

Science and the knowledge it produces have a special, unique quality in the body of human experience. A characteristic of this uniqueness is the capacity of science to provide reliable quantitative predictions of phenomena, within its own domain. No other aspect of human experience has this capability. But this predictive power is the consequence of the way scientific studies evolve. The validation processes of the scientific enterprise are themselves unique and are the basis of the predictive powers of science.

Science advances by a kind of trial-and-error activity, guided by past observations and the interpretations - theories, if you will - of the phenomena that yielded those observations. Foremost is the establishment of the validity of the result. A measurement of a quantity that has been predicted by a very well-established theory is unlikely to stimulate experiments to repeat and validate that result. For many years the theoretical methods available to compute properties of simple atoms were accepted to be capable of generating more accurate values than the experiments that could measure the same properties. But it was useful for investigators to improve the experimental methods and continue to measure the properties.

At the opposite extreme are results that challenge established findings, whether experimental or theoretical. When a new experiment yields a value of a fundamental constant of nature that lies three standard deviations outside the range of previously measured values, people take notice. First, they scrutinize the way the new measurement was made, and if it seems without apparent error, they are likely to repeat the new measurement and to find other ways to measure the quantity in question. There are well- recorded instances of a set of apparently consistent measurements being superceded by such an "outlier" that was subsequently validated.

Another instance-the concept of continental drift, or plate tectonics-illustrates how science treats iconoclastic ideas, far outside accepted dogma, that could not be tested rigorously with the means available when the idea was first proposed, even though it was eventually shown to be correct. Without validation, scientific skepticism determined the fate of the concept until the key measurements could be made.

On the other hand, there are also clear instances in which the surprising result was shown to be erroneous. The most obvious recent example is the purported achievement of cold fusion. Its potential importance, had it been valid, helped stimulate many researchers to examine the work and, in a matter of months, to discredit it and show how the erroneous results came about.

In between are results that are potentially quite important and not strikingly inconsistent with accepted ideas. One is the recent report that element 118 had been observed, a result expected to be difficult to obtain, but consistent with ideas of nuclear structure. Such reports are certain to be examined critically, but not with the alacrity of a report of cold fusion. The self correction process of science inevitably and inexorably brings anyone who wants to build on previous work to do something that will test the correction of that work. While it may be a long time before anything appears to challenge the results, as soon as any apparent inconsistency arises, the challenge inevitably begins. This continual validation and insistence on consistency with previous knowledge is what insured the ability of science to make reliable quantitative predictions.

The examples discussed thus far have been those in which the reported new results have, correct or not, been presented by researchers who believed in their validity. The scientific community is now being asked to scrutinize cases of fraudulent claims of scientific results.

When any new result is presented to the scientific community, the tacit presumption is that it is honest. However, this is essentially irrelevant to the question of the validity of the substantive scientific result, which will regardless be subject to the standard validation processes that make science work. It is especially important to keep quite separate the question of how and whether science is successful at maintaining its self-validation, and the question of how to recognize and deal with misconduct.

Consider the following hypothetical case. A report appears of an experiment that is claimed to yield a rather striking result. Clever investigators carry out real experiments that show that the reported result is correct, while in the meantime, incontrovertible evidence comes to light that the initial report was based on an experiment that was never conducted. The initial claim was fraudulent, although the reported "result" was ultimately found to be correct.

The procedures that test the validity of scientific information can sometimes also test for malfeasance, evidenced by the recent case at Bell Laboratories. The identical noise distributions in two spectra, presented as independent and different, is simply inconsistent in precisely the sense that we use inconsistencies to test the validity of any scientific result. Because of the very nature of noise, we can think of only one way that the two noise distributions could be essentially identical-by their actually being two representations of the same spectrum. The extreme improbability of such an event carries with it a very strong implication of deliberate misrepresentation.

I believe that scientific self-correction is functioning and is indeed maintaining the validity of the body of scientific knowledge. It may occur slowly, but it does occur. It must occur if a result is to be used in building further science. On the other hand, the issue of maintaining ethical behavior and discouraging (and punishing) its opposite in the scientific enterprise is a different issue that does not have such an obvious resolution, and needs scrutiny and careful thought.

R. Stephen Berry is the James Franck Distinguished Service Professor of Chemistry at the University of Chicago. He is also the home secretary of the National Academy of Sciences.

APS encourages the redistribution of the materials included in this newspaper provided that attribution to the source is noted and the materials are not truncated or changed.

Editor: Alan Chodos
Associate Editor: Jennifer Ouellette