Two Reviewers, One Dataset, Two Reports

The 1995 evaluation is the cleanest illustration in the whole subject of how far honest people can diverge on identical material, and it survives because both reports were published rather than merged. Utts and Ray Hyman were given the same body of remote viewing trials from the government programme. Utts concluded that psychic functioning had been established by the data, that the effect had been replicated at independent laboratories, and that further work should move from proving existence to understanding mechanism.

Hyman concluded that the effects were statistically real but that the studies shared design features which could produce exactly such effects without any anomalous transfer of information, and that a consistent artefact is indistinguishable from a consistent phenomenon until someone varies the design. Both later published their reports in the Journal of Scientific Exploration, and each wrote a short response to the other. Neither changed position. The disagreement is not about arithmetic, on which they largely agree, but about what unexplained regularity in data licenses you to conclude, and that is a question about inference rather than about statistics. Readers should note that Utts is a living person with a continuing academic career, and that this article describes her published positions and nothing else.