The Result Gets Filed Before the Data Does

Pre-registration methodology requires a researcher to publicly file a study's hypotheses, sample size and planned analysis before collecting or examining the data, closing off the common but usually invisible practice of trying many analyses after the fact and reporting only the one that produced a striking result. The Center for Open Science launched its Open Science Framework in 2013, building the infrastructure and the norm that let pre-registration move from an occasional practice into a standard expectation across psychology and related behavioral fields, following mounting evidence that a large share of published findings in those fields could not be reproduced by independent replication attempts.

Studies claiming a measurable effect from prayer, meditation or ritual practice are exactly the kind of research most vulnerable to the after the fact selection pre-registration is designed to prevent, since a large enough set of measured outcomes will almost always produce some statistically significant result by chance alone if a researcher is free to choose which outcome to report afterward. The atlas weights a pre-registered finding on religious practice more heavily than an equivalent unregistered one, and notes explicitly when a widely cited claim about a practice's measurable effects was never pre-registered.