Mind & behavior
A psychology journal published a case for precognition. Then came replication
Bem's unusual experiments are best read beside a precise attempt to repeat one of them. The disagreement is a useful lesson in what replication can establish.
Practice after the test
Normally, practice comes before a memory test. Daryl Bem's 2011 paper reversed that order. In one experiment, participants first saw words and recalled what they could; only afterward did a computer select some words for additional practice. Bem reported that words chosen for later practice had already been recalled more successfully.
If reliable, that pattern would be extraordinary: a future random selection would be associated with an earlier response in the predicted direction. The paper presented nine experiments using several such reversals of familiar psychological procedures. Its publication made a provocative claim available for examination, rather than settling the existence of precognition.
Repeating a particular claim
Stuart Ritchie, Richard Wiseman and Christopher French subsequently conducted three preregistered replications of Bem's ninth experiment. Their combined sample contained 150 participants. The experiments did not reproduce the reported recall advantage; their 2012 PLOS ONE paper reported a combined one-tailed p-value of 0.83.
They used Bem's software and an almost identical procedure, with disclosed changes such as British vocabulary substitutions. That specificity is important. They were testing one operational claim about later word practice and earlier recall, not every possible assertion about paranormal experience. A failure in this test is evidence against that effect under those conditions; it is not an experiment on every conceivable form of precognition.
Why decide the analysis first?
A preregistration records intended procedures before researchers inspect outcomes. Its purpose is to distinguish a planned test from a pattern discovered while looking through the results. Both activities can be valuable, but they answer different questions. A surprising pattern can suggest an idea; a fresh, prospectively specified test asks whether that idea survives another opportunity to be wrong.
This distinction becomes especially important when there are many plausible analyses. One can examine different subgroups, outcomes or exclusion rules. Even honest researchers can find something striking among many comparisons. Documenting the plan makes the amount of flexibility visible and gives later readers a fairer basis for judging a reported probability.
An archive should preserve the disagreement
Read the original experiment and the replication together. Compare the sample, software, timing, outcome definition and deviations before comparing their conclusions. A replication's title is not a substitute for those details, just as publication of the original was not a substitute for independent confirmation.
The useful intellectual move is to make the claim smaller and more testable. Instead of asking whether a journal proved that people can see the future, ask whether this particular procedure reliably predicts a specified difference in recall. That question leaves room for a decisive result, for methodological criticism and for further tests. It also prevents a single dramatic paper from becoming a permanent certainty merely because an archive preserves it.
Sources and further reading
- Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect ↗
Feeling the Future; Experiments 8–9, retroactive facilitation of recall
- Ritchie, Wiseman and French, Failing the Future (2012) ↗
Abstract; Introduction; Methods; Results