The concept of “priming” has gained a lot of adherents in psychological research, but is it real? If I lead a group of students to think about the challenges faced by the elderly, do the students walk more slowly because of that “priming?” Several independent studies have produced significant p-values and were published in high-impact journals. As is common in research, however, these studies used relatively few subjects. A recent meta-analysis that attempts to combine the data among these studies, however, concludes that the significant findings in the individual studies were illusory. This negative result was published to relatively little fanfare.
The example from “priming” is illustrative of a greater problem for reproducible research. Under-powered studies are the norm. Academic journals have a clear bias toward positive findings, and negative results frequently fail to be published!
Authors: Ulrich Schimmack, Moritz Heene, and Kamini Kesavan
We computed the R-Index for studies cited in Chapter 4 of Kahneman’s book “Thinking Fast and Slow.” This chapter focuses on priming studies, starting with John Bargh’s study that led to Kahneman’s open email. The results are eye-opening and jaw-dropping. The chapter cites 12 articles and 11 of the 12 articles have an R-Index below 50. The combined analysis of 31 studies reported in the 12 articles shows 100% significant results with average (median) observed power of 57% and an inflation rate of 43%. The R-Index is 14. This result confirms Kahneman’s prediction that priming research is a train wreck and readers of his book “Thinking Fast and Slow” should not consider the presented studies as scientific evidence that subtle cues in their environment can have strong effects on their behavior outside their awareness.
In 2011, Nobel Laureate Daniel Kahneman…
View original post 3,684 more words