Reconstruction of a Train Wreck: How Priming Research Went off the Rails

The concept of “priming” has gained a lot of adherents in psychological research, but is it real? If I lead a group of students to think about the challenges faced by the elderly, do the students walk more slowly because of that “priming?” Several independent studies have produced significant p-values and were published in high-impact journals. As is common in research, however, these studies used relatively few subjects. A recent meta-analysis that attempts to combine the data among these studies, however, concludes that the significant findings in the individual studies were illusory. This negative result was published to relatively little fanfare.

The example from “priming” is illustrative of a greater problem for reproducible research.  Under-powered studies are the norm. Academic journals have a clear bias toward positive findings, and negative results frequently fail to be published!

Replicability-Index

Authors:  Ulrich Schimmack, Moritz Heene, and Kamini Kesavan

Abstract:
We computed the R-Index for studies cited in Chapter 4 of Kahneman’s book “Thinking Fast and Slow.” This chapter focuses on priming studies, starting with John Bargh’s study that led to Kahneman’s open email.  The results are eye-opening and jaw-dropping.  The chapter cites 12 articles and 11 of the 12 articles have an R-Index below 50.  The combined analysis of 31 studies reported in the 12 articles shows 100% significant results with average (median) observed power of 57% and an inflation rate of 43%.  The R-Index is 14. This result confirms Kahneman’s prediction that priming research is a train wreck and readers of his book “Thinking Fast and Slow” should not consider the presented studies as scientific evidence that subtle cues in their environment can have strong effects on their behavior outside their awareness.

Introduction

In 2011, Nobel Laureate Daniel Kahneman…

View original post 3,684 more words

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s