Blast from the past

Lizzie told me about this paper, “Bidirectionality, Mediation, and Moderation of Metaphorical Effects: The Embodiment of Social Suspicion and Fishy Smells,” which reports:

As expected (see Figure 1), participants who were exposed to incidental fishy smells invested less money (M = $2.53, SD = $0.93) than those who were exposed to odorless water (M = $3.34, SD = $1.02), planned contrast i(42) = 2.07, p = .05, Cohen’s d = 0.83, or fart spray (M = $3.38, SD = $1.23), i(42) = 2.22, p = .03, d = 0.78.

Fart spray!

The paper faithfully follows Swann’s 18 rules for success in social priming research.

I was surprised to see that people were still doing this sort of thing . . . but then I looked at the paper more carefully. Journal of Personality and Social Psychology, 2012. Just one year after they’d published that ESP paper. To criticize a psychology journal for publishing this sort of thing in 2012 would be like mocking someone for sporting a mullet in the 1980s.

Of course, just cos a paper is on a funny topic and just cos it follows the cargo-cult-science template, it doesn’t mean that it is wrong. I guess I’ll believe it when I see a preregistered replication, not before. In the meantime, just recall that experimental results can be statistically significant and look super-clean but still not replicate. The garden of forking paths is not just a slogan, it’s a real thing that can easily lead researchers to fool themselves; hence the need to be careful.

P.S. All that said, it’s still not as bad as “Low glucose relates to greater aggression in married couples”: that’s the study where they had people blasting their spouses with loud noises and sticking pins into voodoo dolls.

These issues also arise with published research on more important topics.