Someone sent me this recent article, “Embodying Power: A Preregistered Replication and Extension of the Power Pose Effect,” by Katie Garrison, David Tang, and Brandon Schmeichel.
Unsurprisingly (given that the experiment was preregistered), the authors found no evidence for any effect of power pose.
The Garrison et al. paper is reasonable enough, but for my taste they aren’t explicit enough about the original “power pose” paper being an exercise in noise mining. They do say, “Another possible explanation for the nonsignificant effect of power posing on risk taking is that power posing does not influence risk taking,” but this only appears 3 paragraphs into their implications section, and they never address the question: If power pose has no effect, how did Carney et al. get statistical significance, publication in a top journal, fame, fortune, etc.? The garden of forking paths is the missing link in this story. (In that original paper, Carney et al. had many, many “researcher degrees of freedom” which would allow them to find “p less than .05” even from data produced by pure noise.)
It’s also not clear what makes Garrison et al. conclude, “We believe future research should continue to explore eye gaze in combination body posture when studying the embodiment of power.” If power pose really has no effect (or, more precisely, highly unstable and situation-dependent effects), why is it worth future research at all? At the very least, any future research should consider measurement issues much more carefully.
Perhaps Garrison et al. were just trying to be charitable to Carney et al. and say, Hey, maybe you really did luck into a real finding. Or perhaps psychology journals simply will not allow you to be explicitly say that a published paper in a top journal is nothing but noise mining. Or maybe they thought it more politically savvy to state their conclusions in a subtle way and let readers draw the inference that the original study by Carney et al. was consistent with pure noise.
The downside of subtle politeness
Whatever the reason, I find the sort of subtlety shown by Garrison et al. to be frustrating. For people like me, and the person who sent the article to me, it’s clear what Garrison et al. are saying—no evidence for power pose, and the original study can be entirely discounted. But less savvy readers might not know the code; they might take the paper’s words literally and think that “social context plays a key role in power pose effects, and the current experiment lacked a meaningful social context” (a theory that Garrison et al. discuss before bringing up the “power posing does not influence risk taking” theory).
That would be too bad, if these researchers went to the trouble of doing a new study, writing it up, and getting it published, only to have drawn their conclusions so subtly that readers could miss the point.
Mulligan after mulligan
You may wonder why I continue to pick on power pose. It’s still one of the most popular Ted talks of all time, featured on NPR etc etc etc. So, yeah, people are taking it seriously. One could make the argument that power pose is innocuous, maybe beneficial in that it is a way of encouraging people to take charge of their lives. And this may be so. Even if power pose itself is meaningless, the larger “power pose” story could be a plus. Of course, if power pose is just an inspirational story to empower people, it doesn’t have to be true, or replicable, or scientifically valid, or whatever. From that perspective, power pose lies outside science entirely, and to criticize power pose would be a sort of category error, like criticizing The Lord of the Rings on the grounds that there’s no such thing as an invisibility ring, or criticizing The Rotter’s Club on the grounds that Jonathan Coe was just making it all up. I guess I’d prefer, if business school professors want to tell inspirational stories without any scientific basis, that they label them more clearly as parables, rather than dragging the scientific field of psychology into it. And I’d prefer if scientific psychologists didn’t give mulligan after mulligan to theories like power pose, just because they’re inspirational and got published with p less than .05.
I don’t care about power pose. It’s just a silly fad. I do care about reality, and I care about science, which is one of the methods we have for learning about reality. The current system of scientific publication, in which a research team can get fame, fortune, and citations by p-hacking, and then even when later research groups fail to replicate the study, that even then there is the continuing push to credit the original work and to hypothesize mysterious interaction effects that would manage to preserve everyone’s reputation . . . it’s a problem.
It’s Ptolemy, man, that’s what it is. [No, it’s not Ptolemy; see Ethan’s comment below.]
P.S. I wrote this post months ago, it just happens to be appearing now, at a time in which we’re talking a lot about the replication crisis.