So. I was reading the newspaper the other day and came across a credulous review of the recent book by Amy “Power Pose” Cuddy. The review, by Heather Havrilesky, expressed some overall wariness regarding the self-help genre, but I was disappointed to see no skepticism regarding Cuddy’s scientific claims. And then I did a web search and found a completely credulous CBS News report: “Believe it or not, her studies show that if you stand like a superhero privately before going into a stressful situation, there will actually be hormonal changes in your body chemistry that cause you to be more confident and in-command . . . make no mistake, Cuddy’s work is grounded in science.”
Actually Cuddy’s claims were iffy from the start and suffer a credibility gap, given the failure of a large-scale replication of her key experiment, as discussed a few months ago here under the clever title “Low-power pose” and in careful detail by Joe Simmons and Uri Simonsohn on their blog.
This all inspired me to write, with Kaiser Fung, an article for Slate exploring the mismatch between what one might call external and internal views of science:
– For outsiders, people who read the New York Times or Slate or Malcolm Gladwell or Freakonomics who tune into Ted talks, science is a string of stunning findings by heroic scientists, daring to think outside the box.
– But when insiders see hyped findings about himmicanes or college men with fat arms or ESP or sex ratios of beautiful parents or wobbly stools or embodied cognition or power pose, we laugh or we sigh (depending on our mood), knowing that one more bit of junk science got through the filter.
This is not to say that none of the effects being talked about are real, just that the studies tend to be too noisy to tell us anything useful, and we know by now the problems of creativeness.
The insider-outsider distinction is not always so clear: Daryl Bem and Ellen Langer are Ivy League professors, after all, and even the much-mocked Satoshi Kanazawa teaches at the respected London School of Economics. But all three of these researchers are outsiders when it comes to facing the statistical crisis in science.
Anyway, the focus of our Slate article was the yawning gap, as we put it, between the news media, science celebrities, and publicists on one side, and the general scientific community on the other. Various exceptions aside, it’s my impression that most scientists are a bit embarrassed by headline-grabbing claims on gay genes or ovulation and voting or whatever: We know that Science and Nature and PPNAS sometimes like publishing such papers, and that they can get lots of press, but we don’t take it seriously.
Meanwhile, your ordinary civilian Gladwell-readers can get the impression that these flashy findings are what science is all about.
But . . . then I read the comments on our Slate piece. And what struck me is that nobody came to the defense of the power-pose researchers. But it wasn’t even that. Even more striking was that none of the Slate commenters seemed to take that study seriously in the first place. It wasn’t like: Hey, this is interesting, that much-touted power pose study was in error. It was like: Yeah, what a joke, who’d ever think that that could make sense.
This is good news: Despite all the influence of the New York Times, CBS News, NPR (yes, of course, NPR too), Amy Cuddy’s publisher, and Harvard Business School, still, after all that, 100 Slate readers assume it’s all a scam. That’s good to hear. All the king’s horses etc.
Anyway, I’m not sure what to make of this division between the gullibles and the skeptics. On one side you have the NYT, CBS News, NPR, Science, Nature, PPNAS, Malcolm Gladwell, a major book publisher, the publicity department of the Harvard Business School, and the TED organization (whoever they are). On the other side, Eva Ranehill, Anna Dreber, Chris Chabris, Kaiser Fung, Uri Simonsohn, E. J. Wagenmakers, Arina K. Bones, me, . . . and several dozen random people who write in the comments section of Slate.
P.S. Just to be clear: I don’t think this is a debate about personalities and I’m not trying to personalize this. I’ve never met Amy Cuddy or her coauthors, or, for that matter, Eva Ranehill or any of her coauthors on the paper that reported the non-replication of the power-pose finding. I’ve never met Daryl Bem or Ellen Langer or Satoshi Kanazawa or Malcolm Gladwell either. It’s not about good guys and bad guys. It’s about different experiences and different perspectives. In this case, I was interested to see that these Slate readers had an ambient level of skepticism which actually in this case gave them a clearer perspective than that of NPR editors etc. (I can’t really speak to the sophistication of the Ted talk organizers because maybe they know this is iffy science but they’re hooked on the clicks, I have no idea.)