Skip to content
Archive of posts filed under the Zombies category.

When considering proposals for redefining or abandoning statistical significance, remember that their effects on science will only be indirect!

John Schwenkler organized a discussion on this hot topic, featuring posts by – Dan Benjamin, Jim Berger, Magnus Johannesson, Valen Johnson, Brian Nosek, and E. J. Wagenmakers – Felipe De Brigard – Kenny Easwaran – Andrew Gelman and Blake McShane – Kiley Hamlin – Edouard Machery – Deborah Mayo – “Neuroskeptic” – Michael Strevens – […]

Alan Sokal’s comments on “Abandon Statistical Significance”

The physicist and science critic writes: I just came across your paper “Abandon statistical significance”. I basically agree with your point of view, but I think you could have done more to *distinguish* clearly between several different issues: 1) In most problems in the biomedical and social sciences, the possible hypotheses are parametrized by a […]

The “fish MRI” of international relations studies.

Kevin Lewis pointed me to this paper by Stephen Chaudoin, Jude Hays and Raymond Hicks, “Do We Really Know the WTO Cures Cancer?”, which begins: This article uses a replication experiment of ninety-four specifications from sixteen different studies to show the severity of the problem of selection on unobservables. Using a variety of approaches, it […]

Automated Inference on Criminality Using High-tech GIGO Analysis

Yee Whye Teh writes: You might be interested in this article. My reply was that this is just a big joke, one more bit of hype over a bunch of correlations. Lots of obvious problems with this paper, and it’s too bad that journalists fell for it. And, as an MIT grad, I’m particularly sad […]

“Cheerleading with an agenda: how the press covers science”

Yarden Katz writes: I thought you might be interested in this new (critical) perspective on science journalism: Cheerleading with an agenda: how the press covers science. It’s a topic far less urgent than the election (but related to the broader press failures that have been very visible in politics). My reply: This is an excellent […]

Further evidence that creativity and innovation are stimulated by college sports: Evidence from a big regression

Kevin Lewis sent along this paper from the Creativity Research Journal: Further Evidence that Creativity and Innovation are Inhibited by Conservative Thinking: Analyses of the 2016 Presidential Election The investigation replicated and extended previous research showing a negative relationship between conservatism and creative accomplishment. Conservatism was estimated, as in previous research, from voting patterns. The […]

Air rage update

So. Marcus Crede, Carol Nickerson, and I published a letter in PPNAS criticizing the notorious “air rage” article. (Due to space limitations, our letter contained only a small subset of the many possible criticisms of that paper.) Our letter was called “Questionable association between front boarding and air rage.” The authors of the original paper, […]

It’s not enough to be a good person and to be conscientious. You also need good measurement. Cargo-cult science done very conscientiously doesn’t become good science, it just falls apart from its own contradictions.

Kevin Lewis points us to a biology/psychology paper that was a mix of reasonable null claims (on the order of, the data don’t give us enough information to say anything about XYZ) and some highly questionable noise mining supported by p-values and forking paths. The whole thing is just so sad. The researchers are aware […]

p less than 0.00000000000000000000000000000000 . . . now that’s what I call evidence!

I read more carefully the news article linked to in the previous post, which describes a forking-pathed nightmare of a psychology study, the sort of thing that was routine practice back in 2010 or so but which we’ve mostly learned to at least try to avoid. Anyway, one thing I learned there’s something called “terror […]

As if the 2010s never happened

E. J. writes: I’m sure I’m not the first to send you this beauty. Actually, E. J., you’re the only one who sent me this! It’s a news article, “Can the fear of death instantly make you a better athlete?”, reporting on a psychology experiment: For the first study, 31 male undergraduates who liked basketball […]

It seemed to me that most destruction was being done by those who could not choose between the two

Amateurs, dilettantes, hacks, cowboys, clones — Nick Cave [Note from Dan 11Sept: I wanted to leave some clear air after the StanCon reminder, so I scheduled this post for tomorrow. Which means you get two posts (one from me, one from Andrew) on this in two days. That’s probably more than the gay face study deserves.] […]

Selection bias in the reporting of shaky research: An example

On 30 Dec 2016, a reporter wrote: I was wondering if you’d have some time to look at an interesting embargoed study coming out next week in JAMA Internal Medicine, which seeks to show that gun violence is a social contagion. I know that a few years ago, social contagion studies were controversial and I’m […]

Too much backscratching and happy talk: Junk science gets to share in the reputation of respected universities

Nick Stevenson writes: I agree that it’s disappointing that so many publications that pride themselves on the quality of their journalism – NYTimes, WashPo, Slate, Vox – ran with the EIP’s work, but does the fault really lie with them? This work has been promoted on the conference circuit for years by a full professor […]

Fake polls. Not new.

Mark Palko points me to this article by Harry Enten about a possibly nonexistent poll that was promoted by an organization or group or website called Delphi Analytica. Enten conjectures that the reported data were not fabricated but they’re not a serious poll either but rather some raw undigested output from a Google poll. This […]

“From that perspective, power pose lies outside science entirely, and to criticize power pose would be a sort of category error, like criticizing The Lord of the Rings on the grounds that there’s no such thing as an invisibility ring, or criticizing The Rotter’s Club on the grounds that Jonathan Coe was just making it all up.”

From last year: One could make the argument that power pose is innocuous, maybe beneficial in that it is a way of encouraging people to take charge of their lives. And this may be so. Even if power pose itself is meaningless, the larger “power pose” story could be a plus. Of course, if power […]

Sucker MC’s keep falling for patterns in noise

Mike Spagat writes: Apologies if forty people just sent this to you but maybe it’s obscure enough that I’m the first. It’s a news article by Irina Ivanova entitled, “‘Very unattractive’ workers can out-earn pretty people, study finds.” Spagat continues: You may be able to recognize a pattern here: Tiny, noisy sample Surprise result Journal […]

“Babbage was out to show that not only was the system closed, with a small group controlling access to the purse strings and the same individuals being selected over and again for the few scientific honours or paid positions that existed, but also that one of the chief beneficiaries . . . was undeserving.”

Fernando Martel Garcia writes: Here’s an early reference from the Victorian Age. Enjoy! It’s a news article by Rebekah Higgitt called “Fraud and the decline of science,” subtitled, “Charles Babbage’s accusations of fraudulent science underlined his attack on scientific governance, but were also bitterly personal.” My reply: Wow! I think I’m on Babbage’s side on […]

Just google “Despite limited statistical power”

Here it is. It’s not always clear what people mean by this expression, but sometimes it seems that they’re making the “What does not kill my statistical significance makes it stronger” fallacy, thinking that the attainment of statistical significance is a particular feat in the context of a noisy study, so that they’re (mistakenly) thinking […]

Also holding back progress are those who make mistakes and then label correct arguments as “nonsensical.”

Here’s James Heckman in 2013: Also holding back progress are those who claim that Perry and ABC are experiments with samples too small to accurately predict widespread impact and return on investment. This is a nonsensical argument. Their relatively small sample sizes actually speak for — not against — the strength of their findings. Dramatic […]

I love when I get these emails!

On Jan 27, 2017, at 12:24 PM, ** wrote: Hi Andrew, I hope you are well. I work for ** and we are looking to chat to someone who knows about Freud – I read that you used to be an expert in Freud? Is that correct? Background here.