Skip to content
Archive of posts filed under the Zombies category.

Reasons for an optimistic take on science: there are not “growing problems with research and publication practices.” Rather, there have been, and continue to be, huge problems with research and publication practices, but we’ve made progress in recognizing these problems.

Javier Benitez points us to an article by Daniele Fanelli, “Is science really facing a reproducibility crisis, and do we need it to?”, published in the Proceedings of the National Academy of Sciences, which begins: Efforts to improve the reproducibility and integrity of science are typically justified by a narrative of crisis, according to which […]

I fear that many people are drawing the wrong lessons from the Wansink saga, focusing on procedural issues such as “p-hacking” rather than scientifically more important concerns about empty theory and hopelessly noisy data. If your theory is weak and your data are noisy, all the preregistration in the world won’t save you.

Someone pointed me to this news article by Tim Schwab, “Brian Wansink: Data Masseur, Media Villain, Emblem of a Thornier Problem.” Schwab writes: If you look into the archives of your favorite journalism outlet, there’s a good chance you’ll find stories about Cornell’s “Food Psychology and Consumer Behavior” lab, led by marketing researcher Brian Wansink. […]

“and, indeed, that my study is consistent with X having a negative effect on Y.”

David Allison shares this article: Pediatrics: letter to the editor – Metformin for Obesity in Prepubertal and Pubertal Children A Randomized Controlled Trial and the authors’ reply: RE: Clarification of statistical interpretation in metformin trial paper The authors of the original paper were polite in their response, but they didn’t seem to get the point […]

No, I don’t believe that “Reduction in Firearm Injuries during NRA Annual Conventions” story

David Palmer writes: If you need yet another study to look at, check this out: “Reduction in Firearm Injuries during NRA Annual Conventions.”

Concerns about Brian Wansink’s claims and research methods have been known for years

1. The king and his memory There’s this stunning passage near the end of Josephine Tey’s classic The Daughter of Time. Most of the book is taken up with the main characters laboriously discovering the evidence that Richard III was not really a bad guy, he didn’t really kill those little princes, etc. Having made […]

I fear that many people are drawing the wrong lessons from the Wansink saga, focusing on procedural issues such as “p-hacking” rather than scientifically more important concerns about empty theory and hopelessly noisy data. If your theory is weak and your data are noisy, all the preregistration in the world won’t save you.

This came up in the discussion of yesterday’s post. We’ve discussed theory and measurement in this space before. And here’s a discussion of how the problems of selection bias are magnified when measurements are noisy. Forking paths and p-hacking do play a role in this story: forking paths (multiple potential analyses on a given experiment) […]

Big Oregano strikes again

Paul Alper writes: You recall the University of Maryland chocolate milk cure for concussion [Bigmilk Strikes Again]. A new version of the same sloppiness is discussed here. Alper is linking to a news article, “University of Iowa ignores questions about its oregano ‘cure’ for cancer-wasting syndrome,” by Eric Holland, who writes: At the beginning of […]

Anybody want a drink before the war?

Your lallies look like darts, and you’ve got nanti carts, but I love your bona eke – Lee Sutton (A near miss) I’ve been thinking about gayface again. I guess this is for a bunch of reasons, but one of the lesser ones is that this breathless article by JD Schramm popped up in the Washington Post the other […]

Zoologist slams statistical significance

Valentin Amrhein writes that statistical significance and hypothesis testing are not really helpful when it comes to testing our hypotheses. I’m not quite sure I like the title of Amrhein’s post—“Inferential Statistics is not Inferential”—as I think of parameter estimation, model checking, forecasting, etc., all as forms of inference. But I agree with his general […]

Low power and the replication crisis: What have we learned since 2004 (or 1984, or 1964)?

I happened to run across this article from 2004, “The Persistence of Underpowered Studies in Psychological Research: Causes, Consequences, and Remedies,” by Scott Maxwell and published in the journal Psychological Methods. In this article, Maxwell covers a lot of the material later discussed in the paper Power Failure by Button et al. (2013), and the […]

The Lab Where It Happens

“Study 1 was planned in 2007, but it was conducted in the Spring of 2008 shortly after the first author was asked to take a 15-month leave-of-absence to be the Executive Director for USDA’s Center for Nutrition Policy and Promotion in Washington DC. . . . The manuscript describing this pair of studies did not […]

Hysteresis corner: “These mistakes and omissions do not change the general conclusion of the paper . . .”

All right, then. The paper’s called Attractive Data Sustain Increased B.S. Intake in Journals Attractive Names Sustain Increased Vegetable Intake in Schools. Seriously, though, this is just an extreme example of a general phenomenon, which we might call scientific hysteresis or the research incumbency advantage: When you’re submitting a paper to a journal, it can […]

I’m skeptical of the claims made in this paper

Two different people pointed me to a recent research article, suggesting that the claims therein were implausible and the result of some combination of forking paths and spurious correlations—that is, there was doubt that the results would show up in a preregistered replication, and that, if they did show up, that they would mean what […]


Gary Smith sends along this news article from Jason Samenow, weather editor of the Washington Post, who writes: Three years ago, a scientific study claimed that storms named Debby are predisposed to kill more people than storms named Don. The study alleged that people don’t take female-named storms as seriously. Numerous analyses have since found […]

The Paper of My Enemy Has Been Retracted

The paper of my enemy has been retracted And I am pleased. From every media outlet it has been retracted Like a van-load of p-values that has been seized And sits in star-laden tables in a replication archive, My enemy’s much-prized effort sits in tables In the kind of journal where retraction occurs. Great, square […]

A lesson from the Charles Armstrong plagiarism scandal: Separation of the judicial and the executive functions

[updated link] Charles Armstrong is a history professor at Columbia University who, so I’ve heard, has plagiarized and faked references for an award-winning book about Korean history. The violations of the rules of scholarship were so bad that the American Historical Association “reviewed the citation issue after being notified by a member of the concerns […]

The puzzle: Why do scientists typically respond to legitimate scientific criticism in an angry, defensive, closed, non-scientific way? The answer: We’re trained to do this during the process of responding to peer review.

[image of Cantor’s corner] Here’s the “puzzle,” as we say in social science. Scientific research is all about discovery of the unexpected: to do research, you need to be open to new possibilities, to design experiments to force anomalies, and to learn from them. The sweet spot for any researcher is at Cantor’s corner. (See […]

The retraction paradox: Once you retract, you implicitly have to defend all the many things you haven’t yet retracted

Mark Palko points to this news article by Beth Skwarecki on Goop, “the Gwyneth Paltrow pseudoscience empire.” Here’s Skwarecki: When Goop publishes something weird or, worse, harmful, I often find myself wondering what are they thinking? Recently, on Jimmy Kimmel, Gwyneth laughed at some of the newsletter’s weirder recommendations and said “I don’t know what […]

Alzheimer’s Mouse research on the Orient Express

Paul Alper sends along an article from Joy Victory at Health News Review, shooting down a bunch of newspaper headlines (“Extra virgin olive oil staves off Alzheimer’s, preserves memory, new study shows” from USA Today, the only marginally better “Can extra-virgin olive oil preserve memory and prevent Alzheimer’s?” from the Atlanta Journal-Constitution, and the better […]

It’s . . . spam-tastic!

We’ll celebrate Christmas today with a scam that almost fooled me. OK, not quite: I was about two steps from getting caught. Here’s the email: Dear Dr. Gelman, I hope you do not mind me emailing you directly, I thought it would be the easiest way to make first contact. If you have time for […]