Skip to content
Archive of posts filed under the Zombies category.

Don’t calculate post-hoc power using observed estimate of effect size

Aleksi Reito writes: The statement below was included in a recent issue of Annals of Surgery: But, as 80% power is difficult to achieve in surgical studies, we argue that the CONSORT and STROBE guidelines should be modified to include the disclosure of power—even if less than 80%—with the given sample size and effect size […]

“Tweeking”: The big problem is not where you think it is.

In her recent article about pizzagate, Stephanie Lee included this hilarious email from Brian Wansink, the self-styled “world-renowned eating behavior expert for over 25 years”: OK, what grabs your attention is that last bit about “tweeking” the data to manipulate the p-value, where Wansink is proposing research misconduct (from NIH: “Falsification: Manipulating research materials, equipment, […]

Narcolepsy Could Be ‘Sleeper Effect’ in Trump and Brexit Campaigns

Kevin Lewis sent along this example of what in social science is called the “ecological fallacy.” Below is a press release that I’ve changed in only a few places: UNDER EMBARGO UNTIL MARCH 8, 2018 AT 10 AM EST Media Contact: Public and Media Relations Manager Society for Personality and Social Psychology press@spsp.org Narcolepsy Could […]

“Check out table 4.”

A colleague sent along this article and writes: Check out table 4. this is ERC funded research (the very best of European science get this money). OK, now I was curious, so I scrolled through to table 4. Here it is: Yup, it’s horrible. I don’t know that I’d call it cargo cult science at […]

“It’s Always Sunny in Correlationville: Stories in Science,” or, Science should not be a game of Botticelli

There often seems to be an attitude among scientists and journal editors that, if a research team has gone to the trouble of ensuring rigor in some part of their study (whether in the design, the data collection, or the analysis, but typically rigor is associated with “p less than .05” and some random assignment […]

“We continuously increased the number of animals until statistical significance was reached to support our conclusions” . . . I think this is not so bad, actually!

For some reason, people have recently been asking me what I think of this journal article which I wrote about months ago . . . so I’ll just repeat my post here: Jordan Anaya pointed me to this post, in which Casper Albers shared this snippet from a recently-published paper from an article in Nature […]

Some clues that this study has big big problems

Paul Alper writes: This article from the New York Daily News, reproduced in the Minneapolis Star Tribune, is so terrible in so many ways. Very sad commentary regarding all aspects of statistics education and journalism. The news article, by Joe Dziemianowicz, is called “Study says drinking alcohol is key to living past 90,” with subheading, […]

When anyone claims 80% power, I’m skeptical.

A policy analyst writes: I saw you speak at ** on Bayesian methods. . . . I had been asked to consult on a large national evaluation of . . . [details removed to preserve anonymity] . . . and had suggested treading carefully around the use of Bayesian statistics in this study (basing it […]

The scandal isn’t what’s retracted, the scandal is what’s not retracted.

Andrew Han at Retraction Watch reports on a paper, “Structural stigma and all-cause mortality in sexual minority populations,” published in 2014 by Mark Hatzenbuehler, Anna Bellatorre, Yeonjin Lee, Brian Finch, Peter Muennig, and Kevin Fiscella, that claimed: Sexual minorities living in communities with high levels of anti-gay prejudice experienced a higher hazard of mortality than […]

The competing narratives of scientific revolution

Back when we were reading Karl Popper’s Logic of Scientific Discovery and Thomas Kuhn’s Structure of Scientific Revolutions, who would’ve thought that we’d be living through a scientific revolution ourselves? Scientific revolutions occur on all scales, but here let’s talk about some of the biggies: 1850-1950: Darwinian revolution in biology, changed how we think about […]

Let’s get hysterical

Following up on our discussion of hysteresis in the scientific community, Nick Brown points us to this article from 2014, “Excellence by Nonsense: The Competition for Publications in Modern Science,” by Mathias Binswanger, who writes: To ensure the efficient use of scarce funds, the government forces universities and professors, together with their academic staff, to […]

It was the weeds that bothered him.

Bill Jefferys points to this news article by Denise Grady. Bill noticed the following bit, “In male rats, the studies linked tumors in the heart to high exposure to radiation from the phones. But that problem did not occur in female rats, or any mice,” and asked: ​Forking paths, much? My reply: The summary of […]

Jeremy Freese was ahead of the curve

Here’s sociologist Jeremy Freese writing, back in 2008: Key findings in quantitative social science are often interaction effects in which the estimated “effect” of a continuous variable on an outcome for one group is found to differ from the estimated effect for another group. An example I use when teaching is that the relationship between […]

Amelia, it was just a false alarm

Nah, jet fuel can’t melt steel beams. I’ve watched enough conspiracy documentaries – Camp Cope Some ideas persist long after the mounting evidence against them becomes overwhelming. Some of these things are kooky but probably harmless (try as I might, I do not care about ESP etc), whereas some are deeply damaging (I’m looking at you “vaccines […]

What happens to your career when you have to retract a paper?

In response to our recent post on retractions, Josh Krieger sends along two papers he worked on with Pierre Azoulay, Jeff Furman, Fiona Murray, and Alessandro Bonatti. Krieger writes, “Both papers are about the spillover effects of retractions on other work. Turns out retractions are great for identification!” Paper #1: “The career effects of scandal: […]

I think they use witchcraft

The following came in the email today: On Jul 7, 2018, at 12:58 PM, Submissions <submissions@**.co.in> wrote: Hello Dr. Andrew Gelman, I am Dr. ** [American-sounding name], Research Assistant for the ** Publishing Company contacting you with reference from our Editorial Board. Are you tired of publishing your Manuscript in useless journals and get no […]

Tutorial: The practical application of complicated statistical methods to fill up the scientific literature with confusing and irrelevant analyses

James Coyne pointed me with distress or annoyance to this new paper, “Tutorial: The Practical Application of Longitudinal Structural Equation Mediation Models in Clinical Trials,” by K. A. Goldsmith, D. P. MacKinnon, T. Chalder, P. D. White, M. Sharpe, and A. Pickles. This is the team behind the PACE trial for systemic exercise intolerance disease. […]

On this 4th of July, let’s declare independence from “95%”

Plan your experiment, gather your data, do your inference for all effects and interactions of interest. When all is said and done, accept some level of uncertainty in your conclusions: you might not be 97.5% sure that the treatment effect is positive, but that’s fine. For one thing, decisions need to be made. You were […]

The “Psychological Science Accelerator”: it’s probably a good idea but I’m still skeptical

Asher Meir points us to this post by Christie Aschwanden entitled, “Can Teamwork Solve One Of Psychology’s Biggest Problems?”, which begins: Psychologist Christopher Chartier admits to a case of “physics envy.” That field boasts numerous projects on which international research teams come together to tackle big questions. Just think of CERN’s Large Hadron Collider or […]

Josh “hot hand” Miller speaks at Yale tomorrow (Wed) noon

Should be fun.