Skip to content
Archive of posts filed under the Zombies category.

When anyone claims 80% power, I’m skeptical.

A policy analyst writes: I saw you speak at ** on Bayesian methods. . . . I had been asked to consult on a large national evaluation of . . . [details removed to preserve anonymity] . . . and had suggested treading carefully around the use of Bayesian statistics in this study (basing it […]

The scandal isn’t what’s retracted, the scandal is what’s not retracted.

Andrew Han at Retraction Watch reports on a paper, “Structural stigma and all-cause mortality in sexual minority populations,” published in 2014 by Mark Hatzenbuehler, Anna Bellatorre, Yeonjin Lee, Brian Finch, Peter Muennig, and Kevin Fiscella, that claimed: Sexual minorities living in communities with high levels of anti-gay prejudice experienced a higher hazard of mortality than […]

The competing narratives of scientific revolution

Back when we were reading Karl Popper’s Logic of Scientific Discovery and Thomas Kuhn’s Structure of Scientific Revolutions, who would’ve thought that we’d be living through a scientific revolution ourselves? Scientific revolutions occur on all scales, but here let’s talk about some of the biggies: 1850-1950: Darwinian revolution in biology, changed how we think about […]

Let’s get hysterical

Following up on our discussion of hysteresis in the scientific community, Nick Brown points us to this article from 2014, “Excellence by Nonsense: The Competition for Publications in Modern Science,” by Mathias Binswanger, who writes: To ensure the efficient use of scarce funds, the government forces universities and professors, together with their academic staff, to […]

It was the weeds that bothered him.

Bill Jefferys points to this news article by Denise Grady. Bill noticed the following bit, “In male rats, the studies linked tumors in the heart to high exposure to radiation from the phones. But that problem did not occur in female rats, or any mice,” and asked: ​Forking paths, much? My reply: The summary of […]

Jeremy Freese was ahead of the curve

Here’s sociologist Jeremy Freese writing, back in 2008: Key findings in quantitative social science are often interaction effects in which the estimated “effect” of a continuous variable on an outcome for one group is found to differ from the estimated effect for another group. An example I use when teaching is that the relationship between […]

Amelia, it was just a false alarm

Nah, jet fuel can’t melt steel beams. I’ve watched enough conspiracy documentaries – Camp Cope Some ideas persist long after the mounting evidence against them becomes overwhelming. Some of these things are kooky but probably harmless (try as I might, I do not care about ESP etc), whereas some are deeply damaging (I’m looking at you “vaccines […]

What happens to your career when you have to retract a paper?

In response to our recent post on retractions, Josh Krieger sends along two papers he worked on with Pierre Azoulay, Jeff Furman, Fiona Murray, and Alessandro Bonatti. Krieger writes, “Both papers are about the spillover effects of retractions on other work. Turns out retractions are great for identification!” Paper #1: “The career effects of scandal: […]

I think they use witchcraft

The following came in the email today: On Jul 7, 2018, at 12:58 PM, Submissions <submissions@**.co.in> wrote: Hello Dr. Andrew Gelman, I am Dr. ** [American-sounding name], Research Assistant for the ** Publishing Company contacting you with reference from our Editorial Board. Are you tired of publishing your Manuscript in useless journals and get no […]

Tutorial: The practical application of complicated statistical methods to fill up the scientific literature with confusing and irrelevant analyses

James Coyne pointed me with distress or annoyance to this new paper, “Tutorial: The Practical Application of Longitudinal Structural Equation Mediation Models in Clinical Trials,” by K. A. Goldsmith, D. P. MacKinnon, T. Chalder, P. D. White, M. Sharpe, and A. Pickles. This is the team behind the PACE trial for systemic exercise intolerance disease. […]

On this 4th of July, let’s declare independence from “95%”

Plan your experiment, gather your data, do your inference for all effects and interactions of interest. When all is said and done, accept some level of uncertainty in your conclusions: you might not be 97.5% sure that the treatment effect is positive, but that’s fine. For one thing, decisions need to be made. You were […]

The “Psychological Science Accelerator”: it’s probably a good idea but I’m still skeptical

Asher Meir points us to this post by Christie Aschwanden entitled, “Can Teamwork Solve One Of Psychology’s Biggest Problems?”, which begins: Psychologist Christopher Chartier admits to a case of “physics envy.” That field boasts numerous projects on which international research teams come together to tackle big questions. Just think of CERN’s Large Hadron Collider or […]

Josh “hot hand” Miller speaks at Yale tomorrow (Wed) noon

Should be fun.

Nooooooooooooooooo! (dentists named Dennis, still appearing in 2018)

From today’s NYT: Another finding of note, published in the Journal of Personality and Social Psychology in 2002, is that people gravitate toward places of residence and occupations that resemble their own names. So, the researchers assert, a higher proportion of men named Louis live in St. Louis than would occur at random, and a […]

All Fools in a Circle

A graduate student in psychology writes: Grants do not fund you unless you have pilot data – and moreover – show some statistically significant finding in your N of 20 or 40 – in essence trying to convince the grant reviewers that there is “something there” worth them providing your lab lots of money to […]

“Not statistically significant” != 0, stents edition

Doug Helmreich writes: OK, I work at a company that is involved in stents, so I’m not unbiased, but… http://www3.imperial.ac.uk/newsandeventspggrp/imperialcollege/newssummary/news_2-11-2017-15-52-46 and especially https://www.nytimes.com/2017/11/02/health/heart-disease-stents.html The research design is pretty cool—placebo participants got a sham surgery with no stent implanted. The results show that people with the stent did have better metrics than those with just the […]

Some experiments are just too noisy to tell us much of anything at all: Political science edition

Sointu Leikas pointed us to this published research article, “Exposure to inequality affects support for redistribution.” Leikas writes that “it seems to be a really apt example of “researcher degrees of freedom.’” Here’s the abstract of the paper: As the world’s population grows more urban, encounters between members of different socioeconomic groups occur with greater […]

Write your congressmember to require researchers to publicly post their code?

Stephen Cranney writes: For the past couple of years I have had an ongoing question/concern . . . In my fields (sociology and demography) much if not most of the published research is based on publicly available datasets; consequently, replicability is literally a simple matter of sending or uploading a few kilobytes of code text. […]

No, there is no epidemic of loneliness. (Or, Dog Bites Man: David Brooks runs another column based on fake stats)

[adorable image] Remember David Brooks? The NYT columnist, NPR darling, and former reporter who couldn’t correctly report the price of a meal at Red Lobster? The guy who got it wrong about where billionaires come from and who thought it was fun to use one of his columns to make fun of a urologist (ha […]

“Eureka bias”: When you think you made a discovery and then you don’t want to give it up, even if it turns out you interpreted your data wrong

This came in the email one day: I am writing to you with my own (very) small story of error-checking a published finding. If you end up posting any of this, please remove my name! A few years ago, a well-read business journal published an article by a senior-level employee at my company. One of […]