Skip to content
Archive of entries posted by

A possible defense of cargo cult science?

Someone writes: I’ve been a follower of your blog and your continual coverage of “cargo cult science”. Since this type of science tends to be more influential and common than the (idealized) non-“cargo cult” stuff, I’ve been trying to find ways of reassuring myself that this type of science isn’t a bad thing (because if […]

Learn by experimenting!

A students wrote in one of his homework assignments: Sidenote: I know some people say you’re not supposed to use loops in R, but I’ve never been totally sure why this is (a speed thing?). My first computer language was Java, so my inclination is to think in loops before using some of the other […]

The all-important distinction between truth and evidence

Yesterday we discussed a sad but all-too-familiar story of a little research project that got published and hyped beyond recognition. The published paper was called, “The more you play, the more aggressive you become: A long-term experimental study of cumulative violent video game effects on hostile expectations and aggressive behavior,” but actually that title was […]

More bad news in the scientific literature: A 3-day study is called “long term,” and nobody even seems to notice the problem. Whassup with that??

Someone pointed me to this article, “The more you play, the more aggressive you become: A long-term experimental study of cumulative violent video game effects on hostile expectations and aggressive behavior,” by Youssef Hasan, Laurent Bègue, Michael Scharkow, and Brad Bushman. My correspondent was suspicious of the error bars in Figure 1. I actually think […]

“The Internal and External Validity of the Regression Discontinuity Design: A Meta-Analysis of 15 Within-Study-Comparisons”

Jag Bhalla points to this post by Alex Tabarrok pointing to this paper, “The Internal and External Validity of the Regression Discontinuity Design: A Meta-Analysis of 15 Within-Study-Comparisons,” by Duncan Chaplin, Thomas Cook, Jelena Zurovac, Jared Coopersmith, Mariel Finucane, Lauren Vollmer, and Rebecca Morris, which reports that regression discontinuity (RD) estimation performed well in these […]

Does adding women to corporate boards increase stock price?

Anton Kasster writes:

This one’s important: How to better analyze cancer drug trials using multilevel models.

Paul Alper points us to this news article, “Cancer Conundrum—Too Many Drug Trials, Too Few Patients,” by Gina Kolata, who writes: With the arrival of two revolutionary treatment strategies, immunotherapy and personalized medicine, cancer researchers have found new hope — and a problem that is perhaps unprecedented in medical research. There are too many experimental […]

This April Fools post is dead serious

Usually for April 1st I schedule a joke post, something like: Why I don’t like Bayesian statistics, or Enough with the replication police, or Why tables are really much better than graphs, or Move along, nothing to see here, or A randomized trial of the set-point diet, etc. But today I have something so ridiculous […]

Judgment Under Uncertainty: Heuristics and Biases

There are some people I’ve never met who send me scientific papers to comment on for the blog. The other day one of these people sent me one of these: it was a published paper covering several topics on which I am an expert, and it seemed like it could be interesting but at the […]

Replication is a good idea, but this particular replication is a bit too exact!

The following showed up in my email one day: From: Subject: Self-Plagarism in Current Opinion in Psychology Date: March 9, 2018 at 4:06:25 PM EST To: “gelman@stat.columbia.edu” Hello, You might be interested in the tremendous amount of overlap between two recent articles by Benjamin & Bushman (2016 & 2018) in Current Opinion in Psychology. The […]

Hey! Free money!

This just came in: On Dec 27, 2017, at 6:55 PM, **@gmail.com wrote: My name is ** and I am a freelance writer hoping to contribute my writing to andrewgelman.com. I would be willing to compensate you for publishing. For my posts, I require one related client link within the body of my article, as […]

Yet another IRB horror story

The IRB (institutional review board) is this weird bureaucracy, often staffed by helpful and well-meaning people but generally out of control, as it operates on an if-it’s-not-allowed-it’s-forbidden principle. As an example, Jonathan Falk points us to this Kafkaesque story from Scott Alexander, which ends up like this: Faced with submitting twenty-seven new pieces of paperwork […]

Combining Bayesian inferences from many fitted models

Renato Frey writes: I’m curious about your opinion on combining multi-model inference techniques with rstanarm: On the one hand, screening all (theoretically meaningful) model specifications and fully reporting them seems to make a lot of sense to me — in line with the idea of transparent reporting, your idea of the multiverse analysis, or akin […]

Heuristics and Biases? Laplace was there, 200 years ago.

In an article entitled Laplace’s Theories of Cognitive Illusions, Heuristics, and Biases, Josh “hot hand” Miller and I write: In his book from the early 1800s, Essai Philosophique sur les Probabilités, the mathematician Pierre-Simon de Laplace anticipated many ideas developed in the 1970s in cognitive psychology and behavioral economics, explaining human tendencies to deviate from […]

The problem with those studies that claim large and consistent effects from small and irrelevant inputs

Dale Lehman writes: You have often critiqued those headline grabbing studies such as how news about shark attacks influence voting behavior, how the time of month/color of clothing influences voting, etc. I am in total agreement with your criticisms of this “research.” Too many confounding variables, too small sample sizes, too many forking paths, poor […]

Bayesian inference for A/B testing: Lauren Kennedy and I speak at the NYC Women in Machine Learning and Data Science meetup tomorrow (Tues 27 Mar) 7pm

Here it is: Bayesian inference for A/B testing Andrew Gelman, Department of Statistics and Department of Political Science, Columbia University Lauren Kennedy, Columbia Population Research Center, Columbia University Suppose we want to use empirical data to compare two or more decisions or treatment options. Classical statistical methods based on statistical significance and p-values break down […]

Spatial patterns in crime: Where’s he gonna strike next?

Wouter Steenbeek writes: I am a criminologist and mostly do spatial analyses of crime patterns: where does crime occur and why in these neighborhoods / at these locations, and so on. Currently, I am thinking about offender decision-making behavior, specifically his ‘location choice’ of where to offend. Hey, how about criminologists instead of looking to […]

Request for a cat picture

Could someone please send me a photo (that I’d have permission to share on this blog) that connects a cat to “heuristics and biases” or “behavioral economics”? Thanks. P.S. Javier Benítez points us to this page of free stock photos of cats. Cool! Still, if anyone has anything particularly appropriate to the topic above, just […]

Debate over claims of importance of spending on Obamacare advertising

Jerrod Anderson points to this post by Paul Shafer, Erika Fowler, Laura Baum, and Sarah Gollust, “Advertising cutbacks reduce Marketplace information-seeking behavior: Lessons from Kentucky for 2018.” Anderson expresses skepticism about this claim. I’ll first summarize the claims of Shafer et al. and then get to Anderson’s criticism. Shafer et al. write: The Trump administration […]

“The problem of infra-marginality in outcome tests for discrimination”

Camelia Simoiu, Sam Corbett-Davies, and Sharad Goel write: Outcome tests are a popular method for detecting bias in lending, hiring, and policing decisions. These tests operate by comparing the success rate of decisions across groups. For example, if loans made to minority applicants are observed to be repaid more often than loans made to whites, […]