Skip to content
Archive of entries posted by

Let’s face it, I know nothing about spies.

I saw this news article: Multiple federal agencies investigated claims that former Indiana basketball coach Bobby Knight groped and verbally sexually harassed several female employees when he gave a speech at the National Geospatial-Intelligence Agency in July 2015, according to newly-released documents. . . . he had slapped a “senior woman … on [her] butt” […]

Low power and the replication crisis: What have we learned since 2004 (or 1984, or 1964)?

I happened to run across this article from 2004, “The Persistence of Underpowered Studies in Psychological Research: Causes, Consequences, and Remedies,” by Scott Maxwell and published in the journal Psychological Methods. In this article, Maxwell covers a lot of the material later discussed in the paper Power Failure by Button et al. (2013), and the […]

Bob likes the big audience

In response to a colleague who was a bit scared of posting some work up on the internet for all to see, Bob Carpenter writes: I like the big audience for two reasons related to computer science principles. The first benefit is the same reason it’s scary. The big audience is likely to find flaws. […]

Of rabbits and cannons

When does it make sense to shoot a rabbit with a cannon? I was reminded of this question recently when I happened to come across this exchange in the comments section from a couple years ago, in the context of the finding patterns in the frequencies of births on different days: Rahul: Yes, inverting a […]

“Write No Matter What” . . . about what?

Scott Jaschik interviews Joli Jensen (link from Tyler Cowen), a professor of communication who wrote a new book called “Write No Matter What: Advice for Academics.” Her advice might well be reasonable—it’s hard for me to judge; as someone who blogs a few hundred times a year, I’m not really part of Jensen’s target audience. […]

How to think about the risks from low doses of radon

Nick Stockton, a reporter for Wired magazine, sent me some questions about radiation risk and radon, and Phil and I replied. I thought our responses might be of general interest so I’m posting them here. First I wrote: Low dose risk is inherently difficult to estimate using epidemiological studies. I’ve seen no evidence that risk […]

Research project in London and Chicago to develop and fit hierarchical models for development economics in Stan!

Rachael Meager at the London School of Economics and Dean Karlan at Northwestern University write: We are seeking a Research Assistant skilled in R programming and the production of R packages. The successful applicant will have experience creating R packages accessible on github or CRAN, and ideally will have experience working with Rstan. The main […]

Use multilevel modeling to correct for the “winner’s curse” arising from selection of successful experimental results

John Snow writes: I came across this blog by Milan Shen recently and thought you might find it interesting. A couple of things jumped out at me. It seemed like the so-called ‘Winner’s Curse’ is just another way of describing the statistical significance filter. It also doesn’t look like their correction method is very effective. […]

The Lab Where It Happens

“Study 1 was planned in 2007, but it was conducted in the Spring of 2008 shortly after the first author was asked to take a 15-month leave-of-absence to be the Executive Director for USDA’s Center for Nutrition Policy and Promotion in Washington DC. . . . The manuscript describing this pair of studies did not […]

One data pattern, many interpretations

David Pittelli points us to this paper: “When Is Higher Neuroticism Protective Against Death? Findings From UK Biobank,” and writes: They come to a rather absurd conclusion, in my opinion, which is that neuroticism is protective if, and only if, you say you are in bad health, overlooking the probability that neuroticism instead makes you […]

Testing Seth Roberts’ appetite theory

Jonathan Tupper writes: My organization is running a group test of Seth Roberts’ old theory about appetite. We are running something like a “web trial” as discussed in your Chance article with Seth. And in fact our design was very inspired by your conversation… For one, we are using a control group which takes light […]

3 quick tricks to get into the data science/analytics field

John McCool writes: Do you have advice getting into the data science/analytics field? I just graduated with a B.S. in environmental science and a statistics minor and am currently interning at a university. I enjoy working with datasets from sports to transportation and doing historical analysis and predictive modeling. My quick advice is to avoid […]

Hysteresis corner: “These mistakes and omissions do not change the general conclusion of the paper . . .”

All right, then. The paper’s called Attractive Data Sustain Increased B.S. Intake in Journals Attractive Names Sustain Increased Vegetable Intake in Schools. Seriously, though, this is just an extreme example of a general phenomenon, which we might call scientific hysteresis or the research incumbency advantage: When you’re submitting a paper to a journal, it can […]

“Heating Up in NBA Free Throw Shooting”

Paul Pudaite writes: I demonstrate that repetition heats players up, while interruption cools players down in NBA free throw shooting. My analysis also suggests that fatigue and stress come into play. If, as seems likely, all four of these effects have comparable impact on field goal shooting, they would justify strategic choices throughout a basketball […]

Return of the Klam

Matthew Klam is back. This time for reals. I’m halfway through reading his new book, Who is Rich?, and it’s just wonderful. The main character is a cartoonist and illustrator, and just about every scene is filled with stunning and hilarious physical descriptions. If I were writing a blurb, I’d call Who is Rich? the […]

I’m skeptical of the claims made in this paper

Two different people pointed me to a recent research article, suggesting that the claims therein were implausible and the result of some combination of forking paths and spurious correlations—that is, there was doubt that the results would show up in a preregistered replication, and that, if they did show up, that they would mean what […]

“No System is Perfect: Understanding How Registration-Based Editorial Processes Affect Reproducibility and Investment in Research Quality”

Robert Bloomfield, Kristina Rennekamp, Blake Steenhoven sent along this paper that compares “a registration-based Editorial Process (REP). Authors submitted proposals to gather and analyze data; successful proposals were guaranteed publication as long as the authors lived up to their commitments, regardless of whether results supported their predictions” to “the Traditional Editorial Process (TEP).” Here’s what […]

What’s Wrong with “Evidence-Based Medicine” and How Can We Do Better? (My talk at the University of Michigan Friday 2pm)

Tomorrow (Fri 9 Feb) 2pm at the NCRC Research Auditorium (Building 10) at the University of Michigan: What’s Wrong with “Evidence-Based Medicine” and How Can We Do Better? Andrew Gelman, Department of Statistics and Department of Political Science, Columbia University “Evidence-based medicine” sounds like a good idea, but it can run into problems when the […]

354 possible control groups; what to do?

Jonas Cederlöf writes: I’m a PhD student in economics at Stockholm University and a frequent reader of your blog. I have for a long time followed your quest in trying to bring attention to p-hacking and multiple comparison problems in research. I’m now myself faced with the aforementioned problem and want to at the very […]

Methodological terrorism. For reals. (How to deal with “what we don’t know” in missing-data imputation.)

Kevin Lewis points us to this paper, by Aaron Safer-Lichtenstein, Gary LaFree, Thomas Loughran, on the methodology of terrorism studies. This is about as close to actual “methodological terrorism” as we’re ever gonna see here. The linked article begins: Although the empirical and analytical study of terrorism has grown dramatically in the past decade and […]