Skip to content
Archive of posts filed under the Decision Theory category.

Should we worry about rigged priors? A long discussion.

Today’s discussion starts with Stuart Buck, who came across a post by John Cook linking to my post, “Bayesian statistics: What’s it all about?”. Cook wrote about the benefit of prior distributions in making assumptions explicit. Buck shared Cook’s post with Jon Baron, who wrote: My concern is that if researchers are systematically too optimistic […]

BREAKING . . . . . . . PNAS updates its slogan!

I’m so happy about this, no joke. Here’s the story. For awhile I’ve been getting annoyed by the junk science papers (for example, here, here, and here) that have been published by the Proceedings of the National Academy of Sciences under the editorship of Susan T. Fiske. I’ve taken to calling it PPNAS (“Prestigious proceedings […]

When considering proposals for redefining or abandoning statistical significance, remember that their effects on science will only be indirect!

John Schwenkler organized a discussion on this hot topic, featuring posts by – Dan Benjamin, Jim Berger, Magnus Johannesson, Valen Johnson, Brian Nosek, and E. J. Wagenmakers – Felipe De Brigard – Kenny Easwaran – Andrew Gelman and Blake McShane – Kiley Hamlin – Edouard Machery – Deborah Mayo – “Neuroskeptic” – Michael Strevens – […]

Alan Sokal’s comments on “Abandon Statistical Significance”

The physicist and science critic writes: I just came across your paper “Abandon statistical significance”. I basically agree with your point of view, but I think you could have done more to *distinguish* clearly between several different issues: 1) In most problems in the biomedical and social sciences, the possible hypotheses are parametrized by a […]

2 quick calls

Kevin Lewis asks what I think of these: Study 1: Using footage from body-worn cameras, we analyze the respectfulness of police officer language toward white and black community members during routine traffic stops. We develop computational linguistic methods that extract levels of respect automatically from transcripts, informed by a thin-slicing study of participant ratings of […]

Response to some comments on “Abandon Statistical Significance”

The other day, Blake McShane, David Gal, Christian Robert, Jennifer Tackett, and I wrote a paper, Abandon Statistical Significance, that began: In science publishing and many areas of research, the status quo is a lexicographic decision rule in which any result is first required to have a p-value that surpasses the 0.05 threshold and only […]

“5 minutes? Really?”

Bob writes: Daniel says this issue https://github.com/stan-dev/stan/issues/795#issuecomment-26390557117 is an easy 5-minute fix. In my ongoing role as wet blanket, let’s be realistic. It’s sort of like saying it’s an hour from here to Detroit because that’s how long the plane’s in the air. Nothing is a 5 minute fix (door to door) for Stan and […]

“From ‘What If?’ To ‘What Next?’ : Causal Inference and Machine Learning for Intelligent Decision Making”

Panos Toulis writes in to announce this conference: NIPS 2017 Workshop on Causal Inference and Machine Learning (WhatIF2017) “From ‘What If?’ To ‘What Next?’ : Causal Inference and Machine Learning for Intelligent Decision Making” — December 8th 2017, Long Beach, USA. Submission deadline for abstracts and papers: October 31, 2017 Acceptance decisions: November 7, 2017 […]

I am (somewhat) in agreement with Fritz Strack regarding replications

Fritz Strack read the recent paper of McShane, Gal, Robert, Tackett, and myself and pointed out that our message—abandon statistical significance, consider null hypothesis testing as just one among many pieces of evidence, recognize that all null hypotheses are false (at least in the fields where Strack and I do our research) and don’t use […]

Abandon Statistical Significance

Blake McShane, David Gal, Christian Robert, Jennifer Tackett, and I wrote a short paper arguing for the removal of null hypothesis significance testing from its current gatekeeper role in much of science. We begin: In science publishing and many areas of research, the status quo is a lexicographic decision rule in which any result is […]

Using black-box machine learning predictions as inputs to a Bayesian analysis

Following up on this discussion [Designing an animal-like brain: black-box “deep learning algorithms” to solve problems, with an (approximately) Bayesian “consciousness” or “executive functioning organ” that attempts to make sense of all these inferences], Mike Betancourt writes: I’m not sure AI (or machine learning) + Bayesian wrapper would address the points raised in the paper. […]

Causal inference using data from a non-representative sample

Dan Gibbons writes: I have been looking at using synthetic control estimates for estimating the effects of healthcare policies, particularly because for say county-level data the nontreated comparison units one would use in say a difference-in-differences estimator or quantile DID estimator (if one didn’t want to use the mean) are not especially clear. However, given […]

God, goons, and gays: 3 quick takes

Next open blog spots are in April but all these are topical so I thought I’d throw them down right now for ya. 1. Alex Durante writes: I noticed that this study on how Trump supporters respond to racial cues is getting some media play, notably over at Vox. I was wondering if you have […]

Looking for the bottom line

I recommend this discussion of how to summarize posterior distributions. I don’t recommend summarizing by the posterior probability that the new treatment is better than the old treatment, as that is not a bottom-line statement!

Local data, centralized data analysis, and local decision making

Jeffrey Wagar writes: Nobel-prize winning economist Oliver Williamson’s excellent book “Markets & Hierarchies” offered a set of rules for when Markets were the better method for solving a problem, or when Hierarchies [Government] was better suited. Yet could it be that some of today’s social problems persist because they are not well suited to being […]

How to design and conduct a subgroup analysis?

Brian MacGillivray writes: I’ve just published a paper that draws on your work on the garden of forking paths, as well as your concept of statistics as being the science of defaults. The article is called, “Characterising bias in regulatory risk and decision analysis: An analysis of heuristics applied in health technology appraisal, chemicals regulation, […]

All cause and breast cancer specific mortality, by assignment to mammography or control

Paul Alper writes: You might be interested in the robocall my wife received today from our Medicare Advantage organization (UCARE Minnesota). The robocall informed us that mammograms saved lives and was available free of charge as part of her health insurance. No mention of recent studies criticizing mammography regarding false positives, harms of biopsies, etc. […]

“Mainstream medicine has its own share of unnecessary and unhelpful treatments”

I have a story and then a question. The story Susan Perry (link sent by Paul Alper) writes: Earlier this week, I [Perry] highlighted two articles that exposed the dubious history, medical ineffectiveness and potential health dangers of popular alternative “therapies.” Well, the same can be said of many mainstream conventional medical practices, as investigative […]

Using statistical prediction (also called “machine learning”) to potentially save lots of resources in criminal justice

John Snow writes: Just came across this paper [Human Decisions and Machine Predictions, by Jon Kleinberg, Himabindu Lakkaraju, Jure Leskovec, Jens Ludwig, and Sendhil Mullainathan] and I’m wondering if you’ve been following the debate/discussion around these criminal justice risk assessment tools. I haven’t read it carefully or fully digested the details. On the surface, their […]

Chris Moore, Guy Molyneux, Etan Green, and David Daniels on Bayesian umpires

Kevin Lewis points us to a paper by Etan Green and David Daniels, who conclude that “decisions of [baseball] umpires reflect an accurate, probabilistic, and state-specific understanding of their rational expectations—as well as an ability to integrate those prior beliefs in a manner that approximates Bayes rule.” This is similar to what was found in […]