Skip to content
Archive of posts filed under the Decision Theory category.

My next 170 blog posts (inbox zero and a change of pace)

I’ve successfully emptied my inbox: And one result was to fill up the blog through mid-January. I think I’ve been doing enough blogging recently, so my plan now is to stop for awhile and instead transfer my writing energy into articles and books. We’ll see how it goes. Just to give you something to look […]

What recommendations to give when a medical study is not definitive (which of course will happen all the time, especially considering that new treatments should be compared to best available alternatives, which implies that most improvements will be incremental at best)

Simon Gates writes: I thought you might be interested in a recently published clinical trial, for potential blog material. It picks up some themes that have cropped in recent months. Also, it is important for the way statistical methods influence what can be life or death decisions. The OPPTIMUM trial (http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(16)00350-0/abstract) evaluated use of vaginal progesterone […]

Fish cannot carry p-values

Following up on our discussion from last week on inference for fisheries, Anders Lamberg writes: Since I first sent you the question, there has been a debate here too. In the discussion you send, there is a debate both about the actual sampling (the mathematics) and about more the practical/biological issues. How accurate can farmed […]

Killer O

Taggert Brooks points to this excellent news article by George Johnson, who reports: Epidemiologists have long been puzzled by a strange pattern in their data: People living at higher altitudes appear less likely to get lung cancer. . . . The higher you live, the thinner the air, so maybe oxygen is a cause of […]

On deck this week

Mon: Killer O Tues: More evidence that even top researchers routinely misinterpret p-values Wed: What makes a mathematical formula beautiful? Thurs: Fish cannot carry p-values Fri: Does Benadryl make you senile? Challenges in research communication Sat: What recommendations to give when a medical study is not definitive (which of course will happen all the time, […]

When do statistical rules affect drug approval?

Someone writes in: I have MS and take a disease-modifying drug called Copaxone. Sandoz developed a generic version​ of Copaxone​ and filed for FDA approval. Teva, the manufacturer of Copaxone, filed a petition opposing that approval (surprise!). FDA rejected Teva’s petitions and approved the generic. My insurance company encouraged me to switch to the generic. […]

Moving statistical theory from a “discovery” framework to a “measurement” framework

Avi Adler points to this post by Felix Schönbrodt on “What’s the probability that a significant p-value indicates a true effect?” I’m sympathetic to the goal of better understanding what’s in a p-value (see for example my paper with John Carlin on type M and type S errors) but I really don’t like the framing […]

On deck this week

Mon: Moving statistical theory from a “discovery” framework to a “measurement” framework Tues: Bayesian Linear Mixed Models using Stan: A tutorial for psychologists, linguists, and cognitive scientists Wed: Going beyond confidence intervals Thurs: Ioannidis: “Evidence-Based Medicine Has Been Hijacked” Fri: What’s powdery and comes out of a metallic-green cardboard can? Sat: “The Dark Side of […]

“Pointwise mutual information as test statistics”

Christian Bartels writes: Most of us will probably agree that making good decisions under uncertainty based on limited data is highly important but remains challenging. We have decision theory that provides a framework to reduce risks of decisions under uncertainty with typical frequentist test statistics being examples for controlling errors in absence of prior knowledge. […]

Replin’ ain’t easy: My very first preregistration

I’m doing my first preregistered replication. And it’s a lot of work! We’ve been discussing this for awhile—here’s something I published in 2013 in response to proposals by James Moneghan and by Macartan Humphreys, Raul Sanchez de la Sierra, and Peter van der Windt for preregistration in political science, here’s a blog discussion (“Preregistration: what’s […]

About that claim that police are less likely to shoot blacks than whites

Josh Miller writes: Did you see this splashy NYT headline, “Surprising New Evidence Shows Bias in Police Use of Force but Not in Shootings”? It’s actually looks like a cool study overall, with granular data, and a ton of leg work, and rich set of results that extend beyond the attention grabbing headline that is […]

Of polls and prediction markets: More on #BrexitFail

David “Xbox poll” Rothschild and I wrote an article for Slate on how political prediction markets can get things wrong. The short story is that in settings where direct information is not easily available (for example, in elections where polls are not viewed as trustworthy forecasts, whether because of problems in polling or anticipated volatility […]

“Most notably, the vast majority of Americans support criminalizing data fraud, and many also believe the offense deserves a sentence of incarceration.”

Justin Pickett sends along this paper he wrote with Sean Roche: Data fraud and selective reporting both present serious threats to the credibility of science. However, there remains considerable disagreement among scientists about how best to sanction data fraud, and about the ethicality of selective reporting. OK, let’s move away from asking scientists. Let’s ask […]

On deck this week

Mon: “Most notably, the vast majority of Americans support criminalizing data fraud, and many also believe the offense deserves a sentence of incarceration.” Tues: Some insider stuff on the Stan refactor Wed: I know you guys think I have no filter, but . . . Thurs: Bigmilk strikes again Fri: “Pointwise mutual information as test […]

Causal and predictive inference in policy research

Todd Rogers pointed me to a paper by Jon Kleinberg, Jens Ludwig, Sendhil Mullainathan, and Ziad Obermeyer that begins: Empirical policy research often focuses on causal inference. Since policy choices seem to depend on understanding the counterfactual—what happens with and without a policy—this tight link of causality and policy seems natural. While this link holds […]

On deck this week

Mon: Americans (used to) love world government Tues: “Positive Results Are Better for Your Career” Wed: “I would like to share some sad stories from economics related to these issues” Thurs: Happiness formulas Fri: “Participants reported being hungrier when they walked into the café (mean = 7.38, SD = 2.20) than when they walked out […]

Informative priors for treatment effects

Biostatistician Garnett McMillan writes: A PI recently completed a randomized trial where the experimental treatment showed a large, but not quite statistically significant (p=0.08) improvement over placebo. The investigators wanted to know how many additional subjects would be needed to achieve significance. This is a common question, which is very hard to answer for non-statistical […]

On deck this week

Mon: How is Brexit different than Texit, Quexit, or Scotxit? Tues: Should this paper in Psychological Science be retracted? The data do not conclusively demonstrate the claim, nor do they provide strong evidence in favor. The data are, however, consistent with the claim (as well as being consistent with no effect) Wed: Individual and aggregate […]

On deck this week

Mon: Clarke’s Law: Any sufficiently crappy research is indistinguishable from fraud Tues: Reduced-dimensionality parameterizations for linear models with interactions Wed: Time-reversal heuristic as randomization, and p < .05 as conflict of interest declaration Thurs: It comes down to reality and it’s fine with me cause I’ve let it slide Fri: Can a census-tract-level regression analysis […]

The answer is the Edlin factor

Garnett McMillan writes: You have argued about the pervasive role of the Garden of Forking Paths in published research. Given this influence, do you think that it is sensible to use published research to inform priors in new studies? My reply: Yes, I think you can use published research but in doing so you should […]