Skip to content
Archive of posts filed under the Decision Theory category.

“The frequentist case against the significance test”

Richard Morey writes: I suspect that like me, many people didn’t get a whole lot of detail about Neyman’s objections to the significance test in their statistical education besides “Neyman thought power is important”. Given the recent debate about significance testing, I have gone back to Neyman’s papers and tried to summarize, for the modern […]


Leonid Schneider writes: I am cell biologist turned science journalist after 13 years in academia. Despite my many years experience as scientist, I shamefully admit to be largely incompetent in statistics. My request to you is as follows: A soon to be published psychology study set on to reproduce 100 randomly picked earlier studies and […]

Medical decision making under uncertainty

Gur Huberman writes: The following crossed my mind, following a recent panel discussion in which David Madigan participated on evidence-based medicine. The panel—especially John Iaonnidis—sang the praise of clinical trials. You may have nothing wise to say about it—or pose the question to your blog followers. Suppose there’s a standard clinical procedure to address a […]

The aching desire for regular scientific breakthroughs

This post didn’t come out the way I planned. Here’s what happened. I cruised over to the British Psychological Society Research Digest (formerly on our blogroll) and came across a press release entitled “Background positive music increases people’s willingness to do others harm.” Uh oh, I thought. This sounds like one of those flaky studies, […]

On deck this week

Mon: Review of The Martian Tues: Even though it’s published in a top psychology journal, she still doesn’t believe it Wed: Turbulent Studies, Rocky Statistics: Publicational Consequences of Experiencing Inferential Instability Thurs: Medical decision making under uncertainty Fri: Unreplicable Sat: “The frequentist case against the significance test” Sun: Erdos bio for kids

On deck this week

Mon: Comments on Imbens and Rubin causal inference book Tues: “Dow 36,000″ guy offers an opinion on Tom Brady’s balls. The rest of us are supposed to listen? Wed: Irwin Shaw: “I might mistrust intellectuals, but I’d mistrust nonintellectuals even more.” Thurs: Death of a statistician Fri: Being polite vs. saying what we really think […]

P-values and statistical practice

What is a p-value in practice? The p-value is a measure of discrepancy of the fit of a model or “null hypothesis” H to data y. In theory the p-value is a continuous measure of evidence, but in practice it is typically trichotomized approximately into strong evidence, weak evidence, and no evidence (these can also […]

To understand the replication crisis, imagine a world in which everything was published.

John Snow points me to this post by psychology researcher Lisa Feldman Barrett who reacted to the recent news on the non-replication of many psychology studies with a contrarian, upbeat take, entitled “Psychology Is Not in Crisis.” Here’s Barrett: An initiative called the Reproducibility Project at the University of Virginia recently reran 100 psychology experiments […]

Uri Simonsohn warns us not to be falsely reassured

I agree with Uri Simonsohn that you don’t learn much by looking at the distribution of all the p-values that have appeared in some literature. Uri explains: Most p-values reported in most papers are irrelevant for the strategic behavior of interest. Covariates, manipulation checks, main effects in studies testing interactions, etc. Including them we underestimate […]

On deck this week

Mon: Constructing an informative prior using meta-analysis Tues: Stan attribution Wed: Cannabis/IQ follow-up: Same old story Thurs: Defining conditional probability Fri: In defense of endless arguments Sat: Emails I never finished reading Sun: BREAKING . . . Sepp Blatter accepted $2M payoff from Dennis Hastert

Performing design calculations (type M and type S errors) on a routine basis?

Somebody writes writes: I am conducting a survival analysis (median follow up ~10 years) of subjects who enrolled on a prospective, non-randomized clinical trial for newly diagnosed multiple myeloma. The data were originally collected for research purposes and specifically to determine PFS and OS of the investigational regimen versus historic controls. The trial has been […]

“The belief was so strong that it trumped the evidence before them.”

I was reading Palko on the 5 cent cup of coffee and spotted this: We’ve previously talked about bloggers trying to live on a food stamp budget for a week (yeah, that’s a thing). One of the many odd recurring elements of these post is a litany of complaints about life without caffeine because… I had already understood […]

On deck this week

Mon: “The belief was so strong that it trumped the evidence before them.” Tues: “Can you change your Bayesian prior?” Wed: How to analyze hierarchical survey data with post-stratification? Thurs: A political sociological course on statistics for high school students Fri: Questions about data transplanted in kidney study Sat: Performing design calculations (type M and […]

Macartan Humphreys on the Worm Wars

My Columbia political science colleague shares “What Has Been Learned from the Deworming Replications: A Nonpartisan View”: Last month there was another battle in a dispute between economists and epidemiologists over the merits of mass deworming.1 In brief, economists claim there is clear evidence that cheap deworming interventions have large effects on welfare via increased […]

On deck this week

Mon: My 2 classes this fall Tues: “Soylent 1.5” < black beans and yoghurt Wed: 0.05 is a joke Thurs: Data-analysis assignments for BDA class Fri: Aahhhhh, young people! Sat: Plaig! (non-Wegman edition) Sun: We provide a service

Hey—Don’t trust anything coming from the Tri-Valley Center for Human Potential!

Shravan sends along this article by Douglas Peters and Stephen Ceci, who report: We selected 12 already published research articles by investigators from prestigious and highly productive American psychology departments, one article from each of 12 highly regarded and widely read American psychology journals with high rejection rates (80%) and nonblind refereeing practices. With fictitious […]

Dan Kahan doesn’t trust the Turk

Dan Kahan writes: I [Kahan] think serious journals should adopt policies announcing that they won’t accept studies that use M Turk samples for types of studies they are not suited for. . . . Here is my proposal: Pending a journal’s adoption of a uniform policy on M Turk samples, the journal should should oblige […]

On deck this week

Mon: Dan Kahan doesn’t trust the Turk Tues: Neither time nor stomach Wed: Reprint of “Observational Studies” by William Cochran followed by comments by current researchers in observational studies Thurs: Hey—Don’t trust anything coming from the Tri-Valley Center for Human Potential! Fri: Harry S. Truman, Jesus H. Christ, Roy G. Biv Sat: Why couldn’t Breaking […]

The secret to making a successful conference presentation

JSM (the Joint Statistical Meetings) are coming up soon, and Jiqiang’s giving a talk on Stan. Here’s the advice I gave him: in 20 minutes, something like this: – What is Stan? – Where does Stan work well? – Current and future Stan research. For JSM audience it could be good to spend some time […]

On deck this week

Mon: The plagiarist next door strikes back: Different standards of plagiarism in different communities Tues: Pro Publica’s new Surgeon Scorecards Wed: How Hamiltonian Monte Carlo works Thurs: When does Bayes do the job? Fri: Here’s a theoretical research project for you Sat: Classifying causes of death using “verbal autopsies” Sun: All hail Lord Spiegelhalter!