Skip to content
Archive of posts filed under the Decision Theory category.

A quick one

Fabio Rojas asks: Should I do Bonferroni adjustments? Pros? Cons? Do you have a blog post on this? Most social scientists don’t seem to be aware of this issue. My short answer is that if you’re fitting mutlilevel models, I don’t think you need multiple comparisons adjustments; see here.

Cross-validation != magic

In a post entitled “A subtle way to over-fit,” John Cook writes: If you train a model on a set of data, it should fit that data well. The hope, however, is that it will fit a new set of data well. So in machine learning and statistics, people split their data into two parts. […]

On deck this week

Mon: All the things that don’t make it into the news Tues: Cross-validation != magic Wed: Of buggy whips and moral hazards; or, Sympathy for the Aapor Thurs: Low-power pose Fri: Should you get the blood transfusion? Sat: “History is the prediction of the present” Sun: What to do to train to apply statistical models […]

“With that assurance, a scientist can report his or her work to the public, and the public can trust the work.”

Dan Wright writes: Given your healthy skepticism of findings/conclusions from post-peer-reviewed papers, I thought I would forward the following from Institute of Educational Sciences. Here is a sample quote: Simply put, peer review is a method by which scientists who are experts in a particular field examine another scientist’s work to verify that it makes […]

On deck this week

Mon: An inundation of significance tests Tues: Stock, flow, and two smoking regressions Wed: What’s the worst joke you’ve ever heard? Thurs: Cracked.com > Huffington Post, Wall Street Journal, New York Times Fri: Measurement is part of design Sat: “17 Baby Names You Didn’t Know Were Totally Made Up” Sun: What to do to train […]

Creativity is the ability to see relationships where none exist

Brent Goldfarb and Andrew King, in a paper to appear in the journal Strategic Management, write: In a recent issue of this journal, Bettis (2012) reports a conversation with a graduate student who forthrightly announced that he had been trained by faculty to “search for asterisks”. The student explained that he sifted through large databases […]

Can talk therapy halve the rate of cancer recurrence? How to think about the statistical significance of this finding? Is it just another example of the garden of forking paths?

James Coyne (who we last encountered in the sad story of Ellen Langer) writes: I’m writing to you now about another matter about which I hope you will offer an opinion. Here is a critique of a study, as well as the original study that claimed to find an effect of group psychotherapy on time […]

Bayesian inference: The advantages and the risks

This came up in an email exchange regarding a plan to come up with and evaluate Bayesian prediction algorithms for a medical application: I would not refer to the existing prediction algorithm as frequentist. Frequentist refers to the evaluation of statistical procedures but it doesn’t really say where the estimate or prediction comes from. Rather, […]

New Alan Turing preprint on Arxiv!

Dan Kahan writes: I know you are on 30-day delay, but since the blog version of you will be talking about Bayesian inference in couple of hours, you might like to look at paper by Turing, who is on 70-yr delay thanks to British declassification system, who addresses the utility of using likelihood ratios for […]

On deck this week

Mon: Bob Carpenter’s favorite books on GUI design and programming Tues: Bayesian inference: The advantages and the risks Wed: Objects of the class “Foghorn Leghorn” Thurs: “Physical Models of Living Systems” Fri: Creativity is the ability to see relationships where none exist Sat: Kaiser’s beef Sun: Chess + statistics + plagiarism, again!

My talk at MIT this Thursday

When I was a student at MIT, there was no statistics department. I took a statistics course from Stephan Morgenthaler and liked it. (I’d already taken probability and stochastic processes back at the University of Maryland; my instructor in the latter class was Prof. Grace Yang, who was super-nice. I couldn’t follow half of what […]

There’s No Such Thing As Unbiased Estimation. And It’s a Good Thing, Too.

Following our recent post on econometricians’ traditional privileging of unbiased estimates, there were a bunch of comments echoing the challenge of teaching this topic, as students as well as practitioners often seem to want the comfort of an absolute standard such as best linear unbiased estimate or whatever. Commenters also discussed the tradeoff between bias […]

On deck this week

Mon: There’s No Such Thing As Unbiased Estimation. And It’s a Good Thing, Too. Tues: There’s something about humans Wed: Humility needed in decision-making Thurs: The connection between varying treatment effects and the well-known optimism of published research findings Fri: I actually think this infographic is ok Sat: Apology to George A. Romero Sun: “Do […]

Collaborative filtering, hierarchical modeling, and . . . speed dating

Jonah Sinick posted a few things on the famous speed-dating dataset and writes: The main element that I seem to have been missing is principal component analysis of the different rating types. The basic situation is that the first PC is something that people are roughly equally responsive to, while people vary a lot with […]

What I got wrong (and right) about econometrics and unbiasedness

Yesterday I spoke at the Princeton economics department. The title of my talk was: “Unbiasedness”: You keep using that word. I do not think it means what you think it means. The talk went all right—people seemed ok with what I was saying—but I didn’t see a lot of audience involvement. It was a bit […]

On deck this week

Mon: A causal-inference version of a statistics problem: If you fit a regression model with interactions, and the underlying process has an interaction, your coefficients won’t be directly interpretable. Tues: He’s looking for probability puzzles Wed: In criticism of criticism of criticism Thurs: A question about physics-types models for flows in economics Fri: What I […]

On deck this month

A causal-inference version of a statistics problem: If you fit a regression model with interactions, and the underlying process has an interaction, your coefficients won’t be directly interpretable. He’s looking for probability puzzles In criticism of criticism of criticism A question about physics-types models for flows in economics What I got wrong (and right) about […]

On deck this week

Mon: Eccentric mathematician Tues: What’s the most important thing in statistics that’s not in the textbooks? Wed: Carl Morris: Man Out of Time [reflections on empirical Bayes] Thurs: “The general problem I have with noninformatively-derived Bayesian probabilities is that they tend to be too strong.” Fri: Good, mediocre, and bad p-values Sat: Which of these […]

Online predictions from ipredict

Following up on our post on PredictWise, Richard Barker points to this fun site of market-based predictions. It’s subtitled, “Buy and sell stocks in future political and economic events.” It’s based in New Zealand so you can bet on wacky propositions such as, “David Carter to be next High Commissioner from New Zealand to the […]

On deck this week

Mon: New book on Bayesian analysis in ecology using Stan Tues: The feather, the bathroom scale, and the kangaroo Wed: Instead of worrying about multiple hypothesis correction, just fit a hierarchical model. Thurs: Political Attitudes in Social Environments Fri: Statistical significance, practical significance, and interactions Sat: Statistical analysis on a dataset that consists of a […]