Skip to content
Archive of posts filed under the Decision Theory category.

The AAA tranche of subprime science, revisited

Tom Daula points us to this article, “Mortgage-Backed Securities and the Financial Crisis of 2008: A Post Mortem,” by Juan Ospina and Harald Uhlig. Not our usual topic at this blog, but then there’s this bit on page 11: We break down the analysis by market segment defined by loan type (Prime, Alt-A, and Subprime). […]

The Golden Rule of Nudge

Nudge unto others as you would have them nudge unto you. Do not recommend to apply incentives to others that you would not want for yourself. Background I was reading this article by William Davies about Britain’s Kafkaesque immigration policies. The background, roughly, is this: Various English politicians promised that the net flow of immigrants […]

Perhaps you could try a big scatterplot with one dot per dataset?

Joe Nadeau writes: We are studying variation in both means and variances in metabolic conditions. We have access to nearly 200 datasets that involve a range of metabolic traits and vary in sample size, mean effects, and variance. Some traits differ in mean but not variance, others in variance but not mean, still others in […]

A potential big problem with placebo tests in econometrics: they’re subject to the “difference between significant and non-significant is not itself statistically significant” issue

In econometrics, or applied economics, a “placebo test” is not a comparison of a drug to a sugar pill. Rather, it’s a sort of conceptual placebo, in which you repeat your analysis using a different dataset, or a different part of your dataset, where no intervention occurred. For example, if you’re performing some analysis studying […]

Don’t calculate post-hoc power using observed estimate of effect size

Aleksi Reito writes: The statement below was included in a recent issue of Annals of Surgery: But, as 80% power is difficult to achieve in surgical studies, we argue that the CONSORT and STROBE guidelines should be modified to include the disclosure of power—even if less than 80%—with the given sample size and effect size […]

Why, oh why, do so many people embrace the Pacific Garbage Cleanup nonsense? (I have a theory).

This post is by Phil, not Andrew. Over the couple of months I have seen quite a few people celebrating the long-awaited launch of a big device that will remove plastic garbage from the Pacific ocean. I find this frustrating because this project makes no sense even if the device works as intended: at best […]

Mouse Among the Cats

Colleen Flaherty asks: Do you get asked to peer review a lot? I’m guessing you do… This new very short paper says it’s not a crisis, though, since only the people who publish the most are getting asked to review a lot… The authors pose two solutions: either we need to “democratize” the system of […]

“Dynamically Rescaled Hamiltonian Monte Carlo for Bayesian Hierarchical Models”

Aki points us to this paper by Tore Selland Kleppe, which begins: Dynamically rescaled Hamiltonian Monte Carlo (DRHMC) is introduced as a computationally fast and easily implemented method for performing full Bayesian analysis in hierarchical statistical models. The method relies on introducing a modified parameterisation so that the re-parameterised target distribution has close to constant […]

How to set up a voting system for a Hall of Fame?

Micah Cohen writes: Our company is establishing a Hall of Fame and I am on a committee to help set it up which involved figuring out the voting system to induct a candidate. We have modeled it somewhat off of the voting for the Baseball Hall of Fame. The details in short: · Up to […]

Hey—take this psychological science replication quiz!

Rob Wilbin writes: I made this quiz where people try to guess ahead of time which results will replicate and which won’t in order to give then a more nuanced understanding of replication issues in psych. Based on this week’s Nature replication paper. It includes quotes and p-values from the original study if people want […]

“Identification of and correction for publication bias,” and another discussion of how forking paths is not the same thing as file drawer

Max Kasy and Isaiah Andrews sent along this paper, which begins: Some empirical results are more likely to be published than others. Such selective publication leads to biased estimates and distorted inference. This paper proposes two approaches for identifying the conditional probability of publication as a function of a study’s results, the first based on […]

Bayesian model comparison in ecology

Conor Goold writes: I was reading this overview of mixed-effect modeling in ecology, and thought you or your blog readers may be interested in their last conclusion (page 35): Other modelling approaches such as Bayesian inference are available, and allow much greater flexibility in choice of model structure, error structure and link function. However, the […]

“Richard Jarecki, Doctor Who Conquered Roulette, Dies at 86”

[relevant video] Thanatos Savehn is right. This obituary, written by someone named “Daniel Slotnik” (!), is just awesome: Many gamblers see roulette as a game of pure chance — a wheel is spun, a ball is released and winners and losers are determined by luck. Richard Jarecki refused to believe it was that simple. He […]

Trapped in the spam folder? Here’s what to do.

[Somewhat-relevant image] It seems that some people’s comments are getting trapped in the spam filter. Here’s how things go. The blog software triages the comments: 1. Most legitimate comments are automatically approved. You write the comment and it shows up right away. 2. Some comments are flagged as potentially spam. About half of these are […]

Response to Rafa: Why I don’t think ROC [receiver operating characteristic] works as a model for science

Someone pointed me to this post from a few years ago where Rafael Irizarry argues that scientific “pessimists” such as myself are, at least in some fields, “missing a critical point: that in practice, there is an inverse relationship between increasing rates of true discoveries and decreasing rates of false discoveries and that true discoveries […]

Don’t call it a bandit

Here’s why I don’t like the term “multi-armed bandit” to describe the exploration-exploitation tradeoff of inference and decision analysis. First, and less importantly, each slot machine (or “bandit”) only has one arm. Hence it’s many one-armed bandits, not one multi-armed bandit. Second, the basic strategy in these problems is to play on lots of machines […]

When LOO and other cross-validation approaches are valid

Introduction Zacco asked in Stan discourse whether leave-one-out (LOO) cross-validation is valid for phylogenetic models. He also referred to Dan’s excellent blog post which mentioned iid assumption. Instead of iid it would be better to talk about exchangeability assumption, but I (Aki) got a bit lost in my discourse answer (so don’t bother to go […]

“Seeding trials”: medical marketing disguised as science

Paul Alper points to this horrifying news article by Mary Chris Jaklevic, “how a medical device ‘seeding trial’ disguised marketing as science.” I’d never heard of “seeding trials” before. Here’s Jaklevic: As a new line of hip implants was about to be launched in 2000, a stunning email went out from the manufacturer’s marketing department. […]

How dumb do you have to be…

I (Phil) just read an article about Apple. Here’s the last sentence: “Apple has beaten earnings expectations in every quarter but one since March 2013.” [Note added a week later: on July 31 Apple reported earnings for the fiscal third quarter.  Earnings per share was $2.34 vs. the ‘consensus estimate’ of $2.18, according to Thomson Reuters.]  

Parsimonious principle vs integration over all uncertainties

tl;dr If you have bad models, bad priors or bad inference choose the simplest possible model. If you have good models, good priors, good inference, use the most elaborate model for predictions. To make interpretation easier you may use a smaller model with similar predictive performance as the most elaborate model. Merijn Mestdagh emailed me […]