Computing marginal likelihoods in Stan, from Quentin Gronau and E. J. Wagenmakers

Gronau and Wagemakers write: The bridgesampling package facilitates the computation of the marginal likelihood for a wide range of different statistical models. For models implemented in Stan (such that the constants are retained), executing the code bridge_sampler(stanfit) automatically produces an … Continue reading

Statistical Practice as Scientific Exploration (my talk on 4 Mar 2024 at the Royal Society conference on the promises and pitfalls of preregistration)

Here’s the conference announcement: Discussion meeting organised by Dr Tom Hardwicke, Professor Marcus Munafò, Dr Sophia Crüwell, Professor Dorothy Bishop FRS FMedSci, Professor Eric-Jan Wagenmakers. Serious concerns about research quality have provoked debate across scientific disciplines about the merits of … Continue reading

The so-called “lucky golf ball”: The Association for Psychological Science promotes junk science while ignoring the careful, serious work of replication

“Not replicable, but citable” is how ‎Robert Calin-Jageman puts it. His colleague Geoff Cumming tells the story: The APS [Association for Psychological Science] has just given a kick along to what’s most likely a myth: The Lucky Golf Ball. Alas! … Continue reading

Megan Higgs (statistician) and Anna Dreber (economist) on how to judge the success of a replication

The discussion started with this comment from Megan Higgs regarding a recent science replication initiative: I [Higgs] was immediately curious about their criteria for declaring a study replicated. In a quick skim of the info in the google form, here … Continue reading

Many perspectives on Deborah Mayo’s “Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars”

This is not new—these reviews appeared in slightly rawer form several months ago on the blog. After that, I reorganized the material slightly and sent to Harvard Data Science Review (motto: “A Microscopic, Telescopic, and Kaleidoscopic View of Data Science”) … Continue reading

Facial feedback: “These findings suggest that minute differences in the experimental protocol might lead to theoretically meaningful changes in the outcomes.”

Fritz Strack points us to this article, “When Both the Original Study and Its Failed Replication Are Correct: Feeling Observed Eliminates the Facial-Feedback Effect,” by Tom Noah, Yaacov Schul, and Ruth Mayo, who write: According to the facial-feedback hypothesis, the … Continue reading

Comments on Limitations of Bayesian Leave-One-Out Cross-Validation for Model Selection

There is a recent pre-print Limitations of Bayesian Leave-One-Out Cross-Validation for Model Selection by Quentin Gronau and Eric-Jan Wagenmakers. Wagenmakers asked for comments and so here are my comments. Short version: They report a known limitation of LOO when it’s … Continue reading

The p-curve, p-uniform, and Hedges (1984) methods for meta-analysis under selection bias: An exchange with Blake McShane, Uri Simonsohn, and Marcel van Assen

Blake McShane sent me some material related to a paper of his (McShane et al., 2016; see reference list below), regarding various methods for combining p-values for meta-analysis under selection bias. His remarks related to some things written by Uri … Continue reading

This Friday at noon, join this online colloquium on replication and reproducibility, featuring experts in economics, statistics, and psychology!

Justin Esarey writes: This Friday, October 27th at noon Eastern time, the International Methods Colloquium will host a roundtable discussion on the reproducibility crisis in social sciences and a recent proposal to impose a stricter threshold for statistical significance. The … Continue reading

When considering proposals for redefining or abandoning statistical significance, remember that their effects on science will only be indirect!

John Schwenkler organized a discussion on this hot topic, featuring posts by – Dan Benjamin, Jim Berger, Magnus Johannesson, Valen Johnson, Brian Nosek, and E. J. Wagenmakers – Felipe De Brigard – Kenny Easwaran – Andrew Gelman and Blake McShane … Continue reading