Some recent blog discussion revealed some confusion that I’ll try to resolve here. I wrote that I’m not a big fan of subjective priors. Various commenters had difficulty with this point, and I think the issue was most clearly stated by Bill Jeffreerys, who wrote: It seems to me that your prior has to reflect [...]
Archive of posts filed under the Bayesian Statistics category.
Philosophy of Bayesian statistics: my reactions to Senn
Continuing with my discussion of the articles in the special issue of the journal Rationality, Markets and Morals on the philosophy of Bayesian statistics: Stephen Senn, “You May Believe You Are a Bayesian But You Are Probably Wrong”: I agree with Senn’s comments on the impossibility of the de Finetti subjective Bayesian approach. As I [...]
The inevitable problems with statistical significance and 95% intervals
I’m thinking more and more that we have to get rid of statistical significance, 95% intervals, and all the rest, and just come to a more fundamental acceptance of uncertainty. In practice, I think we use confidence intervals and hypothesis tests as a way to avoid acknowledging uncertainty. We set up some rules and then [...]
Philosophy of Bayesian statistics: my reactions to Cox and Mayo
The journal Rationality, Markets and Morals has finally posted all the articles in their special issue on the philosophy of Bayesian statistics. My contribution is called Induction and Deduction in Bayesian Data Analysis. I’ll also post my reactions to the other articles. I wrote these notes a few weeks ago and could post them all [...]
How many parameters are in a multilevel model?
Stephen Collins writes: I’m reading your Multilevel modeling book and am trying to apply it to my work. I’m concerned with how to estimate a random intercept model if there are hundreds/thousands of levels. In the Gibbs sampling, am I sampling a parameter for each level? Or, just the hyper-parameters? In other words, say I [...]
Using predator-prey models on the Canadian lynx series
The “Canadian lynx data” is one of the famous examples used in time series analysis. And the usual models that are fit to these data in the statistics time-series literature, don’t work well. Cavan Reilly and Angelique Zeringue write: Reilly and Zeringue then present their analysis. Their simple little predator-prey model with a weakly informative [...]
Judea Pearl on why he is “only a half-Bayesian”
In an article published in 2001, Pearl wrote: I [Pearl] turned Bayesian in 1971, as soon as I began reading Savage’s monograph The Foundations of Statistical Inference [Savage, 1962]. The arguments were unassailable: (i) It is plain silly to ignore what we know, (ii) It is natural and useful to cast what we know in [...]
Stan: A (Bayesian) Directed Graphical Model Compiler
Here’s Bob’s talk from the NYC machine learning meetup. And here’s Stan himself:
Prior beliefs about locations of decision boundaries
Forest Gregg writes:
Bob on Stan
Thurs 19 Jan 7pm at the NYC Machine Learning meetup. Stan‘s entirely publicly funded and open-source and it has no secrets. Ask us about it and we’ll tell you everything you might want to know. P.S. And here‘s the talk.
Bayesian Anova found useful in ecology
David LeBauer points me to this article in PLoS One by Andy Hector, Thomas Bell, Yann Hautier, Forest Isbell, Marc Kéry, Peter Reich, Jasper van Ruijven, and Bernhard Schmid. Here’s the abstract: The idea that species diversity can influence ecosystem functioning has been controversial and its importance relative to compositional effects hotly debated. Unfortunately, assessing [...]
Bayesian Page Rank?
Loren Maxwell writes:
More by Berger and me on weakly informative priors
A couple days ago we discussed some remarks by Tony O’Hagan and Jim Berger on weakly informative priors. Jim followed up on Deborah Mayo’s blog with this: Objective Bayesian priors are often improper (i.e., have infinite total mass), but this is not a problem when they are developed correctly. But not every improper prior is [...]
Bayes in astronomy
David Schminovich points me to this paper by Yu Lu, H. Mo, Martin Weinberg, and Neal Katz: We believe that a wide range of physical processes conspire to shape the observed galaxy population but we remain unsure of their detailed interactions. The semi-analytic model (SAM) of galaxy formation uses multi-dimensional parameterisations of the physical processes [...]
Path sampling for models of varying dimension
Somebody asks:
“Keeping things unridiculous”: Berger, O’Hagan, and me on weakly informative priors
Deborah Mayo sent me this quote from Jim Berger: Too often I see people pretending to be subjectivists, and then using “weakly informative” priors that the objective Bayesian community knows are terrible and will give ridiculous answers; subjectivism is then being used as a shield to hide ignorance. . . . In my own more [...]
Christopher Hitchens was a Bayesian
1. We Bayesian statisticians like to say there are three kinds of statisticians: a. Bayesians; b. People who are Bayesians but don’t realize it (that is, they act in coherence with some unstated probability); c. Failed Bayesians (that is, people whose inference could be improved by some attention to coherence). So, if a statistician does [...]
Rational Turbulence
Kent Osband, author of “Pandora’s Risk: Uncertainty at the Core of Finance,” sent along this paper: Fluids are turbulent when tiny differences in space or time make for gross differences in behavior. The mathematical signature of turbulence is an endless moment or cumulant hierarchy. Bayesian tracking of continuous-time processes turns out to have a similar [...]
Martyn Plummer’s Secret JAGS Blog
Martyn Plummer, the creator of the open-source, C++, graphical-model compiler JAGS (aka “Just Another Gibbs Sampler”), runs a forum on the JAGS site that has a very similar feel to the mail-bag posts on this blog. Martyn answers general statistical computing questions (e.g., why slice sampling rather than Metropolis-Hastings?) and general modeling (e.g., why won’t [...]
Neutral noninformative and informative conjugate beta and gamma prior distributions
Jouni Kerman did a cool bit of research justifying the Beta (1/3, 1/3) prior as noninformative for binomial data, and the Gamma (1/3, 0) prior for Poisson data. You probably thought that nothing new could be said about noninformative priors in such basic problems, but you were wrong! Here’s the story: The conjugate binomial and [...]
David MacKay and Occam’s Razor
In my comments on David MacKay’s 2003 book on Bayesian inference, I wrote that I hate all the Occam-factor stuff that MacKay talks about, and I linked to this quote from Radford Neal: Sometimes a simple model will outperform a more complex model . . . Nevertheless, I believe that deliberately limiting the complexity of [...]
Stan uses Nuts!
We interrupt our usual program of Ed Wegman Gregg Easterbrook Niall Ferguson mockery to deliver a serious update on our statistical computing project. Stan (“Sampling Through Adaptive Neighborhoods”) is our new C++ program (written mostly by Bob Carpenter) that draws samples from Bayesian models. Stan can take different sorts of inputs: you can write the [...]
Bayes wikipedia update
I checked and somebody went in and screwed up my fixes to the wikipedia page on Bayesian inference. I give up.
Of hypothesis tests and Unitarians
Xian, Judith, and I read this line in a book by statistician Murray Aitkin in which he considered the following hypothetical example: A survey of 100 individuals expressing support (Yes/No) for the president, before and after a presidential address . . . The question of interest is whether there has been a change in support [...]
I got 99 comparisons but multiplicity ain’t one
After I gave my talk at an econ seminar on Why We (Usually) Don’t Care About Multiple Comparisons, I got the following comment: One question that came up later was whether your argument is really with testing in general, rather than only with testing in multiple comparison settings. My reply: Yes, my argument is with [...]