My blog discussion with Eyal Shahar (see comments #3 and onward here) reminded me of a persistent challenge I face when talking with outsiders about Bayesian statistics.
Shahar’s basic objection to Bayesian methods is that (a) they’re are based on subjective probability, betting, etc., and (b) that’s not scientific. As he or she put it, “No doubt that Bayesian statistics offer a coherent system of logical inference, but its problem is irrelevance to the business of science.” My reply is to diagree with (a)–I claim that Bayesian data analysis is no more subjective than any other approach, I do not in general think of prior probabilities as degrees of belief, and I am not in general interested in estimating the probability that a hypothesis is true. But somehow it was difficult to make this point.
I had a similar frustrating feeling when talking about 15 years ago with one of my Berkeley colleagues: he said that he didn’t believe in Bayesian statistics, but if he did, he would want to use the real stuff, i.e., subjective probabilities. But there’s nothing in the mathematics that requires the probabilities to be subjective. And, just because some people use betting as a justification for this approach, that doesn’t mean that I have to use that justification.
This is one reason why, in writing Bayesian Data Analysis, we minimized philosophy and focused on empirical justifications, i.e., the method works in lots of examples.
The claim that science is not subjective seems to fly in the face of much of the sociology of science. Put simply, science has to be subjective (at some level) because it's done by people.
I tend to think of Bayesian statistics as a prescription for how to learn. You start out with your pre-data knowledge / beliefs, subjective though it may be, and your beliefs about the likelihood function, stir in some data, and now what should you know / believe? Bayesian statistics tells you what you should believe if you make the best use of the data available to you, given where you started from. It provides you with the best learning mechanism you can devise.
This also addresses Larry Wasserman's comment in his rejoinder to your paper along the lines of, if I believe your subjective prior, why not just show you the data and ask you for your subjective posterior? The response is: because that subjective posterior will have been formed using a very inefficient process, and you can make better use of the data the Bayesian way.
And that's not scientific. I'd hate to ask him what he thinks of psychology. Or the right way to study such problems as depression.
No argument–serious or funny–can convince some of us that scientific knowledge amounts to the shaping of beliefs about reality
–Eyal Shahar (in previously linked correspondence)
Well. There you go.
"its problem is irrelevance to the business of science"
That sound you heard was NMR labs all over the world scratching their head in unison.
Andrew, why didn't you mention Jaynes to Shahar? S/he seems like the kind of person who would find "probability theory is an extension of classical logic — the unique extension of classical logic given a certain small set of axioms" a lot more convincing than Dutch book arguments.
Corey,
Yeah, Jaynes is great–his writings were a big influence on my formulation of bayesian data analysis as assumption/inference/checking/expansion. I don't completely buy Jaynes's philosophy but I like what he did in particular examples. And I like that he presents models as logical rather than subjective entities. He didn't know about weakly informative priors, but I don't hold that against him, considering that I didn't know about weakly informative priors until two years ago.
The "Bayesian is subjective and Frequentist is objective" argument is a red herring. It's simply not true that a Bayesian choice is always subjective and a Frequentist approach is objective. Consider 2 situations. In Situation A
We are in a poorly-understood branch of science and so the form of the likelihood is controversial. A frequentist approach is used and there is not a lot of data. In Situation B we are in a well-understood branch of science and the form of the likelihood is not controversial. A Bayesian approach is used. The number of parameters in the model is small and an informative prior is used. There is a vast amount of data, which therefore swamps the prior.
Here, Situation A clearly represents a subjectve Frequentist situation and Situation B represents an objective Bayesian one. In fact the author's preference for unbiased estimates is a subjective choice. Why not always use minimum variance estimates?
The real issue is robustness: how dependent are my results on my choice of prior and likelihood?
Betting is a useful example of decision-making under uncertainty. For one thing, it involves probabilities that must be applied to one-off events, not just repeatable events.
>" So, is this the foundation of scientific knowledge? A system of inference that is founded on axioms we should accept because otherwise, we are sure to lose a bet?"
Yes. Our calculus of uncertainty is supposed to work *all* the time. If there are situations, such as betting, where our rules give nonsensical results then we need to reject those rules in favour of ones that work all the time.
The frustration experienced by Eyal is there is almost no real Bayesian applications in fields like Epidemiology. Take the American Journal of Epidemiology as an example, you hardly see any keywords about Bayesian in the journal.
People may get an impression that these people don't understand the Bayesian methods. This is not true. As far as I know, they are not sold out to Frequentism. In fact, they are not sold out to anything. They are very pragmatic and skeptical. They will accept anything that works in real life, but also question anything that is too good.
I personally would like to see more real applications using Bayesian methods. This is more convincing than a lot of theoretical works.
Might be helpful to have more discussion distinguishing the prior used as a direct model [index] of uncertainty of a parameter
with it being used as a pragmatic model to provide good out of sample performance of inferences (i.e. "works" in lots of examples rather than providing salient posterior probabilities)
Keith
p.s. Enjoyed (re)reading Eyal Shahar and others posts
Priors are free to be subjective or not – if you want to update subjective probabilities with observation, you can do that, and if you want to update nonsubjective probabilities, you can do that too.
Deriding Bayesian statistics for relying on subjective probabilities would be like saying you hate your car because it takes you to work.
Yu,
For Bayesian methods in epidemiology, search on Sander Greenland.
Andrew,
I know Sander Greenland's work. He is pushing Bayesian methods in Epi. But as the mainstream in Epi stays, there is few applications (even Sander Greenland did not have any serious applications).
> even Sander Greenland did not have any serious applications
I thought his Multiple-bias modeling – JRSSA 2005 had a serious application [Childhood leukemia and EMF) in it.
What do you mean by serious?
(In the rejoinder he clarified that he preferred a full Bayes approach to the application)
Ths in adv
Keith