I came across this article on the philosophy of statistics by University of Michigan economist John DiNardo. I don’t have much to say about the substance of the article because most of it is an argument against something called “Bayesian methods” that doesn’t have much in common with the Bayesian data analysis that I do.
If an quantitative, empirically-minded economist at a top university doesn’t know about modern Bayesian methods, then it’s a pretty good guess that confusion holds in many other quarters as well, so I thought I’d try to clear a couple of things up. (See also here.)
In the short term, I know I have some readers at the University of Michigan, so maybe a couple of you could go over to Prof. DiNardo’s office and discuss this with him? For the rest of you, please spread the word.
My point here is not to claim that DiNardo should be using Bayesian methods or to claim that he’s doing anything wrong in his applied work. It’s just that he’s fighting against a bunch of ideas that have already been discredited, within Bayesian statistics. It’s all well and good to shoot at arguments that are already dead, but I think it’s also a good idea to be aware of the best and most current work in a field that you’re criticizing.
To be specific, see pages 7-8 of DiNardo’s article:
1. DiNardo thinks that Bayesians believe that the data generating mechanism is irrelevant to inference. To which I reply, (a) I’m a Bayesian and I believe that the data generating is relevant to inference, and (b) We discuss this in detail in chapter 7 of BDA.
2. DiNardo thinks that stopping rules are irrelevant to Bayesians. Nope. see the example starting in the middle of page 163 in BDA (all references here are to the second edition).
3. DiNardo thinks that Bayesians think that the problem of “how to reason” has been solved. Nope. A Bayesian inference is only as good as its models. Bayesians are busy developing, applying, checking, and extending new classes of models. Consider, for example, the explosion of research in nonparametric Bayes in recent years. Given that “how to reason Bayesianly” includes model specification, and given that model specification is wide open, we would not at all claim that the problem of reasoning has been solved.
4. DiNardo thinks that Bayesians think that “usual (non-Bayesian) practice is very badly wrong.” Sometimes it is, sometimes not. See BDA and ARM for lots of examples. And see here for some inferences that would be difficult to do using classical inference. (I’m not saying it can’t be done, just that it would require a lot of effort, just to reproduce something that can be done straightforwardly using Bayesian methods.)
5. DiNardo thinks that Bayesians think that “randomization rarely makes sense in those contexts where it is most often employed.” Nope. Forget Urbach (1985), whoever he is. Instead, check out section 7.6 of BDA.
6. DiNardo thinks that Bayesians think that “probability does not exist.” Nope. Check out chapter 1 of BDA, in particular the examples of calibrations of record linkage and football point spreads.
The paradox of philosphizing
DiNardo remarks, perhaps accurately, that the literature on the philosophy of statistics is dominated by Bayesians with extreme and often nutty views. And this frustrates him. But here’s the deal: A lot of the good stuff is not explicitly presented as philosophy.
When we wrote Bayesian Data Analysis, we were careful not to include the usual philosophical arguments that were at that time considered standard in any Bayesian presentation. We decided to skip the defensiveness and just jump straight to the models and the applications. This worked well, I think, but it has led the likes of John DiNardo and Chris Burdzy (as discussed earlier on this blog) to not notice the philosophical content that is there.
And this is the paradox of philosophizing. If we had put 50 or 100 pages of philosophy into BDA (rather than discussing model checking, randomization, the limited range of applicability of the likelihood principle, etc., in separate places in the book), that would’ve been fine, but then we would’ve diluted our message that Bayesian data analysis is about what works, not about what is theoretically coherent. Many people find philosophical arguments to be irritating and irrelevant to practice. Thus, to get to the point, it can be a good idea to avoid the philosophical discussions. But, as the saying goes, if philosophy is outlawed, only outlaws will do philosophy. DiNardo is responding to the outlaws. I hope this blog will wake him up and make him see the philosophy that is all around him every day.
P.S. DiNardo does recognize the diversity of Bayesian approaches:
In what follows, when I [DiNardo] describe something as “Bayesian” I do not mean to suggest any writer in particular holds all the views so attributed here. There is considerable heterogeneity: some view concepts like “the weight of evidence” as important, others do not. Some view expected utility as important, other do not. This is not intended to be a “primer” on Bayesian statistics. Neither is it intended to be a “critique” of Bayesian views. . . . My purpose is not to do Bayesian ideas justice (or injustice!) but rather, to try selectively choose some implications of various strands of Bayesianism and non-Bayesianism for actual statistical practice that highlight their differences so as to be clear to a non-Bayesian perspective.
But . . . it’s all about what goes into the “various strands” that DiNardo selects. If he were to compare the applied relevance of, say, BDA and ARM, to the applied relevance of a classical text such as that of LeCam, I think he’d be seeing quite a different picture in terms of relative usefulness of the Bayesian and non-Bayesian approach.
What I suspect–any readers who know DiNardo can ask him directly–is that he is simply unaware of the modern approach to Bayesian data analysis which is based on modeling and active model checking (“severe testing,” to use the phrase of Deborah Mayo). I don’t expect that seeing my books would make DiNardo a convert to the Bayesian approach, but it might make him realize that practical Bayesians such as myself are not quite as silly as he might imagine.
P.P.S. I cite my own writings because that’s what I’m most familiar with. It should be easy enough to form a similar reply using the words of others.