Xian and I respond to the four discussants of our paper, “Not only defended but also applied”: The perceived absurdity of Bayesian inference.” Here’s the abstract of our rejoinder:
Over the years we have often felt frustration, both at smug Bayesians—in particular, those who object to checking of the fit of model to data, either because all Bayesian models are held to be subjective and thus unquestioned (an odd combination indeed, but that is the subject of another article)—and angry anti-Bayesians who, as we wrote in our article, strain on the gnat of the prior distribution while swallowing the camel that is the likelihood. The present article arose from our memory of a particularly intemperate anti-Bayesian statement that appeared in Feller’s beautiful and classic book on probability theory. We felt that it was worth exploring the very extremeness of Feller’s words, along with similar anti-Bayesian remarks by others, in order to better understand the background underlying controversies that still exist regarding the foundations of statistics. We thank the four discussants of our article for their contributions to our understanding of these controversies as they have existed in the past and persist today.
Why was Feller both so stridently anti-Bayesian and so confident that non-Bayesian methods could solve all statistical problems? We locate Feller’s attitude within a post-WW2 “anti-Bayesian moment” in which Bayesian inference was perceived as a threat to the dominance of non-Bayesian methods, which were mature enough to have solved problems yet new enough to still appear to have limitless promise.
Our article that started off the discussion is here.
Only mildly tangential: did you see today’s xkcd? (http://xkcd.com/1132/). Pretty funny, though perhaps it counts as “smug” Bayesianism?
I saw this xkcd as well, I didn’t quite get it (and I usually love xkcd). Seemed like a rare instance of the author getting something wrong and falling for a fallacy. Maybe I’m way off base and just misunderstanding the cartoon, but isn’t the cartoon based on the fallacy that Deborah Mayo discusses in this post?
http://errorstatistics.com/2012/09/08/return-to-the-comedy-hour-on-significance-tests/
Jeremy:
I agree 100%. Even before I finished reading this cartoon I sent it to the Mayo simulator in my brain, which immediately reported that the cartoon was presenting a cartoonish distortion of frequentist statistics. I will blog on this.
While everyone will make mistakes if they post enough stuff on the internet (lord knows I have), I have to say this one surprised me. There are other xkcd cartoons where the joke is based on a firm grasp of textbook frequentist statistics, like the one about correcting for multiple comparisons in unplanned post-hoc tests. Plus, you can make anything sound stupid if you summarize it badly enough–think of the joke about a Bayesian being a man who expects to see a horse, sees a donkey, and decides he’s seen a mule. xkcd doesn’t usually get its humor from such feeble sources as bad summaries.
I’d be curious to know what the author was thinking when he drew this. Like what Gary Larson did in that book of his where he explained the “backstory” and development of many of his cartoons, including a bunch of cartoons that didn’t work.
We’ll see. Maybe he’ll post a comment on my blog post tomorrow which will address this.
I think it’s an exaggeration to say these distortions are “unrecognizable”. Medical, social science, and psychological research often make this fallacy in statistical analysis, it’s just not as apparent because the low base rate isn’t as obvious as it is here. Ioannidis’s arguments regarding the sparsity of true hypotheses could even be seen as an example of this mistake.
It also seems to me that some frequentists want to have it both ways, they’ll mock Bayesians for tilting at windmills in trying to upend entrenched methodology, but then when it’s pointed out that the current practices have effectively institutionalized statistical fallacies into the scientific literature, they disown any responsibility for it.
“Cartoonish distortion” Isn’t that sort of the point of XKCD? Logic based surrealism. Sure not all frequentists (Mayo e.g)subscribe to such fallacies, but it is not hard to find researchers who do. In this vein: http://xkcd.com/892/
Maybe he should used a different term: maybe Nullists.
Well, I’ve never heard of anyone but the one very odd-sounding Bayesian Mayo links to who actually subscribes to the particular fallacy on which this cartoon seems to be based. So while yes, “cartoonish distortion” is the basis of a lot of humor (not just xkcd’s), if the distortion is so great as to be unrecognizable, it stops being very funny. But I freely grant that that’s very much in the eye of the beholder. Clearly, others commenting here found the frequentist in the cartoon recognizable enough to find the cartoon funny.
Agree. It’s depressingly easy to find researchers who take p-values above/below 0.05 as the arbiter of absolute truth, with zero consideration given to power. Perhaps unsurprisingly, it’s also easy to find statistics instructors teaching people to do things that way.
Came here to see if someone referred to this xkcd comic. Wasn’t disappointed.
Would you mind to link to the comments to the paper? I already read the original paper at Arxiv and this rejoinder. Thanks.
Manoel:
I don’t know if these comments are online.
The cartoon is exaggerated — few cartoons would be funny if they weren’t — but I don’t see the fundamental problem with it. Maybe I’m missing something.
Suppose I had a medical test with a 1/6 false positive rate and a 0% false negative rate. That is, if administered to someone without the disease it has a 1/6 chance of reporting positive. The protocol is to administer the test and, if positive, to administer it again. Assuming independence, the probability of two consecutive false positives is 1/36. Some statisticians would reject the null hypothesis (that the patient is disease free) given 2/2 positive tests. That is ridiculous for the same reason the xkcd example is ridiculous (it ignores prior or base rate information) but is is indeed the practice in some circles, I’m told.
Pingback: How Bayesian Challenge Frequentist « Honglang Wang's Blog