Skip to content

I hate the so-called Fisher exact test: a pointer to one of my favorite articles

I’ve made this point before, but I just received an email on the topic and so I thought I’d point youall to section 3.3 of this article of mine from 2003 where I make the argument in detail.

This article–A Bayesian Formulation of Exploratory Data Analysis and Goodness-of-fit Testing–is one of my favorites. It also features:
– A potted history of Bayesian inference (section 2.1)
– The first published definition (I think) of U-values and P-values (section 2.3)
– A model-checking perspective on the problem of degenerate estimates for mixture models (section 3.1)
– Why this isn’t all obvious (section 5)

The article is based on a presentation I gave a year earlier at a conference. It was supposed to appear in the proceedings volume, but it was late, and the conference organizer was so annoyed he refused to include it. So I published it in the International Statistical Review instead. A year later I published a related article, Exploratory Data Analysis for Complex Models, as a discussion paper in the Journal of Computational and Graphical Statistics. That second article is more coherent, but personally I prefer the International Statistical Review article because it covers so many little topics that don’t fit into existing theories of inference. I think of these examples as analogous to the quantum anomalies that toppled classical physics around 1900. In this case, what I want to topple is classical Bayesian inference–by which I mean Bayesian theory that does not include model building and model checking.

One Comment

  1. Matthieu says:

    My personal feeling about that test is that the semantic has helped its spreading more than anything else. Just because the word "exact" is in the title, it is almost like the Delphic Oracle and conveys an impression of objectivity (or more like it, a scrupulous ignorance). I saw it widely used in genomics and other-omics fields, where I have the impression people routinely put a lot of longlines to fish statistical significance.