This paper discusses the dual interpretation of the Jeffreys– Lindley’s paradox associated with Bayesian posterior probabilities and Bayes factors, both as a differentiation between frequentist and Bayesian statistics and as a pointer to the difficulty of using improper priors while testing. We stress the considerable impact of this paradox on the foundations of both classical and Bayesian statistics.
I like this paper in that he is transforming what is often seen as a philosophical argument into a technical issue, in this case a question of priors. Certain conventional priors (the so-called spike and slab) have poor statistical properties in settings such as model comparison (in addition to not making sense as prior distributions of any realistic state of knowledge). This reminds me of the way that we nowadays think about hierarchical models. In the old days there was much thoughtful debate about exchangeability and the so-called Stein paradox that partial pooling could lead to improved estimates. Nowadays we realize that the key issue is not “exchangeability” (a close-to-meaningless criterion in that it just is the requirement that the data from the different groups be treated symmetrically) but rather the model that is being used for the distribution of the varying parameters. Switch from a normal distribution to, say, a bimodal distribution, and the Stein-like pooling goes away. Lots of anguished philosophy is replaced by probability modeling.