From an email I received the other day:
Things are going much better now — it’s interesting, it feels like with both of my models, parameters are slow to converge or get “stuck” and have trouble mixing when the model is somehow misspecified.
See here for a statement of the folk theorem.
It occurs to me at this very moment that there’s a tension between the folk theorem and parameter expanded (PX) Gibbs sampling. PX copes with a model which is perfectly well-specified but encounters slow mixing caused by extreme misalignment of the (local) principle axes of the conditional posteriors and the directions in which sampling is taking place. (Since the folk theorem isn’t actually a theorem, I call “tension” rather than “contradiction”.)
…In fact, isn’t it fair to say that advances in MCMC are driven primarily by the need to cope with correctly-specified models that mix slowly under most or all available algorithms?
(Not that I’m trying to deprecate the folk theorem as a useful heuristic!)
Corey:
PX is actually the original example (for me) of the folk theorem; see Section 5.2 of this paper from 2004, which is the original source for the Folk Theorem and the Pinocchio Principle (although I hadn’t named them at that point).
I’d respond in more detail but I’m too busy responding to trolls on other comment threads. . . .
Pingback: Somewhere else, part 51 | Freakonometrics