The *anthropic principle* in physics states that we can derive certain properties of the world, or even the universe, based on the knowledge of our existence. The earth can’t be too hot or too cold, there needs to be oxygen and water, etc., which in turn implies certain things about our solar system, and so forth.

In statistics we can similarly apply an anthropic principle to say that the phenomena we can successfully study in statistics will not be too small or too large. Too small a signal and any analysis is hopeless (we’re in kangaroo territory); too large and statistical analysis is not needed at all, as you can just do what Dave Krantz calls the Inter-Ocular Traumatic Test.

You never see estimates that are 8 standard errors away from 0, right?

**An application of the anthropic principle to experimental design**

I actually used the anthropic principle in my 2000 article, Should we take measurements at an intermediate design point? (a paper that I love; but I just looked it up and it’s only been cited 3 times; that makes me so sad!), but without labeling the principle as such.

From a statistical standpoint, the anthropic principle is not magic; it won’t automatically apply. But I think it can guide our thinking when constructing data models and prior distributions. It’s a bit related to the so-called method of imaginary results, by which a model is understood based on the reasonableness of its prior predictive distribution. (Any assessment of “reasonableness” is, implicitly, an invocation of prior information not already included in the model.)

**Using the anthropic principle to understand the replication crisis**

Given that (a) researchers are rewarded for obtaining results that are 2 standard errors from zero, and (b) There’s not much extra benefit to being much *more* than 2 standard errors from zero, the anthropic principle implies that we’ll be studying effects just large enough to reach that level. (I’m setting aside forking paths and researcher degrees of freedom which allow researchers on ESP, beauty and sex ratios, etc., to attain statistical significance from what is essentially pure noise.) This implies high type M errors, that is, estimated effects that are overestimates (see Section 2.1 of this paper).

**A discussion with Ken Rice and Thomas Lumley**

I was mentioning this idea to Ken Rice when I was in Seattle the other day, and he responded:

Re: the anthropic idea, that (I think) statisticians never see really large effects relative to standard errors, because problems involving them are too easy to require a statistician, this sounds a lot like the study of local alternatives, e.g., here and here.

This is old news, of course, but an interesting (and unpublished) variation on it might be of interest to you. When models are only locally wrong from what we assume, it’s possible we can’t ever reliably detect the model mis-specificiation based on the data, but yet that the mis-specification really does matter for inference. Thomas Lumley writes about this here and here.

At first this was highly non-intuitive to me and others; perhaps we were all too used to thinking in the “classic” mode where the estimation problem stays fixed regardless of the sample size, and this phenomenon doesn’t arise. I suspect Thomas’ work has implications for “workflow” ideas too – that for sufficiently-nuanced inferences we can’t be confident that the steps that led them being considered are reliable, at least if they were based on the data alone. Some related impossibility results are reviewed here.

And Lumley points to this paper, Robustness of semiparametric efficiency in nearly-true models for two-phase samples.