“Schools of statistical thoughts are sometimes jokingly likened to religions. This analogy is not perfect—unlike religions, statistical methods have no supernatural content and make essentially no demands on our personal lives. Looking at the comparison from the other direction, it is possible to be agnostic, atheistic, or simply live one’s life without religion, but it is not really possible to do statistics without some philosophy.”

This bit is perhaps worth saying again, especially given the occasional trolling on the internet by people who disparage their ideological opponents by calling them “religious” . . . So here it is:

Sometimes the choice of statistical philosophy is decided by convention or convenience. . . . In many settings, however, we have freedom in deciding how to attack a problem statistically. How then do we decide how to proceed?

Schools of statistical thoughts are sometimes jokingly likened to religions. This analogy is not perfect—unlike religions, statistical methods have no supernatural content and make essentially no demands on our personal lives. Looking at the comparison from the other direction, it is possible to be agnostic, atheistic, or simply live one’s life without religion, but it is not really possible to do statistics without some philosophy. Even if you take a Tukeyesque stance and admit only data and data manipulations without reference to probability models, you still need some criteria to evaluate the methods that you choose.

One way in which schools of statistics are like religions is in how we end up affiliating with them. Based on informal observation, I would say that statis- ticians typically absorb the ambient philosophy of the institution where they are trained—or else, more rarely, they rebel against their training or pick up a philosophy later in their career or from some other source such as a persuasive book. Similarly, people in modern societies are free to choose their religious affiliation but it typically is the same as the religion of parents and extended family. Philosophy, like religion but not (in general) ethnicity, is something we are free to choose on our own, even if we do not usually take the opportunity to take that choice. Rather, it is common to exercise our free will in this setting by forming our own personal accommodation with the religion or philosophy bequeathed to us by our background.

For example, I affiliated as a Bayesian after studying with Don Rubin and, over the decades, have evolved my own philosophy using his as a starting point. I did not go completely willingly into the Bayesian fold—the first statistics course I took (before I came to Harvard) had a classical perspective, and in the first course I took with Don, I continued to try to frame all the inferential problems into a Neyman-Pearson framework. But it didn’t take me or my fellow students long to slip into comfortable conformity. . . .

Beliefs and affiliations are interesting and worth studying, going beyond simple analogies to religion.

P.S. See here for some similar thoughts from a few years ago. The key point is that a belief is not (necessarily) the same thing as a religion, and I don’t think it’s helpful for people to use “religion” as a generalized insult that is applied to beliefs that they disagree with.

19 thoughts on ““Schools of statistical thoughts are sometimes jokingly likened to religions. This analogy is not perfect—unlike religions, statistical methods have no supernatural content and make essentially no demands on our personal lives. Looking at the comparison from the other direction, it is possible to be agnostic, atheistic, or simply live one’s life without religion, but it is not really possible to do statistics without some philosophy.”

  1. I wonder if illuminating a “phylogenetic” tree of a statistical method could be, say using paper citations or advisor/student relationships. This would include the simultaneous development of the same idea and the merging of competing ideas into a single method. Naturally one could do this for Fisherian NHST as Bayesian parameter estimation. No doubt people who study networks have some way to tackle this problem.

    For what it’s worth, I think religions are worth studying from this same perspective as well.

  2. I usually refer to these as “metaphysics”. They are philosophies, but a specific kind of philosophy – the grounding of a statistical practice in claims about how the world is (as distinct from epistemology, which would relate the metaphysics to how we make claims about truth – the value of NHST is an epistemological problem, whereas the perspective of frequentism is a metaphysics).

    I think the “religion” thing from from two sources. First, religions and metaphysics have always been close at hand, because metaphysical claims about the world can’t really be scientific claims – at least not in the Popperian falsification sense. So seeing metaphysical claims as religious claims has some sense, if we think that all non-scientific beliefs about the universe are inherently “religious” in some sense (I don’t necessarily think that, just sayin). Second, for partisans, metaphysics can easily become dogmatic ideologies when applied to the concrete world, and ideology and religion are also closely linked (though you can certainly be a non-religious ideologue). So when we see people making impassioned but non-scientific claims about the nature of the world, it sort of feels like religion, even if its not.

    But in general, I think these two conflations miss the point that, at base, frequentism and bayesianism (as best as I can tell) are metaphysics (or meta-statistics, if you like).

  3. jrc: It’s probably true that the philosophies referred to here are understood by many in a metaphysical way. But they could also be understood in a different way, namely as choices of a point of view; choices of how to explain and to use certain man-made concepts.

    If I model aspects of the world in a frequentist manner, I do *not* think that in the world indeed there are frequentist probabilities taken literally. I rather think that this is a helpful tool to bring some useful structure into my world-view.

    Actually I’m grateful for your comment because I think that “statistical philosophies are taken as metaphysics” is a quite good description of what is wrong with the way these philosophies are understood by too many people. (I’d not be much happier if they’d be understood as “beliefs”, by the way.)

    • Thanks Christian – that is an important point, I just wasn’t clear in my post.

      I think of these as “metaphysics” because, if the assumptions are True, then the statistics work out right. But that doesn’t mean I think that any of these metaphysics is the Truth. I’m with you – I think they are useful metaphors that allow us to do useful things with data. And people should choose which form of thinking is appropriate for the task at hand.

      My point should have been more clear on the fact these are somewhat absurd metaphysics if you try to literally apply them to social science research (or even much “hard science”). Absurd, but, as you say, sometimes useful, and different ones useful for different purposes.

  4. I just think it reflects badly on the credibility of a scientific field when you have a long standing, deep schism within its experts on a core topic. How do we retain objectivity if practical answers depend on a philosophical choice of method?

    • I think its less that the answers depend on a philosophical choice of method, and more that for some applications one philosophy will make more sense in relation to the problem at hand (or make more of the problem make sense). Or, for that matter, that one set of methods has been developed for that problem and that set was developed from some metaphysics/philosophy.

      I would think that when predicting elections, putting some statistical structure on the process by modelling constraints and using information from previous studies makes sense – the opinions of the electorate probably haven’t changed completely overnight. The Bayesian metaphysics provides a theoretical grounding and subsequent set of statistical techniques for doing that.

      But when I’m trying to estimate determinants of people’s labor market or health outomes, I’m focused mostly on capturing some specific change that happened in the world and crafting a statistical model that latches on to that in the data. For that purpose, we (we being econometricians from years passed) have found it convenient to model the outcome Y as some linear function of observables (X) and coefficients (Beta) plus an error term. Conceiving that error term as a draw from a random distribution, while the Betas are interpreted as constant parameters in the world, provides theoretical grounding for the econometric methods.

      Personally, I tend towards anti-realism about most of this. Much of the time I think of error terms as “what we can’t explain” (almost as residuals, but not exactly). That makes it much easier for me to accept that in some cases we might want to use one philosophical ground for conceiving of randomness and its relation to the world, and in other cases another ground. The ground of “its all model mis-specification and the glory of human free will” doesn’t go real far mathematically.

      Maybe I’ll change my mind about the relative values of these ways of thinking as I get better at this, but they all seem valuable to me.

      • If the answers don’t depend on the choice of philosophy it’s hard to justify why each side cares so passionately about which method gets used. The same argument applies, even if one method has some fairly incremental or marginal & variable advantage or disadvantage over the other depending on situation.

        OTOH, you could indeed argue that one philosophy works substantially better at one class of problems & vice versa. Fine. But then one must be able to clearly delineate these problem classes. Can we, with any broad consensus within the profession?

        • If the “choice of philosophy” basically amounts to a different angle from which we look at things, this means different results just mean that the thing looks different when looked at from different angles, i.e., it has different aspects. These aspects may seem contradictory but they are not (unless somebody got something really wrong). What is wrong is that people claim that only one of them can be true.
          Depending on the class of problem, we can have more use for one aspect than another, and we may also believe that it is hard to justify why a certain angle to look at the thing should be preferred to a certain other one, but still, in principle what you see from all points of view adds to the knowledge.
          I don’t think statistics should be embarrassed by the fact that the knowledge that you get if you try to piece all of this together is rather complex and that people may go in different directions if they try to milk a simple message out of it.

        • -1

          Christian,

          I remember last year, Wasserman put out a post where he said it was an important open problem in kernel density estimation to get a CI for the probability at a point. In my understanding of probabilities this is a complete waste of time and is never the right thing to do. Far from being an interesting example, it’s a horrifying example of the how much mathematical talent Frequentism has wasted over the years. So how is that merely a two different ways of getting the same result?

          In Statistical Mechanics, Frequentists believe the goal is write down distributions which are equal to the frequency of occupation of microstates. Bayesians believe that frequency distribution is never known, and the real goal is to write down (non-frequency) distributions which represent our ignorance of the microstate given some macroscopic measurements. Depending on which view you take, it’ll drive your research and results in completely different directions. So how is that merely two different ways to getting the same result?

          Your philosophy of statistics determines your research direction and which questions you think are worth answering. To say that it’s unimportant is to say the direction your research takes is unimportant. For many I’m sure it is because their research is crap anyway, but I doubt it’s true for everyone.

        • Also, pluralism in statistics is great for two things: (1) treating your fellow statisticians decently (i.e. not denying them tenure just because their not on your ‘side’) and (2) allowing you to borrow (steal) mathematics and intuition regardless of their source.

          But pluralism in philosophy of statistics achieves nothing. One’s time is better spent improving your philosophy of statistics rather than adopting multiple versions.

        • Vaguely, and abusing philosophical jargon, I think the different approaches to different specific questions involve different ‘modelling ontologies’, ie primitive concepts/objects, and the questions you ask and the answers you get do in fact depend on these choices. I also don’t think this is necessarily a bad thing.

          Connected with that I think you can have a different ‘modelling epistemology’, ie how you learn about/use your models, even with the same model ontology. So Andrew can say he is falsificationist, even while mostly working with a (somewhat) Bayesian modelling ontology of fundamental objects. Again this is not necessarily a bad thing, but does come with Tower of Babel issues, to continue the religious theme.

          I think it’s also possible to wear more than one hat depending on the problem – sometimes you choose your method/model conditional on the problem, sometimes your problems conditional on your methods/model. I learn as much if not more from the former than the latter.

          The question is how to reconcile all this with your more purely philosophical (what you ‘really’ think) views on ontology/epistemology. Personally if I have to choose I’d try to hold on to roughly consistent epistemological views (general principles of how I like to learn about something) while being more flexible with the ontology (what I’m learning about), varying the latter to fit (model) the problem.

          But again clearly these are often interconnected and the choice of model ontology is as much open to criticism as the choice of methods to learn about the model. Also, often I (shamefully?) do the reverse – choose a model type because I know it and use a somewhat dubious question/calculational method because it can be answered/carried out! This freedom of choice to me is important – both the dogmatic or narrow minded and the unprincipled opportunists have roles to play in contributing to our collective knowledge.

        • I should say that I think that all of what I just said is roughly recapitulating the general points of jrc and Christian, who I broadly agree with.

        • “I should say that I think that all of what I just said is roughly recapitulating the general points of jrc and Christian, who I broadly agree with.”
          Good you say this, otherwise I’d have agreed with your points as if this had doubled the support for both of us. ;-)

  5. Nice article. Maybe a bit too reasonable :-)

    Outside of statistics I’ve certainly been ‘raised’ in certain scientific traditions that have both strong support from those within and vehement opposition from those favouring competing traditions. I’d say it’s pretty common in science, as in life.

    While I’ve often tried to be somewhat pluralistic/opportunistic I’ve definitely found myself being unfairly biased towards my original methods, especially in the short term. Like you say, I’ve already learnt them and can often translate some rough approximation to the ideas I like from other areas into the language I already know.

    On the longer timescale I know I now also use methods or ideas I previously disavowed, probably for ‘religious’ reasons! Which also happens with real life religious folk, presumably.

  6. I found my view of this theological split and apparent sectarianism was satisfied by Andrew’s set up debate between supposed proponents of one side or another at some international congress: what I got from the transcript was no problem at all: what’s wrong with proper formulations of the null hypothesis which Andrew regarded as necessary while his ‘adversary’ spoke encouragingly of using any Bayesian techniques; conclusion: phony debate except about the correct use of p-values?

    I doubt this helps and I’m not a statistician but can somebody point me to an online copy of this debate. I looked for it again on Andrew’s publication list but couldn’t find it.

    Cheers

  7. I stopped worrying about this split (but I’m a programmer, not a statistician) when I read the transcript of a debate on the supposed ‘theological’ split at some international congress of statisticians between Andrew and somebody.

    Iirc Andrew pointed out the need to use the null-hypothesis and his interlocuter voiced no objections to any Bayesian method. The misuse of p-values (a technical question) seems to be more important issue.

    I was looking for a copy of this transcript again a few weeks ago but couldn’t find it. Can somebody please point me to an online copy?

Leave a Reply to Christian Hennig Cancel reply

Your email address will not be published. Required fields are marked *