Skip to content
 

The Trumpets of Lilliput

Gur Huberman pointed me to this paper by George Akerlof and Pascal Michaillat that gives an institutional model for the persistence of false belief. The article begins:

This paper develops a theory of promotion based on evaluations by the already promoted. The already promoted show some favoritism toward candidates for promotion with similar beliefs, just as beetles are more prone to eat the eggs of other species. With such egg-eating bias, false beliefs may not be eliminated by the promotion system. Our main application is to scientific revolutions: when tenured scientists show favoritism toward candidates for tenure with similar beliefs, science may not converge to the true paradigm. We extend the statistical concept of power to science: the power of the tenure test is the probability (absent any bias) of denying tenure to a scientist who adheres to the false paradigm, just as the power of any statistical test is the probability of rejecting a false null hypothesis. . . .

It was interesting to see a mathematical model for the persistence of errors, and I agree that there must be something to their general point that people are motivated to support work that confirms their beliefs and to discredit work that disconfirms their beliefs. We’ve seen a lot of this sort of analysis at the individual level (“motivated reasoning,” etc.) and it makes sense to think of this at an interpersonal or institutional level too.

There were, however, some specific aspects of their model that I found unconvincing, partly on statistical grounds and partly based on my understanding of how social science works within society:

1. Just as I don’t think it is helpful to describe statistical hypotheses as “true” or “false,” I don’t think it’s helpful to describe scientific paradigms as “true” or “false.” Also, I’m no biologist, but I’m skeptical of a statement such as, “With the beetles, the more biologically fit species does not always prevail.” What does it mean to say a species is “more biologically fit”? If they survive and reproduce, they’re fit, no? And if a species’ eggs get eaten before they’re hatched, that reduces the species’s fitness.

In the article, they modify “true” and “false” to “Better” and “Worse,” but I have pretty much the same problem here, which is that different paradigms serve different purposes, so I don’t see how it typically makes sense to speak of one paradigm as giving “a more correct description of the world,” except in some extreme cases. For example, a few years ago I reviewed a pop-science book that was written from a racist paradigm. Is that paradigm “more correct” or “less correct” than a non-racist paradigm? It depends on what questions are being asked, and what non-racist paradigm is being used as a comparison.

2. Also the whole academia-tenure framework seems misplaced, in that the most important scientific paradigms are rooted in diverse environments, not just academia. For example, the solid-state physics paradigm led to transistors (developed at Bell Labs, not academia) and then of course is dominant in industry. Even goofy theories such as literary postmodernism (is this “Better” or “Worse”? How could we ever tell?) exist as much in the news media as in academe; indeed, if we didn’t keep hearing about deconstructionism in the news media, we’d never have known of its existence. And of course recent trendy paradigms in social psychology (embodied cognition, etc.) are associated with self-help gurus, Gladwell books, etc., as much as with academia. I think that a big part of the success of that sort of work in academia is because of its success in the world of business consulting. The wheelings and dealings of tenure committees are, I suspect, the least of it.

Beyond all this—or perhaps explaining my above comments—is my irritation at people who use university professors as soft targets. Silly tenured professors ha ha. Bad science is a real problem but I think it’s ludicrous to attribute that to the tenure system. Suppose there was no such thing as academic tenure, then I have a feeling that social and biomedical science research would be even more fad-driven.

I sent the above comments to the authors, and Akerlof replied:

I think that your point of view and ours are surprisingly on the same track; in fact the paper answers Thomas Kuhn’s question: what makes science so successful. The point is rather subtle and is in the back pages: especially regarding the differences between promotions of scientists and promotion of surgeons who did radical mastectomies.

21 Comments

  1. Andrew – having studied it, would you not agree that physics has a pretty robust concept of “better” and “worse” paradigms – eg Einstein vs Newtonian dynamics, classical v quantum explanations of magnetism etc. They make better – in the sense of quantitatively more reliable – predictions.

    I found the authors’ Beetles model strikingly relevant to the current crisis in high-energy physics, which has been in a state of stagnation since the 1980s (all the hoopla about the Higgs stems from confirmation of theory developed in the 1960s). In particular, attempts to create a quantum theory of gravity are mired in controversy about whether observational support – and thus tenure based on successful predictions – is feasible, or even necessary. This would lead to the “low Youden Index” regime the authors cite as the hallmark of a field that is getting nowhere. And that does seem to be the case in HEP and theoretical cosmology.

  2. Dmitri says:

    On the point about fitness, the intent of the paper seems clear enough, even if it could have been worded better: instead of one species being fitter than the other, in the sense of consistently winning the competition and dominating the ultimate population, the surviving species was sometimes random and dependent on very subtle environmental conditions (e.g. whether one of the founding beetles died soon after the experiment started). I guess it might have been better to say “fitness effects were small and swamped by environmental effects.”

    • Andrew says:

      Dmitri:

      You write, “fitness effects were small and swamped by environmental effects,” but I don’t think this is right: fitness depends on the environment, and fitness effects do not generally exist independent of environmental effects.

      • Terry says:

        “With the beetles, the more biologically fit species does not always prevail.”

        How about this as an example of what the author meant?

        Say there are two species of beetles that are alike in all ways except that one is brown and one is red. Both species live in identical brown-colored trees separated by 10 miles. The red beetles are eaten by birds twice as often as the brown beetles, so the probability that the red beetle will go extinct in the next 10 years is .7, while the probability for the brown beetles is .3. A low-probability snowstorm hits the brown-beetle forest but not the red-beetle forest, and the brown beetles go extinct while the red beetles survive.

        In an objectives sense, the brown beetles are better adapted, but they did not prevail — because of the luck of the draw.

        • Ethan Bolker says:

          “The luck of the draw” affecting environment and hence fitness is part of evolution. The fact that you posit a freak snowstorm doesn’t make the red beetles “objectively fitter”. Absent an asteroid impact many years ago we might all be descended from dinosaurs today rather than from those original mammals.

          • The issue is, if “survival of the fittest” simply means “those that survive were the fit ones” you have tautology. there is then, no science involved, merely definition of words: “fit” means by definition “whatever survived” and so “survival of the fittest” is more or less the same as “survival of those who survived” and there’s no content.

            So, we can’t just move the goalposts to say “whatever survived was obviously the fittest one”

            I think Terry’s example is good, because it assumes that in the absence of a “freak” snowstorm we have data evidence that brown beetles are doing better. The existence of such “freak” snowstorms or comet impacts on the earth or whatever can of course affect fitness, but when something is rare and relatively unpredictable, it won’t general go into whatever our description is of “fitness” because we need to define fitness in a way other than “whatever survives” and the best available way is “what tends to work under usual prevailing conditions” so it’s kind of relative to the relatively *slow* environmental trends, not the sudden jumps.

          • Dan Simpson says:

            In small populations, stochasticity can always lead to strange results. Maybe a useful analogy is that “fitness” is like climate (It doesn’t snow in April), while the actual survival is like the weather (sometimes it snows in April).

            • jrc says:

              “In small populations, stochasticity can always lead to strange results.” Maybe a useful analogy is the Garden of Forking Species?

              Sometimes I think evolutionary explanations for traits in living things are very much like fitting a pattern of statistical significance stars to an under-developed theory of utility-maiximizing behavior – there are just enough parameters that you can explain anything.

              But it turns out that sometimes very “fit” species get wiped out by asteroids leaving a less “fit” group alive and propagating itself; and sometimes people do stupid, self-destructive things.

              • Andrew says:

                Jrc:

                Sure, but the example in the above-linked paper was not unexpected asteroid impact; the example was competition with other species in a given environment. Fitness is all about adaptation to the environment, so it seems like a mistake for someone to say that if a species prospers by eating the eggs of another species’s eggs, that this is not true fitness.

          • Martha (Smith) says:

            +1 to Ethan.

            @Daniel Lakeland (and perhaps some others): I consider the phrase “survival of the fittest” to be about as damaging to scientific understanding as saying “Randomized controlled trials are the Gold Standard for research”. I think of biological “fitness” as a contingent property, so think of evolution as “survival of the fit enough to have survived in the circumstances in which they have so far existed.”

    • Jens Åström says:

      Perhaps they should instead have used the metaphor of “priority effects” in an ecological community, where the first species to colonize an area might not be the “optimal” species for that environment (in this metaphor, not the best belief/idea). It might be due to chance events or it might even be a dispersal specialist (think of a contagious meme). This species (belief/idea) might then hinder the colonization of another species that would be a better competitor if it got to compete at equal terms (e.g. arrive at the same time, or evaluated on equal terms). It sounds like this is might be what they are getting at, although I haven’t read the whole paper.

  3. Totally Ridiculous says:

    ” For example, a few years ago I reviewed a pop-science book that was written from a [flat earth] paradigm. Is that paradigm “more correct” or “less correct” than a [round earth] paradigm? It depends on what questions are being asked, and what [round earth] paradigm is being used as a comparison.”

    Sounds like a ridiculous exercise to me…

  4. Roy says:

    “The Natural Selection of Bad Science”

    http://rsos.royalsocietypublishing.org/content/3/9/160384

    which also gives a tangential link to Stan since McElreath will be speaking at the Stan meet up in the summer.

  5. abdulh says:

    “when tenured scientists show favoritism toward candidates for tenure with similar beliefs, science may not converge to the true paradigm.”

    I wonder what “science” they have in mind when they write that – phrenology perhaps (or something similar..)

  6. NJW says:

    The philosophy of science which you describe, or imply, under point 1. seems surprisingly extreme. Or perhaps I’m misconstruing. Do you have a more lengthy description of these beliefs? Or is there another author’s work which you think captures what you’re the underlying arguments?

  7. Jonathan (another one) says:

    Weighing in late here, but “And if a species’ eggs get eaten before they’re hatched, that reduces the species’s fitness” is not necessarily true. For species in which the number of eggs is not a limiting resource (say that the ecological niche for the animal could accommodate 1,000,000 individuals without exhausting resources but there are 1,000,000,000 aggregate eggs) then fitness might actually be reduced if there aren’t ways to reduce the number of eggs which survive to fill the niche.

    This distinction arises in water permitting for nuclear power plants. Lots of fish eggs are sucked into cooling water intakes, but there are so many eggs that the ensuing number of adult fish are unchanged. Had those eggs survived, the ensuing competition would have been *different* but not necessarily *better* in terms of individual survival or net group fitness.

Leave a Reply