Heuristics for identifying ecological fallacies?

Greg Laughlin writes:

My company just wrote a blog post about the ecological fallacy. There’s a discussion about it on the Hacker News message board.

Someone asks, “How do you know [if a group-level finding shouldn’t be used to describe individual level behavior]?” The best answer I had was “you can never tell without the individual-level data, you should always be suspicious of group-level findings applied to individuals.”

Am I missing anything? Are there any situations in which you can look at group-level qualities being ascribed to individuals and not have to fear the ecological fallacy?

My reply: I think that’s right. To put it another way, consider the larger model with separate coefficients for individual-level and group-level effects. If you want, you can make an assumption that they’re equal, but that’s an assumption that needs to be justified on substantive grounds. We discuss these issues a bit in this paper from 2001. (I just reread that paper. It’s pretty good!)

10 thoughts on “Heuristics for identifying ecological fallacies?

  1. Firebaugh (1978) deals with this. (I’m surprised to find it’s not cited in the paper our host links to, which I haven’t read yet.). Here’s the abstract of the Firebaugh paper:

    Under certain conditions aggregate-level data provide unbiased estimates of individual-level relationships. Here I present these conditions in the form of a single theoretical decision rule: bias is absent when, and only when, the group mean of the independent variable (X) has no effect on Y, with X controlled. This paper introduces this rule, demonstrates it for the general n-variable case, compares it with prior discussions of cross-level inference, and illustrates it with the 1930 census data used by Robinson (1950). The final section discusses the implications of this rule for the converse type of cross-level inference: the use of individual-level data to estimate aggregate-level relationships.

    • Thanks for the reference! This seems like a much better answer to the question “Are there any situations in which you can look at group-level qualities being ascribed to individuals and not have to fear the ecological fallacy?”.

  2. Paul, Bill:

    That stuff was fine for its time (it was around 1994 and 1995 that Gary and I did the actual work of creating and fitting the model) but it doesn’t directly address the question raised by the correspondent. But I haven’t thought much about the problem for several years so maybe there’s some new work in the area that I’m not aware of.

  3. In terms of causality individual effects always run up against the fundamental problem of causal inference. But often you can place bounds on them.

    • Fernando:

      Yes, Gary and I talked about some of those bounds in our book. The bounds are pretty good in some examples and not in others, depending on the data.

  4. To avoid falling prey to ecological fallacies, it’s extremely helpful for statistical analysts to familiarize themselves with a wide range of true stereotypes about race, gender, ethnicity, religion, and so forth.

    For example, in the current debate about the effectiveness of gun control laws, differences in laws between states in deterring homicides are swamped by the racial composition of the states: Blacks, especially urban blacks, have high homicide rates, Hispanics moderate ones, whites moderately low ones, and Asians very low ones. Then, within the white population, there are differences based on ethnicity: e.g., Scots-Irish tend to have higher homicide rates than Swedes or Italians. Likewise, within Hispanics, homicide rates tend to be highest among Caribbeans and lower among Mexicans. Among Mexican-Americans, homicide rates tend to low among new immigrants and higher among American-born ones, lower in Texas than in California, and so forth and so on.

    If you are aware of these major differences, then you might be able to tease out some insights into the effectiveness of various kinds of gun control laws.

    The problem, of course, is that you must bear in mind various HateStats about ethnic differences in homicide rates, which is much frowned upon these days.

    • I can see why ignorant laymen should be aware of the statistical fallacy. But how does it help trained professional statisticians (who are already well conversant with ecological fallacies) to familiarize themselves with your cherry picked examples? What’s the value addition? Or was this a non sequitur just to let you inject your favorite points?

  5. Pingback: Somewhere else, part 32 | Freakonometrics

Comments are closed.