Consider two broad classes of inferential questions:
1. Forward causal inference. What might happen if we do X? What are the effects of smoking on health, the effects of schooling on knowledge, the effect of campaigns on election outcomes, and so forth?
2. Reverse causal inference. What causes Y? Why do more attractive people earn more money? Why do many poor people vote for Republicans and rich people vote for Democrats? Why did the economy collapse?
When statisticians and econometricians write about causal inference, they focus on forward causal questions. Rubin always told us: Never ask Why? Only ask What if? And, from the econ perspective, causation is typically framed in terms of manipulations: if x had changed by 1, how much would y be expected to change, holding all else constant?
But reverse causal questions are important too. They’re a natural way to think (consider the importance of the word “Why”) and are arguably more important than forward questions. In many ways, it is the reverse causal questions that lead to the experiments and observational studies that we use to answer the forward questions.
My question here is: How can we incorporate reverse causal questions into a statistical framework that is centered around forward causal inference. (Even methods such as path analysis or structural modeling, which some feel can be used to determine the direction of causality from data, are still ultimately answering forward casual questions of the sort, What happens to y when we change x?)
My resolution is as follows: Forward causal inference is about estimation; reverse causal inference is about model checking and hypothesis generation.
I’ll illustrate with an example and then introduce some (simple) notation. The forward question is, What is the effect of $ on elections? This is a very general question and, to be answered, needs to be made more precise, for example as follows: Supposing a challenger in a given congressional election race is given an anonymous $10 donation, how much will this change his or her expected vote share? It’s not so easy to get an accurate answer to this question, but the causal quantity is clearly defined.
Now a reverse question: Why do incumbents running for reelection to Congress get so much more funding than challengers? Many possible answers have been suggested, including the idea that people like to support a winner, that incumbents have higher name recognition, that certain people give money in exchange for political favors, and that incumbents are generally of higher political “quality” than challengers and get more support of all types. Various studies could be performed to evaluate these different hypotheses, all of which could be true to different extent (and in some interacting ways).
Now the notation. I believe that forward causal inferences can be handled in a potential-outcome or graphical-modeling framework involving a treatment variable T, an outcome y, and pre-treatment variables, x, so that the causal effect is defined (in the simple case of binary treatment) as y(T=1,x) – y(T=0,x). The actual estimation will likely involve some modeling (for example, some curve of the effect of money on votes that is linear at the low end, so that a $20 contribution has twice the expected effect as $10), but there is little problem in defining the treatment effect. In complex settings it might be useful to employ graphical models; it is not my purpose to discuss these here.
Reverse causal inference is different; it involves asking questions and searching for new variables that might not yet even be in our model. I would like to frame reverse causal questions as model checking. It goes like this: what we see is some pattern in the world that needs an explanation. What does it mean to “need an explanation”? It means that existing explanations—the existing model of the phenomenon—does not do the job. This model might be implicit. For example, if we ask, Why do incumbents get more contributions than challengers, we’re comparing to an implicit model that all candidates get the same. If we gather some numbers on dollar funding, compare incumbents to challengers, and find the difference is large and statistically significant, we’re comparing to the implicit model that there is variation but not related to incumbency status. If we get some measure for “candidate quality” (for example, previous elective office and other qualifications) and still see a large and statistically significant difference between the funds given to incumbents and challengers, then it seems we need more explanation. And so forth. Just as I view graphical exploratory data analysis as a form of checking models (which maybe implicit), I similarly hold that reverse causal questions arise in response to anomalies—aspects of our data that are not readily explained—and that the search for causal explanations is, in statistical terms, an attempt to build a new model that has the ability to reproduce the patterns we see in the world.
What does this mean for statistical practice?
A key theme in this discussion is the distinction between causal statements and causal questions. When Rubin dismissed reverse causal reasoning as “cocktail party chatter,” I think it was because you can’t clearly formulate a reverse causal statement. That is, a reverse causal question does not in general have a well-defined answer, even in a setting where all possible data are made available. But I think Rubin made a mistake in his dismissal. The key is that reverse questions are valuable in that they focus on an anomaly—an aspect of the data unlikely to be reproducible by the current (possibly implicit) model—and point toward possible directions of model improvement.
It has been (correctly) said that one of the main virtues of forward causal thinking is that it motivates us to be explicit about interventions and outcomes. Similarly, one of the main virtues of reverse causal thinking is that it motivates us to be explicit about our model. If we ask: Why do ethnic minorities (compared to whites) in NYC have a higher rate of rodents in their apartments?, we have an implicit model that the infestation rates should be the same, after controlling for x1, x2, x3, etc.
Another theme is the separation of the three steps of data analysis: (1) model construction, (2) inference, and (3) model checking. Inference is the glamour boy but you can’t get far without a model (for those model-haters in the audience, you can replace the word “model” with the phrase “choice of what information to use in your analysis, and choice the form in which you will use that information”), and models are much more effective if you allow yourself to check them and make improvements in response to problems. Rubin dismissed reverse causal reasoning because it can’t be fit into the “inference” step; others have struggled with little success (in my opinion) to construct direct answers to reverse causal questions. It all becomes cleaner when we allow statistical methods to fit into the model-construction and model-checking phases of data analysis. This is similar to how we folded EDA into Bayes by framing graphics as an informal way of model checking.
By formalizing reverse casual reasoning within the process of data analysis, I hope to make a step toward connecting our statistical reasoning to the ways that we naturally think and talk about causality. As LBJ might say: Better to have reverse causal inference inside the statistical tent pissing out than outside pissing in.
P.S. Based on the comments, I think many people missed my central point! Let me say it again:
I think reverse casual questions are important. But I don’t think there are reverse causal answers. That is, I think it’s helpful to ask, Why are incumbents better-funded than challengers? But I don’t think there’s any useful answer to that question. The question reveals a gap between reality and our (implicit) models, but I think the answer to the question must come in the form of a forward causal statement.
It was probably a mistake for me to use the term “reverse causal inference.” In future, I’ll stick with the phrase, “reverse causal question.” And I’ve changed the title of the post accordingly.