Martin Lindquist and Michael Sobel published a fun little article in Neuroimage on models and assumptions for causal inference with intermediate outcomes. As their subtitle indicates (“A response to the comments on our comment”), this is a topic of some controversy. Lindquist and Sobel write:
Our original comment (Lindquist and Sobel, 2011) made explicit the types of assumptions neuroimaging researchers are making when directed graphical models (DGMs), which include certain types of structural equation models (SEMs), are used to estimate causal effects. When these assumptions, which many researchers are not aware of, are not met, parameters of these models should not be interpreted as effects. . . . [Judea] Pearl does not disagree with anything we stated. However, he takes exception to our use of potential outcomes notation, which is the standard notation used in the statistical literature on causal inference, and his comment is devoted to promoting his alternative conventions. [Clark] Glymour’s comment is based on three claims that he inappropriately attributes to us. Glymour is also more optimistic than us about the potential of using directed graphical models (DGMs) to discover causal relations in neuroimaging research . . .
Lindquist and Sobel’s arguments make sense to me, except on one point. They consider a causal setting z -> x -> y, where z is the treatment variable, x is the intermediate outcome, and y is the ultimate outcome, and much of their discussion centers on estimating the causal effect of x on y. I have two difficulties with their perspective:
1. If x is an observed variable that is not directly manipulated, I don’t know if it makes sense to talk about the effect of x on y, unconditional on the intervention that was used to change x. In their example, I’d talk about “the effect of x on y, if x is changed through z.” Different z’s can induce different effects of x on y.
2. Lindquist and Sobel talk about the effect of z on x. If z=0 or 1, they write x(z), so that the causal effect of z on x is x(1) – x(0) (or, more generally, x(1) compared to x(0), but we lose nothing by considering simple differences here). So far, so good.
But I get stuck at the next step, where they define the effect of x on y. If x can equal 0 or 1, they write y(z,x), so that the causal effect of x on y, conditional on z, is y(z,1) – y(z,0). At least, I think that’s what they’re saying.
The trouble is, I don’t see how the two parts of this model fit together. For any given item in the experiment, I think they’re following the rule that x(z) has a particular (although maybe unknown) value. But then I don’t see what it means to look at y(z,1) – y(z,0). For any particular value of z, it seems to me that only one of these two terms is possible. (For example, if x(z)=1, then y(z,1) is defined but y(z,0) seems meaningless.)
I’m not saying that this framework is wrong, just that I don’t understand it.
That said, Lindquist and Sobel’s criticisms of Pearl and Glymour seem sound to me.
P.S. I wrote this last month and put it in the queue. Since then I’ve noticed that Pearl has responded to Lindquist and Sobel; see here. I don’t find Pearl’s response to be so convincing—I agree with Lindquist and Sobel’s statement that the graphical or structural equation modeling expression looks simple and appealing but the underlying assumptions in those expressions are not so clear. But you can judge for yourself; as I wrote in my discussion of the book by Morgan and Winship, it’s good to have muultiple expressions for a model, as different users are looking for different things.
To be specific, Pearl contrasts three expressions of a single model, the causal chain Z—>X—>Y. Here’s Pearl:
Pearl characterizes the third expression is a more meaningful and clear display.
In contrast, Lindquist and Sobel argue that the above graphical expression appears clear only because it sweeps the model’s assumptions under the rug. Lindquist and Sobel write:
None of this seems clear and simple to me! Speaking of clear and simple, I’m reminded of a scene, several decades ago, when a bunch of us on the county math team won some competition, and the prize was that we each got to choose one of several math books. One of the books was called Elementary Linear Algebra, and I remember making a disdainful remark to my friend that I didn’t want something elementary. My friend replied, “Linear algebra is not elementary.” Good point.
Which brings back another memory: our coach for the Mathematical Olympiad program was an unbelievably grumpy old man. At one point he interrupted one of his lectures to rant about how all the calculus books now are wasting their space with applications. At some point, he said, they’re gonna come up with a book called Applied Calculus with Applications. That all seemed natural to me at the time but in retrospect I’m amazed by how brainwashed we all were. There was one kid there who I recall was interested in engineering problems rather than number theory etc., but that was an unusual preference. (I just looked him up and, amazingly, he grew up to be an engineering researcher!) The other thing I remember about the grumpy coach dude, besides his personality (which, in retrospect, was perhaps necessary to keep a bunch of 15-year-old boys in line; even nerds can make trouble), was that he thought it was cheating to use calculus or analytic geometry. His favorite sorts of problems used elaborate arguments from classical geometry and he always felt we should be able to solve these without resorting to technical means.
As I’ve remarked more than once in this space, I feel lucky in retrospect to have been pretty unprepared for the Olympiad program, with the result that I didn’t do very well there, gradually lost interest in this sort of competitive event, and decided I didn’t want to be a pure mathematician. I think it must’ve been really hard on the kids who were top performers but didn’t happen to be Noam. It was easier for those of us in the bottom half of the group.