Ban Chuan Cheah writes:
In a previous post, http://andrewgelman.com/2013/07/30/the-roy-causal-model/ you pointed to a paper on Bayesian methods by Heckman. At around the same time I came across another one of his papers, “The Effects of Cognitive and Noncognitive Abilities on Labor Market Outcomes and Social Behavior (2006)” (http://www.nber.org/papers/w12006 or published version http://www.jstor.org/stable/10.1086/504455). In this paper they implement their model as follows:
We use Bayesian Markov chain Monte Carlo methods to compute the sample likelihood. Our use of Bayesian methods is only a computational convenience. Our identification analysis is strictly classical. Under our assumptions, the priors we use are asymptotically irrelevant.
Some of the authors have also done something similar earlier in:
Hansen, Karsten T. & Heckman, James J. & Mullen, K.J.Kathleen J., 2004. “The effect of schooling and ability on achievement test scores,” Journal of Econometrics, Elsevier, vol. 121(1-2), pages 39-98.
This email was 1) to point out Heckman’s forays into Bayesian methods, and 2) to ask whether it makes sense to model educational attainment as a multilevel model.
On 2), the usual way to model earnings as an outcome is to include educational attainment as an explanatory variable. However, since educational attainment is endogenous/intermediate outcome, Heckman and his co-authors use a structural equation approach which models educational attainment as an index in a separate equation and then earnings separately along with other explanatory variables. The setup looked almost like a multilevel model setup (but not quite) – enough for me to venture to ask whether it is possible to think of doing something like this as a multilevel model as an alternative. Doing a search did not turn up anything that indicated that this has been done before.
Moreover, Patrick Curran “Have Multilevel Models Been Structural Equation Models All Along?” (ftp://184.108.40.206/Transfer/ES_Pubs/ESVal/multilevel_analysis/curran_02_Multilevel_vs_StrucEquatModel.pdf) shows that it is possible to think of MLM as Structural Equations. Might it also not be possible to think of this particular structural model as a multilevel model?
Regarding the quote about “asymptotically invariant”: such results can provide useful insights but in the problems I work on, I am rarely close to the asymptotic limit. Priors can be important, especially when we have real prior information!
Getting closer to the subject at hand, I do think that many factors can predict educational attainment, and that effects can vary, hence it makes sense to consider multilevel models.
Finally, I think you’re probably correct that structural models can be thought of as multilevel models, in the sense that I do think that many factors can predict educational attainment, and that effects can vary, hence it makes sense to consider multilevel models.
P.S. Again, let me repeat my happiness that Heckman is working on Bayesian methods. I hope that the era of blindly anti-Bayesian attitudes (for example, this from David Hendry and this from John DiNardo) is coming to a close, as researchers recognize the relevance of prior information and hierarchical structure in sparse-data settings.