Estimating and summarizing inference for hierarchical variance parameters when the number of groups is small

Posted on by

Chris Che-Castaldo writes:

I am trying to compute variance components for a hierarchical model where the group level has two binary predictors and their interaction. When I model each of these three predictors as N(0, tau) the model will not converge, perhaps because the number of coefficients in each batch is so small (2 for the main effects and 4 for the interaction). Although I could simply leave all these as predictors as unmodeled fixed effects, the last sentence of section 21.2 on page 462 of Gelman and Hill (2007) suggests this would not be a wise course of action:

For example, it is not clear how to define the (finite) standard deviation of variables that are included in interactions.

I am curious – is there still no clear cut way to directly compute the finite standard deviation for binary unmodeled variables that are also part of an interaction as well as the interaction itself?

My reply: I’d recommend including these in your model (it’s probably easiest to do so using Stan), but you’ll need informative priors on these hierarchical variance parameters to get everything to work. In practice this should not be so hard to do. I regret that I have not done much of this in my published work so it’s hard to point you to an example.

Che-Castaldo continues:

My predicament is not an uncommon one in my field. Ecologists do a lot of experiments where the unmodeled “fixed” factors are few in number, each factor has only a few groups (they are often binary), and the interactions are important (for example the ubiquitous 2×2 ANOVA). Estimating effects is not difficult but using Bayesian ANOVA to compute variance components for these types of models seems a lot harder.

As we’ve been discussing a bit on the blog recently, I think it should be possible to use informative priors with scales set based on actual prior information. I do think we will understand this sort of thing better once we have some good examples (and, indeed, my own lack of experience in this area is a big reason why we don’t have such examples in our books).