Testing complicated contrasts in Anova-type models

Alan Bergland writes:

I am a graduate student studying evolutionary biology at Brown University. I am writing you with what I think is a simple question, but I cannot seem to find an answer I feel comfortable with.

I am trying to test a planned contrast using posterior distributions from a mixed model (the mixed model is calculated in lme4, and the simulations in arm). The model is fairly complicated, but at the end of the day, there are two fixed effect treatments with two levels each that I am interested in. Lets call these fixed effects “treatment A” (with levels A and a) and “treatment B” (with levels B and b). I am interested in the interaction between treatment A and treatment B, but have a specific hypothesis about the form of that interaction I would like to test. Specifically, I would like to test if ab is less than Ab & aB=AB.

As you and Jennifer Hill suggest in your Multilevel/Hierarchical models book (p. 20), I could test if ab

My reply:

1. Don’t use the arm’s sim() function for lmer() objects. The current version is wrong; we’re fixing it now, and the replacement should be available in about a month.

2. I don’t recommend testing if aB=AB. At least in the sorts of problems I work on, no two comparisons are exactly equal. I think it makes more sense to estimate the relevant comparison, get the confidence interval, and make a graph. You could also do things like calculate the posterior probability (based on simulations) that ab < AB & |aB - AB|

2 thoughts on “Testing complicated contrasts in Anova-type models

  1. Thanks for the warning on arm::sim.

    What would you recommend as the best way (or just a good way) to generate simulated data from linear models fit with lmer? I'd like to be able to determine HPD intervals and, yes, p-values.

    As I understand it:

    1. mcmcsamp only works for models without correlated random effects, so varying-intercept and -slope models won't work (so mcmcsamp throws an error).

    2. One can do something along the lines of http://glmm.wikidot.com/basic-glmm-simulation. This seems unnecessary for linear models. I'm not so clear I am on this method anyway, but I haven't studied it closely.

    3. arm::sim does not work.

    I've worked a good deal of the way through Gelman & Hill (2009), and so BUGS is also an option. But in many cases the models I'm fitting are not that complex for the amount of data, so lmer seems just fine — and helpful for quickly fitting and checking models.

Comments are closed.