Skip to content
Archive of posts filed under the Multilevel Modeling category.

Birthday analysis—Friday the 13th update, and some model checking

Carl Bialik and Andrew Flowers at (Nate Silver’s site) ran a story following up on our birthdays example—that time series decomposition of births by day, which is on the cover of the third edition of Bayesian Data Analysis using data from 1968-1988, and which then Aki redid using a new dataset from 2000-2014. Friday […]

Point summary of posterior simulations?

Luke Miratrix writes: ​In the applied stats class ​I’m teaching ​on​ hierarchical models I’m giving the students (a mix of graduate students, many from the education school, and undergrads) a taste of Stan. I have to give them some “standard” way to turn Stan output into a point estimate (though of course I’ll also explain […]


Jona Sassenhagen writes: Here is a paper ***, in case you, errrrr, have run out of other things to blog about … I took a look and replied: Wow—what a horrible paper. Really ignorant. Probably best for me to just ignore it!

Controlling for variation in the weather in a regression analysis: Joe and Uri should learn about multilevel models and then they could give even better advice

Joe Simmons and Uri Simonsohn have an interesting post here. Unfortunately their blog doesn’t have a comment section so I’m commenting here. They write this at the end of their post: Another is to use daily dummies. This option can easily be worse. It can lower statistical power by throwing away data. First, one can […]

Multilevel regression

Mike Hughes writes: I have been looking a your blog entries from about 8 years ago in which you comment on the number of groups that is appropriate in multilevel regression. I have a research problem in which I have 6 groups and would like to use multilevel regression. Here is the situation. I have […]

My quick answer is that I would analyze all 10 outcomes using a multilevel model.

David Shor writes: Suppose you’re conducting an experiment on the effectiveness of a pain medication, but in the post survey, measure a large number of indicators of well being (Sleep quality, self reported pain, ability to get tasks done, anxiety levels, etc). After the experiment, the results are insignificant (or the posterior effect size isn’t […]

Actually, I’d just do full Bayes

Dave Clark writes: I was hoping for your opinion on a topic related to hierarchical models. I am an actuary and have generally worked with the concept of hierarchical models in the context of credibility theory. The text by Bühlmann and Gisler (A Course in Credibility Theory; Springer) sets up the mixed models under the […]

Mister P: Challenges in Generalizing from Sample to Population (my talk at the Ross-Royall Symposium at Johns Hopkins this Friday)

Mister P: Challenges in Generalizing from Sample to Population Andrew Gelman, Department of Statistics and Department of Political Science, Columbia University With internet surveys, nonrepresentativeness and nonresponse are bigger concerns than ever. The natural approach is to adjust for more information, demographic and otherwise, to align the sample with the population. We demonstrate the effectiveness […]

“Don’t get me started on ‘cut'”

Brendan Rocks writes: I have a request for a blog post. I’ve been following the debates about ‘cut’ on the Stan lists over the last few years. Lots of very clever people agree that it’s bad news, which is enough to put me off. However, I’ve never fully groked the reasoning. [I think that should […]

Hierarchical models for phylogeny: Here’s what everyone’s talking about

The other day on the Stan users list, we had a long discussion on hierarchical models in phylogeny that I thought might be of general interest, so I’m reconstructing it here. It started with this question from Ben Lambert: I am hoping that you can help me settle a debate. My collaborators and I have […]

Pooling is relative to the model

Ryan Raaum writes: I’m hoping you’ll be willing to shed some light on a question I have regarding “pooling” in modeling. In your book with Jennifer Hill, you lay out two ends of a spectrum for dealing with structured data: (1) “Complete pooling” – ignoring the groups and pooling everything together for an overall average […]

Forking paths vs. six quick regression tips

Bill Harris writes: I know you’re on a blog delay, but I’d like to vote to raise the odds that my question in a comment to discussed, in case it’s not in your queue. It’s likely just my simple misunderstanding, but I’ve sensed two bits of contradictory advice in your writing: fit one complete model all at […]

Where the fat people at?

Pearly Dhingra points me to this article, “The Geographic Distribution of Obesity in the US and the Potential Regional Differences in Misreporting of Obesity,” by Anh Le, Suzanne Judd, David Allison, Reena Oza-Frank, Olivia Affuso, Monika Safford, Virginia Howard, and George Howard, who write: Data from BRFSS [the behavioral risk factor surveillance system] suggest that […]

Stunning breakthrough: Using Stan to map cancer screening!

Paul Alper points me to this article, Breast Cancer Screening, Incidence, and Mortality Across US Counties, by Charles Harding, Francesco Pompei, Dmitriy Burmistrov, Gilbert Welch, Rediet Abebe, and Richard Wilson. Their substantive conclusion is there’s too much screening going on, but here I want to focus on their statistical methods: Spline methods were used to […]

One thing I like about hierarchical modeling is that is not just about criticism. It’s a way to improve inferences, not just a way to adjust p-values.

In an email exchange regarding the difficulty many researchers have in engaging with statistical criticism (see here for a recent example), a colleague of mine opined: Nowadays, promotion requires more publications, and in an academic environment, researchers are asked to do more than they can. So many researchers just work like workers in a product […]

rstanarm and more!

Ben Goodrich writes: The rstanarm R package, which has been mentioned several times on stan-users, is now available in binary form on CRAN mirrors (unless you are using an old version of R and / or an old version of OSX). It is an R package that comes with a few precompiled Stan models — […]

Plausibility vs. probability, prior distributions, and the garden of forking paths

I’ll start off this blog on the first work day of the new year with an important post connecting some ideas we’ve been lately talking a lot about. Someone rolls a die four times, and he tells you he got the numbers 1, 4, 3, 6. Is this a plausible outcome? Sure. Is is probable? […]

“Once I was told to try every possible specification of a dependent variable (count, proportion, binary indicator, you name it) in a regression until I find a significant relationship. That is it, no justification for choosing one specification over another besides finding significance. . . . In another occasion I was asked to re-write a theory section of a paper to reflect an incidental finding from our analysis, so that it shows up as if we were asking a question about the incidental finding and had come up with the supported hypothesis a priori. . . .”

Ethan Bolker points me to this discussion. My reply: As discussed in my paper with Hill and Yajima, I think the best approach is to analyze all comparisons rather than picking just some. If there is prior understanding that some comparisons are more important than others, that understanding can be included as predictors in the […]

Hierarchical modeling when you have only 2 groups: I still think it’s a good idea, you just need an informative prior on the group-level variation

Dan Chamberlain writes: I am working on a Bayesian analysis of some data from a randomized controlled trial comparing two different drugs for treating seizures in children. I have been using your book as a resource and I have a question about hierarchical modeling. If you have the time, I would greatly appreciate any advice […]

Judea Pearl and I briefly discuss extrapolation, causal inference, and hierarchical modeling

OK, I guess it looks like the Buzzfeed-style headlines are officially over. Anyway, Judea Pearl writes: I missed the discussion you had here about Econometrics: Instrument locally, extrapolate globally, which also touched on my work with Elias Bareinboim. So, please allow me to start a new discussion about extrapolation and external validity. First, two recent […]