Mark Johnson writes:
Suppose we want to build a hierarchical model where the lowest level are multinomials and the next level up are Dirichlets (the conjugate prior to multinomials). What distributions should we use for the level above that? I’m not a statistician, but aren’t Dirichlet distributions exponential models? If so, Dirichlets should have conjugate priors, which might be useful for the next level up in hierarchical models. I’ve never heard anyone talk about conjugate priors for Dirichlets, but perhaps I’m not listening to the right people. Do you have any other suggestions for priors for Dirichlets?
My reply: I’m not sure, but I agree that there should be something reasonable here. I’ve personally never had much success with Dirichlets. When modeling positive parameters that are constrained to sum to 1, I prefer to use a redundantly-parameterized normal distribution. For example, if theta_1 + theta_2 + theta_3 + theta_4 = 1, with all thetas constrained to be positive, I’ll use the model,
theta_j = exp(phi_j)/(exp(phi_1)+exp(phi_2)+exp(phi_3)+exp(phi_4), for j=1,2,3,4.
If you give each of the phi_j’s a normal distribution, this is a more flexible model than the Dirichlet: it has 8 parameters (four means and four variances). Well, actually 7, because the means are only identified up to an arbitrary additive constant.
Frederic Bois and I used this distribution for a problem in toxicology, modeling blood flows within different compartments of the body–these were constrained to sum to total blood flow.
In this article, I used a fun stochastic approximation trick to compute reasonable values for the mean and variance parameters for the phi’s.
P.S. Dana Kelly points to this article on the topic by Aitchison and Shen from 1980.