Philipp Doebler writes:
I was quite happy that recently you shared some thoughts of yours and others on meta-analysis. I especially liked the slides by Chris Schmid that you linked from your blog. A large portion of my work deals with meta-analysis and I am also fond of using Bayesian methods (actually two of the projects I am working on are very Bayesian), though I can not say I have opinions with respect to the underlying philosophy. I would say though, that I do share your view that there are good reasons to use informative priors.
The reason I am writing to you is that this leads to the following dilemma, which is puzzling me. Say a number of scientists conduct similar studies over the years and all of them did this in a Bayesian fashion. If each of the groups used informative priors based on the research of existing groups the priors could become more and more informative over the years, since more and more is known over the subject. At least in smallish studies these priors will have an impact on the conclusion, and the impact will increase with time. The worst case might be, that a) there is a form of regression to the mean of outcomes the individual studies and b) the variance of the effect sizes are smaller due to the highly informative priors. In some sense each of the primary studies is boosting its sample size by using informative priors.
While this all makes perfect sense on the level of the primary studies, on the meta-analytic level the studies look as if they had achieved more precise estimates then they actually have and also there might be less heterogeneity observed than there really is. One could even say, that the newer studies are forstalling the meta-analysis.
I am not sure if the above leads to the advice to use non-informative priors on the primary study level, so that primary study level outcomes are not influenced by other studies, or if the above only underlines the need to report outcomes in primary studies for more than one prior.
I would welcome to know your views on this.
Yes, I agree that in a meta-analysis you want to have the original data. It is difficult to combine posterior distributions as there is the risk of counting some information multiple times. Sometimes I say, As a Bayesian I want scientists to report their data non-Bayesianly.