Sanjay Kaul writes,
I would like to hear about your perspective on specification of prior for a Bayesian analysis of a clinical trial. The trial was designed to have 80% power to detect a 25% relative difference in outcomes (presumably reflecting the investigator’s estimate of a “clinically important” difference – MCID).
The results suggested an 18% treatment benefit for the new agent (95% CI 2 to 32%), P = 0.028.
One can estimate that the results MIGHT be compatible with a clinically important treatment effect (25% difference lies within the 95% CI). However, one can explicitly calculate the probability of a 25% effect based on the Bayesian analysis.
1. What prior is appropriate in this setting? Non-informative; skeptical; enthusiastic
If skeptical, do we center the distribution on null difference with 2.5% probability of the treatment effect exceeding MCID (risk ratio <0.75)? If enthusiastic, do we center the distribution on 25% difference with 2.5% probability of the treatment effect exceeding null difference (risk ratio >1.0)?
2. Do we base the prior on power estimation variables, i.e., the 80% of “area under the curve” contain the interval of 25% difference (RR of 1 to 0.75) with 10% distributed on either side of the tail (using the symmetry assumptions).
My reply: It doesn’t seem right to base the prior distribution on the power calculation. The standards for power calculations are much more relaxed than for Bayesian priors: for the power calculation, you make a reasonable assumption and compute the power conditional on that assumption. But the analysis itself isn’t supposed to rely on the assumed model. Once you have the data, the power calculation is irrelevant.
I suppose the best prior distribution would be based on a multilevel model (whether implicit or explicit) based on other, similar experiments. A noninformative prior could be ok but I prefer something weakly informative to avoid your inferences being unduly affected by extremely unrealistic possibilities in the tail of the distribuiton.
I’m curious what Sander Greenland would say here.
P.S. Sanjay Kaul adds:
I [Kaul] should have clarified that my previous example is a REAL CLINICAL TRIAL designed and analyzed with frequentist methods and I am now attempting to analyze the data using Bayesian methods.
For the uniform prior, I use a log odds mean of zero and sd of 2. What values do you recommend for a “weakly informative” prior, especially when there is no previous information available?
My reply: I’d set it up as a logistic regression and then use a Cauchy prior distribution centered at 0 with scale 2.5, as described here.
If this is a clinical trial for an FDA setting (mostly CDER), then it might be better to do a classical power calculation. The main reason is that the agency is concerned with maintaining a 2.5% Type I error, and as Scott Berry showed last year at the joint statistical meetings, this makes the prior irrelevant. Might as well be noninformative, unless of course that results in improper posteriors. (Maybe this result is published somewhere?)
If it's a CDRH (devices) setting, there may be some more give and take, but priors have to be agreed upon with the reviewing division. Chances are priors are noninformative or justified by previous data, or possibly using a hierarchical model with trial as one level.
NIH trials not overseen by the FDA in tandem will probably have a different structure.
I'm warming up to weakly informative priors, and look forward to the day when they can be applied more widely (except when something more informative is obvious).
For the experimental arm you don't want to use informative priors for the analysis unless it's strictly for internal decision-making. Use a noninformative prior (flat if appropriate). In the future Andrew's weakly informative priors will be an acceptable alternative…
For the analysis of control or placebo arm you could use informative priors based on historical evidence – but then you'll need to account for between-trial variation and inflate the prior variance. If you have data on several similar trials, an acceptable way would be, as Andrew already hinted at, is to fit a hierarchical model and use the predictive posterior of the parameter of interest for your prior.
In short, base an informative prior on actual data. I don't like 'enthusiastic' or 'skeptical' priors – they're arbitrary. To evaluate the effect of the prior on "power", compute the operating characteristics of your decision rule.
A key question here is what are the constraints of the clinical trial. If it's FDA they will have regulations you must follow.
But regardless, the key of any trial is designing it in a way such that its results will be confidently integrated into medical decision making. If you get too creative, even if your methods are very well thought out and statistically valid, the trial results may be viewed skeptically solely because your methods are new or different to many clinical readers. That doesn't mean you shouldn't do what it right, but it means you should think about the ability of the consumers of your trial to fully understand the methodology and results. Bayesians are making great strides in medical research. But small steps leading to great strides may help our methods be trusted and their value become evident to clinicians and policymakers.
I would encourage you to at least use the same prior for the treatment and control groups in the your final analysis. Otherwise as John Johnson states, you can't really control for Type I error — assuming you have some dichotomous decision at the end of your trial.
Internally, however, you may use subjective priors. We do this frequently. Using partial data at interim analyses we can calculate posterior probabilities of success. These decisions typically go into stopping accrual or dropping arms but don't contribute to the final analysis. The final analysis we will typically do with equal priors between groups and priors that are for the most part vague. This may be very frequentist in behavior, but it is a compromise we make and it is likely to help your trial results be accepted because they will be based on a familiar foundation.
Another important piece is the role of simulation. This will enable you to calculate power, Type I error rate, etc. And don't just look at summaries over all trials. Inspect what a handful of individual trials are doing in your simulations to assure nothing too unexpected happens once you start running the real trial.
This strategy may seem less than ideal in that it fails to incorporate the Bayesian notion of accumulating evidence. If your trial is not regulatory, then you may have more room to use your own best judgement.