The unbelievable reason that Jennifer Lawrence is using Waic and cross-validation for survival models

Sam Brilleman writes:

I’ve been reading two of your recent papers:

(1) Gelman A, Hwang J, Vehtari A. Understanding predictive information criteria for Bayesian models. Statistics and Computing 2014; 24: 997-1016.

(2) Vehtari A, Gelman A. WAIC and cross-validation in Stan. Submitted. 2014. http://www.stat.columbia.edu/~gelman/research/unpublished/waic_stan.pdf. Accessed: 6 July 2015.

My question in short is: The example you give in paper (2) above applies to a (multilevel) linear regression model. Do you have any advice on whether it is appropriate to calculate WAIC for a survival model (with right-censored survival times) using the log-likelihood approach you outline in paper (2) above? In that setting, would the link between the log predictive density and the log-likelihood break down?

I won’t got into any details here, as this email will get dragged out. Instead I will just leave the question broad, but of course offer to provide specific details on my analysis if you are interested in them. I am actually fitting a joint longitudinal and survival model in Stan so I could, for example, provide details on the log-likelihood function for the joint longitudinal and survival model, the Stan code for fitting the joint model, or the Stan code showing how I have calculated WAIC (or anything else which might be relevant!). I had thought about posting this to the Stan-users groups, but thought I would approach you directly first.

I replied:

I don’t see why this would be a problem, but if there is any question, you can do LOO, which is more directly interpretable.

And Aki replied:

LOO and WAIC are appropriate for survival models with right-censored
survival times. The predictive distribution predicts the event time, but if the left out observation is censored then the log predictive density is the log probability that the event is later than the censoring time just like in the likelihood.

We have used cross-validation, for example, in this papers
http://ceur-ws.org/Vol-1218/bmaw2014_paper_8.pdf
http://www.thelancet.com/journals/lanonc/article/PIIS1470-2045(11)70299-6/abstract

Leave a Reply

Your email address will not be published. Required fields are marked *