Stan PK/PD Tutorial at the American Conference on Pharmacometrics, 8 Oct 2015

Bill Gillespie, of Metrum, is giving a tutorial next week at ACoP:

This is super cool for us, because Bill’s not one of our core developers and has created this tutorial without the core development team’s help. Having said that, we’ve learned a lot from Bill and colleagues on our mailing lists as we were designing ODE solvers for Stan (an ongoing issue—see below for future plans).

Bill’s tutorial is up against a 2-day Monolix tutorial and a 2-day tutorial on R by Devin Pastoor, who’s also been active on our mailing lists recently.

Why Stan for PK/PD?

In case you’re wondering why people would use Stan for this instead of something more specialized like Monolix or NONMEM, it’s because of the modeling flexiblity provided by the Stan language and the effectiveness of NUTS for MCMC. So far, though, we’re in the hole in not having a stiff ODE solver in place. Or a good NONMEM-like event data language on top.

Maybe Bill will jump in with some other motivations.

What’s in Store for Stan’s ODE Solvers?

There’s been lots of behind-the-scenes activity on our ODE solvers—we’re really just getting burned in warmed up.

The next minor release of Stan (2.9) should stop the freezing issue when parameters wander into regions of parameter space that lead to stiff ODEs. And we’ve really sped up the Jacobian calculations when Michael Betancourt realized we were doing a lot of redundant calculation and he and I put a patch in to fix it. We should also allow user-defined control of absolute and relative tolerances.

Next, hopefully by Stan 2.10, we’ll have a stiff solver and maybe a way for users to supply analytic coupled-system gradients and Jacobians. Stay tuned. These new designs are largely being guided by Sebastian Weber and Wenping Wang at Novartis. And of course, by Michael Betancourt working out all the math and Daniel, Michael, and I working out the code with Sebastian’s and Wenping’s input.

We also need to evaluate how well variational inference works for ODE problems. Our early trials are very promising. Then we could replace the max marginal likelihood approach of NONMEM with a very speedy variational inference mechanism allowing much more general models.

There’s more in the works, but the above are the top of our to-do list.

PK/PD Talk with Stan — Thu 8 Oct, 10:30 AM at Columbia: Improved confidence intervals and p-values by sampling from the normalized likelihood

Sebastian Ueckert and France Mentré are swinging by to visit the Stan team at Columbia and Sebastian’s presenting the following talk, to which everyone is invited.

Improved confidence intervals and p-values by sampling from the normalized likelihood

Sebastian Ueckert (1,2), Marie-Karelle Riviere (1), France Mentré (1)

(1) IAME, UMR 1137, INSERM and University Paris Diderot, Paris, France; (2) Pharmacometrics Research Group, Department of Pharmaceutical Biosciences, Uppsala University, Uppsala, Sweden

10:30 AM, Thursday 8 October
1025 School of Social Work Building (CUSSW); 1255 Amsterdam Ave (122 St & Amsterdam)

Asymptotic theory-based statistics such as confidence intervals (CI) and p-values (PVAL) are the basis for most model-driven decisions in drug development. For small sample sizes these approximations do not hold and resampling methods are employed. Sampling from the normalized likelihood function represents an alternative, which with the development of Hamiltonian Monte-Carlo (HMC) methods becomes computationally attractive. In this presentation the results of a comparison between HMC-based sampling and existing approaches for the calculation of CI and PVAL is presented.

The comparison was performed with a simulation study using a one-compartment model and different study sizes. For CI, evaluation was based on runtime, median CI and coverage, and in comparison to CI obtained via covariance matrix, log-likelihood profiling and non-parametric bootstrap. For PVAL, the evaluation was based on runtime, type-I error and power, and in comparison to PVAL obtained via Wald test, log-likelihood ratio test and permutation test. The HMC-based methods were implemented using S with improper or uniform priors for sampling. All asymptotic theory and resampling-based results were obtained in NONMEM 7.3.

The simulations showed good agreement between approaches for large sample sizes and increasing differences for smaller sample sizes. In contrast to most other methods, HMC showed nominal coverage and type-I error at all study sizes. In terms of computation time the HMC-based methods were between 10 and 60 times faster than resampling methods.

In conclusion, CI and PVAL through sampling from the normalized likelihood using HMC yielded results with good theoretical properties at a drastically shorter runtime than resampling methods.