Bayesian Computation, Fall 2011, Andrew Gelman (Statistics G8325, section 002)

Class meets Wed 9:30-12, Mudd Hall 1106B

Research topics. We are working on the following Bayesian computation projects:

– Automated adaptive Hamiltonian Monte Carlo

– Penalized maximum likelihood for multilevel models

– Weakly informative prior distributions

– Multilevel regression and poststratification

– Missing-data imputation using iterative regressions

– Automating graphical displays for checking and understanding models

Skills. In the homeworks, you will work in R to:

– Program the following Markov chain samplers: Gibbs, Metropolis, Metropolis-Hastings, slice, Langevin, Hamiltonian, also variational Bayes

– Make statistical graphs and perform posterior predictive checks

– Check Bayesian inference programs using fake-data simulation

All readings, class notes, and homeworks will be posted as pdfs at http://www.stat.columbia.edu/~gelman/bayecomputation

As a start, you should download and read course.outline.pdf, GelmanBoisJiang1996.pdf, introbookchapters2and3.pdf, bdachapters10and11.pdf, bdaAppendixC.pdf, class1handout.pdf, and, most important, hwk1.pdf and hwk2.pdf

Here is our current plan:

Week 1:

Reading: Gelman, Bois, and Jiang (1996) paper on hierarchical Bayesian toxicokinetics

Homework due 9am Friday of week 1: Set up R and Jags

In class: Review of Bayesian data analysis using toxicokinetics example; plan of course

Week 2:

Reading: Chapters 2-3 on R from my intro book, Chapters 10-11 and Appendix C of Bayesian Data Analysis

Homework due 5pm Tues before class: Gibbs sampler

In class: Review of homework; presentation of Gibbs sampler and Metropolis algorithm

Week 3:

Reading: Gelman and Shirley (2011), Kass et al. (1998), Lunn et al. (2009)

Homework due 5pm Tues before class: Metropolis algorithm

In class: review of homework; presentation of methods for implanting Gibbs and Metropolis and improving their efficiency

Week 4:

Reading: Abayomi et al. (2009) and Su et al. (2011) papers on multiple imputation, chapter 25 of Gelman and Hill

Homework due 5pm Tues before class: Optimize Metropolis

In class: review of homework; presentation of mi

Week 5:

Reading: Gelman (2006) and Gelman et al. (2008) on weakly informative priors

Homework due 5pm Tues before class: Metropolis-Hastings

In class: review of homework; presentation of weakly informative priors

Week 6:

Reading: Neal (2003) on slice sampling, Lax and Phillips (2009) on multilevel regression and poststratification

Homework due 5pm Tues before class: Slice sampling

In class: review of homework; presentation of slice sampling; presentation of mrp

Week 7:

Reading: Description of Langevin algorithm in MacKay (2004), Chung et al. (2011) on avoiding boundary estimates in hierarchical models, Dorie (2011) on sim

Homework due 5pm Tues before class: Langevin

In class: review of homework; presentation of Langevin algorithm, presentation of maximum likelihood and Bayes inference for hierarchical models

Week 8:

Reading: Neal (2011) on Hamiltonian Monte Carlo, Hoffman and Gelman (2011) on the no-U-turn sampler, Carpenter et al. (2011) on Stan

Homework due 5pm Tues before class: Hamiltonian Monte Carlo

In class: review of homework; presentation of HMC, the no-U-turn sampler, and stan

Week 9:

Reading: Description of variational Bayes in MacKay (2004)

Homework due 5pm Tues before class: Variational Bayes

In class: review of homework; presentation of variational Bayes

Week 10:

Reading: Gelman (2004) on Bayesian exploratory data analysis, Gelman and Unwin (2011) on information visualization

Homework due 5pm Tues before class: Exploratory data analysis

In class: review of homework; presentation of Bayesian exploratory data analysis; presentation of lattice graphics in R

Week 11:

Reading: Schofield et al. (2011) on hierarchical spline models for tree rings

Homework due 5pm Tues before class: Fit and graph a model

In class: review of homework; presentation of spline models; presentation of tree-ring analysis

Week 12:

Reading: Chapter 6 of Bayesian Data Analysis, Zheng, Salganik, and Gelman (2006) on “How many X’s do you know?” surveys

Homework due 5pm Tues before class: Posterior predictive check

In class: review of homework; presentation of Bayesian model checking, presentation of social network models

Week 13:

Reading: Cook, Gelman, and Rubin (2006) on Bayesian debugging

Homework due 5pm Tues before class: Fake-data debugging

In class: review of homework; presentation of Bayesian debugging; general discussion

I really, really would be a better person if I could take your class. But since I can’t, two questions:

1. Will you post the homework assignments in the internet? I’d like to try them.

2. Will you post the lectures (or slides) in the internet? I’d like to read them and follow the pace of the course by my own.

thanks

Manoel

Hi,

Any chance of video recording the lectures? This seems like a really good course and I would love tp be able to follow it!

Thanks in advance,

Cheers,

inti

I second that!

OK, so I assume with the title that Stan is what Bob Carpenter mentions here. Interesting to see how it stacks up to MCMCglmm in terms of efficiency and scope of model specification.

Does the course have a webpage with course materials?

Perhaps switch week 13 to the second week.

How many new to Bayesian computations concretely understand that sampling from the posterior is “just” sampling from the conditional distribution – given x and that any sample drawn from P(x,theta) is in the conditional distribution for the particular x observed?

Or maybe it is best to have them learn that last?

Providing the lecture slides would be great!

Debugging is at the end because to debug you have to have a model to fit. I’ve structured the hwk assignments to start with pure computation and only get to modeling at the end.

I like that you’re going the delong route and posting class outlines/details to your blog.

Looks like a great course – I’d love to follow it.

To all:

The course will be a seminar, so not much lecturing by me (I hope), with the exception of some already-prepared material which you can find on my webpage under Presentations. I’m hoping it will be mostly discussion. So, if you’re not actually taking the course, you won’t miss much by not being able to see notes, video, etc.

Any chance of posting links to the homework assignments so others can follow along?

I’d love to see the assignment, both so I can “borrow” them for my Bayes classes and so I can work through them myself!