Skip to content
 

My course this fall on Bayesian Computation

Bayesian Computation, Fall 2011, Andrew Gelman (Statistics G8325, section 002)

Class meets Wed 9:30-12, Mudd Hall 1106B

Research topics. We are working on the following Bayesian computation projects:
– Automated adaptive Hamiltonian Monte Carlo
– Penalized maximum likelihood for multilevel models
– Weakly informative prior distributions
– Multilevel regression and poststratification
– Missing-data imputation using iterative regressions
– Automating graphical displays for checking and understanding models

Skills. In the homeworks, you will work in R to:
– Program the following Markov chain samplers: Gibbs, Metropolis, Metropolis-Hastings, slice, Langevin, Hamiltonian, also variational Bayes
– Make statistical graphs and perform posterior predictive checks
– Check Bayesian inference programs using fake-data simulation

All readings, class notes, and homeworks will be posted as pdfs at http://www.stat.columbia.edu/~gelman/bayecomputation

As a start, you should download and read course.outline.pdf, GelmanBoisJiang1996.pdf, introbookchapters2and3.pdf, bdachapters10and11.pdf, bdaAppendixC.pdf, class1handout.pdf, and, most important, hwk1.pdf and hwk2.pdf

Here is our current plan:

Week 1:
Reading: Gelman, Bois, and Jiang (1996) paper on hierarchical Bayesian toxicokinetics
Homework due 9am Friday of week 1: Set up R and Jags
In class: Review of Bayesian data analysis using toxicokinetics example; plan of course

Week 2:
Reading: Chapters 2-3 on R from my intro book, Chapters 10-11 and Appendix C of Bayesian Data Analysis
Homework due 5pm Tues before class: Gibbs sampler
In class: Review of homework; presentation of Gibbs sampler and Metropolis algorithm

Week 3:
Reading: Gelman and Shirley (2011), Kass et al. (1998), Lunn et al. (2009)
Homework due 5pm Tues before class: Metropolis algorithm
In class: review of homework; presentation of methods for implanting Gibbs and Metropolis and improving their efficiency

Week 4:
Reading: Abayomi et al. (2009) and Su et al. (2011) papers on multiple imputation, chapter 25 of Gelman and Hill
Homework due 5pm Tues before class: Optimize Metropolis
In class: review of homework; presentation of mi

Week 5:
Reading: Gelman (2006) and Gelman et al. (2008) on weakly informative priors
Homework due 5pm Tues before class: Metropolis-Hastings
In class: review of homework; presentation of weakly informative priors

Week 6:
Reading: Neal (2003) on slice sampling, Lax and Phillips (2009) on multilevel regression and poststratification
Homework due 5pm Tues before class: Slice sampling
In class: review of homework; presentation of slice sampling; presentation of mrp

Week 7:
Reading: Description of Langevin algorithm in MacKay (2004), Chung et al. (2011) on avoiding boundary estimates in hierarchical models, Dorie (2011) on sim
Homework due 5pm Tues before class: Langevin
In class: review of homework; presentation of Langevin algorithm, presentation of maximum likelihood and Bayes inference for hierarchical models

Week 8:
Reading: Neal (2011) on Hamiltonian Monte Carlo, Hoffman and Gelman (2011) on the no-U-turn sampler, Carpenter et al. (2011) on Stan
Homework due 5pm Tues before class: Hamiltonian Monte Carlo
In class: review of homework; presentation of HMC, the no-U-turn sampler, and stan

Week 9:
Reading: Description of variational Bayes in MacKay (2004)
Homework due 5pm Tues before class: Variational Bayes
In class: review of homework; presentation of variational Bayes

Week 10:
Reading: Gelman (2004) on Bayesian exploratory data analysis, Gelman and Unwin (2011) on information visualization
Homework due 5pm Tues before class: Exploratory data analysis
In class: review of homework; presentation of Bayesian exploratory data analysis; presentation of lattice graphics in R

Week 11:
Reading: Schofield et al. (2011) on hierarchical spline models for tree rings
Homework due 5pm Tues before class: Fit and graph a model
In class: review of homework; presentation of spline models; presentation of tree-ring analysis

Week 12:
Reading: Chapter 6 of Bayesian Data Analysis, Zheng, Salganik, and Gelman (2006) on “How many X’s do you know?” surveys
Homework due 5pm Tues before class: Posterior predictive check
In class: review of homework; presentation of Bayesian model checking, presentation of social network models

Week 13:
Reading: Cook, Gelman, and Rubin (2006) on Bayesian debugging
Homework due 5pm Tues before class: Fake-data debugging
In class: review of homework; presentation of Bayesian debugging; general discussion

12 Comments

  1. Manoel Galdino says:

    I really, really would be a better person if I could take your class. But since I can’t, two questions:
    1. Will you post the homework assignments in the internet? I’d like to try them.
    2. Will you post the lectures (or slides) in the internet? I’d like to read them and follow the pace of the course by my own.

    thanks

    Manoel

  2. C Ryan King says:

    OK, so I assume with the title that Stan is what Bob Carpenter mentions here. Interesting to see how it stacks up to MCMCglmm in terms of efficiency and scope of model specification.

  3. Zhilin Zhang says:

    Does the course have a webpage with course materials?

  4. anon says:

    Perhaps switch week 13 to the second week.

    How many new to Bayesian computations concretely understand that sampling from the posterior is “just” sampling from the conditional distribution – given x and that any sample drawn from P(x,theta) is in the conditional distribution for the particular x observed?

    Or maybe it is best to have them learn that last?

    Providing the lecture slides would be great!

    • Andrew says:

      Debugging is at the end because to debug you have to have a model to fit. I’ve structured the hwk assignments to start with pure computation and only get to modeling at the end.

  5. I like that you’re going the delong route and posting class outlines/details to your blog.

  6. Tom says:

    Looks like a great course – I’d love to follow it.

  7. Andrew says:

    To all:

    The course will be a seminar, so not much lecturing by me (I hope), with the exception of some already-prepared material which you can find on my webpage under Presentations. I’m hoping it will be mostly discussion. So, if you’re not actually taking the course, you won’t miss much by not being able to see notes, video, etc.

  8. Ewan says:

    Any chance of posting links to the homework assignments so others can follow along?

  9. I’d love to see the assignment, both so I can “borrow” them for my Bayes classes and so I can work through them myself!