Skip to content
 

A fast new automatic Metropolis algorithm?

Ted Dunning points me to this article by J. Andres Christen and Colin Fox:

We [Christen and Fox] develop a new general purpose MCMC sampler for arbitrary continuous distributions that requires no tuning. . . .The t-walk maintains two independent points in the sample space, and all moves are based on proposals that are then accepted with a standard Metropolis-Hastings acceptance probability on the product space. . . . In a series of test problems across dimensions we find that the t-walk is only a small factor less efficient than optimally tuned algorithms, but significantly outperforms general random-walk M-H samplers that are not tuned for specific problems. . . . Several examples are presented showing good mixing and convergence characteristics, varying in dimensions from 1 to 200 and with radically different scale and correlation structure, using exactly the same sampler. The t-walk is available for R, Python, MatLab and C++ at http://www.cimat.mx/~jac/twalk/.

This looks pretty cool to me! I asked Christian Robert what he thought, and he referred me to this blog entry where he presents a bit of a skeptical, wait-and-see attitude:

The proposal of the authors is certainly interesting and widely applicable but to cover arbitrary distributions in arbitrary dimensions with no tuning and great performances sounds too much like marketing . . . there is no particular result in the paper showing an improvement in convergence time over more traditional samplers. . . . Since the authors developed a complete set of computer packages, including one in R, I figure people will start to test the method to check for possible improvement over the existing solutions. If the t-walk is indeed superior sui generis, we should hear more about it in the near future…

I have a lot of big models to fit, so I expect we’ll be trying out many of these different methods during the next year or so.

P.S. Update here.

3 Comments

  1. Anon says:

    I'd be interested in hearing your opinion about what you think should be the current state-of-the-art default sampler. Christian Robert seems to take his population monte carlo as the state-of-the-art, which definitely seems to perform well, but I'd be interested in your take, placing some weight on simplicity.

    Maybe a blog post on the state-of-the-art for default samplers. Where is Bayesian computation headed? What should software like Jags/HBC be implementing first? Reading Liu's or Robert and Cassella's books leaves one pretty confused. The result, for me, has been to stick to MH or slice-sampling, and just suffer the consequences by running stuff over a few days, if necessary. Obviously this leads to suboptimal model experimentation.

  2. rrtucci says:

    I would recommend young researchers (too late for you Gelman :) ) to investigate the possibility of doing MCMC with quantum computers.

  3. Anon says:

    Quantum computing is certainly is an interesting idea, but given the state of quantum computers — basically nonexistent — I doubt it would add much in practical terms for several decades.

    I suspect that better sampling / programming could lead to systems that are 1000x faster using conventional technology. Coding in C gets 10-100x increase in speeds. Duke has a group that has being doing research using MCMC on GPUs that has 100x increase in speed, and GPU technology will increase 500x in the next five years according to industry estimates. Christian Robert has reported virtually linear scaling across machines using importance sampling. In his cosmological application he gets 100x increase in speed, and clusters are now easy to build (e.g. EC2). I tend to think (applied) researchers should focus on making these existing massive improvements more widely available. Sadly there is not much academic reward for this type of work.