Skip to content
 

Can’t Stop Won’t Stop Mister P Beatdown

Ben Highton and Matt Buttice point us to this response addressing some of the issues Jeff Lax raised in his most recent MRP post.

P.S. Jeff replies in comments:

It sounds like we’ve converged. They acknowledge MRP performance is significantly better on average than reported in their new paper in PA and yet performance variation in terms of correlation to “truth” remains higher than some might have thought.

Cool. I hope this sort of blog exchange can be a model of scientific discussion. Instead of a paper just sitting there by itself, it can be openly explored. Ideally, the published paper would include a link to these discussions of Highton, Buttice, Lax, Phillips, and Ghitza, so that readers would automatically get all this information.

3 Comments

  1. Jeffrey Lax says:

    It sounds like we’ve converged. They acknowledge MRP performance is significantly better on average than reported in their new paper in PA and yet performance variation in terms of correlation to “truth” remains higher than some might have thought. I appreciate their attention to these issues. They think the variation too high to feel fully comfortable with MRP. I do not, given that (a) mean absolute errors (the other metric for success) are still often small and (b) in practice one can inspect the estimates and model and bring additional information to the table. I’m working on benchmarks and diagnostics for such variation. -Jeff

  2. dab says:

    I just finished writing a comment in the PubMed thread about the selected papers network idea described here:

    http://docs.selectedpapers.net/intro.htm

    that is apropos to your postscript. I’d be interested in your thoughts on the concept.

Leave a Reply