Skip to content
 

Cleaning up science

David Hogg pointed me to this post by Gary Marcus, reviewing this skeptics’ all-star issue of Perspectives on Psychological Science that features replication culture heroes Jelte Wicherts, Hal Pashler, Arina Bones, E. J. Wagenmakers, Gregory Francis, Hal Pashler, John Ioannidis, and Uri Simonsohn. I agree with pretty much everything Marcus has to say. In addition to Marcus’s suggestions, which might be called cultural or psychological, I also have various statistical ideas that might help move the field forward. Most notably I think we need to go beyond uniform priors and null-hypothesis testing to a more realistic set of models for effects and variation. I’ll discuss more at some other time, but in the meantime I thought I’d share these links.

P.S. Marcus updates with a glass-is-half-full take.

3 Comments

  1. Thomas says:

    Fabio Rojas expresses a similar kind of optimism (in re PS) at orgtheory.

    In my view the problem is deeper than Marcus and Rojas suggest. I think fraud in social science follows the same pattern as fraud in finance; the conditions under which outright fraud is possible are the same as the conditions under which unintentional errors go undetected for a long time.

    As Michael Lewis explained in The Big Short: In the early days of the bubble “the market for subprime mortages simply had no place for people in it who took a dim view of them” (p. 29). As I’ve pointed out before, a similar “bullishness” (expresses by those two recent posts by Rojas and Marcus) can be found in social science. “Yes, yes,” the optimists say, “there have been problems, but that’s just all part of the self-correcting logic of the system. In the end, the marketplace of ideas is efficient.”

    • Andrew says:

      Thomas:

      I agree with you and posted as much on Rojas’s site. I don’t know how much notice that comment got, though, so I think I’ll repost it here.

  2. […] think you agree that many of the “bad statistics/methods” problems & even the “nonreplicability” […]