A pivotal episode in the unfolding of the replication crisis

Axel Cleeremans writes:

I appreciated your piece titled “What has happened down here is the winds have changed”. Your mini-history of what happened was truly enlightening — but you didn’t explicitly mention our failure to replicate Bargh’s slow walking effect. This was absolutely instrumental in triggering the replication crisis. As you know, the article was covered by the science journalist Ed Yong and came shortly after the Stapel affair. It was the first failure to replicate a classic priming effect that attracted so much attention. Yong’s blog post about it attracted a response from John Bargh and further replies from Yong, as you indirectly point to. But our article and the entire exchange between Yong and Bargh is also what triggered an extended email discussion involving many of the actors involved in this entire debate (including E. J. Wagenmakers, Hal Pashler, Fritz Strack and about 30 other people). That discussion was initiated by Daniel Kahneman after he and I discussed what to make of our failure to replicate Bargh’s findings. This email discussion continued for about two years and eventually resulted in further attempts to replicate, as they are unfolding now.

I was aware of the Bargh issue but I’d only read Wagenmakers (and Bargh’s own unfortunate writings) on the issue; I’d never followed up to read the original, so this is good to know. One thing I like about having these exchanges on a blog, rather than circulating emails, is that all the discussion is in one place and is open to all to read and participate.

2 thoughts on “A pivotal episode in the unfolding of the replication crisis

  1. “One thing I like about having these exchanges on a blog, rather than circulating emails, is that all the discussion is in one place and is open to all to read and participate.”

    Yes!

    I am a bit worried about blogs either not having a comments-section or choosing which comments to place. If i am not mistaken the Center for Open Science blog, Datacolada, and APS Observer for instance all have no option to comment.

    Especially for blogs that have many readers and/or prestige, i think giving people the opportunity to comment could be very useful and important from a scientific perspective.

    As an ocasional commenter on other scientific blogs, i recently experienced 2 instances where my (critical) comments seem to have not been placed, which i also find worrying for the same reasons.

  2. Perhaps blogposts without comment-sections (or with comment-section but with censorship) are the new high impact-factor journals (e.g. Science, Nature):

    1) only authors with prestige get a platform to say what they want to and get attention for it, and
    2) discussion is severely limited, seperated from the place where it would be most useful, and perhaps also only possible for those belonging to “the inner circle” with certain connections.

    Another one to add to the list of blogs without comment-sections if i am not mistaken: “bayesianspectacles”, who are going to “discuss” the Crane paper “Why ‘Redefining Statistical Significance’ Will Not Improve Reproducibility and Could Make the Replication Crisis Worse”.

    Also see http://statmodeling.stat.columbia.edu/2017/11/19/no-inferential-thresholds/

    We can all read the post tomorrow or the day after that, and then only some of us can “discuss” it with the author(s) which is probably based on whether we belong to his/her/their “inner circle” of twitter friends or fellow-bloggers. Just like the “good old days” of the “old boys network” !

Leave a Reply to Anonymous Cancel reply

Your email address will not be published. Required fields are marked *