Skip to content
 

No Retractions, Only Corrections: A manifesto.

Under the heading, “Why that Evolution paper should never have been retracted: A reviewer speaks out,” biologist Ben Ashby writes:

The problems of post-publication peer review have already been highlighted elsewhere, and it certainly isn’t rare for a paper to be retracted due to an honest mistake (although most retractions are due to misconduct). Moreover, one could argue that the mistakes in Kokko and Wong’s 2007 paper were sufficient to warrant a retraction as they significantly affected the conclusions. But by that logic, a large number of empirical studies should also be retracted due to incorrect statistical analyses or overreliance on fickle p-values, leading to irreproducible results.

OK, I have no problems so far, except to note that this is never gonna happen.

The part I don’t like is what comes next:

My concern is that the forced retraction of the original paper sends a bad message to the scientific community. Kokko [co-author of the original paper that got retracted, and also of the later paper that pointed out the error] has effectively been penalized for critiquing her own work, when in fact she should be applauded for her honesty.

No no no no no! Retraction is not a “penalty,” it is just a matter of correcting the scientific record.

Maybe there should be no such thing as retraction, or maybe we could ban the word “retraction” and simply offer “corrections.” That would be fine with me. The point is never to “expunge the record,” it’s about correcting the record so that later scholars don’t take a mistaken claim as being true, or proven.

But, to the extent there are retractions, or corrections, or whatever you want to call them: Sure, just do it. It’s not a penalty or a punishment. I published corrections for two of my papers because I found that they were in error. That’s what you do when you find a mistake.

I agree with Ashby when he writes:

These problems need to be highlighted at source (i.e. as corrections/erratum next to the original paper), with readers directed to the new paper for a fuller exploration of the corrected model.

I just don’t see how this differs from any other correction or retraction in the scientific literature. When there is an error, you want it to stay in the record, along with its correction.

Summary

Ashby writes:

In the case of Kokko and Wong (2007), does a retraction achieve anything that an Erratum or Technical Comment could not? In my opinion, no.

I agree and will go one step further. Does a retraction (in the sense of expunging the original material from the published record) ever achieve anything that an Erratum or Technical Comment could not? In my opinion, no.

P.S. Just to elaborate slightly: In general, I don’t imagine this correction procedure will be done by journals. There are just too many papers with serious errors. Even if only 10% of published papers have serious errors, that represents millions of possible corrections to be written, evaluated, and published. So I expect most of this will be done using external post-publication review.

But, when publications do want to correct the record, I think it makes sense to do so with corrections rather than retractions.

27 Comments

  1. KD says:

    >No no no no no! Retraction is not a “penalty,” it is just a matter of correcting the scientific record.

    When career/tenure decisions depend on the number of papers published by an individual, then retractions are a penalty. They aren’t a penalty in some ideal world perhaps, but they are when you are one of the few doing it, and your job depends on it.

  2. Andrew says:

    KD:

    Of course if you publish something that’s wrong, you shouldn’t get credit for publishing something that’s right. It’s not a matter of penalizing, it’s just accuracy.

  3. I think we should introduce the concept of a paper being “superseded” by a later paper. Unlike “correction”, it doesn’t imply that a mistake was made, just that progress was made on that question.

    • Andrew says:

      Luis:

      Superseded is fine. But I think correction is also helpful.

      Consider two scenarios:

      1. Paper A reports an experiment. Paper B reports a more general experiment which allows the results of A to be interpreted in a new light. Paper B supersedes paper A.

      2. Paper A reports an experiment. Paper B, among other things, describes some mistakes in Paper A. Whether or not B supersedes A, we now know that A has an error. It should be corrected.

      • Yes, I understand that distinction (it may even be the case that paper A shows a fact on a smaller sample and paper B confirms it on a larger sample, then B supersedes A even though they point in the same direction).

        I would be fine with superseded being sometimes used as a polite lie. It’d still be an improvement on the status quo where authors will fight against a retraction.

  4. John Mashey says:

    How about plagiarisms? Retractions, or something else?

    • Andrew says:

      John:

      For plagiarism, a note saying that his material was taken from whatever source. I guess there are some cases where so much is plagiarized that the material would have to be removed for legal reasons. Other than that, I’d keep the original paper but just label the source.

      In all these cases I guess it would be good to put some sort of watermark on the paper so that it’s clear to people who encounter it online, that there are corrections.

  5. John Mashey says:

    I think there’s a distinction between:
    a) What it’s called
    b) And whether or not it is removed.
    Generally, I certainly prefer it never be removed, but always with an embedded description, and then some kind of watermarks (which might just be a a note that corrections, errata or replacements exist and where). Among other things, old copies propagate, but at least new ones wouldn’t without the watermarks.

    On the plagiarism/copyright issue, would adding a big diagonal “Plagiarism” in 120pt red type and a source to each page be enough to satisfy copyright holders? (I don’t know answer to that one. It advertises the originl and is miserable to read.)

    • Andrew says:

      John:

      Yes, exactly. I always thought “retractions” were fine, but reading this post by Ashby, I got the impression what when a paper is “retracted,” this involves the removal of the paper. I think it should stay but with corrections. Whether this is called a retraction or a correction doesn’t really concern me.

  6. mark says:

    What if the data was simply fabricated (e.g., the papers published by Diederik Stapel). Would the correction or technical note simply state that the data never existed?
    Of course, even retracted papers continue to get heavily cited in some instances because the journals bury the retraction so that no one becomes aware of the errors.

  7. KD says:

    @Andrew: The problem is penalties are the wrong incentive structure. People will be unwilling to look at their own results too closely for fear that it might have negative consequences to their career. There is a joke that goes around in experimental circles, if your results look good, don’t replicate. While this is a joke, I wouldn’t be surprised if a lot of people subconsciously did something like this.

    To me the right approach is not thru penalties, but by positive reinforcement of retractions. Those who retract need to get credit in some form for their work. Credit that helps their career and work-place requirements. This needs to be fleshed out. But, without such an incentive structure, we can talk all we want about how things should change. But, when one’s career depends on it, one finds a way to justify some serious violations sometimes. To berate such people as doing a disservice to science or to get on a high-horse and talk about morality to me simply avoids dealing with the reality of the situation.

    Are there cases where people have looked and questioned their own results. Surely. But, a few cases don’t make an argument. What we need is to think of the institution, and think about the correct incentive structure to encourage such behaviour. And, I’m afraid “punishment” is too deeply a religious concept for me to accept anyway as a sensible/reasonable solution.

    Note: I am not talking about fabricated data here. There surely needn’t be any positive incentive for retraction of such acts – to me, despite the high profile cases in the last decade, they are far and few between to worry about right now. There are simply bigger fish to fry. I’m talking about erroneous data (thru accidents or improper experiment design…).

    • Andrew says:

      KD:

      Incentives are great. But, just to be clear, I never suggested any penalties!

      You write, “I’m afraid ‘punishment’ is too deeply a religious concept for me to accept anyway as a sensible/reasonable solution.” So let me re-emphasized that I never suggested punishing anybody.

      Indeed, here were my exact words above, regarding the retractions or corrections that I recommend:

      It’s not a penalty or a punishment.

      So if you’re arguing against punishment, you’re not arguing with me. I’m on your side!

      In any case, this discussion is entirely theoretical as in most cases people simply refuse to admit they made a mistake. Even the himmicanes people and the air pollution in China people, to take two recent examples where the analyses were obviously flawed, have as far as I know never considered the possibility that they might have performed inappropriate analyses of their data.

      Heck, even Case and Deaton never acknowledged that they should’ve age-adjusted their death rates and separated them by sex. Even though everyone in demography knows to age-adjust death rates and separate by sex.

      There’s lots of people who really, really don’t like to admit they made a mistake. So I think we can forget about routine use of any correction or retraction process.

    • Rahul says:

      As a practical matter, how many non-robust results were ousted because the authors *themselves* published they were non-replicable?

      I always assumed it was other people who find the non-replicability of results.

  8. KD says:

    @Andrew: Apologies for the misunderstanding! I made a mistake :). It looks like we are indeed in agreement.

  9. Brenton says:

    “Retraction”, rightly or wrongly, carries a strong tone of punishment and many negative consequences (e.g., in tenure review). As such, it should be reserved for cases of gross misconduct (fabrication, falsification, plagiarism). For analysis/design flaws, errors, mistakes, even if quite large (e.g., power posing), a “correction” or “update” appended to the original article (including appropriate changes to the substantive conclusions of the paper–editors must hold authors accountable for making these), is more appropriate because it corrects the record while also not creating (as large) a disincentive for authors to bring forth their own errors. This is especially the case for cases of researcher degrees of freedom–in many cases, authors probably didn’t realize at the time the problematic choices they made; punishment is not a good response to learning.

  10. Hi Andrew,
    I’ve only just come across your blog and found it really interesting. I think your views are similar to those expressed in a proposal that a group of us have been working on and posted to BioRxiv.

    The latest version is here:
    Amending Published Articles: Time To Rethink Retractions And Corrections?
    http://biorxiv.org/content/early/2017/03/24/118356#comment-3230389134

    but you might also like to see the comments on an earlier version too:
    http://biorxiv.org/content/early/2017/03/21/118356

    Please feel free to add your thoughts. We are considering all the feedback we’ve received and hope to revise in due course.

    Best wishes,
    Elizabeth Moylan

Leave a Reply